Introduce yourself

Hey all, I'm Theo. I just finished putting together this forum. Hope you like it, I'm looking forward to being able to interact with all of you. I'm right at the very beginning of my career, but kinda lost for direction. A bit about me, I can be summed up as an absolute lack of focus and abundant curiosity. I'm trying to learn Arabic, Chinese, French, & Tibetan, and I'm super interested in sociology, mathematics, economics, biology, electrical engineering, & applied physics. In undergraduate I got interested in machine learning and taught two basic classes on machine learning for malware analysis and object recognition. Last year I was a PhD student in Machine Learning, but a series of medical problems led to poor performance in the final semester and I had withdraw from the program. My background in machine learning is more the mathematical perspective than the algorithmic, but I'm interested in all aspects of machine learning. I'm kinda of lost for direction since getting expelled from the PhD program and trying to find a path forward, all I know is whatever that path is it will be in data science.

Projects I'm currently working:

Deep Learning for malware analysis: simple project I'm just applying the work of Yu et. al from microsoft research to analyse a set of malware data that I have access to http://research.microsoft.com/pubs/193768/MalwareRandomProjections.pdf

Deep Learning for finance: At the moment I'm working part-time as a sort of consultant and build some deep learning models for trading using deep learning

Hey,

I am Ellen. I have a background in CS and 5 years of post-MSc. experience in mostly (web) backend development. About a year ago I switched companies and careers to go into data engineering, which I find much more fascinating. Right now, I am working in the BI/data analytics team on A/B testing, from all angles: Framework building, statistics, usability research, automated results analysis & visualization, internal consulting, training etc. The only thing I don't do is run actual tests myself.
Beyond work, I study in a remote BSc. program in psychology & sociology, and my goal is to one day combine these interests by working more with social data. To brush up my statistics knowledge, I also (so far) did half of the Coursera DS specialization, but I am not really impressed by it.

Hey Ellen & Theo,

Great to hear your backgrounds.

I went to a school for "Web development" (it was a specialized degree) but I dropped out after the first few months because it wasn't going fast enough. I immediately moved from NC to Chicago without knowing anyone and started freelancing in Ruby programming. Learned it all on the fly as I went. Then I got a job at an agency doing front-end development, then another agency doing full-stack development, then got a job at the white house & state department doing ruby and javascript programming (and tech consulting work), and ever since I've moved around the US (california to upstate new york) and I've been sort of lost. I never expected to get to such a high level so quickly, and I know this sounds horribly humblebraggy - but it honestly threw me for a huge loop.

Ever since then I've been trying to regroup on my goals, and figure out what I want to do. That was in 2014, and I'm still trying to figure it out. Now days I work remotely for a company based out of LA and NYC as a Director of Engineering where I lead teams as we build podcast technology and apps (ios/android). I am really good at management, but it's not my passion. I love people, but it's just not intellectually challenging enough to be of interest to me anymore.

I'm really excited to join this group because over the past 6 months I've realized my ultimate vision is to push the knowledge in our industry forward. I've done enough practical application building working on web apps at scale blah blah, but now its time to dive into theory. I know almost 0% about any computer science things except that which I've learned on youtube and coursera and khan academy etc. Somehow I managed to build and scale massive nodejs apps without knowing what binary search trees are ha. Anyway, I am applying to complete my undergrad degree at Cornell and Harvard in Comp Sci part time, while Im working and eventually want to move into a masters degree and being a professor. Why a professor? I don't necessarily want to do it only for teaching, which I love, but mainly for the research aspect. I know I could do research in other venues but really digging into the academic lifestyle is something I'm interested in. As Cal says, working backwards from a lifestyle we want.

Anyway, sorry for the wall of text, as I was writing I realized I was doing it for my own clarification more than to inform anyone. BUT thanks for reading if you got this far. And thanks Theo for being so cool with all my questions about this stuff thus far.

Thanks for making this forum, Theo!

My name is Sam, and it always seems like my story is a little bit more backwards than
most. I graduated with Maths and Physics and went off to the working world doing
computational stats, which later evolved into a data science role. At the time I got
pretty frustrated with my lack of programming knowledge, and so I wound up coming to
Sillicon Valley and wound up in various roles: teaching, some data modelling, and
full-stack consulting.

These days I do enterprise-level biotech applications, however I would like to get
back into writing functional code and doing machine learning. I've dabbled with the
idea of going back to school, however I'd like to see how far I can go in industry
before committing to a PhD.

Aside from work, I am working on some ml research with a professor and industry
researcher (you can find the original paper here: bactra.org/CSSR).  I also
love what I've seen of the Haskell community and would really like to be more
involved. Luckily, there seems to be a small niche in the quantatative finance sector
for Haskell developers, which is the current end-goal I have in mind for this course.

--

Aside: it looks like new users can't post for 7 days: Theo, would you mind looking into this?

Hi All

I'm Diran.  For the the last two years I have been working as a Business Analyst at an Amazon Fulfillment Centre.  One aspect of my role is to support Operations with data analysis to drive performance and cost.  I enjoy diving deep into data, but I want to move on to solve more difficult, human problems affecting the world.  I want to move into Data Science.  

I don't have any previous qualifications other than high school, I have self taught myself to where I am now.  Excel and SQL are the only tools I currently know.  I have started part time undergraduate studies in Mathematics and Statistics.  I would like to learn machine learning.

I'm a complete beginner and I'm not too sure where I should start.  I'm looking forward to learning from the members of this community, and beginning my journey into the world of Data Science.  Thank you Theo for creating this forum!

It's nice to meet everyone. Thanks so much for creating this forum, Theo!

I am currently a Postdoc in Psychology at Columbia University. I got my Ph.D. in Psychology with a minor in Statistics last year. I am looking to transition out of academia. Given my research experience and statistical training, I am thinking about whether a career as a Data Scientist might be a good fit. A recurring frustration I have with the Data Science field in general is the absolute lack of clarity/agreement about what it constitutes. I understand that this is likely because the field is so new. But it is incredibly frustrating to know what skills you need to learn and what kinds of roles you would be playing if you want to become a Data Scientist. People and companies have such different opinions and expectations about what a data scientist is. Data science could consist of development, statistics, ML, programming...the list goes on and on. I've been looking at job ads, and the requirements they want for a data scientist sound quite ridiculous. How would it be possible for someone to be an expert in advanced statistics, ML, web development AND fluent in all the different kinds of software out there (e.g., Python, R, SAS, SPSS, STATA etc). It is very overwhelming. I'm glad I have the support of this group though!

Yes, also thanks from me, Theo!

Hey all,

I recently graduated from an Electrical and Computer Systems Engineering degree in Melbourne, Australia. But I just moved to San Francisco/Silicon Valley and am job hunting for data science jobs here.

In the long term, I want to reach a PhD+ level of ML/DS without doing a PhD. I'm trying to figure out the best path to do that.

I'm trying to get into data science now because:
1) All my engineering interests seem to involve machine learning - signal processing, emotion recognition, BCI.
2) It's very clear that we're going to have a data explosion soon. Wearables, Internet of Things, Virtual Reality, etc. are going to generate a tonne of data on top of the deluge we already don't know what to do with. So those who know data science are going to be on top.

It seems the standard path for someone like me is getting a PhD, but that seems costly and time consuming. I'm hoping that with deliberate practice and strategy I can end up in the right roles via working in industry.

------
A bit more about my experience. My degree was diverse, but largely hardware focused. Programming (scripting, really) got taught on the side, and I had one class on optimisation/non-basic linear algebra.
During my degree, I worked in a computational neuroscience lab focused on consciousness over the breaks. I spent for months processing and analysing fly brain recordings to see how supposed correlates of consciousness varied when you messed with neurotransmitters in different ways. For my honour's thesis I tried to extract correlates of charisma from speech recordings, comparing these between expert and non-expert presenters.
------

Anyhow, it's really great to meet a whole group of people intent on being amazing at data science! This should be fun.

Hey everyone,

My name is Yoori and I graduated with a finance degree in 2010. Data science is new, and foreign, to me. Apart from following data science related twitter handles, I don't know much about the field or what the day to day entails. Data is interesting to me because I want to develop analytical skills. I'm relieved and super glad this forum was created. I'm looking forward to reading your insights.

Hey all it's great to hear from everyone, it seems we have pretty diverse backgrounds and I'm sure we can learn a lot from each other. I just wanted share some thoughts, this is all my opinion and definitely debatable, but I hope that it provokes some discussions or thoughts, if somebody disagrees and has evidence to prove me wrong I will be really excited.

darkruby -- You said you don't want to do a PhD. I would ask why? It may seem costly and time consuming, but outside of graduate program it would probably take far more time to get the same level of knowledge. I don't want to be negative, but I do want to give you my honest opinion. A PhD (assuming a good advisor and a proper approach) is 5 years of deliberate practice working on small, but challenging projects (ideally chosen by a supportive and helpful adviser). I really don't think there's a good substitute. A PhD has a high opportunity cost in terms of income and takes a long time, but the flip side is you have five years to dedicate to refining your skills. It's really hard to imagine a similar situation if you're trying to do this part-time. Most jobs will have work on programming and you simply won't have sufficient time or energy to work on the side to gain deeper insight. It is definitely possible to get to the same point of a PhD without graduate school, but the odds are against you and it will likely take you far more time than inside a PhD program. Few companies are really going to invest the time or effort to train you to that point. It's important to note that you gain very valuable skills outside of academia, PhDs come out grad school with a lot of knowledge but frequently don't know how to navigate real-world constraints so it's a trade-off. But it's hard to imagine for an average person that you could get the same knowledge as a PhD student in the same time-frame while juggling a full-time job. The whole point of the PhD is sacrificing income and time to get an opportunity to build up skills that you wouldn't have an opportunity to focus on otherwise. To really understand is much more than reading textbooks, you really have to do a variety of projects to gain a deep intuition which will let you make new models (It's very easy to come up with a new model, but without a good understanding, the model is likely to be intractable or be essentially the same hypothesis space as a previous model). If you want to do something outside of academia, I really suggest you try to apply to Baidu's silicon valley AI lab, or similar industry research labs. On another note, can you share some of your interests in signal processing, that's a huge area. Or any papers in emotion recognition that seems really interesting. Finally by any chance do you still have access to the fly brain recordings data in case anybody would like to use it for their project. All in all wish you luck in getting a data science job.

Yoori -- thanks for writing. Quantitative finance is closely tied to data science; for example the CAPM model is essentially just an example of factor analysis. From a very reductionist perspective finance is an application of time series analysis where we search for low dimensional uncorrelated factors. Are you working in finance? If you have time it would be great if you could tell us a bit about so we can get a feel of how you have to go about adapting the textbook data science techniques to a specific domain

Justkeepswimming -- That's an amazing background being a postdoc at Columbia, just in case you didn't know, one of the big names in the field Michael Jordan (professor at Berkeley) did his PhD in psychology too. He has a huge lab so I don't think you'll be able to contact him directly but he might be a really good expert to read up on his history and see how he managed to shift so well from psychology to Machine Learning / Data Science. Another interesting choice would be Sebastian Seung (Princeton) he's a leading researcher in computational neurobiology, but he's also had numerous important publications in machine learning. He might be a good example about how to bridge the two fields. With respect to the job applications, yea I can totally empathize the term data scientist is so broad that it really covers multiple types of positions. Basically there's more or less a continuum between people who mostly do programming and implementation and researchers who develop the models and analyse them for insight. Given you're background you're probably looking at the later group. In that case you should probably concentrate on statistics and ML (web development is for someone more junior, knowledge of systems can be handed off to a hardware expert). In terms of languages I would recommend R in a linux environment. It's one of the more used languages, especially for researchers. SAS & STATA are at this point really used by the government or other huge companies for whom adopting a new language is too expensive. To my knowledge the only advantage of SAS is that it has better support for out-of-core operations than R or python. But that really isn't important as out-of-core situations are increasingly handled by frameworks like spark or hadoop.

Diran - I don't know if you'd be interested by Georgia tech offers an online masters in computer science with a specialization in machine learning. It's part time, extremely cheap (relative to other masters programs) and would give a very good background. You might want to check it out. Beyond that there are a lot of great online classes (free, professors just put all their lectures and assignments online). Can you give me a bit better idea of what sort of problems you want work with, then I can link some resources that might help you.

samstites -- if you have time, I would really like to hear about your paper: your insights, how it fits into the current field of machine learning. Also working with an industry researcher sounds like a great opportunity, can I ask how you managed that. I think for the people who want to advance without grad school that seems like a great opportunity. Would be an ideal situation for me. Are there a lot of similar opportunities? Finally what sort of stuff do you in biotech? I thought biotech really didn't have much use of data science; are there some overlaps? I'd really like to learn a new area to apply data science.

Ellen - It sounds like the rest of us are trying to get into data science, but you're already in data science. If you have some time could you elaborate on the workflow you mentioned in your post. I'd really like to learn more about how data moves in industry

Super interesting to hear about everyone's background! It's very cool that we all have such different experiences and training.

Theo, thanks for the great tips! I will def. research the people you have mentioned.

For anyone who is wondering about whether to pursue a Ph.D.: my advice, as someone who went through a Ph.D. program (6 years), is NO! I know this might be an unpopular answer, but I think there are huge problems with the higher education system. Aside from the high tuition rates that universities are charging and the downright horrifying treatment of their instructors, the system is fundamentally broken. For example, although the main goal of a Ph.D. program is to train one for an academic position, there are VERY, VERY few academic jobs available. You can argue that you don't want to be a professor, that you want a Ph.D. so you can transfer that knowledge into industry. Well, a Ph.D. does not transfer well into industry. Given the main goal of Ph.D. programs is to train one to become an academic, most programs pay very little attention to preparing their graduates for non-academic professions. Training for technical skills that are important in industry is abysmal, and many professors are often very behind on new advances in technology and software (I am sure this might be different for engineering or computer science departments, but this is certainly true for the majority of traditional Ph.D. programs IMO). For example, most academics in my field (social sciences) use SPSS and SAS for their statistical analysis; very few use open source softwares like Python. As a result, the technical skills that I spent years and years learning were not helpful or valued in industry.

There are a billion other reasons why I believe getting a Ph.D. is not worth it (personally or professionally). I have included some relevant readings here (keep in mind that this is just a snapshot: there are many, many more writings about this):

http://www.nature.com/nature/journal/v472/n7343/full/472259b.html

http://www.economist.com/node/17723223

http://www.slate.com/articles/life/culturebox/2013/04/there_are_no_academic_jobs_and_getting_a_ph_d_will_make_you_into_a_horrible.html

http://www.salon.com/2014/09/21/professors_on_food_stamps_the_shocking_true_story_of_academia_in_2014/

http://www.nature.com/news/2011/110420/full/472276a.html

http://www.theguardian.com/higher-education-network/blog/2014/may/23/so-many-phd-students-so-few-jobs

https://chroniclevitae.com/news/724-the-job-market-recovery-that-never-came

http://theprofessorisin.com/its-ok-to-quit/

If I sound bitter, that is because I am. Although I learned a great deal about my field and myself through my graduate training program, I would not advise ANYONE to follow the same path. I really believe the degree is not a wise investment of one's time, energy, or money.

Justkeepswimming I would like to respond that the situation is very different depending on the field. Without wanting to offend I would bet that there are relatively more grants for computer science departments than for psychology. In computer science the expectation is that you would be supported throughout the entire program and career prospects with a CS PhD are quite bright. I fully acknowledge that the situation is quite different in other fields, social sciences struggle to get funding, biology is suffering from an oversupply of graduates, etc. In many fields graduates cannot find work and suffers decades as adjunct professors. However I think most people in data science would go for a PhD in computer science or statistics. In these fields (particularly computer science), funding isn't an issue. You will be expected to work 10-15 hours a week as a TA if you don't receive a fellowship. The articles you linked above are mostly about PhDs in general, this is very bad as it overlooks that PhD outcomes have high variation depending on the degree (and since the the majority of grad students are in lower earning subjects, this gives a strong downward weight). In undergrad we all knew this; a BS in computer science will earn several times the amount a BA in Slavic studies will. In pure terms of potential income, a bachelors in social work, drama, arts, etc is probably not worth the cost. It's similar in graduate school, you do a PhD in English literature, it will be a very hard road to secure funding and difficult to find work after graduation. A PhD in computer science is on the high end of this continuum. I would wager that if you go the computer science department at Columbia you'd find that the graduates are fully funded and are not struggling to find work after graduation.

Hi Theo,

Not offended at all. I am biased given my only experience is in the social sciences. I can definitely see how Ph.D. programs in Computer Science are different and are likely much more lucrative and beneficial for one's career, especially in the data sciences. I agree that articles like the ones I cited above focus on the Ph.D. field in general and mask individual variations, though I still think it is important to be aware of the larger issues plaguing our higher education system. Perhaps the takeaway is that one should research a program carefully when deciding to pursue a Ph.D. (considering facts like one's concentration-- invest in fields like Computer Science and not the life sciences/social sciences, graduation rates, funding, graduate student placement in non-academic fields etc). Thanks for offering an alternative view!

Hey Theo!

I'm currently a customer specialist for an energy supplier company, and not working in finance. On the side, I'm slowly learning the basics of mathematical thinking and need to learn stats 101.

Thanks for taking the time to write detailed advice, Theo! Much appreciated.

I agree that doing a PhD in a relevant area would be a reliable way to get where I want to go, and that it's damn hard to get there in another way. I think it's the right advice. But I guess I'm also willing to wager on my ability to beat the odds. I reckon my determination, strategy, and work ethic are above average. I'll just have to see if I made the right bet . . .

On the topic of Baidu . . . I met Andrew Ng today. Didn't ask about openings though.

I'll think of which papers in image recognition might be useful. I have the fly brain recordings, but 1) It's a really large dataset - dozens or more GB. 2) I'm not sure if I have permission to share it. 3) It's probably not worth it, I'm sure there's a much better EEG dataset available online to play with.

Hello. Going to throw a quick introduction in here.

I currently run a team at Microsoft focused on answering questions around Windows 10 servicing through data.

I've been digging into general data science type things, but I'm hoping to move into more machine learning. Trying to figure out how to direct my current career momentum in the right direction. Moving trains turn slowly though.

I've done neural networks and machine learning off and on over the years, but never really at scale, and never with deep practice. I want to have a chance to develop real mastery here.

Hey Theo,

I guess given the vague definition of data science, in some companies what I do would be considered DS. I never thought of it this way, I call myself a data (applications) engineer. What I mostly do is build tooling to automatically compile and create (large) datasets, and facilitate analysis of those datasets, whereas DS to me is more about analyzing data, and in particular with a predictive/ML focus.

There is no specific workflow I use, since my tasks vary a lot. I might use anything from qualitative research (for UX), to digesting textbooks and academic papers (to get the stats part right), to common coding processes (design, implement & test, iterate).

From my research for TP, I noticed that the range of tasks I do in my work is pretty unique, most people in my area are much more specialized on either data visualization & UX (with much more experience in that area) or backend coding (with usually a stronger focus on building ML-based components than what I do). I find that a challenge for my expert selection, so have to cast a wide net.

Theo, Thank you for creating the forum
This my quick intro.

Hi All!

I'm a practicing Physician in the field of Hospital Medicine (Hospitalist). I'm just starting to study the basics of how to get into machine learning like Stats 101 and R. My interest in machine learning when started when we work on identifying patients that will deteriorate. It is well known that majority of patient that went into cardiac arrest will have signs and symptoms (or data) that will warn us so we can intervene before the serious event will happen. We have a very crude way of doing this which is just designating a score to every variable and adding it up to trigger an alert or visual display. I would like to create a model to be able to do the same thing.

If you have a guide on where to start to achieve my goal that will be great.
I’m willing to put all my effort in studying and start at the very foundation to learn machine learning.
Just like now I would like to ask all of your opinion on what statistical package to use.

Is it better to study R or Python?

Any input is highly appreciated.
Thanks,

Btw, an interview with a top data scientist about his career path in case you guys are interested! :]

https://www.quora.com/What-is-a-data-scientists-career-path-1