NaClhv

I've been working as a data scientist for some time now. This is a change for me. My background is in physics, and my previous work was mostly in education, teaching in the STEM fields. So this is an exciting period in my life. Career changes often are, and data science in particular is an exciting field, having been called "the sexiest job of the 21st century" - a description that I'll not complain about.

All this excitement has lead to people asking a number of questions about my job - in fact, I recently ran into an old friend who was contemplating a similar kind of career change, and I told her that I loved my new job. She then asked me the following questions:

1) Why do you love being a data scientist? Why does the job fit you like a glove?
2) What do you do day-to-day? What's a typical week like?
3) What were the challenges starting out? What are they now, some time later?
4) What are the best and worst parts of the job?
5) What are the top 3 actions you took to successfully land a job in data science?
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
7) Anything else I'm missing?

Here are my answers:

1) Why do you love being a data scientist? Why does the job fit you like a glove?

Doing data science is like all the fun parts of doing physics, without any of the drudgery. Iteration times are much faster. If I'm curious about something, I can just look in the data, instead of having to go get the liquid nitrogen, and wait for the vacuum system to pump down, and for the coarse approach to finish, etc. I can typically produce something meaningful in hours or days, instead of weeks or months.

I never was very good in the lab, with all the physical equipment. I was better in a more theoretical, abstracted setting. I felt more at home when sitting in front of a computer, or thinking about a math problem on a whiteboard, rather than working with physical lab equipment. Collecting physical data is hard - it's what ends up taking all the time and effort in physics. But in data science, there's already so much data out there that much more of my time could be spent on exploring, processing, analyzing, and acting on that data. I can do things like that for hours or days on end. So a data science job fits well with my talents and temperament.

The lengthy and time-consuming nature of physical data collection was another part of science that I didn't particularly like. It was hard to stay motivated - I can work hard on a problem for an extended period of time, but if there was no real sign of progress after a few days of focused effort, that would be very discouraging to me. This happened to me frequently in physics, and it probably ultimately prevented me from doing well in grad school.

By contrast, data science projects generally give me some real return on my investment within hours of putting in the effort. In addition, working in industry as opposed to academia means that the results of my work impact the real world much more quickly. This higher rate of reward for my efforts helps me stay motivated and is much more suited for my personality.

On a more big-picture level, I like the idea that my primary job is to think soundly about data. That, in the abstract, is a profound activity, closely connected to the depth of the nature of the universe. In that sense I still feel that I am trying to "know the thoughts of God", in much the same way that I felt when I was exploring the "big questions" of physics.

2) What do you do day-to-day? What's a typical week like?

I usually do some programming, some SQL data pulls, visualize data by making graphs, attend meetings, etc. These pieces get put together to answer a data question that's relevant to my employer, like "How effective is this feature in our business in serving our customers?" or "Which of our potential customers would benefit the most from which of our offerings?" I usually work on answering several of these questions per week, with some of the larger questions taking several weeks to answer completely.

There is also an element of teaching and education, as part of the role of a data scientist is helping others understand what the data is saying and what actions we can take in response to it. This is great for me, as it scratches my itch for teaching and personal interactions - more things that I couldn't often get while sitting in the basement in a physics lab.

3) What were the challenges starting out? What are they now, some time later?

Coming from a background in physics, there were some new things I had to pick up. Obviously, I had to ramp up on the programming end. I had picked up a smattering of obscure, little-used programming languages throughout my physics career, but I finally learned more useful languages like Python or JavaScript some time before switching to data science. Apart from general programming, there's also some specific sets of modules used by data scientists, and I had to become proficient in those as well. I also picked up some statistics.

In all these thing my physics background served me well. A great deal of learning to do data science was just learning to implement the things that I could already think of to do, in a specific development environment. But the foundation of the necessary abstract concepts and the mathematical tools were already in place.

Now that I've been in the field for a while, I'm still challenged to continually improve and add to my skill set. I see more of what's out there now - there's so much to branch out to that it's hard to say what I should learn next. There's also the other, more nebulous, non-technical questions that I have to answer, like "how can I maximize the impact that my work will have in my company?" or "how should I split my time between exploring new ideas and doing what others have asked of me?" I didn't have much time to contemplate these questions when I first started, but I find that they're becoming more important now.

4) What are the best and worst parts of the job?

My favorite part of the job is probably something that I've mentioned above - the idea that I'm sharpening my data interpretation skills, that my job is basically to think soundly about data.

It's also great to see my ideas make a difference in the real world very quickly. People are having a different experience right now because of what I've done. My co-workers come to me for help with the data, and my recommendations determine how they do their job and how our customers interact with our product.

I also like the regular, but flexible, hours for my job. I don't have to worry about being "on-call" like some software engineers do, where they have to get up and do work at 3am on a weekend when the website totally breaks or something. I work during normal working hours, but I can easily switch around things like coming in late one morning for staying late another day.

There really aren't that many bad parts. I guess I sometimes end up waiting for the computer to process things, which you sometimes have to do when you're looking at billions of rows of data. Sometimes the work can get a little boring, when I have to repeat a similar, routine analysis. Some of my work ends up not having an impact, either because other people ignore it or because the conclusions end up not being actionable - and that's annoying for the same reason that making a difference is satisfying. But overall, the good definitely outweighs the bad.

5) What are the top 3 actions you took to successfully land a job in data science?

First, I learned a great deal of programming on my own. I picked up Python and JavaScript, because the school I was working at asked me to teach these, and because I wanted to do more with my blog. Some of these pre-career change projects can be found on the other posts on this blog. The programming skills, combined with the quantitative mindset I already had from my science background, positioned me well for a data science job.

Second, I applied to a data science boot camp. Insight is a well-known program, as is The Data Incubator. There are many others. Seriously, if you're wondering about a career in data science, apply to one of these programs now. Just the application process taught me a ton of things, ranging from "what does a data scientist do?" to "how do I scrape a webpage?" to "what specific tools do I need to become proficient in?" It will answer a lot of the questions that any potential future data scientist would have.

Third, I applied to jobs. Usual job seeker advice here - be tenacious, send to multiple employers, continue to practice interviewing, etc. If you've learned your craft well, you're tremendously valuable to potential employers - so believe in yourself and keep at it.

6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?

I'm happy with how things turned out - and I see now that I was already on this trajectory from 2 years ago - so I'd tell myself to keep going. As for anyone with a science or mathematics background contemplating a career change, I'd say go for it. At least, try applying to one of those boot camps.

I'd also say this, on a more "big picture" level: I think that computing, and data science in particular, is the pre-eminent field of our age, in our current moment in human history. I first got into physics because of people like Einstein or things like space travel or nuclear power - but these are very much 20th century endeavors. I still love physics, and there are certainly still some interesting activities there - but computing and data is where all the really exciting things are now happening.

7) Anything else I'm missing?

I can't think of anything really "missing" from the questions above - so here's a summary instead. I love my new job. I love my new career, considered as its own academic field. It has a number of varied characteristics that all fit my traits and needs exceedingly well: the required talents and skills, the reward schedule, the mix of group and solitary activities, the working hours, the connection with all other fields of study, etc. Of course, every job is bound to have some downsides, but I'm very content overall - I couldn't expect much more from a new career. If anyone is thinking about a career in data science, I'd encourage them to at least give it a serious pursuit.