Dear Advisor: How to work with a data scientist?

May 2022

You keep hearing that “data is the new oil” and are considering hiring a Data Scientist to help you. But do you need one? How do you evaluate their skills? What are some tips to make sure the collaboration is off to a good start?

I’ve been in the Data space for over 10 years, managing, hiring and mentoring Data professionals – and collaborating with executives of companies of all shapes, sizes and industries on data-driven initiatives. Here’s my advice to help you answer these questions

Step 1: Evaluate – do you need a DS?

As you may know, hiring and retaining knowledgeable people in tech is hard, especially now. Let’s make sure you actually do need to hire one – and if you do, whether that should/not be a data scientist.

Is/will data be core to helping you make data-driven decisions about your business?
Do you know what’s your most pressing, high-impact, low-hanging fruit business question you’d like to know the answer to, to help you make better, actionable decisions?
Do you currently have data in a fairly accessible format?

I would argue that the answer should be ‘no’ if you don’t know whether it exists – or if you’re not sure about the answer to this question. :)

Are you currently using your data to already understand what’s been going on in your business historically and now?
Are you looking for someone to help you streamline/automate (semi-)known repeatable motion(s) to acquire/retain customers?
Did you know that there are no guarantees in predictive models/ML/AI? And that it’s different from software engineering, and comes with its own unique challenges?

If you've answered ‘no’ to any of the questions above, at this point-in-time, hiring a Data Scientist is not the right solution to solving your pain point.

Question: OK, I won’t hire a Data Scientist just yet. Whom would you recommend?

If data isn’t/won’t be core to helping you make data-driven decisions about your business, I imagine that an investment into data and data science will not be worth it.
I share advice on how to formulate and prioritize business questions on my blog and for dot.la
If you don't have data in a fairly accessible format, or just aren’t sure, here’s advice around evaluating your data quality, in the context of trying to answer the business question (from the previous bullet).
- - Spoiler alert: it’s not a fractional Chief data Officer, but a fractional CTO or Data Architect can better help you set-up/update the infrastructure and begin/continue collecting data.
I imagine there’s a lot going on at your company. If you (understandably) don’t currently understand everything that’s going on in your business, I’d recommend tackling the most pressing business question first. Then iteratively tackling the next most impactful business question, which should also iteratively improve your product positioning and market fit.
If you’re still (understandably) trying to find out what those repeatable motions are, I share advice on how to get started in my talk on the Product Market Fit roadmap and more about ML + repeatability here.
I point out some of the predictive modeling challenges in these blog posts:
- - Hard to estimate how long it may take to develop a predictive model, because software ML has its own unique challenges, including:
  - Promising predictive model is performing differently on live customers than it did historically
  - Your product may be too early for ML/AI

Step 2: Recruit – how do you hire a DS?

It sounds like a Data Scientist may be the right hire at this point. Great! Here’s advice to guide your hiring process, including tailoring the job description to the skills that will help you solve your processing business challenge.

Question: OK, but what if I’m not technical enough to evaluate their skills – and also don’t have anyone on my team to help. What do you recommend?

That’s a great question! I recommend leaning on your technical advisor(s), board member(s), fractional data executives (like me) – or someone with a Data background you trust in your network, to help you interview candidates.

Step 3: Collaborate – how to successfully work together?

You’re now ahead of the curve. You have an actionable business question you need help trying to answer. Go ahead and share that context with your new hire; here’s a template I use– and includes the recommendation of discussing checkpoints along the way.

Notice that in this case, you're sharing target outcomes, not asking the new hire to implement “method X”, which may or may not be applicable.

I also recommend sharing and finding out more about how each of you collaborates best. I share advice with strategies and talking points here, on how to stay on the same page throughout the collaboration.

At the end of the collaboration, I highly recommend a post-mortem, to evaluate:

What went well?
What didn’t go so well?
What can be improved for next time? This is especially important to discuss if the deliverable didn’t help you make the decision you needed.

Step 4: Maintain – any advice on long-term data science strategy?

Question: You mentioned that data and data science are expensive investments. If data is core to my business, will there ever come a time when I can stop listening to data?

That’s a great question! In my (potentially biased) opinion, the answer is “most likely not” for these 3 reasons:

If data is core to your business, any predictive models that account for business insights/logic/knowledge tend to perform better than those that don’t.

Anything you build that becomes internally-/externally-customer facing, you need to support and maintain, as long as it’s getting used. For this reason, my typical advice to founders is, if functionality you’re considering developing won’t be core to the business, I’d recommend buying it, so that you can outsource support.
While repeatable motions to engage your customers are derived from historical customer actions, there is a way to make data collection be forward-looking, to help you identify promising product features or even pivots. I share this advice in my “Product Market Fit roadmap” (slide 17):

Rather than sporadically asking your customers if they’ll be disappointed without the product, to try to optimize for the “magic 40%”, I highly recommend asking for feedback -- as part of the product itself, to help you get ideas for new product features.

I show an example of what Amazon’s doing. Each product has its own drop-down and asks for more information. This way later you can evaluate quality, price, etc. for each item.

At one start-up I collaborated with, I read through the feedback and actually came up with a way to improve the return process that was really specific to the company’s business -- that would reduce churn and improve profit, which I wouldn't have thought of otherwise.

Parting Advice, Caveats, Challenges and Pitfalls

I’ll admit, this 4-step process sounds deceptively simple. What can go wrong?

It may take 6+ months to hire
You and your team may encounter many challenges working with data, such as:
- - Not all of the required data was captured to be able to answer the business question. I share advice on the types of data I recommend collecting ethically and legally
  - Data was captured but it’s not clean
  - Hard to estimate how long it may take to develop a predictive model, especially if this is the first time someone will be working with data
  - Promising predictive model is performing differently on live customers than it did historically

Challenges notwithstanding, you’ll learn something new – whether that’s about your customers, your product, or business – or how to better collaborate on the next project.

Are you getting enough applicants? Do you need to improve your job description? Please schedule this flat-fee session.

Good luck! If you’d like more support at any step of this process, please reach out. I’d like to help you understand and solve your customers’ pain points and support your team.