Issue #2: Misalignment of Incentives
Even if you treat data as a process vs a project, that doesn’t necessarily translate into success. There are companies that spend endless time on the infrastructure, tooling and instrumentation but then miss something extremely important.
Getting an organization to use data requires behavior change among the individuals.
Changing behavior is extremely difficult. There is typically one culprit behind the lack of change: a misalignment of incentives and rewards.
Teams and individuals will ultimately do what they are rewarded for. So if you want to change behavior, you have to make sure that the behavior is part of what they are rewarded for. There are multiple types of rewards:
- Financial Rewards (bonuses/raises/equity)
- Progression Rewards (promotions)
- Authoritative Recognition (good job from boss/superior, etc)
- Peer Recognition (“good job” from peers, etc)
When you look around your team, how often are you delivering one of those rewards related to the use of data?
Here’s the problem:
If the use of data is not rewarded, then the work that it takes to instrument and analyze data will be seen as friction to doing what is rewarded, or what’s perceived as a “good job” within the company.
If the use of data is rewarded, then the work it takes to incorporate data will be seen as an ally to doing a good job.
Then ask yourself a second question: How big / often is that reward in relation to other things you reward for?
How big/often the use of data is rewarded compared to other rewards signal importance/priority. Managers need to be careful how they weight different factors as part of rewards.
What to do instead
1. Every team needs a KPI
Every team needs a KPI as part of their measure of success. Having a KPI for each team aligns the use of data to their work vs positioning data as a friction point.
Just setting the KPI isn’t enough, though. These 3 things need to happen:
- The team needs have a sense of ownership over the KPI.
- If they don’t feel like they own it, they will view it as someone else’s problem.
- Every single person on the team needs to understand the KPI and have easy access to viewing it.
Often times only the PM or a couple people on the team will truly understand it.
The KPI needs to be part of the four types of rewards, but not the only thing the team is rewarded for.
There are other important factors for product teams like shipping velocity, product quality, etc. Like most things, it is a balance.
2. Design systems for each type of reward
Go through each of the four reward types and design systems to reward for the use of data. What do I mean by systems?
Here’s a quick example around a system for Authoritative Recognition Reward: the best managers I know have checklists they use to prep for every 1:1. An item on the checklist might be “Did this person display the use of data in their work?”
If yes, make sure to give recognition for it. It is too easy to forget these types of things unless you have specific prompts.
3. Communicate the rewards clearly
Don’t assume that team members know how they get promotions, bonuses, praise. You need to put it in writing, make it explicit, and reinforce. You have to over communicate in order for it to get through.
When you promote someone, don’t just announce “Congrats to Jane Smith on her promotion to Sr. Product Manager.” Back the announcement up with examples of the type of work and behavior that the person has displayed which led to the promotion.
Issue #3: Data team becomes the bottleneck
If you solve process vs project and incentives and data starts to become valuable in the company it creates a couple of new problems.
The first is that the data team can end up becoming a bottleneck. This stems from a the data team taking an “ownership” mindset to the data, i.e. “We own the data.”
But ownership thinking leaves out one important point:
Every team has a “customer.”
The data team’s customers are the other people using the data internally: data analysts, product managers, engineers, marketers, etc.
In service to these internal customers, the data team needs to act like any other product team:
- They need to define their customer segments
- They need to understand customer needs
- They need to deliver the most compelling solution
- And they have to iterate!
In other words, their output must be able to enable other teams’ output, rather than them being the exclusive owners.
Issue #4: Brilliant Answers, Useless Questions
There’s a second issue that arises once data has become valuable and recognized, and that’s when people start noodling on data for data’s sake. Whether it’s because they find the data process intellectually fascinating, or just to seem smart, productive, etc. in the end, it’s data masturbation 🙂
Data for data’s sake is an alluring trap, but only creates “brilliant answers to useless questions,” as Ken Rudin (Head of User Growth & Analytics at Google, former Head of Analytics at Facebook) says.
Rudin reminds us that even though it’s alluring to increase knowledge, insight alone isn’t enough — results and impact are the true goal of analytics. To that end, data teams should make sure they’re asking the right questions, not just coming up with more and more answers.
Rudin has two suggestions for how to create a data process that actually serves business needs:
1. Hire analysts that are business savvy, not only academic about data or its tools.
When you interview candidates, don’t focus on just “How do we calculate this metric?” Ask them “In this business scenario, what metrics do you think would be important?”
2. Make data everyone’s thing.
Rudin’s team at Facebook ran a two week “Data Camp” for not just the data team but for everyone at the company. It gives the wider organization a common language to frame questions that data can answer.
A solid data process will be informed by asking good questions that result in a real business impact (Rudin also makes a great point that data teams must be accountable for not just “actionable insights” but the fact that those necessary actions get taken).
As data provides answers and actions, it helps shape better and better questions.
As data’s answers translate to impact, people will be incentivized to maintain and even improve data systems, leading to more data capacity, and stronger answers to better questions.