My company has grown very rapidly in the last couple years and as the demands on the product team have grown significantly, the value around adding the Product Analysis function also grew. As more and more new team members joined, we needed to change the structure of our team to be more data-driven. Out of this need, we defined four guiding principles for this new Product Analysis function:
Own Instrumentation Suite - Setup and Maintenance
Heap, Optimizely, Google Analytics, A/B testing, and other measurement tools all fall under this new function. Previously, these applications were managed by different teams. These teams were unaware if we launched new pages or changed URLs, and that led to data quickly becoming outdated. It made sense for these to all be centrally owned by someone with deep product knowledge, so that we could ensure the data was current.
Provide Product-Level Insights on Usage
This function is all about knowing the customer. Where are they falling out of the funnel? What are the bounce rates? How can we improve engagement with our products? The Product Analysis function helps us see what the customers see. This includes identifying how we can improve the loan application to provide the best and fastest experiance for our customers.
Guide, Monitor, and Design Experiments
The drop off points identified in our Heap funnels, are turned into opportunities to work with Product Managers and Designers to create experiments that solve the root cause of that dropoff. Running A/B tests is a big way we validate these hypotheses, and although we want every A/B test to “win” we know that many will not, but can still provide great insights.
Reporting
As we’ve grown, our metrics have evolved from being project-based to product-driven based on our customers’ needs. The Product Analyst ensures our OKRs are on track, guiding us at biweekly meetings on how we’re progressing on those goals and also sharing what we have learned from our experiments.
This is what worked for us, but I always love learning from others. Please add your thoughts in the comments below on what you’ve done to build this function on your product team.