Starting off with Product Analytics
Google Analytics (GA) was always considered to be the default tool for all things related to web analytics. But that was before I built my first mobile app and realized that GA was not the best tool to help me understand what a customer does inside the app. That’s when I first got introduced to Firebase and a host of other tools and frameworks that kick-started my #ProductAnalytics journey. The more you understand the role of Product Analytics in your product development journey, the more you will really appreciate the different lens that each of these tools takes.
I started researching competing products in this space. Mixpanel stood out as the most advanced app for what I was looking for with #ProductAnalytics, and it had pivoted itself to focus purely on this topic, which was evident from it’s powerful analysis and reporting capabilities. Others that I reviewed included Amplitude, Clevertap, Webengage, Moengage, and Netcore. But my final decision was based on my own (but brief) analysis of the product demos and feedback from references and friends in the startup and tech space. Each tool has its own strengths and weaknesses, so you must take your time and evaluate the one that suites your needs the best.
I next started pouring through the extensive documentation that Mixpanel has for its product. And everything pointed to just one topic — Identify your #NorthStarMetric and Key Performance Indicators (#KPIs). It is really important to understand and collectively decide on what you are measuring for and why. Your North Star Metric must meet 3 key criteria — lead to revenue, measure progress, and reflect customer value. Your KPIs must add up to your north star metric. And if you don’t have your KPIs for your business and your product clearly defined, measuring can become an endless and fruitless exercise. So if you are the Product Manager heading this initiative, take a step back, take a deep look at the product and business, and map these out. What you lay down will evolve over time as you continuously question yourself, but starting to think along these lines is essential.
My team and I evaluated the framework that Mixpanel provides, but settled down on the more popular #AARRR marketing framework, as we felt it helped us better define KPIs for our funnel. Once the product team defined the overall metrics, we discussed these with key members across ALL functions (design, operations, marketing, technology and product) to debate on what’s being measured, solicit their feedback, and ensure we were all working towards a common goal. This also enabled us all to finalize our business KPIs for FY22, and every function’s KPI were further defined as a subset of this metric.
The next step was to document our north star metric, add the AARRR framework under it, and then define what each letter in the AARRR framework meant for us. Once we were clear on what each covers, we defined the metrics and the data source that fed into each AARRR aspect. As you do this exercise with your team, take some time to define these carefully, identify all the sources from where the data can be collected, and debate them in detail. Ask your team to challenge your assumptions.
We found that there is no one source where all this comes from. For us, the key sources of data are as follows:
We are still working on our referral metrics, but those will only come into play when our referral system is setup. But narrowing down this core list of metrics too was challenging, as many key metrics are still left out. Reaching this is a small milestone in itself, as after this its all about execution and learning. We jointly agreed to setup our systems to measure these parameters before adding anything more. No TPS reports for us!
In the next post, I’ll elaborate on the steps we took to setup all this in our app, and the key learnings we got out from this process. There is a lot of trial and error that needs to be done to get it right, and a constant learning process as you discover more. So if you are in this game, be prepared to take a focused and methodical approach to the madness of #ProductAnalytics.
