Ah the world of product usage. This is an idea that doesn’t need any pitching. Any company understands the value of it right away. Knowing how your users are engaging with your product is critical to building a better product.
How can you build a better product if you don’t know what features your users love? What about the features that they don’t care about? This is where your product usage comes in.
While the idea is simple, the execution of it can be quite hard. Analytics is a world of complexity but that changes today. My goal is to make product usage easy to understand and to start with. I’m a fan of aiming for “good enough” instead of perfection.
The tricky thing about “good enough” in data is that it feels weird to people. Incomplete data, missing data points, and imperfect tools all seem wrong to companies. The magic is making decisions DESPITE the lack of perfection.
Let’s start by talking about how to make product usage easy.
There’s no need to complicate this process. Finding the usage of your product can be as easy as 1,2 and 3. The catch is that you’ll want to add a step 4.
This step 4 will consume months of your time and resources. It could be anything. Achieving 100% accuracy, researching tools, collecting more data. Remember that as you read through this post.
We always want to start with the goal in mind. This could be higher retention, better onboarding, or more engagement. Some of these may feel like buzzwords (and they are) so we could also talk about higher revenue per user, lower acquisition costs, and lower churn.
The goal matters because you have limited time and resources. Time is obvious but resources are often misunderstood. The main resource that you’ll be limited by is engineering time. Almost everything that you will see in this article requires help from engineers. This may translate to setting up tracking, or tools.
Engineering teams are busy. As in, “we don’t have time for the next 10 years busy”. They are typically working on the actual product which is quite important. Nonetheless, you’ll need to work with them. This means that you want to avoid wasting this time.
This brings us back to the goal. What is the biggest area for improvement in your product? This could also be the biggest unknown. It’s common for me to meet a company that only has napkin math on key metrics like retention and churn. Sometimes, metrics are crappy regardless of what you do.
Once you have a goal, we’ll want to translate it into specific questions. Here are some examples:
Good questions will get you good answers. You can then iterate to even better questions.
The second step is to figure out what data you need and where it is coming from. This is the “technical” part of finding your product usage. If you love talking about tools, you will LOVE this step.
Most of the data that you will care about will be in the form of events. These are the actions that users take within your product. Signing up, uploading a profile picture, and watching a video are all possible events.
Since you have limited engineering resources, we can’t just track everything possible. We need to prioritize what we actually need. This is where a tracking plan comes in. This could be a spreadsheet or a software product. Either way, you want to determine the events, event properties, and user properties that are relevant to you.
I wrote an entire post on how to create tracking plans so I’ll refer you to that. Your tracking plan will also take into account what tools you will be using. You might be looking at tools like Google Analytics, Mixpanel, Amplitude, Segment.com, Heap, or something else. I have guides for most of these tools and any of them will do the job. Don’t forget about laying the foundation for automation too.
A caution tale on tools. I had a client who had way too many tools. They were spending upwards of $150K per year and barely using any of them. This is exactly where YOU don’t want to end up. Start small and iterate upwards.
Finally, you need to work with your engineering team to figure out where the data is coming from. This could be from the client (web or mobile apps) or from the backend. The former is easier to get started with while the latter is more accurate over the long term. The decision will come down to who’s helping you implement this.
The third step is to visualize the data. If you did step 1 and 2 correctly, this step should be a breeze. You got your questions and you have the data, you just need to put them together. Here are some considerations for this step:
I have a whole section on the dashboard design below but keep in mind the two different potential roles here. The first role is to summarize your KPIs aka a dashboard. The second role is to dig through your data for answers. Your selected tool may do both or just one.
That’s it. The process of finding your product usage is simple but not easy. Focus on getting these first 3 steps right. You can’t score a home run if you haven’t run through the first 3 bases first. You can also think of these steps as your initial design of a data-driven culture.
Let’s dive deeper into how to build a real-time product usage dashboard.
Everyone loves a good dashboard. It’s a great way to show what is going on in the company but it can also cause panic. You need to make sure that your dashboard is designed properly, otherwise, you will run into unnecessary issues.
Let’s start with the contents. Less is more. Remember that, don’t add too much. Start with a single dashboard that summarizes the most important product-related metrics e.g. north star metric. Take them through a logical sequence so you could start with sign up related metrics, move to the onboarding, cover retention, and finish with revenue. This is a great place to look at what Brian Balfour calls “success signals”.
The dashboard I described above might have 1-2 charts for each section. Don’t try to solve every possible problem here. People will get overwhelmed with too much data or if the data seems to contradict itself.
Next, choose the right time period. This will depend on the frequency of use of your product. Monthly tends to work well with B2B products while weekly or even daily could work well for consumer products. Funnels in particular will limit the data period as you want to have enough time for people to fully complete them.
Be sure to also add comparison time periods. It’s interesting to know what happened in the last 30 days but it’s even more helpful to know how this compares to a rolling average or the previous time period.
Finally, you can start to answer two questions: why do we think this happened and what do we think will happen?
Data on its own might not answer these questions but you can add annotations explaining specific charts. This is especially useful whenever you have unexpected spikes or drops. This lets you anticipate questions and answer them as you analyze the data.
A good dashboard can help everyone in the company feel more at ease with their work. Just make sure that it is designed properly before prime time.
I now want to show you real examples of how companies analyze their product usage. All of these companies are clients but their information will be anonymized. The name of the company is irrelevant as the principles will be beneficial on their own.
Analyzing Product Launches
Launching a new product can be stressful especially if you’re redesigning significant elements of your product. This example is a B2B product and they were launching their V2. We wanted to easily track the impact of this launch on our core KPIs.
We were using Mixpanel to track all the data and we first made sure that we had tracking for all the new features of V2. This meant adding new events.
We then took advantage of the “Impact” report that Mixpanel offers. It allows us to take a “launch event” and then compare the performance before and after the launch. Launch event here simply means an event that is associated with the V2. In this case, we had an event that was exclusive to V2 which fit perfectly.
Besides setting up the report, Mixpanel does all the heavy lifting here. We also created reports showing users switching between V1 and V2. The company used this in their outreach efforts to figure out why they were switching.
Finally, we created a brand new dashboard just for tracking the transition to V2. The entire process provided much-needed data to everyone involved. Even engineers were able to see the impact of their work in real-time.
Determine Cross-Platform Usage and Retention
The second example is for a cross-platform product available on the web, iOS, and Android. One of the main questions that this team had was around how users were engaging across the different platforms. Were they switching between the web and mobile apps consistently?
For this project, we used Amplitude to track all of our data. The cross-platform report showed us that most users tended to stick with the platform that they signed up for.
However, the retention rate of the users who did switch was higher than for those who stay isolated. Using the product across the web and mobile was a clear sign of “power” users. This can be something that could be incentivized in the product design and notification strategy.
Finally, the company was able to run different retention scenarios to determine what behaviors seem to be driving usage.
These examples are all built on the basic 1, 2 and 3 framework that I mentioned earlier. Finding your product usage doesn’t have to be complex. You can start small and eventually build-up to the “fun” techniques like machine learning.