The world changes over time. That’s a fact whether we like it or not. When looking into your numbers, you might notice factors that seem to change for no real reason but time. Understanding why these changes occur and why your products perform how they do is the heart of product performance analysis and the key to your business’ growth. 

What makes a great product? Is it audience reception? Hitting a certain number of units sold? Making a certain amount of profit?

The answer depends on the industry, business, and specific strategic goals. A profitable product can be considered a failure, while a break-even one is a resounding success.

In order to determine your product’s performance and analyze it with your business plans in mind, you need a method that yields information on the individual product level. In other words, you need to employ product performance analysis.

What is product performance analysis?

Product performance analysis is all about measuring how well your product is doing with respect to the goals that you set out to achieve. It’s a process conducted by higher management, as they’re the ones with the information necessary to make such evaluations.

Unlike other analysis methods, product performance analysis is highly individualized, targeting a specific product at a time. It’s extremely rare for a business to launch multiple products with the same goals in mind, after all.

This means that every time you launch a new product, you’ll need to perform product performance analysis from scratch. But worry not! Once you’ve got the hang of it, the process of data gathering and analysis is actually quite simple, and similar methods can be used each time.

After all, it’s not how the data is being gathered that changes but what data you need.

The advantages of analyzing product performance

Before we get into the nitty gritty of how product performance analysis works, let’s talk about some benefits.

What we’ve laid out below will give you a general idea of what product performance analysis can do for you. Granted, this isn’t a complete list of the benefits it provides. That would stretch the word count into the high thousands as we’d need to get into really specific, niche situations. 

Let’s see what you have to gain.

Increased knowledge of customer-product interaction

Knowing how your customers interact with your products is vital when you’re in the product development stages of your project.

When designing a product, the first thing you should think about is what exactly that product will be used for. Then come the following questions:

• What will it do? 
• Who will use it? 
• Why does it beat out the competition? 

Information from customer-product interactions will help you answer these questions in a data-driven way, by giving you insight into the minds of your customers. It’ll tell you what they want from your product, how you can improve it, and where you’re hitting the mark.

Performing customer-product interaction is especially important in software fields, where the ever-changing nature of the market means that you must keep a close eye on what your customers want from you.

Adobe has certainly taken this to heart, with their Acrobat 2020 version featuring several unique features that are clearly based on customer interaction data. 

For example, the software giant added a color customization capability in the Fill & Sign tool, allowing you to choose specific colors for your signature.

On top of that, the 2020 version includes:

• Improved accessibility features.
• Support for the use of tablet pens when using the software.
• Increased control of signature panels.
• Touchbar support for MacBooks.

Many of these improvements are most likely born out of going over user data, telling Adobe that:

• Users with accessibility needs were using their software.
• Users were using tablet pens and other similar devices while using their software.
• Users were signing documents with their software.
• A high proportion of users were using MacBooks.

While the software industry benefits significantly from customer-product interaction, especially due to its measurable nature and ease of data gathering, it’s not the only one.

Other sectors can enjoy the fruits of such analysis by gathering user feedback via surveys or other organic manners like customer reviews. With the right analytic tools, you can discover a trove of data that will help you improve your products.

Reduced customer churn

It’s also been well-documented that acquiring new customers is significantly more costly than retaining your current ones. 

That’s why many businesses must invest in churn-reducing strategies. These include:

• Lightning-fast customer support.
• Behavior-based predictive analytics.
• Personalized and focused promotions.

Another way is to give your customers what they want and need. By tapping into the information you’ve acquired from your product performance analysis, you may be able to adjust certain aspects of your product to keep your customers happy. 

Twitter, for instance, is rolling out an Edit function to select users, something that users of the platform have been asking for for years. 

Improving your products is an excellent place to start, but it goes deeper than that. You need to make products that are not only useful but ones that understand your customers’ needs.

Recognizing user engagement drivers

It’s all well and good knowing what your customers like about your product, what they use, and what they don’t, but understanding the drivers of these is another matter entirely.

This is especially true for offline, physical products, where the link between what a product is nominally intended for and what it is actually used for can be radically different without any indication. 

I’m sure you’ve probably seen one of these, whether it’s what you made when you were in school or simply in passing.

It’s certainly recognizable as a spider decoration, but what isn’t always known is that the legs are made from pipe cleaners. These days, it’s very rare to come across one being used for cleaning pipes, actually.

Looking from the perspective of a product performance analysis, making arts and crafts with these would have been totally unexpected. Yet, it’s the main user engagement driver for pipe cleaners these days. It just goes to show, what you think a product will do isn’t always what it will end up doing.

A better understanding of your customer base

When attempting to create products, one of the first questions that are asked is often “who are we making this for.”

Tying in with user engagement, different demographics of consumers tend to have different habits, different ways of doing things, and different features that they want out of a product.

It’s true that everyone is an individual, and that no two people will want exactly the same thing. However, you can still make generalizations.

It’s not stereotyping to say that people over 70 will want large text options in media, after all that demographic tends to have worse eyesight than those younger than them. If you’re targeting over 70s, you should therefore include these options.

However, it isn’t always clear who is using your products. Looking at the case of the pipe cleaner spiders, the customer base for pipe cleaners is radically different from what was expected at the time of launch. 

Manual laborers vs. schoolchildren is quite a big difference, after all.

Product performance analysis can give you information on your customer base, who they are, where they’re based, what they do, etc. Information is power after all, and you can make your products more specialized and therefore more relevant if you know exactly who is buying them.

Of course, it’s entirely possible to end up with several different demographics buying your products. In fact, that’s likely the norm in a lot of markets.

What happens then? You can continue adding features that appeal to all demographics to make a product that’s a good average, or you can specialize.

One company that’s been on the ball with specialization in recent years is Wizards of the Coast. Their Magic: the Gathering cards are purchased by two main types of consumers:

• Those who want to play the game because they find it interesting.
• Those who want to collect the cards because they find them aesthetically appealing.

Starting in 2019, Wizards began making special cards with alternate artworks, designed to appeal to the second cohort of customers. You can see below the differences between these cards and the regular releases, with one being practical and easy to read, and the other more focused on appearance.

These alternate editions of cards proved a huge success. As a distinct entity, they didn’t appear in regular product editions and so appealed to the second cohort without any backlash from the first.

How do you perform Product Performance Analysis?

Below we’ve laid out a list of steps for you to follow in order to create your analyses. With these templates in hand, you’ll be able to make adjustments that would fit the specific needs of your businesses and be on your way to a successful analysis.

#1 Set your goals

In an ideal world, all you’ll have to do is push the “analyze my product’s performance” button. In reality, setting your goals means deciding how to analyze your product. In other words, the first step is to determine what you want to achieve with your product. 

What would you consider a success? A product that’s a failure in one regard might be successful elsewhere. Some products are launched solely to make money, others have different aims. 

This question will dictate the metrics you’ll be using. Is it units sold? A certain profit margin? Beating out a competitor?

Then, you’d need to set up the timespan you’ll be looking at. Remember, some products take time to reach their full potential.

Let’s take a look at some examples to get a better feel of how it works. 

Promotional items

I’m sure you’ve seen those small gift bags given away at events and conventions. You might’ve handed some yourself.

As you may know, the purpose of these items isn’t to make a profit (not directly anyway), but to give event attendees a physical reminder of your presence there and warm them up towards your brand. 

To measure success, you may use “items given” as a metric. Perhaps a better one would be expecting an increase in traffic on your website in the days following the event. 

Raising awareness

Some products or campaigns can be created solely to bring attention to your brand. 

A good example of a campaign made to raise awareness is PlayStation’s Play and Plant initiative. For every player that reaches a certain section of the video game Horizon: Forbidden West, the corporation will plant a tree in their name.

 

Players won’t buy the game to plant trees, but this initiative generates positive headlines for PlayStation and raises awareness of both the game and the brand.

Measuring the success of similar awareness efforts isn’t easy. 

Awareness is the first stage in the marketing funnel, and it may take a while for potential leads to convert into paying clients. Your best bet to assess this is by monitoring traffic, leads, and conversions over time, trying to find an uplift in these metrics.

Boost sales of another product

Whenever you look to purchase a new phone charging cable, it’s super cheap. In fact, you’ll probably wonder how on earth any profit is made from them. 

That’s because they’re not in fact designed to make a profit, but to support the use of the mobile phones that particular business sells too. 

With different types of phones come different connectors. If you purchase an iPhone right now, for example, you’ll need a different charging cable than you would if you purchased an Android. 

Put yourself in a buyer’s shoes for a second. 

If a particular brand of phone needs a charging cable that is expensive or difficult to find, you’re likely to look elsewhere for a complete set. 

You might even decide to buy a different model of phone when your charging cable breaks, rather than simply replace the cable if it’s costly or tedious to do. 

Thus, phone charging cables can be seen not as profit makers, but as support to the phones that a maker sells. Their success would be measured not in profits, but in customer retention rates. 

Ultimately, it’s up to you to decide, and with any luck, you’ll be able to pick sensible options that align with your business goals.

#2 Define the relevant data

Once you know what you’re looking for, the next step is determining which data is relevant. 

If you’re looking at product features, you’ll look to detect statistics and other product data sources.

If profit margins are your game, look to financial figures.

This step follows from the first, but the metrics that you use aren’t always obvious. If in doubt, think back to your overall business goals as an organization. Aligning these with your data collection will greatly assist in choosing metrics.

#3 Pick your methods of data collection

Once you’ve decided what metrics you want to track, the next step is to decide what methods you’re going to use to collect the data. 

Once again, this step is heavily dependent on the previous. The data you want to collect will determine the method of collection, with only a few different options available.

Are you looking for quick, easy-to-collate product ratings? Look to STAR reviews and similar.

Do you want detailed information on your product’s capabilities and how they meet your customers’ needs? You’ll need to search for more detailed review data.

Data collection can be done both in retrospect and in real time. Real-time data is usually considered more up-to-date as customer opinions can shift over time, but it is also far more difficult to collect.

Some ways of obtaining real-time data include sending out regular surveys, conducting incentivized review programs, and using software to obtain new information from the internet.

#4 Collect & analyze customer data

Once the first three steps have been decided, it’s time to put your data collection plan into action. 

The time scale of your plan will vary depending on your particular plan, but in general data collection should go fairly quickly.

When the data is collected, you should make sure to format it correctly. It’s no good having all the data in the world if your analysis software can’t read it. 

This is especially true when pulling data from multiple sources that might have different internal means of formatting. 

What analysis tools you use, and what software you use, will again be determined by your needs. In the next section we’ve laid out a few different ways of performing analysis that can be useful in different contexts, and with any luck, you’ll be able to determine which one is right for you.

#5 Put the information you’ve obtained into action.

Finally, you need to act on the information you’ve found. There’s no use in doing all this analysis if you’re not going to do anything about the situation, after all.

When acting on your information, it’s important to keep in mind that this is all retrospective. Information quickly goes out of date, especially in the case of volatile markets such as fashion or software.

Ultimately, what you do will be heavily dependent on your situation, the information you’ve found, and the market you’re in. There isn’t really a right or wrong way of acting, however, we can give general advice on scenarios that might pop up.

• Are you seeing that customers aren’t happy with a change? Rolling it back will be the way to go.
• Are your products seen as outdated? You’ll need to look at your designs again.
• Do you see a drop in sales? There will be a reason for that, you need to look further.
• Are your competitors’ products seen as more desirable? Take a look at them and see what they’re offering that you’re not.

Analyzing your data

Product performance analysis has several functions, with the one you pick being the one most relevant to your goals. Below you’ll find some of the more common ones, how they function, and why they’re important.

Bear in mind that many analysts use multiple categories in order to get a more rounded view. However, the more types of analysis you use the more likely it is that you could get mixed signals from your data, so be careful!

Funnel analysis

Funnel analysis is all about diagnosing your sales funnel, a term used for customer journey analysis that starts with marketing, goes through to sales, then ends with purchasing. 

It’s about the actions the customer takes, rather than the actions of your business.

Sales funnel analysis will look at the process’s separate parts, from the first interactions your customers have with you, to the end of the line when the purchase is made.

Trend analysis

Trend analysis in this case means tracking customer behavior. Opinions and beliefs change over time, and the behavior that these create will change accordingly.

Trend analysis uses both current and historic data to determine where the market is headed. It’s not always obvious when a change is beginning, so you need to be careful. What looks like a small rise in a figure could actually be the beginning of a drop, or vice-versa.

In short, only trust your data if it’s showing these trends over a long period of time.

It’s extremely useful to split customers into their demographics when performing trend analysis, as different demographics tend to have different behaviors overall. This is especially great for age groups since younger consumers tend to prefer a more tech-heavy experience than older ones.

Cohort analysis

This one is trickier to define. A cohort is similar in some ways to a demographic, that being that they are all customers who experience common things. That being said, it’s not quite the same as the event can also be factors like the month or time of day when the purchase occurred.

Cohort analysis is extremely useful as it tells you facts about your products over time. 

Are you seeing an increased number of complaints about product defects in one month, but not the one before it? It’s likely that your manufacturing tools have developed a fault.

Did reviews drop after a competitor launched their product? Customers are probably comparing your product to your competitors and finding that they fall short.

What’s difficult with this type of analysis is determining which cohort a customer will fall into. This is much easier in ecommerce, where everything takes place online, but can be done with proper data tracking for offline purchases too.

Customer journey analysis

While this might at first glance seem to be the same as sales funnel analysis, you need to remember that a customer’s journey doesn’t always stop at the point of purchase.

Customers are a valuable advertising tool, with it being well known that a friend referring a product makes it far more likely that someone will purchase from you than if they have no information.

Customer journey analysis tracks a customer’s behavior in the following stages:

• Awareness.
• Acquisition.
• Adoption.
• Assimilation.
• Advocacy.

In each stage, the customer becomes more and more likely to do some of your marketing for you. Analyzing these steps will allow you to see what makes a loyal customer, and how you might improve the process.

You can read more about customer journey analysis here.

A/B tests

A/B tests are a means of checking if you’ve made the right decisions. They involve two metrics, one that you have control over, and one that you measure the outcome of.

Take, for instance, the color of the packaging you use for your product. This can be your controlled metric.

By changing the packaging from purple to orange, you can then measure the overall impact that this has on the other variable. This way you can optimize the things you can control about your product.

A/B tests are often used in the product testing stage with test audiences, as this is much less costly than rolling out the changes across an existing product. 

That being said, test audiences do miss things. Limited-time releases can serve the same purpose as tests, with the changes potentially being made permanent or into an offshoot if they prove popular.

Coca-Cola Raspberry, for example, was first sold on a trial basis in New Zealand back in 2005. It was discontinued as planned at the end of the year, but has since been brought back and expanded to four continents.

What metrics you should use for product performance analysis?

When the word metrics is used, what it means is anything, absolutely anything, that can be measured and recorded about a product over time. This can be something as inconsequential as the volume levels of production machines, or as important as sales figures.

When we’re talking useful metrics, it refers to anything that can be used as a measure of a product’s success. 

We’ve laid out a few of the more important ones below, but be aware that it isn’t a be-all, end-all list. Any factor you think is important can be taken into account – it’s up to you to determine what is, and isn’t relevant.

Business-oriented metrics

These metrics mostly look at things from the perspective of your business and it’s relevant processes.

Revenue per product

Let’s start with the big one. Revenue.

The amount of profit you’re making is extremely important, as the end goal of most businesses is to make money and stay afloat.

The amount of revenue you make per product sold determines your profit margin. It will also tell you if it’s worth expanding into new territories where shipping costs will be higher, and whether or not the product would be worth keeping at all.

It all depends on the strategy you’re using. Some products rely on a high volume of sales with a low-profit margin, others the inverse. Keep your strategy in mind when analyzing this metric.

Cost to acquire new customers

It’s often said that it’s easier to keep existing customers than it is to attract new ones. While true, more customers is always better.

The cost of acquiring a new customer will give you information on how you should proceed with your marketing plans. Is your product one that will easily attract new faces? If so, you should keep it at the front and center of future marketing efforts.

On the other hand, if your product isn’t very attractive to new customers and will likely cost a lot to obtain, you’re better off focusing on keeping your existing customer base.

Customer lifetime value (CLV)

Customer lifetime value refers to the amount that a single customer is worth to your business over the entire span of your relationship with them. This can be anywhere from a single purchase to a loyal, lifetime subscriber.

CLV is a useful metric as it will tell you about customer behavior. It ties in with the customer retention rate metric that we’ll talk about further down, but is a more numerical indication with other functions.

How far should you go when attempting to retain customers? At what point should you be fine seeing them go? CLV helps you determine at what point it’s still profitable to try and keep a customer. 

Customer-oriented metrics

These metrics are all about your customer base, and how they interact with your product.

Revenue per customer

How much is a particular customer spending with you? Can you expect and rely on a high volume of purchases from a small group of people, or should you aim to market to a wider audience?

Revenue per customer ties into the customer’s lifetime value, but is aimed at short-term analysis in the immediate future, rather than the long term.

Customer sentiment toward products

How do customers feel about your products? Remember, emotion is a huge driver in choice when purchasing a product. Customers are willing to pay up to 140% more with a brand they’ve had positive experiences with in the past.

It’s the main reason we’ve created Sentimate, the world’s first AI-powered product insight engine.

With Sentimate you will have access to the combined might of all publicly available reviews on a product, using sentiment analysis to detect customer sentiment in highly contextualized situations. 

Customer retention rate

The proportion of customers who can be considered loyal is another great source of information.

How many of your customers stay with you after purchase? Can you expect repeat purchases, or should you aim your marketing more toward one-time customers?

Your customer retention rate will be highly contextual, depending on factors such as industry, sales model, and target demographics. By factoring that in, you can learn a lot more about your products.

For instance, USA Pan offers a lifetime guarantee on some of its products. That’s a literal lifetime, meaning that at any point between purchasing from them and your death, they’ll replace your products for you if they break.

Obviously, a low customer retention rate for these products isn’t a problem. In fact, even if it were zero then they wouldn’t see a problem with it.

If you sell a product with a limited lifespan, on the other hand, you’d be concerned if your retention rate was low. Context is key to this metric!

eCommerce-only metrics

We’ve split this section off from the main body since it’s so specific to analyzing the way ecommerce functions. 

eCommerce has a huge advantage in product performance analysis, since by it’s nature, information is communicated back to the hosts. I’m sure you’ve seen a little pop-up when an app crashes, asking to send a report.

The advantage is even greater in the case of self-hosted websites, in which the software that runs it is hosted on your own internal servers and therefore can be analyzed in the most minute of details. You don’t need permission to analyze your own files or to gather data on how your own servers are running, after all.

These e-commerce-specific metrics are heavily indicative of how well your assets will function, and of how they’re being received by users. While analyzing them might not be that straightforward, a means of gathering these metrics should be built into them in order to make data collection simpler.

Task times

How long does it take your customers to perform certain tasks? Is it around what you’d expect or does it take longer?

See, the thing about navigating websites or apps is that once you know what to do you’re completely fine. Developers will naturally know what software does and how to perform tasks, as they know it inside and out.

Your users, on the other hand, aren’t so lucky. If a task takes them significantly longer to perform than you think it should, they’re probably having problems. 

Whether that’s with finding the correct links, the interface layout, etc., is up to you to find out, but a task time will tell you that there is a problem in the first place.

Task exits

On the flip side, there are task exits. Is there a particular task or feature that a high proportion of users exit from? If so, you’ve probably identified a pain point that needs to be fixed.

Any place where many users stop using your website or app is an absolutely huge problem. It’s something that’s big enough to stop them from interacting with you altogether because of how much it frustrates them, which is the exact opposite of a good customer experience.

Errors logged

Errors are pretty self-explanatory, and they’re expected in self-hosted websites and apps more than template ones. It’s also a fact that nobody can release a piece of software without them, so you need to keep an eye on where they are logged if you’re using someone else’s software as a template.

The places where the most errors are logged can indicate where users are having trouble with your media, but it can also indicate the number of users using that feature. This will help greatly when prioritizing which to begin patching. 

A major bug that only affects a few users might be considered less important than a minor one that affects everyone. It’s up to you to decide, but error logging information can help you there.

Users unsubscribed

I’m sure you’ve all seen the little boxes that pop up when you unsubscribe from a service or mailing list. The ones that ask you “why did you unsubscribe” and potentially “here’s an incentive to not do that.”

The amount of users unsubscribing from your mailing lists or from notifications are useful, as it can indicate when problems began occurring and assist you with tracking them down. However, if you’re given the reason that users unsubscribe it will assist you far, far more.

(De)activation of features

Ah, features. The shining light and also the bane of software’s existence.

When dealing with modular apps, users can activate and deactivate certain features as they please. With this information, you can determine what features users like and dislike, as well as how certain cohorts use your app.

Naturally, this will give you insight into what future features you might want to add, what current features you should accelerate the development of, and what you might want to drop.

Session durations

Session durations are another fine source of information. The amount of time customers spend on your website or app can give you insight into how your customers interact with it.

Are they mostly spending five minutes here and there? It’s likely that you’re being used to find specific items that they trust you for.

Are sessions longer? It’s likely that users are either exploring your features or simply that they use you regularly as a reliable seller. You can cross-reference with your sales statistics to check this. 

Unique user rates

The number of users that are connected to you at any one time is useful information. The amount of unique users is much more interesting.

Your software will log every individual account’s interactions with it, thus allowing you to track the number of unique users at any point in time. This will let you see interesting pieces of information such as:

• The average lifetime a customer will interact with you.
• Whether or not new users are joining at the same rates.

This tells you how users are interacting with you.

Are they connected with you for short periods of time and be done with it? It’s likely that you’re being used to find the odd item that they can’t find elsewhere.

Are you seeing consistent use by the same users over time? You’re likely a lynchpin of their daily routines.

What happens if my data conflicts?

You’re never going to get the complete picture from a single piece of data. That’s generally why we try to analyze several in order to get a more rounded view of things.

Unfortunately, when you have multiple pieces of data that you’re tracking, some of them will inevitably conflict. If that happens, don’t panic! There is usually a solution, though it might not be immediately obvious.

Check your methodology

The first thing you should do when you’re seeing conflicting data is re-check your methods. If there’s a flaw in one of them, that might explain why you’re getting mixed signals.

If you find no flaws, you must conclude that the data is sound. But how can that be? How can two completely opposite things be true at the same time?

Well, you’ve got to consider that no two pieces of data are collected the same way. There’s always going to be a bias depending on the method. There might be a factor at play here that accounts for the discrepancies.

Online surveys are more likely to be completed by younger customers, for instance. If one piece of information was gathered online, and the other offline, they may be coming from different cohorts. 

Examine your cohorts

The solution there is simply to split your data to account for the differences between them.

Next, if you find no distinct difference between the cohorts that gave you the data, look to when or where in the customer journey the data was collected. Let’s take a look at customer sentiment, which tells you how customers feel about your product.

Are you dealing with after-purchase? At the point of purchase? 

Customers are much less likely to post a review as time goes on after the purchase. Thus, any review that’s posted a long time after purchase is likely to be heavily weighted. It’s also an unfortunate fact that the longer the wait the more likely that review is to be negative.

Thus, data that says customers are both pleased with your product and have a burning hatred of it, can both be correct. 

Consider randomness

If this step reveals no differences, the only thing to do is simply chalk it up to the uniqueness of the data. 

Each piece of data you’ll ever collect is unique. While there may be overall trends it’s entirely possible to end up with completely contradicting pieces of information by sheer chance.

The key here is to remember that no analysis will ever account for everything that a customer thinks. There’s always the chance that you had a boom in the popularity of a certain feature one month, then had it drop away the next, for example.

In the case of directly conflicting data, remember that it will have been pulled from different individual customers, with thoughts and feelings of their own. Customers aren’t cogs in a machine, their thoughts aren’t predictable and sometimes you just have to shrug your shoulders and re-do your analysis.

Wrapping up

Conducting product performance analysis is key to understand what your customers are after, allowing you to cater to their needs.

It’s not a one-and-done thing, though. 

You’ll need to constantly assess the viability of your products in the ever-changing marketplace. The good news is that it gets much easier once you’ve got the hang of it and have the tools for the job.

Remember that product performance analysis is usually done after the product has launched. To better prepare yourself for these stressful and joyous occasions, it’s best to have a solid go-to-market strategy. Check out our complete guide for more.