When updating a model one tends to compare the effectiveness of any new activities / campaigns against those that were previously captured by the model. If there are any changes, understanding the reason for this is key. The metric that people often use to compare effectiveness is the Return on Investment (ROI). Depending on whether you’re interested in the revenue or profit returns, the ROI is calculated as:
So, if your ROI is different, is it due to changes in the uplift from the activity, the cost of running it, or both? There are several factors that we should consider, let’s take a more detailed look at some of the main ones.
When you’re investigating the various elements, ensure that you’re making a fair comparison when doing this. Does the ROI for the earlier campaigns include any carryover effects? If so, you should ensure that the carryover, or at least an estimate of it, has been included for the new campaign too.
Financial effect: Has the revenue and/or profit per unit of your KPI changed since the previous campaign? The same volume uplift might have different financial impacts over time. To isolate the impact of this, you can try calculating the ROI using the same revenue/profit per unit for both campaigns to see what difference this makes.
Media costs: How has the cost of media varied between campaigns? By this we mean the cost per unit of media e.g. cost per impression, not the total spend (we’ll look at this later). Changes in cost per unit can be due to general media inflation/deflation or seasonal fluctuations in costs, as well as changes in the size and position/location of the adverts. You can get more specific details on what has changed from your media agency. As we did with changes in the financial data, you can also isolate the impact of changes in media costs by assuming the same cost per unit for each campaign.
Effectiveness: Check the volume uplift per unit of activity (e.g. GRP, impression). This ratio strips out the impact of changes in media costs and the financials for your product. If the effectiveness has changed, investigate if this has been caused by changes in:
Budget invested: Have you moved further up or down the response curve (i.e. further into / out of diminishing returns)? Look at where the points on the curve are for the weights used in the latest campaign versus the previous campaigns.
Flighting: Some flighting/laydown patterns are more efficient than others, i.e. are less likely to be into diminishing returns. You can run a scenario to test the difference caused by changes in the flighting.
Carryover or adstocks: The impact from previous periods’ activity will also influence the position on the response curve. The rate of carryover and size of gap before the new campaign might push you further along the response curve and into diminishing returns.
As you can see, there are many elements that can cause the ROI for an activity to change. A higher ROI isn’t always due to a better performing campaign.