.png)
I remember being told early in my career that marketing mix models are much like sausages – they need to look good from the outside and do the job, but you are better off not knowing too much about how they are made. The point being that some ‘manipulation’ might be required to make a model work as intended and generate the outputs needed (for example to meet expectations or align with previous results). Not to mention any bias or hidden agendas that might be involved.
This type of thinking contributed to the perception of MMM as something of a ‘black box’, which undermined trust and limited the decision-making influence that MMM could have within client organisations. Without a clear understanding of the methodology and assumptions, marketers may legitimately question the validity of the findings and feel uncertain about basing critical decisions on them.
Transparency is undoubtedly a much better approach that can alleviate these concerns, so what does transparency really mean in the context of MMM?
It is largely a question of process and bringing the team along – being able to explain what you did and why, and what impact it had. A transparent approach provides visibility into the model’s data sources, assumptions and calculations. It allows marketers to understand, validate and fine-tune the methodology, increasing trust, enabling customization and promoting collaboration.
Let’s break these points down in a little more detail.
Model-building process:- Transparency starts with clearly defining the steps involved in marketing mix modeling. This includes data collection, variable selection, model building, and validation.
- There are lots of decisions to make during a modeling project, so it’s important to be open about what data was used, what assumptions were made (both explicit and implicit) and the methodology used to test different model permutations and ultimately arrive at a ‘final’ model that forms the basis of reporting and recommendations.
- Transparent modeling requires clarity on how the various marketing inputs (such as media spend, promotions, pricing strategies, etc.) are incorporated into the model, and where that data comes from.
- If there are issues with the data these should be discussed openly, and stakeholders need to be aware of any transformations that are applied to the raw data before it is used for modeling.
- As the statistician George Box famously said: “All models are wrong, some are useful.
- Every model has limitations so modellers shouldn’t feel the need to pretend their model is the single ‘right’ one. By all means justify the model by describing the thorough process and approach that’s been taken, but openly discussing the strengths and weaknesses and taking on feedback is a better way to build trust and potentially improve the models.
- A transparent model should be interpretable so stakeholders can understand how each marketing input contributes to outcomes (e.g., sales, revenue).
- Visualizations, coefficients, and sensitivity analyses aid in interpretation, and specialised modeling software can bring this understanding of cause and effect to life.
- Open-source modeling solutions might seem to offer the ultimate in transparency, but we have seen clients actually lose transparency, control & understanding when adopting code that was purely meant to be efficient for data science, rather than commercial tools that helped to structure the analysis and visualize results so that all relevant parties could participate in the modeling process.
- Transparent modeling includes comprehensive reporting. Ideally this goes beyond the top-line results presentation and includes detail on the model performance, underlying calculations and uncertainty around the parameter estimates.
- Many vendors won’t provide outputs such as model equations and response curves to their clients due to concerns about sharing IP, but opening these things to scrutiny further empowers those making decisions on the back of the analysis.
- Thorough documentation encourages accountability and ensures that the process can be replicated in the future.
- A transparent process means not just delivering a report at the end of the project but involving stakeholders throughout.
- This means consulting them at all stages, involving them in the assumptions that are made, and understanding the implications of making different assumptions on the end results.
Ultimately when stakeholders can see and understand the modeling process, they are more likely to trust the results and the decisions based on those results. Not everyone wants or needs to see all the details, but transparency is a better default than a black box and is always a reassuring option to have.
If you’re paying an external vendor to build your marketing mix model and don’t know what’s really going on in the process, maybe it’s time to pay a visit to the sausage factory!
At ScanmarQED transparency is at the heart of our MMM proposition – our specialized modeling software is 100% transparent and we believe in a collaborative approach to all analysis - please contact us if you’d like to find out more.
