Beyond the Ghosts: Unmasking the realities of AI and MMM

A reliance on the past 

ChatGPT, the well-known AI language model, leverages pre-existing data to train its models. By analyzing vast amounts of historical written information, the system learns patterns, relationships, and trends that enable it to generate valuable insights. The strength of this AI lies in its ability to utilize pre-existing knowledge to provide contextually appropriate responses and recommendations. 

Similarly, MMM relies on historic performance to provide statistical confidence in media measurement. By analyzing past data on media consumption, advertising campaigns, consumer behavior, and buying trends, MMM models can identify patterns and extract meaningful insights. This historical data forms the basis for optimizing media planning, understanding audience reach, and measuring advertising effectiveness. MMM's reliance on past performance allows businesses to make data-driven decisions and allocate resources more effectively. 

This reliance on the past brings to mind a book¹ in a fantastical London where magic is real. In the narrative, a police detective investigates a machine claiming to be the genuine first example of AI technology, but it turns out to be powered by a paranormal force, controlled by ghosts. This depiction it would seem – current AI technology not controlled by the paranormal – is limited by the knowledge it has been presented with and the experience it has been shown (much like MMM trained on past data). Are these akin to a machine controlled by "ghosts"? Are the ghosts of past writing, past planning, and trusted methodologies holding us back from being able to identify the true opportunities outside the boundaries of previous experience? An accurate response curve can’t be estimated without sufficient data points in the same way that ChatGPT can’t generate precise content without being fed just the right information. 

Difficulties with data 

Court cases originating on either coast of the US, in the New York and San Francisco legal systems, have made headlines due to the involvement of some well-known names including Sarah Silverman and George RR Martin. These cases raised against ChatGPT, center on copyright law and the usage of Shadow Libraries (online databases holding, as you might imagine, not quite legal copies of published work). Authors are challenging the AI tech, calling for firmer policing of how the models are trained and calling for their involvement, through the use of their words, to be properly acknowledged and compensated. 

Should we assume that these cases will be the tip of an ever-growing iceberg? They are already stretching further afield from the language models, into the visual arts where AI is being used to create artwork and facing similar challenges in copyright law. 

An additional avenue causing issues for AI tech sits at the other end of the process – not in the ingestion of data but in the results. There are times when an AI language model like ChatGPT may respond to questions with a false claim or a totally untrue quote, which are termed “hallucinations”. These untrue quotes are initially troublesome as they falsely educate the user, but secondly that untruth may pervade and spread like a sour rumor. There are legal cases here too, in which personalities are suing for defamation against AI companies. 

We see parallels with analysis work; if an MMM piece claims some less-than-true-results, for whatever reason - perhaps the stakeholder sponsoring the piece has influenced the write-up, perhaps an item in a results deck has been taken out of context, or perhaps the analyst has purely got it wrong! These false results, these rumors that may spread around the company, can be harmful and MMM results need to be written carefully, with this outcome in mind to avoid uncomfortable untruths spreading on the grapevine. 

Jumping back to historic data in the case of MMM, we do not regularly see newsworthy examples of legal calamities when sourcing datasets for analysis. It is true though that every MMM model build is beset with some form of data difficulty. A marketing analyst can regularly identify trouble obtaining clean, consistent, and repeatable media and marketing data to feed a model and the transfer and storage of this sensitive sales and financial data is of top priority for all credible MMM providers. Even though we seem to stay outside the courtroom, MMM is not a headache-free data game. 

Interestingly, the legality of holding and using personal data is starting to boost the regard for MMM as an alternative to MTA (digital attribution based on individual customer journeys). MMM datasets can be created with anonymous, homogenized data; this gives companies who are keen to avoid things like GDPR in Europe a route around data security barriers. 

A shared need for human delivery 

Earlier this year, an East London venue hosted an AI-powered rave, amusingly titled “Algorithm”. The rave was met with mixed reviews, some saying the music felt "dry and empty". 

It is this risk of hollow delivery that highlights the power of a human touch. A good DJ will sense when their ravers are feeling energized, or when they need things to be dropped down a notch, and hence mix their set to meet the atmosphere. Likewise, a good presenter can sense when a room perhaps needs more time to digest information or more context on which to trust the results on the current slide. A human presenter can help explain (or at least hypothesize about) 'why' the content is as it is, guide a confused audience by reiterating fundamental points, and describe the journey that has been taken to arrive at the material seen in the room. 

Both rave and MMM presentation can be greatly improved by the artistry of the demonstration, and more generally both AI creations and MMM results can hit the spot more successfully with humanized delivery. 

A reliance on human intervention  

We’re seeing that both MMM and ChatGPT need the human touch to be most useful. The example that stands out when we consider ChatGPT is the undeniable reliance on good prompts. Prompts are the way in which we set up a chat session with ChatGPT, guiding it to answer in a manner suitable for a certain audience, to a certain level of detail, or with a specific tone of voice for example. Good prompting – the equivalent of training a new inexperienced team member – is crucial to gain the most from using the tool and can be crafted by a skilled human operator. Good training too needs to be available for these human operators of ChatGPT which adds to the line of human contact needed to get the best results from this form of AI. 

When we consider MMM, the need for a hand to steer the content of a model can be vital, as well as the voice to interpret results. Imagine an output consisting simply of a regression equation; this would not suffice for the CMO in a large retail company and even output presented in a comprehensive fashion, but which holds insights on factors that do not make sense for the business in question, would be useless. In these situations, the results need to be honed to meet the needs of this CMO, answering specific business questions, illustrating actions they can take, and matching their business vernacular, for example.  

In both cases, human intervention is crucial to start us off in the right direction and bring us back to fulfilling the brief. Without it, AI and MMM can produce indecipherable or nonsensical outcomes. 

Let’s now move on from unpicking the shared dependencies, and challenges, of these tools and think instead about what they can learn from one another and from one another’s traits. Our initial point already laid the foundation: to overcome reliance on past experience, the approaches can both improve through adaptation. 

An example of adaptation resides in the crossover space between AI and MMM, where a much-developed area already exists. MMM model generation and optimization, using AI or machine learning in the form of genetic algorithms, make use of adaptation and improvement. Something that nature has shown us is key to improvement; survival of the fittest relies on mutations, mutations that may produce unfit individuals but can also produce individuals who excel and thus reproduce. Introducing mutation into machine learning gives us the ability to utilize genetic algorithms in model building and optimization and opens possibilities giving diagnostics in seconds that a human would take days if not months to discover. 

One might wonder – is this mutation mechanism, deftly wielded by nature and essential in the evolution of our species to reach our intellect levels, the missing skill of some AI tools?  

A human author can create something wholly new and an artist can devise a new way to see the world. Imagine the buzz felt when George Orwell released “Nineteen Eighty-Four”, or the awe when Italian Renaissance artists unveiled the use of perspective. In contrast, watching the AI-generated spectacle of Serena Williams play her younger self does give entertainment, but would sports fans be satisfied watching the stars of yesteryear play in a never-ending loop? Are movie fans going to be satisfied with watching AI-de-aged Robert de Niros and Harrison Fords populate the silver screen for decades to come? Do AI-generated songs, fabricating recording artists’ voices or mixing music in the place of a DJ, keep the musos’ ears contented? 

It’s possible that the worlds of literature, music, sports, and movies need new blood, new moves, and new faces to entertain and thrill. AI needs to adapt to create like humans, (or more realistically be used in unison with human creators), not just copy, otherwise, algorithms will be overtaken by fitter options. As we’ve touched on in earlier paragraphs, this behavior of copying can in itself cause issues for AI tech, as the right to copy is not always given. 

Being able to adapt also brings a step change in the value of an MMM model. A much-maligned feature of the MMM approach stems from its focus on the past too. In order to boost MMM’s power for future decision-making, it needs to adapt to be most relevant for what is coming, not what has happened. To this idea, creating models using some form of time-dependent coefficients can encourage results to be more relevant for recent and therefore upcoming activity. And adaption of media response curves designed specifically to give well-behaved optimization results, or even have a heightened or lowered contribution based on expectations of the future, can create superior planning tools. 

Both AI and MMM need to keep mutating and adapting; to introduce novel content, encapsulate changing trends, shifting consumer behaviors, and evolving market dynamics, and blend it all with real-time information and contextual understanding to ensure relevant, pleasing products. 

Lastly, let’s touch upon more tasks where the two tools can collide and produce something special. There are fewer examples of these tasks in the current market, but let’s hope we see many developments in the coming years. Think of the possibilities:  

• AI-generated descriptions of datasets, freeing analyst’s time and ensuring items aren’t missed. 
• AI-generated descriptions of MMM model results, open up the opportunities for less analytical teams to gain MMM benefits. (Although we must remember the limitations we’ve touched upon when thinking about the need for human interaction and intervention in our above paragraphs.) 
• AI-based natural language tools for ‘reading results’, aiding end-user interaction; allowing a client to gain a better feel for their results without the need for extra official, possibly costly, delivery from consultants. 
• AI usage to improve the ability to search for models with ‘good’ characteristics. Envisage a step on from the model searches seen in our own proprietary tool, to seek out the traits of models that best answer business questions, manage client expectations, and enable actionable results.  

But these ideas we must leave here. Let’s keep our eyes open for new tools in the marketplace that encompass both AI and MMM to offer practitioners like us some exciting benefits.  

In conclusion, the exploration of the shared characteristics between AI and MMM reveals an interplay between these two disciplines. By recognizing their reliance on past data and the barriers this can raise, the need for human delivery, and the significance of human intervention, we gain a deeper understanding of their intricacies. 

It is through adaptation and continuous improvement that both AI and MMM can truly thrive. As we envision a future where these tools evolve and collaborate, the possibilities become even more exciting.  

¹ "False Value", Rivers of London book 8, by Ben Aaronovitch