Open-source MMM vs strataQED

One of the most frequent questions we get asked today is how our technology compares with some of the new open-source MMM solutions offered by Facebook and Google.  

The right choice will, of course, vary company-by-company depending on the strategy and the composition of the team, but we recently started working with a global consumer electronics company that switched from Facebook’s Robyn to strataQED after an extended test of both solutions (alongside Google’s lightweight MMM) based on the same datasets. 

The decisive factor for them, once they were satisfied with testing the three offerings, was that strataQED could produce equivalent model output (with easy set-up and faster run-time) whilst offering out-of-the-box forecasting and optimization capabilities that are not straightforward to replicate using the alternatives. 

A great outcome for us; however, let’s evaluate both. 

Aren’t open-source platforms more flexible/customizable?  

Firstly, many of the open-source offerings are great initiatives which include some innovative techniques. Being open source, they offer the promise of flexibility via customizing the code and transparency via reading the existing code. Interestingly, what we’ve heard from clients (and from our own experience with these solutions) is that the UX is less developed, meaning it can take quite a lot of work to make them functional and there are lots of small practical problems that can make them quite hard to use and even to understand.  

We can see that open-source platforms are viewed as providing the ultimate flexibility and control, but ironically, in many business situations you won’t have the resources to make it as transparent as the leading proprietary supported platforms already are. Open-source code used in-house is a challenge. As it isn’t commercially supported, it presents a high risk of misuse in the hands of under-resourced users. To be fully operational and fit for purpose, it also requires an experienced data science resource, a MMM data analyst, as well as someone with marketing skills and a strong business understanding. The risk in open source is really about implementing via data science without specific MMM expertise. This might sound like a surprising risk, but MMM is a highly specialized expertise. The two common challenges are around selecting and preparing data inputs and interpretation of results. Open-source packages do save data science teams time, but they do not educate on the overall process. In fact, we spoke with a brand team that brought modeling in-house (writing their own code) and found that they actually lost transparency, control and understanding – vs. previous use of commercially supported modeling systems. The unexpected gap came from the fact that the commercial tools helped to structure the analysis and visualize results so that all relevant parties could participate in the measurement (and even modeling) process; whereas the internal code was purely meant to be efficient for data science, falling short on collaboration and transparency.   

Management at other firms should take this as a warning to think carefully about the costs and risks of the larger measurement process and not simply the “model fitting cost”.  Media measurement is fundamentally a cross functional process (marketing, finance, creative, agencies, etc.). Transparency in measurement is not about access to source code, it is about bringing the team along. The current reality is that open source is for programmers and the leading commercial MMM platforms are designed to support the entire team. 

But isn’t open-source MMM cheaper? 

At first glance it may seem so, but if you factor in the total cost of ownership, strataQED can be overall better for the company wallet.  

What we offer with strataQED (our MMM software) is a fully developed user interface both for modeling and optimization that is production ready, offers lots of control over how things work (no black-boxes) and is quick and easy to get started; not to mention, it comes with a full training and support package.  

Conclusion 

While both open-source and commercially supported packages are viable ways to build internal MMM capabilities, the leading commercial packages (including our strataQED platform) are often a more comprehensive, more transparent, lower risk and even come with a lower long-term cost of ownership vs. efforts based on open-source packages. 

Our strataQED platform provides an end-to-end support to prepare data, fit models, interpret and communicate results, build forecasts, and optimize budgets. If your business is looking for a comprehensive marketing measurement platform that will enable in-house capabilities while limiting the business risk of building from scratch, then strataQED is clearly the better fit.