The Future of Media Measurement: The Role of Panels in Big Data

As audiences increasingly migrate to CTV, the old norms of TV measurement no longer apply. A new approach is required - one that is best served through a combination of big data and calibration panels. While smaller in size than big data, the importance of calibration panels’ role in ensuring the accuracy of future measurement solutions should not be underestimated. So what role will the panel play? This is a question we are asked often, afterall, we are one of the industry’s only person-level panels. I was happy to get to the heart of this question at the Summer TV Data and Measurement Summit sponsored by NextTV where I discussed the future of the panel with industry gurus Nielsen’s Mainak Mazumdar, Beth Rockwood of WarnerMedia, Peter Sedlarcik from Havas Media Group, and Howard Shimmel of Janus (former CRO of Turner). We all agreed panels are pivotal to the future of measurement.

As the WFA outlines in their Cross-Media Measurement Framework, effectively measuring media across digital, TV and CTV is a complex problem that will take multiple data sets to solve. Both census-level big data and panels will be part of the solution working together to provide the most comprehensive measurement of audience behavior and engagement with all types of media, whether digital or TV.  

In this new model, big data is the foundation and panels are just one ingredient. The simple truth is that big data works better if it is calibrated by panels. Calibration panels can, and should be used to fine tune census-level data to remove biases, fill in gaps, and ensure accuracy by acting as a common source of truth. 

What Makes a Calibration Panel Effective for Unified Media Measurement

At TVision, we’ve already begun supporting several big data providers who are looking to develop next-gen measurement solutions. We’ve found that in order for panels to work effectively as calibration data sets, as our does, they must meet these criteria: 

• Person-level data - At TVision, we have a saying, “Households don’t buy products, people do.” Any effective measurement solution must accurately report the marketer’s ability to reach specific decision-makers, and not a generic household. Person-level is a critical panel requirement because, unfortunately, a significant percentage of big data sets still rely on household-level information. Person-level data from panels enable big data sets to normalize their reported viewer behavior based on who is actually watching.  For example, household-level data might show ESPN to have an equal number of male and female viewers - assuming an equal number of males and females live in the household. Person-level insights from a panel can help calibrate the big data set to learn what percentage of ESPN viewers are typically male or female. 
• Second-by-second analysis - With video commercial air times commonly less than 30 seconds, and increasingly just six seconds, the industry needs insight into second-by-second viewer behavior in order to effectively measure video ads. If data is only available on a minute-by-minute basis, measurement companies may assume a viewer has seen an ad when in fact they have tuned away. Second-by-second data is critical for accurately reporting metrics for ad delivery and engagement.  
• Single-source across devices - Single-source panels measure the behavior of the same individuals across devices, regardless of if they are viewing linear or CTV. They enable apples-to-apples analysis and provide a comprehensive view of how people are consuming media. The WFA framework sums it up nicely: a single-source panel “acts as the arbiter of truth, providing benchmarks for the use and overlap of media consumption across channels and screens.”
• Passive measurement - Measuring passive panelist behavior enables continuous, accurate understanding of viewer behavior. There is little opportunity for human error or biases to impact the data. For instance, passive measurement helps provide correct co-viewing metrics since it doesn’t rely on the panelists to self-identify that they are in the room. Passive measurement can also be trusted to provide more accurate person-level identification of guests, children, and other non-heads of household. 
• Transparency - Panel data shouldn’t be a black box. If assumptions or adjustments have been made within the panel data in order to provide statistical significance, it needs to be clear what those are so they can be factored into the combined model. Upfront transparency is key for removing any bias that could be exaggerated as it gets scaled up within the larger data set. 
• Flexibility - The future of measurement is complex. We need the entire industry to work together to solve our common problems. While some measurement companies may be looking for an out-of-the-box solution, innovation will be more likely if companies can manipulate the data on their own. For providers who want to focus on the unique value their solution provides, a one-size-fits-all panel data set will be too limiting. If panels provide access to sessionized data, the data scientists at these measurement companies can more efficiently work with the panel data to get what they need for their own models.
  
  

A Future with Multiple Currencies for Media Measurement

This is an exciting time for our industry. We all know we’re in desperate need of new currencies, ones that place value on the effectiveness of ads. While there is certainly a continued place for media ratings, I propose that we work toward a future that supports multiple currencies. After all, different marketers have different goals for their campaigns. Brand marketers care about attention, whereas direct marketers are more focused on immediate conversions. To align success, these varying goals need to be supported with currencies that are built into the campaign from media planning to measurement. 

As TV embraces big data, every company should have as much runway as possible to contribute to our new path forward. Measurement providers shouldn’t be boxed into one panel provider with one vision of measurement - there is room for multiple panels as we redefine how we measure and value media. Let’s not forget, a multi-panel ecosystem is good not only for innovation, but also as a check and balance to ensure the wrong trends don’t go unnoticed, or worse, expanded upon as big data is modeled up. 

There is plenty of room for innovation and improvement. If our industry can unlock the power of big data, we can solve numerous problems. We don’t need to have all our eggs in one basket, and frankly, the cost of panel data shouldn’t eat up all the research budgets that could otherwise drive innovation. Our industry’s best future will unfold when measurement providers have flexibility in their approach as they work to harness the power of big data and the accuracy of panels together.