Supply Shaping and the Quest to “See” the Entire Internet

Magnetic, my employer, is an artificial intelligence company that operates a variety of AI-driven advertising solutions, including a fully-automated, AI-powered DSP. The machine decides who to target and how much to pay for an ad. This decision is based on Magnetic’s rich consumer profiles, and how well each profile “correlates” with a typical converter for one of our live campaigns.

Our machine learning methods have been performing exceptionally well since Magnetic embraced AI years ago. We brought more decisioning under our AI’s control over time, and significantly scaled back the manual parts of media buying.

The next step in Magnetic’s journey to full AI-automation is well underway. AI now pilots our exchange partners to control our media supply.

Ad impressions: too much of a good thing

Managing exchanges was still a highly manual task in late 2017 and early 2018. We had to hold monthly internal meetings to review QPS (queries per second) allocated to exchanges, blacklists, brand safety levers, potential publishers deals, and overall performance metrics.

Managing exchange-specific lists of preferred publishers became especially time consuming. The number of ad opportunities Magnetic received doubled while overall value stayed the same, thanks to header bidding and exchanges reselling impressions between each other. We applied short term fixes to optimize our path to supply, but they could not be our long term solutions.

Please, only send consumers we care about

Last October, we committed to largely automating supply management by 2019.

We reached out to our exchange partners to see if they could “pre-filter” ad opportunities based on rules generated by our AI. Enough of our existing exchange partners had such whitelisting capabilities to reach critical mass. None, however, had done pre-filtering at such scale.

Our first step was to identify all consumers our AI may target. We maintain about 350 million consumer profiles for North America. Each exhibits human (i.e. repeated and consistent) behavioral patterns over the last 330 days – such as searching multiple times for travel related terms across one of 350K+ partner sites.

We trimmed down the user whitelist in 2 ways. First, we used a probabilistic model to exclude low value profiles unlikely to be considered by at least one of our running campaigns. Low value users account for less than 5% of predicted conversions, but represent the majority of traffic.

Second, we customized the whitelist for each exchange by removing unmatched users (those who do not have a linked cookie with the exchange).

Then, we extended each whitelist to include device IDs belonging to any of these target consumers, as per our cross device graph. Because in-app conversion measurement is unreliable, we only target mobile app users if they are predicted to convert on a website.


The final lists of 500MM+ cookies and device IDs was synced with each exchange. Time is of the essence here, as we update our profiles continuously and the value of any newly added consumer can tail off in a matter of minutes.

The tweaks that matter

More tweaks were needed to ensure maximum performance.

First, we still needed to receive some unbiased traffic to continuously A/B test the performance of our whitelisted traffic.

Second, we had to whitelist placements that generated a high level of clicks. We run a fair amount of click-optimized campaigns, and we did not want to miss out on some high-CTR domains even when the consumer is unknown to us.

For a similar reason, we chose not to filter video traffic. Most video campaigns are optimized toward completed view, and the context of the ad often matters more than the consumer seeing the video.

Spiking win rate

As soon as an exchange enables supply shaping, we saw a significant hike in gross win rate (the win rate based on all received ad opportunities). Some of these gains eroded as we launched the tweaks described above to preserve clicks. But in general, gains have been holding pretty well.

Such gains translated immediately to cost saving, since processing ad opportunities is the biggest infrastructure cost.

But more importantly, for a company like Magnetic, Supply Shaping™ enabled us to remove the QPS cap entirely and address the entire internet. For us, this is the guarantee of delivering maximum possible conversion rate to our clients.

Exchanges must embrace DSP’s supply path optimization

We are slowly rolling out Supply Shaping™ and consumer whitelisting to all of our exchange partners that can support it. In 12 months, Magnetic will no longer do direct business with any exchange that does not have adequate Supply Shaping capabilities.

Magnetic may be a pioneer, but I expect most media buys to be AI-driven in a few short years. Exchanges will need to beef up their capabilities to enable DSPs to optimize their supply path.

DMPs Must Evolve to Stay Relevant in an AI World

Originally posted on blog

Brett House, VP of Global Product Marketing at Nielsen Marketing Cloud, recently penned an article in ExchangeWire that breaks down the extreme hype around marketing AI. He states that many AI solutions only provide “batch” or offline learning (instead of real-time), and a lack of true transparency is clouding the potential of AI in marketing and advertising.

I can say the same about DMPs.

AI Decisioning is the Future of Ad Buying

I work for a company that’s all about using AI to maximize ad performance. Every minute, we collect millions of signals from all over the web (mostly search and browsing behavior data) and update our user profiles in real-time. Our AI works amazingly on our in-house DSP to deliver CPA’s at a fraction of what even the most skilled AdOps person can do manually. After all, there are just too many variables for a human being to beat a computer at that game.

With all our DSP has to offer, many agency clients prefer to use their own DSP. Learning a new tool can be a laborious task for some, or they may have deep integrations between their DSP of choice and their backend systems.

Regardless of why, last year we set out to launch Magnetic Live Audiences (“MLA”) for that reason. Our AI creates custom, real-time audiences that MLA packages as third-party segments to be activated on the DSP the advertiser is using.

That’s where a DMP often comes into play. DMPs have been used traditionally to aggregate, manage and deliver audiences from data producer to data consumer. For example, from an advertiser’s CRM system to demand-side platform (DSP).

DMPs Are Stuck in the Past

You would think that, in 2018, using a Data Management Platform (DMP) to deliver a multitude of custom audiences in real-time to any buying platform is table stakes. Haven’t DMPs touted “just in time” audience management for ages?

Not so fast.

In recent months, I’ve been feuding with leading DMPs to distribute our audience quickly and efficiently. Most established DMPs have difficulties handling the volume of data needed to sync audiences that are custom-built for each campaign and updated continuously.

At the same time, consumer behavior is notoriously fickle. Think about somebody adding an expensive phone to a shopping cart on one of our many eCommerce partners. Suddenly the value of this user skyrockets. A shopping cart action is one of the most valuable signals that a user is about to convert. Unfortunately, this “value boost” will last a few minutes, hours at most.

Now, picture us struggling to communicate to DMPs that our shopper’s cookie or device ID is suddenly worth 100x more for a Samsung campaign than it was just one minute before. DMPs will take hours to ingest our updated segment, more hours to process and approve the change, and a few more hours to match that user and re-export to our target DSP.

To make matters worse, the DSP has its own ingestion and processing lag. Some established DMPs even require manual operation on their UI to complete the update. When things go wrong, the DMP and DSP point fingers at each other while we are left scratching our heads. In most cases, by the time our phone shopper ID is ready for activation on the DSP where our Samsung campaign is running, the phone has been ordered, delivered, and activated.

A Superfluous Layer Between Data Seller and Data Buyers

Most of DMPs’ business is still to distribute “standard” catalogs of pre-packaged segments that are static for weeks (or even months). These are the same segments that nobody on the buying side trusts. Because of this legacy, DMPs tend to sit between the data seller (us in this case) and the data buyer (the DSP). This additional “hop” creates a refresh latency and decreases the all-important match rate (i.e. how many of our user cookies can be mapped to the cookie seen by the DSP).

However, in the new AI-driven world of custom segments and real-time online learning, this simply won’t do. It should come as no surprise that we are seeing much better performance with direct integration with DSPs than when distributing our audiences through a DMP.

But don’t DMPs offer more bells and whistles? It’s true that DMPs come with a better UI, easier data onboarding, and fancy analytics and attribution models. But performance is what actually matters. That’s why AI-driven data sellers like Magnetic will gravitate toward direct integrations with DSPs.

Leading DSPs Will Build Powerful Capabilities to Ingest Real-time Audiences

For all these reasons – in the end, independent DMPs will either be absorbed by established DSPs or major marketing clouds, or they will die. So where does that leave us?

  1. Refreshing audience segments (in minutes or less) will soon become the norm, and DSPs will get us there. AI-driven custom audiences are indeed the future of data selling. Most of the leading DSPs understand that. DMPs gave us quick reach, but top-notch performance will require close integration with the buying platform. DSPs and audience sellers alike will continue to innovate and quickly bring new capabilities to the forefront to maximize performance from AI-driven custom audiences. Stalled, static behavioral segments will soon be a thing of the past.
  2. Audience sellers will gain necessary control over the bid price. This can be tricky as media buyers need the flexibility to adjust price according to overall campaign objectives and performance to date. The Trade Desk “base bid multiplier” feature offers a smart trade off here. While the trader can set a base bid, the audience seller can set a multiplier for each user to adjust the bid price according to the value of the user. Some platforms also offer multiple pricing options. For example, going beyond fixed CPM fees and offering percent-of-media or CPA.