Supply Shaping and the Quest to “See” the Entire Internet

Magnetic, my employer, is an artificial intelligence company that operates a variety of AI-driven advertising solutions, including a fully-automated, AI-powered DSP. The machine decides who to target and how much to pay for an ad. This decision is based on Magnetic’s rich consumer profiles, and how well each profile “correlates” with a typical converter for one of our live campaigns.

Our machine learning methods have been performing exceptionally well since Magnetic embraced AI years ago. We brought more decisioning under our AI’s control over time, and significantly scaled back the manual parts of media buying.

The next step in Magnetic’s journey to full AI-automation is well underway. AI now pilots our exchange partners to control our media supply.

Ad impressions: too much of a good thing

Managing exchanges was still a highly manual task in late 2017 and early 2018. We had to hold monthly internal meetings to review QPS (queries per second) allocated to exchanges, blacklists, brand safety levers, potential publishers deals, and overall performance metrics.

Managing exchange-specific lists of preferred publishers became especially time consuming. The number of ad opportunities Magnetic received doubled while overall value stayed the same, thanks to header bidding and exchanges reselling impressions between each other. We applied short term fixes to optimize our path to supply, but they could not be our long term solutions.

Please, only send consumers we care about

Last October, we committed to largely automating supply management by 2019.

We reached out to our exchange partners to see if they could “pre-filter” ad opportunities based on rules generated by our AI. Enough of our existing exchange partners had such whitelisting capabilities to reach critical mass. None, however, had done pre-filtering at such scale.

Our first step was to identify all consumers our AI may target. We maintain about 350 million consumer profiles for North America. Each exhibits human (i.e. repeated and consistent) behavioral patterns over the last 330 days – such as searching multiple times for travel related terms across one of 350K+ partner sites.

We trimmed down the user whitelist in 2 ways. First, we used a probabilistic model to exclude low value profiles unlikely to be considered by at least one of our running campaigns. Low value users account for less than 5% of predicted conversions, but represent the majority of traffic.

Second, we customized the whitelist for each exchange by removing unmatched users (those who do not have a linked cookie with the exchange).

Then, we extended each whitelist to include device IDs belonging to any of these target consumers, as per our cross device graph. Because in-app conversion measurement is unreliable, we only target mobile app users if they are predicted to convert on a website.

 

The final lists of 500MM+ cookies and device IDs was synced with each exchange. Time is of the essence here, as we update our profiles continuously and the value of any newly added consumer can tail off in a matter of minutes.

The tweaks that matter

More tweaks were needed to ensure maximum performance.

First, we still needed to receive some unbiased traffic to continuously A/B test the performance of our whitelisted traffic.

Second, we had to whitelist placements that generated a high level of clicks. We run a fair amount of click-optimized campaigns, and we did not want to miss out on some high-CTR domains even when the consumer is unknown to us.

For a similar reason, we chose not to filter video traffic. Most video campaigns are optimized toward completed view, and the context of the ad often matters more than the consumer seeing the video.

Spiking win rate

As soon as an exchange enables supply shaping, we saw a significant hike in gross win rate (the win rate based on all received ad opportunities). Some of these gains eroded as we launched the tweaks described above to preserve clicks. But in general, gains have been holding pretty well.

Such gains translated immediately to cost saving, since processing ad opportunities is the biggest infrastructure cost.

But more importantly, for a company like Magnetic, Supply Shaping™ enabled us to remove the QPS cap entirely and address the entire internet. For us, this is the guarantee of delivering maximum possible conversion rate to our clients.

Exchanges must embrace DSP’s supply path optimization

We are slowly rolling out Supply Shaping™ and consumer whitelisting to all of our exchange partners that can support it. In 12 months, Magnetic will no longer do direct business with any exchange that does not have adequate Supply Shaping capabilities.

Magnetic may be a pioneer, but I expect most media buys to be AI-driven in a few short years. Exchanges will need to beef up their capabilities to enable DSPs to optimize their supply path.

nToggle: A Hot Product but No Customer

Yesterday’s acquisition of nToggle by Rubicon was a big surprise.

nToggle offers technology in big need right now. All programmatic platforms saw massive spike in impression volume, resulting from the combined effect of Header Bidding and Reselling.

In order to protect their infrastructure, DSPs and SSPs need to eliminate bid requests that have a low probability of converting, including duplicated impressions. This filtering mechanism is often referred to as “intelligent throttling” or “supply path optimization” (SPO).

A few startups lined up to solve this industry-wide problem. nToggle became the best known of them.

You would think nToggle had customers lining up to license their technology. So why selling?

SPO is too important to outsource

From what I heard, most platforms are developing SPO in-house. SPO is mission critical for platforms, who must offer scale without spending a fortune on hardware.

There is also strategic value in SPO.

Most SSPs want to do reselling. Reselling AdX inventory, for example, means double bid requests even for the largest platforms. Fill rate on resold inventory is tiny, so SPO is a must.

For DSP, the strategic impetus is more about scale for highly targeted campaigns, and in effective campaign optimization.

SPO must identify and eliminate intermediaries who take the higher margins. In particular, DSPs will start integrating directly to the header bidder or ad servers of the largest publishers, and they will need to de-dupe impressions they can buy directly.

Rubicon Was Getting Desperate

Officially, Rubicon is buying nToggle to offer clean, de-duped inventory to their DSP clients as a differentiation move. I don’t buy that one minute.

Rubicon was simply slow at developing SPO in-house, reminiscent of how they missed the boat on header bidding in 2015.

As Rubicon realized their competitors were filling up their pockets with reselling impressions, they needed a jump start on SPO. Buying nToggle gave them the technology and manpower to catch up with the widespread practice.

A $38MM mistake.