Supply Shaping and the Quest to “See” the Entire Internet

Magnetic, my employer, is an artificial intelligence company that operates a variety of AI-driven advertising solutions, including a fully-automated, AI-powered DSP. The machine decides who to target and how much to pay for an ad. This decision is based on Magnetic’s rich consumer profiles, and how well each profile “correlates” with a typical converter for one of our live campaigns.

Our machine learning methods have been performing exceptionally well since Magnetic embraced AI years ago. We brought more decisioning under our AI’s control over time, and significantly scaled back the manual parts of media buying.

The next step in Magnetic’s journey to full AI-automation is well underway. AI now pilots our exchange partners to control our media supply.

Ad impressions: too much of a good thing

Managing exchanges was still a highly manual task in late 2017 and early 2018. We had to hold monthly internal meetings to review QPS (queries per second) allocated to exchanges, blacklists, brand safety levers, potential publishers deals, and overall performance metrics.

Managing exchange-specific lists of preferred publishers became especially time consuming. The number of ad opportunities Magnetic received doubled while overall value stayed the same, thanks to header bidding and exchanges reselling impressions between each other. We applied short term fixes to optimize our path to supply, but they could not be our long term solutions.

Please, only send consumers we care about

Last October, we committed to largely automating supply management by 2019.

We reached out to our exchange partners to see if they could “pre-filter” ad opportunities based on rules generated by our AI. Enough of our existing exchange partners had such whitelisting capabilities to reach critical mass. None, however, had done pre-filtering at such scale.

Our first step was to identify all consumers our AI may target. We maintain about 350 million consumer profiles for North America. Each exhibits human (i.e. repeated and consistent) behavioral patterns over the last 330 days – such as searching multiple times for travel related terms across one of 350K+ partner sites.

We trimmed down the user whitelist in 2 ways. First, we used a probabilistic model to exclude low value profiles unlikely to be considered by at least one of our running campaigns. Low value users account for less than 5% of predicted conversions, but represent the majority of traffic.

Second, we customized the whitelist for each exchange by removing unmatched users (those who do not have a linked cookie with the exchange).

Then, we extended each whitelist to include device IDs belonging to any of these target consumers, as per our cross device graph. Because in-app conversion measurement is unreliable, we only target mobile app users if they are predicted to convert on a website.


The final lists of 500MM+ cookies and device IDs was synced with each exchange. Time is of the essence here, as we update our profiles continuously and the value of any newly added consumer can tail off in a matter of minutes.

The tweaks that matter

More tweaks were needed to ensure maximum performance.

First, we still needed to receive some unbiased traffic to continuously A/B test the performance of our whitelisted traffic.

Second, we had to whitelist placements that generated a high level of clicks. We run a fair amount of click-optimized campaigns, and we did not want to miss out on some high-CTR domains even when the consumer is unknown to us.

For a similar reason, we chose not to filter video traffic. Most video campaigns are optimized toward completed view, and the context of the ad often matters more than the consumer seeing the video.

Spiking win rate

As soon as an exchange enables supply shaping, we saw a significant hike in gross win rate (the win rate based on all received ad opportunities). Some of these gains eroded as we launched the tweaks described above to preserve clicks. But in general, gains have been holding pretty well.

Such gains translated immediately to cost saving, since processing ad opportunities is the biggest infrastructure cost.

But more importantly, for a company like Magnetic, Supply Shaping™ enabled us to remove the QPS cap entirely and address the entire internet. For us, this is the guarantee of delivering maximum possible conversion rate to our clients.

Exchanges must embrace DSP’s supply path optimization

We are slowly rolling out Supply Shaping™ and consumer whitelisting to all of our exchange partners that can support it. In 12 months, Magnetic will no longer do direct business with any exchange that does not have adequate Supply Shaping capabilities.

Magnetic may be a pioneer, but I expect most media buys to be AI-driven in a few short years. Exchanges will need to beef up their capabilities to enable DSPs to optimize their supply path.

DMPs Must Evolve to Stay Relevant in an AI World

Originally posted on blog

Brett House, VP of Global Product Marketing at Nielsen Marketing Cloud, recently penned an article in ExchangeWire that breaks down the extreme hype around marketing AI. He states that many AI solutions only provide “batch” or offline learning (instead of real-time), and a lack of true transparency is clouding the potential of AI in marketing and advertising.

I can say the same about DMPs.

AI Decisioning is the Future of Ad Buying

I work for a company that’s all about using AI to maximize ad performance. Every minute, we collect millions of signals from all over the web (mostly search and browsing behavior data) and update our user profiles in real-time. Our AI works amazingly on our in-house DSP to deliver CPA’s at a fraction of what even the most skilled AdOps person can do manually. After all, there are just too many variables for a human being to beat a computer at that game.

With all our DSP has to offer, many agency clients prefer to use their own DSP. Learning a new tool can be a laborious task for some, or they may have deep integrations between their DSP of choice and their backend systems.

Regardless of why, last year we set out to launch Magnetic Live Audiences (“MLA”) for that reason. Our AI creates custom, real-time audiences that MLA packages as third-party segments to be activated on the DSP the advertiser is using.

That’s where a DMP often comes into play. DMPs have been used traditionally to aggregate, manage and deliver audiences from data producer to data consumer. For example, from an advertiser’s CRM system to demand-side platform (DSP).

DMPs Are Stuck in the Past

You would think that, in 2018, using a Data Management Platform (DMP) to deliver a multitude of custom audiences in real-time to any buying platform is table stakes. Haven’t DMPs touted “just in time” audience management for ages?

Not so fast.

In recent months, I’ve been feuding with leading DMPs to distribute our audience quickly and efficiently. Most established DMPs have difficulties handling the volume of data needed to sync audiences that are custom-built for each campaign and updated continuously.

At the same time, consumer behavior is notoriously fickle. Think about somebody adding an expensive phone to a shopping cart on one of our many eCommerce partners. Suddenly the value of this user skyrockets. A shopping cart action is one of the most valuable signals that a user is about to convert. Unfortunately, this “value boost” will last a few minutes, hours at most.

Now, picture us struggling to communicate to DMPs that our shopper’s cookie or device ID is suddenly worth 100x more for a Samsung campaign than it was just one minute before. DMPs will take hours to ingest our updated segment, more hours to process and approve the change, and a few more hours to match that user and re-export to our target DSP.

To make matters worse, the DSP has its own ingestion and processing lag. Some established DMPs even require manual operation on their UI to complete the update. When things go wrong, the DMP and DSP point fingers at each other while we are left scratching our heads. In most cases, by the time our phone shopper ID is ready for activation on the DSP where our Samsung campaign is running, the phone has been ordered, delivered, and activated.

A Superfluous Layer Between Data Seller and Data Buyers

Most of DMPs’ business is still to distribute “standard” catalogs of pre-packaged segments that are static for weeks (or even months). These are the same segments that nobody on the buying side trusts. Because of this legacy, DMPs tend to sit between the data seller (us in this case) and the data buyer (the DSP). This additional “hop” creates a refresh latency and decreases the all-important match rate (i.e. how many of our user cookies can be mapped to the cookie seen by the DSP).

However, in the new AI-driven world of custom segments and real-time online learning, this simply won’t do. It should come as no surprise that we are seeing much better performance with direct integration with DSPs than when distributing our audiences through a DMP.

But don’t DMPs offer more bells and whistles? It’s true that DMPs come with a better UI, easier data onboarding, and fancy analytics and attribution models. But performance is what actually matters. That’s why AI-driven data sellers like Magnetic will gravitate toward direct integrations with DSPs.

Leading DSPs Will Build Powerful Capabilities to Ingest Real-time Audiences

For all these reasons – in the end, independent DMPs will either be absorbed by established DSPs or major marketing clouds, or they will die. So where does that leave us?

  1. Refreshing audience segments (in minutes or less) will soon become the norm, and DSPs will get us there. AI-driven custom audiences are indeed the future of data selling. Most of the leading DSPs understand that. DMPs gave us quick reach, but top-notch performance will require close integration with the buying platform. DSPs and audience sellers alike will continue to innovate and quickly bring new capabilities to the forefront to maximize performance from AI-driven custom audiences. Stalled, static behavioral segments will soon be a thing of the past.
  2. Audience sellers will gain necessary control over the bid price. This can be tricky as media buyers need the flexibility to adjust price according to overall campaign objectives and performance to date. The Trade Desk “base bid multiplier” feature offers a smart trade off here. While the trader can set a base bid, the audience seller can set a multiplier for each user to adjust the bid price according to the value of the user. Some platforms also offer multiple pricing options. For example, going beyond fixed CPM fees and offering percent-of-media or CPA.

nToggle: A Hot Product but No Customer

Yesterday’s acquisition of nToggle by Rubicon was a big surprise.

nToggle offers technology in big need right now. All programmatic platforms saw massive spike in impression volume, resulting from the combined effect of Header Bidding and Reselling.

In order to protect their infrastructure, DSPs and SSPs need to eliminate bid requests that have a low probability of converting, including duplicated impressions. This filtering mechanism is often referred to as “intelligent throttling” or “supply path optimization” (SPO).

A few startups lined up to solve this industry-wide problem. nToggle became the best known of them.

You would think nToggle had customers lining up to license their technology. So why selling?

SPO is too important to outsource

From what I heard, most platforms are developing SPO in-house. SPO is mission critical for platforms, who must offer scale without spending a fortune on hardware.

There is also strategic value in SPO.

Most SSPs want to do reselling. Reselling AdX inventory, for example, means double bid requests even for the largest platforms. Fill rate on resold inventory is tiny, so SPO is a must.

For DSP, the strategic impetus is more about scale for highly targeted campaigns, and in effective campaign optimization.

SPO must identify and eliminate intermediaries who take the higher margins. In particular, DSPs will start integrating directly to the header bidder or ad servers of the largest publishers, and they will need to de-dupe impressions they can buy directly.

Rubicon Was Getting Desperate

Officially, Rubicon is buying nToggle to offer clean, de-duped inventory to their DSP clients as a differentiation move. I don’t buy that one minute.

Rubicon was simply slow at developing SPO in-house, reminiscent of how they missed the boat on header bidding in 2015.

As Rubicon realized their competitors were filling up their pockets with reselling impressions, they needed a jump start on SPO. Buying nToggle gave them the technology and manpower to catch up with the widespread practice.

A $38MM mistake.

Art-driven or Data-driven Creatives?

“Should ad creatives be driven by art or by data?”, asked Rob Rasko to a panel he was moderating yesterday on advertising, before calling for a show of hands.

Both panel and audience were split. Half thought UX and visual appeal should be top of mind for creative agencies, the other half believed data-driven optimization shall guide ad content creation.

“Scale is an addictive drug” trumped Warren Zenna from Havas, “and we’ve gotten to a point where monetization is getting in the way of good user experiences.”


Better high performing than “artsy”

Programmatic is all data driven. Marketers are getting more comfortable with targeting directly specific audiences, and focus less on which publications their ads show into.

More importantly, offering non-standard creatives does not scale. These “high impact” creatives look awesome on paper, but brands just don’t have the assiduity to build non-standard creatives. Too many operational issues.

IAB did invest in standardizing native creatives over the last year, but adoption has been slow and there is little appetite for exotic format standardization any time soon.

“Is anybody involved in programmatic advertising?” I asked around, perplexed.

Nobody was but me.

AppNexus/Index Partnership: What’s in it for Index?

AppNexus and Index announced a broad partnership on server-side header bidding at the IAB leadership meeting last week. On stage, Brian O’Kelley also mentioned full interoperability between Index, AppNexus and PubMatic header bidding wrappers.

Such a partnership, if executed, would ultimately doom Index.

Google and Facebook in Focus

AppNexus seems to be the main benefactor here, particularly AOS, the company’s Ad Server.

Google benefited greatly from its ad server monopoly. DFP has first look on the world’s best ad placements, and Google successfully sneaked in an “AdX tax” within DFP dynamic optimization, an early attempt at allowing programmatic demand to compete with direct demand.

But in 2017, AppNexus has a unique opportunity to strike. Premium publishers are switching to a programmatic-first model, and must rethink their monetization stack. At the same time, Google seems confused over its programmatic strategy, or fails to grasp the importance of DFP for its ad business.

The server-side header bidding deal with Index will enable AppNexus to cross-sell its publisher suite, including AOS for managing direct campaigns. One SaaS platform to fully unify direct and indirect demand.

Brillant move by AppNexus.

End-to-End Monetization Stack

AppNexus also gets access to Index’ valuable supply.

Somehow, Index’ black-box Header Bidder solution proved more popular with Comscore 100 pubs than AppNexus’ own open source offering, PreBid. Publishers were not ready to embrace the self-service open source solution, and instead favored Index’ out-of-the-box and well supported header bidding offering.

AppNexus must secure direct access to the best ad inventory as it builds an end-to-end monetization stack with the scale and feature set to compete with Google and Facebook

Pure-Play SSPs in a Weak Spot

You would think Index came to the negotiation table with in strong position. They surfed the wave of header bidding beautifully and wholeheartedly, and scooped up many top publishers as they switched to header bidding. Quite impressive for a network that used to specialize in annoying pop-unders back in the days.

Yet, with this partnership, these same publishers will be stirred toward AppNexus if they want to move their monetization capabilities back to the server. And most publishers will do just that.

Puzzling move by Index.

Index must believe its options are limited. With header bidding moving back server-side, the deck is being reshuffled once more, and this time the giants of AdTech will not be caught napping.

Publishers are hiring programmatic expertise as fast as they can, and are ready to bring their monetization stack fully in-house at last.

Index Will Have Regrets

No doubt Index was lured by AppNexus’ extensive unique demand.

But owning the demand means AppNexus will ultimately call the shots.

With header bidding, DSPs see the same impressions from multiple SSPs simultaneously, making it easy to prioritize one source over another. This practice, known as Supply Path Optimization, will gain traction in 2017.

AppNexus will find it easy to “punish” or favor some SSPs without hurting its own business, giving it great bargaining power, and ultimately control over Index’ supply.

And if AppNexus must torpedo this partnership to catch up with Google, so be it.

Supply Path Optimization: The Kiss of Death to SSPs

Over the last year, the ad tech industry has witnessed the fast rise of header bidding and similar publisher-side mechanisms aimed at unifying programmatic auctioning for a given impression.

For demand-side platforms (DSPs), this means more liquidity, but also a three-fold increase in the volume of bid requests to handle. In the old waterfall auctioning model, only one impression was sent to many buyers, whereas header bidding is fundamentally a “many-to-many” model. Craig Mytton illustrated this mechanic perfectly in a LinkedIn post last year.

As a result, DSPs are now flooded with duplicated impressions, threatening the stability of their platform and the integrity of their campaigns.

“Header Bidding has an exponential effect on the volume of impressions received from publishers” noted Ian Trider, Director of RTB platform operations at Centro, a DSP. “Two header bidding buyers means twice the number of impressions sent, three header bidders mean three times more impressions, and so on.”

With 70% of top publishers using header bidding, this redundancy will have a profound impact on the business dynamic among programmatic actors. At the same time DSPs will be forced to de-dupe impressions, the economics along the supply chain will become more transparent.

Middlemen’s Margins in Full View

For the first time, DSPs can see how aggressive intermediaries are along the supply chain. Brian O’Kelly provided sample math in a recent blog post. In short, SSPs that bid highest will maximize revenue in the short term, but undermine campaign performance in the long term.

Like AppNexus, DSPs are responding with supply chain optimization (SPO), essentially prioritizing which SSPs are allowed to submit bids for a given app. Basically, the old publisher-side waterfall is being replaced by an advertiser-side whitelist. Only supply sources on this whitelist are allowed to send impressions for a given page or app.

In time, I believe machine learning will make these whitelists more dynamic and efficient. Looking at recent bid patterns and fill rates per ad placement, DSPs will predict which SSPs or supply sources they should privilege when bidding.

This new practice of supply path optimization will become part of any modern buying stack.

Eco 101: Perfect Competition Equals No Margin

For SSPs, this is bad news.

This pricing transparency, along with low switching costs, will put pressure on their margins.

Whichever SSP charges the highest margin will quickly see their fill rates tank, as DSPs around the world shift their buying pattern to other supply sources available for the same publisher.

OpenRTB recommends the domain name or app bundle in any bid request, so DSPs can easily identify who the originating publisher is.

IAB and TAG will do their bit by offering mechanisms within the bid stream to uniquely identify a placement across platforms. There is also discussions of  each intermediary adding a “stamp” to the ad chain, similarly to the blockchain of a bitcoin.

Reselling premium inventory will become a lot harder for those platforms that do not have access to valuable data or failed to lock exclusive access to supply.

M&A’16: Data plays, Telco Power, Enterprise Software Vendors, Dentsu

Telcos dream of becoming media companies. AT&T $85Bn acquisition of Time Warner bested Verizon shopping spree.

AT&T splashed a mere $4.8Bn for Yahoo in July to boost up its AOL eclectic publishing empire with its myriad of AdTech platforms. Why are pipe providers so obsessed with content is a bit of a mystery. Leveraging their user’s data to boost advertising yield on their acquired properties, we’re told. Herd mentality I’d say.

Microsoft acquisition strategy? Let’s throw everything at the wall and see what sticks. LinkedIn, acquired for $26Bn in June, is actually a rising star in publishing. And yes, there is first party data as well…

The other deep-pocketed enterprise software vendors continued building up their programmatic stack. Adobe acquired video DSP TubeMogul for $540MM. Oracle gobbled up AddThis in January and Crosswise in April, a lesser-known cross-device mapping vendor.

TapAd, the leading cross-device vendor, was scooped by a small European mobile carrier. No obvious synergy. Independent cross-device vendors have no future, with the like of Facebook and Liveramp pretty much giving away their cross-device graphs.

IBM deserves a special mention for gobbling up 4 digital consultancies & marketing agencies in Q2. Ressource/Ammirati,, Aperto and Bluewolf will be folded into iX, IBM’s in-house agency, with the aim of expanding its footprint beyond IBM core offering.

Media love to talk about Chinese buying premium assets. Yet this is more of a trickle than an onslaught. Some unknown Chinese consortiums bought Mobile SSP Smaato for $148MM in June, for $900MM in August, and, mysteriously and expensively, AppLovin for $1.42Bn in September.

Rovi, the leading content guide channel, bought TiVo for $1.1Bn in April, mostly for its patents and technological assets. The combined company, called TiVo, hope to become a leader in addressable programmatic TV, possibly the hottest segment in digital advertising going forward.

The Private Equity firm who bought Mediaocean last year also bought Marketo for $1.8Bn in May. I have no idea what they are trying to achieve.

Forensiq, a minor fraud detection vendor rooted in domain spoofing detection, was acquired by Impact Radius in June. Expect a lot more acquisitions in the inane fraud detection space next year, as industry-wide initiatives emerge at last to tackle NHT and viewability standards.

Vector finally put Sizmek out of its misery in August, buying the topsy-turvy mobile AdTech platform for a mere $122MM. Sizmek is a great example of how difficult it is to grow by acquisition in the fast moving AdTech sector.

Last, but not least, Dentsu bought 2 US digital companies: Email marketer Merkel in august for $1.5Bn, then trading desk Accordant one month later.

2017 Native Predictions

I and the other members of the IAB Native Standardization committee wrote our predictions for native advertising in 2017.

My notes:

Native standardization by the IAB has bridged the gap between social marketing and the programmatic display ecosystem.

At PulsePoint, our social marketing & sponsored content distribution platform used to be siloed from our programmatic offering.

Native standardization enabled us to distribute native-style display ads across hundreds of publishers, leveraging existing OpenRTB integrations.

Similarly, our DSP partners started scaling up native campaigns without worrying about the how their ads would blend within publisher content, enjoying far greater engagement and reach than with banner.

In 2017, we anticipate native advertising to be our fastest growing channel, slightly ahead of pre-roll video.

In-feed will still make up the bulk of native ad revenue, as many of PulsePoint traditional publishers focus on text-heavy or user-generated content.

We have big ambition for native video for next year. Constraints are mostly demand-side, but both publishers and advertisers are getting more comfortable with embedding video content within in-feed ads, at a substantial CPM premium over static media.”

This said, and as a few of my co-members noted, adoption of native has been below expectations in 2016.

Video is the main reason, as Kayla Wilson noted: “DSPs de-prioritized [native buys] when they realized this year was actually all about in-app video”.

Facebook exited the OpenRTB ecosystem


Once upon a time, Facebook had grand ambitions for becoming an end-to-end programmatic advertising platform, capable of rivaling Google. It bought Atlas – a display ad server -, LiveRail – the leading video ad server -, launched its FBX exchange, and was planning DSP capabilities.

No more. Facebook spent the better part of this year scaling down its advertising stack to 2 key areas: selling media on its O&O properties, and leveraging its “social graph” to sell third-party media.

OpenRTB no more

In less than a year, Facebook has mostly exited the OpenRTB-based programmatic ecosystem to focus on selling direct instead.

First to get the boot was FBX. Facebook’s exchange has been moribund for years, once it became clear that Facebook would not onboard its mobile inventory. The ax finally came in January 2016.

LiveRail video platform followed shortly. Facebook terminated its third party publishers. “Too many bots to police”, the company said. Welcome to my world.

In march, Facebook abandoned plans to add DSP features to its Atlas platform, for buying media across third-party publishers, leveraging Facebook’s deep targeting capabilities.

Facebook is still mulling about programmatic native and video, where supply is cleaner and yield higher. But no more challenge to Google’s DoubleClick in the open market.

Insights into insights

So Facebook Audience Network (“FAN”) is what’s left of the company’s grandiose ambitions. The old-school network is rumored to gross $2Bn this year from 3 million advertisers, yet is no pinnacle of innovation.

Atlas will survive as an analytics and attribution platform, leveraging its social graph for better audience and cross-device insights.

Facebook  will be alright. The social network controls 20% of display inventory in the internet, and the most extensive trove of user data.

It should do more with it.

Addressing iOS users is getting harder

Today Apple announced that iOS 10 users can blank entirely their Device IDs (“IDFA”) to advertising SDKs and mobile browsers.

No short term impact

Not a big news in itself, as few users have opted in to blank their device IDs  – 17% according to a recent study, much less in our network -. iOS 9 already had a similar block feature, that still allowed advertisers to use opted-out Device IDs for cross device targeting, reporting and conversion tracking.

After today’s change, advertisers will simply rely more on device fingerprinting to uniquely identify iOS users that opted out of sharing their Device IDs.

But once more Apple is tightening the screw on programmatic advertisers.

A worrying long term trend

Safari has long been a cookieless environment, and the iOS app ecosystem it taking that direction.

Clearly, advertising platforms should start looking seriously into fingerprinting technology and location-based profiling, to ensure iOS users can be addressed in the long run.

Device fingerprinting to the rescue

AdTruth, and BlueCava offer technology to probabilistically identify a device based on its physical characteristics, IP patterns and other behavioral factors.

These vendors claim to be a reliable substitute for the hardware’s Device ID in 94% of cases. Our internal tests have shown a lower but still acceptable match rate within our network.

Alternatively, Augur.js in an open source library to do device fingerprinting. We haven’t tried it, but worth considering.