The Six Hours Between Programmatic and Real Time

A lot can happen in six hours: You can fly from one coast to the other or run a marathon. You can even play in a Super Bowl or two!

Mainstream DSPs can take six hours or more to report the media buys that were made on your behalf. Digital media vendors boast about the latest and greatest in real-time bidding, yet they are willing to label something that takes almost an entire workday “real time.” That's as if a head football coach prepared a playbook, handed it off to his quarterback, got on a plane to London and then waited until he landed to find out who won.

That is not real time. Let's compare Madison Avenue to Wall Street. You can tell eTrade to buy stocks for you up to a limit and the system will buy for you all day. You can then see what buys were made on your behalf with up-to-the-minute visibility.

So if the technology exists, why is there a delay in real time in the online advertising world? There are dozens of excuses, most commonly that it's either too hard or too expensive. Some companies claim that there is too much data to translate quickly, or that marketers don’t really need true real time. These are lazy rationalizations.

The real reason that it takes six hours to get real-time information is that the DSPs, agencies and exchanges profit from the lack of transparency, and marketers won't hold them accountable.

When cell phones were first introduced, people questioned if they really needed a mobile phone. They were slow to adopt the technology. Sure, you could talk on the phone before mobile phones were invented, but now you are able to do all sorts of things you could never do before. Someone had to invent the cell phone before people realized that they couldn’t live without real-time communication. The same thing will be true when actual real time hits online marketing. What is acceptable today will become anachronistic.

Speed is inevitable. Years ago, on Wall Street, people questioned the need for speed. Those that embraced speed ahead of the curve not only ended up making a lot of money, they enabled the average person to buy stocks for a lot less than before, thanks to increased liquidity and transparency. The large monopolies that tried to hold back ended up losing out against change.

Agencies and their vendors should realize that real time is inevitable, regardless of whether or not they embrace it. If they don’t do it, someone else will. Marketers must insist on true real time, and agencies need to be held accountable.

If your agency can’t explain its bids to you in real time, then maybe you need a new agency.

Tags: rtb
Recommend (40) Print RSS
7 comments about "The Six Hours Between Programmatic and Real Time".
  1. Nate Carter from eEffective , March 17, 2014 at 12:49 p.m.
    Ted can you flush out what you mean by this statement? "The real reason that it takes six hours to get “real-time” information is that the DSPs, agencies and exchanges profit from the lack of transparency, and marketers won't hold them accountable." How do they profit? Thanks for the clarification.
  2. Myles Younger from Canned Banners , March 17, 2014 at 1:37 p.m.
    Agreed that reporting will eventually become truly real time, and it may take a bold newcomer to "wake up" the big, lumbering incumbents, but there are valid barriers to offering real-time reporting. First, most current DSPs are built as extensions of ad servers, exchanges, and reporting systems that are 5+ years old and were never designed to offer true real-time reporting; overhauling and migrating these platforms will be a truly massive undertaking. In addition, a DSP that's plugged into multiple exchanges often has to take a "lowest common denominator" approach and offer advertisers the reporting speed of its slowest exchange partner. Otherwise data is going to be revised so often and so drastically that advertisers will never have any confidence in the "real-time" numbers they're seeing. Another thing to keep in mind is that there are workarounds. For example: using a separate impression and/or click tracking vendor that offers real-time reporting. Lastly, your stock market analogy may be less apt than you think. A lot of (nearly all?) trades aren't cleared until the end of the day or overnight when markets are closed (and keep in mind that the "market" for ads NEVER closes). In other words, the trades and quotes themselves may be real-time, just as RTB is real-time, but the tallying up of results is done in bulk.
  3. Alexis Wrightson from Affectv , March 17, 2014 at 1:47 p.m.
    I think he means the optimisations people do cannot be reported on until the day after in some cases. Therefore, you could spend a small fortune running a strategy that may yield no fruit. It is exactly right and the problem with a lot of DSPs.
  4. Frost Prioleau from Simpli.fi , March 17, 2014 at 1:59 p.m.
    For the record, not all DSPs have a 6 hour delay in reporting. My company, Simpli.fi, is a DSP and our reporting, which is very granular and transparent, typically runs at about a one-hour delay. We could likely make this faster, but we haven't sensed a need beyond that. Ted- What is your ideal? By the way, for all the exchanges that we are integrated with we get clearing prices within minutes...so there is not a hold up on the exchange end for us at least.
  5. Ted Yang from MediaCrossing , March 17, 2014 at 4:41 p.m.
    @Alexis - Yes that's right. @Nate - For example, intermediaries profit when you spend more than you normally would on a strategy. Even a simple typo can result in buying something you didn't intend, so there is no incentive for them to be more transparent, or even efficient. Many players in our ecosystem are cost-plus: they reap the benefit of you acting on 6-hour old information. @Frost - Near real-time (15-20min delay) is an achievable and useful goal. I agree the "need" isn't quite there, again because advertisers need to be trained on the benefits of quick results.
  6. Vadim Tsemekhman from MediaCrossing , March 18, 2014 at 9:32 a.m.
    Because the impression cost on the exchange is very volatile, only a short-term average can help predict and optimize spend. Anything that averages over hours makes such predictions totally unreliable, and, intentionally or not, the DSP wins by getting higher bids than necessary. A bigger problem is the inability to optimize, which would have been made only somewhat easier with real-time reporting. DSP's are not disclosing how and whether they optimize, and near real-time reporting would have at least opened the opportunity for the client to look, evaluate and adjust the optimization strategy. Smoothing over long period of time removes any granularity, and the client is left with only very high-level optimization. Ad networks might have this real-time optimization capability but intentionally keep it to themselves because their incentives are quite different.
  7. Mike Driscoll from Metamarkets , March 19, 2014 at 10:02 a.m.
    Ted - You suggest that a six-hour delay exists in part because "DSPs, agencies and exchanges profit from the lack of transparency" -- but I believe a less conspiratorial reason why most DSPs do not provide real-time reporting: this capability is costly to build, and many are skeptical of the return they'll get on that investment. Most firms in ad:tech rely on batch-oriented systems, powered by a decade-old technology called Hadoop, to crunch their terabytes of server logs and generate reports. This typically introduces anywhere from a 15-minute to six-hours to even 24-hours of delay. At Metamarkets, we spent last year building a truly real-time pipeline for our clients (with a six-second, not a six-hour, delay). But we've have to do some convincing to get our clients to embrace this new approach: it's often hard to imagine the value of a new technology until you have it. The number of use cases -- from alerts on campaign overspend and fraudulent activity, to "debugging" of in-flight campaigns -- is increasingly a core part of a programmatic marketer's toolkit. Opinions like yours will help more people realize that real-time bidding markets, like Wall Street's, will benefit tremendously from real-time insights.