• by Ray Schultz , Columnist, July 27, 2018

The New York Times sends almost 4 billion emails a year to its customers. So it is no small thing when the paper overhauls its email platform. 

In a story today, the Times reports that it moved its infrastructure to the cloud to the cloud earlier this year. The piece is noteworthy on two fronts. First, it is a detailed case study of an enterprise email system. And it is a remarkable display of transparency. Here is a summary.

Most of the Times’ emails are newsletters for audiences that vary in size from tens of thousands to millions. But a small portion are transactional emails such as paid subscription confirmations.

Before this, the paper was saddled with an unscalable system full of legacy code. So it decided to move to the cloud and build a platform that could grow with demand. To this end, a serverless architecture built on top of Google Cloud Platform.

The Times writes that “the core sending infrastructure relied on layers of archaic and often undocumented scripts to select audiences and compile emails. Countless frustrating hours of debugging and coaxing were needed to address the near-weekly failures.” 

In technical terms, the Times relies on “a GAE API with Datastore to manage each individual user data object. Every time a user object is modified, a record of the updates to the data (e.g. a new subscription to a given newsletter product) is published to a Pub/Sub from which it is then inserted into BigQuery, GCP’s data warehouse that runs SQL-like queries.”

The paper continues: “GAE’s quick scaling means that many instances of the email compiler can spin up very quickly to handle the potentially millions of compile tasks coming from the Pub/Sub, and this architecture makes it easy to add additional API calls as data sources for email content (such as the personalization API or an A/B testing framework) without re-tuning the scaling.” Transactional email dispatches also go through this pathway.

The result? The Times writes that the news system enables it to:

• Keep track of a reader’s newsletter subscriptions 
• Quickly select audiences when sending newsletters
• Provide an admin UI for editors to assemble email content and schedule newsletters 
• Compile and send individual emails to each recipient
• Provide customer APIs for other teams to send transactional emails

The system also provides these benefits: 

• It now rarely takes more than 10 minutes to deliver a newsletter to a large audience, in contrast to the old system, which could take hours 
• Since launch, there has been “a dramatic reduction” in weekly system failures 
• Problems are easy to flag, thanks to a suite of alerts and monitoring dashboards
• The data storage setup is transparent, and covers everything from GDPR compliance to measuring email recipient engagement 
• It easily facilitates integration of additional API call or data source

The Times concludes: “We now have confidence that we can rely on a platform that’s scalable, resilient, and sustainable. Rather than simply trying to keep things running, we’re now able to focus on building out features and tools that will help editors create more engaging newsletters to build even deeper relationships with our readers.