Why The Globe and Mail trusts an AI to run its paywall

Sophi not only automates the metering of stories, but it also controls story placement on the homepage.

For the past decade, publishers have utilized metered paywalls to grow their subscription businesses. Under that model, a reader gets to view a certain number of free articles before a paywall pops up and requires them to subscribe. 

But how many free articles should a user encounter before they hit a paywall?

Increasingly, the answer to that question is: It depends. Publishers are starting to roll out dynamic paywalls that assign varying weights to different kinds of stories. If you’re reading business news, for instance, you may only get to read three free articles before hitting a paywall, but if you’re perusing real estate listings you might get unlimited free access.

The Globe and Mail has taken the idea of the dynamic paywall to the next level: it’s developed a sophisticated AI that’s able to analyze user behavior and determine the exact moment that a reader is most likely to subscribe. The AI is so powerful that the newspaper’s editors now allow it to automate the placement of stories on its homepage and social media.

I recently sat down with Gordon Edall, the person who runs the product team that developed the AI. We talked about how the paywall was initially designed, his experience recruiting data scientists, and why the Globe and Mail is licensing its AI product to other publishers.

To listen to the interview, subscribe to The Business of Content on your favorite podcast player. If you scroll down you’ll also find some transcribed highlights from the interview.

iTunesStitcher/  OvercastSpotifyGoogleYouTubeAudible

This transcript has been edited for clarity.

Building the initial paywall

Gordon has spent most of his career in media, moving back and forth between the product and editorial sides of the business. He started at The Globe and Mail in 2006 as a business editor. “My day would start at 6:00 AM, and it was really a job of setting up the editorial agenda for the day, even before most of the reporters were in,” he said. “It was very much a pure editorial function.” In that role, he constantly found himself diving into audience data and mining insights on what kind of content seemed to generate the most engagement from readers. This was in an era when very few legacy print publishers were capitalizing on that kind of data.

Eventually, Gordon began to migrate over to the product side, first to build out the company’s financial tools — things like stock quote pages and stock filters — and then to develop mobile applications. His success on those projects led to him being tapped on the shoulder in 2011 to help build The Globe and Mail’s new paywall.

Given the recent success The New York Times had seen with its subscription product, Gordon’s team quickly settled on a solution that mixed freemium and metered models. “A paywall is really this sort of mechanism for introducing artificial scarcity to drive conversion,” he said. The challenge for his team was to figure out the exact point to introduce that scarcity.  “From the very beginning, the paywall was built on the idea that some content was reserved for subscribers only and that some content was going to be metered.”

They developed a number of fixed rules about how the paywall would behave, and they used a traffic light concept to enforce these rules. The premium content that always went behind a paywall was labeled red. Metered content was yellow.  “We always had some content that we knew would be free for everyone,” Gordon said. “This is your service journalism, social justice stories related to policy change, and your brand halo stories.” Those were labeled green. It was ultimately up to the editors to decide which colors got assigned to stories.

After lots of testing, the paywall team rolled out a fourth color: Blue. If a reader landed on a blue story, it counted toward a meter, but they would never see a paywall while on a blue story. For example: let’s say the meter was set at four free stories; a user could land on five blue stories and never see the paywall. They’d have to subsequently land on a yellow story before triggering a pop-up. “The creation of blue was the realization that we were allowed to build a paywall that was hard to understand, and that you didn't necessarily have to have it be something that was easily understood by everyone in every moment they were on the site.”

Building Sophi

As the paywall grew more sophisticated, it became clear to Gordon’s team that they would need to rely on something better than simple human judgement to assign colors to the hundreds of articles the outlet published every week. In other words, they needed to build up their data science and machine learning capabilities. “We were looking for people with a background in predictive modeling, and there was sort of a recognition that there are other types of businesses in Canada that have really figured out how to manage customer relationships. We were looking in the telecom space. We were looking in the financial services space, and then we were really lucky because we were able to reach into the heart of Research in Motion” — the company behind the Blackberry — “which employed a small predictive modeling group.” The Globe and Mail ended up recruiting Greg Doufas, the leader of that group, as its CTO. He then brought over several of his colleagues and now runs a team of 50 data scientists, many of whom are PhDs. 

This data team began building out an AI product that it later named Sophi. At first, Sophi merely assigned colors to stories to guide the paywall, but it didn’t take long for the data scientists to realize its potential for optimizing the website, specifically its homepage. Choosing stories for the homepage was a job that typically fell to the editorial team, so Gordon and his colleagues had to tread lightly in terms of winning over the newsroom’s support.

The product team developed a system in which Sophi alerted editors when a particular story was overperforming or underperforming on the homepage. “It figures out whether a story is living up to expectations or not based on the total amount of promotion it's getting.” Almost as soon as this program rolled out, engagement and conversions skyrocketed, enough to convince senior editors that Sophi was providing real value. 

Eventually, this triggered a lightbulb moment. “David Walmsley, our editor in chief, was looking over the shoulder of his best and brightest assigning editors, the people running the home pages, and they were basically nudging the red circles down and nudging the blue circles up on our pages all day long. So in a moment of shocking awareness of the potential to change the way his people were doing work, he actually reached out to the data science group to say, ‘what would happen if you guys just let the computer start to make some of the decisions about which stories should run where?’” In other words, the newsroom was volunteering to give up homepage control to Sophi. 

Since then, Sophi has taken on more and more functions, from recommending content on pages to even scheduling out social media posts. Flash forward a few years, and the results speak for themselves. “When we first started this, advertising was around 70% of our revenue and reader payments were about 30% of our revenue. Today, reader payments at the Globe and Mail make up 70% of our revenue.” Though the proportions have switched, the outlet has continued to grow its advertising revenue at a faster rate than most of its competitors. 

Licensing Sophi to others

Now that Sophi proved its effectiveness, the Globe and Mail has begun offering it as a SaaS product. “We work with paywalls vendors. We work with CMS platforms. We have some really clean integrations with Arc, with WordPress, and with some more CMSs that we'll be announcing in the future as well. We make it really easy to integrate predictive capabilities into your existing stack and drive performance directly. We have some really fascinating stuff we've been doing on the print side of things where we’re using Sophi to lay out the print editions of newspapers. We can produce camera-ready pages, a full newspaper run in under a minute in a world where that used to be a four or five hour task for a good design group.”

Once Sophi is up and running at a new publisher, it takes about three months of data collection before it’s able to fully optimize the content. “We're really seeing pretty high performance states reached within the first month, and by the end of three months, you're very close to what we expect to be an optimal run rate.” Typically, it doesn’t take the full three months for publishers to see substantial returns on their investment. While human editors and journalists will always control the creative side of content production, it’s just simply impossible for a human being to compete with the optimization capabilities of a well-trained AI.

Do you like this newsletter?

Then you should subscribe here:

Simon Owens is a tech and media journalist living in Washington, DC. Follow him on TwitterFacebook, or LinkedIn. Email him at simonowens@gmail.com. For a full bio, go here.