On May 13, 2021, Knight Foundation announced a new, $3 million initiative to help local news organizations harness the power of artificial intelligence. Below, researchers outline an industry survey they conducted to help Knight understand the landscape.

Automation algorithms are all around us, detecting fraudulent use of our credit cards, determining what you see in your social media feed, and displaying shoe ads that follow you around online.

But how are news organizations using artificial intelligence, machine learning, and other algorithms for automation? We set out to survey the industry to help Knight Foundation understand the landscape and spot possibilities for future funding.

We collected 130 projects, focused primarily on projects done within the past three years. We drew from our own knowledge in data journalism and machine learning for journalists, as well as interviews, outreach into journalism-technology networks, examples described at conferences, and research done on the topic, including work done by JournalismAI project at the London School of Economics and by Jonathan Stray.

Augmenting reporting is a major focus

Almost half of the projects we surveyed used AI for “augmenting reporting capacity” These projects comb through large document dumps with machine learning, detect breaking news events in social media, and scrape Covid-19 data from government websites. 

The second significant area AI is used in journalism is for “reducing variable costs.” That includes tools that automate the process of transcription, tagging of images and videos, and story generation. The category of projects that used AI for “optimizing revenue” — including dynamic paywalls, recommendation engines, and the digitization of a news organization’s archives — ranks third. 

“Engagement” counts efforts to corral audience input, such as the algorithm KPCC / LAist used for sorting thousands of COVID-19 questions into manageable buckets. “Self-critique” includes work to foster gender and racial balance in an organization’s stories, and “news reporting” refers to situations where the result is the story, such as the real-time election needle at the New York Times.

It’s worth noting that some projects may cover a few different purposes; we chose the primary purpose based on what appeared to be the intent of the project. For example, a machine learning algorithm that combs through documents for keywords augments reporting capacity and at the same time reduces variable costs. We put that under “augmenting reporting capacity.” Understanding why newsrooms use AI can help both industry insiders and outside stakeholders identify the demands from the field. 

Next, for the same AI projects surveyed, we looked into where they appeared in the news pipeline. Placing them in the whole picture of the news ecosystem helps reveal the spots for improvements. 

From newsgathering to product development to subscriber acquisition and retention, newsrooms have used AI across the entire news production process. But as the following chart shows, when we talk about AI in newsrooms, we seem to lean heavily on the newsgathering part of the process and maybe do not pay as much attention to the product or the business side of the ecosystem.

Most AI projects happen at larger news organizations

Not surprisingly, national and global AI efforts far outpace those at the local level by simple project count. The larger organizations have more resources, in time, people, and money, to devote to innovation and experimentation. They also may have larger upsides for those investments.

AI projects require people with specific skills. But people we spoke to people at smaller news organization who have those capabilities — including newsroom and product engineers — who simply do not have the bandwidth. They must devote to their existing obligations.

Justin Myers, the data editor at The AP, said it was difficult to do AI projects in a big newsroom if the person with the skills was not specifically hired for that reason, not to mention for even higher hurdles for local newsrooms. “Finding someone with the time, skills and resources is hard for a newsroom. Finding a project where the level of effort pays off is hard.” 

But the need for AI for local newsrooms is equally, if not more urgent. 

The work AI can do is “work that reporters could do without machines, but it would take much longer. I see a big benefit of AI as the reallocation of resources, especially for smaller newsrooms,” said John Conway, vice president of WRAL Digital at Capitol Broadcasting Group.

A third of the projects at large newsrooms could be repeated for smaller ones
Software designed to automatically tag pictures may not be necessary for news organizations who don’t process thousands of images per day, but we estimated that  44 of the 130 projects surveyed could be adapted for smaller-scale use.

Here is a list of highly repeatable use cases of AI at the local level:

• Templated sports, schools, real estate, and other stories.
• Reporting tools including transcription services, entity extraction from documents, claim/fact identification, social media event detection.
• Engagement helpers, such as the KPCC / LAist Covid-19 question sorting system.
• Dynamic paywalls and subscriber prediction algorithms.
• Recommendation engines.
• Photo searching and tagging systems.
• Homepage curation systems.
• Self-critique systems, monitoring gender and racial bias in stories.

Many of the use cases fall into the realm of tools or systems that could operate almost invisibly inside a news organization’s subscriber or a content management system.

Given the diversity of publishing and business management platforms on a local level, it may be a high hurdle to put these AI-driven tools in place. That said, among news organizations already using a common platform, such as Newspack for WordPress, solving the adaptation problem once would benefit many outlets.