How newsrooms use open-source tech to build in-house tools

Although the news industry is typically characterized as fiercely competitive, the philosophy of the open source movement is encouraging The Times and The Sunday Times, BBC News Labs, and The Wall Street Journal to make their newsroom tools available to others in the industry.

Jacqui Maher, an interactive journalist at BBC News Labs, outlined to the news:rewired conference, in London today, a series of her team’s projects that use APIs and open-source software. Hooked, an experiment using comics in digital storytelling designed specifically for audiences in Sub-Saharan Africa and Ghana, couldn’t have been developed without the work of one of her friends at al Jazeera America, she said..

She added:

Wherever we can in the Newslabs, if there’s something already around that can get us 90% of the way there, we’ll absolutely use it.

Nick Petrie, the deputy head of digital at The Times and the Sunday Times, told the conference about his organization’s open-source tool CardKit. Used extensively by his social media team, it provides a custom template to upload with Twitter posts that might not have striking images accompanying them.

CardKit’s code has been made publicly available on Github and, Emily Cadman of the Financial Times, who moderated the panel, said that her newsroom had customized the code for its use as well.

Petrie said:

A lot of what we do is built on shoulders of other people’s hard work, so we try to contribute back when we can.

Delegates were told by the respective panellists how some projects, such as the BBC’s natural language processing tool, The Juicer, and Deck, The Times and Sunday Times’ newsletter layout tool, remain in-house technologies only. However, this isn’t always to protect in-house developments so much as a response to the cost of rewriting code to make it publicly available.

Elliot Bentley, graphics editor at the Wall Street Journal, outlined to delegates how he developed the mapping tool Pinpoint a few months ago for use when the organisation’s in-house cartographers were out of the office. Using Leaflet basemaps and the Googlemaps API, it makes significant use of open-source programs, he said.

Bentley adapted Pinpoint for open-source use as well, but said it cost him an extra week of work following the tool’s development. Months after making the code public, he hasn’t seen it adopted by any other news organisations.

He said:

I think maybe as an industry we often open-source these things but they’re not so easy to adopt and adapt for other people. And it is a real challenge. I haven’t seen other organisations using Pinpoint even though it’s been out for a few months.

Petrie said that while it’s important to try to contribute back to open-source development, his team can’t always document their open-source projects to make them user-friendly. He said that going back and documenting code comes at the cost of lost time on other internal projects.

But by participating in communities of journalists and developers like the groups Hacks/Hackers and Journocoders, Petrie said developers can acknowledge and contribute to the work of others.

Planning to make software open-source from the beginning of development can also provide a solution for newsrooms looking to make their tools publicly available. Bentley said the earlier a decision is taken to start open-sourcing, the easier it is, saving time and money for developers.

“What I took away from this is this is now a mature part of big newsrooms,” Cadman said to wrap up the session. “This is now something that’s getting rolled out for everyone.”

You might be interested in …