How The Markup mapped outdated infrastructure to disparities in internet pricing across America

Internet access is an essential lifeline to learn, work, access public services and fully participate in today’s society. But for many American households, broadband internet is still too expensive, too slow and out of reach.

In 2022, The Markup’s Aaron Sankin, Investigative Reporter, and Leon Yin, Investigative Data Journalist, embarked on a months-long investigation that revealed the country’s poorest neighborhoods disproportionately pay higher prices for slower internet. The resulting project, Still Loading, was a winner in the Excellence in Technology Reporting category in 2023 Online Journalism Awards. 

More than a dozen other newsrooms have applied guidance and data from The Markup to investigate internet pricing disparities in their own area, including in cities like Seattle and Portland (InvestigateWest), Detroit (Outlier Media), Minneapolis (Sahan Journal) and others. Among several resources, the team built tools to help people search for better deals on broadband internet and partnered with Consumer Reports for wide distribution of the comparison-shopping guide. Most recently, The Markup partnered with Capital B to amplify the stories of Black Americans in the rural South who still have no or limited options to access high-speed internet.

We wanted to learn more about the elements that raised this project to award-winning status. Aaron generously shared about what motivated the initial investigation, lessons for other journalists and the kind of impact that technology reporting like this can create for communities.

What inspired this particular project’s inquiry into connectivity and the digital divide?

“Still Loading” came on the heels of another Markup investigation looking into a predictive policing algorithm that disproportionately directed officers to patrol marginalized neighborhoods. That policing algorithm story was all about the lingering reverberations of America’s long history of race-based housing segregation. If certain neighborhoods were more likely to be subjected to increased police patrols, we wondered, could the deals offered by internet service providers (ISPs) also be distributed inequitably?

Our project began with a data collection method. Three years before The Markup’s investigation, a team of researchers at Princeton published a study that used data scraped from the websites of ISPs to show how a nationwide map of broadband connectivity maintained by the Federal Communications Commission had widespread, systematic inaccuracies. Once we heard about this way to gather connectivity data, we realized we could use it to map other elements of the digital divide. 

Prior to using this scraping method to  collect data directly from ISP websites, there was no publicly accessible way to determine what level of connectivity was offered to U.S. households at the address level. At the time, the FCC’s map only showed the highest level of connectivity offered within each Census block.

Once we built scrapers and reviewed our initial data, we quickly realized that connectivity varied dramatically from one house to the next, and furthermore, one household’s sluggish connection was frequently offered for the same price as their neighbor’s fast one.

These realizations suggested a hypothesis that we spent the better part of a year working to prove: that ISPs were offering disproportionately significantly worse deals to parts of cities that were less White, lower income, and historically redlined.

“Reporting on the digital divide for this story revealed to us that the reason why certain areas in major cities had better internet service than others was largely about where ISPs decided to upgrade their networks.”

A lot of work went into encouraging other journalists and the general public to look into the digital divide where they live. What insights and breakthroughs did the project team experience that others can learn from?

Outside of our initial investigation, almost all the work we’ve done in the “Still Loading” series is aimed at empowering others to take our story one step further.

• A how-to guide for local reporting: Shortly after publication, we got a slew of responses from regulators, lawmakers and members of the public who wanted to know if our investigation applied to their neighborhood. We published a guide showing how other news outlets could localize our findings for their own communities, which over a dozen did.
• Mapping slow internet speeds in 45 U.S. cities: We developed custom, address-level interactive maps, allowing readers to see the digital divide in their communities with their own eyes. We had to develop these maps ourselves since most widely-used, commercially available web mapping tools send user data to third-parties, which violates The Markup’s privacy promise to readers that we would never send their data into the online advertising ecosystem. 
• Build and analyze your own dataset: The next step in this series was creating a guide allowing members of the public to investigate internet disparities themselves. Our initial investigation only looked at four ISPs operating in 38 major U.S. cities. Many people asked us to expand our investigation to internet providers and communities we hadn’t originally examined. But replicating our exact investigation would require scraping websites and analyzing large amounts of data, which takes considerable programming and statistical skills. Our guide was built to avoid those barriers of entry—no programming or intense statistical analysis skills required.
• Creating the address sampler: We partnered with Big Local News to develop the United States Place Sampler—a free, public tool for obtaining a random sample of addresses from any city, county or state in the country. We also published an easy-to-use Google Sheets template that automatically pulls in census data about the addresses you collect from the Place Sampler, so people can immediately get demographic information, like median income and race, about the addresses they want to test. Finally, we taught readers how to look up these local addresses on ISP websites, then record the prices and speeds offered.
• Answering “So what?” — a tool to find better internet deals: Giving people tools to analyze disparity wasn’t enough. People are already paying way too much for crummy internet service, and those people need solutions that can help them as quickly as possible. That’s why we published a second guide showing people how to check if another internet service provider offers a faster or cheaper plan using the FCC’s recently released nationwide broadband availability map. As a bonus, our guide also taught readers how to identify and report errors, which helps ensure billions of dollars in federal broadband funding go to places where the money is most urgently needed.

For distribution, we partnered with Consumer Reports to get this comparison-shopping guide in front of their civic-minded, price-conscious readership (hundreds of their readers have followed the guide), and we partnered with the Detroit Documenters, who found and reported errors in the FCC map minutes after they started. 

The lesson here is that publishing an investigation can be a jumping-off point for a larger project. We could make the methods we developed during the investigation accessible for readers to use themselves.

Thinking about the existing coverage and analysis on internet access, what work is left to do?

Reporting on the digital divide for this story revealed to us that the reason why certain areas in major cities had better internet service than others was largely about where ISPs decided to upgrade their networks. In certain areas, they had invested in the technology necessary to provide state-of-the-art connections, whereas others were stuck with decades-old technology.

Our investigation was, in effect, working to make a map of ISPs’ infrastructure based on the speeds they were offering at each address. It gave us a hazy picture of where ISPs were laying their newest infrastructure, but our view was far from complete.

The FCC recently proposed a new requirement that ISPs file documents showing where they are upgrading their systems. This dataset, once it is made publicly available, will provide a new opportunity to hold ISPs accountable if their new technology deployments are upholding the digital divide rather than attempting to bridge it.

Beyond The Markup’s work, what are other inspiring journalism projects that others should keep an eye on?

Lighthouse Reports and WIRED’s Suspicion Machines project is a great example of exposing an otherwise black-box system to accountability. It explained the effects of a fraud-detection algorithm by learning how the system worked and then feeding synthetic people into it to see how each of them was scored. The piece did a great job of making an otherwise complex, and potentially dry, issue visceral and accessible.

Grist’s Ghosts of Polluters Past** investigation into toxic lead contamination in Santa Ana, California is also a deeply inspiring piece of journalism. Not only did reporter Yvette Cabrera hand-collect over 1,600 soil samples from across the city, but she was also able to directly tie modern-day environmental pollution to zoning decisions and racially-restrictive deed covenants made near a century ago that funneled the city’s minority population into the most toxic neighborhoods. It’s always great to see journalists building datasets themselves by physically going out into the world and gathering data.

** Ghosts of Polluters Past was a finalist for the Knight Award for Public Service in the 2022 Online Journalism Awards. Get to know other honorees in this category.

Learn more

Launched in 2000, the Online Journalism Awards are the only comprehensive set of journalism prizes honoring excellence and innovation in digital journalism around the world. The call for entries for the 2024 awards is open April 4–May 9.