What we've learned from meta's big research project and other thoughts
The biggest news on the transparency front over the last few weeks is that the first 4 papers in meta’s big research collaboration around the US 2020 election were finally released.
I think this is an important project, I think very highly of all the researchers involved, and I think it’s likely that this eventually represents one of, if not the single most, important collection of data about social media and democracy that exists. I’m excited the first papers are out and eagerly looking forward to seeing the rest. In the meantime, though, here are the first four papers:
Science: “Asymmetric ideological segregation in exposure to political news on Facebook”
Nature: “Like-minded sources on Facebook are prevalent but not polarizing”
That being said, the project and the research have some real limitations, some of which are worth touching on as we think about researcher access to social media.
The big things that stood out to me were:
When it comes to the question about polarization, I think ultimately the paper released actually doesn’t tell us very much, unfortunately. I know very few quantitative political scientists who would have expected anything other than a null result because of how the experiment was designed & that we know how hard it is to change people’s political opinions & identities. Of course, sometimes null results are how you build knowledge but I think in this case, the experiments were too short-lived and took place within a political climate that was already too politically calcified (especially by the final few months of a two-year Presidential campaign cycle). To get to something more definitive, I think the real question is about what happens with a much longer exposure (see Tom’s great write-up) and also the specific effect on media & elites (see Ravi and Juliana), not to mention what the effects are within other information ecosystems with different affordances than the U.S. one.
The research *certainly* doesn’t tell us that social media plays no role and meta’s clumsy attempts to try and make that claim by overstating the findings of that paper were disappointing. At one point, they apparently put out the statement that there is a “growing body of research showing there is little evidence that social media…has any meaningful impact on key political attitudes, beliefs or behaviors”. Even for corporate spin, that’s a wild statement. But it’s also a good reminder (along with Mike’s thoughtful commentary and Justin’s bigger picture analysis) about why the real answer is wholly independent research required by the force of law. I hope meta isn’t still trying to make those arguments behind closed doors with regulators.
It’s obviously taken way too long to get the results out and unfortunately, that clouds a lot of how relevant the results still are, even as we debate exactly what they mean. Among other things, we know that the Facebook app has undergone three foundational changes since the research was conducted (much more recommended content, much more short-form videos, and far less politics).
I think we also need to be ok asking smaller questions. For instance, I think asking whether “social media is good for democracy” is basically the epistemic equivalent of tilting at digital windmills. The best research in this space should try and focus on narrower questions and narrower effect sizes, even among narrower audiences. The platforms we’re trying to study are too big, are made up of too many divergent and diverse use cases, and exist in too many heterogeneous communities that are themselves shaped and influenced by complex systems. That’s not in any way to suggest throwing up our hands but it’s about being efficient and strategic about the questions we ask and optimizing for impact & understanding instead of headlines & catharsis. Moreover, when you’re talking about social media, small effects can lead to big impacts. Even if it’s simply avoiding the opportunity costs of chasing after bad ideas.
In the end, I couldn’t help but imagine a world in which this sort of research was coming out regularly and how much time & energy & resources it would save us, as well as how much more progress we could make on actually addressing risks & potential harms. A world where we weren’t just looking back at elections years later and celebrating who was right & who was wrong but studying them & adjusting to them as they were happening. A world in which we weren’t just constantly studying the United States but countries & elections all over the world, especially ones that were particularly vulnerable. A world in which a diverse group of researchers and experts could all help contribute to asking the right questions, parsing through what the answers meant, and designing solutions. A world in which this sort of project wasn’t unusual at all.
For me personally, transparency isn’t about answering any single set of questions, especially of a set of systems that existed years ago. It’s an infinite game, not a finite one. We need ongoing, heterogeneous mechanisms to constantly be able to study, monitor and learn about the information ecosystems we live within…to both understand the platforms and also to understand ourselves. Both are changing constantly.
In other transparency news:
Over the course of this year, I worked with a bunch of folks from the Integrity Institute to put together a high-level overview of social media transparency. We hope it’s useful for lawmakers and regulators who are thinking about this topic.
TikTok released its first-ever Ad Library in Europe as a part of a wave of coming transparency requirements in the Digital Services Act. It’s far from perfect but it’s a huge step forward in transparency. However, as of this moment, it doesn’t seem like there will be any Brussels effect. In other words, here in the U.S., we’re going to keep falling behind until we pass something like PATA or DSOSA. You can read more here.
Google also launched its first-ever transparency center. It’s incredibly helpful to have all of their efforts together in one place and among other things, makes it clear how complex these programs are. However, that being said, it’s also a bit amazing that Google never had anything like this before and is only showing up now because of the coming wave of requirements out of the EU.
Earlier this year I became a Knight Fellow at George Washington’s Institute for Data, Democracy and Politics. I posted a bit about what I’ve been doing on my LinkedIn profile but one of the big reasons for joining is that I've been working closely with Rebekah Tromble for several years now on data access and I've been a huge admirer of the incredible work she's done in the space, especially in Europe. She's also building up a team of some really talented scholars and operators who all care about transparency, including Anna Lenhart, Jeff Allen, and Sahar Massachi.
In other news:
I recently discovered Daniel Danger and am totally hooked. Reminds me of the old days of frantically refreshing to try and land a Mondo poster (which I never successfully did sadly).
As a parent, I love the art of picking books to read with my kids (during the brief window when I have any influence over it course) and I discovered this list recently. My new parenting goal is to try and read as many of them as possible with my kids. Speaking of kid’s books, Aaron Becker is by far one of my favorite kid’s illustrators and authors. His Wordless Trilogy is stunning but I somehow just learned he had a new book (The Tree and the River) come out a few months ago.
Based on a summer of kid birthdays and kid-led karaoke, I’m convinced that 75% of Imagine Dragons fanbase are kids under 12.
Despite their hopes that they could avoid it, news and politics have made it onto Threads (which was always inevitable). It’s a good reminder that successful products are usually defined as much by their users as they are by any product manager or roadmap. And even more, when it comes to a real-time discussion app, nothing generates more new & exciting content than updates on what’s happening in the world. It’s very unclear to me if there’s actually a large market for real-time discussion about non-news (that’s substantially different from influencers commenting on their IG posts) but I guess we’ll see. Maybe it’ll eventually just be a sports app?
I have a lot of good friends who are in the clean energy and climate space and it’s really fascinating to try and follow all the amazing work that is happening. In that vein, Basis Climate is an awesome new marketplace for clean energy tax credits unlocked by the IRA and despite being only a few months old, they are already killing it. It’s true that it’s also co-founded by my brother but I swear, I’m totally objective.
Speaking of clean energy, this website is fascinating and I find myself just checking it randomly throughout the week.