Leading experts in Rice University’s School of Social Sciences are exploring pioneering methods to push the boundaries of research. This series features Rice social scientists who are utilizing new and innovative uses of artificial intelligence (AI) to enhance scholarship. Today’s featured faculty is Erik Peterson, associate professor of political science.
Completing his fourth year at Rice University, political scientist Erik Peterson studies local media outlets and messaging in American politics, with a specific interest in how they relate to local governments. Peterson seeks to understand the ways that politics are affected by the decline of local news mediums, such as newspapers and television, and by people’s shifting news habits to digital sources.
“I'm interested in how this shift in news sources connects to people’s opinions on politics and how they decide between different candidates,” said Peterson. “I also seek to understand the impact that preferred news sources have on various political outcomes.”
In recent years, transcripts of local government meetings, such as school board meetings and city council meetings, have become increasingly available in digital formats to fulfill public records requirements. Many local governments choose to meet these requirements by posting videos of meetings online. Peterson is incorporating AI into his research by utilizing large language models (LLMs) to evaluate these sizable corpuses of text.
“Within these transcripts, people talk over each other, and the meetings can last for many hours. It is a laborious effort to structure them,” said Peterson. “Now, using LLMs, we're able to observe at a very granular level what people are talking about in these local governance arenas, and we can explore different ways to measure local government performance and people's engagement with their local governments.”
Peterson and his fellow researchers first attempted to use LLMs for this purpose two years ago, but found the LLMs could not reliably and systematically extract data from the meetings or measure sentiment in the way the team had hoped. However, Peterson observed a recent shift in LLM performance.
“The LLMs have really turned a corner, at least for this application,” said Peterson. “As someone who is interested in local media and local government, integrating this tool with data sources that have been incredibly labor intensive to retrieve and difficult to use in any sort of systematic way has made the whole process much more feasible to incorporate into research.”
Many researchers use AI to verify their methods; Peterson is doing the opposite.
“We have been using human validation to verify the use of AI,” said Peterson. “One of our graduate students who co-authored the project, Maranda Joyce, watched hours of school board meetings after the LLMs classified them to verify that they had produced accurate assessments.”
Peterson noted that using LLMs allows for new, more generalizable research that can incorporate multiple studies or longer time periods, instead of a single case study.
“Often in the social sciences, a theory might be held up, or it might be difficult to push it forward, because we don't have the means to systematically measure the concepts or the variables we want or to study subjects in different settings,” said Peterson. “With LLMs, we can take multiple individual, labor intensive studies – that were well done but constrained due to limited time and resources – and hopefully scale them up to understand important questions about local media and local politics.”
In addition to conducting research, Peterson teaches courses on American politics, with a focus on research methodology. He noted the challenges he has encountered with incorporating AI into teaching.
“Part of the writing process that is most valuable is learning how to think in a clear fashion and iterate on your own ideas,” said Peterson. “In the fall, I plan to incorporate AI as a tool for specific parts of the class but also indicate that it's not advised for use in other areas.”
AI tools have social implications that need to be understood, Peterson noted. As computer scientists are often designing these tools, social scientists can address how the tools are used and how they will alter or complement the way people access and process information.
“Social scientists often help to address the various trade-offs that exist in these domains,” said Peterson. “There are many ways we can attempt to understand the evolution of AI in light of past changes, and we can also think about the implications in a new light and try to understand what's changing as the system evolves.”