Lisa Argyle

@lpargyle.bsky.social

Associate Professor of Political Science at Purdue. Study: Political Conversation, Polarization, Computational Social Science, LLMs

Ok, thread time. Closed-ended survey responses are efficient and easy to analyze, but limit what respondents can say. Open-ended responses are useful for letting respondents answer with more depth in their own words (as opposed to yours).

Jon Green@jongreen.bsky.social · 2y ago

Online/open access in @polanalysis.bsky.social w/ @willrhobbs.bsky.social: a theory and method for inferring attitudes in open-ended survey responses www.cambridge.org/core/journal...

Abstract: Past work on closed-ended survey responses demonstrates that inferring stable political attitudes requires separating signal from noise in “top of the head” answers to researchers’ questions. We outline a corresponding theory of the open-ended response, in which respondents make narrow, stand-in statements to convey more abstract, general attitudes. We then present a method designed to infer those attitudes. Our approach leverages co-variation with words used relatively frequently across respondents to infer what else they could have said without substantively changing what they meant—linking narrow themes to each other through associations with contextually prevalent words. This reflects the intuition that a respondent may use different specific statements at different points in time to convey similar meaning. We validate this approach using panel data in which respondents answer the same open-ended questions (concerning healthcare policy, most important problems, and evaluations of political parties) at multiple points in time, showing that our method’s output consistently exhibits higher within-subject correlations than hand-coding of narrow response categories, topic modeling, and large language model output. Finally, we show how large language models can be used to complement—but not, at present, substitute—our “implied word” method.

Recent study by Lisa Argyle, Chris Bail et al. showing that using an LLM to suggest rephrasings in conversations about divisive topics can improve perceived conversation quality and feelings of democratic reciprocity — without any apparent manipulation of political views. doi.org/10.1073/pnas...