Manon Revel

tl;dr: We find that some reward models used for making AI models safer resist learnnig a significant portion of their training split in that they give higher rewards to rejected entries in a number of pairs in the alignment dataset. In other words, they go against human preferences. Oh, and an AI-labeler agrees with the reward model to disagree with human labels.

Reinforcement Learning from Human Feedback (RLHF) aims to align language models (LMs) with human values by training reward models (RMs) on binary preferences and using these RMs to fine-tune the base LMs. Despite its importance, the internal mechanisms of RLHF remain poorly understood. SEAL: Systematic Error Analysis for Value Alignment introduces new metrics to evaluate the effectiveness of modeling and aligning human values, namely feature imprint, alignment resistance, and alignment robustness.

We test our method with widely used open-source components (the Anthropic/hh-rlhf preference dataset and OpenAssistant's RMs). The experiments reveal significant imprints of target features and a notable sensitivity to spoiler features. We observed a 26% incidence of alignment resistance (where RMs fail to match human preferences on the training split). These findings underscore the importance of scrutinizing both RMs and alignment datasets for a deeper understanding of value alignment.

Accepted at AAAI-25     Joint work with Matteo Cargnelutti, Tyna Eloundou, and Greg Leppert

tl;dr: Democratic institutions are old. Let's innovate (with new theories!)

Our collective imagination has been captured by the mirage that democracy equals election. Representative democracy (as a collective decision-making process) happens in stratified layers: the filtering of candidates, the selection of representatives, the sense-making and deliberation happening amongst the representatives, and the decision time. If democracy is characterized by the inclusion and equal treatment of all group members, election concentrates these democratic qualities at the selection stage—one-person-one-vote does not guarantee that the filtering, deliberation, or decision-making happen democratically. Alternative models of democracy propose to repurpose democratic institutions aimed at a holistic account of equality and inclusion in the decision-making pipeline. Most prominently, scholars have discussed re-introducing democracy by lot—whereby randomly selected citizens deliberate to reach highly consensual decisions.

This system (called lottocracy, or sortition) was famously in place in the Greek city-state of Athens that accommodated up to a thousand randomly selected officials and 30,000 over-aged citizens. In modern societies, my chances of being selected in my lifetime may be less than 5%: should I feel more included as an episodical voter or as a hypothetical decision-maker? In How to Open Democratic Representation to the Future? I argue that representation in democracy is due for an upgrade: we need innovative representative mechanisms as well as renewed democratic theories that account for the novel societal and technological conditions under which we live. If there is no such thing as an ultimate form of representation, there is no such thing as a static democracy.

Presented at the Harvard International Reimagining Democracy Workshop, Workshop on Long Term Risks and Future Generations, Harvard University's Ash Center for Democratic Innovation

Social media platforms mediate a significant fraction of human communication and attention. The impact of social media on society has been under increased scrutiny, and concerns over its effects have motivated varied and sometimes contradictory government regulation around the world. In this review article, we offer two ways of mapping the space of social media regulation: viewing social media either (i) as an architecture impacted by design choices, or (ii) as a market governed by incentives. We survey the most prominent regulatory approaches globally (both enacted and proposed), with an emphasis on the United States and the European Union, and position these options within the two maps. We conclude by discussing the fundamental trade-offs associated with different interventions, comparing jurisdictions, and highlighting paths forward in the context of the potentially-conflicting rights and interests of relevant stakeholders.

Covered in Tech Policy Press with Unregulated Ways to Regulate Social Media

Joint work with Nate Lubin, Kalie Mayberry, Dylan Moses, Luke Thorburn and Andrew West

The digitization of news publishing has resulted in new ways for advertisers to reach readers, including additional native advertising formats that blend in with news. However, native ads may redirect attention off-site and affect the readers' impression of the publishers. Using a combination of observations of ad content across many publishers and two large randomized experiments, we investigate the characteristics of a pervasive native ad format and compare the impact of different native ads characteristics on perceived news credibility. Analyzing 1.4 million collected ad headlines, we found that over 80% of these ad headlines use a clickbait-style and that politics is among the most common topics in ads (The effects of native advertisement on the US news industry).

In two randomized experiments (combined n=9,807), we varied the style and content of native ads embedded in news articles and asked people to assess the articles’ credibility (Native advertising and the credibility of online publishers.). Experiment 1 (n=4,767) suggested that different publishers were impacted differently by the ads and motivated the more detailed design of Experiment 2 (n=5,040). This latter experiment used hundreds of unique combinations of ads, articles, and publishers to study effects of clickbait and political ads. Findings from this pre-registered experiment provide evidence that clickbait and, to a lesser extent, political ads, substantially reduce readers' perception of the articles' credibility: publishers using clickbait native ads may trade short-term revenues for audience trust.

Presented at the International Conference on Computational Social Science (IC2S2), MIT Schwarzman College of Computing Launch, and Technology, Management, and Policy (TMP) Consortium

Covered in MIT News: Understanding how people make sense of information in the information age

Joint work with Adam Berinsky, Dean Eckles, Ali Jadbabaie and Amir Tohidi

The resonance of media output plays an important role in the age of misinformation and fake news. While scholars have extensively studied resonance, they have mostly focused on whether and why particular messages align with the predispositions of their intended audience rather than systematically analyzing how they are interpreted by the wider population. Based on a computational text analysis of the media output from more than a hundred different outlets in France and weekly surveys of what people have retained from the news during the same period, this paper investigates the ways in which media coverage trigger different types of resonance in accordance with people’s diverse interpretations and utilizations of the messages to which they have been exposed (Varieties of resonance: The subjective interpretations and utilizations of media output in France). We theoretically argue that resonance is not just an objective alignment between a message and one’s predispositions, but also a subjective interpretation and utilization of the message heard. We empirically identify three different types of subjective resonance: one used for problem-solving, one that is problem-aggravating, and another one that is problem-generating. This research contributes to a better understanding of the mechanisms of resonance by expanding on previous works on the cognitive, emotional, and interactional dimensions of resonance.

Presented at the American Sociological Association (ASA) Communication, Information Technologies, and Media Sociology (CITAMS) by Bo Yun Park

Joint work with Adrien Abecassis and Bo Yun Park

Individuals with tinnitus are highly heterogeneous in terms of etiology, the manifestation of symptoms, and the way they manage their condition. Most of these patients are likely to seek hearing health information and social support online via various websites or social media platforms. Indeed, information is easily accessible online. Further, in absence of evidence-based tinnitus care, patients with similar symptoms can regroup, share experiences, and exchange tips. Even after consultation with healthcare providers, some of the patients continue seeking information online when they feel they did not get satisfying information about treatment options and/or about their prognosis. The present study was aimed at examining the discussions around tinnitus in Reddit posts from 12,000 users over 8 years, using various Natural Language Processing (NLP) techniques ( Online discussions about tinnitus: What can we learn from natural language processing of Reddit posts? ). We examined the free-texts posts to understand the types of conversation about tinnitus in an online forum and the way in which people with tinnitus reach out to other people for support (informational, emotional, etc.) when coping with their conditions. We hope that this can provide insights, complementary to those collected in the clinical environment, to reflect on new ways to support tinnitus patients.

Work presented at the Virtual Conference on Computational Audiology (Best video pitch awards)

Covered in Alter Ago Le Mag, Manon Revel: En deça et au-delà des algorithmes

Joint work with Ryan Boyd, Aniruddha Deshpande, Alain Londero, Vinaya Manchaiah, Guillaume Palacios and Pierre Ratinaud

Alternative realities are troubling democracy. Eager to discuss these issues beyond academia, I have explored avenues to discuss the risks associated with an impoverished debate space. In Can We Talk? An Argument for More Dialogue in Academia, I explain why I worry about the lack of critical thinking and open-mindedness in academic environments.

Organized and moderated the panel Data weaponized, data scrutinized: a war on information at the Women In Data Science Conference

Wrote Internet, you lie! (in French) for the French parliamentary journal L'Hémicycle

Organised a joint workshop with Zivvy Epstein and Maurice Jakesch