ECHO: Extracting Community Insights for Policymaking

Abstract

This study analyzes over 6.5 million posts and 56 million comments from relevant Reddit communities to surface worker experiences across various industries since the advent of ChatGPT. We employ a novel LLM-assisted methodology to efficiently capture diverse stakeholder perspectives on AI's impact on work. Our findings reveal that AI's effect on employment is more nuanced than simple job displacement, with evidence of workers adapting to AI-driven changes by leveraging AI tools or transitioning to new roles.

Methodology

We collected data from 20 subreddits across three categories: creatives, professionals, and educators.

We used LLM-assisted thematic analysis to extract and categorize AI-related work anecdotes.

We combined expert analysis with LLM-assisted insights to identify key themes and select representative quotes.

Dataset

Dataset period: December 2022 to July 2024
Total posts: Over 6.5 million
Total comments: Over 56 million

Industry	Stakeholders	Subreddits
Creatives	Writers (freelancers, screenwriting, creative writers, poets, journalists), Musicians, Artists, Actors	r/freelanceWriters/, r/screenwriting, r/creativewriting, r/Poetry, r/Writers, r/Writing, r/Journalism, r/Music, r/Musicians, r/ArtistLounge, r/VoiceActing
Professionals	Lawyers, Doctors, Nurses, Software Engineers	r/Ask_Lawyers, r/Paralegal, r/Nursing, r/Medicine, r/SoftwareEngineering, r/SoftwareDevelopment, r/DevelopersIndia
Educators	Teachers	r/Teachers/, r/Education

Labor Risks Anecdotes

Job Displacement: Anecdotes discussing people losing work or being laid off due to AI tools.
Career Transitions: Anecdotes about people adapting their careers in response to AI, including reskilling or changing roles.
AI-enhanced Work: Anecdotes where people are adopting AI tools to enhance productivity and streamline their workflows.

Industry	Job Displacement (1)	Career Transitions (2)	AI-enhanced Work (3)	Total
Creatives	773	119	330	1222
Professionals	589	389	482	1460
Educators	63	60	332	455
Overall	1425	568	1144	3137

We also have anecdotes from other systemic risks, including global AI divide, market concentration risks, environmental risks, privacy risks, and copyright infringement. Our dataset includes opinions and media reports, but we focus on anecdotes for this consultation.

Results

We used this methodology to submit a response to the EU AI Office's consultation on the first Code of Practice for providers of General-Purpose AI (GPAI) systems in September 2024. This consultation aims to establish rules for GPAI providers to address potential systemic risks. Our submission focused on highlighting labor market risks, an aspect not explicitly covered in the original consultation form.

You can view our full submission to the GPAI consultation here: Original GPAI Submission Document

Our analysis resulted in spreadsheets containing the following columns:

subreddit_name: The name of the subreddit where the post was found
key: A unique identifier for the post
quote: The relevant excerpt from the post
summary: A summary that captures context from other parts of the post, including post text, surrounding comments, and replies
category: The category of the anecdote (1, 2, or 3)

To recreate the URL of the original post: reddit.com/r/subreddit_name/comments/key (unless deleted)

Interactive Spreadsheets

Limitations

Author Positionality: Our background in computer science and public policy studying AI's societal impacts has influenced the thematic analysis.
Selection Bias: The use of Reddit and specific subreddits may not fully represent all worker demographics or experiences.
LLM Errors: The model may have made errors in data extraction and risk classification.
Geographical Limitations: The data may be skewed towards certain regions or countries.
Potential Misinformation: Some anecdotes may contain inaccurate information or exaggerations.

Authors

Name	Affiliation	Email
Varun Rao	Princeton University	varunrao@princeton.edu
Kyler Zhou	Princeton University	kyler@princeton.edu
Sayash Kapoor	Princeton University	sayashk@princeton.edu
Arvind Narayanan	Princeton University	arvindn@cs.princeton.edu

Acknowledgements

We thank Reddit user u/Watchful1 for the Subreddit data and James Mellody for pointing us to the relevant source. We also acknowledge the feedback of Connor Dunlop and Lara Groves from the Ada Lovelace Institute.

Source Code

The source code for this project is available on GitHub