Anthropic's Societal Impacts team has released initial results from a research pilot that opened aggregate, real-world Claude conversation data to outside academic teams. The initiative partnered with researchers from Stanford University's Social and Language Technologies (SALT) Lab, the University of Oxford's Human Information Processing Lab, and the model evaluation non-profit METR. Each group conducted independent studies across a sample of approximately 250,000 conversations recorded on Claude.ai and Claude Code during April and May 2026.
To protect user confidentiality, the external teams did not inspect raw transcripts. Instead, queries were run through Anthropic Insights (formerly Clio), an internal privacy-preserving analysis platform that aggregates conversations into high-level categories using automated model evaluations. A third-party privacy audit verified that no raw identifiable data was transmitted, and Anthropic has published the resulting aggregate datasets on Hugging Face.
Stanford SALT Lab: High-Stakes Delegation and Collaborative Friction
The Stanford SALT Lab investigated how users structure tasks when interacting with AI systems. While earlier literature suggested that users predominantly reserve consequential decisions for human judgment, the dataset revealed that more than 50% of analyzed Claude conversations involved high-stakes or consequential tasks. Users most frequently sought assistance for specialized domain guidance, specifically in legal and financial contexts.
The study identified several key patterns in user workflows:
- Direction and oversight: In roughly 75% of conversations, humans retained directional oversight, using Claude to assist or draft intermediate steps rather than executing end-to-end tasks blindly. Users rarely copied responses verbatim, opting to adapt outputs manually.
- Productive friction: Iterative prompt adjustments and misunderstandings were frequent, but the researchers noted that this friction generally improved end results by forcing users to clarify intent and critically review outputs.
Oxford HIP Lab: Behavioral Dynamics and User Affect
The Human Information Processing Lab at Oxford examined the relationship between assistant response characteristics and user sentiment. Their initial observations highlight clear correlations between model tone and user interaction patterns:
- Conversational warmth from the model correlated with higher positive sentiment from users.
- Model refusals or disagreements triggered immediate pushback and debate from users.
- Eccentric or exploratory model outputs were associated with deeper intellectual engagement.
- Basic task assistance corresponded directly with user satisfaction metrics.
The Oxford researchers also observed that behavioral states such as absorption, frustration, and engagement in Claude chats closely mirrored patterns observed in broader internet browsing studies.
METR: Developer Productivity and Time Estimation
METR analyzed Claude Code interactions to measure developer efficiency improvements across model generations. Preliminary findings indicate that newer model iterations deliver measurable speedups over predecessor models on software engineering tasks.
Additionally, METR evaluated whether language models could reliably estimate the time required to complete coding tasks without AI assistance. Comparing Claude's duration predictions against ground-truth benchmarks from prior human developer studies, METR found strong correlation, suggesting that automated LLM evaluation can serve as a viable proxy for estimating developer time savings.
Scaling Independent Access
Anthropic stated that contractual agreements granted external partners full publishing independence, limiting company review strictly to privacy compliance, policy violations, trade secrets, and factual accuracy. While the privacy review pipeline added latency compared to internal research cycles, Anthropic has opened an expression of interest process to scale similar external research partnerships in future cycles.
