Notes for week 2 of my course on AI Agents for Managers are up! This week we talked about Retrieval-Augmented Generation and where it fits into businesses. augmentedreturns.substack.com/p/ai-agents-... @uclaanderson.bsky.social

AI Agents for Managers (2): Putting agents into context, and context into agents.

Course notes for week 2 of my MBA course

augmentedreturns.substack.com

🚨 New Working Paper! 🚨 Time to talk about "The Household Impact of GenAI", covered by the @WSJ ! In this paper, we ask what GenAI is doing to productivity outside of work - and find evidence of large impacts. arxiv.org/pdf/2603.03144 w/ Miao Ben Zhang (USC) & Michael Blank (Stanford) 🧵(1/n)

More on our conference encouraging AI writing and also selecting papers using AI: augmentedreturns.substack.com/p/launching-...

What happens when we encourage AI use for research and also use it to review papers? We are running an experiment to find out: the UCLA Human × AI Finance conference! Write a finance paper with AI in 4 weeks (by 3/18). AI agents review the submissions: humanxaifinance.org 🧵 1/

Human × AI Finance

Write a finance paper with AI. Get reviewed by AI. Top four presented at the Fink Center Conference at UCLA Anderson, April 24, 2026.

humanxaifinance.org

Vibe-coding’s complement is unit testing. If the writing of code is commodified, the validation of output becomes scarce.

Randomized trial AI for legal work finds Reasoning models are a big deal: Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28% There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level

🚨 New working paper with Caitlin Gorback! We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵

A new generation of AIs: Claude 3.7 and Grok 3 https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37 #AI #Claude #Grok

A professional workflow for translation using LLMs - Simon Willison. https://simonwillison.net/2025/Feb/2/workflow-for-translation/

🧪 A @ai.nejm.org study found that GPT-4 translation of discharge notes significantly improved patient understanding, especially for Black, Hispanic, and older patients. AI-driven readability could help close health literacy gaps. 🩺💻 tagging: @downing.bsky.social @thelizarmy.bsky.social

A Cross-Sectional Study of GPT-4–Based Plain Language Translation of Clinical Notes to Improve Patient Comprehension of Disease Course and Management

The 21st Century Cures Act provides patients with access to their clinical notes, but most patients, particularly those with limited health literacy, have difficulty understanding and utilizing the...

ai.nejm.org

I am worried LLM researchers sometimes bury the lede with regard to "should we trust these systems". Framing below is: LLMs are failing to "earn human trust". But it turns out it's the humans who cannot be trusted - even seeing the LLM's answer, the humans do worse than the LLM!

I think one way of describing these results to our future LLM overlords is “inference time compute scaling leads to thinking more but not to thinking better - in humans”

todd gureckis@toddgureckis.bsky.social · 2y ago

Our paper on if you can incentivize rule induction in humans with money is finally out (answer is: it appears to be a very weak/0-ish effect in contrast to the huge effect of financial incentives on rote, repetitive tasks). credit to pamop, ben newell & dan bartels psycnet.apa.org/fulltext/202...

Generative AI has flaws and biases, and there is a tendency for academics to fix on that (85% of equity LLM papers focus on harms)… …yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that

Let me try to formalize some thoughts about Gen AI adoption that I have had, which I will call "The Bedazzlement Curve". Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".

Reporting on AI adoption rates tends to show the importance of having priors.

In many examples of people actually implementing Gen. AI-based workflows, building the automation requires experience with the task at hand - suggesting that there might be upskilling / demand for experienced workers in those areas at least in the short-medium term, rather than simple “replacement”

Qian Li@qianli.dev · 2y ago

Consider this: an agent handling refunds might have to run a complex workflow. It first records in the DB that a refund is pending, then sends an email to an admin for verification, then uses Stripe to process the refund, then records the refund’s success in the DB, then sends another email.

The new Deep Research feature from Google feels like one of the most appropriately "Google-y" uses of AI to date, and is quite impressive. I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.

Was very surprised to stumble across a graph from my own research in a presentation by Benedict Evans today! He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2

It's incredibly encouraging that even models for analytical purposes, like o1, can recite Shakespeare. This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.

I had to tell a Ph.D. student today what o1 is. "The future is already here, it's just not evenly distributed"

I found this Stratechery framework for how technologies evolve by the interplay of hardware, input modes, and applications quite thought-provoking - with innovations in one triggering innovations in the other dimensions. Science seems to progress in similar ways. stratechery.com/2024/the-gen...

The Gen AI Bridge to the Future

Generative AI is the bridge to the next computing paradigm of wearables, just like the Internet bridged the gap from PCs to smartphones.

stratechery.com

I think firms worrying about AI hallucination should consider some questions: 1) How vital is 100% accuracy on a task? 2) How accurate is AI? 3) How accurate is the human who would do it? 4) How do you know 2 & 3? 5) How do you deal with the fact that humans are not 100%? Not all tasks are the same.

how do researchers use LMs in their work & why? we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on: 🐟 which research activities (eg coding, writing) 🐠 benefits vs risks 🦈 willingness to disclose findings in @simonaliao.bsky.social's thread 🧵

Simona Liao@simonaliao.bsky.social · 2y ago

Hi everyone, I am excited to share our large-scale survey study with 800+ researchers, which reveals researchers’ usage and perceptions of LLMs as research tools, and how the usage and perceptions differ based on demographics. See results in comments! 🔗 Arxiv link: arxiv.org/abs/2411.05025

How can managers identify GenAI use cases? I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own! I called it the "BEAST" framework for finding LLM use cases - see details below.

Happy ChatGPT Day to those who celebrate it! This birthday AI is being very demure, very mindful, even on its special day.

The batch API pricing for major LLMs seems like great price discrimination in favor of academic use cases and I am not complaining.

Does Generative AI upskill or downskill the jobs that are affected? One perspective (not necessarily the correct one!) is that of the "paradox of automation" where workers need MORE training and specialized skills as those tasks that cannot be automated are the specialized ones...

One good use case of LLMs for research that I have found: rapidly going deeper on existing literature reviews to find the papers most relevant to me. Steps: 1. Paste in lit review 2. Ask for web search on all the papers, to get titles and abstracts => Saves me lots of separate searches

The actual usage pattern of Generative AI look very different from a simple "automation = bad for workers" perspective: firms often use Gen. AI to ENHANCE worker capabilities - we should be interested in how this leads to a restructuring of workplaces and the assignment of different tasks to jobs.

This is a fascinating study, and speaks to the vast perception gap between what LLMs are capable of and what people think they are capable of. Doctors using ChatGPT for diagnosis performed *worse* than ChatGPT alone, because they kept second-guessing the model. www.nytimes.com/2024/11/17/h...

A.I. Chatbots Defeated Doctors at Diagnosing Illness

A small study found ChatGPT outdid human physicians when assessing medical case histories, even when those doctors were using a chatbot.

nytimes.com