Leveraging 30,000 stock predictors and AI to generate 288 finance papers with custom hypotheses, highlighting risks of automated research and HARKing (Hypothesizing After Results are Known) in academia, from Robert Novy-Marx and @VelikovMihail https://www.nber.org/papers/w33363

Very excited to release a major revision to our paper on algorithmic collusion by large language models. #EconSky

Yannai A. Gonczarowski@yannaigonch.bsky.social · 2y ago

🚨Major new version🚨 Algorithmic Collusion by Large Language Models Joint w/ @sarafish.bsky.social & @ranshorrer.bsky.social LLMs are automating many business decisions. Pricing might be next (or is already). What if multiple firms, in good faith, to use off-the-shelf-LLMs for pricing? 1/3 #EconSky

Researchers used AI to generate 288 complete academic finance papers predicting stock returns, complete with plausible theoretical frameworks & citations. Each paper looks and reads as legit. They did this to show how easy it now is to mass produce "credible" research. Academia isn't ready.

Really cool work that raises questions about how we think about science and progress. The push towards pre registration has benefits but foregoing HARKing has many costs too. The optimal balance is not obvious!

Mihail Velikov@mvelikov.bsky.social · 2y ago

An academic paper has excellent empirical evidence & hypotheses that perfectly match the patterns in the data. One catch: AI wrote the hypotheses after seeing the results. Should this matter? New paper w/ Robert Novy-Marx on AI-Powered (Finance) Scholarship🧵 papers.ssrn.com/sol3/papers....

Cool results that raise interesting questions about a swath of asset pricing papers. Also really like the assaying framework they use.

Mihail Velikov@mvelikov.bsky.social · 2y ago

An academic paper has excellent empirical evidence & hypotheses that perfectly match the patterns in the data. One catch: AI wrote the hypotheses after seeing the results. Should this matter? New paper w/ Robert Novy-Marx on AI-Powered (Finance) Scholarship🧵 papers.ssrn.com/sol3/papers....

An academic paper has excellent empirical evidence & hypotheses that perfectly match the patterns in the data. One catch: AI wrote the hypotheses after seeing the results. Should this matter? New paper w/ Robert Novy-Marx on AI-Powered (Finance) Scholarship🧵 papers.ssrn.com/sol3/papers....