Matthew Kenney
@baykenney.bsky.social
Founder - Algorithmic Research Group. Previously: Senior ML Engineer at Apple, Asst. Research Prof at Duke University, Duke Data Science | PSU '15 | Cornell '11
There's rightly a lot of excitement around Karpathy's autoresearch. We've been studying at ARG for a couple years now: what happens when you put an AI agent in a loop and let it run experiments, evaluate results, and iterate without you. We've built a bunch of benchmarks and tools to measure this. 🧵
Over the past ~2 years I’ve been working hard on agents, models, and datasets to understand what recursive self-improvement might look like, and what path supporting infrastructure for this line of research might take. Very excited to open source some of that work starting today
Very excited to launch this little tool that we’ve been building. ScoutML is an API built for AI researchers and agents that includes a ton of metadata on each paper. It’s been super helpful for us as we run our research agents internally. x.com/algoresearch...
Algorithmic Research Group on X: "At ARG, we're laser-focused on understanding recursive self-improvement. We're confident that as models scale, RSI will accelerate the frontier of AI at ever-increasing speeds. Over the past year, we've created benchmarks, agents, and AI systems to measure how this might happen. https://t.co/JyOPFSB8DJ" / X
At ARG, we're laser-focused on understanding recursive self-improvement. We're confident that as models scale, RSI will accelerate the frontier of AI at ever-increasing speeds. Over the past year, we've created benchmarks, agents, and AI systems to measure how this might happen. https://t.co/JyOPFSB8DJ
x.com
A lot of ML tools help you implement. Not many help you think. When I’m exploring a new research direction, I don’t want another search engine or citation graph. I want something that’s actually read the literature, can suggest promising directions, and helps me reason through tradeoffs.
ARG is on Bluesky! Please follow here: @algoresearch.bsky.social
good post on 2025 ai safety research directions: alignment.anthropic.com/2025/recomme...
Recommendations for Technical AI Safety Research Directions
alignment.anthropic.com
Back in Pennsylvania, drinking schuylkill county coal cracker (boilo) and making pierogies
AI for science could be more impactful than chatbots. It is already helping win Nobel prizes and accelerating drug development and materials discovery. Today we published an essay about it: why it matters, how it’s happening and its implications. Here is a summary from an econ / social sci lens.
Important point that the open protocol makes extracting data from bluesky easy. Can't have it both ways. I like the protocol and think this site is well designed, but that means anyone can and will analyze these posts (if there is value to them, which I'm honestly less convinced of than some)
Stella Biderman@stellaathena.bsky.social · 2y agoOpenAI doesn't want this dataset. Any company that wants Bluesky data will press a button and go get all posts on Bluesky. The very design of the website makes that trivial to do. This doesn't help OpenAI *at all*.
Jeremy Howard @howard.fm · 2y agoAn AI researcher that wants to stop big tech owning everything was permabanned here for releasing a dataset of 2M posts. A librarian received death threats for a 1M post dataset. The EU funded the creation of this dataset of 235M posts months ago, and… nothing? zenodo.org/records/1108...
A dataset of 1 million or 2 million Bluesky posts is completely irrelevant to training large language models. The primary usecase for the datasets that people are losing their shit over isn't ChatGPT, it's social science research and developing systems that improve Bluesky.
Jeremy Howard @howard.fm · 2y agoDid you know that 99% of email today is spam? Your inbox isn’t 99% spam because AI is used to filter it. The same 99% will happen here too, but if AI researchers continue to get perma-banned for making available the datasets needed to filter it, it’s going to make this platform unusable.
Wait what even is this platform. This is insane
Jeremy Howard @howard.fm · 2y agoAn AI researcher that wants to stop big tech owning everything was permabanned here for releasing a dataset of 2M posts. A librarian received death threats for a 1M post dataset. The EU funded the creation of this dataset of 235M posts months ago, and… nothing? zenodo.org/records/1108...
(re-posting from X) Can we get AI to accelerate AI research and development? I’m excited to release ML Research Benchmark, an agentic benchmark of 7 ML conference competition tasks. Paper: arxiv.org/abs/2410.22553 Tasks: github.com/AlgorithmicR... Agent: github.com/AlgorithmicR...
ML Research Benchmark
Artificial intelligence agents are increasingly capable of performing complex tasks across various domains. As these agents advance, there is a growing need to accurately measure and benchmark their c...
arxiv.org
I somehow ended up following a ton of OCaml people and now I’m like I should probably learn OCaml
I've created an initial Grumpy Machine Learners starter park. If you think you're grumpy and you "do machine learning", nominate yourself. If you're on the list, but don't think you are grumpy, then take a look in the mirror. go.bsky.app/6ddpivr