Tessera
Folgen

Hamel Husain

@hamel.bsky.social

evals evals evals. https://evals.info

126 Posts6638 Follower655 Folgt
Hamel Husain@hamel.bsky.social · 9mo ago

👀 Animals have been assigned. Scheduled to print fall 2026! We have iterated on this with over 3k students (and continue to do so). We give our students access to the full draft as part of our evals course (link in bio)

Bild
Hamel Husain@hamel.bsky.social · 10mo ago

I recently sat down with Lenny Rachitsky to discuss why AI Evals are becoming the most sought after skill for product builders. As a bonus, we step through an end-to-end example of building an eval in a spreadsheet so everyone can understand. See reply for links.

Hamel Husain@hamel.bsky.social · 11mo ago

This one is going to be spicy. 80% of the time I've seen a graph DB in production, it's been an overcomplicated mess (especially in AI applications). In this talk, Jo and I will discuss when GraphDBs are overkill and when they actually make sense. Sign up here:

You Don't Need a Graph DB

Many teams adopt graph databases believing they need specialized tools for relationship data, adding unnecessary complexity to their stack. This session reveals that for most use cases, the…

maven.com

Kyle Baxter@kbaxter.bsky.social · 11mo ago

For technical domains especially, getting non-DSes involved in analyzing outputs is vital. It’s hard to build anything good without it bc v1s almost always have major fail modes. Finding the appropriate system design—let alone optimizing—requires a tight coupling of output analysis and system design

Hamel Husain@hamel.bsky.social · last yr.

Can non-data scientists write AI Evals? The answer is nuanced and not just "Yes". @eugeneyan.com and I discuss this in the context of the "analyze-measure-improve" cycle from our course. Links to more resources in the reply

Kyle@kylestratis.com · last yr.

As genAI projects mature, proper evals are becoming table stakes for production deployment. But how do we evaluate probabilistic machines? Looking forward to learning about the latest techniques and best practices from @hamel.bsky.social and Shreya Shankar next month!

AI Evals For Engineers & PMs by Hamel Husain and Shreya Shankar on Maven

Learn proven approaches for quickly improving AI applications. Build AI that works better than the competition, regardless of the use-case.

maven.com

Eugene Yan@eugeneyan.com · last yr.

@hamel.bsky.social & his wisdom on evals, error analysis, looking at your data is what we need. Here are his 10 Don'ts: • Don't skip error analysis • Don't skip looking at your data • Don't gatekeep who can write prompts • Don't let zero users be a roadblock • Don't be blindsided by criteria drift

Hamel Husain@hamel.bsky.social · last yr.

The most critical part of RAG is the R (Retrieval). In this lesson, Doug Turnbull will share how we can go beyond simple hybrid search to optimize retrieval. He'll share his bag of tricks from over a decade of optimizing search systems. maven.com/p/29a33a/hyb...

Hybrid Search Is Just The Beginning: Optimizing the R in RAG

You may have implemented hybrid search, and that's a great first step. In this session, Doug will share his experience building advanced search systems at Shopify and Reddit to provide you with a clea...

maven.com

Hamel Husain@hamel.bsky.social · last yr.

The most critical part of RAG is the R (Retrieval). In this lesson, Doug Turnbull will share how we can go beyond simple hybrid search to optimize retrieval. He'll share his bag of tricks from over a decade of optimizing search systems. maven.com/p/29a33a/hyb...

Hybrid Search Is Just The Beginning: Optimizing the R in RAG

You may have implemented hybrid search, and that's a great first step. In this session, Doug will share his experience building advanced search systems at Shopify and Reddit to provide you with a clea...

maven.com