March Voice AI Meetup - Wednesday the 5th lu.ma/ffpyl57n

Gemini 2.0 Flash is competitive with GPT-4o on: - TTFT, - instruction following, - function calling, and - natural conversation dynamics. GPT-4o was ahead on all of these attributes by a wide enough margin that using any other LLM for voice AI mostly didn't make sense. Now there's competition!

Memory for voice AI agents (and composite function calling) ... There are several ways to store (and later, retrieve) conversation state. One of the simplest is just to define a couple of functions and use your local filesystem! Here, @chadbailey.net shows how to do that, using Gemini 2.0 Flash.

Sean DuBois is one of my favorite people to talk to about WebRTC, audio and video, designing good libraries, and hacking in general. Sean is the creator of Pion. Pion is an Open Source WebRTC implementation that is influential and very widely used (including at OpenAI, where Sean works).

My favorite part of the DeepSeek-V3 Technical Report is the stuff about the all-to-all communication kernels. (Mostly in section 3.2.2. "Efficient Implementation of Cross-Node All-to-All Communication.")

Using Gemini search metadata in a voice AI application Filipi added support in Pipecat for Google Gemini's `groundingMetadata`. This makes it easy to do things like: - link to URLs - log searches for observability - use specific search result chunks for RAG youtu.be/oL9w-3Hbag0

Google Gemini search metadata

YouTube video by Daily

youtu.be

Voice AI programming with Gemini² and Cursor Adrian built a Gemini voice + vision AI agent that writes software indirectly, collaborating with a human and with Gemini running inside Cursor. Really nice glimpse of the future (and nice example of a "multi-agent" architecture). youtu.be/0VFZWZfU0vw

Pipecat 0.0.53 release is out today. 31 entries in the Changelog, including: 🌓 Frame observers — for implementing loggers, debuggers, and pipeline tools 🌓 Heartbeat frames — pipeline traversal timing and warnings if system frames get blocked anywhere in the pipeline github.com/pipecat-ai/p...

Search is a built-in tool in the Gemini Multimodal Live API. Here's an iOS starter project that shows: - how to to use the Gemini search built-in tool - combining the built-in search with custom functions Here's the code: github.com/pipecat-ai/p... youtube.com/shorts/7jX7l...

Gemini Multimodal Live search tool + iOS native app

YouTube video by Daily

youtube.com

Maslow's hierarchy of voice AI u are here ⤵️ ◻️◻️◻️◻️🟦◻️◻️◻️◻️ ◻️◻️◻️🟦🟦🟦◻️◻️◻️ ◻️◻️🟦🟦🟦🟦🟦◻️◻️ ◻️🟦🟦🟦🟦🟦🟦🟦◻️ 🟦🟦🟦🟦🟦🟦🟦🟦🟦 Network transport ▶️ Turn detection ▶️ Interruption handling ▶️ Natural voices ▶️ Tool use www.youtube.com/watch?v=tAQW...

Pipecat Flows - open source Voice AI agent builder

YouTube video by Daily

youtube.com

"Voice AI in 2025" panel recording from the Voice AI Meetup last night. Thank you to panelists Karan Goel, Niamh Gavin, Shrestha Basu Mallick, and Swyx. And thank you to Chroma for hosting the meetup in their fantastic office in SF. www.youtube.com/live/B6zTwHh...

Christian built a voice AI assistant to control Spotify. Tech: - Google Gemini 2.0 - Pipecat - Deepgram - Cartesia Code is here: github.com/pipecat-ai/s... youtu.be/q6v-3BQem3Y

Gemini + Pipecat demo — Spotify Voice AI Assistant

YouTube video by Daily

youtu.be

Gemini Multimodal Live API + iOS + WebRTC Nice walk-through from Paul: youtu.be/nU3K8h_pkeQ 📣 set up a voice client in your iOS app 📶 specify WebSockets or WebRTC for network transport 🏓 attach a delegate to handle lifecycle events (for example "connected", "LLM ready")

Sunday morning listening ... and hacking.

Today's reminder of how early we are in the generative AI/deep learning technology transition: moved a moderately complex prompt to a different LLM and 150% of my evals broke. 150% because evals I didn't even have (but, obviously, needed) broke, too.

They know what they’re doing over there in Cupertino (and Shenzhen).

iOS + Gemini Multimodal Live + WebRTC Filipi Fuchter added an iOS example to the Pipecat "Simple Chatbot" repo. With the Pipecat iOS SDK, you can build apps that use Gemini Multimodal Live and Gemini Flash with WebRTC, WebSockets, and HTTP networking.

I had a lot of fun talking to Eric Landau about the state of Voice AI at the end of 2024, what's coming in 2025, what the pain points are today if you're scaling voice AI agents in production, and — of course — the importance of data tooling and evals. open.spotify.com/episode/5Fjj...

Kwin Kramer | Building the Future of Real-Time AI with Daily and PipeCat: Insights on Multimodal Systems and Developer Tools

Deep Learning Leaders · Episode

open.spotify.com

The first Voice AI Meetup of the new year is next week. If you'll be in the San Francisco area on Tuesday evening, come join the fun. It's been great to see the voice/conversational AI community grow over the past year. lu.ma/it312noz

Voice AI Meetup — Voice AI in 2025 · Luma

Join us for the first Voice AI Meetup of the year, hosted by Daily and Chroma. As always, there will be food, great conversations, and a few demos. The main…

lu.ma

Which LLM should you use for your voice agents? The team at Coval does a lot of interesting work with synthetic data and evaluations for voice AI agents. I've been working with them on evals for Gemini 2.0. They wrote up some of their results so far, here: www.coval.dev/blog/scripte...

Scripted Evaluation Framework for Large Language Models: A Controlled Approach to Comparative Analysis - My Framer Site

Coval is a simulation & evaluation platform for voice and chat agents. Start your free trial today!

coval.dev

all you need is beam search

The voice-to-voice AI Pareto frontier ... Gemini 1.5 Flash occupies an interesting place in the capabilities matrix for voice AI. It's fast, very inexpensive, has a long context window, and has native audio input. I've been experimenting with Gemini a lot. Here's an interesting Pipecat pipeline:

Sunset. Double overhead day.

Public library terrace. San Francisco west side.

Team Suparova at the @supabase / @ycombinator hackathon. There was a four-participant limit on the team size. We have five, but two are robots. Last night was a very long session with lots of tiny little screws and some heavy ifconfig action.

I've been having a lot of fun writing code that uses the OpenAI Realtime API. I wrote up my notes from the past month: getting started, technical details, use cases that are the best fits for this API right now, code snippets you can borrow. www.latent.space/p/realtime-api

OpenAI Realtime API: The Missing Manual

Everything we learned, and everything we think you need to know, from technical details on 24khz/G.711 audio, RTMP, HLS, WebRTC, to Interruption/VAD, to Cost, Latency, Tool Calls, and Context Mgmt

latent.space

Lightscreen AI is live on Product Hunt today: producthunt.com/posts/lights... Very nicely thought through technical job interview platform. I've been having a lot of fun conversations lately about how AI is impacting UI design ...