A lot of extraction systems have the same problem: EITHER cram thousands of possible values into the schema OR leave the field open & fix mistakes later The better option: keep the structure fixed and narrow the valid values at runtime See how it works: blog.dottxt.ai/external-constraints

The .txt team is growing. We're looking for a: - Founding DevRel Engineer, - Member of Technical Staff - LLM Inference, and - a Rust Developer Apply here: jobs.ashbyhq.com/dottxt

Be our developer zero. Join .txt as our Founding DevRel Engineer. Apply here: jobs.ashbyhq.com/dottxt

We changed the way we handle multimodal inputs in Outlines. Most libraries will give the prompt a special role, separate it from the other inputs. We don’t anymore.

Check out this cool project by @aastroza.bsky.social to automatically convert any TV stream into a real-time movie script! Amazingly cool project using Outlines to enforce movie script formatting, and generally an impressive project. Here's the entire JSON schema he uses. github.com/aastroza/tvtxt

GitHub - aastroza/tvtxt: [WIP] AI that "reads" live TV and writes it as a movie script in real-time.

[WIP] AI that "reads" live TV and writes it as a movie script in real-time. - aastroza/tvtxt

github.com

Another example from our internal context-free grammar hackathon. You can force a model to generate an acrostic, i.e. make each line start with a letter in a word -- HELLO in this case. Grammars are cool.

During our internal hackathon, one of our teams (named "Too Many Cooks") wrote a grammar-powered recipe generator. The language model can ONLY generate text consistent with this recipe format. Here's a recipe for the universe.

🎵 Cameron with your eyes so bright 🎵 won't you join our Discord tonight

.txt@dottxtai.bsky.social · last yr.

We've got a Discord channel if you want to talk about AI engineer, compound AI systems, low overhead agents (though who knows what that means), constrained decoding, or whatever you're building. Come on by! discord.gg/ErZ8XnCmkQ

We've got a Discord channel if you want to talk about AI engineer, compound AI systems, low overhead agents (though who knows what that means), constrained decoding, or whatever you're building. Come on by! discord.gg/ErZ8XnCmkQ

Loved this paper. Smart idea. Basically they "guess and check" whether a token would appease the compiler when an LLM is doing code generation.

.txt@dottxtai.bsky.social · last yr.

"Type-Constrained Code Generation with Language Models" is a relatively new paper that addresses a common challenge with LLM-generated code. The researchers are from ETH Zurich and UC Berkeley.

"Type-Constrained Code Generation with Language Models" is a relatively new paper that addresses a common challenge with LLM-generated code. The researchers are from ETH Zurich and UC Berkeley.

Type-Constrained Code Generation with Language Models

Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedure does not model…

arxiv.org

My talk "Building Knowledge Graph-Based Agents with Structured Text Generation" at PyData Global 2024 is now available on YouTube: www.youtube.com/watch?v=94yu... #PyData #PyDataGlobal @dottxtai.bsky.social @pydata.bsky.social

Alonso Silva - Building Knowledge Graph-Based Agents with Structured Text Generation

YouTube video by PyData

youtube.com

Do people like this type of thread? They're fun to write.

.txt@dottxtai.bsky.social · last yr.

We recently came across an interesting paper that helps LLMs be better at handling domain-specific languages like database queries or probabilistic programming languages, using an approach called "grammar prompting". Link + brief thread below.

We recently came across an interesting paper that helps LLMs be better at handling domain-specific languages like database queries or probabilistic programming languages, using an approach called "grammar prompting". Link + brief thread below.

Check out the sieves library from Mantis NLP! sieves is a library for structured document processing. It works with Outlines! Go check out the library, it's pretty cool. Repo: github.com/mantisai/sie...

GitHub - MantisAI/sieves: NLP tasks with zero- and few-shot models.

NLP tasks with zero- and few-shot models. . Contribute to MantisAI/sieves development by creating an account on GitHub.

github.com

We're considering adding an llms.txt file to the upcoming Outlines v1.0 documentation to make it more machine readable! Opinions welcome. github.com/dottxt-ai/ou...

We're delighted to sponsor the AI User Conference! It's next week, April 15th-17th. Find us at online or at San Francisco's delightful Fort Mason. Tickets: www.aiuserconference.com @cameron_pfiffer will be presenting a design for Minerva, a frontier space colony resource management system.

Check out our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org on the Weaviate Podcast!

Cameron@cameron.stream · last yr.

I'm on a podcast for @weaviate.bsky.social with Connor Shorten! We talk about structured generation Youtube: www.youtube.com/watch?v=3PdE... Spotify: creators.spotify.com/pod/profile/... Blog: connorshorten300.medium.com/structured-o... #AI @dottxtai.bsky.social

YOOOOOOOOOOOOOOOOO I got to teach a @deeplearningai.bsky.social course! I'm so happy. It was a really fun experience -- go take a look at the course and let us know how it went. bit.ly/4cdQlzG

Getting Structured LLM Output

Learn how to generate structured outputs to power production-ready LLM software applications.

bit.ly

.txt@dottxtai.bsky.social · last yr.

We are excited to announce our partnership with Andrew Ng and @deeplearningai.bsky.social! Our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org created a course on building AI systems using structured output from LLMs. Oh, and it’s free. bit.ly/4cdQlzG

We are excited to announce our partnership with Andrew Ng and @deeplearningai.bsky.social! Our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org created a course on building AI systems using structured output from LLMs. Oh, and it’s free. bit.ly/4cdQlzG

Getting Structured LLM Output

Learn how to generate structured outputs to power production-ready LLM software applications.

bit.ly

People love our Outlines library. We figured we'd release our next big product -- Inlines! Inlines are high-performance inline roller blades for the discerning AI engineer in your life. Reply in the comments and we'll add you to the pre-order list!

Wednesday, April 2nd, at 8am PST. Stay tuned.

Outlines supports "recursive schemas", which are JSON schemas that refer to themselves. For example, here is a simple schema to generate a recursive concept map. This defines a concept, which has a title, a description, and a list of related concepts with the same schema.

Outlines v1.0 has a few interface changes coming up. One is the addition of the Application class, which combines a prompt templating function and an output type. Then, you call that application with the language model you want to use, as well as the arguments to the prompt function.

Here's what happens if you disable the use of "R" and "r" in the thinking block of Qwen 7B R1. We asked if roses are red, and it invented the phrase "conclusion point" to use instead of "answer". It eventually got the right answe.

What features do you want to see in Outlines?

Here's the code for this one: github.com/theophilec/f...

GitHub - theophilec/foudinge: Mapping the French Culinary Universe

Mapping the French Culinary Universe. Contribute to theophilec/foudinge development by creating an account on GitHub.

github.com

.txt@dottxtai.bsky.social · last yr.

Théophile Cantelobre built a map of France's culinary universe by extracting structured data from restaurant reviews using Outlines. They created a knowledge graph connecting chefs and restaurants that traditional prompting couldn't achieve.

Is that our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org hanging out with Andrew Ng? Wonder why.

Let us know how you're using this!

Cameron@cameron.stream · last yr.

Here's how you get structured output from R1 using Outlines. gist.github.com/cpfiffer/e98...

Highly recommend this post by @simonwillison.net about extracting structured data from text with LLMs. Simon puts it well: "the single most commercially valuable application of LLMs is turning unstructured content into structured data."

Structured data extraction from unstructured content using LLM schemas

LLM 0.23 is out today, and the signature feature is support for schemas—a new way of providing structured output from a model that matches a specification provided by the user. …

buff.ly