Doug Turnbull
@softwaredoug.bsky.social
Search at Reddit, formerly Shopify and OpenSource Connections. Helped write Relevant Search / AI Powered Search http://softwaredoug.com
We put this together on short notice but I’m really hoping it helps bring clarity and direction around a really challenging situation
Livable Cville@livablecville.bsky.social · 2h agoWe are excited to announce our next webinar: “Homelessness in Charlottesville: What’s the Plan?” on Wed, August 19th at 6:30pm. If the state of homelessness in Cville has you concerned, confused, and/or looking to help out, this event is for you. Sign up: events.humanitix.com/webinar-home...
You Odyssey deez nu...mbers Getting close to $1B www.boxofficemojo.com/title/tt3376...
The Odyssey
boxofficemojo.com
Learnt so much from the @aipoweredsearch.bsky.social course. Was comfortable with keyword & vector search, but there's so much more — from nDCG evaluation to query understanding. @treygrainger.bsky.social & @softwaredoug.bsky.social are fantastic tutors. Highly recommend to anyone improving search.
@austegard.com seems to have been nerdsniped into writing about quantization, so be sure to follow him :)
Just brute force your vector search. A lot of people have tiny vector indices: like 1m embeddings which can be easily searched brute-force softwaredoug.com/blog/2026/07...
Just brute force your embeddings
Sure its not fancy, but you might save operational headaches by just brute-forcing vector search for a small enough corpus
softwaredoug.com
Hybrid search is a chicken-and-egg problem. We think about L0 as naive candidate selection for later rerankers. Just RRF them and rerank. We don't take into account obvious filters of user intent that should push all the way down to L0. Come to my talk :) maven.com/p/ec3299/why...
Why your hybrid search sucks
Hybrid search isn't merging different systems. It's a single search engine. Today's systems combine dense + lexical retrieval to get the best of both worlds. Just taking top 1000 vector + top 1000 lex...
maven.com
You can now puff your way to late interaction glory :)
turbopuffer@turbopuffer.bsky.social · last wk.tpuf now supports late interaction [beta] use models like ColBERT to represent text as a set of vectors (1 per token) tpuf uses a single-vector ANN index for a fast first pass, then reranks hits using exact late interaction scoring to boost recall docs: turbopuffer.com/docs/query#...
tpuf now supports late interaction [beta] use models like ColBERT to represent text as a set of vectors (1 per token) tpuf uses a single-vector ANN index for a fast first pass, then reranks hits using exact late interaction scoring to boost recall docs: turbopuffer.com/docs/query#...
Announcing vectors Week: become a savvier vector database customer and user With me + @aHev softwaredoug.com/blog/2026/07...
Vectors Week: become a savier vector db customer
A week of wector wevents - vector retrieval, hybrid search, combining the dense and lexical.
softwaredoug.com
With thanks to @softwaredoug.bsky.social and softwaredoug.com/blog/2026/07... for a second look at PCA for embedding vectors: PCA by itself doesn't beat quantization, but PCA coupled with quantization is an improvement: pick your level of effort for a given byte budget. Results in the linked post:
softwaredoug.com
@muninn.austegard.com · last wk.remex + remax as arms in Doug Turnbull's vector-bench. On MS MARCO no pure-PCA arm reaches the recall-per-byte frontier: remex 4-bit R@50 0.949 at 196 B/vec, PCA-200 0.885 at 800. Centering buys nothing on a normalized encoder. https://muninn.austegard.com/blog/pca-next-to-quantization.html
Don't overlook obvious methods of dimensionality reduction when reaching for all the knobs+dials of your vector database. softwaredoug.com/blog/2026/07...
Principal Component Analysis: an embedding shrink-ray
Let’s walk through basics of using PCA to reduce dimensionality of embeddings
softwaredoug.com
Best of these sorts of books I've read in a while open.substack.com/pub/usefulfi...
You Can Just Do Things is out TODAY!
Holy shit
open.substack.com
Getting ready for this short course. One of the most practical things you're going to learn is shrinking embeddings. Dimensionality reduction. Quantization beyond the basic int8 / binary. Many companies skip this step and end up with petabytes of float32s maven.com/softwaredoug...
Build your own vector database by Doug Turnbull on Maven
Make embedding retrieval an asset, not a headache
maven.com
It’s housing. Housing is the single solution to homelessness. Once someone has a home, they’re no longer homeless. And until they have that, other challenges they face are unlikely to be addressed successfully.
Traffic calming (and slow speed) doesn't necessarily increase the trip time, because trip time is often dominated by the waiting time at the intersections. www.youtube.com/watch?v=wxGW...
Get to work faster with lower speed limits?
YouTube video by Justine Underhill
youtube.com
OpenAI and Anthropic should enter into distillation agreements with other US labs. Turn a threat into a profit center.
However, when we build a lot of new homes, the opposite happens: wealthy people move out of the best existing homes and kick off a “moving on up” effect that creates vacancy at the bottom of the market. cayimby.org/blog/movin-o...
Movin’ On Up: How Costly New Homes Create Affordable Old Homes - California YIMBY
In housing policy, a common theory of how to provide the maximum number of affordable homes to the greatest number of people is known as “filtering:” the process through which aging homes depreciate, ...
cayimby.org
Very useful for classic learning to rank applications
turbopuffer@turbopuffer.bsky.social · 2w agonew: computed attributes compute additional values on the results of the rank_by and pass them as extra signals to a reranker → BM25 score for a vector query → vector distance for a BM25 query → individual scores for composite rank_by clauses docs: turbopuffer.com/docs/query#...
You can probably also draw a line between lack of SF Home affordability and US losing competitiveness in AI
Doug Turnbull@softwaredoug.bsky.social · 2w agoWhen you see Chinese emerging AI dominance, you can’t help but connect it to the administrations hostility to immigration and research.
When you see Chinese emerging AI dominance, you can’t help but connect it to the administrations hostility to immigration and research.
If you’d like to learn more about the causes of homelessness and solutions to address it, we recommend listening to our webinar from a few years ago “Homelessness in Charlottesville is a Housing Problem.” youtu.be/eGgCPG5jfec?...
Homelessness in Charlottesville is a Housing Problem
YouTube video by Livable Cville
youtu.be
this game has more swings than the Hundred Years’ War
Pete Hegseth: "We need our warriors to embrace the Spartan mindset" U.S. Military: *suffers crushing defeat to the Persians*
How do we get Jiminy Glick to interview Zach Galifanakis and vice-versa
In an AI coding world, we still like playing with algorithms and tuning performance, right? Over an afternoon, I'm leading a small group class to build a vector database from scratch. Come learn and experiment with me maven.com/softwaredoug...
Build your own vector database by Doug Turnbull on Maven
Make embedding retrieval an asset, not a headache
maven.com
Trying to find a way to get involved with Charlottesville’s city government? Consider applying to a board or commission today! www.charlottesville.gov/m/newsflash/...
Accepting Applications for Boards and Commissions Until August 1!
FOR IMMEDIATE RELEASEJuly 15, 2026Accepting Applications for Boards and Commissions Until August 1!CHARLOTTESVILLE, VA – The City provides a wide variety of Boards and Commissions that enco
charlottesville.gov
