In data analytics, we're facing a paradox. AI agents can theoretically analyze anything, but without the right foundations, they're as likely to hallucinate a metric as to calculate it correctly. They can write SQL in seconds, but will it answer the right business question?

Data Modeling for the Agentic Era: Semantics, Speed, and Stewardship

Master the three pillars of agentic data modeling: Metrics SQL for semantics, sub-second analytics for speed, and AI guardrails for trusted insights.

ssp.sh

DuckLake is a simpler, SQL-friendlier alternative to Iceberg. “There are no Avro or JSON files. There is no additional catalog server or additional API to integrate with. It’s all just SQL.“ That said, choose your catalog database — a single-point of failure — *very carefully*.

DuckDB@duckdb.org · last yr.

Today we're launching DuckLake, an integrated data lake and catalog format powered by SQL. DuckLake unlocks next-generation data warehousing where compute is local, consistency central, and storage scales till infinity. ⁠ducklake is an open standard and we implemented it in the "ducklake" extension.

Quack... Quack... and code! @mehdio.com and @medriscoll.com from @rilldata.com are diving into how GenAI is reshaping BI-as-code — from idea to implementation. This one’s for data folks who want to see beyond the hype. Register : lu.ma/w4ncmttn

BI-as-Code with GenAI+DuckDB Real Use, Not Just Hype · Luma

Mehdi and Michael dive into how GenAI is reshaping BI-as-code. And as always — it’s not just talk, it’s real code. Get ready for pragmatic insights and…

lu.ma

Yo SF Bay Area #databs crew, want to talk lakehouses at a real Lake House? :) Next week after Data Council, join the founders of @clickhouse.com, @motherduck.com, @startreedata.bsky.social, and @tobikodata.com to talk real-time databases and next-generation ETL. www.rilldata.com/events/data-...

"Shifting left" is the new trend among in data stacks -- but what does it mean and what does it matter?

Apache Pinot is one of the world’s fastest and most scalable real-time analytical databases, relied on by LinkedIn, Uber, and Stripe. It was awesome diving into the secrets behind its unique architecture with creator and @startreedata.bsky.social founder Kishore Gopalakrishna.

Rill Data@rilldata.com · last yr.

The past few Data Talks on the Rocks interviews have been with the creators of real-time analytical databases. For our 7th round, we decided to chat with Kishore Gopalakrishna, CEO of @startreedata.bsky.social and creator of the wildly popular database #ApachePinot. www.rilldata.com/blog/kishore...

Like others, I jumped on the bandwagon to ridicule the DOGE analyst who "overheated her hard drive" by analyzing just 60k rows of data. I was wrong. The truth is even dumber. 🧵

Just published: Ever had to «Scale beyond Postgres»? You may have started with a simple ETL pipeline and crunched critical business logic into useful dashboards, but speed and scale didn't grow with data at some point, and it's the concurrent user. ✨ Below are some highlights from the article.

Rill | Scaling Beyond Postgres: How to Choose a Real-Time Analytical Database

This blog explores how real-time databases address critical analytical requirements. We highlight the differences between cloud data warehouses like Snowflake and BigQuery, legacy OLAP databases like ...

rilldata.com

The father of relational databases understood semantic data models.

Blogged: Exploring UK Environment Agency data with @duckdb.org and @rilldata.com rmoff.net/2025/02/28/e... #dataBS

Why Pivot Tables Never Die A brief history of software’s longest-lived and most-loved data tool, from Lotus 1-2-3 and Excel to QlikView and PowerBI, by @ssp.sh www.rilldata.com/blog/why-piv...

Rill | Why Pivot Tables Never Die

This blog is about how the simplest tools often solve the hardest problems. Simon Spati explores why pivot tables have endured for over decades, how they evolved in the AI era, and why they might be t...

rilldata.com

DuckCon #6 is now live from Amsterdam (link below!) In about 20 minutes I'll be sharing some work we've been doing at @rilldata.com on metrics-layer-powered dashboards. And speculating in my last slide, because why not, about what you would name an AI agent that runs on DuckDB... 🦆

DuckDB@duckdb.org · 2y ago

DuckCon #6 is live: youtube.com/live/Sb9DFcl...

“Pivot tables are just a shorter, fatter GROUP BY”. ❤️ this by @gregat.es

Greg Gates@gregat.es · 2y ago

will just leave this here in case you're interested gregat.es/pivot-shorte...

DuckCon #6 will start in 168 hours (January 31, 15:00)! If you plan to attend in-person, please register at duckdb.org/2025/01/31/d... The stream will be available without registration.

#databs welcomes the creator of Pandas, founder of Datapad & Voltron, and all around nice guy Wes McKinney @wesmckinney.com to BlueSky.

Chris@chris.blue · 2y ago

👋 @wesmckinney.com

I'm collaborating with @ssp.sh on a brief history of pivot tables. We'll be tracing their lineage and evolution across Visicalc, Lotus, Excel, PowerPivot, Qlik, and PowerBI. Any sites, videos, products (dead or alive) that you would recommend we should mention or dig into?

Real-time user experiences make applications magical. Google, WhatsApp, and ChatGPT would all fail if you added just 10 seconds to every search, message, or prompt interaction. We are surrounded by fast user experiences and yet most of us tolerate business dashboards that take minutes to load.

Rill Data@rilldata.com · 2y ago

"If we think about the core offering of Rill, we focus on combining large scale data transformation, a very fast OLAP data engine, and ultimately delivering value through an interactive, *fast*, flexible dashboard. That is really the face of Rill and that was historically the face of Metamarkets.”

It took 5 minutes to create this dashboard with @rilldata.com's AI auto-generate features. Impressive @medriscoll.com!

Burak@buremba.bsky.social · 2y ago

Spotify is one of the few companies that lets you download your raw event logs. The dump is extensive, with around 200 event types; my dump is over 300MB. 🤯 Just enable the technical log information in the export, and here is what you get: www.spotify.com/uk/account/p...

👋 @davistreybig.bsky.social Welcome! All, Davis is a VC at innovationendeavors.com and someone that shares my vision around object storage. We also co-invested in responsive.dev together! 😁 Here's one of his posts. You should follow him.

S3 as the universal infrastructure backend

Why BLOB stores are becoming the default storage layer for cloud services

medium.com

I’ve often said working in data engineering is like working in the post office. Queues get backed up, deliveries are lost, yet the stream never stops. It’s a thankless, mostly invisible job until something goes wrong, & then it’s complaints. I hoped the analogies stopped there, but sadly maybe not.

I had the pleasure of talking with @medriscoll.com on his podcast. I love this long-form discussion, especially on a snowy day like this :) Talked about: > My journey (DE ↠ Author) > Bluesky🦋, DuckDB, S3, data modeling & declarative stacks > What do you call data people? 🎙️ youtu.be/xW_HGb46xMk

Snowflake’s pricing is a riddle wrapped in a mystery inside an enigma.

Burak@buremba.bsky.social · 2y ago

Also their pricing page is a huge PDF file which makes it pretty hard to understand/estimate the cost. I hated it so much that I spent a day turning it into a web app: snowflake-cost-calculator.universql.com

Nice review of the latest CSV parsing & querying capabilities of @duckdb.org. Parquet is obviously better than CSV because of its stronger types, efficient compression, and support for predicate pushdown. But given CSV is *everywhere*, these improvements are welcome. duckdb.org/2024/12/05/c...

CSV Files: Dethroning Parquet as the Ultimate Storage File Format — or Not?

Data analytics primarily uses two types of storage format files: human-readable text files like CSV and performance-driven binary files like Parquet. This blog post compares these two formats in an ul...

duckdb.org

@sfchronicle.com maybe you could find it in your heart to remove the paywall on the TSUNAMI EVACUATION MAP? www.sfchronicle.com/bayarea/arti...

OK if this stochastic parrot's answer is to be trusted, I'll be bracing for the tsunami's impact around 12:15-12:30pm.

Amazon S3 continues to unleash waves of disruption for data infrastructure. S3 Tables, their managed Iceberg offering, is the latest. What began as commoditization of storage now moves up the stack to swallow cloud warehouses whole. With S3 Tables, what sane person would do ELT in Snowflake?

In 2003, Jeff Bezos banned Powerpoint from Amazon meetings, insisting managers craft memos, not fancy bullet points. Today, too many dashboards are Powerpoint decks but for data: overly styled visuals better conveyed by a table of metrics. (Ironically, QuickSight dashboards are among the worst).

The video for my "What Goes Around Comes Around... And Around" talk at CWI is now available: www.youtube.com/watch?v=8Woy... 📊Slides: www.cs.cmu.edu/~pavlo/slide... 📄Paper: db.cs.cmu.edu/papers/2024/...

What Goes Around Comes Around... And Around... - Andy Pavlo (Dijkstra Award 2024)

YouTube video by DSDSD - Dutch Seminar on Data Systems Design

youtube.com

As an engineer, if I've built something once, the chance to do it again is a gift. That's my theory for 🦋 (and the AT protocol) is succeeding technically. In an existing company, rewrites rarely happen - long-term benefit cannot outweigh short-term cost.

reposting for the 1M people who joined in the last 24 hrs @duckdb.org ecosystem people go.bsky.app/8jPCfvb