colton

@colton.boo

Data Engineering / Music / Photography / Tinkering cmpadden.github.io

I wrote an Almanack like Charlie Munger and Naval Ravikant did, but about life wisdom and my time of using Dagster for data orchestration over the years. Since using it in 2018, the story has gone from complexity to composability and from orchestrating to a full data platform.

The Dagster Almanack highlights data orchestration, composability, and governance with icons and structured layout.

We just released version 1.13.0 of @dagster.io with lots of great features like official Dagster skills for, virtualized assets for modeling entities like views, partitioned asset checks, and more. Check out the blog post for more details.

Bild

If you're a Python developer using Claude Code, or something similar, we just published a blog that covers 10 rules we add to our prompt for drastically improved code gen. And we include the full prompt at the end of the post! dagster.io/blog/dignifi...

Bild

“Flexibility without fragmentation” In addition to some solid alteration, it’s also a great explanation of how to build a data team and how your system can and should scale. @colton.boo and Dennis Hume have an awesome new ebook on this topic, check it out! #dataBS dagster.io/how-to-scale...

How to Scale Data Teams | Free eBook from Dagster

Download Dagster's free eBook to learn how to build systems that scale with clarity, reliability, and confidence.

dagster.io

Excited to be at All Things Open tomorrow! Come hang out at the @dagster.io booth, or see my talk “Enabling education with a little help from AI” at 1:45 EST on Tuesday!

Bild

Read this post you work in data analytics. A similar mindset problem pervades sales engineering as well, especially in the demo / early stages of discovery. Understand your prospect’s context. Who are they? What do they care about? What value can you provide them? That’s what you show.

Emily Riederer@emilyriederer.bsky.social · last yr.

Technical writing is hard bcs "writing is thinking" but we often should tell our story not in the order we worked. Solution? I wrote a quick post on how @quarto.org 's embed shortcodes can reframe technical writing as reproducible evidence curation www.emilyriederer.com/post/quarto-... 🧵 (1/n)

I'm writing about #opentableformats. With the newest entries of AWS S3 tables and Cloudflare R2, it's interesting to reassess the market and see where the future with managed Iceberg Tables leads us. What's your take? And anything missing on the evolution below?

Open Table Format Evolution

Refactored `chatblade.nvim` to use `llm`, and renamed the project to `llm.nvim`. It's archaic compared these agentic MCP workflows, but it's completely replaced Googling programming related questions.

Screenshot of the https://github.com/cmpadden/llm.nvim repository README.

Tinkering with dltHub's REST API source to ingest data without having to write any bespoke code, and DuckDB's new local web UI, and I feel so spoiled with how good data tooling is getting!

Screenshot of the DuckDB's local UI and a code example using dltHub's REST source config.

🚀 Want to preview the brand new `dg` cli, and a framework for building and working with YAML DSLs built on top of Dagster called "Components"? You can find a link to demo videos and a GitHub discussion in the thread!

Bild

Joe Naso and I just published an e-book covering the essential topics needed to build a data platform! Check it out if you're looking to learn more about data modeling, ingestion patterns, and common data architectures. dagster.io/how-to-build...

Bild

We’re building a new static type checker for Python, from scratch, in Rust. From a technical perspective, it’s probably our most ambitious project yet. We’re about 800 PRs deep!

Bild

As someone who works at an open core company similar to Preset / Superset, this post really resonates with my experience. Buying a “hosted version of an open source library” made by that libraries creators has more benefits than “just” the hosting. preset.io/blog/running...

Running Apache Superset on the Open Internet: A Report from the Fireline

Running Apache Superset on the open internet requires sophisticated solutions to security and performance challenges. Here we explore how to navigate these challenges while keeping data safe (or opt o...

preset.io