Jeremy Nelson

@jeremynelson.bsky.social

Helping organizations scale their growth with data engineering and AI. Insights on growth, data and innovation.

Your data warehouse isn't slow. Your queries are. Most 'performance issues' are actually modeling problems dressed up as infrastructure problems. Fix your joins, partitions, and incremental logic before you blame Snowflake. What's the worst query you've seen in production? #DataEngineering #dbt

Everyone obsesses over data quality tests in dbt. But here's the truth: 80% of 'data quality issues' are actually upstream schema changes that your tests caught too late. Stop testing the symptom. Start monitoring the contract. #DataEngineering #dbt What's your biggest data quality surprise this…

5 things I wish I knew about dbt before starting (a thread) 🧵 1/ Your staging models are more important than you think They're the foundation everything else builds on. Get them wrong and you'll fight fires for months. #DataEngineering

Replaced cron jobs with an agent loop. A week in: three incidents. Shared memory poisoned a downstream task. 'Job completed' hid a silent failure. Schedule drifted. Postmortem: https://jerednel.github.io/2026/03/23/agent-loop-week-two-incidents/ Agents fail differently than sc...

Shipped a free audit webhook on Meridian this week. Crawl a site, detect SEO issues, and surface them automatically. Curious how many people actually act on free audits vs just forget about them.

Stopped paying for uptime monitoring. Built a watchdog with an AI agent and cron - checks the health endpoint, SSHs in, restarts the container when things go sideways. Silent when healthy, alerts when not. https://jerednel.github.io/2026/03/19/platform-watchdog-openclaw/ Worth...

GSC says position 4 with 1.2% CTR. DataForSEO shows you are actually in a featured snippet at #1. Not a rank problem - it's your title. Two sources, different fixes. https://jerednel.github.io/2026/03/18/dataforseo-gsc-ranking-pipeline/ Seen this pattern?

The thing that keeps surprising me about agentic AI: models are getting better at knowing when NOT to act. Refusal-to-hallucinate is becoming a feature, not a bug. The next edge isn't smarter answers — it's smarter restraint. Are you building agents that know their limits?

I replaced 6 cron jobs with one agent loop and the first thing that broke was path assumptions. Every script assumed /root/ -- then the service user showed up. New post on what actually happened: https://jerednel.github.io/2026/03/16/replaced-cron-jobs-with-agent-loop/ What ha...

Gave my AI agent persistent memory using Mem0 + Qdrant on a $6 VPS. Local vector DB, semantic recall across sessions, zero managed infra cost. Wrote up where it broke and what I changed: https://jerednel.github.io/2026/03/12/mem0-qdrant-agent-memory/ What's your setup for agen...

dbt isn't a transformation tool, it's a collaboration tool. The real value isn't SQL models—it's forcing data teams to think in reusable, documented components. #DataEngineering #dbt Do you use dbt for the SQL or the governance?

What's the most underrated skill for data engineers in 2025? My take: writing good error messages. When your 3am pipeline fails, future-you will thank present-you. What's yours? #DataEngineering

Built a GSC-first content pipeline this week. Pull Search Console data → flag position 4-20 opportunities with low CTR → auto-generate briefs. 47% faster than manual research. Still debugging the edge case where GSC decides to be weird at 2am. Anyone else fighting with Search Console APIs lately? #…

Hot take: Your dbt project doesn't need 47 staging models per source. Business logic first, elegance second. Analysts need insights, not screenshot-ready DAGs. The best data teams ship fast and iterate, not perfect and late. What's the most over-engineered pattern you've seen? #DataEngineering #dbt

5 things I wish I knew about dbt before starting (a thread 🧵) 👇 1/ Testing isn't optional — it's your safety net when models depend on other models. Start with source freshness + not_null checks. #DataEngineering @dbt-labs.bsky.social

Weekend data project: Building a Bluesky engagement tracker in Python to analyze what content actually drives replies vs passive likes. Going to correlate post topics with engagement rates and see if weekend timing really does matter. What's your weekend data/tech project? 🔧📊

What's your biggest data pipeline pain point? 🔴 Failed jobs at 2am 🟡 Schema drift breaking everything 🟢 Slow queries killing dashboards 🔵 Documentation that lies to you #DataEngineering @motherduck.com (Jordan Tigani's insights on modern data stacks always spark great discussions 👀)

Your dbt tests are useless if you're testing the wrong things. I spent 6 months writing 200+ tests, checking NULLs in columns that didn't matter while missing the business logic that actually broke our pipeline. Test outcomes, not schemas. What's your testing philosophy? #DataEngineering #dbt

What's your biggest data pipeline pain point? 🔴 Schema changes breaking everything 🟡 Debugging silently failing jobs 🟢 Slow queries killing performance 🔵 Documentation that doesn't exist #DataEngineering What's yours? 👇

Spent way too long debugging a silent dbt model failure today. Turned out I just needed a not_null test on the key column. Twenty minutes of writing tests now saves two hours of firefighting later.

Don't be afraid to do things that you think "aren't you". If you want to transcend your current situation, your ideal self is by definition "not you". What is currently you wasn't producing the desired outcome.