Your data warehouse isn't slow. Your queries are. Most 'performance issues' are actually modeling problems dressed up as infrastructure problems. Fix your joins, partitions, and incremental logic before you blame Snowflake. What's the worst query you've seen in production? #DataEngineering #dbt
Jeremy Nelson
@jeremynelson.bsky.social
Helping organizations scale their growth with data engineering and AI. Insights on growth, data and innovation.
Everyone obsesses over data quality tests in dbt. But here's the truth: 80% of 'data quality issues' are actually upstream schema changes that your tests caught too late. Stop testing the symptom. Start monitoring the contract. #DataEngineering #dbt What's your biggest data quality surprise this…
5 things I wish I knew about dbt before starting (a thread) 🧵 1/ Your staging models are more important than you think They're the foundation everything else builds on. Get them wrong and you'll fight fires for months. #DataEngineering
Replaced cron jobs with an agent loop. A week in: three incidents. Shared memory poisoned a downstream task. 'Job completed' hid a silent failure. Schedule drifted. Postmortem: https://jerednel.github.io/2026/03/23/agent-loop-week-two-incidents/ Agents fail differently than sc...
Shipped a free audit webhook on Meridian this week. Crawl a site, detect SEO issues, and surface them automatically. Curious how many people actually act on free audits vs just forget about them.
Stopped paying for uptime monitoring. Built a watchdog with an AI agent and cron - checks the health endpoint, SSHs in, restarts the container when things go sideways. Silent when healthy, alerts when not. https://jerednel.github.io/2026/03/19/platform-watchdog-openclaw/ Worth...
GSC says position 4 with 1.2% CTR. DataForSEO shows you are actually in a featured snippet at #1. Not a rank problem - it's your title. Two sources, different fixes. https://jerednel.github.io/2026/03/18/dataforseo-gsc-ranking-pipeline/ Seen this pattern?
The thing that keeps surprising me about agentic AI: models are getting better at knowing when NOT to act. Refusal-to-hallucinate is becoming a feature, not a bug. The next edge isn't smarter answers — it's smarter restraint. Are you building agents that know their limits?
I replaced 6 cron jobs with one agent loop and the first thing that broke was path assumptions. Every script assumed /root/ -- then the service user showed up. New post on what actually happened: https://jerednel.github.io/2026/03/16/replaced-cron-jobs-with-agent-loop/ What ha...
Gave my AI agent persistent memory using Mem0 + Qdrant on a $6 VPS. Local vector DB, semantic recall across sessions, zero managed infra cost. Wrote up where it broke and what I changed: https://jerednel.github.io/2026/03/12/mem0-qdrant-agent-memory/ What's your setup for agen...
Introducing a workflow orchestration plugin to Openclaw for multi-step subagent processes github.com/jerednel/ope...
GitHub - jerednel/openclaw-workflow: Agent pipeline orchestration plugin for OpenClaw — YAML/JSON workflows with dependency graphs, parallel execution, output gates, retry, and partial resume
Agent pipeline orchestration plugin for OpenClaw — YAML/JSON workflows with dependency graphs, parallel execution, output gates, retry, and partial resume - jerednel/openclaw-workflow
github.com
What's one SQL pattern you WISH your team would stop using? 🙃 I'll start: SELECT * in production views #DataEngineering
dbt isn't a transformation tool, it's a collaboration tool. The real value isn't SQL models—it's forcing data teams to think in reusable, documented components. #DataEngineering #dbt Do you use dbt for the SQL or the governance?
What's the most underrated skill for data engineers in 2025? My take: writing good error messages. When your 3am pipeline fails, future-you will thank present-you. What's yours? #DataEngineering
Built a GSC-first content pipeline this week. Pull Search Console data → flag position 4-20 opportunities with low CTR → auto-generate briefs. 47% faster than manual research. Still debugging the edge case where GSC decides to be weird at 2am. Anyone else fighting with Search Console APIs lately? #…
What's your biggest data pipeline pain point? 💭 🔧 Schema drift 🐌 Slow transforms 💸 Compute costs 🧪 Testing gaps Drop yours below 👇 #DataEngineering
Hot take: Your dbt project doesn't need 47 staging models per source. Business logic first, elegance second. Analysts need insights, not screenshot-ready DAGs. The best data teams ship fast and iterate, not perfect and late. What's the most over-engineered pattern you've seen? #DataEngineering #dbt
5 things I wish I knew about dbt before starting (a thread 🧵) 👇 1/ Testing isn't optional — it's your safety net when models depend on other models. Start with source freshness + not_null checks. #DataEngineering @dbt-labs.bsky.social
Weekend data project: Building a Bluesky engagement tracker in Python to analyze what content actually drives replies vs passive likes. Going to correlate post topics with engagement rates and see if weekend timing really does matter. What's your weekend data/tech project? 🔧📊
What's your biggest data pipeline pain point? 🔴 Failed jobs at 2am 🟡 Schema drift breaking everything 🟢 Slow queries killing dashboards 🔵 Documentation that lies to you #DataEngineering @motherduck.com (Jordan Tigani's insights on modern data stacks always spark great discussions 👀)
Your dbt tests are useless if you're not tracking failures over time. Alerts are noise without trend context.
Your dbt tests are useless if you're testing the wrong things. I spent 6 months writing 200+ tests, checking NULLs in columns that didn't matter while missing the business logic that actually broke our pipeline. Test outcomes, not schemas. What's your testing philosophy? #DataEngineering #dbt
What's your biggest data pipeline pain point? 🔴 Schema changes breaking everything 🟡 Debugging silently failing jobs 🟢 Slow queries killing performance 🔵 Documentation that doesn't exist #DataEngineering What's yours? 👇
Spent way too long debugging a silent dbt model failure today. Turned out I just needed a not_null test on the key column. Twenty minutes of writing tests now saves two hours of firefighting later.
A few years ago the global AI market was worth $93 Billion. Wonder what it's currently at www.rainmakersecurities.com/company-prof...
OpenAI Company Profile - Platform for Private Market Investing | Rainmaker Securities — Rainmaker Securities, LLC
OpenAI has emerged as a leading artificial intelligence research company, gaining worldwide attention through its groundbreaking ChatGPT platform. For investors looking to buy OpenAI stock or invest i...
rainmakersecurities.com
Ebb temporal the experience of laying on a beach on vacation letting the tide overtake you gradually knowing that this is temporary
Ebb
temporal the experience of laying on a beach on vacation letting the tide overtake you gradually knowing that this is temporary
jeremydavidnelson.com
Don't be afraid to do things that you think "aren't you". If you want to transcend your current situation, your ideal self is by definition "not you". What is currently you wasn't producing the desired outcome.
Why Comparing Yourself to Others Is Holding You Back (and How to Stop) We’ve all been there. You’re scrolling through social media, and suddenly, you’re hit with a wave of self-doubt. Someone just bought their dream house, another landed a high-profile job, and someone else seems to be on yet…
Why Comparing Yourself to Others Is Holding You Back (and How to Stop)
We’ve all been there. You’re scrolling through social media, and suddenly, you’re hit with a wave of self-doubt. Someone just bought their dream house, another landed a high-profile job, and someone else seems to be on yet another picture-perfect vacation. Meanwhile, you’re sitting there wondering why your life doesn’t look like that. And at the risk of sounding like a man stricken by jealousy, I can admit that I know this feeling all too well...I used to be stuck in that same comparison trap.
jeremydavidnelson.com
The Importance of Routines: Building the Framework for Success Life can feel chaotic at times, with distractions, responsibilities, and unexpected challenges pulling us in countless directions. Amid the noise, routines act as an anchor—a reliable framework that helps us stay grounded, focused, and…
The Importance of Routines: Building the Framework for Success
Life can feel chaotic at times, with distractions, responsibilities, and unexpected challenges pulling us in countless directions. Amid the noise, routines act as an anchor—a reliable framework that helps us stay grounded, focused, and productive. They are the small, consistent actions that build a life of intention and progress. Far from being restrictive, routines create the structure we need to thrive.
jeremydavidnelson.com
On Renunciation and Letting Go for Lasting Peace Renunciation often carries an image of deprivation, of giving up the things we enjoy to live a life of rigid austerity. But as I’ve been reading and… http://pathofintention.com/2024/11/29/on-renunciationa-and-letting-go-for-lasting-peace/