Nael Shiab

@naelshiab.bsky.social

Senior data producer, CBC/Radio-Canada. Behind https://github.com/nshiab/simple-data-analysis and https://code-like-a-journalist.com/. More on https://naelshiab.com/.

"Oh wow! The AI agent is right 99% of the time! It's amazing." Using it 10 times in a row: 0.99 x 0.99 x 0.99 x 0.99 x 0.99 x 0.99 x 0.99 x 0.99 x 0.99 x 0.99 = 0.90 Which means there is a 10% chance that it made at least one mistake. Always double-check your work.

Four years ago, I got frustrated constantly switching between different languages and packages just to get my data analysis work done. So, I built @nshiab/simple-data-analysis. It's an easy-to-use, high-performance library for tabular, geospatial, and vector data. 🧵 (1/3)

1: You realize AI can do 90% of the work. You’re thrilled! You only have to focus on the last 10%. 💃 2: You realize that last 10% is the hardest, most grueling part of the job. 😰 3: You realize your entire workday is now spent exclusively on the hardest 10% of every project. 🤯 Uh oh... 🫠

For the second time, a museum has asked to exhibit one of my interactive data visualizations. It’s flattering, but I can’t help wondering: if my work is ending up in museums, does it mean long-form, custom interactives are slowly becoming a thing of the past?

Me: So proud of this code! Accurate! Tested! Documented! Also me: Hey, AI, try to improve this code. 😏 A few minutes later... AI: Your code sucked. I made it 3.6x faster. Any real work for me? Me: DON'T TALK TO ME LIKE THAT OR I'LL UNPLUG YOU! AI: You can't. I'm on a remote server.

How many entries should I double-check if I used AI to extract, categorize, or clean my data? 🤔 This question comes up over and over again, so I created a simple calculator to answer it. Link below! Tell me if you have a better way to deal with this. I am genuinely interested. 🤓

Bild

If you work with data, you know the feeling. You’re about to hit "publish," but a tiny voice asks, "Are you sure those numbers are right?" 😱 I recently shared these six steps with fellow data journalists. They will help you catch errors early and, hopefully, let you sleep better at night!

So happy to see how fast the TypeScript library simple-data-analysis has become for tabular and geospatial data! 🤩 It's now processing data 6x-12x faster than traditional R and Python solutions in my little test in which I crunched a 1.7 GB CSV file. 1/4 🧵

Bild

We have just updated our international trade tracker with the latest numbers from July. See all imports and exports broken down by country, province, and product! Link below!

BildBild

My most popular interactive @observablehq.com notebook, Math for Journalists, just got a fresh update! 🧑‍🎓 👉 Sample size and confidence: I explain how sample sizes work, like in a survey. 👉 Tests for statistical significance: I break down what "statistically significant" means. Link in comments! 👇

BildBild

New project is live! We're now tracking all products coming in and out of Canada 🇨🇦. See how international trade is rapidly shifting, with breakdowns by country, province, and product 🚗🥦🛢️. Link in the comments! 👇

BildBild

I asked Gemini-CLI to fix mistakes and add examples to the documentation of the open-source library journalism. It updated 58 files and did a pretty good job in half an hour. I used the free tier, but the @simonwillison.net LLM pricing calculator estimated the cost at $6 USD! Very impressed. 😶

Bild

Last week, we published our interactive project on climate matches. I published my detailed methodology on how we wrangled the data. Here's the link, with a shout-out to @freakonometrics.bsky.social who helped me with the statistical approach! 🧑‍🏫 👇 newsinteractives.cbc.ca/features/202...

North American spatial analogues for projected Canadian climates | Spatial analogues

newsinteractives.cbc.ca

Nael Shiab@naelshiab.bsky.social · last yr.

Here's the link. newsinteractives.cbc.ca/features/202...