✨ New Narwhals expression: `nw.corr` 👥 Supported in both group-by and window contexts 🌐 Polars, DuckDB, PyArrow, PySpark, pandas...all supported 🗒️ PS. please don't draw conclusions about ice cream sales and drownings from this post
@marcogorelli.bsky.social
Senior Software Engineer at Quansight Labs Pretty focused on Narwhals and Polars these days, generally interested in dataframes and time series
Bwlch Llanberis (Welsh folk tune) Named after a scenic road in Snowdonia, this is one of my favourite Welsh melodies
Tafliad Carreg (Welsh folk tune) This one was written by Mike Lease, and it means "A stone's throw" Here's my attempt at it. First time tuning my guitar to DADGAD, and my initial impression is that standard tuning might be a mistake
✨ @marimo.io now supports Pyrefly as an LSP, helping you to catch errors before you even execute your code! ⚙️ To enable it: - Install pyrefly - Settings -> User settings -> Enable pyrefly 🤸 This was a fun one to contribute to (🙏 Note that the ty type checker is also supported)
✨ New @pola.rs feature: if you misspell a time zone, it will suggest the correct one to you 🤸♂️This was a fun one to work on, Levenshtein and all!
✨ New Polars features: `min_by` and `max_by` 💡 Get the first (or last) value of a column ordered by another expression 🌊🦄 Narwhals will have this feature too, making it available to @arrow.apache.org , pandas, @duckdb.org , and more
😱🙀 The pandas 3.0 release-candidate is here! 💥 Will it break your code? 💡 Test it with `uv pip install -U --pre pandas` to find out! 🌊🦄 Narwhals users can relax, everything's taken care of for you, no need to do anything ☺️
✨ The next @pola.rs release will support lazy pivot! 💡 All you need is to specify the unique values to pivot on
✨ Narwhals now supports Daft! 🔌Just install the `narwhals-daft` plugin, and then `narwhals.from_native` will accept Daft DataFrame! 💡 No extra dependencies requires, computation will remain native (and distributed, and lazy!) until you call `collect`
✨ New @pola.rs feature: `list.agg` 💡 Compute any Expr aggregation on a list column, just use `pl.element()` 🔜 Coming soon, in version 1.35!
🥧🧵 Narwhals now tests against Python 3.14 (pi-Python!) free-threading build 👉 For more on free-threaded Python, my colleague Nathan's blog post is an excellent read, link in comments
✨ Narwhals now supports `format` across all backends 🧵 Create new string columns using placeholders and expressions 🌊🦄 Yet another piece of the @pola.rs API brought to pandas, @duckdb.org , PySpark, PyArrow, and more!
✨ NumPy's MaskedArray class is now type-complete 🏷️ Enjoy improved code completion and typing! 🙏 Thank you to Quansight Labs and Meta for having facilicated and funded the work (blog post in comments)
🌝 The next @astropy.org release will support generic dataframe tables (like @pola.rs ) 🌊🦄 Powered by Narwhals
💡 Life-pro-tip: enable inline type hints 🏷️ Your editor will display inferred types for variables you haven't annotated 🕵️ Debugging those red squiggly lines will become much easier
✨🐼 Pandas 3.0 will have `pd.col` syntax 💡 You can use it for clean chaining in `assign` and `loc` 🚀 It's happening, the PR just got merged!
✨ New @pola.rs feature - you can now chain `name` expressions! 🌊🦄 As of Narwhals 2.0 (yes, we've released 2.0 🚀), this feature is available there as well! Little example below with our friend @duckdb.org
💡 "There's nothing for permanent than a temporary fix" Case in point, this pandas comment which was added [checks git log] ... 12 years ago 🙈
"If the U.S. can't get its greenhouse emissions under control, they need to face real consequences" said the rest of the world.
Canada's National Observer@nationalobserver.com · 12mo ago"If Canada can't get these wildfires under control, they need to face real consequences," said Wisconsin state Rep. Calvin Callahan in a news release Wednesday. "We won't sit back while our air becomes a health hazard."
💡 Today in open source: a 17-year-old NumPy commit promising that a new feature will be coming soon Any day now..
🎢 When the Q&A goes off-script 🤣 🇯🇵 Japanese math rock, 🅿️ porpoises and unfortunate aliases, and 🎤 Taylor Swift: 🥈All of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference 🙏 Thanks @pydatalondon.bsky.social for inviting me!
Marco Gorelli - How Narwhals brings Polars, DuckDB, PyArrow, & pandas together | PyData London 25
YouTube video by PyData
youtu.be
🥳 The next version of @mlflow.org will support @pola.rs ! 😩 Slightly disappointingly, they do so by converting to pandas 🌊🦄 Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below 👇
🙀 It's scary to merge a 2,000+ line PR! 😇 But, this one was so thorough that we just had to ship it 🏆 Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis! 💡You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!
✨ @pola.rs now supports `list.filter` 🪄 The `list` namespace is now even more turbocharged ⚡ 👇 Full release notes for 1.30
✨ PySpark 4.0 is out 🌊🦄 And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)
✨ Narwhals now supports `fill_null` with forwards and backwards strategies 🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more! 🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎
