Max Zhdanov
@maxxxzdn.bsky.social
PhD candidate at AMLab with Max Welling and Jan-Willem van de Meent. Research in physics-inspired and geometric deep learning.
🤹 New blog post! I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025 blog: maxxxzdn.github.io/blog/erwin/ paper: arxiv.org/abs/2502.17019 Compressed version in the thread below:
Can only speak for my ICML reviewing batch, but the hack of putting scary, convoluted and wrong math still works.
🤹 Excited to share Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems joint work with @wellingmax.bsky.social and @jwvdm.bsky.social preprint: arxiv.org/abs/2502.17019 code: github.com/maxxxzdn/erwin
Interestingly, something similar can be done for irregular data structures - point clouds - with a couple of tricks including hierarchical trees and FlexAttention🌳 1/N 🧵
Sung Kim@sungkim.bsky.social · last yr.DeepSeek's NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference! Core components of NSA: • Dynamic hierarchical sparse strategy • Coarse-grained token compression • Fine-grained token selection
the number of snarky comments about Musk from anime pfps on Twitter lately left me wondering how much of the right turn of the group is just teenage contrarianism
🚨 Excited to share our work just published in Physical Review Letters with @wellingmax.bsky.social, @jwvdm.bsky.social, @berndensing.bsky.social, Marjolein Dijkstra and René van Roij: doi.org/10.1103/Phys.... Details below 👇
Learning Neural Free-Energy Functionals with Pair-Correlation Matching
The intrinsic Helmholtz free-energy functional, the centerpiece of classical density functional theory, is at best only known approximately for 3D systems. Here we introduce a method for learning a ne...
doi.org
This is how it should be: one file, ctrl+c ctrl+v runnable from a notebook. I really do not want to mess with omegaconf, hydra or, the worst, a custom configuration system just to figure out how your model runs.
The comments section is depressing to read. I guess automation is inevitable in a progressing society, but such incidents do bring a bitter flavour to the current advancements.
Happy 2025. I Lost My Freelance Writing Job To AI.
YouTube video by Alex Wei
youtube.com
grabbing the whole node on a cluster for an interactive session for a week should be considered antisocial behavior
🚨 FINAL REMINDER 🚨: Multiple Postdoc and PhD positions in our AI + Physics cluster in DAMTP, Cambridge! Deadlines: - Postdoc: Jan 5th (Sunday) - PhD: Jan 7th (Tuesday) More info below – Please share with researchers and students who might be interested in joining us!
🚨 PhD position alert! 🚨 I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...
PhD Position in Mechanistic Interpretability
PhD Position in Mechanistic Interpretability
vacatures.uva.nl
If you are attending #NeurIPS2024🇨🇦, make sure to check out AMLab's 11 accepted papers ...and to have a chat with our members there! 👩🔬🍻☕ Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list! 🧵1 / 12
I will be at NeurIPS this week. @amlab.bsky.social will be presenting 10 papers in total, and I myself am involved with 3 (see thread). I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!
At a certain point, I was so inspired by the geomancer paper by @davidpfau.com et al. that I decided to apply for a PhD in causality/GDL at ELLIS. All those buzzwords in my motivational letter just to find later that I was writing about casual ML all along 🤦♂️
Marvin Schmitt@marvin-schmitt.com · 2y agoSearch Google Scholar for "casual inference" (not causal inference).
Yesterday @ellisamsterdam.bsky.social hosted the yearly NeurIPS-Fest, a pre-party for NeurIPS with a keynote talk, poster session, drinks and bites! 🍺🍻 The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥
Fun paper led by Julia Balla: "A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing". Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month. 📄: arxiv.org/abs/2410.20516 💻: github.com/smsharma/eqn...
A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing
Efficiently processing structured point cloud data while preserving multiscale information is a key challenge across domains, from graphics to atomistic modeling. Using a curated dataset of simulated ...
arxiv.org
Arvind, me, and Jonah released a new pre-print on some pen and paper analysis of fundamental failure modes and old school stability analysis for neural PDEs typically used in AI for Science application. arxiv.org/abs/2411.15101. 1/n
What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning
Differentiable Programming for scientific machine learning (SciML) has recently seen considerable interest and success, as it directly embeds neural networks inside PDEs, often called as NeuralPDEs, d...
arxiv.org
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
Meet our Lab's members: staff, postdocs and PhD students! :) With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋 go.bsky.app/8EGigUy
New year's resolution: make it to the grumpy ML starter pack
Hi everyone! This is AMLab :) Looking forward to share our research here on 🦋 !
Yesterday was my first day at FAIR Chemistry as a research scientist. Looking for interns for summer 2025!
I was playing with the fast multipole method this summer, mainly by translating the fantastic pybbfmm library from torch to jax. showcase: simulation of an infection spread over the British Isles (10M agents, 3s to simulate with jax, 3x over pytorch).
