Small but important quality of life update by @rerun.io: Customizable frustum colors. Finally we can distinguish estimates and ground truth in the same 3D view ;)

Have a sim vehicle in jolt-rust working with stacked hinges for wheel rotation and steering on two of four wheels, no suspension, visualized in @rerun.io:

✨ Massive Pipeline Refactor → One Framework for Ego + Exo Datasets, Visualized with @rerun.io 🚀 After a refactoring, my entire egocentric/exocentric pipeline is now modular. One codebase handles different sensor layouts and outputs a unified, multimodal timeseries file that you can open in Rerun.

MVP of Multiview Video → Camera parameters + 3D keypoints. Visualized with @rerun.io

Trying to wrap my head around fwd/bwd kinematics for imitation learning, so I built a fully‑differentiable kinematic hand skeleton in JAX and visualized it with @rerun.io new callback system in a Jupyter Notebook. This shows each joint angle and how it impacts the kinematic skeleton.

Exciting news for egui: there is a draft branch for switching the text handling to Parley, which will bring support for color emojis, right-to-left text, access to system fonts, and much more! github.com/emilk/egui/p...

@rerun.io v0.23 is finally out! 🎉 I’ve extended my @gradio-hf.bsky.social annotation pipeline to support multiview videos using the callback system introduced in 0.23.

Visualized with @rerun.io, I’ve integrated video‑based depth estimation into my robot‑training pipeline to make data collection as accessible as possible—without requiring specialized hardware.

I extended my previous @rerun.io and @gradio-hf.bsky.social annotation pipeline for multiple views. You can see how powerful this is when using Meta's Segment Anything and multi-view geometry. Only annotating 2 views, I can triangulate the other 6 views and get masks extremely quickly!

@pablovelagomez.bsky.social · last yr.

Here’s a sneak peek using @rerun.io and @gradio-hf.bsky.social for data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.

Fun use of Rerun: www.youtube.com/watch?v=pxRG...

Re: Can You Fool A Self Driving Car? Testing variations reveal what went wrong

YouTube video by Parallel Domain

youtube.com

Here’s a sneak peek using @rerun.io and @gradio-hf.bsky.social for data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.

Using @rerun.io , I established a baseline from the HoCAP dataset and conducted a qualitative comparison among the ground-truth calibrated cameras, Dust3r, and VGGT—all within rerun. The improvements are evident in both the camera parameters and the multi-view depth map/point cloud.

Rerun keeps growing! I’m so happy we started this thing, and that we’re building in the open, in Rust 🤘

Rerun@rerun.io · last yr.

1/ We just raised $17M to build the multimodal data stack for Physical AI! 🚀 Lead: pointnine.com With: costanoa.vc, Sunflower Capital, @seedcamp.com Angels including: @rauchg.blue, Eric Jang, Oliver Cameron, @wesmckinney.com , Nicolas Dessaigne, Arnav Bimbhet Thesis: rerun.io/blog/physica...

1/ We just raised $17M to build the multimodal data stack for Physical AI! 🚀 Lead: pointnine.com With: costanoa.vc, Sunflower Capital, @seedcamp.com Angels including: @rauchg.blue, Eric Jang, Oliver Cameron, @wesmckinney.com , Nicolas Dessaigne, Arnav Bimbhet Thesis: rerun.io/blog/physica...

More progress towards building a straightforward method to collect first-person (ego) and third-person (exo) data for robotic training in @rerun.io. I’ve been using the HO-cap dataset to establish a baseline, and here are some updates I’ve made (code at the end)

Finally finished porting mast3r-slam to @rerun.io and adding a @gradio-hf.bsky.social interface. Really cool to see it running on any video I throw at it, I've included the code at the end

I'm working towards an easy method to collect a combined third-person and first-person pose dataset starting Assembly101 from Meta, with near real-time performance via @rerun.io visualization. The end goal is robot imitation learning with Hugging Face LeRobot

Following up on my prompt depth anything post, I'm starting a bit of a miniseries where I'm going through the tutorials of Lerobot to understand better how I can get a real robot to work on my custom dataset. Using @rerun.io to visualize code: github.com/rerun-io/pi0...

Rerun 0.22 is out! 🔎🟡🔜🔵 The release brings long-requested entity filtering for finding data faster in the Viewer, significantly simplified APIs for partial & columnar updates, and many other enhancements. vimeo.com/1054160833

Rerun 0.22: entity filtering

This is "Rerun 0.22: entity filtering" by Nikolaus West on Vimeo, the home for high quality videos and the people who love them.

vimeo.com

Egui's improved ui rendering & more is of course also going to show up in the next Rerun release! :)

emilk@ernerfeldt.bsky.social · 2y ago

Introducing egui 0.30! This adds `egui::Scene`: a pannable, zoomable container for other UI elements. This release also makes frames and corner radius more in line with how CSS and Figma works. We’ve also improved the crispness of the rendering, and a lot more!

Recently, I've been playing with my iPhone ToF sensor, but the problem has always been the abysmal resolution (256x192). The team behind DepthAnything released PromptDepthAnything that fixes this. Using @rerun.io to visualize. Links at the end of the thread

Entity filtering coming up in the UI 👀

emilk@ernerfeldt.bsky.social · 2y ago

Using egui_kittest for screenshot tests is an absolute game changer docs.rs/egui_kittest/

Rerun 0.21 is out! With it we’re introducing graphs in Rerun www.youtube.com/watch?v=071S...

New Graph View in Rerun 0.21

YouTube video by Rerun

youtube.com

We're #hiring for another super fun role: a #Robotics #ML Engineer! 1/4

This year, while writing my master's thesis, I found this great blog post from @rerun.io that shows the structure of #rosbags. I recommend it to anyone who wonders how rosbags work. It helps understand why not all rosbags can be easily recovered when your robot's battery dies 🪫. rerun.io/blog/rosbag

From the Evolution of Rosbag to the Future of AI Tooling

Thirteen years ago, Willow Garage released ROS (the Robot Operating System) and established one of the standard productivity tools for the entire robotics industry.

rerun.io

Sneak peak: we're working on graphs in Rerun 👀

Check out Pablo Vela Gomez's presentation at our Show & Tells! 🎥 Discover the ultimate 3D vision development experience with pixi, @rerundotio, @huggingface, and @Gradio. Watch now: www.youtube.com/watchv=ff7Bh... This video is one of the examples in his presentation:

We implemented undo in @rerun.io by storing the viewer state in the same type of in-memory database we use for the recorded data. Have a look (sound on!)

World grid for our 3D views is coming along nicely!

Introducing “MAGiC-SLAM: Multi-Agent Gaussian Globally Consistent SLAM”! We do SLAM with novel view synthesis capabilities on multiple simultaneously operating agents! vladimiryugay.github.io/magic_slam/i... 1/7