Turns out you can make btrfs dedup about 10x faster by just... reading the data before you deduplicate it. FIDEDUPERANGE byte-compares every range first, and if it has fallen out of the page cache that is a cold read. Prime it and it isn't. github.com/martinus/oans
GitHub - martinus/oans: Fast, safe filesystem deduplication for btrfs & xfs — a duperemove fork.
Fast, safe filesystem deduplication for btrfs & xfs — a duperemove fork. - martinus/oans
github.com