Commit Graph

15 Commits

Author SHA1 Message Date
Joe Thornber
040e3bfc2d Lot's of work on thin_restore 2021-03-24 14:20:20 +00:00
Joe Thornber
1ae62adec6 work in progress 2020-11-18 14:33:56 +00:00
Joe Thornber
9995751dde [thin_check (rust)] Provide 3 different report types.
ProgressBar for interactive use.
Simple for when not a tty
and quiet for -q
2020-08-17 13:10:32 +01:00
Joe Thornber
08e3ea948e [thin_check (rust)] rename block_manager.rs -> io_engine.rs 2020-08-10 08:29:32 +01:00
Joe Thornber
fa4ea3e2d9 [functional-tests] port some of the cache_check tests to Rust 2020-08-07 14:30:00 +01:00
Joe Thornber
cc2582b8b1 [thin_check (rust)] factor out pdata/btree.rs 2020-08-03 12:37:32 +01:00
Joe Thornber
1398cf31d1 [thin_check (Rust)] work in progress 2020-07-27 15:53:42 +01:00
Joe Thornber
3cf6307762 Merge branch 'master' into 2020-06-13-thin-check-rewrite 2020-07-27 15:53:26 +01:00
Joe Thornber
d03dac8f75 [thin_shrink] write test harness 2020-06-26 16:44:47 +01:00
Joe Thornber
861b2f21ff [thin_shrink] Write thinp xml format reader and writer.
Still need to tidy up the error handling.
2020-06-24 13:32:47 +01:00
Joe Thornber
29d56f62a5 wip 2020-06-14 08:17:46 +01:00
Joe Thornber
380632565d [thin_metadata_{pack/unpack}] Fix get_nr_blocks()
It didn't work with block devices.
2020-06-10 14:38:42 +01:00
Joe Thornber
409a660082 [pack/unpack] Get the functional tests working again.
There's some hard coded version numbers in the tests, but I'm
leaving for now since I'll rewrite in Rust to avoid too much
of a proliferation of languages.
2020-06-09 13:03:39 +01:00
Joe Thornber
db5a71a53c [rust tools] Get the tools version from autoconf 2020-06-09 10:23:52 +01:00
Joe Thornber
61de3f9287 [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one.
The Rust implementation is multithreaded, performs better in general and
does custom compression of btree nodes to achieve much better compression
ratios.  unpack also checksums expanded metadata to validate it.

Format version has jumped to 3, no backwards compatibility, but I think
that's ok since we never made a release that contained the C++ version
of these tools.

Benchmarks
==========

On an 8 core, 16 hyperthread machine.

metadata 1G, full:

      Pack size    pack time     unpack time
------------------------------------------------------
C++      193M        50.3s          6.9s (no verify)
Rust      70M         1.4s          1.8s (verify)

metadata 16G, sparse:

       Pack size    pack time     unpack time
------------------------------------------------------
C++      21M          68s           1s   (no verify)
Rust      4M           8.6s         0.5s (verify)
2020-06-09 09:15:00 +01:00