emo/thin-provisioning-tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,893 Commits 2 Branches 0 Tags 3.7 MiB
60b65ebe7a
Commit Graph

45 Commits

Author SHA1 Message Date
Ming-Hung Tsai
ce94ba73a5 [cache_restore (rust)] First draft 2021-05-28 03:19:42 +08:00
Ming-Hung Tsai
b7bf82b8f2 [all] Fix newline in version string 2021-05-12 12:03:52 +08:00
Ming-Hung Tsai
965fbb6e8f [all] Apply cargo fmt, and fix clippy warnings 2021-05-11 23:53:31 +08:00
Ming-Hung Tsai
5baeab4a5c Merge branch 'main' into rust-cache-tools 2021-05-11 23:16:08 +08:00
Joe Thornber
d9a96758b0
Merge pull request #168 from mingnus/2020-10-09-thin-restore-rewrite 
Merge recent changes in thin_restore
 2021-05-04 09:21:40 +01:00
Ming-Hung Tsai
43e433149b [all] Apply cargo fmt 2021-05-04 16:10:23 +08:00
Ming-Hung Tsai
cf4b937ade [cache_check (rust)] Check space map counts 
- Support space map checking and auto-repair
 2021-04-23 16:23:21 +08:00
Ming-Hung Tsai
e1628f9004 [cache_check (rust)] Add more checks 
- Check array indices continuity
- Support ignore_non_fatal
- Report blocknr of IoErrors
- Report array block indeices
 2021-04-21 23:35:22 +08:00
Ming-Hung Tsai
239ff7dfa1 [cache_check (rust)] Add more checks 
- Report errors
- Support reading partially broken bitset
  - The output is a bitmap of 2-bit entries, indicating availability of bits
 2021-04-21 14:09:38 +08:00
Ming-Hung Tsai
860b3ca7d2 [cache_check (rust)] Add more checks 
- Check the version-1 dirty flag
- Check mappings against the origin device size, if the cache is clean
- Check superblock fields
 2021-04-11 20:18:25 +08:00
Joe Thornber
c41b2e8ec3 Some interface changes to let dm-unit use the thin checking code more easily. 2021-04-08 10:45:50 +01:00
Joe Thornber
040e3bfc2d Lot's of work on thin_restore 2021-03-24 14:20:20 +00:00
Ming-Hung Tsai
eb3d181f95 [cache_dump (rust)] First draft of cache_dump 2021-03-03 12:27:51 +08:00
Joe Thornber
a4ba01cacd
Merge pull request #160 from mingnus/rust-cache-tools 
First draft of cache_check in Rust
 2021-02-24 09:53:51 +00:00
Ming-Hung Tsai
2bb3bf65b7 [cache_check (rust)] Implement basic functions 2021-02-24 17:39:11 +08:00
Joe Thornber
c3c6d37aea Fix a lot of clippy warnings 2021-02-08 10:38:21 +00:00
Ming-Hung Tsai
ca7e79a828 Merge commit 'b67b587' into main 2021-01-12 02:50:12 +08:00
Joe Thornber
ba7fd7bd2b [thin_check (rust)] Make --sync-io the default. 
For some systems you have to adjust the ulimits to get io_uring
to work, so we now default to using sync io.

Also added --async-io flag.
 2020-12-02 11:33:05 +00:00
Joe Thornber
37ea0280df [thin_restore] first pass at btree_builder. 
No tests yet
 2020-10-26 12:05:27 +00:00
Joe Thornber
f60ae770c2 [thin_explore] Explore devices tree, including path support. 2020-10-15 11:53:09 +01:00
Joe Thornber
e9fbcc31de [thin_dump (rust)] First pass at thin_dump. 
Doesn't include --repair.

This includes <def> and <ref> sections for shared regions.
 2020-09-28 15:45:13 +01:00
Joe Thornber
66b6a1ba48 [thin_check (rust)] --superblock-only, --skip-mappings, INFO fields 2020-09-24 13:55:58 +01:00
Joe Thornber
b67b587a10 [thin_shrink] Add comment pointing people at Nikhil's PoC 2020-09-22 12:01:17 +01:00
Joe Thornber
819fc6d54c [thin_explore] accept a node path on the command line 
Helpful to examine thin_check failures.
 2020-09-22 10:47:04 +01:00
Joe Thornber
b193d19603 [thin_check (rust)] output complete node paths with errors. 
This can be used with thin_explore
 2020-09-18 11:16:09 +01:00
Joe Thornber
bc058f8baf [thin_check (rust)] BTree values must now implement Copy 2020-09-18 10:06:33 +01:00
Joe Thornber
bcfb9a73a1 [thin_explore] display ranges where possible 2020-09-18 09:12:51 +01:00
Joe Thornber
8493cf7081 [thin_explore] First code drop 2020-09-16 15:10:01 +01:00
Joe Thornber
67a54b4ebc [thin_check (rust)] add --auto-repair switch 2020-08-18 11:47:42 +01:00
Joe Thornber
239ae6b6ec [thin_check (rust)] factor out spawn_progress_thread 2020-08-18 09:48:51 +01:00
Joe Thornber
e8d7e5cf1e [thin_check (rust)] move report creation to top level 2020-08-17 16:05:06 +01:00
Joe Thornber
4e4b7ca2b1 [thin_check (rust)] add --sync-io flag 
Makes it easier to switch between engines
 2020-08-10 11:24:50 +01:00
Joe Thornber
1398cf31d1 [thin_check (Rust)] work in progress 2020-07-27 15:53:42 +01:00
Joe Thornber
3cf6307762 Merge branch 'master' into 2020-06-13-thin-check-rewrite 2020-07-27 15:53:26 +01:00
Joe Thornber
3618da3a12 [thin_shrink] Get test framework built 
This verifies data has been moved to the correct place.
 2020-06-29 10:49:40 +01:00
Joe Thornber
07da5704d5 [thin_shrink] add --no-copy 2020-06-26 08:57:26 +01:00
Joe Thornber
31abc468be [thin_shrink] Stub the copier 2020-06-25 15:28:45 +01:00
Joe Thornber
259eef9eee [thin_shrink] calculate remaps 2020-06-25 10:44:57 +01:00
Joe Thornber
3f1b776359 [thin_shrink] Add NoopVisitor 2020-06-24 13:55:08 +01:00
Joe Thornber
861b2f21ff [thin_shrink] Write thinp xml format reader and writer. 
Still need to tidy up the error handling.
 2020-06-24 13:32:47 +01:00
Joe Thornber
29d56f62a5 wip 2020-06-14 08:17:46 +01:00
Joe Thornber
7a85e47a20 [rust] squash clippy (linter) warnings 2020-06-10 16:46:38 +01:00
Joe Thornber
409a660082 [pack/unpack] Get the functional tests working again. 
There's some hard coded version numbers in the tests, but I'm
leaving for now since I'll rewrite in Rust to avoid too much
of a proliferation of languages.
 2020-06-09 13:03:39 +01:00
Joe Thornber
db5a71a53c [rust tools] Get the tools version from autoconf 2020-06-09 10:23:52 +01:00
Joe Thornber
61de3f9287 [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. 
The Rust implementation is multithreaded, performs better in general and
does custom compression of btree nodes to achieve much better compression
ratios.  unpack also checksums expanded metadata to validate it.

Format version has jumped to 3, no backwards compatibility, but I think
that's ok since we never made a release that contained the C++ version
of these tools.

Benchmarks
==========

On an 8 core, 16 hyperthread machine.

metadata 1G, full:

      Pack size    pack time     unpack time
------------------------------------------------------
C++      193M        50.3s          6.9s (no verify)
Rust      70M         1.4s          1.8s (verify)

metadata 16G, sparse:

       Pack size    pack time     unpack time
------------------------------------------------------
C++      21M          68s           1s   (no verify)
Rust      4M           8.6s         0.5s (verify)
 2020-06-09 09:15:00 +01:00