emo/thin-provisioning-tools
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,827 Commits 2 Branches 0 Tags
239ff7dfa172b04b8d2bf629ca2dc67b4d4ff1ac
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Joe Thornber d0675dd7bf Add missing header. 
So many errors from one issue
 2021-03-04 11:19:04 +00:00
Joe Thornber 7e869bb8e0 Code review of cache_dump/check 
Added support for metadata format2 to cache_dump.  Nothing tested.
 2021-03-04 11:13:08 +00:00
Joe Thornber e6c6275aea Code review of src/pdata/array* 2021-03-03 10:27:57 +00:00
Joe Thornber fdcc09c27e [cache_dump] Squash some clippy warnings 2021-03-03 09:48:15 +00:00
Joe Thornber 3cf0dba469 [cache_dump] fix how needs_check flag is checked 2021-03-03 09:47:42 +00:00
Joe Thornber 3d9b32e509 Merge branch 'main' of github.com:jthornber/thin-provisioning-tools into main 2021-02-26 15:26:33 +00:00
Joe Thornber d4299a00d0 Make pack_node() and calc_max_entries() public for dm-unit 2021-02-26 15:25:53 +00:00
Joe Thornber 441a000c52 Merge pull request #163 from mingnus/rust-cache-tools 
[cache (rust)] Fix merge conflicts
 2021-02-24 12:33:50 +00:00
Joe Thornber a4ba01cacd Merge pull request #160 from mingnus/rust-cache-tools 
First draft of cache_check in Rust
 2021-02-24 09:53:51 +00:00
Joe Thornber 78c93ce09b Merge pull request #162 from mingnus/main-thin-debug 
[dbg] Enhance debugging tools
 2021-02-22 08:28:07 +00:00
Joe Thornber 58cd881340 Fix regression where era_restore wouldn't work with devices. 
check_file_exists() had an extra parameter added with a default, which was
the wrong default for era_restore.
 2021-02-17 15:15:57 +00:00
Joe Thornber 763b2d14b2 [io_engine] Add Block::zeroed() constructor 2021-02-09 14:34:26 +00:00
Joe Thornber c3c6d37aea Fix a lot of clippy warnings 2021-02-08 10:38:21 +00:00
Joe Thornber e62fa3e835 Merge branch 'main' of github.com:jthornber/thin-provisioning-tools into main 2021-02-04 10:01:17 +00:00
Joe Thornber 6f30d17f04 version.rs includes VERSION directly. 
So you no longer need to run ./configure to build the rust tools
 2021-02-04 10:00:32 +00:00
Joe Thornber 8dbbe7fe27 Merge pull request #161 from mingnus/main-thin-debug 
[dbg] Integrate thin_debug, and add cache_debug
 2021-02-04 10:00:09 +00:00
Joe Thornber 9733ceb949 Merge pull request #158 from mingnus/main 
[cache_writeback] Support offset within the source and destination devices
 2021-01-20 11:06:21 +00:00
Joe Thornber 76ac202463 Merge pull request #157 from mingnus/main 
[build] Enable building the dev-tools for functional tests
 2021-01-12 09:02:43 +00:00
Joe Thornber 443b3c8f0b [io_engine (rust)] get_nr_blocks() wasn't handling block devices. 
Now calls file_utils::file_size()
 2020-12-02 15:20:14 +00:00
Joe Thornber ba7fd7bd2b [thin_check (rust)] Make --sync-io the default. 
For some systems you have to adjust the ulimits to get io_uring
to work, so we now default to using sync io.

Also added --async-io flag.
 2020-12-02 11:33:05 +00:00
Joe Thornber 0e4622f337 [Rust tools] squash lots of warnings 2020-12-01 11:50:32 +00:00
Joe Thornber 327fc80fb0 Merge branch 'main' into 2020-10-09-thin-restore-rewrite 2020-11-25 11:20:28 +00:00
Joe Thornber 83eea11d12 Merge pull request #155 from mingnus/thin-check-fix-metadata-leaks 
thin_check fixes
 2020-11-24 10:30:35 +00:00
Joe Thornber 5af95167b4 Merge pull request #154 from mingnus/main 
thin development tools update
 2020-11-24 09:31:23 +00:00
Joe Thornber 1ae62adec6 work in progress 2020-11-18 14:33:56 +00:00
Joe Thornber 37ea0280df [thin_restore] first pass at btree_builder. 
No tests yet
 2020-10-26 12:05:27 +00:00
Joe Thornber f60ae770c2 [thin_explore] Explore devices tree, including path support. 2020-10-15 11:53:09 +01:00
Joe Thornber c42b623e39 Merge branch '2020-08-13-thin-check-rewrite' into main 2020-10-09 11:21:12 +01:00
Joe Thornber e9fbcc31de [thin_dump (rust)] First pass at thin_dump. 
Doesn't include --repair.

This includes <def> and <ref> sections for shared regions.
 2020-09-28 15:45:13 +01:00
Joe Thornber a88ae3ca18 [thin_check (rust)] factor out device detail 2020-09-25 09:59:16 +01:00
Joe Thornber 66b6a1ba48 [thin_check (rust)] --superblock-only, --skip-mappings, INFO fields 2020-09-24 13:55:58 +01:00
Joe Thornber 34052c540c [thin_check (rust)] Reinstate walk_node_threaded 2020-09-24 09:40:38 +01:00
Joe Thornber b67b587a10 [thin_shrink] Add comment pointing people at Nikhil's PoC 2020-09-22 12:01:17 +01:00
Joe Thornber f4c3098e02 [thin_check (rust)] fix bug in key range splitting. 
Ranges were not being ommitted when a block was ommitted due to
being shared and already visited.
 2020-09-22 10:47:23 +01:00
Joe Thornber 819fc6d54c [thin_explore] accept a node path on the command line 
Helpful to examine thin_check failures.
 2020-09-22 10:47:04 +01:00
Joe Thornber b193d19603 [thin_check (rust)] output complete node paths with errors. 
This can be used with thin_explore
 2020-09-18 11:16:09 +01:00
Joe Thornber bc058f8baf [thin_check (rust)] BTree values must now implement Copy 2020-09-18 10:06:33 +01:00
Joe Thornber bcfb9a73a1 [thin_explore] display ranges where possible 2020-09-18 09:12:51 +01:00
Joe Thornber 8493cf7081 [thin_explore] First code drop 2020-09-16 15:10:01 +01:00
Joe Thornber 5168621f02 [thin_check (rust)] Optimise SyncIoEngine 
*_many no longer get/put for each block
 2020-09-02 13:28:16 +01:00
Joe Thornber 44142f657a [thin_check (rust)] Add error handling to io_engine interface 2020-09-02 12:57:47 +01:00
Joe Thornber b82307d8a5 [thin_check (rust)] drop O_DIRECT for the sync_io engine. 
O_DIRECT slows us down, and there's no correctness reason for having it.
 2020-08-21 11:39:41 +01:00
Joe Thornber cda92de441 [thin_check (rust)] Add a threaded version of btree walk. 
Bottom level mappings use this if there are few devices.  Performance
is a bit slower for io_uring, and much slower for sync io (which I think
is due to io scheduling which I can't do much about).
 2020-08-21 10:10:49 +01:00
Joe Thornber b01a0a46d1 [thin_metadata_pack/unpack] Use Vec::with_capacity() to avoid reallocs. 
Gives a small speed boost to both pack and unpack.
 2020-08-21 09:14:54 +01:00
Joe Thornber c9a759b4e8 [thin_check (rust)] Use vec::with_capacity() to avoid reallocations. 2020-08-21 09:00:21 +01:00
Joe Thornber 2cc2dffab5 [thin_check (rust)] Make NodeVisitor::visit non mut. 
Preparation for making btree_walk multithreaded
 2020-08-20 11:05:14 +01:00
Joe Thornber a1c206b774 [thin_check (rust)] NodeVisitor only needs to see leaf nodes 2020-08-20 10:55:38 +01:00
Joe Thornber 936e06e132 [thin_check (rust)] Remove some unused params from NodeVisitor::visit 2020-08-20 10:46:06 +01:00
Joe Thornber 1999343d2f [thin_check (rust)] squash a couple of warnings 2020-08-20 10:33:02 +01:00
Joe Thornber 0372e689e5 Merge branch '2020-08-13-thin-check-rewrite' 2020-08-19 14:32:40 +01:00
Joe Thornber 7834d661e2 [thin_check (rust)] auto repair space map leaks 2020-08-19 14:31:01 +01:00
Joe Thornber cdd0beb527 [thin_check (rust)] Change io_engine trait to use slices rather than Vecs 2020-08-18 12:57:05 +01:00
Joe Thornber 2aa6859502 [thin_check (rust)] add write support to io_engine 2020-08-18 12:52:16 +01:00
Joe Thornber 67a54b4ebc [thin_check (rust)] add --auto-repair switch 2020-08-18 11:47:42 +01:00
Joe Thornber 8eec84fbec [thin_check (rust)] introduce ASpaceMap type alias 2020-08-18 11:06:15 +01:00
Joe Thornber 04f3ba5a33 [thin_check (rust)] Pass ctx to check_space_map 2020-08-18 10:59:04 +01:00
Joe Thornber 4beb2db337 [thin_check (rust)] Factor out check_mapping_bottom_level 2020-08-18 10:53:11 +01:00
Joe Thornber 239ae6b6ec [thin_check (rust)] factor out spawn_progress_thread 2020-08-18 09:48:51 +01:00
Joe Thornber 65799f9a14 Merge pull request #150 from mingnus/thin-check-fix-metadata-leaks 
Add space-map counting tests
 2020-08-18 08:50:32 +01:00
Joe Thornber e8d7e5cf1e [thin_check (rust)] move report creation to top level 2020-08-17 16:05:06 +01:00
Joe Thornber 5743e3e9ba [thin_check (rust)] Add title method to reports 2020-08-17 15:36:21 +01:00
Joe Thornber 9995751dde [thin_check (rust)] Provide 3 different report types. 
ProgressBar for interactive use.
Simple for when not a tty
and quiet for -q
 2020-08-17 13:10:32 +01:00
Joe Thornber 7466cd7182 [functional-tests (rust)] TestDir now creates dir in ./ 
O_DIRECT doesn't work with /tmp/
 2020-08-14 10:54:31 +01:00
Joe Thornber e1cfc3866b [thin_check (rust)] Mappings top level weren't being ref counted 2020-08-13 14:43:19 +01:00
Joe Thornber 092447d17a [thin_check (rust)] remove some dead code 2020-08-13 14:30:04 +01:00
Joe Thornber 2fa732a93c [functional tests] Port some of Hank's sh tests to Rust. 
Run with 'cargo test'
 2020-08-13 14:20:29 +01:00
Joe Thornber a8a2f560ec [build] add symlinks for thin_generate_{mappings/damage} 2020-08-13 13:47:51 +01:00
Joe Thornber b0e7520fbf [thin/superblock (rust)] Unpack flags 2020-08-13 13:46:07 +01:00
Joe Thornber c254ebe384 [functional-tests] Port 3 --auto-repair tests from Scheme to Rust 2020-08-12 13:03:01 +01:00
Joe Thornber 5a16f21199 [functional tests] Fixup a few tests that broke when we bumped the version nr. 2020-08-12 12:33:16 +01:00
Joe Thornber bf202d076b Merge branch '2020-06-13-thin-check-rewrite' 2020-08-12 11:18:28 +01:00
Joe Thornber d6d93c3157 bump version to 0.9.0 2020-08-12 10:46:26 +01:00
Joe Thornber 2860b4c0b1 [thin_check] rename --fix-metadata-leaks to --auto-repair 
This flag will cover all trivial fixes, such as data leaks.
 2020-08-12 10:45:20 +01:00
Joe Thornber ebcfc322c0 [build] Update CHANGES file 2020-08-12 10:35:03 +01:00
Joe Thornber d9965a3906 Merge pull request #149 from mingnus/thin-check-fix-metadata-leaks 
thin_check enhancements
 2020-08-12 10:28:26 +01:00
Joe Thornber afa3f2f04d [thin_check (rust)] Rename Spinner -> Reporter 2020-08-12 10:25:06 +01:00
Joe Thornber 544335ae4a [thin_check (rust)] Send all reporting through the Spinner. 
This means the spinner doesn't overwrite messages.
 2020-08-12 09:35:21 +01:00
Joe Thornber 3757e1d947 [thin_check (rust)] check metadata space map 2020-08-12 08:02:29 +01:00
Joe Thornber e65d2dec6f [thin_check (rust)] Add progress bar 2020-08-11 13:44:33 +01:00
Joe Thornber 34425521e2 [thin_check (rust)] change BTreeWalker to use a space map rather than seen bitset 2020-08-11 10:50:43 +01:00
Joe Thornber 50bde693a1 [thin_check (rust)] Factor out pdata/unpack 2020-08-10 15:42:10 +01:00
Joe Thornber 55ee4bfad8 [thin_check (rust)] replace IndexVisitor with a call to btree_to_map 2020-08-10 14:56:39 +01:00
Joe Thornber e28c602c3d [thin_check (rust)] factor out btree_to_map() fn 2020-08-10 14:45:35 +01:00
Joe Thornber cbc9c2c72a [thin_check (rust)] Improve data_sm handling 2020-08-10 12:56:41 +01:00
Joe Thornber d5444d2255 [thin_check (rust)] sm bitmap entries were being unpacked incorrectly. 2020-08-10 12:55:05 +01:00
Joe Thornber b915257e10 [thin_check (rust)] Fix race in btree walking. 
The seen bitset was locked once to test, and separately to insert.
 2020-08-10 12:30:12 +01:00
Joe Thornber 4e4b7ca2b1 [thin_check (rust)] add --sync-io flag 
Makes it easier to switch between engines
 2020-08-10 11:24:50 +01:00
Joe Thornber 0f865856ed [thin_check (rust)] Improve SyncIoEngine. 
Now opens the file multiple times so different threads can do io in parallel.
 2020-08-10 10:44:47 +01:00
Joe Thornber f0df17af9e [thin_check (rust)] Get SyncIoEngine working again. 2020-08-10 08:59:02 +01:00
Joe Thornber 08e3ea948e [thin_check (rust)] rename block_manager.rs -> io_engine.rs 2020-08-10 08:29:32 +01:00
Joe Thornber fd0c0ffc1d [thin_check (rust)] data space map now checked. 2020-08-08 16:42:32 +01:00
Joe Thornber 4054b1be4c [thin_check (rust)] Switch to BTreeMap. 
It's faster.
 2020-08-08 14:58:13 +01:00
Joe Thornber 1e4a038b41 [thin_check (rust)] Reimplement CoreSpaceMap 
We now use a simple vector of elements that can hold 'nr thin devs'.  Much faster.
 2020-08-08 13:29:30 +01:00
Joe Thornber 7cf239b878 [thin_check (rust)] speed up CoreSpaceMap 2020-08-08 12:36:13 +01:00
Joe Thornber ec8f7b7fa8 [thin_check (rust)] Keep track of data block ref counts 
as we walk the mapping tree.
 2020-08-08 09:54:16 +01:00
Joe Thornber 8f76371bb2 [functional-tests] Fix clippy warnings 2020-08-07 15:41:21 +01:00
Joe Thornber fa4ea3e2d9 [functional-tests] port some of the cache_check tests to Rust 2020-08-07 14:30:00 +01:00
Joe Thornber 4a0582bb5d [thin_check (rust)] start decoding the space maps. 2020-08-06 07:51:48 +01:00
Joe Thornber 904d9b0c84 [functional-tests] port thin_metadata_unpack tests to Rust. 
cargo test
 2020-08-05 10:00:44 +01:00
Joe Thornber d2678fdf27 [functional-tests] port thin_metadata_pack tests to Rust. 
cargo test
 2020-08-05 09:09:18 +01:00