Joe Thornber
fdcc09c27e
[cache_dump] Squash some clippy warnings
2021-03-03 09:48:15 +00:00
Joe Thornber
3cf0dba469
[cache_dump] fix how needs_check flag is checked
2021-03-03 09:47:42 +00:00
Ming-Hung Tsai
eb3d181f95
[cache_dump (rust)] First draft of cache_dump
2021-03-03 12:27:51 +08:00
Ming-Hung Tsai
fde0e0e2b8
[cache (rust)] Add Mapping::is_dirty()
2021-03-03 12:14:46 +08:00
Ming-Hung Tsai
74fcb9d505
[cache (rust)] Fix data types
2021-02-26 23:31:12 +08:00
Joe Thornber
3d9b32e509
Merge branch 'main' of github.com:jthornber/thin-provisioning-tools into main
2021-02-26 15:26:33 +00:00
Joe Thornber
d4299a00d0
Make pack_node() and calc_max_entries() public for dm-unit
2021-02-26 15:25:53 +00:00
Ming-Hung Tsai
a7ecfba2b4
[cache (rust)] Fix merge conflicts
2021-02-24 20:21:10 +08:00
Joe Thornber
a4ba01cacd
Merge pull request #160 from mingnus/rust-cache-tools
...
First draft of cache_check in Rust
2021-02-24 09:53:51 +00:00
Ming-Hung Tsai
2bb3bf65b7
[cache_check (rust)] Implement basic functions
2021-02-24 17:39:11 +08:00
Joe Thornber
763b2d14b2
[io_engine] Add Block::zeroed() constructor
2021-02-09 14:34:26 +00:00
Joe Thornber
c3c6d37aea
Fix a lot of clippy warnings
2021-02-08 10:38:21 +00:00
Joe Thornber
6f30d17f04
version.rs includes VERSION directly.
...
So you no longer need to run ./configure to build the rust tools
2021-02-04 10:00:32 +00:00
Ming-Hung Tsai
faa371c208
[cache (rust)] Implement cache primitives
2021-02-03 15:18:47 +08:00
Ming-Hung Tsai
087a4711a5
[array (rust)] Implement array basis
2021-02-03 15:18:41 +08:00
Ming-Hung Tsai
ca7e79a828
Merge commit 'b67b587' into main
2021-01-12 02:50:12 +08:00
Joe Thornber
04e0eb3a66
[thin_restore (rust)] rewrite the btree_builder
...
Now copes with adding shared leaves.
2020-12-09 13:22:32 +00:00
Joe Thornber
443b3c8f0b
[io_engine (rust)] get_nr_blocks() wasn't handling block devices.
...
Now calls file_utils::file_size()
2020-12-02 15:20:14 +00:00
Joe Thornber
ba7fd7bd2b
[thin_check (rust)] Make --sync-io the default.
...
For some systems you have to adjust the ulimits to get io_uring
to work, so we now default to using sync io.
Also added --async-io flag.
2020-12-02 11:33:05 +00:00
Joe Thornber
0e4622f337
[Rust tools] squash lots of warnings
2020-12-01 11:50:32 +00:00
Joe Thornber
1ae62adec6
work in progress
2020-11-18 14:33:56 +00:00
Joe Thornber
37ea0280df
[thin_restore] first pass at btree_builder.
...
No tests yet
2020-10-26 12:05:27 +00:00
Joe Thornber
f60ae770c2
[thin_explore] Explore devices tree, including path support.
2020-10-15 11:53:09 +01:00
Joe Thornber
e9fbcc31de
[thin_dump (rust)] First pass at thin_dump.
...
Doesn't include --repair.
This includes <def> and <ref> sections for shared regions.
2020-09-28 15:45:13 +01:00
Joe Thornber
a88ae3ca18
[thin_check (rust)] factor out device detail
2020-09-25 09:59:16 +01:00
Joe Thornber
66b6a1ba48
[thin_check (rust)] --superblock-only, --skip-mappings, INFO fields
2020-09-24 13:55:58 +01:00
Joe Thornber
34052c540c
[thin_check (rust)] Reinstate walk_node_threaded
2020-09-24 09:40:38 +01:00
Joe Thornber
b67b587a10
[thin_shrink] Add comment pointing people at Nikhil's PoC
2020-09-22 12:01:17 +01:00
Joe Thornber
f4c3098e02
[thin_check (rust)] fix bug in key range splitting.
...
Ranges were not being ommitted when a block was ommitted due to
being shared and already visited.
2020-09-22 10:47:23 +01:00
Joe Thornber
819fc6d54c
[thin_explore] accept a node path on the command line
...
Helpful to examine thin_check failures.
2020-09-22 10:47:04 +01:00
Joe Thornber
b193d19603
[thin_check (rust)] output complete node paths with errors.
...
This can be used with thin_explore
2020-09-18 11:16:09 +01:00
Joe Thornber
bc058f8baf
[thin_check (rust)] BTree values must now implement Copy
2020-09-18 10:06:33 +01:00
Joe Thornber
bcfb9a73a1
[thin_explore] display ranges where possible
2020-09-18 09:12:51 +01:00
Joe Thornber
8493cf7081
[thin_explore] First code drop
2020-09-16 15:10:01 +01:00
Joe Thornber
5168621f02
[thin_check (rust)] Optimise SyncIoEngine
...
*_many no longer get/put for each block
2020-09-02 13:28:16 +01:00
Joe Thornber
44142f657a
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 12:57:47 +01:00
Joe Thornber
b82307d8a5
[thin_check (rust)] drop O_DIRECT for the sync_io engine.
...
O_DIRECT slows us down, and there's no correctness reason for having it.
2020-08-21 11:39:41 +01:00
Joe Thornber
cda92de441
[thin_check (rust)] Add a threaded version of btree walk.
...
Bottom level mappings use this if there are few devices. Performance
is a bit slower for io_uring, and much slower for sync io (which I think
is due to io scheduling which I can't do much about).
2020-08-21 10:10:49 +01:00
Joe Thornber
b01a0a46d1
[thin_metadata_pack/unpack] Use Vec::with_capacity() to avoid reallocs.
...
Gives a small speed boost to both pack and unpack.
2020-08-21 09:14:54 +01:00
Joe Thornber
c9a759b4e8
[thin_check (rust)] Use vec::with_capacity() to avoid reallocations.
2020-08-21 09:00:21 +01:00
Joe Thornber
2cc2dffab5
[thin_check (rust)] Make NodeVisitor::visit non mut.
...
Preparation for making btree_walk multithreaded
2020-08-20 11:05:14 +01:00
Joe Thornber
a1c206b774
[thin_check (rust)] NodeVisitor only needs to see leaf nodes
2020-08-20 10:55:38 +01:00
Joe Thornber
936e06e132
[thin_check (rust)] Remove some unused params from NodeVisitor::visit
2020-08-20 10:46:06 +01:00
Joe Thornber
1999343d2f
[thin_check (rust)] squash a couple of warnings
2020-08-20 10:33:02 +01:00
Joe Thornber
7834d661e2
[thin_check (rust)] auto repair space map leaks
2020-08-19 14:31:01 +01:00
Joe Thornber
cdd0beb527
[thin_check (rust)] Change io_engine trait to use slices rather than Vecs
2020-08-18 12:57:05 +01:00
Joe Thornber
2aa6859502
[thin_check (rust)] add write support to io_engine
2020-08-18 12:52:16 +01:00
Joe Thornber
67a54b4ebc
[thin_check (rust)] add --auto-repair switch
2020-08-18 11:47:42 +01:00
Joe Thornber
8eec84fbec
[thin_check (rust)] introduce ASpaceMap type alias
2020-08-18 11:06:15 +01:00
Joe Thornber
04f3ba5a33
[thin_check (rust)] Pass ctx to check_space_map
2020-08-18 10:59:04 +01:00
Joe Thornber
4beb2db337
[thin_check (rust)] Factor out check_mapping_bottom_level
2020-08-18 10:53:11 +01:00
Joe Thornber
239ae6b6ec
[thin_check (rust)] factor out spawn_progress_thread
2020-08-18 09:48:51 +01:00
Joe Thornber
e8d7e5cf1e
[thin_check (rust)] move report creation to top level
2020-08-17 16:05:06 +01:00
Joe Thornber
5743e3e9ba
[thin_check (rust)] Add title method to reports
2020-08-17 15:36:21 +01:00
Joe Thornber
9995751dde
[thin_check (rust)] Provide 3 different report types.
...
ProgressBar for interactive use.
Simple for when not a tty
and quiet for -q
2020-08-17 13:10:32 +01:00
Joe Thornber
e1cfc3866b
[thin_check (rust)] Mappings top level weren't being ref counted
2020-08-13 14:43:19 +01:00
Joe Thornber
092447d17a
[thin_check (rust)] remove some dead code
2020-08-13 14:30:04 +01:00
Joe Thornber
2fa732a93c
[functional tests] Port some of Hank's sh tests to Rust.
...
Run with 'cargo test'
2020-08-13 14:20:29 +01:00
Joe Thornber
b0e7520fbf
[thin/superblock (rust)] Unpack flags
2020-08-13 13:46:07 +01:00
Joe Thornber
afa3f2f04d
[thin_check (rust)] Rename Spinner -> Reporter
2020-08-12 10:25:06 +01:00
Joe Thornber
544335ae4a
[thin_check (rust)] Send all reporting through the Spinner.
...
This means the spinner doesn't overwrite messages.
2020-08-12 09:35:21 +01:00
Joe Thornber
3757e1d947
[thin_check (rust)] check metadata space map
2020-08-12 08:02:29 +01:00
Joe Thornber
e65d2dec6f
[thin_check (rust)] Add progress bar
2020-08-11 13:44:33 +01:00
Joe Thornber
34425521e2
[thin_check (rust)] change BTreeWalker to use a space map rather than seen bitset
2020-08-11 10:50:43 +01:00
Joe Thornber
50bde693a1
[thin_check (rust)] Factor out pdata/unpack
2020-08-10 15:42:10 +01:00
Joe Thornber
55ee4bfad8
[thin_check (rust)] replace IndexVisitor with a call to btree_to_map
2020-08-10 14:56:39 +01:00
Joe Thornber
e28c602c3d
[thin_check (rust)] factor out btree_to_map() fn
2020-08-10 14:45:35 +01:00
Joe Thornber
cbc9c2c72a
[thin_check (rust)] Improve data_sm handling
2020-08-10 12:56:41 +01:00
Joe Thornber
d5444d2255
[thin_check (rust)] sm bitmap entries were being unpacked incorrectly.
2020-08-10 12:55:05 +01:00
Joe Thornber
b915257e10
[thin_check (rust)] Fix race in btree walking.
...
The seen bitset was locked once to test, and separately to insert.
2020-08-10 12:30:12 +01:00
Joe Thornber
4e4b7ca2b1
[thin_check (rust)] add --sync-io flag
...
Makes it easier to switch between engines
2020-08-10 11:24:50 +01:00
Joe Thornber
0f865856ed
[thin_check (rust)] Improve SyncIoEngine.
...
Now opens the file multiple times so different threads can do io in parallel.
2020-08-10 10:44:47 +01:00
Joe Thornber
f0df17af9e
[thin_check (rust)] Get SyncIoEngine working again.
2020-08-10 08:59:02 +01:00
Joe Thornber
08e3ea948e
[thin_check (rust)] rename block_manager.rs -> io_engine.rs
2020-08-10 08:29:32 +01:00
Joe Thornber
fd0c0ffc1d
[thin_check (rust)] data space map now checked.
2020-08-08 16:42:32 +01:00
Joe Thornber
4054b1be4c
[thin_check (rust)] Switch to BTreeMap.
...
It's faster.
2020-08-08 14:58:13 +01:00
Joe Thornber
1e4a038b41
[thin_check (rust)] Reimplement CoreSpaceMap
...
We now use a simple vector of elements that can hold 'nr thin devs'. Much faster.
2020-08-08 13:29:30 +01:00
Joe Thornber
7cf239b878
[thin_check (rust)] speed up CoreSpaceMap
2020-08-08 12:36:13 +01:00
Joe Thornber
ec8f7b7fa8
[thin_check (rust)] Keep track of data block ref counts
...
as we walk the mapping tree.
2020-08-08 09:54:16 +01:00
Joe Thornber
8f76371bb2
[functional-tests] Fix clippy warnings
2020-08-07 15:41:21 +01:00
Joe Thornber
fa4ea3e2d9
[functional-tests] port some of the cache_check tests to Rust
2020-08-07 14:30:00 +01:00
Joe Thornber
4a0582bb5d
[thin_check (rust)] start decoding the space maps.
2020-08-06 07:51:48 +01:00
Joe Thornber
197e4ffbfd
[thin_check (rust)] Rename ValueType trait to Unpack
2020-08-05 08:01:02 +01:00
Joe Thornber
1d44025584
[thin_check (rust)] Walk the top level and bottom level of the mapping tree separately
2020-08-04 12:11:36 +01:00
Joe Thornber
f56ea2d031
[thin_check (rust)] walk devices tree.
2020-08-03 16:22:08 +01:00
Joe Thornber
1368227a71
[thin_check (rust)] add btree node checks
2020-08-03 15:04:59 +01:00
Joe Thornber
cc2582b8b1
[thin_check (rust)] factor out pdata/btree.rs
2020-08-03 12:37:32 +01:00
Joe Thornber
7243f95380
[thin_check (rust)] Drop nr threads down to 4
...
We get very little benefit from threads atm. Need to improve IO handling first.
2020-07-30 10:17:36 +01:00
Joe Thornber
de172147d3
[thin_check (rust)] squash a lot of warnings
2020-07-30 10:12:51 +01:00
Joe Thornber
f7623e6264
[thin_check (rust)] remove spurious mutex.
2020-07-30 09:59:02 +01:00
Joe Thornber
8146fba9d2
[thin_check (rust)] Move mutex inside IoEngines.
...
Makes it easier to share an engine between different threads.
2020-07-30 08:47:16 +01:00
Joe Thornber
4f120911d2
[thin_check (rust)] First attempt at multithreading the tree walking.
...
Still using a single io engine, so little benefit.
2020-07-29 16:38:52 +01:00
Joe Thornber
d5597d5d36
[thin_check (rust)] Factor out tree walking code
2020-07-29 11:12:03 +01:00
Joe Thornber
e9abdd9c88
[thin_check (rust)] Switch to a different io_uring crate.
...
This one works.
2020-07-28 12:57:30 +01:00
Joe Thornber
a90294e279
[thin_check (rust)] read many blocks at once.
...
We need to switch to io_uring to really get the benefit of this.
2020-07-28 11:45:25 +01:00
Joe Thornber
062a1b8a2c
[thin_check (rust)] Walk mapping tree.
...
Sync IO, no checks beyond checksumming.
2020-07-28 10:51:48 +01:00
Joe Thornber
1398cf31d1
[thin_check (Rust)] work in progress
2020-07-27 15:53:42 +01:00
Joe Thornber
3cf6307762
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 15:53:26 +01:00
Joe Thornber
e9055f5d7d
Merge branch 'master' of github.com:jthornber/thin-provisioning-tools
2020-07-22 16:09:33 +01:00
Joe Thornber
b7e02d0ae4
[thin_metadata_pack] FIx bug where pack would try and read too much.
2020-07-09 13:53:46 +01:00