emo/thin-provisioning-tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,973 Commits 2 Branches 0 Tags 3.7 MiB
9253117132
Commit Graph

1973 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ming-Hung Tsai
9253117132 [tests] Add cache fixtures 
- Generate temp xml and metadata files
- Correct existing tests to use cache fixtures
- Fix cache xml generation
 2021-09-16 21:55:46 +08:00
Ming-Hung Tsai
0acc57d17f [thin_restore (rust)] Fix errors in tests 
- Move error messages to stderr (fixes unwritable_output_file,
  input_file_not_found, and missing_input_option)
- Validate XML structures implicitly (fixes corrupted_input_data)
- Check output file size (fixes tiny_output_file)
- Provide XML error context
- Fix mk_valid_input()
 2021-09-16 21:55:46 +08:00
Ming-Hung Tsai
aaa84aa35a [tests] Enable tests for rust thin_repair 2021-09-16 21:55:46 +08:00
Ming-Hung Tsai
ea8dcb54d0 [tests] Fix the file size for testing 2021-09-16 21:55:46 +08:00
Ming-Hung Tsai
737cf2f67d [tests] Refine trait names 2021-09-16 21:55:20 +08:00
Ming-Hung Tsai
47d39d1efa [tests] Do not assume no stderr with thin_dump 2021-09-10 23:50:36 +08:00
Joe Thornber
9aa36f017a
Merge pull request #188 from mingnus/2021-08-05-thin-repair-find-roots 
Support broken superblock in thin_repair
 2021-09-10 13:59:16 +01:00
Ming-Hung Tsai
0d87619a93 [thin_repair/thin_dump (rust)] Show repairing progress 2021-08-27 17:30:56 +08:00
Ming-Hung Tsai
71d47ef58b [thin/cache (rust)] Validate superblock checksum 2021-08-27 11:17:27 +08:00
Ming-Hung Tsai
8a822cebec [thin_repair/thin_dump (rust)] Support rebuilding superblock 2021-08-27 02:17:57 +08:00
Ming-Hung Tsai
213442bffd [all (rust)] Fix cosmetic names for argument values 2021-08-27 02:17:57 +08:00
Ming-Hung Tsai
434d24d10a [thin_repair (rust)] Support setting missing superblock fields automatically 
While rebuilding a superblock, the transaction_id, data_block_size, and
nr_data_blocks could be inherited from the current superblock if available
(assumed it's a multiple-activated copy, so supposed partially valid),
or derived from the probed data mappings and device details, that saves
the hassle of manual input.

Note that either the current superblock or user-overrides must be compatible
with the probed values, thus the filters are applied.
 2021-08-27 02:17:57 +08:00
Ming-Hung Tsai
5dad1097c3 [thin_repair (rust)] Add searching for missing roots 
Based on the method of commit 9e20465
 2021-08-27 02:17:57 +08:00
Ming-Hung Tsai
b7132440d0 [space_map (rust)] Encapsulate implementations 2021-08-27 00:08:21 +08:00
Joe Thornber
15189aa28b
Merge pull request #187 from mingnus/2021-08-24-thin-repair-find-roots-fixes 
Fix issues in finding roots
 2021-08-26 16:23:24 +01:00
Ming-Hung Tsai
e6f17d4b4f [thin_repair/thin_dump] Exclude unwanted btree nodes 2021-08-24 22:26:22 +08:00
Ming-Hung Tsai
d2d6ab7926 [thin_repair/thin_dump] Check consistency of thin_ids before running a regular dump 2021-08-24 22:26:22 +08:00
Ming-Hung Tsai
d1e8168fb6 [thin_repair/thin_dump] Change the label type for empty leaves 
Empty leaves now are treated as bottom-level leaves, so that empty
devices could be recovered.
 2021-08-24 22:26:22 +08:00
Ming-Hung Tsai
c2e6db74b9 [thin_repair/thin_dump] Remove unused function parameter 2021-08-24 22:26:22 +08:00
Ming-Hung Tsai
c286041f25 [thin_repair/thin_dump] Fix sorting of data mapping candidates 
- Fix the references for sorting. The timestamp statistics is stored
  in node_info corresponding to the second element.
- Fix the timestamp comparison routine. The mapping root with more recent
  blocks should have higher priority.
 2021-08-24 22:26:15 +08:00
Joe Thornber
b58e42bb95
Merge pull request #185 from mingnus/2021-07-29-btree-builder-fixes 
Fix reference counting in btree construction
 2021-08-06 08:35:46 +01:00
Ming-Hung Tsai
ec44d043e8 [space_map (rust)] Restrict metadata space map size 2021-08-06 13:22:13 +08:00
Ming-Hung Tsai
c167403212 [space_map (rust)] Fix the maximum value of reference counts 2021-08-06 13:22:13 +08:00
Ming-Hung Tsai
e158dc7601 [btree_builder] Fix issues with under populated shared nodes 
A pre-built node could be under populated (less than half-full) due to
following reasons:

- A single shared leaf generated by dm-thin key removal. The residency
  could drop below 50%, until it reaches the merge threshold (33% or 44%,
  depends on its context).
- A shared root, which could have any possible nr_entries.
- Underfull shared nodes (less than 33% residency) caused by kernel issues.

To avoid producing under populated nodes, those kinds of pre-built nodes,
except the roots, will be merged into their siblings.
 2021-08-06 13:22:13 +08:00
Ming-Hung Tsai
bd39b570ef [btree_builder] Fix reference counts of unshifted child values 
The ref counts now are handled in a straightforward method:
The pre-built subtrees serve as snapshots of potentially shared nodes.
They hold an initial ref count on the pre-built nodes and their children.
The temporary ref counts are later dropped at the end of device building.

This way fixes the ref counts of unshifted nodes, thus the 'reserve'
operation introduced by commit 6d16c58 is reverted.
 2021-08-05 23:41:23 +08:00
Ming-Hung Tsai
052c9f90ea [thin/cache_repair (rust)] Fix file open options 2021-08-05 23:41:23 +08:00
Joe Thornber
63b8b9fc40 Clarify a namespace. 
See issue 184.
 2021-08-04 13:46:30 +01:00
Joe Thornber
03d0651d0a
Merge pull request #183 from mingnus/2021-07-08-thin-repair 
First pass at thin_repair and cache_repair
 2021-07-29 13:22:36 +01:00
Ming-Hung Tsai
3dc01bf962 [tests] Use IntoIterator on the array argument (requires Rust 1.53) 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
7239204b01 [cache_repair (rust)] First pass at cache_repair 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
4b9766846e [cache (rust)] Prepare for cache_repair 
- Adapt function interfaces for repairing purpose
- Finalize the metadata in the Restorer
- Make the Restorer public
 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
85581cdf5a [thin_repair (rust)] First pass at thin_repair 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
dccd844714 [thin_restore (rust)] Prepare for thin_repair 
- Finalize the metadata in the Restorer
- Make the Restorer public
 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
d42bde371f [thin_dump (rust)] Prepare for thin_repair 
- Adapt function interfaces for repairing purpose
- Pull out reuseable structures
 2021-07-27 23:51:20 +08:00
Ming-Hung Tsai
56d7da66aa [thin_dump (rust)] Tidy up 
- Do not use shared internals as Defs
- Build Defs with common leaf sequences only, that could forms
  more common Defs between partially shared subtrees.
 2021-07-27 23:45:02 +08:00
Ming-Hung Tsai
7002a8ae8d [all (rust)] Pull out structures for intermediate representation 
Also fix the data type for thin timestamp
 2021-07-21 18:46:06 +08:00
Joe Thornber
6d31d4def2
Merge pull request #182 from mingnus/2021-06-23-functional-tests 
Clean up function test commons
 2021-07-21 11:27:05 +01:00
Ming-Hung Tsai
222b4f0902 [tests] Do not perform Path to str conversion for compatibility 2021-07-21 18:00:20 +08:00
Ming-Hung Tsai
16190f0f9a [tests] Pull out common submodules 
New modules: fixture, process, program, target, and thin
 2021-07-21 17:50:34 +08:00
Ming-Hung Tsai
66b49e6f3d [tests] Change the path type to OsString for compatibility 2021-07-21 17:27:13 +08:00
Joe Thornber
b92151d527
Merge pull request #181 from mingnus/2021-06-23-functional-tests 
Functional tests revised
 2021-07-19 14:01:37 +01:00
Ming-Hung Tsai
a50c9d97e2 [thin_metadata_unpack] Fix truncated output file on invalid input 
Check the input header before creating or truncating the output file
 2021-07-09 01:22:22 +08:00
Ming-Hung Tsai
6cecf0f673 [file_utils] Check the file type to prevent unexpected writes by thin_repair 2021-07-09 01:22:22 +08:00
Ming-Hung Tsai
87ada9b493 [tests] Add basic tests for cache_dump 2021-07-09 01:22:22 +08:00
Ming-Hung Tsai
f395bab7be [tests] Use traits to specify test parameters 
To deal with variety in target attributes and their expected outputs,
the test parameters are categorized into traits, thus the test program
could define test parameters in a more structured way, without having
to pass multiple tightly-coupled parameters to test functions.
 2021-07-09 01:22:22 +08:00
Ming-Hung Tsai
12ef69c31b [tests] Pull out common tests on i/o options into reusable modules 
- Introduce modules for testing input/output options
- Provide macros for generating test cases
- Hide details of subprocess execution
 2021-07-08 22:02:44 +08:00
Ming-Hung Tsai
6660fde3c4 [tests] Refine the test naming and error messages 
- Make the naming of test cases less ambiguous, e.g., rename
  "missing_input_file" to "missing_input_arg" or "input_file_not_found"
- Unify the error messages on input/output options
 2021-07-08 01:05:15 +08:00
Ming-Hung Tsai
d00388f68a [thin_shrink] Support short options 2021-07-07 16:06:04 +08:00
Ming-Hung Tsai
1526ab3472 [all] Apply cargo fmt, and fix clippy warning of branches_sharing_code 2021-07-02 16:17:52 +08:00
Joe Thornber
5ac9ae2dae
Merge pull request #180 from mingnus/2021-06-25-update-deps 
Update config scripts and Rust dependencies
 2021-06-28 15:44:59 +01:00