Commit Graph

1941 Commits

Author SHA1 Message Date
dccd844714 [thin_restore (rust)] Prepare for thin_repair
- Finalize the metadata in the Restorer
- Make the Restorer public
2021-07-27 23:51:20 +08:00
d42bde371f [thin_dump (rust)] Prepare for thin_repair
- Adapt function interfaces for repairing purpose
- Pull out reuseable structures
2021-07-27 23:51:20 +08:00
56d7da66aa [thin_dump (rust)] Tidy up
- Do not use shared internals as Defs
- Build Defs with common leaf sequences only, that could forms
  more common Defs between partially shared subtrees.
2021-07-27 23:45:02 +08:00
7002a8ae8d [all (rust)] Pull out structures for intermediate representation
Also fix the data type for thin timestamp
2021-07-21 18:46:06 +08:00
6d31d4def2 Merge pull request #182 from mingnus/2021-06-23-functional-tests
Clean up function test commons
2021-07-21 11:27:05 +01:00
222b4f0902 [tests] Do not perform Path to str conversion for compatibility 2021-07-21 18:00:20 +08:00
16190f0f9a [tests] Pull out common submodules
New modules: fixture, process, program, target, and thin
2021-07-21 17:50:34 +08:00
66b49e6f3d [tests] Change the path type to OsString for compatibility 2021-07-21 17:27:13 +08:00
b92151d527 Merge pull request #181 from mingnus/2021-06-23-functional-tests
Functional tests revised
2021-07-19 14:01:37 +01:00
a50c9d97e2 [thin_metadata_unpack] Fix truncated output file on invalid input
Check the input header before creating or truncating the output file
2021-07-09 01:22:22 +08:00
6cecf0f673 [file_utils] Check the file type to prevent unexpected writes by thin_repair 2021-07-09 01:22:22 +08:00
87ada9b493 [tests] Add basic tests for cache_dump 2021-07-09 01:22:22 +08:00
f395bab7be [tests] Use traits to specify test parameters
To deal with variety in target attributes and their expected outputs,
the test parameters are categorized into traits, thus the test program
could define test parameters in a more structured way, without having
to pass multiple tightly-coupled parameters to test functions.
2021-07-09 01:22:22 +08:00
12ef69c31b [tests] Pull out common tests on i/o options into reusable modules
- Introduce modules for testing input/output options
- Provide macros for generating test cases
- Hide details of subprocess execution
2021-07-08 22:02:44 +08:00
6660fde3c4 [tests] Refine the test naming and error messages
- Make the naming of test cases less ambiguous, e.g., rename
  "missing_input_file" to "missing_input_arg" or "input_file_not_found"
- Unify the error messages on input/output options
2021-07-08 01:05:15 +08:00
d00388f68a [thin_shrink] Support short options 2021-07-07 16:06:04 +08:00
1526ab3472 [all] Apply cargo fmt, and fix clippy warning of branches_sharing_code 2021-07-02 16:17:52 +08:00
5ac9ae2dae Merge pull request #180 from mingnus/2021-06-25-update-deps
Update config scripts and Rust dependencies
2021-06-28 15:44:59 +01:00
8bb4aaef8f [build] Create the destdir for solely installing rust tools 2021-06-28 08:19:30 +08:00
725ad1d9b0 [manpages] Make the footer backward compatible 2021-06-28 08:19:30 +08:00
857e3a7d3d [manpages] Make the header and footer backward compatible 2021-06-28 08:19:30 +08:00
6f1a6a59dd [manpages] Update txt2man to 1.7.1
Signed-off-by: Kay Lin <i@v2bv.net>
2021-06-26 13:40:56 +08:00
2d201c5483 [build] Update config.guess and config.sub to version 2021-05-24 2021-06-26 13:38:56 +08:00
b7c3969747 [build] Update dependencies
- Update fixedbitset to 0.4
- Update indicatif to 0.16
- Update libc to 0.2.*
- Update nix to 0.21
- Update nom to 6.2.*
- Update dependencies with cargo-update
2021-06-26 00:59:37 +08:00
c18cd42d35 [build] Update dependencies
- Update crc32c to 0.6, which allows it to be built on AArch64.

  - Update base64 to 0.13
  - Update byteorder to 0.14
  - Update io-uring to 0.4
  - Update libc to 0.2.83
  - Update nix to 0.19
  - Update nom to 6.0.1
  - Update quick-xml to 0.20
  - Update rand to 0.8
  - Update tempfile to 3.2
  - Update tui to 0.14

Signed-off-by: Kay Lin <i@v2bv.net>
2021-06-26 00:08:58 +08:00
4905c0eb73 [build] Update .gitignore 2021-06-26 00:08:58 +08:00
cf27a2cf4f Merge pull request #179 from mingnus/2021-06-03-cache-restore-fixes
Minor fixes
2021-06-23 07:55:59 +01:00
2cb84236d4 [all (rust)] Tidy command line options 2021-06-23 14:33:28 +08:00
cd48f00191 [all (rust)] Make sync-io the default
Multithreaded sync-io has performance similar to async-io. Also,
sync-io saves the hassle of setting ulimits to get io_uring working
on some systems (commit ba7fd7b). Now we default to sync-io, and
leave async-io as a hidden option for testing and benchmarking.
2021-06-23 14:33:28 +08:00
361d19adaa [space_map (rust)] Fix cache hit with async-io 2021-06-23 14:33:28 +08:00
5dd2e81bf0 Merge pull request #178 from jwakely/boost-iostreams
[build] Remove -lboost_iostreams linker flag
2021-06-23 07:27:19 +01:00
6b7e66d8f9 [build] Remove -lboost_iostreams linker flag
This was previously needed for thin-provisioning/thin_metadata_pack.cc
but that file was rewritten in Rust and no longer needs Boost. The flag
causes every binary to have a completely redundant depedency on
libboost_iostream.so, which is an issue for RHEL packaging.
2021-06-22 21:23:54 +01:00
8e609458c2 Merge pull request #177 from mingnus/2021-06-03-cache-restore-fixes
Fix restoration tools
2021-06-22 09:50:49 +01:00
fba2adbe56 [tests] Fix minor test errors
- Fix command line arguments
- Adapt error messages for rust implementations
2021-06-21 23:27:51 +08:00
2bd3c17578 [tests] Fix version string checking 2021-06-21 23:27:51 +08:00
bfc7f96d9f [tests] Enable testing the rust targets
This is a temporary solution for development.
Usage: cargo test --features rust_tests
2021-06-21 23:27:51 +08:00
7daff7350a [thin/cache_dump (rust)] Add buffered outputs, and fix command line options 2021-06-21 23:27:51 +08:00
c71132c056 [space_map (rust)] Do not use an extra block set for building metadata sm
The blocks storing metadata itself are located continuously within
a certain reserved range, hence there's no need to use a block set
as the representation.
2021-06-21 23:11:57 +08:00
7ab97a9aae [space_map (rust)] Fix nr_free in index entries 2021-06-16 01:39:31 +08:00
4b7b3658ff [thin/cache_restore (rust)] Build the metadata space map in-place
That avoids cloning the source space map
2021-06-16 01:39:31 +08:00
9ab8dfa283 [space_map (rust)] Fix bitmap packing
- Allow packing unaligned number of bitmap entries that could happen
  with the last bitmap. Unused trailing entries are zeroed by memset.
- Fix none_free_before in index_entry
2021-06-16 01:39:31 +08:00
3fda9cc1f8 [thin_restore (rust)] Do not iterate mapping tree leaves twice 2021-06-16 01:39:31 +08:00
de7e79fc06 [btree_builder] Rename Builder to BTreeBuilder for clarity 2021-06-16 01:39:31 +08:00
6d16c5816e [btree_builder] Fix reference counts of btree nodes
A leaf node should be counted only if it is referenced by some internal
nodes, since the leaves generated by <def> sections might be unshifted
and merged with exclusive mappings or other shared defs, or they might
not even be used by any of the devices if the xml was tampered. The
internal nodes should be handled in the same manner.

The new 'reserve' operation is designed for this purpose. Applications
could reserve a block for later use, without setting its ref count
immediately. This way saves the hassles of tracking unused leaves.
2021-06-16 01:39:31 +08:00
9e061a03a8 [space_map (rust)] Do not reset search position 2021-06-11 20:51:49 +08:00
88e7f8fd69 [array_builder] Simplify array building process 2021-06-11 20:51:49 +08:00
101028ed5f Merge pull request #176 from mingnus/2021-04-28-coverity-fixes
Clear thin superblock flags in restored metadata
2021-06-09 10:10:44 +01:00
60b65ebe7a [space_map (rust)] Fix uninitialized bytes in index block 2021-06-08 20:17:24 +08:00
c32517f827 [thin] Clear superblock flags in restored metadata
The needs_check flag is unnecessary for a restored metadata since
it is assumed clean and has no errors
2021-06-07 19:06:56 +08:00
01aac6c1c1 Merge pull request #175 from mingnus/2021-04-28-coverity-fixes
Fix resource leak
2021-06-04 15:22:24 +01:00