Commit Graph

1579 Commits

Author SHA1 Message Date
Joe Thornber 8aaea7f45b Merge pull request #142 from mingnus/thin-generate-metadata-wip
thin_generate_metadata minor fixes
2020-06-24 12:29:31 +01:00
Ming-Hung Tsai 19e482fd2f [thin_generate_mappings] Update help text 2020-06-24 12:26:15 +08:00
Ming-Hung Tsai 8006e18173 [thin_pool] Fix the number of mapped blocks 2020-06-24 12:26:14 +08:00
Ming-Hung Tsai 084ac4b950 [io_generator] Simplify the class interface 2020-06-24 12:26:09 +08:00
Joe Thornber fe754d81a4 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-06-19 14:50:51 +01:00
Joe Thornber 1fd0bfd4e8 Merge pull request #141 from mingnus/thin-generate-metadata-wip
Complete the major features of thin_generate_metadata, and introduce thin_generate_mappings
2020-06-19 14:46:52 +01:00
Ming-Hung Tsai 8fa59553c4 [thin_generate_mappings] Introduce the mapping generation tool 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 621b466333 [all] Eliminate duplicated sector_t typedefs 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai a345b894d3 [math_utils] Add is_power_of_two() 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 5260a87c0b [base] Move math_utils to namespace base 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 12d7178199 [base] Introduce io_generator 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai e724a72045 [btree] Implement btree::remove() 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai ef8e94b22c [thin_generate_metadata] Support setting transaction id and metadata snapshots 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 524de134a4 [thin_pool] Support creating metadadata snapshot 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai e33c60c7e2 [thin_pool] Add close_thin() 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 89409927b4 [thin_generate_metadata] Support device deletion 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai a5614f7db2 [thin_pool] Complete device deletion 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai f803c44e93 [btree] Implement btree::destroy() to delete the entire tree
Also decrease the reference count of the mapped values.
(e.g., bitmap blocks of btree_index_store)
2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 4e1ec9326b [thin_pool] Tidying
- Remove unused headers
- Use predefined types
2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 598696bc32 [thin_pool] Implement synchronous mapping insertion and removal 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 383ae78871 [thin_pool] Determine shared blocks while looking up mappings 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai a842b4ecfd [thin_generate_metadata] Make use of thin_pool.cc 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai 66b519db83 [thin_pool] Fix snapshot creation 2020-06-19 17:22:19 +08:00
Ming-Hung Tsai 2cce8a51a7 [thin_pool] Fix mapped block time 2020-06-19 17:22:19 +08:00
Ming-Hung Tsai 7f7ba950ef [thin_pool] Complete device creation
- Set up the device details
- Cache device details of opened devices
- Update changed devices at once while committing the pool
2020-06-19 17:22:19 +08:00
Ming-Hung Tsai 77adb08c3f [thin_pool] Replace the member pointer by references 2020-06-17 19:09:24 +08:00
Ming-Hung Tsai 0b058a2850 [thin_pool] Encapsulate metadata into the pool class 2020-06-17 19:09:24 +08:00
Ming-Hung Tsai 5810846409 [thin] Add more constructors to device_details 2020-06-17 19:09:24 +08:00
Joe Thornber d1cf87712b Bump version to 0.9.0-rc1 2020-06-15 12:50:58 +01:00
Joe Thornber b8eebf59c7 Update CHANGES file 2020-06-15 12:49:16 +01:00
Joe Thornber 7d625eb6ff [src/file_utils.rs] Don't print to stderr in file_exists. 2020-06-15 12:48:19 +01:00
Joe Thornber 0d2b49aae7 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-06-15 11:29:01 +01:00
Joe Thornber bb64189d26 [thin_check] update thin_check manpage. 2020-06-15 11:27:15 +01:00
Joe Thornber dc5bb3559b [build] remove need for zlib.
Disappeared with the C++ version of thin_metadata_pack
2020-06-12 19:09:19 +01:00
Joe Thornber 3e24264442 Merge branch '2020-05-27-check-data-space-map' 2020-06-12 13:42:35 +01:00
Joe Thornber 0b5afc6cb0 [thin_check] fix bugs in thin_check -m 2020-06-12 13:41:47 +01:00
Joe Thornber 0ab509578f [*_check] Remove the inane "This is not the btree you are looking for." message. 2020-06-11 13:05:03 +01:00
Joe Thornber 7a85e47a20 [rust] squash clippy (linter) warnings 2020-06-10 16:46:38 +01:00
Joe Thornber ace8b39a8a Merge branch 'master' into 2020-05-27-check-data-space-map 2020-06-10 15:12:46 +01:00
Joe Thornber f90010e22a [pack/unpack] fix bug in file_utils::file_size() 2020-06-10 15:07:40 +01:00
Joe Thornber 380632565d [thin_metadata_{pack/unpack}] Fix get_nr_blocks()
It didn't work with block devices.
2020-06-10 14:38:42 +01:00
Joe Thornber 324c0050bf [install-rust-tools] add dep to generate man pages 2020-06-10 10:32:36 +01:00
Joe Thornber 549c0b9499 Merge branch 'master' into 2020-05-27-check-data-space-map 2020-06-10 10:15:16 +01:00
Joe Thornber 821628b004 [pack/unpack] add missing file. 2020-06-09 14:59:13 +01:00
Joe Thornber 86704deacb [thin_check] Add support for --metadata-snap.
Not tested yet.
2020-06-09 14:25:19 +01:00
Joe Thornber 01a7462ece Merge branch 'master' into 2020-05-27-check-data-space-map 2020-06-09 13:39:46 +01:00
Joe Thornber 409a660082 [pack/unpack] Get the functional tests working again.
There's some hard coded version numbers in the tests, but I'm
leaving for now since I'll rewrite in Rust to avoid too much
of a proliferation of languages.
2020-06-09 13:03:39 +01:00
Joe Thornber db5a71a53c [rust tools] Get the tools version from autoconf 2020-06-09 10:23:52 +01:00
Joe Thornber 61de3f9287 [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one.
The Rust implementation is multithreaded, performs better in general and
does custom compression of btree nodes to achieve much better compression
ratios.  unpack also checksums expanded metadata to validate it.

Format version has jumped to 3, no backwards compatibility, but I think
that's ok since we never made a release that contained the C++ version
of these tools.

Benchmarks
==========

On an 8 core, 16 hyperthread machine.

metadata 1G, full:

      Pack size    pack time     unpack time
------------------------------------------------------
C++      193M        50.3s          6.9s (no verify)
Rust      70M         1.4s          1.8s (verify)

metadata 16G, sparse:

       Pack size    pack time     unpack time
------------------------------------------------------
C++      21M          68s           1s   (no verify)
Rust      4M           8.6s         0.5s (verify)
2020-06-09 09:15:00 +01:00
Joe Thornber c48851e747 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-06-01 11:14:24 +01:00