emo/thin-provisioning-tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5168621f02
thin-provisioning-tools/Cargo.lock

836 lines
21 KiB
Plaintext
Raw Normal View History

[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
name = "adler"
version = "0.2.3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ee2a4ec343196209d6594e19543ae87a39f96d5534d7174822a3ad825dd6ed7e"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "aho-corasick"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.7.13"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "043164d8ba5c4c3035fec9bbee8647c0261d788f3474306f93bb65901cae0e86"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"memchr",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "ansi_term"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ee49baf6cb617b853aa8d93bf420db2383fab46d314482ca2803b40d5fde979b"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"winapi",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
[[package]]
name = "anyhow"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
version = "1.0.32"
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
checksum = "6b602bfe940d21c130f3895acd65221e8a61270debe89d628b9cb4e3ccb8569b"
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "arrayvec"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.5.1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "cff77d8686867eceff3105329d4698d96c2391c176d5d03adc90c7389162b5b8"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "atty"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"hermit-abi",
"libc",
"winapi",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "autocfg"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "1.0.1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "cdb031dd78e28731d87d56cc8ffef4a8f36ca26c38fe2de700543e627f8a464a"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[functional-tests] port some of the cache_check tests to Rust
2020-08-07 19:00:00 +05:30
[[package]]
name = "base64"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3441f0f7b02788e948e47f457ca01f1d7e6d92c693bc132c22b087d3141c03ff"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "bitflags"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "byteorder"
version = "1.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "08c48aae112d48ed9f069b33538ea9e3e90aa263cfa3d1c24309612b1f7472de"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
[[package]]
name = "cc"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "1.0.59"
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "66120af515773fb005778dc07c261bd201ec8ce50bd6e7144c927753fe013381"
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "cfg-if"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.1.10"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "clap"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "2.33.3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "37e58ac78573c40708d45522f0d80fa2f01cc4f9b4e2bf749807255454312002"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"ansi_term",
"atty",
"bitflags",
"strsim",
"textwrap",
"unicode-width",
"vec_map",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "console"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0b1aacfaffdbff75be81c15a399b4bedf78aaefe840e8af1d299ac2ade885d2"
dependencies = [
"encode_unicode",
"lazy_static",
"libc",
"regex",
"terminal_size",
"termios",
"unicode-width",
"winapi",
"winapi-util",
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "crc32c"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "77ba37ef26c12988c1cee882d522d65e1d5d2ad8c3864665b88ee92767ed84c5"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "crc32fast"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ba125de2af0df55319f41944744ad91c71113bf74a4646efff39afe1f6842db1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"cfg-if",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
[[package]]
name = "duct"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f90a9c3a25aafbd538c7d40a53f83c4487ee8216c12d1c8ef2c01eb2f6ea1553"
dependencies = [
"libc",
"once_cell",
"os_pipe",
"shared_child",
]
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "encode_unicode"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a357d28ed41a50f9c765dbfe56cbc04a64e53e5fc58ba79fbc34c10ef3df831f"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "env_logger"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "44533bbbb3bb3c1fa17d9f2e4e38bbbaf8396ba82193c4cb1b6445d711445d36"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"log",
"regex",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_shrink] calculate remaps
2020-06-25 15:14:57 +05:30
[[package]]
name = "fixedbitset"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "0.3.1"
[thin_shrink] calculate remaps
2020-06-25 15:14:57 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "4e08c8bc7575d7e091fe0706963bd22e2a4be6a64da995f03b2a5a57d66ad015"
[thin_shrink] calculate remaps
2020-06-25 15:14:57 +05:30
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "flate2"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "1.0.17"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "766d0e77a2c1502169d4a93ff3b8c15a71fd946cd0126309752104e5f3c46d94"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
dependencies = [
"cfg-if",
"crc32fast",
"libc",
"miniz_oxide",
]
[[package]]
name = "futures"
version = "0.3.5"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "1e05b85ec287aac0dc34db7d4a569323df697f9c55b99b15d6b4ef8cde49f613"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"futures-channel",
"futures-core",
"futures-executor",
"futures-io",
"futures-sink",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-channel"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f366ad74c28cca6ba456d95e6422883cfb4b252a83bed929c83abfdbbf2967d5"
dependencies = [
"futures-core",
"futures-sink",
]
[[package]]
name = "futures-core"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59f5fff90fd5d971f936ad674802482ba441b6f09ba5e15fd8b39145582ca399"
[[package]]
name = "futures-executor"
version = "0.3.5"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "10d6bb888be1153d3abeb9006b11b02cf5e9b209fda28693c31ae1e4e012e314"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"futures-core",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-io"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de27142b013a8e869c14957e6d2edeef89e97c289e69d042ee3a49acd8b51789"
[[package]]
name = "futures-macro"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0b5a30a4328ab5473878237c447333c093297bded83a4983d10f4deea240d39"
dependencies = [
"proc-macro-hack",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "futures-sink"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f2032893cb734c7a05d85ce0cc8b8c4075278e93b24b66f9de99d6eb0fa8acc"
[[package]]
name = "futures-task"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bdb66b5f09e22019b1ab0830f7785bcea8e7a42148683f99214f73f8ec21a626"
dependencies = [
"once_cell",
]
[[package]]
name = "futures-util"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8764574ff08b701a084482c3c7031349104b07ac897393010494beaa18ce32c6"
dependencies = [
"futures-channel",
"futures-core",
"futures-io",
"futures-macro",
"futures-sink",
"futures-task",
"memchr",
"pin-project",
"pin-utils",
"proc-macro-hack",
"proc-macro-nested",
"slab",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "getrandom"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "7abc8dd8451921606d809ba32e95b6111925cd2906060d2dcc29c070220503eb"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"cfg-if",
"libc",
"wasi",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "hermit-abi"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.1.15"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "3deed196b6e7f9e44a2ae8d94225d80302d81208b1bb673fd21fe634645c85a9"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"libc",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "indicatif"
version = "0.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7baab56125e25686df467fe470785512329883aab42696d661247aca2a2896e4"
dependencies = [
"console",
"lazy_static",
"number_prefix",
"regex",
]
[thin_check (rust)] Switch to a different io_uring crate. This one works.
2020-07-28 17:27:30 +05:30
[[package]]
name = "io-uring"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a31e11f8867575fc79a3e73e5f554d0b7386bc4a6f469039e8a83136c724fd81"
dependencies = [
"bitflags",
"libc",
]
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
[[package]]
name = "json"
version = "0.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "078e285eafdfb6c4b434e0d31e8cfcb5115b651496faca5749b88fafd4f23bfd"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "lazy_static"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "lexical-core"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.7.4"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "db65c6da02e61f55dae90a0ae427b2a5f6b3e8db09f58d10efab23af92592616"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"arrayvec",
"bitflags",
"cfg-if",
"ryu",
"static_assertions",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "libc"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "0.2.76"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "755456fae044e6fa1ebbbd1b3e902ae19e73097ed4ed87bb79934a867c007bc3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "log"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.4.11"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "4fabed175da42fed1fa0746b0ea71f412aa9d35e76e95e59b192c64b9dc2bf8b"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"cfg-if",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "memchr"
version = "2.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "3728d817d99e5ac407411fa471ff9800a778d88a24685968b36824eaf4bee400"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "miniz_oxide"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "0.4.1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "4d7559a8a40d0f97e1edea3220f698f78b1c5ab67532e49f68fde3910323b722"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"adler",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
[[package]]
name = "nix"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "50e4785f2c3b7589a0d0c1dd60285e1188adac4006e8abd6dd578e1567027363"
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"bitflags",
"cc",
"cfg-if",
"libc",
"void",
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "nom"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "6.0.0-alpha1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"lexical-core",
"memchr",
"version_check",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "num-derive"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "0.3.2"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "6f09b9841adb6b5e1f89ef7087ea636e0fd94b2851f887c1e3eb5d5f8228fab3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"proc-macro2",
"quote",
"syn",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "num-traits"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.2.12"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ac267bcc07f48ee5f8935ab0d24f316fb722d7a1292e2913f0cc196b29ffd611"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"autocfg",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "num_cpus"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "05499f3756671c15885fee9034446956fff3f243d6077b91e5767df161f766b3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"hermit-abi",
"libc",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "number_prefix"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17b02fc0ff9a9e4b35b3342880f48e896ebf69f2967921fe8646bf5b7125956a"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[[package]]
name = "once_cell"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "1.4.1"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "260e51e7efe62b592207e9e13a68e43692a7a279171d6ba57abd208bf23645ad"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
[[package]]
name = "os_pipe"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb233f06c2307e1f5ce2ecad9f8121cffbbee2c95428f44ea85222e460d0d213"
dependencies = [
"libc",
"winapi",
]
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[[package]]
name = "pin-project"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
version = "0.4.23"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
checksum = "ca4433fff2ae79342e497d9f8ee990d174071408f28f726d6d83af93e58e48aa"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"pin-project-internal",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[[package]]
name = "pin-project-internal"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
version = "0.4.23"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
checksum = "2c0e815c3ee9a031fdf5af21c10aa17c573c9c6a566328d99e3936c34e36461f"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "pin-utils"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "ppv-lite86"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "0.2.9"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "c36fa947111f5c62a733b652544dd0016a43ce89619538a8ef92724a6f501a20"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[[package]]
name = "proc-macro-hack"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
version = "0.5.18"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Walk mapping tree. Sync IO, no checks beyond checksumming.
2020-07-28 15:21:48 +05:30
checksum = "99c605b9a0adc77b7211c6b1f722dcb613d68d66859a44f3d485a6da332b0598"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
[[package]]
name = "proc-macro-nested"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eba180dafb9038b050a4c280019bbedf9f2467b61e5d892dcad585bb57aadc5a"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "proc-macro2"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "1.0.19"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "04f5f085b5d71e2188cb8271e5da0161ad52c3f227a661a3c135fdf28e258b12"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"unicode-xid",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_shrink] Write thinp xml format reader and writer. Still need to tidy up the error handling.
2020-06-24 18:02:47 +05:30
[[package]]
name = "quick-xml"
version = "0.18.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "3cc440ee4802a86e357165021e3e255a9143724da31db1e2ea540214c96a0f82"
[thin_shrink] Write thinp xml format reader and writer. Still need to tidy up the error handling.
2020-06-24 18:02:47 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"memchr",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "quickcheck"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "a44883e74aa97ad63db83c4bf8ca490f02b2fc02f92575e720c8551e843c945f"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"env_logger",
"log",
"rand",
"rand_core",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "quickcheck_macros"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "608c156fd8e97febc07dc9c2e2c80bf74cfc6ef26893eae3daf8bc2bc94a4b7f"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"proc-macro2",
"quote",
"syn",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "quote"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "1.0.7"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "aa563d17ecb180e500da1cfd2b028310ac758de548efdd203e18f283af693f37"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"proc-macro2",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "rand"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"getrandom",
"libc",
"rand_chacha",
"rand_core",
"rand_hc",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "rand_chacha"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "f4c8ed856279c9737206bf725bf36935d8666ead7aa69b52be55af369d193402"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"ppv-lite86",
"rand_core",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "rand_core"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"getrandom",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "rand_hc"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"rand_core",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_shrink] write test harness
2020-06-26 21:14:47 +05:30
[[package]]
name = "redox_syscall"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.1.57"
[thin_shrink] write test harness
2020-06-26 21:14:47 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "41cc0f7e4d5d4544e8861606a285bb08d3e70712ccc7d2b84d7c0ccfaf4b05ce"
[thin_shrink] write test harness
2020-06-26 21:14:47 +05:30
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "regex"
version = "1.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "9c3780fcf44b193bc4d09f36d2a3c87b251da4a046c87795a0d35f4f927ad8e6"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"aho-corasick",
"memchr",
"regex-syntax",
"thread_local",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "regex-syntax"
version = "0.6.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "26412eb97c6b088a6997e05f69403a802a92d520de2f8e63c2b65f9e0f47c4e8"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
[thin_shrink] write test harness
2020-06-26 21:14:47 +05:30
name = "remove_dir_all"
version = "0.5.3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "3acd125665422973a33ac9d3dd2df85edad0f4ae9b00dafb1a05e43a9f5ef8e7"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"winapi",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "ryu"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "71d301d4193d031abdd79ff7e3dd721168a9572ef3fe51a1517aba235bd8f86e"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
[[package]]
name = "shared_child"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8cebcf3a403e4deafaf34dc882c4a1b6a648b43e5670aa2e4bb985914eaeb2d2"
dependencies = [
"libc",
"winapi",
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
name = "slab"
version = "0.4.2"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "c111b5bd5695e56cffe5129854aa230b39c93a305372fdbb2668ca2394eea9f8"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "static_assertions"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "1.1.0"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "strsim"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "8ea5119cdb4c55b55d432abb513a0429384878c15dde60cc77b1c99de1a95a6a"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "syn"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
version = "1.0.39"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
[thin_check (rust)] Add error handling to io_engine interface
2020-09-02 17:27:47 +05:30
checksum = "891d8d6567fe7c7f8835a3a98af4208f3846fba258c1bc3c31d6e506239f11f9"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"proc-macro2",
"quote",
"unicode-xid",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_shrink] write test harness
2020-06-26 21:14:47 +05:30
[[package]]
name = "tempfile"
version = "3.1.0"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "7a6e24d9338a0a5be79593e2fa15a648add6138caa803e2d5bc782c371732ca9"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"cfg-if",
"libc",
"rand",
"redox_syscall",
"remove_dir_all",
"winapi",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "terminal_size"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a14cd9f8c72704232f0bfc8455c0e861f0ad4eb60cc9ec8a170e231414c1e13"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "termios"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f0fcee7b24a25675de40d5bb4de6e41b0df07bc9856295e7e2b3a3600c400c2"
dependencies = [
"libc",
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "textwrap"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "d326610f408c7a4eb6f51c37c330e496b08506c9457c9d34287ecc38809fb060"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"unicode-width",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "thinp"
version = "0.1.0"
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"anyhow",
[thin_check (rust)] move report creation to top level
2020-08-17 20:22:12 +05:30
"atty",
[functional-tests] port some of the cache_check tests to Rust
2020-08-07 19:00:00 +05:30
"base64",
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"byteorder",
"clap",
"crc32c",
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
"duct",
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"fixedbitset",
"flate2",
"futures",
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
"indicatif",
[thin_check (rust)] Switch to a different io_uring crate. This one works.
2020-07-28 17:27:30 +05:30
"io-uring",
[functional-tests] Move thin_check functional tests to Rust. They'll be run as part of 'cargo test' now.
2020-07-31 15:34:12 +05:30
"json",
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"libc",
"nix",
"nom",
"num-derive",
"num-traits",
"num_cpus",
"quick-xml",
"quickcheck",
"quickcheck_macros",
"rand",
"tempfile",
"thiserror",
[thin_check (rust)] First attempt at multithreading the tree walking. Still using a single io engine, so little benefit.
2020-07-29 21:08:52 +05:30
"threadpool",
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
]
[[package]]
name = "thiserror"
version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "7dfdd070ccd8ccb78f4ad66bf1982dc37f620ef696c6b5028fe2ed83dd3d0d08"
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"thiserror-impl",
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
]
[[package]]
name = "thiserror-impl"
version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "bd80fc12f73063ac132ac92aceea36734f04a1d93c1240c6944e23a3b8841793"
[thin_metadata_{pack,unpak}] use anyhow in toplevel.
2020-06-21 16:20:24 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"proc-macro2",
"quote",
"syn",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "thread_local"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "d40c6d1b69745a6ec6fb1ca717914848da4b44ae29d9b3080cbee91d72a69b14"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"lazy_static",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[thin_check (rust)] First attempt at multithreading the tree walking. Still using a single io engine, so little benefit.
2020-07-29 21:08:52 +05:30
[[package]]
name = "threadpool"
version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d050e60b33d41c19108b32cea32164033a9013fe3b46cbd4457559bfbf77afaa"
dependencies = [
"num_cpus",
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "unicode-width"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.1.8"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "9337591893a19b88d8d87f2cec1e73fad5cdfd10e5a6f349f498ad6ea2ffb1e3"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "unicode-xid"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.2.1"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "f7fe0bb3479651439c9112f72b6c505038574c9fbb575ed1bf3b797fa39dd564"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "vec_map"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "f1bddf1187be692e79c5ffeab891132dfb0f236ed36a43c7ed39f1165ee20191"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "version_check"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "b5a972e5669d67ba988ce3dc826706fb0a8b01471c088cb0b6110b805cc36aed"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
[[package]]
name = "void"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d"
[thin_metadata_{pack/unpack}] Fix get_nr_blocks() It didn't work with block devices.
2020-06-10 19:08:42 +05:30
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "wasi"
version = "0.9.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "winapi"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
version = "0.3.9"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
dependencies = [
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[thin_check (rust)] Add progress bar
2020-08-11 17:52:14 +05:30
[[package]]
name = "winapi-util"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70ec6ce85bb158151cae5e5c87f95a8e97d2c0c4b001223f33a334e3ce5de178"
dependencies = [
"winapi",
]
[thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. The Rust implementation is multithreaded, performs better in general and does custom compression of btree nodes to achieve much better compression ratios. unpack also checksums expanded metadata to validate it. Format version has jumped to 3, no backwards compatibility, but I think that's ok since we never made a release that contained the C++ version of these tools. Benchmarks ========== On an 8 core, 16 hyperthread machine. metadata 1G, full: Pack size pack time unpack time ------------------------------------------------------ C++ 193M 50.3s 6.9s (no verify) Rust 70M 1.4s 1.8s (verify) metadata 16G, sparse: Pack size pack time unpack time ------------------------------------------------------ C++ 21M 68s 1s (no verify) Rust 4M 8.6s 0.5s (verify)
2020-06-09 13:45:00 +05:30
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
Merge branch 'master' into 2020-06-13-thin-check-rewrite
2020-07-27 20:23:26 +05:30
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
Reference in New Issue Copy Permalink