emo/thin-provisioning-tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,721 Commits 2 Branches 0 Tags 3.7 MiB
67a54b4ebc
Commit Graph

420 Commits

Author SHA1 Message Date
Ming-Hung Tsai
0f2918e989 [thin_generate_metadata] Implement format_metadata and create_thin 2020-05-28 17:30:15 +08:00
Ming-Hung Tsai
0c6d132774 [thin_generate_metadata] Preparation for revising the features 2020-05-28 15:09:20 +08:00
Joe Thornber
e92ae10408 [thin_metadata_pack/unpack] Add version field to header. 2020-05-26 10:59:13 +01:00
Joe Thornber
a934a34d22 [thin_metadata_pack/unpack] Move the file magic nr before the compressed stream. 
Makes it easier to spot whether it's a pack file or not.
 2020-05-26 10:53:14 +01:00
Ming-Hung Tsai
c0106fc0b3 [thin_metadata_pack] Improve block scanning performance 2020-05-23 20:32:09 +08:00
Joe Thornber
0e1700fbe9 [thin_metadata_pack] First pass at pack/unpack 2020-05-22 14:11:48 +01:00
Joe Thornber
b7d20bce48 [dev-tools] Update dev tools to use non-template version of block_manager. 2020-05-01 14:11:33 +01:00
Joe Thornber
4313469475 [all] Switch from boost::shared_ptr -> std::shared_ptr. 
Shared_ptr has moved into the standard library since these tools were
first written.
 2020-04-30 15:02:43 +01:00
Joe Thornber
e801cc607b [block_manager] Hard code block size to 4k. 
We're never going to use anything other than 4k, and by hard coding it we
avoid making block_manager a template.
 2020-04-30 14:30:01 +01:00
Joe Thornber
ad79b627a4 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-04-08 12:30:26 +01:00
Joe Thornber
0098803345 [thinp_*] Make mtree_ref_counter derive from ref_counter. 
Slight performance hit, but I'd rather everything was derived
from ref_counter.

Also implement it!  Not sure why this wasn't done before.
 2020-04-08 12:24:37 +01:00
Joe Thornber
0ccee1759e [various] Squash a couple of clang warning wrt friend declarations. 2020-04-08 12:18:24 +01:00
Joe Thornber
5e347dddbf [various] remove dead code. 
Squashes warnings from clang++
 2020-04-08 12:12:59 +01:00
Arusekk
3ed3925370 Fix compilation with uClibc 
Closes #12
 2020-03-25 15:58:02 +01:00
Ming-Hung Tsai
955e11bc28 [block-cache] fix potential file descriptor leak 
Encapsulate file descriptor into an object, to ensure that an fd will be
closed properly while exception raised, e.g., the block_cache throws
exception during the block_manager's construction.
 2020-02-23 15:41:16 +08:00
Ming-Hung Tsai
c85ea5ef76 [thin_check] factor out metadata_checker 2020-02-17 11:42:24 +00:00
Joe Thornber
5f2c3bed69 Add some ftests, and fixup whitespace from Nikhil's work 2019-10-28 11:52:21 +00:00
Nikhil Kshirsagar
91b4714e38 Corrupt superblock repair 2019-10-25 15:02:56 +05:30
Joe Thornber
0fc7529c01 [cache_repair, thin_repair] fix bug introduced in recent patch 
I hadn't realised that check_file_exists() also checked that it was
a regular file, which we don't want for the couple of uses I recently
added.

This patch adds an optional arg must_be_regular_file, and defaults
it to true, preserving the original behaviour.  The recent additions
have this set to false.
 2019-10-14 09:21:38 +01:00
Joe Thornber
1dd7b454bb [thin_repair, cache_repair] Check input file exists earlier 
and zero superblock if we fail part way through a repair.

bz1499781
 2019-10-08 14:34:24 +01:00
Joe Thornber
733c7df798 [thin_repair/thin_dump] Fix some more spurious error messages when doing repair. 
Repair was falling back to non-repair behaviour if it thought the roots
were ok.  Now if --repair is specified the same dumping code is always
executed.
 2019-06-03 14:03:24 +01:00
Joe Thornber
e45a211902 [thin/metadata_dumper] Change a couple of confusing class names. 
metadata_emitter is actually a visitor that passes on it's data
to an encapsulated emitter object.

metadata_emitter -> metadata_emit_visitor
metadata_tree_emitter -> metadata_tree_emit_visitor
 2019-06-03 11:07:32 +01:00
Joe Thornber
0624ded67f [thin_repair, thin_dump] Don't look for better roots if the ones in the sb are ok. 
Calculating the most recent trees is inexact, so trees other than the ones
in the superblock, which presumably are the most recent, could be chosen.
 2019-05-27 16:43:32 +01:00
Joe Thornber
5e6ddbf2da [thin_dump, thin_repair] Fix bad metadata_damage exception 
The first pass of the repair process scans all metadata working out the
largest orphan btrees.  This scan doesn't use as much validation as
the btree_walk function which subsequently gets called.

This patch catches any exceptions thrown by the btree walk function
and removes that btree from consideration.
 2019-05-27 16:06:59 +01:00
Joe Thornber
65264911c1 [thin_repair, thin_dump] --repair falg wasn't being passed through properly. 
Recent regression.
 2019-05-27 09:04:55 +01:00
Joe Thornber
595ddfc50f [thin_repair/thin_dump] repair flag wasn't being set internally. 
Introduced by recent changes
 2019-05-03 15:15:01 +01:00
Joe Thornber
9e20465fd1 [thin-repair, thin_dump] When repairing we now hunt for the best btree roots. 
We've had a trickle of users who accidentally activate the same pool on a
VM and host at the same time.  Typically the host doesn't do any IO, but
the kernel will still rewrite the superblock on shutdown.  This leaves
the superblock pointing to very out of date btree roots and so we get
massive metadata loss.

This patch changes thin_repair, and thin_dump --repair.  They now hunt
for the most recent, undamaged and consistent roots of the device and
mapping trees, and use that as the starting point of the repair.
 2019-04-17 12:17:13 +01:00
Joe Thornber
b027a1039f [thin-provisioning] Add a new constructor for the metadata class 
This one takes a superblock.  Used when repairing.
 2019-04-17 12:15:46 +01:00
Joe Thornber
b777a77023 [thin-provisioning] pass the tm by reference to the metadata object 2019-04-04 15:06:41 +01:00
Joe Thornber
aace49cdd6 [thin_repair] work in progress 2019-03-29 12:56:31 +00:00
Joe Thornber
d1a41d01cc [thin-check] print out some info fields. 
These are consumed by lvm.
 2018-11-09 12:43:21 +00:00
Joe Thornber
de7c9a5781 [thin_journal_check] Checks journal of block manager activity. 
You need to apply doc/bm-journal.patch to create the journal.

thin_journal_check confirms that if the machine had crashed at any time
during the test run no metadata corruption would have occured.
 2018-09-24 14:51:46 +01:00
Joe Thornber
70cdfe12a2 [thin_journal_check] first code drop 2018-08-28 13:12:20 +01:00
Ming-Hung Tsai
9837feaee5 [thin_scan] refactoring and performance improvement (#111) 2018-08-15 07:56:45 +01:00
Dennis Schridde
1311447a94 [build] Include boost/optional/optional_io.hpp in thin-provisioning/emitter.h (#102) 
This fixes following build failure with Boost 1.67.0:
```
In file included from /usr/include/boost/optional/optional.hpp:33,
                 from /usr/include/boost/optional.hpp:15,
                 from ./thin-provisioning/emitter.h:23,
                 from contrib/thin_sexp_emitter.cc:2:
/usr/include/boost/optional/optional.hpp: In instantiation of ‘std::basic_ostream<_CharT, _Traits>& boost::operator<<(std::basic_ostream<_CharT, _Traits>&, const boost::optional_detail::optional_tag&) [with CharType = char; CharTrait = std::char_traits<char>]’:
./base/indented_stream.h:31:9:   required from ‘{anonymous}::indented_stream& {anonymous}::indented_stream::operator<<(const T&) [with T = boost::optional<unsigned int>]’
contrib/thin_sexp_emitter.cc:105:29:   required from ‘void {anonymous}::sexp_emitter::kv(const char*, const T&) [with T = boost::optional<unsigned int>]’
contrib/thin_sexp_emitter.cc:29:21:   required from here
/usr/include/boost/optional/optional.hpp:1481:3: error: static assertion failed: If you want to output boost::optional, include header <boost/optional/optional_io.hpp>
   BOOST_STATIC_ASSERT_MSG(sizeof(CharType) == 0, "If you want to output boost::optional, include header <boost/optional/optional_io.hpp>");
   ^~~~~~~~~~~~~~~~~~~~~~~
make: *** [contrib/Makefile:15: contrib/thin_sexp_emitter.o] Error 1
```

boost/optional/optional_io.hpp exists since at least Boost 1.34.0, so it is
 safe to include in older versions, too.
 2018-06-13 08:42:30 +01:00
Ming-Hung Tsai
582e424560 [space-map-disk] fix the number of free blocks (#93) 2018-04-17 14:00:44 +01:00
Joe Thornber
1433d11f59 [thin-provisioning/metadata] Use latest metadata version (2) 2018-04-17 13:32:49 +01:00
Joe Thornber
abb32897e2 [thin_dump] human_readable emitter didn't print the metadata_snap. 2018-04-17 10:29:31 +01:00
csonto
5b5aa971a0 Detect XML in *_check tools (#86) 
* [*_check] Detect XML in cache_check and era_check

This is based on previous commit b10d8d4440.

* [*_check] Fix typo in check_superblock
 2017-10-05 13:47:10 +01:00
Joe Thornber
3c761e6822 [all tools] Factor out open_bm() and open_tm. 
Many duplicates of this code.
 2017-10-05 11:53:40 +01:00
Joe Thornber
99e8459be3 [thin_rmap] tweak an error message. 2017-10-05 08:22:58 +01:00
Joe Thornber
24931b3bda [thin_check] --override-mapping-root 2017-10-04 11:43:13 +01:00
Joe Thornber
5b92f410ec [*_restore] if things go wrong wipe the superblock. 
So we don't leave the metadata device with partially restored metadata.
 2017-09-28 14:39:24 +01:00
Joe Thornber
48e7ab89a5 [*_dump] Fix segfault when given a tiny metadata file 2017-09-21 10:22:38 +01:00
Joe Thornber
313716028e [thin-metadata-size] -V was exiting with 1 2017-09-19 11:29:35 +01:00
Joe Thornber
3790f2e79c [thin_rmap] Fail if no regions are supplied. 2017-09-19 11:05:00 +01:00
Joe Thornber
f7d79339e8 [man pages] Various updates. 2017-09-18 16:10:56 +01:00
Joe Thornber
b10d8d4440 [thin_check] Spot XML and be helpful. 2017-09-15 15:22:04 +01:00
Joe Thornber
a424417475 [thin_delta] long opts weren't being terminated properly. 
This caused a seg fault with an unknown switch
 2017-09-15 12:28:19 +01:00
Ming-Hung Tsai
b7d418131d Spin-off syscall-related file operations (#78) 
* [file_utils] spin-off syscall-related file operations

1. Eliminate the potential circular dependency between
   persistent-data/block.h and persistent-data/file_utils.h,
   if the former one wants to include the latter.
2. Avoid namespace pollution by removing the "using namespace std"
   declaration in block.tcc.
3. Correct the header hierarchy: base/xml_utils.h now no longer
   depends on the higher-level persistent-data/file_utils.h

* [file_utils] support block files in get_file_length()
 2017-04-29 18:51:52 +01:00
Joe Thornber
4c311cf6d4 Switch an auto_ptr to a unique_ptr 2017-03-13 13:20:52 +00:00
Thanos Makatos
73e583e662 remove debug print 2017-02-06 13:11:10 +03:00
Thanos Makatos
0c78d8d0c5 fix compilation for gcc 5.4.0 2016-09-07 13:49:36 +00:00
Ming-Hung Tsai
45dbc2593a [thin_ll_restore] check preallocation of output file 2016-08-10 23:41:25 +08:00
Ming-Hung Tsai
6863db6f86 [thin] fix duplicated counting of mapped blocks in restore emitter 2016-08-10 23:40:48 +08:00
Joe Thornber
e6562c6110 thin_generate_metadata 2016-07-15 11:23:07 +01:00
Joe Thornber
52de2dd38e Merge branch 'v0.7-devel' of github.com:jthornber/thin-provisioning-tools into v0.7-devel 2016-07-11 15:51:58 +01:00
Joe Thornber
c2fdbad322 Merge branch '2016-07-07-improve-thin-repair-error-message' into v0.7-devel 
Conflicts:
	Makefile.in
	VERSION
	configure.ac
	persistent-data/file_utils.cc
 2016-07-11 15:51:30 +01:00
Joe Thornber
4779fb9b80 [various] Improve documentation 
Output file must be preallocated.
 2016-07-11 14:53:03 +01:00
Ming-Hung Tsai
7eac48793c [space map disk] tidy up: add const qualifier 2016-06-20 00:40:10 +08:00
Ming-Hung Tsai
3439dbfdfc [metadata_counter] hide count_trees() and count_space_maps() 2016-06-20 00:30:26 +08:00
Ming-Hung Tsai
9e7af6b677 [metadata_counter] remove explicit try/catch when counting data space map 
Unlike metadata_index_store, the constructor of btree_index_store
doesn't throw exceptions.
 2016-06-20 00:30:26 +08:00
Ming-Hung Tsai
e8f1bda1a5 [thin] store the device id in single_mapping_tree_damage_visitor for error reporting 
1. fix the damage type for single_mapping_tree_damage_visitor
2. walk_mapping_tree() now requires the device id
3. update metadata_dumper and thin_ls for the new walk_mapping_tree()
 2016-06-20 00:30:25 +08:00
Joe Thornber
a3eac8d5d8 Merge branch 'v0.7-devel' of github.com:jthornber/thin-provisioning-tools into v0.7-devel 2016-06-14 16:33:00 +01:00
Joe Thornber
642740e1b6 Merge branch '2016-03-08-cache-writeback' into v0.7-devel 2016-06-14 16:30:16 +01:00
Ming-Hung Tsai
e985b8b3be [thin_scan][thin_ll_*] fix the data type for getopt_long return value to int 
For toolchains represent char as unsigned type
 2016-05-20 00:54:31 +08:00
Ming-Hung Tsai
55ecf87439 [thin_ll_restore] fix the data type of blocknr to uint64_t 2016-05-20 00:53:51 +08:00
Joe Thornber
3fb4177656 [thin_trim] more bug fixing 2016-04-08 17:07:04 +01:00
Joe Thornber
3c02151a4f Merge branch 'master' into v0.7-devel 
Conflicts:
	persistent-data/file_utils.cc
	persistent-data/file_utils.h
 2016-04-05 13:13:10 +01:00
Thanos Makatos
431f2ab50a allow tmakatos (a binary) emitter to be statically compiled 
It is trivial to implement static compilation of the rest of the 3rd party
emitters.
 2016-04-06 15:56:09 +03:00
Joe Thornber
866986b883 [thin_trim] FInally get thin_trim emitting the right discards 2016-04-05 12:27:00 +01:00
Ming-Hung Tsai
f20e2a0f40 [thin_check] cleanup: use metadata_counter 2016-04-05 16:09:58 +08:00
Ming-Hung Tsai
c8aabf2948 [metadata_counter] fix repeated counting of trees 2016-04-05 16:09:58 +08:00
Ming-Hung Tsai
c6c5086068 [thin_ll_dump] cleanup: use btree_node_checker 2016-04-05 16:09:58 +08:00
Joe Thornber
2288fc4565 Merge branch 'master' into 2016-02-03-fix-thin-trim 2016-03-31 16:39:54 +01:00
Joe Thornber
c7813e07e4 [thin_dump] --format custom=<shared lib> 
Allow people to use their own emitters held in a shared library.

Put a trivial emitter in contrib/ as an example.
 2016-03-24 13:59:08 +00:00
Joe Thornber
872a933072 [thin_dump] start refactoring format code 2016-03-23 10:52:04 +00:00
Joe Thornber
4370f048c0 [thin_dump] --skip-mappings 2016-03-22 15:01:37 +00:00
Joe Thornber
ad4fd30a27 Merge branch 'v0.7-devel' of github.com:jthornber/thin-provisioning-tools into v0.7-devel 2016-03-22 14:47:11 +00:00
Joe Thornber
a3f4940f1e [thin_dump] --dev-id 2016-03-22 14:46:33 +00:00
Joe Thornber
8b2ec7a3aa Merge branch 'master' into v0.7-devel 
Conflicts:
	VERSION
	persistent-data/space-maps/disk.cc
	persistent-data/space-maps/disk.h
 2016-03-22 12:04:51 +00:00
Joe Thornber
b0e2d0e6a0 [thin_repair, metadata] Try harder to get the number of data blocks 
when opening the metadata.
 2016-03-21 17:17:57 +00:00
Joe Thornber
2cf0c84ce3 [thin_repair] we need to read the space maps after all to get the data 
volume size.
 2016-03-21 15:36:49 +00:00
Joe Thornber
e5363621ba Merge pull request #51 from mingnus/v0.6.2-repairtool 
[thin_ll_dump] Fix option -m to accept optional argument
 2016-03-20 14:36:10 +00:00
Ming-Hung Tsai
055623b90f [thin_ll_dump] Fix option -m to accept optional argument 2016-03-19 13:59:38 +08:00
Joe Thornber
0bc126fbeb Merge branch 'v0.7-devel' into 2016-03-08-cache-writeback 
Conflicts:
	thin-provisioning/commands.h
 2016-03-08 16:00:31 +00:00
Joe Thornber
5cbef4f6ef [thin_ll_*] move these to dev tools 2016-03-08 15:52:12 +00:00
Joe Thornber
b8659853bb [cache_writeback] stub cache_writeback 2016-03-08 15:27:22 +00:00
Seth Jennings
38f67676e4 get stats in bytes 2016-03-07 15:42:24 -06:00
Joe Thornber
e670a27638 Merge pull request #46 from mingnus/v0.6.2-repairtool 
Code cleanup for new dev-tools
 2016-03-06 16:15:35 +00:00
Ming-Hung Tsai
220ece1dc8 [thin] Cleanup: move methods before data member declaration 2016-03-06 23:21:09 +08:00
Ming-Hung Tsai
8232feb855 [thin_scan] Cleanup: remove unnecessary variables 2016-03-06 11:42:38 +08:00
Ming-Hung Tsai
9f15c5589f [thin_scan] Cleanup: move methods before data member declaration 2016-03-06 11:42:38 +08:00
Joe Thornber
752163b681 Merge pull request #45 from mingnus/v0.6.2-repairtool 
Minor fixes for new dev-tools
 2016-03-05 15:36:12 +00:00
Ming-Hung Tsai
6fb5f8241d [thin_scan] Factor out range manipulation code 2016-03-05 22:31:09 +08:00
Ming-Hung Tsai
400613305c [thin_ll_restore] Cleanup: use device_details' constructor 2016-03-05 14:12:58 +08:00
Ming-Hung Tsai
08219a60b6 [thin] Add default constructor to struct device_details 2016-03-05 14:12:13 +08:00
Joe Thornber
8676198e76 [file_utils] rename get_nr_blocks() -> get_nr_metadata_blocks() and introduce get_nr_blocks() 2016-03-04 10:43:58 +00:00
Joe Thornber
286f70ceac Merge remote-tracking branch 'remotes/github-m-h-tsai/v0.6.2-repairtool' into 2016-02-29-mingus-merge 
Conflicts:
	Makefile.in
	VERSION
	thin-provisioning/commands.cc
	thin-provisioning/commands.h
	thin-provisioning/thin_delta.cc
 2016-02-29 10:29:28 +00:00
Ming-Hung Tsai
08a7093cd5 [thin_ll_dump][thin_ll_restore] first draft 2016-02-27 15:24:28 +08:00