mirror of
https://git.qwik.space/left4code/left4code.neocities.org.git
synced 2025-07-28 00:13:38 +05:30
apr 19th changes
This commit is contained in:
Left4Code
126
courses/digital_forensics.html
Normal file
126
courses/digital_forensics.html
Normal file
@@ -0,0 +1,126 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Left 4 Code</title>
|
||||
<link rel="icon" type="image/x-icon" href="../favicon/favicon.ico">
|
||||
<link rel="stylesheet" type="text/css" href='../style.css'>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="../index.html">Home</a>
|
||||
<a href="../blog.html">Blog / Courses</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">Digital Forensics Using Linux</h1>
|
||||
|
||||
<p>This whole page is currently subject to change, I'm figuring things out.</p>
|
||||
<p>This "Course" will be formatted in such a way where you can view any section you want, I'll provide what you need to know at the top of it, and it's up to you if you want to follow that advice or not.</p>
|
||||
<dl>
|
||||
<hr>
|
||||
<p>[*Note*] This course got it's list of tools from <a href="https://tsurugi-linux.org/documentation_tsurugi_linux_tools_listing_2024.php">this Tsurugi Linux page</a> if this course ever becomes outdated (probably will unless I do community-submitted git integration) you can always find an updated list of tools there.</p>
|
||||
<hr>
|
||||
<p>[*Also Note*] Everything with a "[✅]" means the section exists and "[❌]" means it does not. [🛠️] means I'm currently working on it.</p>
|
||||
<hr>
|
||||
</dl>
|
||||
|
||||
<h3 class="blog-header">Filesystem Imaging & hashing</h3>
|
||||
<ul>
|
||||
<li><a href="itscoming.html">dd — [❌]</a></li>
|
||||
<li><a href="itscoming.html">ddc3dd — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Guymager — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Cyclone — [❌]</a></li>
|
||||
<li><a href="itscoming.html">ddrescuer — [❌]</a></li>
|
||||
<li><a href="itscoming.html">ftkimage — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Guymager — [❌]</a></li>
|
||||
<hr>
|
||||
<li><a href="hash_forensics/gtkhash.html">GtkHash — [✅]</a></li>
|
||||
<li><a href="hash_forensics/shasum.html">sha*sum — [✅]</a></li>
|
||||
<li><a href="itscoming.html">hashcat — [🛠️]</a></li>
|
||||
<li><a href="itscoming.html">hashid — [❌]</a></li>
|
||||
<li><a href="itscoming.html">OpenTimestamps — [❌]</a></li>
|
||||
</ul>
|
||||
<h3 class="blog-header">Data Acquisition</h3>
|
||||
<ul>
|
||||
<li><a href="itscoming.html">Acquire — [❌]</a></li>
|
||||
<li><a href="itscoming.html">artifactcollector — [❌]</a></li>
|
||||
<li><a href="itscoming.html">AVML — [❌]</a></li>
|
||||
<li><a href="itscoming.html">LiME — [❌]</a></li>
|
||||
<li><a href="itscoming.html">unix_collector — [❌]</a></li>
|
||||
<li><a href="itscoming.html">velociraptor — [❌]</a></li>
|
||||
</ul>
|
||||
<h3 class="blog-header">Logfile Locations on Win & Lin, & Reading Logfiles</h3>
|
||||
|
||||
<ul>
|
||||
<li><a href="itscoming.html">Windows CommonLog — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Linux CommonLog — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Reading logfiles— [❌]</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h3 class="blog-header">Memory Analysis</h3>
|
||||
<ul>
|
||||
<li><a href="itscoming.html">Rekall — [❌]</a></li>
|
||||
<li><a href="itscoming.html">volatility — [❌]</a></li>
|
||||
<li><a href="itscoming.html">volUtility — [❌]</a></li>
|
||||
</ul>
|
||||
<h3 class="blog-header">Common Types of Steganography & Detection</h3>
|
||||
|
||||
<ul>
|
||||
<li><a href="itscoming.html">StegHide — [❌]</a></li>
|
||||
</ul>
|
||||
<h3 class="blog-header">Network Forensics</h3>
|
||||
<ul>
|
||||
<li><a href="itscoming.html">SNORT — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Wireshark — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Kismet — [❌]</a></li>
|
||||
<li><a href="itscoming.html">NetworkMiner — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Squery — [❌]</a></li>
|
||||
</ul>
|
||||
<h3 class="blog-header">Blockchain Forensics</h3>
|
||||
|
||||
<ul>
|
||||
<li><a href="itscoming.html">etherscan — [❌]</a></li>
|
||||
<li><a href="itscoming.html">blockchair — [❌]</a></li>
|
||||
<li><a href="itscoming.html">WalletSleuth — [❌]</a></li>
|
||||
</ul>
|
||||
|
||||
<h3 class="blog-header">Metadata forensics</h3>
|
||||
|
||||
<ul>
|
||||
<li><a href="metadata_forensics/mat2.html">mat2 — [✅]</a></li>
|
||||
<li><a href="metadata_forensics/exiftool.html">ExifTool — [✅]</a></li>
|
||||
</ul>
|
||||
|
||||
<h3 class="blog-header">Putting Together a Timeline & Report</h3>
|
||||
|
||||
<ul>
|
||||
<li><a href="itscoming.html">plaso — [❌]</a></li>
|
||||
<li><a href="itscoming.html">Timeline Explorer — [❌]</a></li>
|
||||
<li><a href="itscoming.html">timeliner — [❌]</a></li>
|
||||
<li><a href="itscoming.html">timesketch — [❌]</a></li>
|
||||
<li><a href="itscoming.html">DFTimewolf — [❌]</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
</section>
|
||||
</div>
|
||||
|
||||
</body>
|
||||
<footer>
|
||||
<h5>This entire site and it's material are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License</h5>
|
||||
<h5>and is not allowed to be used for Commercial purposes or educational purposes which require tuition (or any money at all) to access.</h5>
|
||||
<p><a href="https://creativecommons.org/licenses/by-nc/4.0/">https://creativecommons.org/licenses/by-nc/4.0/</a></p>
|
||||
|
||||
</footer>
|
||||
|
||||
</html>
|
||||
89
courses/hash_forensics/gtkhash.html
Normal file
89
courses/hash_forensics/gtkhash.html
Normal file
@@ -0,0 +1,89 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en"><head>
|
||||
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="icon" type="image/x-icon" href="favicon/favicon.ico">
|
||||
<title>Left4Code - (gtkhash)</title>
|
||||
<base href="../../">
|
||||
<link rel="stylesheet" type="text/css" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="index.html">Home</a>
|
||||
<a href="blog.html">Blog</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">gtkhash for Hashing Files from a GUI <p>(and introduction to hashing)</p></h1>
|
||||
<p>Since this is the beginning page in the hashing section, I will explain the concept of hashing here, what is it, what it does, why it's useful, how to use it. This should be quite a small introduction as most of gtkhash is very self-explanatory and you can figure everything out yourself just by clicking around if you really wanted to. Personally, I like the GUI sometimes, it's not too complex, you don't have to have the manual up side-by-side with another terminal tab while trying to do something, and while it's not typically as fast as a cli program in terms of use and output, it's still nice to know how to use.</p>
|
||||
|
||||
<h3 class="blog-header">What you need to know (To get the most out of this!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> A tiny bit of familiarity with the terminal.. But don't worry! It's just for installing gtkhash. (you could just use your GUI software installer to install it instead if you really do not like the terminal.)
|
||||
|
||||
<b>—</b> Some determination to read, mentally digest, practice, and learn for yourself. That's it.. Really.
|
||||
</div>
|
||||
<h3 class="blog-header">What this page covers (To Not Waste Your Time!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Quick introduction to hashing (what is it, why it's useful.)
|
||||
<b>—</b> Installing gtkhash
|
||||
<b>—</b> Using gtkhash
|
||||
<b>—</b> Using different hashing algorithms in gtkhash
|
||||
<b>—</b> Saving hash output to a file using gtkhash
|
||||
<b>—</b> The different view modes of gtkhash
|
||||
<b>—</b> Closing notes
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3 class="blog-header">A Quick Introduction to Hashing (The What, and the why!)</h3>
|
||||
<h4>The What!</h4>
|
||||
<p>Let's start with the what. What is hashing? From the way I learned it, hashing is the process of taking some input data, running it through a mathematical algorithm, which then spits out a unique alphanumerical string called a hash. Hashing is designed to be only one way and ""Hopefully!"" not reversible through brute force. So when all is right in the world for a hashing algorithm, a string of data has only one equivalent hash, and the original string can not be derived from the hash. </p>
|
||||
<h4>The Why!</h4>
|
||||
<p>Why do we need hashing? Well, hashing is useful because with it, you can verify that the integrity of data remains the same, it can increase the security of a password database for example, because all the server needs to do is compare the hash output with the password received instead of comparing strings directly, and it can provide checksums from developers for things like your web browser. Without hashing, data could be modified by anything from another person, to passing electrons or cosmic mysteries without any real way to tell that something has happened!</p>
|
||||
|
||||
<h3 class="blog-header">Installing gtkhash</h3>
|
||||
<p>To install gtkhash, you can either install it using your fancy GUI Linux software store for your specific distribution (synaptic maybe?) or just install it by opening that big scary terminal and typing the following if you're using a debian-based distribution.</p>
|
||||
|
||||
<pre class="preformatted">sudo apt install gtkhash</pre>
|
||||
|
||||
<p>To use gtkhash, you can either open it up from your extra extra fancy start menu, or open that scary terminal up again (I know, it'll be the last time for this section, I swear.) And type the following command:</p>
|
||||
|
||||
<pre class="preformatted">gtkhash &</pre>
|
||||
|
||||
<p>Then run:</p>
|
||||
|
||||
<pre class="preformatted">disown -r</pre>
|
||||
|
||||
<p>this command should run gtkhash as it's own process not directly attached to the terminal, you should then be able to close the terminal and gtkhash stays open... I hope. If not then you'll have to figure out a solution on your own.</p>
|
||||
|
||||
<h3 class="blog-header">Using gtkhash</h3>
|
||||
|
||||
<p>To use gtkhash, it's pretty simple, upon launching the program you will be greeted with a file box to select a file, a box to check a file against a checksum file or data from a checksum file, and the remaining boxes for the output of the different hashing algorithms. All you need to do is make a file (or pull one from a location in your file-system like /bin) and then click "hash" in the bottom right, which should spit out the hashes for MD5, SHA1, SHA256, and CRC32 by default.</p>
|
||||
<h3 class="blog-header">Using different hashing algorithms in gtkhash</h3>
|
||||
|
||||
<p>I'll be honest, the default 4 hashing algorithms you get will probably be all you need for the rest of time, but let's say you for some reason really need a hash using the GOST algorithm for example, well head over to that tab labeled "edit" in the top left of the gtkhash window and go down to the "preferences". From there you will be quickly overwhelmed with the 7 billion hash functions, just pick the ones you want by clicking the boxes and it will be added to where those original default 4 algorithms were in gtkhash, and if you specifically hate those 4 default ones, you can remove them the same way.</p>
|
||||
|
||||
<h3 class="blog-header">Saving the output from gtkhash to a text file</h3>
|
||||
<p>To save the results of your epic hashing session, head over to the top left of the gtkhash window and click "file", and the go to "save digest file", from there you will be able to name the hash file. What I would recommend is to save the file with the same name as the original file you hashed, and then with the appropriate extension (.sha1, .md5, .sha256, whatever) so if you hashed the file Timothys_Starbucks_Order.txt using sha1, you would save it as Timothys_Starbucks_Order.sha1, this stops gtkhash from getting confused and it should put the hash in the check box like it's supposed to.</p>
|
||||
<h3 class="blog-header">Using the different View Modes of gtkhash</h3>
|
||||
|
||||
<p>In gtkhash, there are multiple different view modes that you can use which allow you to do different things like instead of hashing a file, you can hash the word "test" for example, and instead of hashing a single file, you can hash multiple files! You can do this by clicking the "view" tab, selecting whatever mode you want, and then do what you need with it.</p>
|
||||
<h4>Conclusion</h4>
|
||||
<p>gtkhash is a nifty piece of software that you can use without knowledge of the terminal to create digests of files. What I didn't show was the ability to use gtkhash as an extension in your file manager, for example nautilus, but there's probably a guide somewhere on the internet if you're looking for that specifically. If you want to check to see if gtkhash has an addon for your specific file manager, you can use synaptic and search gtkhash to get the list of addons for whatever file manager they support, but they support most of the big ones and I added the package names so all you would need to do is sudo apt install it. thunar (thunar-gtkhash), nemo (nemo-gtkhash), nautilus (nautilus-gtkhash), caja (caja-gtkhash). Just install the package name in parentheses and it should work without a problem.</p>
|
||||
<p>That's all, we're using the terminal next do to the same thing!</p>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
</body></html>
|
||||
109
courses/hash_forensics/shasum.html
Normal file
109
courses/hash_forensics/shasum.html
Normal file
@@ -0,0 +1,109 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en"><head>
|
||||
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="icon" type="image/x-icon" href="favicon/favicon.ico">
|
||||
<title>Left4Code - (sha*sum)</title>
|
||||
<base href="../../">
|
||||
<link rel="stylesheet" type="text/css" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="index.html">Home</a>
|
||||
<a href="blog.html">Blog</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">sha*sum for hashing files from the command line</h1>
|
||||
<p> To clear up any confusion, when I refer to sha*sum, I'm referring to most of the command line hashing programs that come with most linux distributions by default. (md5sum, sha512sum, sha256sum) So just replace sha*sum with the cli hash utility you're currently using. If you want to see what hashing utilities you have on your system, you can have a look in /bin to see what you've got! This is a quick and dirty way to see.</p>
|
||||
<pre class="preformatted">ls /bin | grep sum</pre>
|
||||
|
||||
<h3 class="blog-header">What you need to know (To get the most out of this!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Basic understanding of the Linux command line (bash). Specifically, do you understand output and input redirection and pipes ('<b>></b>', <b>'<'</b>, and '<b>|</b>')
|
||||
|
||||
<b>—</b> Some determination to read, mentally digest, practice, and learn for yourself.
|
||||
|
||||
<b>—</b> How to use the manpages (run "man man" without the double quotes in your terminal if you don't know) this is so you can always use the manpages if this course doesn't get completed or updated. I want to teach you to fish, not give you fish.
|
||||
|
||||
<b>—</b> Whenever I put carat symbols outside of something, don't add them to the command in your actual terminal, ex: <yourfile> should be typed in your terminal as yourfile, or whatever you want to name it, it's just a placeholder, you get it.
|
||||
</div>
|
||||
<h3 class="blog-header">What this page covers (To Not Waste Your Time!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Quick introduction to hashing things with the command line
|
||||
<b>—</b> What can be hashed
|
||||
<b>—</b> Some techniques to make hashing more effective
|
||||
<b>—</b> Saving hash output to a file
|
||||
<b>—</b> Binary mode
|
||||
<b>—</b> Taking a hash from a file and comparing it against a file to be hashed
|
||||
<b>—</b> Closing notes
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3 class="blog-header">A Quick Introduction to Hashing Using the Command Line!</h3>
|
||||
<p>If you've already read the gtkhash section of the course, you'll know the basics of how hashing works from a high level, and why it's useful for a forensic investigation, to avoid wasting your time, I will quickly explain how to use 99% of command line hashing tools and then go into further detail about some things that are a little more advanced. (moar content!)</p>
|
||||
<p>Starting off with the basics of the basics, if you want to hash a file using the command line, type the following (obviously remember to change sha*sum to your preferred hash program!)</p>
|
||||
|
||||
<pre class="preformatted">sha*sum <file_you_want_to_hash></pre>
|
||||
|
||||
<p>So if all you wanted to know was how to get the hash of a file, that should do it. It will just print the hash of the corresponding file to standard output in the terminal. Now if you want to learn some more things you can do involving bash and these utilities, stick around.</p>
|
||||
|
||||
<h3 class="blog-header">What can be hashed?</h3>
|
||||
<p>Something you might commonly hear after you've used Linux enough is "in Linux, everything is a file". So if that's the case, then technically we could hash anything on the system, couldn't we? Let's see some common examples of things that can be hashed! A fun little list I've cobbled together shows you some of the fun things you can use sha*sum on.</p>
|
||||
|
||||
<h4><b>1 —</b> Output from other programs (this will come in handy later!)</h4>
|
||||
<pre class="preformatted">echo "Hello" | sha*sum</pre>
|
||||
<h4><b>2 —</b> /dev/null !</h4>
|
||||
<pre class="preformatted">sha*sum /dev/null</pre>
|
||||
<h4><b>3 —</b> File Metadata!</h4>
|
||||
<pre class="preformatted">mat2 -s <your_file> | sha*sum</pre>
|
||||
<h4><b>4 —</b> The Git Repo for this course!</h4>
|
||||
<pre class="preformatted">wget https://git.i2pd.xyz/Left4Code/L4C_Forensics_CTF/ -O h1.html && sha*sum $_</pre>
|
||||
<p>Basically, you can hash whatever your heart desires if you're thinking hard enough. I'll manipulate some of the above examples to instead be forensics-oriented in the next section.</p>
|
||||
|
||||
<h3 class="blog-header">Some Techniques to Make Hashing Effective for Forensics</h3>
|
||||
<p>Take this scenario for example. You're a forensic investigator and need to always be completely sure that the content given to you by someone (let's say a laptop hard-drive) will keep it's integrity and it can always be verified that nobody has modified it. How would we do that? Well.. With that new knowledge about hashing you just learned, you know that we can use it to hash the files on the drive. But let's go a step further, remember that saying "In Linux, everything is a file"? This includes drives. So instead of hashing out every single file on the drive, just hash the drive file! If anything on the drive changes, the hash will change when verified again and then you can restore from a backup or take the necessary action based on your hashing precaution. To hash a drive, it's pretty simple, you can first use another command line utility (dd) to generate a drive image, then hash it! (In the example below, sdX will need to be changed to the drive you actually want to hash.)</p>
|
||||
<h4>Quick word of <b><u>WARNING</u></b>, the command below this message will create a complete disk image clone to the actual size of your drive, running this command will effectively fill up all space on your drive. If you still want to run this, maybe get a small usb drive, put some files on it, and create a disk image and hash from that instead to get comfortable with the process.</h4>
|
||||
<pre class="preformatted">sudo dd if=/dev/sdX of=/<your_dir>/<drive_dump> bs=4M status=progress && sha*sum <drive_dump></pre>
|
||||
<p>Once we generate the .dd file for the target drive and generate the hash for it, we would theoretically be able to pass this to another investigator without the fear of it being modified and nobody knowing about it.</p>
|
||||
<p>If you don't have the disk space to copy your entire drive to another one. Then you can run this command which will directly generate a hash from your drive and only read from it and not write to it.</p>
|
||||
<pre class="preformatted">sha*sum /dev/sdX</pre>
|
||||
<p>If you want to check what drives you have available to be hashed on the system, you can use the following command to check:</p>
|
||||
<pre class="preformatted">lsblk</pre>
|
||||
<p>This would only be for the cases where you can't use dc3dd, because it has the ability to hash the .dd file immediately after and this is not necessary. However, using sha*sum on files can still be useful for things like creating hash databases, getting known hashes and inputting them into something like autopsy or sleuthkit to automatically scan for them when looking through a drive, and hashing a live linux system. </p>
|
||||
|
||||
<h3 class="blog-header">Saving the output from sha*sum to a text file</h3>
|
||||
<p>When you normally work with sha*sum, you will not be able to save the output of the hash you generate to a file, there's no -o option and it just prints to standard output so you'll have to use the shell to save the output to a file.</p>
|
||||
<pre class="preformatted">sha*sum <file_youre_hashing> > <output_file></pre>
|
||||
<h3 class="blog-header">Reading files in binary mode</h3>
|
||||
|
||||
<p>When specifying sha*sum to read in binary mode with the -b flag, this is specifically used so that binary and other files which need very careful attention to detail are read properly, sha*sum does this by reading the input file byte by byte instead of text character by text character, it is very rare that you will ever use this, but it's good to know that it exists if you need to use it for very specific circumstances where a file is presenting two different hashes depending on the mode specified.</p>
|
||||
<pre class="preformatted">sha*sum -b <file_youre_hashing> > <output_file></pre>
|
||||
<h3 class="blog-header">Comparing hash files to generated hashes with sha*sum</h3>
|
||||
<p>This is honestly useful even without the forensic context, it's important to verify the hashes of software you're downloading to ensure that the software is coming from the developers and has not been modified by a third party. The complete version of this involves using pgp keys in combination with the hashes, but to keep this simple (and also because I have no idea how to do it, when I figure it out I will update this) I will just show the check functionality for sha*sum</p>
|
||||
<p>Let's use this scenario: I'm the developer, and I want to prove to the user that the executable they are downloading comes from me and has not been tampered with, I would first hash the executable to a file like so:</p>
|
||||
<h4>The Developer:</h4>
|
||||
<pre class="preformatted">sha*sum --tag <the_executable> > <the_hash_file></pre>
|
||||
<p>the "--tag" makes it so sha*sum won't throw a beginning error when you check the hash file against the file you're running sha*sum on and sha*sum will add a little more content to the ouput file showing the correlation of the file and the hash, not putting --tag does not negatively impact sha*sum's ability to check the hash file compared to what it is being ran against. I would then include the hash file and the executable file together so that the user can download both, then, as the user, I would download both and then run the following command:</p>
|
||||
<h4>The User:</h4>
|
||||
<pre class="preformatted">sha*sum -c <the_executable> <the_hash_file></pre>
|
||||
<p>The output of this command should say OK somewhere in the terminal, if it does not, and says FAILED: checksum did not match, then you know someone's up to some funny business and you probably shouldn't install that piece of software.</p>
|
||||
|
||||
<h4>Conclusion</h4>
|
||||
<p>This covers most of the functionality of the sha*sum utilities and the md*sum utilities. With this you should be able to hash basically anything you want and be able to check and verify that hashes you receive are correct and actually coming from a valid source.</p>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
</body></html>
|
||||
28
courses/itscoming.html
Normal file
28
courses/itscoming.html
Normal file
@@ -0,0 +1,28 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Left 4 Code</title>
|
||||
<link rel="icon" type="image/x-icon" href="../favicon/favicon.ico">
|
||||
<link rel="stylesheet" type="text/css" href='../style.css'>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="../index.html">Home</a>
|
||||
<a href="../blog.html">Blog / Courses</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">It's on the way, don't worry!</h1>
|
||||
|
||||
</footer>
|
||||
|
||||
</html>
|
||||
116
courses/metadata_forensics/exiftool.html
Normal file
116
courses/metadata_forensics/exiftool.html
Normal file
@@ -0,0 +1,116 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="icon" type="image/x-icon" href="favicon/favicon.ico">
|
||||
<title>Left4Code - (exiftool)</title>
|
||||
<base href="../../">
|
||||
<link rel="stylesheet" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="index.html">Home</a>
|
||||
<a href="blog.html">Blog / Courses</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">exiftool For Metadata Viewing, adding, and removal</h1>
|
||||
<hr>
|
||||
<p>exiftool is the ffmpeg of metadata tools, a billion options (most of which you'll never even comprehend exist), decent guides if you look hard enough for them, and will most likely copy commands from a website into your terminal to use this utility (hopefully this guide makes it so you don't have to do that as much.)</p>
|
||||
<hr>
|
||||
<p>Before diving in, much like that last metadata guide I did on mat2, don't trust what I do completely for inputting commands if the data you're working on is of vital importance to you or someone else, make a backup (3, 2, 1 rule if it's that serious please.) before you continue reading. But if you don't care about what you do to the data you're working on, go right ahead and continue!</p>
|
||||
<hr>
|
||||
<p>[NOTE] <a href=https://exiftool.org/examples.html>exiftool.org/examples.html</a> is a really solid website which probably has all of the information query commands you would ever want to know with nice little descriptions based on what you want to enter given the little html buttons. This site is awesome for this and I won't gatekeep this information and just pick out what I think is useful from the manual, this course intends to give you a jumping point and throw the kitchen sink of resources your way to go further, I don't know who you are and you may need this tool for a different purpose than what I outline, so to cover all my bases, it's there. Also read the manual, or learn to read it. I mention how you can learn to read and navigate man-pages in the mat2 section ("man man")</p>
|
||||
<p>Additionally, if you are reading the man-pages and are wondering to yourself, "Hmm, I want to see the tags with their ID's, but I don't know where the heck 'Image::Exiftool::TagNames(3pm)' even is! Well, fear not. All you have to do read these pages, much like <b>man 7 man</b> to read the 7'th page of the man manual as an example, you can do <b>man 3pm Image::Exiftool::TagNames</b> to see the tag information and any other pages that you might want with this format."</p>
|
||||
|
||||
<h3 class="blog-header">What this page covers (To Not Waste Your Time!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Warnings about the use of exiftool (same warnings as mat2)
|
||||
<b>—</b> Installing exiftool using apt
|
||||
<b>—</b> Understanding exiftool's tag system
|
||||
<b>—</b> How to view metadata using exiftool
|
||||
<b>—</b> How to edit metadata using exiftool
|
||||
<b>—</b> How to copy metadata using exiftool
|
||||
<b>—</b> Closing notes and a fun challenge!
|
||||
</div>
|
||||
|
||||
|
||||
<h4>Warnings before using exiftool</h4>
|
||||
<p>Final warning, back up the data you're running exiftool on if it is of importance to you. Once you get the hang of it, then you can get a little less cautious.. But I didn't tell you that. Safety first!</p>
|
||||
<h3 class="blog-header">Installing exiftool</h3>
|
||||
<p>exiftool is available in most debian repositories and can probably be installed using the command:</p>
|
||||
<pre class="preformatted">sudo apt install exiftool</pre>
|
||||
<h3 class="blog-header">Understanding exiftool's tag system</h3>
|
||||
<p>This is probably going to get really rambly, and will 1000% be updated as I read this massive man-page more and more, but before I go into the tagging system I'd like to acknowledge how insanely packed this utility is with features you'll probably never use, it's awesome.</p>
|
||||
<p>As far as the tagging system goes, from what I can see so far, if you run the man command above to show the image tags, you'll eventually stumble upon a massive list(s) with the table in the format of the following:</p>
|
||||
<div style="white-space: pre-wrap">
|
||||
Tag ID | Tag Name | Group | Writable
|
||||
</div>
|
||||
|
||||
<p>This page contains probably every Exif, IPTC, XMP, GPS, GeoTiff, PLUS, ICC_Profile, PrintIM, Photoshop, Apple, NikonSettings, Canon, CanonCustom, CanonVRD, Casio, DJI, FLIR, FujiFilm, GE, HP, JVC, Kodak, Leaf, etc, etc, (I haven't made it past 35% of the man-page yet by the way) tag you will ever have the possibility of seeing in your lifetime and the lifetime of everyone else on planet Earth until the heat death of the universe.</p>
|
||||
<p>The entire tag page for this utility is 30k+ lines long (and that's just the tag page), it's insane when you look at the Author section of the man-page and see a single name. Phil Harvey.</p>
|
||||
<p>So appreciate the author, if you ever make it in forensics and have some money to spend, donate some of it to him at exiftool.org</p>
|
||||
<p>Now, after you've comprehended how bananas this tool is for metadata extraction, I'll show you how to utilize the unique tags for viewing metadata.</p>
|
||||
<p>If you want to read specific exif sections on a file, you can do the following:</p>
|
||||
<pre class="preformatted">exiftool -s -Make -Orientation <yourfile></pre>
|
||||
<p>Notice how I specified the tags in title case as shown in the 3pm Image::Exiftool::TagNames man-page under EXIF Tags. I'm making this clear to you because the formatting is different from the internal tags used compared to what gets shown to you on standard output in the terminal, namely tags involving spaces. In the above command, -s means shorten output (and -S means remove whitespace!) you don't need these options, as long as you specify those tags, It'll show those tags.</p>
|
||||
|
||||
|
||||
<h3 class="blog-header">Viewing metadata with exiftool (files & folders)</h3>
|
||||
<p><b>To view most of the metadata with exiftool, we can use the command:</b></p>
|
||||
<pre class="preformatted">exiftool -a -u -g1 <yourfile></pre>
|
||||
<p>this pulls a lot of the metadata in the file, but does not pull all of it for performance reasons, if you want to pull absolutely everything exiftool can get from the file, you can run the following:</p>
|
||||
<pre class="preformatted">exiftool -a -u -g1 -ee3 -api RequestAll=3 <yourfile></pre>
|
||||
<p>Would you like to output your metadata in HTML, JSON, XML, a PHP array, or even add it to a csv file? Well do I have something for you! <b>-j</b> for json, <b>-h</b> for html, <b>-X</b> for XML, <b>-php</b> for php, and <b>-csv >$(pwd)/<yourcsv></b> to output the meta into a csv! (if you don't know what $(pwd) does, then look it up, it shouldn't be too hard to figure out. I'd love to refer you to my linux course, but sadly. It's not made.... Yet.... Maybe.)</p>
|
||||
|
||||
<h3 class="blog-header">Writing metadata with Exiftool</h3>
|
||||
<p>Much like reading the metadata using exiftool, to write to specific tags they need to be specified, from reading the manual I can see that the important writing modes are "-", "+", and "=". Just specify the specific tag you want to edit using the specific tag you want with a tag in front of it and the specific operation you want to do with that tag after it.</p>
|
||||
<h4>Completely changing the value at the Comment tag</h4>
|
||||
<pre class="preformatted">exiftool -Comment="Hello there, this is a metadata test!" <CoolFile></pre>
|
||||
<h4>Removing a tag's data from the file</h4>
|
||||
<pre class="preformatted">exiftool -Comment-="Hello there, this is a metadata test!" <CoolFile></pre>
|
||||
<h4>Adds 8 hours to the value already at the createdate tag</h4>
|
||||
<pre class="preformatted">exiftool -createdate+=8 <yourfile></pre>
|
||||
<p>If you look through the man-page where all the tags are listed (find the command at the top of the page), you can also find larger metatags which when used can modify multiple elements at the same time, meaning that for example you could add all the IFD0 tags to a file at the same time. This functionality should be 99% of what you need for the basic tasks of writing or editing metadata to a file for plain forensic lab purposes.</p>
|
||||
<p>I almost forgot the most important part, you can remove all metadata from a file like:</p>
|
||||
<h4>Remove all metadata exiftool can from a file</h4>
|
||||
<pre class="preformatted">exiftool -all= <yourfile></pre>
|
||||
|
||||
<h3 class="blog-header">Copying metadata from one file to another with exiftool</h3>
|
||||
<p>Now that's something I don't think you can do with mat2, exiftool is able to rip the metadata out of a file and put it into another, you can do so like this:</p>
|
||||
<pre class="preformatted">exiftool -tagsfromfile <yourfile_1> <yourfile_2></pre>
|
||||
<p>This will copy all of the metadata out of the first file and write it to the second file. In the manual examples page it also states how you can copy certain metadata tags individually from one file to another, it's kind of hard to find though so I will add an example of how to do it below:</p>
|
||||
|
||||
<h4>Takes tags from yourfile_1, selects the IFD0 tags, then writes them to yourfile_2</h4>
|
||||
<pre class="preformatted">exiftool -TagsFromFile <yourfile_1> -CommonIFD0 <yourfile_2></pre>
|
||||
|
||||
<h3 class="blog-header">Conclusion</h3>
|
||||
<p>exiftool is such a massive utility that I obviously won't be able to cover everything it can do, but hopefully the exiftool.org forums and the man-pages will be enough for you to find what you need if it wasn't outlined here, but if you're doing forensics what I've written here is probably all you'll need for reading metadata for an investigation.</p>
|
||||
<h3 class="blog-header">Challenge (BKFLAG)</h3>
|
||||
<p>Let's have a little throw back to 2012 when <a href="https://archive.org/details/originalbkflimage">this fun image</a> showed up on a little web forum back in the day. It has the metadata and GPS location in it still (Cartwheel76 and Zubes, thank you!). To complete this challenge, follow these guidelines (or don't, figure something else out that solidifies all this learning!)</p>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>1)</b> Head over to the <a href=https://git.i2pd.xyz/Left4Code/L4C_Forensics_CTF/src/branch/master/Metadata%20Forensics>L4C Forensics Git Repository</a> for this course and download the gpg file in addition to the BKFL photo.
|
||||
|
||||
<b>2)</b> Use exiftool (and mat2 if you read the guide) to determine what kind of phone took the photo
|
||||
|
||||
<b>3)</b> Copy the phone exact model (ex. Oneplus 7 Pro) [The capitalization of the phone model matters!] from exiftool and paste it into the gpg decrypt prompt when you run gpg on the encrypted file from the terminal in order to decrypt it and claim your prize of 1 hackerman cat photo, YOU NEED GPG TO DO THIS!!
|
||||
|
||||
<pre class="preformatted">sudo apt install gpg</pre>
|
||||
<pre class="preformatted">gpg BKFLAG.gpg</pre>
|
||||
<b>4)</b> Modify the phone model to a different model of phone (or just say something funny or mess with the cat photo's metadata in whatever way you want)
|
||||
</div>
|
||||
|
||||
|
||||
</section>
|
||||
</div>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
89
courses/metadata_forensics/mat2.html
Normal file
89
courses/metadata_forensics/mat2.html
Normal file
@@ -0,0 +1,89 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="icon" type="image/x-icon" href="favicon/favicon.ico">
|
||||
<title>Left4Code - (mat2)</title>
|
||||
<base href="../../">
|
||||
<link rel="stylesheet" type="text/css" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<span>Left4Code</span>
|
||||
</header>
|
||||
|
||||
<nav>
|
||||
<div>
|
||||
<a href="index.html">Home</a>
|
||||
<a href="blog.html">Blog / Courses</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container">
|
||||
<section>
|
||||
<h1 class="blog-header">mat2 For Metadata Viewing and Removal</h1>
|
||||
<p>mat2 (Metadata Anonymization Toolkit) is a pretty nice simple tool written in python that supports the viewing and removal of metadata for various file formats like pdf, docx, jpg, png, zip, etc etc, you get it. It's purpose seems to be a quicker way to extract the juicy important information out of files without cracking open exiftool and going bananas. Simple command arguments, simple life.</p>
|
||||
|
||||
<h3 class="blog-header">What you need to know (To get the most out of this!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Basic understanding of the Linux command line (bash) Basically, do you understand a basic command like ls -alh
|
||||
|
||||
<b>—</b> Some determination to read, mentally digest, practice, and learn for yourself.
|
||||
|
||||
<b>—</b> How to use the manpages (run "man man" without the double quotes in your terminal if you don't know) this is so you can always use the manpages if this course doesn't get completed or updated. I want to teach you to fish, not give you fish.
|
||||
|
||||
<b>—</b> Whenever I put carat symbols outside of something, don't add them to the command in your actual terminal, ex: <yourfile> should be typed in your terminal as yourfile, or whatever you want to name it, it's just a placeholder, you get it.
|
||||
</div>
|
||||
<h3 class="blog-header">What this page covers (To Not Waste Your Time!)</h3>
|
||||
<div style="white-space: pre-wrap">
|
||||
<b>—</b> Warnings about mat2 and potential data loss (only worry if wiping metadata)
|
||||
<b>—</b> Installing mat2 using apt
|
||||
<b>—</b> How to view metadata using mat2 quickly
|
||||
<b>—</b> Running mat2 on folders & files for showing and wiping
|
||||
<b>—</b> Figuring out what file-types mat2 supports
|
||||
<b>—</b> metadata removal without backing up the file
|
||||
<b>—</b> lessening the amount of data removed with mat2
|
||||
<b>—</b> Closing notes
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<h4>Warnings when it comes to using mat2</h4>
|
||||
<p>Straight to the point, if you are running mat2 for wiping on a file which it's core hashed integrity is of utmost importance or if you want to make completely sure that the file you are removing the metadata from will still open afterwords, make a backup of the file first, mat2 does this by default so you should be fine, but if you want to make double sure, back one up for yourself too. Quick note, I've never had mat2 tweak out on me after my years of using it, so this is just a defcon 1 level precaution in case you have one really bad day. With that out of the way, let's get into it.</p>
|
||||
<h3 class="blog-header">installing mat2</h3>
|
||||
<p>mat2 is available in most distribution repositories and can probably be installed using the command:</p>
|
||||
|
||||
<pre class="preformatted">sudo apt install mat2</pre>
|
||||
<h3 class="blog-header">Viewing metadata with mat2 (files & folders)</h3>
|
||||
<p><b>To view metadata with mat2, we can use the command:</b></p>
|
||||
<pre class="preformatted">mat2 -s <yourfile></pre>
|
||||
<p>or run it on a directory to recursively check everything inside of it!</p>
|
||||
<pre class="preformatted">mat2 -s yourdirectory/</pre>
|
||||
<p>You can specify the -V option (note the capital) to display verbose debug information about what mat2 is doing, this can be added as a flag regardless of wiping or viewing metadata, but it's really only been useful in my experience when removing metadata.</p>
|
||||
<pre class="preformatted">mat2 -Vs yourdirectory/</pre>
|
||||
<p><u>mat2 can also be run on zip files to get at least the creation date of the files inside it and maybe some more metadata, cool to know.</u></p>
|
||||
|
||||
<h3 class="blog-header">File-types mat2 supports for wiping & viewing</h3>
|
||||
<p>mat2 supports many files like flac, a bunch of mpeg family file-types, jpeg, png, html and a bunch of others. A full exhaustive list of the supported file-types for mat2 can be found by running:</p>
|
||||
<pre class="preformatted">mat2 -l</pre>
|
||||
|
||||
<h3 class="blog-header">Metadata Removal with mat2</h3>
|
||||
<p>By default, if the -s parameter is not passed to mat2, it will remove all of the metadata it can from the file, this means that by learning how to show the metadata of a file with mat2, you also learned how to wipe the metadata of a file with mat2! Simply remove the -s from the previous commands you used above to wipe the metadata from the selected file or directory.</p>
|
||||
<pre class="preformatted">mat2 <yourfile></pre>
|
||||
<p>Also by default, when wiping metadata with mat2 it will run the process on a separate instance of the file and upon success will load the cleaned file into the working directory the file is in with the new extension .cleaned appended to the end of the file name. <u> Meaning NoMetadata4U.pdf would turn into NoMetadata4U.pdf.cleaned</u></p>
|
||||
|
||||
<p>You can additionally use the -l flag to lessen the amount of metadata that mat2 removes, and use the --inplace flag to not create a backup of the file before scrubbing the metadata from it.</p>
|
||||
<pre class="preformatted">mat2 -l <yourfile></pre>
|
||||
<pre class="preformatted">mat2 --inplace <yourfile></pre>
|
||||
|
||||
<h4>Conclusion</h4>
|
||||
<p>This is about 90% of mat2's functionality and will get you through most metadata viewing and wiping tasks. luckily, if you installed mat2 through a package repository, it probably comes with a manpage, so read that if you would like information on the one or two flags I didn't include here.</p>
|
||||
<h4>[Challenge approaching!!!]</h4>
|
||||
<p>In the Exiftool lesson, there will be a fun challenge for you to do where you can use a combination of both mat2 and exiftool to complete a CTF which you can put on the certification document as proof of completion for this section.</p>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user