How, why, and when you should hash check

There is some convenience to commercial software, but this often comes with the caveat of being closed source. Here at ProPrivacy, we applaud the transparency of open source, as it is the only definitive way to know that a program is what it says it is on the tin with no funny business.

It’s wonderful that open source software can be used by anyone as the building blocks to other programs, but this is, in fact, a double-edged sword. This makes it difficult to tell whether the software you have just downloaded is what was intended by the developer(s) or something that has been modified somewhere along the way.

Thankfully, a partial solution to this can be found by checking cryptographic hashes.

What is a cryptographic hash?

A cryptographic hash is a checksum or digital fingerprint derived by performing a one-way hash function (a mathematical operation) on the data comprising a computer program (or other digital files).

Any change in just one byte of the data comprising the computer program will change the hash value. The hash value is, therefore, a unique fingerprint for any program or other digital files. 

What is a hash check?

Ensuring that a program has not been tampered with, or just corrupted, is a fairly simple matter of calculating its hash value and then comparing it with the hash checksum provided by its developers.

If it's the same, then you have a reasonable degree of confidence that the program you have downloaded is exactly the same as the one published by its developer. If it is not, then the program has been changed in some way.

The reasons for this are not always malicious (see below), but a failed hash check should set alarm bells ringing. 

Problems with hash checks

You may have detected a note of caution when singing the praises of hash checks...

Integrity but not authentication

Hash checks are useful for ensuring the integrity of files, but they do not provide any kind of authentication. That is, they are good for ensuring the file or program you have matches the source, but they provide no way of verifying that the source is legitimate. 

For example, fake websites exist which distribute malicious versions of popular open source software such as KeePass. Many of these websites even provide hash checksums for the programs they supply and, were you to check these against the fake program, they would match. Oops.

Mathematical weaknesses

An additional problem is that mathematical weaknesses can mean that hashes are not as secure as they should be. 

The MD5 algorithm, for example, remains a highly popular hash function, despite its known vulnerability to collision attacks. Indeed, even SHA1 is no longer considered secure in this regard.

Despite this, MD5 and SHA1 remain the most popular algorithms used to generate hash values. SHA256, however, remains secure.

Developer laziness

Developers sometimes update their programs with bug fixes and new features, but neglect to publish an updated hash checksum. This results in a failed hash check when you download and try to verify their program.

This is, of course, nowhere near as serious as a hash check giving malicious software a pass, but it can degrade trust in the ecosystem, resulting in people not bothering to check the integrity of files they download...

Cryptographic hashes vs digital signatures

Most of the problems with cryptographic hashes are fixed by the use of digital signatures, which guarantee both integrity and authentication.

Developers who are happy to use proprietary code can automatically and transparently validate signatures when their software is first installed, using mechanisms such as Microsoft, Apple, or Google PKI (public key infrastructure) technologies.

Open source developers do not have this luxury. They have to use PGP, which is not natively supported by any proprietary operating system, and why no equivalent of Microsoft, Apple or Google PKIs exist in Linux. 

So PGP digital signatures must be verified manually. But PGP is a complete pig to use and is not a simple process, as a quick glance at our guide to checking PGP signatures in Windows will demonstrate.

Neither is the actual signing process for developers, who are well aware that in the real world few people bother to check digital signatures manually, anyway.

Cryptographic hashes are nowhere near as secure as PGP digital signatures, but they are much easier to use, with the result that many developers simply choose to rely on them instead of digitally signing their work.

You really should hash check (if no digital signature is present)

This is a less than ideal situation, and you should always check an open source program’s digital signature when one is available. If one is not, however, then checking its cryptographic hash is much better than doing nothing.

As long as you are confident about the source (for example you are sure it's from the developer’s real website, which has not been hacked to display a fake cryptographic hash), then checking its hash value provides a fair degree of confidence that the software you have downloaded is the software its developer intended for you to download. 

If neither a digital signature nor a checksum is available for open source software, then do not install or run it. 

How to hash check

The basic process is as follows:

If they are identical, then you have the file the developer intended you to have. If not, then it has either become corrupted or has been tampered with.

If an SHA256+ hash is available, check against that. If not, then use SHA1. Only as a last resort should you check against an MD5 hash. 

The easy way (all systems)

The simplest way to generate the hash value of files is by using a website such as Online Tools. Just select the kind of hash value you need to generate, then drag-and-drop the required file into the space provided and the relevant hash value will be generated. 

Example

We want to check the integrity of the KeePass installer file that we have downloaded from the KeePass.org website (which we know to be the correct domain). The website publishes MD5, SHA1, and SHA256 hashes for all versions of its KeePass, so we will check the SHA256 for the version we downloaded.

1. We make a note of the correct hash value, as published on the KeePasswebsite.

2. We then visit the Online Tools website, select File Hash: SHA256, and drag our downloaded KeePassinstaller file into the space provided.

3. We compare the output with the checksum published on the KeePass website and they are identical. So assuming that the KeePass website has not been hacked to display false hash values, we can be confident that we have downloaded an untampered-with file. Yay!

    

   Windows, macOS, and Linux also have built-in hash value functions which can be accessed via the command-line...

Windows

This method works out-of-the box in Windows 10, while Windows 7 users need to first update Windows PowerShell with Windows Management Framework 4.0.

To obtain an SHA256 hash, right-click Start -> Windows PowerShell and type:

Get-FileHash [path/to/file]

For example:

Get-FileHash C:\Users\Douglas\Downloads\KeePass-2.43-Setup.exe

MD5 and SHA1 hashes can be calculated using the syntax:

Get-FileHash [path to [path/to/file] -Algorithm MD5

and

Get-FileHash [path to [path/to/file] -Algorithm SHA1

For example:

Get-FileHash C:\Users\Douglas\Downloads\KeePass-2.43-Setup.exe -Algorithm MD5

macOS

Open Terminal and type:

openssl [hash type] [/path/to/file]

Hash type should be md5, SHA1, or SHA256. 

For example, to check the SHA256 hash for the Windows KeePass installer (just to keep things simple for this tutorial), type:

openssl sha256 /Users/douglascrawford/Downloads/KeePass-2.43-Setup.exe

Linux

Open Terminal and type either:

Md5sum [path/to/file]Sha1sum [path/to/file]

or

Sha256sum [path/to/file]

For example:

sha256sum /home/dougie/Downloads/KeePass-2.43-Setup.exe