feature/NewHashingAlgorithms #6

ProfessionalUwU · 2023-07-04T00:32:46+02:00

ProfessionalUwU commented

2023-07-04 00:32:46 +02:00

Breaking changes

Redis

I am now using redis to overcome the limitations of SQLite.
This also means that any setup before will be broken by this update.
If you want to save the hashes to a database then you can use the saveToSqlite option.

New features

New hashing algorithms

MurmurHash is considerably faster than MD5 and XxHash is even faster than MurmurHash when tested on mixed file sizes.

Buffer size

The buffer size of the two new hashing algorithms can be configured with a commandline option.
The default value is 4096 bytes. Higher values are recommended for faster drives.

Progress bar

The program finally has a progress bar so one will know how far along the checksumming process is.

# Breaking changes ## Redis I am now using [redis](https://stackexchange.github.io/StackExchange.Redis/) to overcome the limitations of SQLite. This also means that any setup before will be broken by this update. If you want to save the hashes to a database then you can use the saveToSqlite option. # New features ## New hashing algorithms 1. [MurmurHash](https://github.com/odinmillion/MurmurHash.Net) 2. [XxHash](https://github.com/uranium62/xxHash) MurmurHash is considerably faster than MD5 and XxHash is even faster than MurmurHash when tested on mixed file sizes. ## Buffer size The buffer size of the two new hashing algorithms can be configured with a commandline option. The default value is 4096 bytes. Higher values are recommended for faster drives. ## Progress bar The program finally has a progress bar so one will know how far along the checksumming process is.

ProfessionalUwU added the

Kind/Breaking

Kind/Enhancement

Kind/Feature

labels 2023-07-04 00:32:46 +02:00

ProfessionalUwU self-assigned this 2023-07-04 00:32:46 +02:00

ProfessionalUwU added 8 commits 2023-07-04 00:32:46 +02:00

New hashing algorithm f65108425c

Add new hashing algorithms 9d6b1385c8

Add XxHash
Code cleanup

Add option for choosing hash algo d80a5f5e6b

Major advancments 02a0bddd7e

New options
Switched to wal (Write ahead log)
Now using redis to cache filehashes
New option to dump redis data into sqlite database

feature/progressBar (#4 ) b011f08172

# Add progress bar
Added a progress bar so that one can see how far along the program is.

Reviewed-on: #4
Co-authored-by: ProfessionalUwU <andre.fuhry@hopeless-cloud.xyz>
Co-committed-by: ProfessionalUwU <andre.fuhry@hopeless-cloud.xyz>

Fix logging 336eb5b73d

Runtime dependency d7aa7a6d98