Major changes from 1.9.0-jumbo-1 (May 2019) in this bleeding-edge version:

- Bug fixes, most importantly in OldOffice-opencl format, which could give

false negatives on some or many builds. [Solar, magnum, others; 2019]

- Documentation fixes and improvements. [Solar, magnum, others; 2019]

- Improved our portable hi-res timer for nano-second resolution on most

archs, and using uint64_t instead of double. This timer isn't used much

yet, but may replace other timers in the future. [magnum; 2019]

- Never print the "spinning wheel" unless stderr is a tty. [magnum; 2019]

- Added new option --rules-skip-nop for when you already ran some attack

on some slow format without rules and now want to run it *with* rules.

[magnum; 2019]

- Mute warning about low buffer-fill in case the difference in time appears

to correspond to less than one second. [Solar/magnum; 2019]

- Dropped the RelaxKPCWarningCheck config option as it was obsoleted with

the above logic [magnum; 2019]

- If "Many salts" test exceeded 90% of target, force finish so not to get

annoying/confusing messages. [Solar/magnum; 2019]

- Always use fcntl() locks. Handle spurious EAGAIN despite waiting for lock

(likely due to ulimits or server limits). [magnum; 2019]

- Implement some tricks for automagically decrease risk of file lock clashes

when running a large number of nodes. [magnum; 2019]

- Bugfix in zip2john for stream archives in certain situations. [magnum; 2019]

- Add --fork ability to ZTEX formats. [Denis Burykin; 2019]

- tezos2john: Added the ability to validate Tezos Seed Words.

[LordDarkHelmet; 2019]

- Added --length=N which is a shortcut for --min-length=N --max-length=N

[magnum; 2019]

- Add functions for querying host's total or available physical memory, if

possible. Among other things, this will be used for detecting when an OpenCL

buffer that is backed by an equal size host buffer gets too big.

[magnum; 2019]

- Single mode: Add new config option SingleMaxBufferAvailMem. If true, actual

amount of physical memory will override SingleMaxBufferSize (it may increase

or decrease). If that option isn't set but SingleMaxBufferSize is explicitly

set to zero, no limit will be applied! [magnum; 2019]

- When running "-test -mask" benchmark, don't disable self-tests. Instead

perform a non-mask self-test before the mask-mode benchmark. [magnum; 2019]

- Default to benchmarking with mask. For benchmarking without mask, use the

new --no-mask option. [magnum; 2019]

- Add format match symbol '#' for searching in format_name. For example,

--format=#ipmi,+opencl will pick the RAKP-opencl format. We already had

the '@' symbol for searching in alorithm name. [magnum; 2019]

- Reduce repeated log messages with fork/node/MPI. [magnum; 2019]

- Allow wildcard for --users/groups/shells options. Also makes them case

insensitive when wildcard is used. [magnum; 2019]

- When ShowUIDinCracks = Y in config and format is WPAPSK, actually print

gid instead, as that is the access point's MAC address. [magnum; 2019]

- Show "same-salt boost" figure if applicable. Add john.conf boolean options

ShowSaltProgress and ShowRemainOnStatus for some extra status output.

[magnum; 2019]

- dmg2john: Recognize .backupbundle as an alias for .sparsebundle

[magnum; 2019]

- Add option --no-keep-guessing that "turns off" FMT_NOT_EXACT just like the

old --keep-guessing option "turns it on". [magnum; 2019]

- Worked out a new KISS solution "FMT_BLOB" for non-hash formats, so they can

still benefit from same-salt optimizations/exploits. [magnum; 2019]

- Converted WPAPSK formats to the new FMT_BLOB scheme, with excellent

improvement in performance, especially for single mode. [magnum; 2019]

- Converted Office formats to the new FMT_BLOB scheme. [magnum; 2019]

- Converted oldOffice CPU format to the new FMT_BLOB scheme. The GPU version

uses internal mask so may be hard or impossible to convert. [magnum; 2019]

- Converted RAR3 formats to the new FMT_BLOB scheme. [magnum; 2019]

- zip2john: Refactor and support reading central directory. [srett; 2019]

- Added support for Enpass 6.x database files. [dsmith, magnum; 2019]

- Single mode: Fix a buffer overflow when max. length is decreased.

[magnum; 2020]

- Keys [<] and [>] can now be used to decrease/increase verbosity level.

[magnum; 2020]

- At verbosity level 2, only print admin cracks to screen. [magnum; 2020]

- Added --single-user-seed=FILE option for single mode. FILE is a wordlist

with seeds per username (user:password[s] format). [magnum; 2020]

- Added a workaround for some nvidia bug causing keepass-opencl to fail using

recent drivers. [magnum; 2020]

- Added a warning for devices only supporting OpenCL 1.0. [Claudio; 2020]

- Added -mlc option to unique - "max. length considered". [magnum; 2020]

- 7z formats: Added support for BCJ and other preprocessors sometimes used

before compression. Before this we had false negatives for such files.

Also updated LZMA code from upstream, to version 19.00. [magnum; 2020]

- Upgraded Unicode stuff from version 11.0.0 to 13.0.0. This affects the

external modes Dumb16, Dumb32, Repeats16 and Repeats32. It also affects

the Easter egg option "--subsets=full-unicode" which BTW supersedes the

external modes with far better performance. [magnum; 2020]

- Add support for ssh new-style private keys encrypted using `aes256-ctr`

cipher. [vkhromov; 2020]

- Implement full checking in SSH OpenCL format, and stop it as well as the

CPU SSH format from defaulting to --keep-guessing. Bugs were squashed as

well. For any remaining problems with false-positives (if any), user can

selectively use the --keep-guessing option. [magnum; 2020]

- Add support for --format=LIST [magnum; 2020]

- If command-line or config-file rules are given in UTF-8 but we're utilizing

a legacy internal codepage, convert them to that codepage. Mask mode already

had the corresponding feature. [magnum; 2020]

- Improve handling of using stdin or pipe vs. reading keystrokes. This

includes a new option --force-tty that will set the terminal up for

reading status/quit keystrokes even if we're not the foreground process.

[magnum; 2020]

- Make keypress [d] special: It will also emit a status line, but not until

the current batch has been run among all salts. Under some circumstances

that may take a good while. [magnum; 2020]

- Add command-line option --[no-]loader-dupe-check, overriding config file.

Mark the latter as deprecated in config file comments. [magnum; 2020]

- Add alternative syntax to --salts option for loading "N most populated

salts" as opposed to "salts having at least N hashes". [magnum; 2020]

- Add support for Telegram Desktop >= 2.2.0/2.1.14b, CPU-only for now.

[philsmd, magnum; 2020]

- Drop support for --fix-state-delay option, and instead do something better

in the cracker engine - by default, and for all modes. [magnum; 2020]

- Addition of mosquitto2john.py for cracking of Eclipse Mosquitto passwd files.

[blackfell; 2021]

- Final fixes for zip2john selection of 1-byte or 2-byte early-reject checks,

as well as exactly what source to use for that check, after seeing several

issues and digging into the specs. Note that a re-run of zip2john is needed

for correct cracking. [magnum; 2021]

- Allow any node range divisible by fork (or MPI) count. [Solar, magnum; 2021]

- Fix autoconf selection of arch.h for Apple M1. Improve ARM pseudo-intrinsics

portability (would fail on clang). Allow RAR formats to build on ARM where we

detect it can handle unaligned memory access. [magnum; 2021]

- Add options --subsets-prefer-short and --subsets-prefer-small to Subsets

mode, affecting candidate order. [magnum; 2021]

- Support "./configure --without-openssl" on all platforms, which excludes much

functionality, but allows the build to complete. (Previously, this was only

supported on macOS and required use of Apple's CommonCrypto.) [Solar; 2021]

- Repair "make -f Makefile.legacy linux-mic" (easy cross-compilation for first

generation Xeon Phi) and make it not depend on zlib by default. The OpenSSL

and GMP dependencies are now also easy to exclude by commenting out lines in

Makefile.legacy, for all targets. [Solar; 2021]

- Drop support for Apple's CommonCrypto, it was defunct for many years anyway.

[magnum; 2021]

- Improved Monero support: Now supports legacy wallets from before 2016 that are

not using JSON format yet. [patrickd; 2021]

- Add --without-unrar option to configure, for building without the non-free

ClamAV unrar code (crippling the RAR v3 formats a little). Simply deleting

src/unrar*.[ch] will infer that option as well. [magnum; 2021]

- Support --target-encoding with --make-charset, for generating charsets for

formats like LM that need to be in a legacy codepage. [magnum; 2021]

- Incremental: Allow supplying a charset file name as "mode", with no config

entry required. [magnum; 2021]

- Cracker: Re-transfer salt every 30 seconds even if only one salt is loaded.

This minimizes impact if a device-side salt gets thrashed. [magnum; 2021]

- pwsafe2john.py: Fixed parsing bug of iteration count, that led to

uncrackable hashes in the past. [NecroMortis; 2021]

- Python 3 compatibility and other improvements in various *2john.py scripts

[exploide; 2019-2024]

- Add new option --catch-up=NAME, for running a new session only until it

reaches the candidates tried count of a different, existing and paused

session, then exit. [magnum; 2021]

- Add BestCrypt Volume Encryption V4 format. [Jean-Christophe Delaunay; 2021]

- Addition of bestcryptve2john.py for cracking of BestCrypt Full Disk Encryption.

[trounce1 / HN; 2021]

- Added missing code to the Office formats for cracking Office 2007 documents

with a 256-bit key (apparently very uncommon). The CPU format would bail

with a message (hence the issue reported) but unfortunately the OpenCL format

just produced false negatives until now. [magnum; 2021]

- Added formats for cracking cryptoSafe vaults. They are loaded as-is: There's

no "cryptosafe2john" or such involved. [magnum; 2021]

- Added sense2john.py: pfSense/OPNsense config.xml hash extract script.

[Private Wolf; 2021]

- The Zip (WinZip) formats got several optimizations and the OpenCL version now

handles large data sizes much better. [magnum; 2021]

- Added apop2john.py: script to extract and format POP3 APOP challenge/responses

[Mark Silinio; 2021]

- Added option --mask-internal-target=N for overriding format's idea of how

large portion of a mask should be generated on device-side (or completely

disabling internal mask). [magnum; 2021]

- Add a format class "vector" for matching vector capable OpenCL formats. This

doesn't include bitslice formats. It matches formats that will run SIMD

using the default GPU, or per the --device option. [magnum; 2021]

- Revised regex mode's support for librexgen, to version >= 2.1.5 [Jan Starke,

magnum; 2021]

- 7z2john.pl: Update from upstream. [magnum; 2021-2022]

- 7z formats: Added support for Delta encoding. [magnum; 2021]

- Added support for using a FIFO as a wordlist. [magnum; 2021]

- OpenCL: If building from a cached kernel fails, just build a new one instead

of bailing out. [magnum; 2021]

- cisco2john: Correction for decoding of "password 7". It turned out what we

had was incomplete. [magnum; 2021]

- OpenCL on NVIDIA (Linux only): Reduce the amount of CPU busy-wait in a number

of OpenCL formats. [Solar, magnum; 2021]

- OpenCL LM/DEScrypt: Override kernel selection with env. variable. [magnum;

2021]

- Added support for ENCDataVault and encdatavault2john.py script

[sylvainpelissier; 2021]

- Optimize tezos-opencl by computing ed25519_publickey() and BLAKE2b on-device,

removing the CPU bottleneck. This work was funded by the Tezos Foundation.

[Solar; 2021]

- OpenCL SHA-2: Roll the loops to reduce code size and register pressure, and

speed up OpenCL formats using this code on many devices (most notably, +50%

for pbkdf2-hmac-sha512-opencl on NVIDIA Maxwell and Pascal GPUs) while not

hurting most others tested so far. [Solar; 2021]

- Default wordlist: Replace with overlap of HIBP v8 100+ hits and RockYou.

[Solar; 2022]

- Wordlist mode: Optionally log per-rule statistics (PerRuleStats setting).

[Solar; 2022]

- Default wordlist rules: Replace with Best-by-score, which is auto-re-ordered

(via PerRuleStats) subset of optimized Single and Extra rules. [Solar; 2022]

- Print detailed status on 's' keypress and help message on 'h'. [Solar; 2022]

- Add opportunistic --dupe-suppression for wordlist(+rules) and PRINCE, enabled

by default when rules are in use. [Solar; 2022]

- Added suport for PBKDF2-SHA256 key derivation algorithm of PrivateAccess and

ENCDataVault. [sylvainpelissier; 2022]

- Add NT-long (handles up to 110 characters), implemented as a thin dynamic.

[magnum; 2022]

- Fuzz and harden many formats' hash parsing code. [Aleksey Cherepanov; 2022]

- Update the Streebog code (a.k.a. GOST R 34.11-2012). The new version doesn't

require SIMD but currently doesn't support BE. [magnum; 2022]

- Various fixes for drivers that support OpenCL 2.0 (such as nvidia and macOS)

[magnum; 2022]

- Many NSEC3 format enhancements. [Ralf Sager; 2022]

- Added support for Cardano legacy's encrypted secret keys. [ilap; 2022]

- Added cardano2john.py: exports secret.key to john format. [ilap; 2022]

- Added support for cracking blockchain wallet v4. [Solar; 2022]

- Add OneRuleToRuleThemAll and OneRuleToRuleThemStill. [Solar; 2022, 2023]

- Added krb5tgs-opencl (etype 23 TGS-REP), with device-side mask acceleration.

This includes improvements to the shared RC4 OpenCL code and related changes

to other formats. [magnum; 2023]

- Added support for cracking SNTP-MS "timeroast". [magnum; 2023]

- Add NT-long-opencl (password length of up to 125 bytes). [magnum; 2023]

- NT-opencl: 64-bit binary size. Some good performance boost depending on

number of hashes loaded. [magnum; 2023]

- Add Coinomi wallet support (coinomi2john.py). [Solar; 2023]

- Added krb5tgs-sha1[-opencl] formats for etypes 17 and 18. [magnum; 2023]

- Support for cracking InnoSetup generated Unicode installers. [Dhiru; 2023]

- Optimize the overstrike/insert and update the "All" rulesets. [Solar; 2023]

- Add Keplr wallet support (format, keplr2john.py). [Alain Espinosa; 2023]

- Added shell.nix with opencl support on AMD and Intel. [lambdajack; 2023]

- Added modern, overhauled version of pdf2john.py.a [benjamin-awd; 2023]

- Avoid hard build-time dependency on Perl (is now optional). [Solar; 2023]

- Add argon2-opencl format (Argon2 on GPU support) based on Ondrej Mosnáček's

OpenCL code. [Alain Espinosa, Solar; 2023]

- Load the OpenCL library dynamically (solves issues with its discovery on

many Windows systems). [Alain Espinosa, Solar, Claudio; 2023]

- Added a local copy of the OpenCL headers. [Alain Espinosa; 2023]

- Add Armory wallet support (accepts btcrecover data extracts), along with

optimization of shared SHA-512 SIMD code and its other uses. [Solar; 2024]

- electrum2john.py and Electrum format: Support seed_version above 13 (with

risk of false negatives on untested/future versions, but currently tested

working up to 59). [magnum, Layder76, Solar; 2022, 2024]

- Add Combinator external mode (combines words into pairs). [Solar; 2024]

- Add Shuffle external mode (tries permutations of characters). [Solar; 2024]

- External mode compiler: Compile-time evaluate most constant subexpressions.

[Solar; 2024]

- New version of radius2john.py correcting support for attacks 3.1 and 3.3

from "An Analysis of the RADIUS Authentication Protocol" by Joshua Hill,

much closer to the Perl version radius2john.pl.

[k4amos; 2024]

- Added support for SM3. [SamuraiOcto; 2024]

- Added support for Astra Linux crypt variants using GOST R 34.11-94 or GOST R

34.11-2012. [magnum; 2024]

- More optimal bitslice DES S-box expressions for NVIDIA GPUs and AVX-512.

[Sovyn Y., Solar; 2024]

- Add tokenize.pl (identify and replace most common multi-character tokens in a

wordlist or password list e.g. to augment incremental mode). [Solar; 2024]

- Add OpenCL version of PDF format. This also corrected bugs in handling of

v3 RC-40 hashes as well as more obscure key lengths that are probably not

used anywhere. [magnum; 2024]

- Add bitlocker2john.py to improve BitLocker hash extraction. [holly-o; 2024]

- Sync with latest upstream Argon2 and BLAKE2 code. For Argon2, this adds

the Argon2id flavor and AVX2 and AVX-512 support. [magnum, Solar; 2024]

- Add KDBX4 support to KeePass formats (and keepass2john). This means Argon2

support as well, and keepass-argon2-opencl is made a separate format.

[magnum; 2024]

- Upgraded Unicode stuff from version 13.0.0 to 16.0.0. This affects the

external modes Dumb16, Dumb32, Repeats16 and Repeats32. It also affects

the Easter egg option "--subsets=full-unicode" which BTW supersedes the

external modes with far better performance. This time some definitions of

character classes got (very minor) changes - this can affect resuming old

jobs [magnum; 2024]

- Dropped our old AES-NI code in favor of the AES code from mbedTLS, which

supports AES-NI (Intel) as well as AES-CE (Arm). The new code kicks in for

any format using AES. Boosts of up to 13x seen on Intel and 7x on MacBook

M1 (those are for the KeePass format with AES-KDF, which is extreme because

all the heavy lifting is AES). [magnum, Solar; 2024]

- Revised shared OpenCL AES code for a 3-6x boost (depending on device), again

affecting many formats but only significant for a handful of them. [magnum;

2024]

- Add fvde2john.py to improve FileVault 2 hash extraction. [holly-o; 2024]

- Add oracle2john.py and fix the o5logon format to support passwords longer

than 16. [k4amos; 2024]

Major changes from 1.8.0-jumbo-1 (December 2014) to 1.9.0-jumbo-1 (May 2019):

- Updated to 1.9.0 core, which brought the following relevant major changes:

- Optimizations for faster handling of large password hash files (such as

with tens or hundreds million hashes), including loading, cracking, and

"--show". These include avoidance of unnecessary parsing (some of which

creeped into the loader in prior jumbo versions), use of larger hash

tables, optional use of SSE prefetch instructions on groups of many hash

table lookups instead of doing the lookups one by one, and data layout

changes to improve locality of reference. [Solar; 2015-2017]

- Benchmark using all-different candidate passwords of length 7 by default

(except for a few formats where the length is different - e.g., WPA's is 8

as that's the shortest valid), which resembles actual cracking and hashcat

benchmarks closer. [Solar, magnum; 2019]

- Bitslice DES implementation supporting more SIMD instruction sets than

before (in addition to our prior support of MMX through AVX and XOP on

x86(-64), NEON on 32-bit ARM, and AltiVec on POWER):

- On x86(-64): AVX2, AVX-512 (including for second generation Xeon Phi),

and MIC (for first generation Xeon Phi).

- On Aarch64: Advanced SIMD (ASIMD).

[Solar, magnum; 2015-2019]

- Bitslice DES S-box expressions using AVX-512's "ternary logic" (actually,

3-input LUT) instructions (the _mm512_ternarylogic_epi32() intrinsic).

[DeepLearningJohnDoe, Roman Rusakov, Solar; 2015, 2019]

(In jumbo, we now also use those expressions in OpenCL on NVIDIA Maxwell

and above - in fact, that was their initial target, for which they were

implemented in both JtR jumbo and hashcat earlier than the reuse of these

expressions on AVX-512.)

See also:

- https://www.openwall.com/lists/announce/2019/04/12/1 1.9.0 core release

- Added FPGA support for 7 hash types for ZTEX 1.15y boards ("./configure

--enable-ztex", requires libusb). Specifically, we support: bcrypt,

descrypt (including its bigcrypt extension), sha512crypt & Drupal7,

sha256crypt, md5crypt (including its Apache apr1 and AIX smd5 variations) &

phpass. As far as we're aware, several of these are implemented on FPGA

for the very first time. For bcrypt, our ~119k c/s at cost 5 in ~27W greatly

outperforms latest high-end GPUs per board, per dollar, and per Watt. For

descrypt (where we have ~970M c/s in ~34W) and to a lesser extent for

sha512crypt & Drupal7 and for sha256crypt, our FPGA results are comparable to

current GPUs'. For md5crypt & phpass our FPGA results are much worse than

current GPUs'; we provide support for those hashes to allow for more (re)uses

of those boards. We also support multi-board clusters (tested by Royce

Williams for up to 16 boards, thus 64 FPGAs, all sharing a USB 2.0 port on a

Raspberry Pi 2 host). For all 7 hash types, we have on-device candidate

password generation for mask mode (and hybrid modes applying a mask on top of

host-provided candidates from another cracking mode) and on-device hash

comparison (of computed hashes against those loaded for cracking). We

provide pre-built bitstreams (5 of them, two of which support two hash types

each due to our use of multi-threaded soft CPU cores interfacing to

cryptographic cores) and full source project trees. [Hardware design and

host code by Denis Burykin, project coordination by Solar Designer, testing

also by Royce Williams, Aleksey Cherepanov, and teraflopgroup. 2016-2019.

See also:

- doc/README-ZTEX, src/ztex/fpga-*/README.md

- [List.ZTEX:Devices] and [ZTEX:*] john.conf sections

- https://www.openwall.com/lists/john-users/2019/03/26/3 bcrypt

- https://www.openwall.com/lists/john-users/2019/03/29/1 descrypt

- https://www.openwall.com/lists/john-users/2019/02/03/1 sha512crypt, Drupal7

- https://www.openwall.com/lists/john-users/2019/01/12/1 sha256crypt

- https://www.openwall.com/lists/john-users/2019/04/01/1 md5crypt, phpass

- https://www.techsolvency.com/passwords/ztex/ Royce Williams' cluster

- https://www.ztex.de/usb-fpga-1/usb-fpga-1.15y.e.html board specifications

These are old (introduced in 2011-2012), mostly ex-Bitcoin-miner boards with

four Spartan-6 LX150 FPGAs per board. ZTEX sold these boards for 999 EUR

(plus EU VAT if applicable) in 2012 with the price gradually decreasing to

349 EUR (plus VAT) in 2015, after which point the boards were discontinued.

Used boards were commonly resold on eBay, etc. (often in significant

quantities) in 2014 to 2016 for anywhere from $50 to 250 EUR, but are now

unfortunately hard to find. We support both German original and compatible

US clones of the boards.

- Dropped CUDA support because of lack of interest. We're focusing on OpenCL,

which is more portable and also runs great on NVIDIA cards (in fact, much

better than CUDA did for us before, due to our runtime auto-tuning and

greater focus on getting OpenCL right).

- We now have 88 OpenCL formats, up from 47 in 1.8.0-jumbo-1. (The formats may

be listed with "--list=formats --format=opencl".)

- Added 47 OpenCL formats: androidbackup-opencl, ansible-opencl,

axcrypt-opencl, axcrypt2-opencl, bitlocker-opencl, bitwarden-opencl,

cloudkeychain-opencl, dashlane-opencl, diskcryptor-aes-opencl,

diskcryptor-opencl, electrum-modern-opencl, enpass-opencl, ethereum-opencl,

ethereum-presale-opencl, fvde-opencl, geli-opencl, iwork-opencl,

keepass-opencl, keystore-opencl, krb5asrep-aes-opencl, lm-opencl,

lp-opencl, lpcli-opencl, mscash-opencl, notes-opencl, office-opencl,

openbsd-softraid-opencl, pbkdf2-hmac-md4-opencl, pbkdf2-hmac-md5-opencl,

pem-opencl, pfx-opencl, pgpdisk-opencl, pgpsda-opencl, pgpwde-opencl,

raw-sha512-free-opencl, salted-sha1-opencl, sappse-opencl, sl3-opencl,

solarwinds-opencl, ssh-opencl, sspr-opencl, telegram-opencl, tezos-opencl,

truecrypt-opencl, vmx-opencl, wpapsk-pmk-opencl, xsha512-free-opencl.

- Dropped 6 OpenCL formats (functionality merged into other OpenCL formats):

odf-aes-opencl, office2007-opencl, office2010-opencl, office2013-opencl,

ssha-opencl, sxc-opencl.

[Dhiru Kholia, magnum, Sayantan Datta, Elena Ago, terrybwest, Ivan Freed;

2015-2019]

- We now have 407 CPU formats, up from 381 in 1.8.0-jumbo-1 (including

pre-defined dynamic formats), or 262 non-dynamic CPU formats, up from 194 in

1.8.0-jumbo-1, despite having dropped many obsolete ones. (The formats may

be listed with "--list=formats --format=cpu".)

- Added 80 CPU formats (not including pre-defined dynamic formats): adxcrypt,

andotp, androidbackup, ansible, argon2, as400-des, as400-ssha1, axcrypt,

azuread, bestcrypt, bitlocker, bitshares, bitwarden, bks, dashlane,

diskcryptor, dominosec8, dpapimk, electrum, enpass, ethereum, fortigate256,

fvde, geli, has-160, itunes-backup, iwork, krb5-17, krb5-3, krb5asrep,

krb5tgs, leet, lp, lpcli, md5crypt-long, monero, money, multibit, net-ah,

notes, nsec3, o10glogon, o3logon, oracle12c, ospf, padlock, palshop,

pbkdf2-hmac-md4, pbkdf2-hmac-md5, pem, pgpdisk, pgpsda, pgpwde, phps2,

plaintext, qnx, racf-kdfaes, radius, raw-sha1-axcrypt, raw-sha3, saph,

sappse, scram, securezip, signal, sl3, snmp, solarwinds, sspr, stribog-256,

stribog-512, tacacs-plus, tc_ripemd160boot, telegram, tezos, vdi, vmx,

wpapsk-pmk, xmpp-scram, zipmonster.

- Dropped 12 CPU formats (not including pre-defined dynamic formats):

aix-smd5, efs, md4-gen, nsldap, nt2, raw-sha, raw-sha1-ng, raw-sha256-ng,

raw-sha512-ng, sha1-gen, ssh-ng, sxc. Their functionality is available in

other formats - e.g., AIX smd5 hashes are now supported by our main

md5crypt* formats.

[Dhiru Kholia, JimF, magnum, Fist0urs, Rob Schoemaker, MrTchuss,

Michael Kramer, Ralf Sager, bigendiansmalls, Agnieszka Bielec, Ivan Freed,

Elena Ago, Claudio Andre, Solar; 2015-2019]

- Several old formats got support for additional underlying hash, KDF, and/or

cipher types under their previous format names, making them more general -

e.g., the OpenBSD-SoftRAID format now supports bcrypt-pbkdf. [Dhiru, others]

- Several file archive formats got better support for file format variations,

large file support, and/or more complete verification (no longer producing

false positives, and thus no longer needing to continue running after a first

seemingly successful guess). [magnum, philsmd, JimF, others?]

- Added many new pre-defined dynamic format recipes. See run/dynamic.conf.

[Dhiru, JimF, Remi Dubois, Ivan Novikov; 2015-2018]

- Added dynamic compiler mode that can handle simple custom algorithms on CPU

(including with automatic use of SIMD) - e.g. "sha1(md5($p).$s)" - without

any programming - just state that very string on the command line as

"--format=dynamic='sha1(md5($p).$s)'". This is somewhat of a hack, but it

has clever self-testing so if it seems to work chances are it really does.

Available features include tens of fast hash functions (from common like MD5

to exotic ones like Whirlpool), string concatenation, encoding/decoding,

conversion to lowercase or uppercase, and references to the password, salt,

username, and string constants. See doc/DYNAMIC_EXPRESSIONS. [JimF; 2015]

- Many formats now make better use of shared code, often with optimizations

and/or SIMD support that was previously lacking. [magnum, JimF; 2015-2019]

- Shared code for reversing steps in MD4/MD5/SHA-1/SHA-2, boosting several fast

hash formats. [magnum; 2015]

- We added a terrific "pseudo-intrinsics" abstraction layer, which lets us use

the one same SIMD source code for many architectures and widths. [Zhang Lei,

magnum, JimF; GSoC 2015, 2015-2019]

- Where relevant, all SIMD formats now support AVX2, AVX-512 (taking advantage

of AVX-512BW if present), and MIC, as well as NEON, ASIMD, and AltiVec -

almost all of them using said pseudo-intrinsics (except for bitslice DES

code, which comes from JtR core and uses its own pseudo-intrinsics for now).

[magnum, Zhang Lei, JimF, Solar; GSoC 2015, 2015-2019]

- When AES-NI is available, we now use it more or less globally, sometimes with

quite significant boost. [magnum; 2015]

- Runtime CPUID tests for SSSE3, SSE4.1, SSE4.2, AVX2, AVX512F, and AVX512BW

(AVX and XOP were already present from 1.8 core), making it possible for

distros to build a full-featured fallback chain for "any" x86 CPU (including

along with fallback from OpenMP-enabled to non-OpenMP builds when only one

thread would be run). See doc/README-DISTROS. [magnum; 2015, 2017, 2018]

- Countless performance improvements (in terms of faster code, better early

rejection, and/or things moved from host to device-side), sometimes to single

formats, sometimes to all formats using a certain hash type, sometimes

globally. [magnum, Claudio, Solar, others; 2015-2019]

- Better tuning (by our team) of candidate password buffering for hundreds of

CPU formats, as well as optional auto-tuning (on user's system, with

"--tune=auto" and maybe also with "--verbosity=5" to see what it does) for

all CPU formats, both with and without OpenMP. [magnum; 2018-2019]

- Many OpenCL formats optimized and/or re-tuned to be friendly to newer NVIDIA

and AMD GPUs, and to newer driver and OpenCL backend versions. Some OpenCL

formats gained generally beneficial optimizations (for older hardware too),

and notably our md5crypt-opencl is now about twice faster on older AMD GPUs

as well. [Claudio, Solar, magnum; 2019]

- Many improvements to OpenCL auto-tuning (which is enabled by default), where

we try to arrive at an optimal combination of global and local work sizes,

including addition of a backwards pass to retry lower global work sizes in

case the device was not yet fully warmed up to its high-performance clock

rate when the auto-tuning started (important for NVIDIA GTX 10xx series and

above). [Claudio, magnum, Solar; 2015, 2019]

- When auto-tuning an OpenCL format for a real run (not "--test"), tune for the

actually loaded hashes (as opposed to test vectors) and in some cases for an

actual candidate password length (inferred from the requested cracking mode

and its settings). [magnum; 2017, 2019]

- Nearly all OpenCL formats now do all post-processing on GPU, so don't need

more than one CPU core. Post-processing on CPU is kept where it presumably

wouldn't run well on a GPU (e.g. RAR or ZIP decompression), but for them we

often have excellent early-reject - often even on device-side. [magnum,

Dhiru; 2018-2019]

- Graceful handling of GPU overheating - rather than terminate the process,

JtR will now optionally (and by default) sleep until the temperature is below

the limit, thereby adjusting the duty cycle to keep the temperature around

the limit. (Applies to NVIDIA and old AMD drivers. We do not yet have GPU

temperature monitoring for new AMD drivers.) [Claudio, Solar; 2019]

- We've switched from 0-based to 1-based OpenCL device numbers for consistency

with hashcat. (We also use 1-based numbers for ZTEX FPGA boards now.)

[Claudio, magnum, Solar; 2019]

- More efficient session interrupt/restore with many salts. Previously, we'd

retest the current set of buffered candidate passwords against all salts; now

we (re)test them only against previously untouched salts. This matters a lot

when the candidate password buffers are large (e.g., for a GPU), target hash

type is slow, and different salt count is large. [JimF; 2016-2017]

- PRINCE cracking mode ("--prince[=FILE]") added due to kind contribution by

atom of hashcat project. It's not a rewrite but atom's original code, with

additions for JtR session restore and some extras. PRINCE is a wordlist-like

mode, but it combines multiple words from a wordlist to form progressively

longer candidate passwords. See doc/PRINCE. [atom, magnum; 2015]

- Subsets cracking mode added ("--subsets[=CHARSET]"), which exploits the

weakness of having too few different characters in a password even if those

come from a much larger set of potential characters. A similar mode was

already present as an external mode (originally by Solar) but the new mode is

way faster, has full Unicode support (UTF-32 with no limitations whatsoever)

and unlike that external mode it also supports session restore.

See doc/SUBSETS. [magnum; 2019]

- Hybrid external mode added. This means external mode can produce lots of

candidates from a single base word. See "External Hybrid Scripting" in

doc/EXTERNAL and "Hybrid_example", "Leet", and "Case" external modes in the

default john.conf and the "HybridLeet" external mode in hybrid.conf.

[JimF, Christien Rioux; 2016]

- Stacking of cracking modes improved. Mask can now be stacked after any other

cracking mode, referring to that other mode's output "word" as "?w" in the

mask. See doc/MASK. The experimental "--regex" mode can be stacked before

mask mode and after any other cracking mode. [magnum, JimF; 2015-2016]

- Rules stacking. The new option "--rules-stack" can add rules to any cracking

mode, or after the normal "--rules" option (so you get rules x rules).

[magnum; 2018]

- Support for what used to be hashcat-specific rules. The ones that did not

clash with our existing commands just work out-of-the-box. Support for the

ones that clash can be turned on/off at will within a rule set (using lines

"!! hashcat logic ON" / "!! hashcat logic OFF"). See doc/RULES-hashcat.

[JimF, magnum; 2016, 2018]

- Added third-party hashcat rule sets to run/rules/ and referenced them from

separate sections as well as from [List.Rules:hashcat] in default john.conf,

so "--rules=hashcat" activates most of them. Our "--rules=all" also invokes

these rules, but only as the last step after completing our usual rule sets.

[magnum, individual rule set authors; 2018]

- Support for giving short rule commands directly on the command line,

including with preprocessor, e.g. "--rules=:[luc]$[0-9]" to request

"lowercase, uppercase, or capitalize, and append a digit" (30 rules after

preprocessor expansion). The leading colon requests this new feature, as

opposed to requesting a rule set name like this option normally does.

[JimF; 2016]

- Support for running several rule sets once after another, e.g.

"--rules=wordlist,shifttoggle". [JimF; 2016]

- Enhanced the "single crack" mode (which targets hashes with candidate

passwords derived from related information such as their corresponding

usernames) to be reasonable to use on massively-parallel devices such as

GPUs in some cases, which was never the case before (we advised for this mode

to always be used purely on CPU). This is achieved through buffering of much

larger numbers of candidate passwords per target salt (deriving them from

application of a larger number of mangling rules) and teaching the rest of

this mode's logic to cope up with such extensive buffering. As part of this

change, means were added for limiting this mode's memory usage (relevant when

hashes having a lot of different salts are loaded for cracking), most notably

the "SingleMaxBufferSize" setting in john.conf (4 GB by default).

See doc/MODES. [magnum; 2018]

- Added means for supplying global seed words for "single crack" mode, from

command line ("--single-seed=WORD[,...]") or file ("--single-wordlist=FILE").

[magnum; 2016]

- Wordlist mode: Better suppression of UTF-8 BOMs at a little performance cost.

[magnum; 2016]

- Unicode support is now at version 11.0.0, and we also added a few legacy

codepages. [magnum; 2018]

- UTF-32 support in external modes. This made an awesome boost to Dumb16/32

and Repeats16/32 modes. [magnum; 2018]

- Use our own invention "UTF-32-8" in subsets mode, for a significant boost in

final conversion to UTF-8. In the future we will likely make much more use

of this trick. [magnum; 2018]

- Full Unicode/codepage support even for OpenCL - most notably for formats like

NT and LM. [magnum; 2014-2019]

- Perfect hash tables for quick matching of computed against loaded hashes on

GPU, used by many of our fast hash OpenCL formats. So far, tested for up to

320 million SHA-1 hashes, which used up 10 GB of GPU memory and 63 GB of host

memory. For comparison, when using CPU only (and bitmaps along with simpler

non-perfect hash tables), the same hashes need 25 GB on host only, but the

attack runs slower (than on-device mask mode, see below). [Sayantan; 2015]

- On-device mask mode (and compare) implemented in nearly all OpenCL formats

that need device-side mask acceleration. Unlike most (maybe all) other

crackers, we can do full speed cracking (or e.g. hybrid wordlist + mask)

beyond ASCII, e.g. cracking Russian or Greek NT hashes just as easy as

"Latin-1" - and without any significant speed penalty. [Sayantan, Claudio,

magnum; 2015-2019]

- Many improvements to mask mode, including incrementing lengths with

stretching of masks (so you can say e.g. "-mask=?a -min-len=5 -max-len=7".

[Sayantan, magnum; 2015, 2018]

- Uppercase ?W in mask mode, which is similar to ?w (takes another cracking

mode's output "word" for construction of a hybrid mode) but toggles case of

all characters in that "word". [Sayantan; 2015]

- Extra (read-only) pot files that will be considered when loading hashes (such

as to exclude hashes previously cracked on other systems, etc.)

[magnum, JimF; 2015]

- Improved support for huge "hashes" (e.g. RAR archives) by introducing

shortened pot entries and an alternate read line function that can read

arbitrarily long lines. [magnum, JimF; 2016]

- A negative figure for "--max-run-time=N" will now abort after N seconds of

not cracking anything. [magnum; 2016]

- Improved logging with optional full date/time stamp ("LogDateFormat",

"LogDateFormatUTC", "LogDateStderrFormat" in john.conf). [JimF; 2016]

- JSON interface for frontends (like Johnny the GUI) to use in the future for

querying stuff. [magnum, Aleksey Cherepanov; 2017, 2019]

- Many updates to *2john programs for supporting more/newer input formats and

runtime environments (e.g., Python 3 compatibility).

[Dhiru, magnum, philsmd, Albert Veli, others; 2015-2018]

- wpapcap2john: Support for more link types, more/newer packet types,

more/newer algorithms e.g. 802.11n, anonce fuzzing, pcap-ng input format;

dropped hard-coded limits in favor of dynamic allocations.

[magnum; 2014-2018]

- More extensive self-tests with "--test-full" and optional builtin formats

fuzzer with "--fuzz" (when built with "./configure --enable-fuzz").

[Kai Zhao; GSoC 2015]

- "configure" options "--enable-ubsan" and "--enable-ubsantrap" for building

with UndefinedBehaviorSanitizer (we already had "--enable-asan" for building

with AddressSanitizer in 1.8.0-jumbo-1). [Frank, JimF; 2015, 2017]

- "configure" options "--disable-simd" and "--enable-simd=foo" to easily build

without SIMD support or for a particular SIMD instruction set (other than the

build host's best). [magnum, JimF; 2017]

- Default to not enable OpenMP for fast hash formats where OpenMP scalability

is too poor and we strongly recommend use of "--fork" instead. Accordingly,

"configure" option "--disable-openmp-for-fast-formats" is replaced with its

opposite "--enable-openmp-for-fast-formats". [magnum; 2015]

- Lots of improvements and tweaks to our usage of autoconf. [magnum]

- Many bug fixes, cleanups, unifications, and so on. Many fixes initiated from

source code, static, or runtime checking tools like ASan, UbSan, and fuzzers.

[magnum, Frank, Claudio, Solar, Christien Rioux, others; 2015-2019]

- Many fixes for big-endian architectures and/or for those that don't allow

unaligned access.

[magnum, JimF, Claudio, Frank, Solar, others]

- Many improvements to documentation, although we're still lagging behind.

[magnum, Frank, Solar, others]

- Far more extensive use of Continuous Integration (CI), where pull requests

can't be merged until passing numerous tests on different platforms. This is

mostly part of our development process and not the release, although some

CI-related files do exist in our released tree.

[Claudio, magnum; 2015-2019]