doc/MASK

Mask mode is a fast way to produce password candidates given a "mask" that
describes what the words should look like.

A mask may consist of:

- Static letters.
- Ranges in [aouei] or [a-z] syntax. Or both, [0-9abcdef] is the same as
     [0-9a-f].
- Placeholders that are just a short form for ranges, like ?l which is
     100% equivalent to [a-z].
- ?l lower-case ASCII letters
- ?u upper-case ASCII letters
- ?d digits
- ?s specials (all printable ASCII characters not in ?l, ?u or ?d)
- ?a full 'printable' ASCII. Note that for formats that don't recognize case
     (eg. LM), this only includes lower-case characters which is a tremendous
     reduction of keyspace for the win.
- ?B all 8-bit (0x80-0xff)
- ?b all (0x01-0xff) (the NULL character is currently not supported by core).
- ?h lower-case HEX digits (0-9, a-f)
- ?H upper-case HEX digits (0-9, A-F)
- ?L lower-case non-ASCII letters
- ?U upper-case non-ASCII letters
- ?D non-ASCII "digits"
- ?S non-ASCII "specials"
- ?A all valid characters in the current code page (including ASCII). Note
     that for formats that don't recognize case (eg. LM), this only includes
     lower-case characters which is a tremendous reduction of keyspace.
- Placeholders that are custom defined, so we can e.g. define ?1 to mean [?u?l]
  ?1 .. ?9 user-defined place-holder 1 .. 9
- Placeholders for Hybrid Mask mode:
  ?w is a placeholder for the original word produced by the parent mode in
     Hybrid Mask mode.
  ?W is just like ?w except the original word is case toggled (so PassWord
     becomes pASSwORD).

Mask Mode alone produces words from the mask, for example ?u?l?l will generate
all possible three-letter words, with first character uppercased and the
remaining in lowercase.

Hybrid (a.k.a Stacked) Mask means we use e.g. a wordlist with or without rules
(or any other cracking mode), and then apply the mask to each word.  So with a
mask of ?w?d?d and an input word (from the parent cracking mode) of "pass",
it will produce "pass00", "pass01" and so on until "pass99".  Hybrid Mask can
be stacked upon any other mode except single.  Hybrid Mask can even be applied
after hybrid regex, eg "prince -> regex -> mask".

For most fast GPU formats, mask mode (including hybrid) is several orders of
magnitude faster than any other cracking mode, as the mask (or part of it) is
applied on GPU side.  Hybrid mask can thus be used as a GPU accelerator for
any mode except single.  To benchmark the speed of such format using a mask,
use "--test --mask" (using some default mask) or optionally with a specific
mask, eg. "--test --mask=?a?a".  You can list all formats featuring internal
mask using "--list=formats -format=mask".

External filters can be applied too, and will be applied last of all.  The
"longest" chain is thus "wordlist -> rules -> regex -> mask -> filter".  Using
external filters with "GPU side mask" will cause a somewhat undefined behavior
though: The filter will be applied before the GPU-side mask has finished the
word!

You can define custom placeholders for ?1 .. ?9 using command line e.g. -1=?l?u
or in john.conf section [Mask].

There is a default mask in john.conf too (defaulting to same as hashcat).
This should be used with -max-len (and possibly -min-len) to do any good.

The -max-len=N option will truncate the mask so no words longer than N are
produced.

The -min-len=N option will skip generation of words shorter than N.

If not in "hybrid mask" mode, and either -min-len or -max-len option was used,
we will iterate lengths (as in "incremental mask") from -min-len to -max-len
(or format's min or max length, if one was not given).  So to produce all
possible words from 3 to 5 letters, use -mask=?l -min-len=3 -max-len=5.  In
case the specified mask is shorter, the last part of it will be expanded, for
example "-mask=?u?l -max-len=5" will use an effective mask of ?u?l?l?l?l.
Whenever using incremental mask, the ETA at any given time shows estimated
time to complete the *current* length, as opposed to the whole run.

You can escape special characters with \.  So to produce a literal "?l" you
could say \?l or ?\l and it will not be parsed as a placeholder.  Similarly,
you can escape dashes or brackets to prevent them from being parsed as
specials.  To produce a literal backslash, use \\.

There is also a special hex notation, \xHH for specifying any character code.
For example, \x41 is "A" and \x09 is the code for TAB.


Examples:

Mask            custom mask / hybrid input      example output  num candidates
pass                                            pass            1
pw?d                                            pw3             10
?w?d?d?d                password                password123     1000x
?w?s?w                  Bozo                    Bozo#Bozo       33x
?w?s?W                  Bozo                    Bozo#bOZO       33x
0x?1?1:?1?1:?1?1        -1=[0-9a-f]             0xde:ad:ca      16777216
?3?l?l?l                -3=?l?u                 Bozo, hobo      913952
[Pp][Aa@][Ss5][Ss5][Ww][Oo0][Rr][Dd]            P@55w0rD        1296

We have on-device mask mode support for most fast hash types for which
we have OpenCL support at all.  Further, such on-device mask support can
be used along with a host-provided stream of partial candidate passwords
to form a variety of hybrid modes.  For example, these are all valid:

Test any 7-character printable ASCII strings, with a reasonable number
of the mask positions being processed on device (JtR decides to
optimally split the mask between host and device):

--mask='?a?a?a?a?a?a?a'

Ditto:

--mask='?a' --min-length=7 --max-length=7

Ditto, but for range of lengths 1 to 8:

--mask='?a' --min-length=1 --max-length=8

Can also use length ranges with more complex masks, where the last mask
component would be the one extended to higher lengths:

--mask='start?l?d' --min-length=7 --max-length=14

Use the host's incremental mode to test strings of digits in a smart
order, then append 3 more digits in a dumber order on device, no
specific length (so let incremental mode switch lengths back and forth
like it usually does):

--incremental=digits --mask='?w?d?d?d'

Ditto, but limit this to (total) length of 8 (adjusts incremental mode
to only produces length 5 for its portion):

--incremental=digits --mask='?w?d?d?d' --min-length=8 --max-length=8

Have the host generate all-lowercase substrings in a smart order, then
prepend an uppercase letter and append 3 digits, at least some of this
on device:

--incremental=lower --mask='?u?w?d?d?d'

Read a wordlist on host, apply wordlist rules on host, then append 3
digits on device:

-w=word.lst --rules=jumbo --mask='?w?d?d?d'

Duplicate words, then append 3 digits on device:

-w=word.lst --mask='?w?w?d?d?d'

Another way to write that:

-w=word.lst --rules=':d' --mask='?w?d?d?d'

Test strings consisting of uppercase words, 4 digits (processed on
device), then lowercase versions of the same words:

-w=word.lst --mask='?W?d?d?d?d?w'

And so on.  Many of these option names may be abbreviated.