Thanks for the review @pyrmont. It'd be a good test to check the lexer against various lenses in https://github.com/hercules-team/augeas/tree/master/lenses (both curdir and tests/). Is there an easy way to do do that?

@raphink Unfortunately, not really. Rouge comes with a command-line tool bin/rougify and you could use it to lex files using ./bin/rougify highlight <path_to_file>. (You can change the theme by adding -t <theme_name. as a switch.)

Does that help at all?

Yes it does help, thanks!

I've run it against a few files. These examples generate errors:

test test_user get "root
@my\ admin\ group
EXAMPLE\\\\cslack
MY\ EX-AMPLE\ 9\\\\cslack\ group
" =
  { "spec"
    { "user" = "@my\ admin\ group" } }

test Sudoers.parameter_integer_bool
  put "umask = 022"
  after rm "/umask/negate"; set "/umask" "022" = "!!!!umask = 022"

test Xml.doctype put dt_with_entities after
    rm "/\!DOCTYPE/\!ENTITY[2]";
    set "/\!DOCTYPE/\!ENTITY[. = \"sys-ent\"]/SYSTEM/#systemliteral"
        "other-file.xml"

test Xml.lns get "<a password=\"my\!pass\" />" =
  { "a" = "#empty"
    { "#attribute" { "password" = "my\!pass" } } }

So essentially:

• semicolons
• escaped characters (\ , \\, \!) in strings or backslashes without escapes, such as "(DOMAIN\linux_users)"

There, I've run rougify on all lenses and their tests in the Augeas project, without seeing errors. If my changes are fine by you, I'm ok with the current output of the lexer.

@raphink Thanks for going through and trying out the lexer with those other files. Looks like you caught some tricky bugs! I've merged it into master and this will be pushed out to RubyGems as v3.20.0 on Tuesday 9 June. Thanks again! 🎉

Great, thanks for your help @pyrmont