Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a lexer for emails #1567

Merged
merged 7 commits into from
Oct 13, 2020
Merged

Add a lexer for emails #1567

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
bd45692
Add a lexer for emails
smokris Jul 22, 2020
fbf5ab9
Use separate states for header and body text. Add support for RFC 28…
smokris Jul 22, 2020
ce9d980
Remove unnecessary `/m`s.
smokris Jul 22, 2020
9a5486d
Add colon into subject line of visual sample
pyrmont Jul 23, 2020
b88a9eb
Match RFC specifications more closely
pyrmont Jul 23, 2020
98c03dc
Add support for quoted lines
pyrmont Sep 8, 2020
bd87220
Add newline matching
pyrmont Sep 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use separate states for header and body text. Add support for RFC 282… 
…2 section 2.2.3 "Long Header Fields".
  • Loading branch information
@smokris
smokris committed Jul 22, 2020
commit fbf5ab9e099b6a9603233b39c1d030d0831eb591
34 changes: 28 additions & 6 deletions lib/rouge/lexers/email.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,26 +16,48 @@ def self.detect?(text)
return true if text.start_with?('From: ')
smokris marked this conversation as resolved.
Show resolved Hide resolved
end

state :root do
start do
push :header
end

state :header do
rule %r/^(From|To|Cc|Bcc):\s/, Keyword, :address
rule %r/^Date:\s/, Keyword, :date
rule %r/^Subject:\s/, Keyword, :subject
rule %r/^[A-Z][A-Za-z0-9-]+:\s/, Keyword, :other_header
rule %r/\n/m, Text::Whitespace, :pop!
smokris marked this conversation as resolved.
Show resolved Hide resolved
end

state :root do
rule %r/\n/m, Text::Whitespace
rule %r/^>.*/, Comment
rule %r/^>.*$/, Comment
rule %r/^--\s\n/m, Comment::Doc, :signature
rule %r/.*/, Text
rule %r/.*$/, Text
end

state :header_line_and_continuation do
rule %r/\n(?=[^ \t])/m, Text::Whitespace, :pop!
rule %r/\n/m, Text::Whitespace
end

state :address do
rule %r/[^\n]+\n/m, Name, :pop!
mixin :header_line_and_continuation
rule %r/.*$/, Name
end

state :date do
rule %r/[^\n]+\n/m, Literal::Date, :pop!
mixin :header_line_and_continuation
rule %r/.*$/, Literal::Date
end

state :subject do
rule %r/[^\n]+\n/m, Name::Label, :pop!
mixin :header_line_and_continuation
rule %r/.*$/, Name::Label
end

state :other_header do
mixin :header_line_and_continuation
rule %r/.*$/, Literal::String
end

state :signature do
Expand Down
12 changes: 12 additions & 0 deletions spec/visual/samples/email
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,23 @@
From: Me <me@example.com>
To: You <you@example.com>
Cc: Somebody <somebody@example.com>,
And One More <andonemore@example.com>
Bcc: Secret Person <secretperson@email.com>
X-Spam-Status: Definitely not spam
Date: Tue, 21 Jul 2020 15:14:03 +0000
Subject: A very important message
that continues onto the next line.

Greetings and salutations.

>> A second-level quotation.
>
> Please investigate. Thank you.

I have investigated.

Note: This header-like like shouldn't be highlighted
since it's part of the body text.

--
This message is highly confidential and will self-destruct.