Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a lexer for emails #1567

Merged
merged 7 commits into from
Oct 13, 2020
Merged

Add a lexer for emails #1567

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
bd45692
Add a lexer for emails
smokris Jul 22, 2020
fbf5ab9
Use separate states for header and body text. Add support for RFC 28…
smokris Jul 22, 2020
ce9d980
Remove unnecessary `/m`s.
smokris Jul 22, 2020
9a5486d
Add colon into subject line of visual sample
pyrmont Jul 23, 2020
b88a9eb
Match RFC specifications more closely
pyrmont Jul 23, 2020
98c03dc
Add support for quoted lines
pyrmont Sep 8, 2020
bd87220
Add newline matching
pyrmont Sep 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Match RFC specifications more closely
  • Loading branch information
@pyrmont
pyrmont committed Jul 23, 2020
commit b88a9eb8dc908537755eeb806e5381b765427895
59 changes: 14 additions & 45 deletions lib/rouge/lexers/email.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,63 +5,32 @@ module Rouge
module Lexers
class Email < RegexLexer
tag 'email'
aliases 'eml', 'mail'
filenames '*.eml', '*.mail'
aliases 'eml', 'e-mail'
filenames '*.eml'
smokris marked this conversation as resolved.
Show resolved Hide resolved
mimetypes 'message/rfc822'

title "Email"
desc "An email message"

smokris marked this conversation as resolved.
Show resolved Hide resolved
def self.detect?(text)
return true if text.start_with?('From: ')
end

start do
push :header
end

state :header do
rule %r/^(From|To|Cc|Bcc):\s/, Keyword, :address
rule %r/^Date:\s/, Keyword, :date
rule %r/^Subject:\s/, Keyword, :subject
rule %r/^[A-Z][A-Za-z0-9-]+:\s/, Keyword, :other_header
rule %r/\n/, Text::Whitespace, :pop!
end

state :root do
rule %r/\n/, Text::Whitespace
rule %r/^>.*$/, Comment
rule %r/^--\s\n/, Comment::Doc, :signature
rule %r/.*$/, Text
push :fields
end

state :header_line_and_continuation do
rule %r/\n(?=[^ \t])/, Text::Whitespace, :pop!
rule %r/\n/, Text::Whitespace
state :fields do
rule %r/[:]/, Operator, :field_body
rule %r/[^\n\r:]+/, Name::Tag
rule %r/[\n\r]/, Name::Tag
end

state :address do
mixin :header_line_and_continuation
rule %r/.*$/, Name
state :field_body do
rule(/(\r?\n){2}/) { token Text; pop!(2) }
smokris marked this conversation as resolved.
Show resolved Hide resolved
rule %r/\r?\n(?![ \v\t\f])/, Text, :pop!
smokris marked this conversation as resolved.
Show resolved Hide resolved
rule %r/[^\n\r]+/, Name::Attribute
rule %r/[\n\r]/, Name::Attribute
end

state :date do
mixin :header_line_and_continuation
rule %r/.*$/, Literal::Date
end

state :subject do
mixin :header_line_and_continuation
rule %r/.*$/, Name::Label
end

state :other_header do
mixin :header_line_and_continuation
rule %r/.*$/, Literal::String
end

state :signature do
rule %r/.*/m, Comment::Doc
state :root do
rule %r/.*/m, Text
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
end
end
end
Expand Down
5 changes: 0 additions & 5 deletions spec/lexers/email_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,10 @@

it 'guesses by filename' do
assert_guess :filename => 'foo.eml'
assert_guess :filename => 'foo.mail'
end

it 'guesses by mimetype' do
assert_guess :mimetype => 'message/rfc822'
end

it 'guesses by source' do
assert_guess :source => 'From: Me <me@example.com>'
end
end
end