Proposal for escaping within lexed content #1152
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Good point, we do depend on #1151. I will rebase once that is merged. |
jneen
changed the title
pyrmont
reviewed
Jun 3, 2019
dblessing
approved these changes
Jun 3, 2019
There was a problem hiding this comment.
Looks and sounds fine from my perspective. I like that it is disabled by default. I'm interested in hearing from @stanhu on whether or how he thinks we might be able to utilize this in GitLab. If not, that's fine. It's still interesting for others.
|
I have rebased, and now I believe it's ready to merge pending yall's +1. |
|
I also added escape functionality to the CLI, which is where I confess I actually need it at this moment, for use in the TeX formatter I'm working on. |
jneen
changed the title
This was referenced Jun 19, 2019
ole
added a commit
to ole/middleman-syntax
that referenced
this pull request
Nov 10, 2022
Option in config.rb:
activate :syntax, :escape => true
This enables Rouge's Escape lexer (introduced in rouge-ruby/rouge#1152), which is useful for formatting/highlighting individual pieces of a code block. This lexer passes the contents of <! … !> sections unchanged as a raw value into the resulting HTML, effectively allowing you to add inline HTML formatting into a code block at arbitrary places.
WARNING: As described in rouge-ruby/rouge#1152, this can be a major security risk if you don't control the input to the lexer. Don't enable this option for formatting e.g. user-generated content.
Example Markdown (Kramdown):
~~~swift
func add(_ x: Int, y: Int) -> Int {
return <!<span style="text-decoration: underline;"> !>x + y<!</span>!>
}
~~~
ole
added a commit
to ole/middleman-syntax
that referenced
this pull request
Nov 10, 2022
Option in config.rb:
activate :syntax, :escape => true
This enables Rouge's Escape lexer (introduced in rouge-ruby/rouge#1152), which is useful for formatting/highlighting individual pieces of a code block. This lexer passes the contents of <! … !> sections unchanged as a raw value into the resulting HTML, effectively allowing you to add inline HTML formatting into a code block at arbitrary places.
WARNING: As described in rouge-ruby/rouge#1152, this can be a major security risk if you don't control the input to the lexer. Don't enable this option for formatting e.g. user-generated content.
Example Markdown (Kramdown):
~~~swift
func add(_ x: Int, y: Int) -> Int {
return <!<span style="text-decoration: underline;"> !>x + y<!</span>!>
}
~~~
joshuap
added a commit
to honeybadger-io/middleman-syntax
that referenced
this pull request
Feb 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We often receive requests to do things such as underline sections of code, or embed arbitrary html inside of highlighted content. This PR is a proposal to address such features, using a similar tactic as Pygments.
We provide a special
Escapelexer, which is capable of taking user-defined escape delimiters (default<!and!>) and rendering their contents into theEscapetoken (also from Pygments). In the HTML formatter (and potentially other formatters), we can allow theEscapetoken to pass as a raw value.IMPORTANT
There is a major security concern with this feature: in some applications (such as Gitlab), an untrusted user will control all of the input text, the lexer choice, and the lexer options. We cannot allow such users to pass unescaped HTML into the output.
For this reason, escaping is disabled by default - the base
Formatterclass will filter allEscapetokens and replace them withErrortokens before passing the token stream to thestreammethod. In order to enable this feature, a user must explicitly callRouge::Formatter.enable_escape!. It can also be enabled thread-locally withRouge::Formatter.with_escape { ... }and explicitly disallowed withRouge::Formatter.disable_escape!. I think this should have the correct secure defaults and extension opportunities for most applications, but I would appreciate additional security review.cc @stanhu