Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix highlighting of YAML block strings #1235

Merged
merged 2 commits into from
Jul 3, 2019

Conversation

ashmaroli
Copy link
Contributor

Currently, YAML block strings are allotted as Name::Constant tokens.
(Note that the [String] "Mark McGwire" and block strings are colored differently).

YAML Visual Test - before

Ideally, there shouldn't be a differentiation.
Result after fix:

YAML Visual Test - after

@pyrmont
Copy link
Contributor

pyrmont commented Jun 30, 2019

Definitely looks like the right token for multiline strings!

One thing I'm catching on is that it seems like a really strange mistake for @jneen to have made. I checked the git blame and the rule was apparently added with the original commit:

rule /[^\n\r\f\v]+/, 'Name.Constant'

It's possible that it was an oversight and yet—I don't know—it's difficult to imagine an oversight like that happening completely unintentionally.

Based on the location of the rule, I wonder if it was meant to be a catch-all for some other type of value and is sort of unintentionally capturing multiline strings. @ashmaroli Can you think of any values for which Name::Constant would be the correct token here and that would have the wrong token if it's changed to Str?

@pyrmont pyrmont added the author-action The PR has been reviewed but action by the author is needed label Jun 30, 2019
@ashmaroli
Copy link
Contributor Author

Can you think of any values for which Name::Constant would be the correct token here

I think object tags and booleans would be best suited for this name. However, I think there are more bugs in this lexer. For example, unclosed single quote has no effect and num literals are highlighted as strings..etc.:

bool_true: true
true: "true"
bool_false: false
false: 'false'
number: 123456
str_number: '123456'
email_regex: !ruby/regexp '/^([\w.%+-]+)@([\w-]+\.)+([\w]{2,})$/i'

@pyrmont
Copy link
Contributor

pyrmont commented Jul 1, 2019

@ashmaroli I guess what I was getting at with my question was more whether the change in this PR would affect the existing behaviour of the lexer. It sounds like:

  1. it does by causing certain values that were being tokenised as Constants to be tokenised as Strings; but
  2. these instances were all errors because they weren't actually strings (such as the number literals).

Does this mean that in the same way that you fixed the number literals in #1239, we should have separate PRs to address the other bugs?

@ashmaroli
Copy link
Contributor Author

This PR does affect existing behavior. Like you've mentioned, tokens incorrectly rendered as Constants will now be Strings. To my knowledge, YAML doesn't have constants per se. Just tags.

From the visual test, all tags, booleans and null are highlighted properly. Therefore IMO, we should just wait for our users to report anything amiss once this gets shipped.

@pyrmont
Copy link
Contributor

pyrmont commented Jul 3, 2019

@ashmaroli Yeah, doesn't look like Constant is one of YAML's types so I'm guessing it was just an unintentional mistake. I'll merge this in.

@pyrmont pyrmont merged commit 75676e7 into rouge-ruby:master Jul 3, 2019
@ashmaroli ashmaroli deleted the fix-yaml-block-string branch July 3, 2019 06:39
@pyrmont pyrmont removed author-action The PR has been reviewed but action by the author is needed labels Jul 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants