Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite Syntax for ST4 #130

Closed
wants to merge 21 commits into from
Closed

Rewrite Syntax for ST4 #130

wants to merge 21 commits into from

Conversation

deathaxe
Copy link

@deathaxe deathaxe commented May 1, 2023

This PR completely rewrites LESS.sublime-syntax as extension of ST's default CSS.sublime-syntax using latest syntax engine features.

  1. CSS.sublime-syntax is vendored for backward compatibility reasons for all ST builds not shipping [CSS] Make CSS a good base syntax for extensions sublimehq/Packages#3717

    Note

    Once a ST build with the PR merged is released, an appropriate release of this package should be added with vendored CSS removed, so it can benefit from future improvements of ST's CSS.

  2. Default CSS syntax tests are vendored as syntax_test_css.less to verify all existing CSS expressions are still scoped correctly.

    Only those invalid-syntax tests have been removed, which behave different caused by LESS.

  3. Symbol Lists and Symbol Index is refactored to reflect syntax changes and apply some functionality from base CSS.

  4. Folding rules for ST's syntax based folding are added

  5. Syntax specific settings are added to adjust word separators so - is included into identifiers.

Benchmark:

description before after diff
real world CSS file with 10.000 lines of code 478ms 30ms -94%
syntax_test_less.less from previous revision 36ms 8ms -78%

Requirements:

This PR is currently compatible with ST 4126+.

If it is intented to be released with ST4107 Regexp.sublime-syntax needs to be vendored, too.

deathaxe added 2 commits May 1, 2023 13:49
This commit completely rewrites LESS.sublime-syntax as extension of
ST's default CSS.sublime-syntax using latest syntax engine features.

1. CSS.sublime-syntax is vendored for backward compatibility reasons
   for all ST builds not shipping sublimehq/Packages#3717

   Note: Once a ST build with the PR merged is released, an appropriate
   release of this package should be added with vendored CSS removed,
   so it can benefit from future improvements of ST's CSS.

2. Default CSS syntax tests are vendored as syntax_test_css.less to
   verify all existing CSS expressions are still scoped correctly.

   Those invalid-syntax tests have been removed, which behave different
   caused by LESS.

3. Symbol Lists and Symbol Index is refactored to reflect syntax changes
   and apply some functionality from base CSS.

4. Folding rules for ST's syntax based folding are added

5. Syntax specific settings are added to adjust word separators so `-`
   is included into identifiers.

Benchmark:

- real world CSS file with 10.000 lines of code
  before: 478ms   after: 30ms  diff: -94%

- syntax_test_less.less from previous revision
  before: 36ms    after: 8ms   diff: -78%
@braver
Copy link
Collaborator

braver commented May 5, 2023

This PR is currently compatible with ST 4126+.

Alright, when we're going to ship, we can pre-fix tag stuff.

@braver
Copy link
Collaborator

braver commented May 5, 2023

How do you feel about mixins? This is one area where I find LESS annoying (as a language): any class can be used as a mixin by "calling it". So this is effectively no longer a selector, but a function call.

Screenshot 2023-05-05 at 18 05 56

See: https://lesscss.org/features/#mixins-feature

@deathaxe
Copy link
Author

deathaxe commented May 5, 2023

I am aware of that, but found it of little value to give it a special scope. We'd need to extend the property-or-selector context by another set of 1 or 2 branches to scope (maybe parametrised) mixin definitions and mixin calls. It would be possible - is it worth it?

A mixin definition can also look very much like a normal class #lib.mixin { ... } So how to distinguish normal classes from mixin definitions?

Without definitions not sure if mixin calls are worth the effort then. The primary use case would be to drive goto definition, symbol list, goto reference.

I'd be otherwise fine to highlight them very much the same like selectors in a first step.

The same is with mixins in values. For simplicity reasons I just scoped them meta.selector.less at the moment. The former use of meta.alias is wrong, imho.

Tests/syntax_test_css.less Outdated Show resolved Hide resolved
@braver
Copy link
Collaborator

braver commented May 5, 2023

A mixin definition can also look very much like a normal class #lib.mixin { ... } So how to distinguish normal classes from mixin definitions?

That's is by definition impossible: any normal classes can be used as a mixin. It's only on the usage of that mixin that you can differentiate. And when a mixin is used it always has the () to "call" it (without the parentheses is deprecated).

Without definitions not sure if mixin calls are worth the effort then

Without a definition you can't do "find usages", so in that sense it adds no value. Is it worth it to perhaps get a different color so you can find your mixins between all the properties and selectors? Well I guess the entire idea behind the syntax in the first place is that it looks the same. So scoping them the same is totally acceptable. For me personally at least :)

@braver
Copy link
Collaborator

braver commented May 5, 2023

Ok, I'm just trying really hard to find holes, that I come up with such niche things is a testament to the quality of your work here. One final thing before I call it a week (I'll have a deeper look at some later time):

Screenshot 2023-05-05 at 18 21 44

In this situation it can't figure out that I'm writing a property, so I don't get the right suggestions. If I add another blank line before the media query, or when I add the property value and a ;, it clears up. Not sure if that's a clear explanation of what happens, but you can see in the screenshot that it's currently expecting a selector instead of a property.

@deathaxe
Copy link
Author

deathaxe commented May 5, 2023

color: could be a property or a custom element tag with : maybe starting a pseudo class. The current approach to distinguish selectors from properties without maintaining huge and never complete lists of predefined tokens is just to try to scope the thing as property: value;. This branch succeeds if a ; ) or } is found directly after the key-value pair.

- match: (?=[;)}])
pop: 1
# otherwise it is part of a selector
- match: ''
fail: property-or-selector

Otherwise an incomplete selector is assumed.

I'd need to look into whether and how it is possible to improve that edge case.

@braver
Copy link
Collaborator

braver commented May 5, 2023

Well color isn’t a valid element and a custom element needs two words and a dash IIRC, but even then to make something work you’d need to maintain a list… and we already agree that that’s not a good idea.
For me it’s totally acceptable that in some cases the scopes can only be correctly set once the statement is completed. I’ve always found this (selector with pseudo class, or property?) the trickiest bit and never gotten good enough at writing syntaxes to handle it properly. It gets really tricky when a closing semi-colon after a value is not required. That would be an area I would do additional testing.

Other than that, testing with some larger code sets and hunting for edge cases, I defer to you for the scope naming… and it looks really solid already.

@deathaxe
Copy link
Author

deathaxe commented May 6, 2023

HTML also supports "foreign tags", which are any form of valid identifier. A good example are SVG tags. But anything else is fine, too. I guess CSS therefore should be able to highlight them as well. Thus the "custom element needs two words and a dash" rule is unfortunately not sufficient.

I think catching valid CSS at-rules may be something which can easily be added to bailout from property values early, as known at-rules can't be variables and there are not too many.

I aggree that this pseudo-class vs. property issue is very tricky, espececially if no hard coded lists of tokens are to be maintained.

In general CSS's completions try hard to add the semicolon after : automatically so it should normally be of no problem to get value completions.

Scope names are mainly just taken from core CSS. I am not too happy with all of them, but I hesitate to change them because it might break many legacy color schemes.

I am looking into adding some more tests about it.

deathaxe added 5 commits May 6, 2023 20:31
Maintain compatibility with ST4107-4125.

We could also use builtin regexp, but this would mean to adjust syntax
tests depending on ST builds.
@deathaxe
Copy link
Author

deathaxe commented May 6, 2023

Support for mixins was added and property vs. selector detection slightly improved. Further improvements are probably possible, but that may take some time for investigation and may be done in a future PR as well.

This PR als vendors ST4149's RegExp syntax to maintain compatibility with ST4107-4126 without adjusting syntax tests etc.

I've got an st4149 branch as well which drops both CSS and RegExp.

deathaxe added 9 commits May 7, 2023 11:06
This commit fixes a wrong assumption of `;` denoting mixin definitions.
Exclude them from normal goto definition / goto reference functionality.
This commit...
1. introduces a `less-values` context to avoid messing with
   `less-group-body`.
2. Fixes detection of mixin calls of detached rulesets.
This commit ...

1. scopes mixin parameters of parametrized mixins `variable.parameter`
2. scopes parameter lists `meta.selector.parameters meta.group`
3. adjusts selector's symbol list definition

This way scoping/highlighting is adjusted to how parameter/arguments
lists of function definitions vs. function calls are scoped.
1. more specific selectors with regards to exclude `punctuation`
2. add mixins to reference index to enable Goto Reference feature
3. add parametrized mixin parameters to symbol index to enable
   Goto Definition feature
This commit uses same contexts for both property-level and value-level mixins.
Both property-level mixins and values are terminated by the same tokens.
By also including comments, another edge case in detached rulesets is fixed
where mixins were scoped `meta.selector.css` before.
Seems better placed next to less-expressions
Syntaxes/LESS.sublime-syntax Outdated Show resolved Hide resolved
Syntaxes/LESS.sublime-syntax Outdated Show resolved Hide resolved
Syntaxes/LESS.sublime-syntax Outdated Show resolved Hide resolved
Syntax after incomplete mixin brackets may break until end of document without
a 2nd-level bailout.
@braver
Copy link
Collaborator

braver commented May 14, 2023

Alright, looking good… let’s do it like this:

  • You move your fork to the Sublime Text org
  • Set up the correct prefixed tags for backwards compatibility
  • We can keep this PR open so I have another chance to look at it and maybe give some feedback next week
  • Set up a PR on package control to replace the Less entry with the fork

@deathaxe
Copy link
Author

All prepared. Repo moved, you added as maintainer, tags added retarget PR filed.

@deathaxe deathaxe closed this May 18, 2023
@michaelblyons
Copy link

@deathaxe deathaxe deleted the st4 branch June 27, 2023 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants