Partial rewrite of PHP lexer #368
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
julp
force-pushed
the
php_lexer_rewritten
branch
from
a1b21d5 to
8ac7fc7
Compare
julp
force-pushed
the
php_lexer_rewritten
branch
from
8ac7fc7 to
1ac94d2
Compare
lib/rouge/lexers/php.rb
Outdated
|
|
||
| DEFAULTS = Hash.new(Error).tap do |h| | ||
| # :in_scripting => Error, | ||
| # :var_offset => Error, |
There was a problem hiding this comment.
Actually, could you explain to me what the DEFAULTS hash is for?
There was a problem hiding this comment.
Some states are "inherited" in few others, so to set the current token's type, I use this hash to map it to the correct type based on the current (top level) state's name. eg to highlight a , in "...,..." as Str::Double but in an expression like "$a[,]" as an error.
Do you have a better approach in mind?
|
Hi @julp! Thanks for your work. I'm going to leave this for later, because it looks like this needs a good deal of review, which I'll get to when things have settled down a bit more. |
jneen
added
lexer-request
* translation of PHP's lexer into rouge
* add lexer option short_open_tag to permit highlighting as if
short_open_tag were off when disabled (enabled by default)
* add some missing features:
+ binary numbers (introduced by PHP 5.4.0)
+ string interpolation in backquoted strings
+ Unicode codepoints escape syntax (\u{...}) (introduced by PHP 7.0.0)
* stricter syntax:
+ <?php is case insensitive
+ a "blank" have to follow <?php (but not <?= nor <?)
+ for doc comments, /** is followed by at least one whitespace
+ keywords are case insensitive
+ function/method names are also case insensitive
* add some missing keywords:
+ type declarations (PHP 7, including void planned for 7.1.0 - master)
+ 7.0.0: class (anonymous classes), yield from
+ 5.5.0: finally
+ 5.4.0: callable, insteadof, trait, __TRAIT__
+ 5.3.0: goto, __NAMESPACE__, __DIR__
+ function, const added to keywords (can't manage them with a rule anymore, it
conflicts with use - eg: use constant \A\B\C; as introduced by PHP 5.6.0)
+ others: casts, instanceof, __CLASS__, __FUNCTION__, __METHOD__, __halt_compiler
+ self even if it's not really a reserved word
* cleanup in keywords, removal of:
+ predefined constants (eg: E_ERROR or PHP_OS) - which, besides, are case sensitive
+ predefined classes (as stdclass)
julp
force-pushed
the
php_lexer_rewritten
branch
from
f6747e0 to
b70a812
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
short_open_tag were off when disabled (enabled by default)
0b...) (introduced by PHP 5.4.0)\u{...}) (introduced by PHP 7.0.0)<?phpis case insensitive<?php(but not<?=nor<?)/**is followed by at least one whitespacevoidand nullable types from PHP 7.1)class(anonymous classes),yield fromfinallycallable,insteadof,trait,__TRAIT__goto,__NAMESPACE__,__DIR__function,constadded to keywords (can't manage them with a rule anymore, itconflicts with
use- eg:use constant \A\B\C;as introduced by PHP 5.6.0)instanceof,__CLASS__,__FUNCTION__,__METHOD__,__halt_compilerselfeven if it's not really a reserved wordE_ERRORorPHP_OS) - which, besides, are case sensitivestdClass)Also fixes: #338 and #348
To give it a try, (temporarily) replace
gem 'rouge'in your Gemfile by: