I like the idea of being able to reinitialize a Lexer with a new set of options. However, there might be something even simpler for the use case at hand, which I'll describe.

The difficulty with Lexer.find_fancy is that you don't know what the Lexer will be when preparing the options to pass in. This could be solved by allowing Lexer.find_fancy to accept a block which provides access to the resolved lexer class and compiled options. The block would return the lexer instance, replacing the default behavior.

Basically, we're looking to control this line:

lexer_class && lexer_class.new(additional_options)

Here's how that might look in practice:

start_inline_if_php = ... # user-defined

lexer = Lexer.find_fancy lang do |lexer_class, opts|
  if lexer_class.tag == 'php' && start_inline_if_php
    opts[:start_inline] = true
  end
  lexer_class.new opts
end

We get all the benefits of Lexer.find_fancy while still having control over how the lexer is instantiated. And that would solve the chicken-egg problem.

I should add that this would dovetail really nicely with guess too, when you absolutely have no idea what lexer you're going to get. You would now have a callback post-guess to react to the result.

In this latest commit, I've factored out a method called Lexer.lookup_fancy, which does not instantiate a lexer, but instead returns a pair of [lexer_class, opts] - I think this is a simpler way to meet the requirement. Your example would be:

lexer_class, opts = Lexer.lookup_fancy(lang)
lexer_class ||= Rouge::Lexers::PlainText

if lexer_class.tag == 'php' && start_inline_if_php
  # use string keys for lexer options (lexer option keys are user-provided, thus unsafe for symbols)
  # and use reverse_merge in case of a manual start_inline=false in the CGI options
  opts.reverse_merge!('start_inline' => true)
end

lexer_class.new(opts)

If you don't have reverse_merge then opts['start_inline'] = true unless opts.key?('start_inline') works just as well.

I will, however, note that a Lexer instance and a pair of lexer class and options are not that different when you get right down to it...

What's the argument against the block argument? That seems to me like the more idiomatic way to do it in Ruby. And it's already used throughout the Rouge API. Returning multiple arguments from a function is always a bit awkward, especially in comparison.

I'd make the case that the block shouldn't be called if no lexer is resolved.

I didn't intend to split the discussion across two PRs. The fact the suggestions to @jneen's solution are visible in the PR list for Rouge is an unfortunate side effect of this branch being in the Rouge project.

Here's the explanation for my suggested changes:

Currently, revisions to Lexer are being discussed in #1188. This PR is intended to improve the solution provided in that branch.

The code proposed in #1188 includes a method Lexer.lookup_fancy that returns a Lexer subclass. This is in addition to the existing method Lexer.find_fancy that returns an instance of a Lexer subclass. This difference is not evident from the names of the methods and so this commit renames lookup_fancy to find_fancy_class.

Separately, the commit also adds support to consumers passing a block to find_fancy. It is not envisaged that block passing will be common but supporting it allows for consumers to use idiomatic Ruby to customise the instantiation of a lexer.

Finally, tests are added.

Please discuss here rather than in #1195.

@pyrmont Can you do a git pull upstream master onto this branch..? Thanks.

@jneen Should this branch be reworked to be part of the v4.0 release?

Superseded by #1565.