Fixes #643
Fixes #648
Fixes #841

The problem is that with the fix performed in #607, when you have something like this:

const pure = () => {
  return (
      <Foo
        // one
        foo={123}
        // two
        bar="woof"
      />
  );
}

We currently perform reScanJsxToken() on the closing brace for the foo attribute value.

This results in a SyntaxKind.JsxText token like this:

"}\n        // two\n        bar="woof"\n      />\n  );\n}\n"

So the // two never gets correctly identified. If we hadn't performed the reScanJsxToken() on the } closing brace for the JSX expression, the // two would have been tokenized correctly.

I have fixed this by changing the rescan call added in the PR referenced earlier to only apply if the JSX expression is inside a JSX element (which addresses the case it was added to fix to begin with).

Honestly, this feels like a bit of a band-aid solution. Admittedly, I don't have a lot of experience with the typescript scanner but I feel as though this is the best solution without reworking the method a bit. Looking at some of the tokens we end up with after rescanning sometimes, there seems to be some sacrifices made in order to keep this as simple as possible and just locate the comments (which is a totally fair and valid design decision).

I feel like a better way to make these rescan decisions would be to move forward to the next meaningful token before making the decision to rescan. Like, if we move forward past the } in this example:

const fifth = <div>{}http://example.com</div>;

And then look at the AST for the next token (http which ends up as SyntaxKind.Identifier without rescanning), the AST will tell us that we're on a JSXText node. So if we know the proceeding token was a closing brace for a JSXExpression, we can check the AST to see if we're on a JSXText node before making the decision to rescan. Even if the next token is a whitespace or a new line, the AST we have will already know that's a JSXText node.

I think this is a better approach than basing it on the type of parent for the JSXExpression (the change I made to fix the issue in this PR), since the issue is that we don't want the jsx text nodes to get tokenized incorrectly (which is what resulted in the text after http: getting incorrectly picked up as a comment). We could also probably benefit from this approach for the > token logic too.

If you want, I can update this PR and use the technique described above instead of the current fix I have applied.

Anyways, I have no experience working with the TS scanner output so I could be way off in my conclusions here, but the fact that we end up with tokens identified as JsxText that look like }http://example.com (notice the } included) also indicates that we're a little off in the approach here (although one can argue that for the sake of extracting the comments it's irrelevant).