Currently JDK 8 and JavaFX is required to build and run the plugin.

The JetBrains JDK 8 distribution is recommended for development. As some parts of the code still support Java 8 and JavaFX 8, you can’t develop with Java 11 at the moment. The plugin will still run in environments with Java 11.

JetBrains JDK 8

Comes with a bundled JavaFX, download here: JetBrains OpenJDK 8 This way you can develop with the JDK that is also running the JetBrains IDE.

Oracle JDK 8

Meets both dependencies, JavaFX is bundled with it out-of-the box. This is easy to give development a start if it is already installed on your machine, but will be different from most user’s installations.

OpenJDK 8

Is usually distributed without JavaFX; you probably need to install JavaFX manually. This is usually the least preferred approach, although in practice Linux users often run JetBrains IDEA with the JDK provided by their local Linux distribution (that now and then leads to difficulties).

This plugin is built using Gradle. If you build or run it the first time it will download the community edition of IntelliJ automatically. You don’t need to install Gradle, you just need to install Java and make it available in the path.

If you have developed the plugin before it changed to Gradle you might want to remove the contents of your .idea folder to trigger a re-import of the Gradle project.

To build this plugin, you need to run:

The ZIP file with plugin to distribute will be located in build/distributions.

To run the plugin for development you’ll need to start

To run all tests and the CheckStyle validations you’ll need to start

Lexing chops the input file into a stream of tokens. Each token has a type and a snippet of characters.

The standard to do this in IntelliJ is JFlex.

The heart of lexing is asciidoc.flex. It defines multiple states, and uses a lot of functionality and tweaking to parse AsciiDoc. You can add new token types as you go in AsciiDocTokenTypes. Ensure to update the list TOKENS_TO_MERGE if consecutive identical types of the tokens should be merged. If the content of the token types should be spell-checked, add the token types to the list of tokens in AsciiDocSpellcheckingStrategy.

Once you change asciidoc.flex, you should run gradlew compileJava to generate the parser code.

A test suite for the lexer in AsciiDocLexerTest. I recommend running it from the IDE. Each test case contains a doTest() method that parses one snippet of AsciiDoc and compares it to an expected “golden master” result.

A typical developer workflow for enhancing the lexer looks like this:

1. change asciidoc.flex in the IDE, adding new entries to AsciiDocTokenTypes as needed

2. run gradlew compileJava on the command line

3. add a test case to AsciiDocLexerTest and run it from the IDE

4. if lexing doesn’t work yet as expected repeat from step 1 when

5. if lexing returns the expected result, update the expected parameter in the test

Here some JFlex rules for AsciiDoc together with an explanation of the why:

Look ahead rules

Look ahead rules are considered slow in JFlex, but they give the power to recognize tokens only when there is a matching closing token.

A slash (/) separates the matching pattern from the look ahead.

Example from parsing typographic quotes

{TYPOGRAPHIC_QUOTE_START} / [^\*\n \t] {WORD}* {TYPOGRAPHIC_QUOTE_END}

End of line and end of file parsing

JFlex supports $ to describe the end of a line as look-ahead. But this doesn’t work at the end of a file. To match the end of a file, the lexer uses the fact that JFlex will match the longest rule first (including any look-ahead rules). So first match the end of a line, then when not at the end of a line (of the same length), and then the end of the file (the shortest rule).

Example: matching a delimiter at the end of the file

// delimiter at end of line
{LISTING_BLOCK_DELIMITER} $ { /* ... */ }

// delimiter not at end of line
{LISTING_BLOCK_DELIMITER} / [^\-\n \t] { /* ... */ }

// delimiter at end of file (shorter match than the two above)
{LISTING_BLOCK_DELIMITER} { /* ... */ }

Stateful parser

To parse bold, italic and monospace text (that can be nested) there is a set of boolean variables to memorize the current text style. They are reset at the end of a block (like in regular Asciidoctor). The function textFormat() uses them to determine the current token type from a combination of these flags.

Other states memorize the length of block separator line to find the matching closing separator.

qualifying matches, push back and state change

After a match the Java code checks additional conditions like if this is an unconstrained position in the stream. If the code decides to discard the match, two possible strategies out of several are:

1. push back all but the first character, and return the token type for the single character (for example when an double-asterisk occurs, but no bold text is to end here, see {DOUBLEMONO} in the lexer).

2. push back the complete text and continue with a different state using yybegin() (for example when matching a {HEADING_OLDSTYLE} in the MULTILINE state).

3. some of the expressions can be prefixed with a backslash (\) to escape the expression. Use isEscaped() to check if it has been escaped.

Unfortunately, the parser can’t continue with other matches in the same state. To work around this issue blocks are parsed first in state MULTILINE, then in state SINGLELINE, and finally INSIDE_LINE to implement a hierarchy and some ordering of matches.

auto-completion

Expressions described above match expressions once they have their closing syntax completed and it is essential for the correct highlighting. To support autocomplete the matching must handle an expression where only the left part of the expression exists.

A special case is in the parser to support autocompletion, as IntelliJ inserts a special string when parsing the content for autocompletion (named auto-complete in our parser).

In the case for references (<<ref>>) there are two rules, one for regular parsing and highlighting, one without:

// regular
{REFSTART} / [^>\n]+ {REFEND} { yybegin(REF); return AsciiDocTokenTypes.REFSTART; }
// auto-complete
{REFSTART} / [^>\n ]* {AUTOCOMPLETE} { yybegin(REFAUTO); return AsciiDocTokenTypes.REFSTART; }

Highlighting is coloring the text in the editor.

The file AsciiDocSyntaxHighlighter defines one TextAttributesKey to each entry in AsciiDocTokenTypes parsed during lexing. Currently several tokens have the same highlighting ASCIIDOC_MARKER, so users have the same color for the pointy brackets around references references (<<ref>>)and markers for bold (*bold*).

Once you add a new TextAttributesKey, you should either

1. reference an existing color (like ASCIIDOC_COMMENT references DefaultLanguageHighlighterColors.LINE_COMMENT) OR

2. add a color the AsciiDoc themes AsciidocDefault.xml and AsciidocDarcula.xml

Once you add a new token you will need to add it to AsciiDocColorSettingsPage so users can customize the colors of their theme. This class references also SampleDocument.adoc and AsciiDocBundle.properties, therefore you’ll probably need to change these two files as well.

The central class and method to create AsciiDoc from HTML is AsciiDoc.render(). It is implemented as a singleton.

It registers custom Asciidoctor extensions that are needed for improve the preview. It also enables custom extensions in the .asciidoctorconfig folder.

There is a JeditorHtmlPanel (for Swing) and a JavaFxHtmlPanel (for JavaFX) preview.

The JavaFX preview is the current default preview. It is available when the user is running 64bit JDK with JavaFX (the default JDK for JetBrains IDE).

For the JavaFX preview the HTML is enriched with CSS and JavaScript.

The JavaFX preview uses JavaScript to scroll the preview to the current position: once the user moves the cursor, the cursor line is transmitted to the preview using scrollToLine() and repositions the preview using JavaScript.

When the user interacts with the JavaFX preview (for example clicks on a text or a link), there is a bridge JavaPanelBridge back from JavaScript to Java to trigger actions like scrolling the editor or opening a link in the browser.

You can log information from the JavaFX preview the preview using

this will call the method JavaPanelBridge#log, an inner class of JavaFxHtmlPanel.

1. Update CHANGELOG.adoc with th latest changes for the release

2. Push all changes to GitHub

3. Create a Release in the GitHub releases. This allows you to also create a tag. Name the tag like the release (for example: 0.28.2)

4. Travis CI will then build the release The plugin.xml included in the build will contain the release version and the most recent entries from the change log. TravisCI will publish a binary to:

   • GitHub releases (attached as asciidoctor-intellij-plugin.zip)

   • EAP (early access program) repository, see Adding the EAP Repository for more information.

1. Edit the CHANGELOG.adoc and remove the “(preview …​)” additions here

2. Copy the link of the asciidoctor-intellij-plugin.zip on GitHub releases to the clipboard

3. Go to the JetBrains plugin repository and upload the plugin to the stable repository using ‘Get file from URL’