Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improvement: replace react-syntax-highlighter with shiki #463

Merged
merged 14 commits into from
Feb 26, 2024

Conversation

abvthecity
Copy link
Contributor

@abvthecity abvthecity commented Feb 25, 2024

What changed?

In this PR, we've:

  1. fully removed react-syntax-highlighter and replaced it with a custom rehype plugin built on top of shiki.
  2. all syntax highlighting is now computed server-side
  3. line numbers are generated using special CSS, and bash/shell uses $ and > instead of line numbers
  4. the order of the example json is now sorted to match the order that they appear in the API reference (this enhances readability, and hovering over property keys don't cause the code snippet scroll-area to jump all over the place).

Why make this change?

  • react-syntax-highlighter is heavyweight, and requires a lot of work to make it dynamic, so that the initial JS is small. shiki ensures that every language and theme is dynamically loaded
  • the customization options in react-syntax-highlighter is limited, requiring us to implement only a single color scheme, whereas shiki allows us to swap between a number of built-in themes, as well as enable providing custom TextMate theme files.
  • light-mode and dark-mode are handled by shiki out-of-the-box, letting us apply the themes via CSS variables rather than using javascript. this ensures that no hydration failures occur and the theme is loaded before the DOM is ready.
  • with shiki, we can offload syntax highlighting to occur on the server-side, significantly reducing the runtime burden from the client side
  • we now have the option to implement several shiki transformers as well as twoslash for more interactive and rich code snippets

Why is this PR so large?

  • implemented custom rehype plugin, which required bumping up our dependency of unified from 2 -> 3, in addition to a few other util packages
  • in order to support server-side code snippets, i've introduced a major refactor of resolver.ts which now outputs safely serializable JSON, which required touching a bunch of places where the data structure is consumed.
  • by moving syntax highlighting to the server-side, we actually significantly increase the size of the JSON payload b/c we are passing back the HAST for each code snippet. to reduce the payload size, i've went ahead and removed duplicate content, such as the unresolved api from the server. This required removing usage of pathResolver from the frontend, so I had to replicate the functionality using the new resolved data structure

What's next?

  • add shiki transformers
  • add twoslash integration for hoverable json property descriptions
  • improve handling the HAST (currently, the HAST is turned in the JSX and we're parsing the JSX in CodeBlock/CodeBlocks. Instead, this should be parsed in the rehype step)

Before:
Screenshot 2024-02-26 at 2 46 29 PM

After:
Screenshot 2024-02-26 at 2 46 45 PM

Before:
Screenshot 2024-02-26 at 2 48 03 PM

After:
Screenshot 2024-02-26 at 2 48 29 PM

Before:
Screenshot 2024-02-26 at 2 49 01 PM

After:
Screenshot 2024-02-26 at 2 49 27 PM

Before:
Screenshot 2024-02-26 at 2 53 31 PM

After:
Screenshot 2024-02-26 at 2 53 44 PM

Copy link

vercel bot commented Feb 25, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
fern-dev ✅ Ready (Inspect) Visit Preview Feb 26, 2024 8:49pm
3 Ignored Deployments
Name Status Preview Updated (UTC)
fern-prod ⬜️ Ignored (Inspect) Visit Preview Feb 26, 2024 8:49pm
fontawesome-cdn ⬜️ Ignored (Inspect) Visit Preview Feb 26, 2024 8:49pm
private-docs ⬜️ Ignored (Inspect) Visit Preview Feb 26, 2024 8:49pm

@abvthecity abvthecity merged commit 16f569c into main Feb 26, 2024
11 of 12 checks passed
@abvthecity abvthecity deleted the ajiang/shiki-impl branch February 26, 2024 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant