Context

• We want to convert the paragraph length and sentence length assessments to use HTML Parser for the following reasons:
  • To improve the robustness of the assessments' analysis processing
  • To improve the accuracy of the highlighting
  • To make it possible to output the client ID of the blocks that are long or the blocks that contain long sentences
    • This is especially necessary for implementing the Yoast AI Optimize for both assessments

Summary

This PR can be summarized in the following changelog entry:

• [yoastseo] Converts sentence length and paragraph length assessments to use HTML parser in their analysis.
• [yoastseo] Converts sentence length and paragraph length assessments highlighting approach from search and replace to position-based approach.
• [yoastseo] Enables Yoast AI Optimize button for sentence length and paragraph length assessments.
• Introduces more robust HTML processing and highlighting for the sentence length and paragraph length assessments.
• [shopify-seo] Introduces more robust HTML processing and highlighting for the sentence length and paragraph length assessments.

Relevant technical choices:

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

Optimize AI button in Free

• Install and activate Yoast SEO
• Go through the steps as described in this scenario
  for sentence length and paragraph length assessments
• Confirm that you get the expected result

For the scenarios below, also install and activate Yoast SEO Premium. If building the plugin, pair this PR with this PR in Premium: https://github.com/Yoast/wordpress-seo-premium/pull/4527

Paragraph length assessment

• Follow the testing steps as described in this document to test the paragraph length analysis
• Run the test in WordPress and in Shopify

Sentence length assessment

• Follow the testing steps as described in this document to test the sentence length analysis
• Run the test in WordPress and in Shopify

Relevant test scenarios

• Changes should be tested with the browser console open
• Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
• Changes should be tested on different editors (Default Block/Gutenberg/Classic/Elementor/other)
• Changes should be tested on different browsers
• Changes should be tested on multisite

Test instructions for QA when the code is in the RC

• QA should use the same steps as above.

QA can test this PR by following these steps:

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

• No further impact other than what is already covered in the testing instructions.

UI changes

• This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

• This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

• I have written documentation for this change. For example, comments in the Relevant technical choices, comments in the code, documentation on Confluence / shared Google Drive / Yoast developer portal, or other.

Quality assurance

• I have tested this code to the best of my abilities.
• During testing, I had activated all plugins that Yoast SEO provides integrations for.
• I have added unit tests to verify the code works as intended.
• If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
• I have written this PR in accordance with my team's definition of done.
• I have checked that the base branch is correctly set.

Innovation

• No innovation project is applicable for this PR.
• This PR falls under an innovation project. I have attached the innovation label.
• I have added my hours to the WBSO document.

Fixes https://github.com/Yoast/lingo-other-tasks/issues/114
Fixes https://github.com/Yoast/lingo-other-tasks/issues/471
Fixes https://github.com/Yoast/wordpress-seo-premium/issues/4524