Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend exception list for German nouns with umlaut in plural #21796

Open
wants to merge 8 commits into
base: trunk
Choose a base branch
from

Conversation

hannaw93
Copy link
Contributor

@hannaw93 hannaw93 commented Nov 5, 2024

Context

  • Many nouns with -umlaut (e.g. singular rand and plural ränder) were incorrectly stemmed because our exception list of words with multiple stems was not exhaustive. This PR (paired with the premium-configuration PR) fixes the problem above.

Summary

This PR can be summarized in the following changelog entry:

  • [wordpress-seo-premium] Improves keyphrase recognition for German plural noun forms with an umlaut.
  • [shopify-seo] Improves keyphrase recognition for German plural noun forms with an umlaut.
  • [yoastseo] Adds a separate check for German plural nouns containing umlaut.

Relevant technical choices:

  • Nouns that attach an umlaut in plural sometimes attach a suffix such as -e or -er as well. However, since those suffixes are covered in the stemmer, we only add the stem variants to the exception list (the variant with and without an umlaut).
  • The words that have umlaut in their stem and an ending that that looks like a valid suffix but it's actually part of the stem (e.g. läden ) are saved in a separate list umlautException. This list is checked before the stemming steps are executed so that these words don't get overstemmed.

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

⚠️ Until the premium configuration data is deployed, testing can only be done through the content analysis app.
Content analysis app

  • Go to apps/content-analysis and run the app (run yarn install and yarn start)
  • Fill in the Locale field with de

Wordpress

  • Install and activate Yoast SEO and Yoast SEO Premium
  • Set the site language to German

SCENARIO 1: plurals with umlaut and suffix -er

  • Create a post
  • Add wörter or loch as the keyphrase
  • Add the following words to the post: wort, löcher.
  • Go to Keyphrase density and Keyphrase distribution assessments and make sure that both forms are recognized as forms of the keyphrase

SCENARIO 2: plurals with umlaut and irregular case suffix -n

  • Create a post
  • Add äpfeln or müttern as the keyphrase
  • Add the following words to the post: apfel, mutter.
  • Go to Keyphrase density and Keyphrase distribution assessments and make sure that both forms are recognized as forms of the keyphrase
  • Remove the above two words from the post and add äpfel, mütter instead.
  • Go to Keyphrase density and Keyphrase distribution assessments and make sure that both forms are recognized as forms of the keyphrase

SCENARIO 3: plurals with umlaut ending on -er

  • Create a post
  • Add wörter or loch as the keyphrase
  • Add the following words to the post: wort, löcher.
  • Go to Keyphrase density and Keyphrase distribution assessments and make sure that both forms are recognized as forms of the keyphrase

SCENARIO 4: plurals with umlaut ending on -e

  • Add ausflüchte or macht as the keyphrase
  • Add ausflucht anywhere in the post and mächte in the introduction of the post
  • Go to Keyphrase density and make sure ausflucht is recognized as a form of the keyphrase
  • Go to Keyphrase in introduction and make sure mächte is recognized as a form of the keyphrase

SCENARIO 5: plurals with umlaut and no suffix

  • Add kästen or schwager as the keyphrase
  • Add kasten to the post and schwäger to the metadescription
  • Go to Keyphrase density make sure kasten is recognized as a form of the keyphrase
  • Go the Keyphrase in metadescription assessment and make sure that schwäger is recognized as the form of the keyphrase

Shopify

Create a product and repeat the steps above from Step 2.

Relevant test scenarios

  • Changes should be tested with the browser console open
  • Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
  • Changes should be tested on different editors (Default Block/Gutenberg/Classic/Elementor/other)
  • Changes should be tested on different browsers
  • Changes should be tested on multisite

Test instructions for QA when the code is in the RC

  • QA should use the same steps as above.

QA can test this PR by following these steps:

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

UI changes

  • This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

  • This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

  • I have written documentation for this change. For example, comments in the Relevant technical choices, comments in the code, documentation on Confluence / shared Google Drive / Yoast developer portal, or other.

Quality assurance

  • I have tested this code to the best of my abilities.
  • During testing, I had activated all plugins that Yoast SEO provides integrations for.
  • I have added unit tests to verify the code works as intended.
  • If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
  • I have written this PR in accordance with my team's definition of done.
  • I have checked that the base branch is correctly set.

Innovation

  • No innovation project is applicable for this PR.
  • This PR falls under an innovation project. I have attached the innovation label.
  • I have added my hours to the WBSO document.

Fixes https://github.com/Yoast/lingo-other-tasks/issues/390

@hannaw93 hannaw93 added changelog: enhancement Needs to be included in the 'Enhancements' category in the changelog Shopify This PR impacts Shopify. labels Nov 5, 2024
@hannaw93 hannaw93 changed the title add tests for each group of vowel changing to umlaut nouns Extend exception list for German nouns with umlaut in plural Nov 5, 2024
@hannaw93 hannaw93 marked this pull request as ready for review November 5, 2024 13:53
@hannaw93 hannaw93 added this to the 24.0 milestone Nov 7, 2024
@mhkuu mhkuu removed this from the 24.0 milestone Nov 8, 2024
@coveralls
Copy link

coveralls commented Nov 18, 2024

Pull Request Test Coverage Report for Build ebadf77212deffa09c4d43d5593470d8dca688a4

Details

  • 8 of 8 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.008%) to 57.493%

Totals Coverage Status
Change from base Build 4864684a3cf54d83e326b48ea33600e4728faafe: 0.008%
Covered Lines: 13426
Relevant Lines: 23009

💛 - Coveralls

…internal/determineStem.js


Update code comment
@agnieszkaszuba
Copy link
Contributor

CR: Looks good to me, I only added some comments for the premium configuration PR, which will require adding some unit tests and test instructions to this PR 🙂

Copy link
Contributor

@FAMarfuaty FAMarfuaty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! ✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog: enhancement Needs to be included in the 'Enhancements' category in the changelog Shopify This PR impacts Shopify.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants