Change js to jar dependency

[zane-neo]

Description

There're two types of dependency in our repo: on jar which usually under org.opensearch: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/ and on plugin which usually under org.opensearch.plugin: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/.
For most common cases, we should depend on jars instead of plugins, because depend on plugin is a workaround and depend on jar is the most natural way.

We're depending on sql's plugin because sql doesn't publish jars for ppl.jar and protocol.jar and sql team refused to publish these jars, so we have to extract them from sql plugin zip.

Job-scheduler publishes jars to this location: org.opensearch:opensearch-job-scheduler-${version}: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/opensearch-job-scheduler/, so we can depend on jar directly. This PR is to make this change.

For Anomaly-detection, the jars also not published to org.opensearch: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/ , and the reason I assume is missing publishAllPublicationsToStagingRepository in scripts/build.sh, for now to not break compilation,
we can only extracting jars from ad plugin. If ad in the future publishes jars to sonatype, we can change ad's dependency to jar's approach just like job-scheduler.

For forced versions: I published ad jars in my local so I can change ad dependency to jar type but found there's jar version conflicts, so forced the versions, after publish the PR, I found ad jars are not in sonatype, so rolled back ad to plugin dependency but didn't remove this force version. I've update this part to remove them.

Issues Resolved

Change js to jar dependency

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d2e9c72) 81.76% compared to head (13f6fc3) 81.76%.

Additional details and impacted files

@@            Coverage Diff            @@
##               main      #74   +/-   ##
=========================================
  Coverage     81.76%   81.76%           
  Complexity       86       86           
=========================================
  Files             8        8           
  Lines           521      521           
  Branches         70       70           
=========================================
  Hits            426      426           
  Misses           54       54           
  Partials         41       41           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ohltyler]

Could you add more details on the reasoning for this change, what it is changing exactly, and what the blockers are in AD and SQL plugin to do the same? From my understanding it is that AD, for example, doesn't have jars in sonatype? I see it does exist there for job-scheduler. Do we have a path forward to add?

Additionally, I see you added forced versions for some dependencies - was this due to jarhell issues when changing the dependency model for JS? I'm worried this will be more of an issue if more plugins with their dependencies are added this way.

[zane-neo]

Could you add more details on the reasoning for this change, what it is changing exactly, and what the blockers are in AD and SQL plugin to do the same? From my understanding it is that AD, for example, doesn't have jars in sonatype? I see it does exist there for job-scheduler. Do we have a path forward to add?

Additionally, I see you added forced versions for some dependencies - was this due to jarhell issues when changing the dependency model for JS? I'm worried this will be more of an issue if more plugins with their dependencies are added this way.

@ohltyler Updated the description, please take a look.

[ohltyler]

Thanks for the details! Makes sense to me.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/skills/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/skills/backport-2.x
# Create a new branch
git switch --create backport/backport-74-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 edf31505ba25c38aa61bb5908e827dc784e62ee6
# Push it to GitHub
git push --set-upstream origin backport/backport-74-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/skills/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-74-to-2.x.