Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: expose plugin extension points into main package, developer docs + sample #713

Merged
merged 5 commits into from
Oct 21, 2024

Conversation

coopernetes
Copy link
Contributor

@coopernetes coopernetes commented Sep 9, 2024

  • update PluginLoader to use load-plugin's native Node module resolution. this library already
    handles loading of files vs modules in node_modules/ so let's use that directly
  • add configuration-based plugin loading and deprecate environment variables
  • new documentation page for plugin development. loading, writing, etc.
  • add PullActionPlugin + unify config on just "plugins"
  • add explicit exports to make importing @finos/git-proxy more straight forward for plugins

In addition, this commit adds both CommonJS and ES module examples of plugins and refactors them
into a more reflective package. It also includes updates to the site documentation, specifically
in the Development section, which now includes details about plugins and contribution guidelines.

  • fix issue with json-schema-for-humans producing bad output due to post-processing (script step
    no longer required)
  • docs: add section on docs on how to update the config schema + re-generate reference doc
  • fix: don't package website with git-proxy, update .npmignore on npm pack/publish

Side note:
The version was bumped from 1.3.4 to 1.3.5-alpha.0. This was necessary for validating the new package behaviour on a version that was distinct from the latest version of the public @finos/git-proxy package. This can be rolled into a cumulative release instead of an "alpha" test release. reverted this change, now using the version straight from main.

Copy link

netlify bot commented Sep 9, 2024

Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit a7b799c
🔍 Latest deploy log https://app.netlify.com/sites/endearing-brigadeiros-63f9d0/deploys/6716afc426b8ac000881ab08

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

Attention: Patch coverage is 91.11111% with 8 lines in your changes missing coverage. Please review.

Project coverage is 60.08%. Comparing base (661bdea) to head (a7b799c).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
src/plugin.js 92.72% 4 Missing ⚠️
src/config/index.js 50.00% 3 Missing ⚠️
src/proxy/routes/index.js 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #713      +/-   ##
==========================================
+ Coverage   57.75%   60.08%   +2.32%     
==========================================
  Files          46       46              
  Lines        1598     1631      +33     
==========================================
+ Hits          923      980      +57     
+ Misses        675      651      -24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@rgmz rgmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts that may or may not be helpful.

src/plugin.js Show resolved Hide resolved
src/plugin.js Show resolved Hide resolved
src/plugin.js Show resolved Hide resolved
src/proxy/index.js Outdated Show resolved Hide resolved
@maoo
Copy link
Member

maoo commented Sep 16, 2024

Thanks @rgmz for the review! Can someone look at his comments? And since we're there, you may probably want to update the branch with latest changes from main.

I read through the PR, code looks clean and docs seems to be solid; sadly - with OSFF in 2 weeks - it's hard for me to find time to test https://deploy-preview-713--endearing-brigadeiros-63f9d0.netlify.app/docs/development/plugins , so if anyone would be interested to go through these docs and test it locally, it would be great! Sections that are IMO particularly interesting are https://deploy-preview-713--endearing-brigadeiros-63f9d0.netlify.app/docs/development/plugins#via-npm-packages and https://deploy-preview-713--endearing-brigadeiros-63f9d0.netlify.app/docs/development/plugins#developing-new-plugins

FYI - @vaibssingh @divinetettey @bill-n

config.schema.json Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
plugins/README.md Outdated Show resolved Hide resolved
plugins/README.md Outdated Show resolved Hide resolved
@JamieSlome
Copy link
Member

Codecov Report

Attention: Patch coverage is 66.33663% with 34 lines in your changes missing coverage. Please review.

Project coverage is 57.52%. Comparing base (3880ed1) to head (2c64d63).

Files with missing lines Patch % Lines
src/proxy/chain.js 15.00% 17 Missing ⚠️
src/plugin.js 81.42% 13 Missing ⚠️
src/config/index.js 50.00% 3 Missing ⚠️
src/proxy/routes/index.js 50.00% 1 Missing ⚠️
Additional details and impacted files
☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here.

Any chance we can bump up the test coverage before merging?

Copy link
Member

@JamieSlome JamieSlome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🍰 Address comments and happy to merge! ❤️

@coopernetes coopernetes force-pushed the feat/new-plugin-system branch from 007b5d7 to 6d539de Compare October 2, 2024 17:55
@coopernetes
Copy link
Contributor Author

Codecov Report

Attention: Patch coverage is 66.33663% with 34 lines in your changes missing coverage. Please review.

Project coverage is 57.52%. Comparing base (3880ed1) to head (2c64d63).

Files with missing lines Patch % Lines
src/proxy/chain.js 15.00% 17 Missing ⚠️
src/plugin.js 81.42% 13 Missing ⚠️
src/config/index.js 50.00% 3 Missing ⚠️
src/proxy/routes/index.js 50.00% 1 Missing ⚠️
Additional details and impacted files
☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here.

Any chance we can bump up the test coverage before merging?

I can add additional tests if necessary. Note that these code paths have not had previous tests so some additional refactoring may be required. Mind if we incorporate this in a follow-up PR? It's more important to give developers adequate docs for plugin development ahead of improving test coverage for existing code that is known to be working. I've added tests for the most critical new code paths. This PR adds e2e tests for plugins to be installed & consumed which do validate these missing code paths - it's just not reflected in coverage.

@JamieSlome
Copy link
Member

JamieSlome commented Oct 2, 2024

Codecov Report

Attention: Patch coverage is 66.33663% with 34 lines in your changes missing coverage. Please review.

Project coverage is 57.52%. Comparing base (3880ed1) to head (2c64d63).

Files with missing lines Patch % Lines
src/proxy/chain.js 15.00% 17 Missing ⚠️
src/plugin.js 81.42% 13 Missing ⚠️
src/config/index.js 50.00% 3 Missing ⚠️
src/proxy/routes/index.js 50.00% 1 Missing ⚠️
Additional details and impacted files
☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here.

Any chance we can bump up the test coverage before merging?

I can add additional tests if necessary. Note that these code paths have not had previous tests so some additional refactoring may be required. Mind if we incorporate this in a follow-up PR? It's more important to give developers adequate docs for plugin development ahead of improving test coverage for existing code that is known to be working. I've added tests for the most critical new code paths. This PR adds e2e tests for plugins to be installed & consumed which do validate these missing code paths - it's just not reflected in coverage.

Happy 🤝 @coopernetes - once you've resolved all remaining conversations, happy for you to proceed and we can merge, and release 🎉 I'll get the LinkedIn juices flowing and put together a social media post as well.

@JamieSlome
Copy link
Member

Honestly can't wait to have the discussion about community plugins ❤️

src/plugin.js Outdated Show resolved Hide resolved
…s + sample

- update PluginLoader to use load-plugin's native Node module resolution. this library already
  handles loading of files vs modules in node_modules/ so let's use that directly
- add configuration-based plugin loading and deprecate environment variables
- new documentation page for plugin development. loading, writing, etc.
- add PullActionPlugin + unify config on just "plugins"
- add explicit exports to make importing @finos/git-proxy more straight forward for plugins

In addition, this commit adds both CommonJS and ES module examples of plugins and refactors them
into a more reflective package. It also includes updates to the site documentation, specifically
in the Development section, which now includes details about plugins and contribution guidelines.

- fix issue with json-schema-for-humans producing bad output due to post-processing (script step
  no longer required)
- docs: add section on docs on how to update the config schema + re-generate reference doc
- fix: don't package website with git-proxy, update .npmignore on npm pack/publish
- remove actions which are used in internal fork from chain
- add default config value for plugins (an empty array)
@coopernetes coopernetes force-pushed the feat/new-plugin-system branch from 356e356 to e0915ce Compare October 21, 2024 19:29
- add test cases for getChain + executeChain to validate the processor execution
- remove extraneous function to create a loader and just use the constructor
- address review comments
@coopernetes coopernetes force-pushed the feat/new-plugin-system branch from e0915ce to a7b799c Compare October 21, 2024 19:47
@coopernetes
Copy link
Contributor Author

There's a few outstanding things which I'll handle in a follow-up PR (for example, updating all the version strings to the latest 1.3.10). Will merge so we can close #539

@coopernetes coopernetes merged commit 0b1bc4a into finos:main Oct 21, 2024
14 checks passed
@coopernetes coopernetes deleted the feat/new-plugin-system branch October 21, 2024 20:01
@JamieSlome
Copy link
Member

@coopernetes - woo 🎉

We are currently at version 1.3.10. Should we bump to 1.4.0 given that we are introducing a new API structure? More looking for your thoughts here rather than prescribing as not entirely sure 👍

Psingle20 pushed a commit to Psingle20/git-proxy that referenced this pull request Nov 27, 2024
feat: expose plugin extension points into main package, developer docs + sample
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants