Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide all services related to Legend deployment #327

Open
eddie-knight opened this issue Mar 8, 2023 · 8 comments
Open

Provide all services related to Legend deployment #327

eddie-knight opened this issue Mar 8, 2023 · 8 comments
Assignees
Labels
epic policy wg reproducible infra wg Allocated to reproducible infrastructure working group runtime validation wg

Comments

@eddie-knight
Copy link
Contributor

eddie-knight commented Mar 8, 2023

This issue follows an ongoing discussion related to our project's position within FINOS: Our services should be inarguably valuable to other projects in the ecosystem.

The team will need to identify the resources required for a complete Legend deployment, and subsequently create the policies, IaC, and validation packs for each service. This will benefit any Legend users who want CFI resources to deploy their instance of Legend.

As a first step for the policy development, we will solicit information regarding the policies used by the FINOS infrastructure team for the Legend instance they host. Subsequent policies will require guidance from Legend end users. This will allow the RI and RV groups to begin work on those services.

@abdullahgarcia
Copy link
Contributor

@eddie-knight , who's our primary contact for Legend? Can we please involve him/her in this issue?

@eddie-knight
Copy link
Contributor Author

eddie-knight commented Mar 12, 2023

I suspect @maoo can help us find the info we need to plan this out

@maoo
Copy link
Member

maoo commented Mar 13, 2023

Hi @abdullahgarcia and @eddie-knight !

You can find all the info you need on https://legend.finos.org/docs/getting-started/installation-guide - I suppose that the Docker compose file gives a very clear idea on how to deploy. Also note that there is a Juju integration for legend on https://github.com/finos/legend-juju-bundle .

If you have any further question, the best way to engage with the Legend team is via https://github.com/finos/legend/issues

If you have questions related with our production environment on legend.finos.org/studio , I'm the right person.

Hope this helps!

@abdullahgarcia
Copy link
Contributor

Thanks @maoo, will have a look!

@eddie-knight
Copy link
Contributor Author

RFC @maoo

It looks like we just need these three elements to prepare an infrastructure for the legend deployment... could you take a look to see if we missed anything here? After we have these child modules built, we'll try out a deployment to see if we can provide a recommended tf config for the end-to-end deploy.

  • EKS (w/ VPC)
  • S3
  • Mongo

@maoo
Copy link
Member

maoo commented Mar 23, 2023

  • EKS (w/ VPC)

I'd suggest creating a user in the AWS CFI account that is able to create and tear down EKS clusters; I can see that we already have a user on the CFI (FINOS) AWS IAM user (and group), with a custom policy called CSC-Terraform-Policy

Maybe we can reuse this group/policy and just create a new user?

  • S3

I believe that this is used only for CDK deployments; is this what you intend to use?

  • Mongo

I'd suggest using a container for this; please note that Mongo acts as a session cache, so there is no need to persist this data.

Hope this helps!

@eddie-knight
Copy link
Contributor Author

Thanks @maoo!
@thinkl33t @AdrianHammond @ml4

For the development purposes, it looks like we'll just need to finalize EKS and set up a Mongo child module. For dev/test purposes we'll follow Mao's guidance and make sure we're able to deploy legend using our modules, then we can make a pull request to Legend to see if they want to list that config example it as an installation quickstart.

Subject to y'alls feedback, I think the next step is to create the mongo child module repo.

@eddie-knight eddie-knight moved this from To Do to In Progress in CFI - Main Project Kanban Apr 12, 2023
@eddie-knight
Copy link
Contributor Author

@thinkl33t could you link any associated RI WG issues to this epic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic policy wg reproducible infra wg Allocated to reproducible infrastructure working group runtime validation wg
Projects
Status: In Progress
Archived in project
Development

No branches or pull requests

5 participants