Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: GitLab CI model and base classes #1357

Merged
merged 23 commits into from
Jan 2, 2022
Merged

feat: GitLab CI model and base classes #1357

merged 23 commits into from
Jan 2, 2022

Conversation

dontirun
Copy link
Contributor

Related to #137, #942

This PR adds the model and some opinionated base classes for GitLab CI configurations (which should be reviewed). This currently does not tie the classes into a project configuration. I have not added any tests, but would be happy to add them as this PR progresses.

Base schema from here
Initial TypeScript Interfaces (before opinionated modifications) generated from here


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@codecov-commenter
Copy link

codecov-commenter commented Dec 20, 2021

Codecov Report

Merging #1357 (60bc5ff) into main (d90284c) will decrease coverage by 0.31%.
The diff coverage is 87.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1357      +/-   ##
==========================================
- Coverage   88.06%   87.74%   -0.32%     
==========================================
  Files         132      141       +9     
  Lines        5109     5679     +570     
  Branches     1207     1450     +243     
==========================================
+ Hits         4499     4983     +484     
- Misses        610      694      +84     
- Partials        0        2       +2     
Impacted Files Coverage Δ
src/cdk/consts.ts 100.00% <ø> (+36.36%) ⬆️
src/dev-env.ts 83.33% <0.00%> (ø)
src/java/index.ts 100.00% <ø> (ø)
src/python/index.ts 100.00% <ø> (ø)
src/release/index.ts 100.00% <ø> (ø)
src/typescript/typescript-typedoc.ts 20.00% <0.00%> (ø)
test/util.ts 96.15% <ø> (+0.26%) ⬆️
src/semver.ts 13.33% <8.33%> (-6.67%) ⬇️
src/task-runtime.ts 12.31% <21.42%> (-0.28%) ⬇️
src/logger.ts 59.61% <29.41%> (-6.35%) ⬇️
... and 143 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5899dd...60bc5ff. Read the comment docs.

Copy link
Contributor

@Chriscbr Chriscbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great start! I think it would be awesome to have GitLab components in projen.

I've added some initial comments below. Be sure to add tests, and if you could also write up a short high level summary in docs/gitlab.md it would be very helpful for understanding the API as well.

src/gitlab/configuration-model.ts Outdated Show resolved Hide resolved
src/gitlab/configuration-model.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/gitlab-configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
@dontirun dontirun requested a review from Chriscbr December 28, 2021 22:20
docs/gitlab.md Outdated Show resolved Hide resolved
docs/gitlab.md Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Show resolved Hide resolved
* Add additional services.
* @param services The services to add.
*/
public addServices(...services: (string | Service)[]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does specifying a service with a string have the same meaning as using { name: 'the-string' }? if so, I feel like this is an implementation detail that probably shouldn't be covered in our API -- plus introducing type unions makes the API more obtuse when this gets compiled to other languages using JSII.

Can we simplify the API like this maybe:

Suggested change
public addServices(...services: (string | Service)[]) {
public addService(options: Service) {

(Users can call this in a for loop to add multiple services if they want)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with removing the string option from the API, but I think we should still use the spread operator here. Many of the other APIs within projen use the spread operator and I think it would be better to be consistent.

Additionally I don't think a for loop will be helpful in this case. I think users would end up doing something like the following.

addService({ name: 'my-sql-service'})
addService({ name: 'my-nosql-service'})
addService({ name: 'my-gitlab-api-service'})

I don't think something like this would make much sense either

const services = [...]
for (const service of services){
   addService(service)
}
Suggested change
public addServices(...services: (string | Service)[]) {
public addServices(...services: Service[]) {

src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
Co-authored-by: Christopher Rybicki <crybicki98@gmail.com>
@dontirun
Copy link
Contributor Author

I previously missed that the Global services keyword is deprecated.

In general it might make more sense to make a property in the CiConfiguration Class for each of the properties in the Default interface (ex. defaultAfterScript) and reassemble everything in the renderCI method.

This would also give more flexibility in adding more methods to help the user configure these properties

  private renderCI() {
    return {
      default: { afterScript: this.defaultAfterScript, ... },
      include:
        this.include.length > 0 ? snakeCaseKeys(this.include) : undefined,
      pages: snakeCaseKeys(this.pages),
      services:
        this.services.length > 0 ? snakeCaseKeys(this.services) : undefined,
      variables:
        Object.entries(this.variables).length > 0 ? this.variables : undefined,
      workflow: snakeCaseKeys(this.workflow),
      stages: this.stages.length > 0 ? this.stages : undefined,
      ...snakeCaseKeys(this.jobs),
    };
  }
}

@dontirun dontirun requested a review from Chriscbr December 30, 2021 01:03
Copy link
Contributor

@Chriscbr Chriscbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more small changes and I think this PR is about ready to ship 👍

On further thought, I'm not totally sure what the best practices are here regarding leaving class properties public or private, since this class is both like an L1 (not a lot of opinionated defaults) but also like an L2 (it some provides convenience methods). I want to say it probably makes sense to make most of the properties private by default so that they can be changed as needed in the future and make the class more L2-like, but I don't have a strong opinion on this.

@eladb do you have any guidance here?

src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
src/gitlab/configuration.ts Outdated Show resolved Hide resolved
* Add additional services.
* @param services The services to add.
*/
public addServices(...services: (string | Service)[]) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with removing the string option from the API, but I think we should still use the spread operator here. Many of the other APIs within projen use the spread operator and I think it would be better to be consistent.

Additionally I don't think a for loop will be helpful in this case. I think users would end up doing something like the following.

addService({ name: 'my-sql-service'})
addService({ name: 'my-nosql-service'})
addService({ name: 'my-gitlab-api-service'})

I don't think something like this would make much sense either

const services = [...]
for (const service of services){
   addService(service)
}
Suggested change
public addServices(...services: (string | Service)[]) {
public addServices(...services: Service[]) {

src/gitlab/configuration.ts Outdated Show resolved Hide resolved
dontirun and others added 2 commits December 30, 2021 17:40
Co-authored-by: Christopher Rybicki <crybicki98@gmail.com>
@dontirun
Copy link
Contributor Author

dontirun commented Dec 30, 2021

Just a few more small changes and I think this PR is about ready to ship 👍

On further thought, I'm not totally sure what the best practices are here regarding leaving class properties public or private, since this class is both like an L1 (not a lot of opinionated defaults) but also like an L2 (it some provides convenience methods). I want to say it probably makes sense to make most of the properties private by default so that they can be changed as needed in the future and make the class more L2-like, but I don't have a strong opinion on this.

@eladb do you have any guidance here?

My thoughts here were to make this closer to a L1, but with some private properties (with convenience methods) to prevent the creation of CI files with invalid field configurations.

In the future I would like something closer to the GithubProject (or what comes out of this RFC) where standard jobs are created for the different workflows that projen uses for GitHub (Release, build, etc). That future GitlabProject would probably have more private properties as it would be closer to a L3

@eladb
Copy link
Contributor

eladb commented Jan 2, 2022

Please read this to avoid merge hell

Hola! apologies for hassle. I've enabled Prettify as part of #1453. This means that most of the source files have changed and you will likely see quite a lot of merge conflicts when you try to merge from main.

But there's an easy way to handle this.

First, merge your branch from this tag:

$ git fetch origin
$ git merge 1020d0ad3f95bb4e1201f8640d440ffb43ccf230

Edit the .projenrc.js file in your branch and add this to line 57 (right
after codeCov: true):

  prettier: true,

Now, run:

$ git clean -fdx .
$ yarn
$ bash projen.bash
$ yarn eslint
$ git add .
$ git commit -m 'format code'

This will reformat all the code in your branch to match main.

Now, merge from main:

$ git merge origin/main

Sorry again for the inconvenience...

@mergify mergify bot merged commit 2ef39f8 into projen:main Jan 2, 2022
mergify bot pushed a commit that referenced this pull request Apr 27, 2022
…nes (#1800)

#1357 added the types to build GitLab CI pipelines. The [base schema](https://gitlab.com/gitlab-org/gitlab/-/raw/master/app/assets/javascripts/editor/schema/ci.json) unfortunately broke the implementation of caching into two entities. In the [manual](https://docs.gitlab.com/ee/ci/yaml/#cache), this is only one entity.

In the current state, caching between pipeline runs is actually not very usable. You can only express when to cache, not what. 

This PR adds the types needed to generate cache statements according to the GitLab reference for [GitLab 14.10](https://docs.gitlab.com/14.10/ee/ci/yaml/).

---
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants