Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] v4 uniquely named artifacts fail with 409 #481

Closed
schmidtw opened this issue Dec 18, 2023 · 4 comments
Closed

[bug] v4 uniquely named artifacts fail with 409 #481

schmidtw opened this issue Dec 18, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@schmidtw
Copy link

What happened?

When using a the new v4 version of this action, my workflow can no longer upload multiple results from multiple jobs, despite the artifacts having unique names.

In a prior step, we output the list of files in each job & they are all unique.
image

What did you expect to happen?

The files would upload until the limit of 10 is encountered.

How can we reproduce it?

I can't share the workflow since it is an GH hosted enterprise context, but I have a matrix that produces 10 different jobs and each of the 10 jobs produce 2 uniquely named files. Each job then does this:

      - name: Archive Results
        if: |
          always() &&
          inputs.dry-run == 'false'
        uses: actions/upload-artifact@v4
        with:
          name: Image Metadata
          path: |
            ${{ inputs.working-directory }}/images/*.txt
            ${{ inputs.working-directory }}/images/*.yml

All but the first job fail with a 409.

Anything else we need to know?

This worked fine in v3.1.3. This is running in a Github hosted enterprise context in case that makes any difference.

What version of the action are you using?

4.0.0

What are your runner environments?

self-hosted, linux

Are you on GitHub Enterprise Server? If so, what version?

No response

@schmidtw schmidtw added the bug Something isn't working label Dec 18, 2023
@schmidtw
Copy link
Author

I was able to recreate an example showing the problem I am running into here: https://github.com/schmidtw/upload-problem-example/actions/runs/7257051532

This happens regardless of OSS or Enterprise.

@konradpabjan
Copy link
Collaborator

@schmidtw

When using a the new v4 version of this action, my workflow can no longer upload multiple results from multiple jobs, despite the artifacts having unique names.

In your example the artifact names are not unique. Looking at the workflow file for https://github.com/schmidtw/upload-problem-example/actions/runs/7257051532/workflow

You have the following bit of code that runs for each job in the matrix

      - name: Archive Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: Image Metadata

Image Metadata is the name of artifact. This issue is that you have 8 jobs in your matrix and each one of them is uploading an artifact with the same name which is Image Metadata. Do the thing (x86_64, aws, rocky8, a, any, cd, na) for example is able to upload an artifact with the name Image Metadata because it is the first job that ran, but then the other 7 jobs try to do the same thing and it fails.

With v4 we call this out in the breaking changes that artifacts are scoped to a job so multiple jobs cannot upload to the same artifact: https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes

Due to how Artifacts are created in this new version, it is no longer possible to upload to the same named Artifact multiple times. You must either split the uploads into multiple Artifacts with different names, or only upload once. Otherwise you will encounter an error.

A bit more documentation here: https://github.com/actions/upload-artifact?tab=readme-ov-file#not-uploading-to-the-same-artifact

What you have to do to fix this is ensure that name is unique for each job.

${{ matrix.machine }}-${{ matrix.cloud }}-${{ matrix.region }}-${{ matrix.arch }}-${{ matrix.distro }}-${{ matrix.stage }}-${{ matrix.theater }}

      - name: Archive Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: Image Metadata ${{ matrix.machine }}-${{ matrix.cloud }}-${{ matrix.region }}-${{ matrix.arch }}-${{ matrix.distro }}-${{ matrix.stage }}-${{ matrix.theater }}

@Borda
Copy link

Borda commented Dec 19, 2023

In this case, when you want to restore past behavior (it was convenient and with execution in sequence, it was also safe), you would need to use GH CLI to download all artifacts on a loop and copy the content to a single/new artifact, but will all these artifacts available when the workflow has not finished yet...

@schmidtw
Copy link
Author

Thank you for the explanation. For some reason I didn't realize the name was not unique. Your suggestion works perfectly. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants