Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: read the release assets asynchronously #552

Merged
merged 1 commit into from
Dec 11, 2024

Conversation

xen0n
Copy link
Contributor

@xen0n xen0n commented Dec 6, 2024

Previously all assets were being read synchronously into memory, making the action unsuitable for releasing very large assets. Because the client library allows stream body inputs (it just forwards it to the underlying fetch implementation), just do it.

Fixes: #353

@xen0n
Copy link
Contributor Author

xen0n commented Dec 6, 2024

Note: I haven't tested this change myself, because my workflow takes 2 hours to finish and I have just finished one run, so I'm just uploading my 8GiB of assets manually this time to fix the release. Testing is welcome.

Previously all assets were being read synchronously into memory, making
the action unsuitable for releasing very large assets. Because the
client library allows stream body inputs (it just forwards it to the
underlying `fetch` implementation), just do it.

The idea is also suggested by @enumag in
softprops#353 (comment).

Fixes: softprops#353
Signed-off-by: WANG Xuerui <git@xen0n.name>
Copy link
Collaborator

@chenrui333 chenrui333 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@chenrui333 chenrui333 merged commit 64f1fa1 into softprops:master Dec 11, 2024
1 check passed
@chenrui333
Copy link
Collaborator

@xen0n thanks a lot for your contribution!

@xen0n xen0n deleted the async branch December 11, 2024 03:21
@pattacini
Copy link

pattacini commented Dec 19, 2024

Unfortunately, this PR doesn't solve the problem reported in #353 (comment), at least in my case.

I was trying to upload a bunch of big files using globbing and the action failed with this fatal error:

##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
##[error]The operation was canceled.
Here's my action, in case it helps:
name: Create Release

on: 
  workflow_dispatch:
    inputs:
      name:
        description: 'Release Name'
        required: true
        default: 'Project JL2 Camozzi Release v0.0.0'
      body:
        description: 'Release Body Message'
        required: true
        default: 'The asset provided by this release contains the whole material stored in the repository.'
      tag:
        description: 'Release Tag'
        required: true
        default: 'v0.0.0'
      zip_size:
        description: 'Max size of zip files in GB'
        required: true
        default: '1'

env:
  DIR_PREFIX: project-jl2-camozzi

jobs:
  doRelease:
    name: "Release"
    runs-on: ubuntu-latest
    environment: release_environment

    steps:
      - name: Release Inputs
        run: |
          echo "Release Inputs:"
          echo "- Name: ${{ inputs.name }}" 
          echo "- Body: ${{ inputs.body }}" 
          echo "- Tag: ${{ inputs.tag }}" 
          echo "- Zip Size: ${{ inputs.zip_size }}g"

      - name: Clone Branch master
        uses: actions/checkout@main
        with:
          ref: master
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/master
          lfs: true

      - name: Clone Branch mech
        uses: actions/checkout@main
        with:
          ref: mech
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/mech
          lfs: true

      - name: Clone Branch sim
        uses: actions/checkout@main
        with:
          ref: sim
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/sim
          lfs: true

      - name: Clone Branch code
        uses: actions/checkout@main
        with:
          ref: code
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/code
          lfs: true

      - name: Clone Branch astrial/code
        uses: actions/checkout@main
        with:
          ref: astrial/code
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/astrial/code
          lfs: true
          
      - name: Clone Branch astrial/hailo-dev
        uses: actions/checkout@main
        with:
          ref: astrial/hailo-dev
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/astrial/hailo-dev
          lfs: true 

      - name: Clone Branch firmware
        uses: actions/checkout@main
        with:
          ref: firmware
          path: ${{ env.DIR_PREFIX }}-${{ inputs.tag }}/firmware
          lfs: true

      - name: Configure Git
        run: |
          git config --global push.default upstream
          git config --global user.name "GitHub Actions"
          git config --global user.email "actions@github.com"

      - name: Tag Branches
        run: |
          declare -a branches=("master" "mech" "sim" "code" "astrial/code" "astrial/hailo-dev" "firmware")
          for branch in ${branches[@]}; do
            echo "tagging ${branch}..."
            cd ${GITHUB_WORKSPACE}/${DIR_PREFIX}-${{ inputs.tag }}/${branch}
            git tag -a ${branch}-${{ inputs.tag }} -m "version ${{ inputs.tag }}"
            git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git
            git push origin --tags
            rm -Rf .git
          done

      - name: Create Assets
        run: |
          zip -r -s ${{ inputs.zip_size }}g ${DIR_PREFIX}-${{ inputs.tag }}.zip ${DIR_PREFIX}-${{ inputs.tag }}/
          
      - name: Create Release
        uses: softprops/action-gh-release@v2
        with:
          tag_name: master-${{ inputs.tag }}
          name: ${{ inputs.name}}
          body: ${{ inputs.body }}
          generate_release_notes: false
          make_latest: true
          files: |
            ${{ env.DIR_PREFIX }}-${{ inputs.tag }}.*

cc @xen0n @chenrui333

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The runner has received a shutdown signal
3 participants