Fix #19110: Implement job to copy mis-placed translation images #21378

U8NWXD · 2024-12-04T23:33:17Z

Overview

This PR fixes [BUG]: Problems with image when translating lessons into Portugese #19110.
This PR does the following: Copies images referenced in translation suggestions from exploration/<exploration_id>/assets/image/ to exploration_suggestions/<exploration_id>/assets/image/ in GCS.
(For bug-fixing PRs only) The original bug occurred because: PR Fix #19110: Copy translation's target exploration images regardless if submitter uploaded new images or not #20188 caused images to be copied for new translation suggestions, but the job introduced in this PR is required to copy images for existing translation suggestions.

Essential Checklist

Please follow the instructions for making a code change.

I have linked the issue that this PR fixes in the "Development" section of the sidebar.
I have checked the "Files Changed" tab and confirmed that the changes are what I want to make.
I have written tests for my code.
The PR title starts with "Fix #bugnum: " or "Fix part of #bugnum: ...", followed by a short, clear summary of the changes.
I have assigned the correct reviewers to this PR (or will leave a comment with the phrase "@{{reviewer_username}} PTAL" if I can't assign them directly).

Testing doc (for PRs with Beam jobs that modify production server data)

A testing doc has been written: https://docs.google.com/document/d/1EnfjZIqUBMSkO1ZU1W6b_7owreX8Z5gqa2uLwCzxsHI/edit?tab=t.0
(To be confirmed by the server admin) All jobs in this PR have been verified on a live server, and the PR is safe to merge.

Proof that changes are correct

I manually verified that before #20188 was merged, translation suggestion images resided solely under exploration_suggestions/ in GCS. Therefore, I reproduced this state in a local development server as follows:

Log in as an admin user and grant yourself curriculum manager, release coordinator, and translation admin roles.
From the admin page, generate new structures.
Go to /create/25 to edit one of the newly-generated explorations. Add an image to the first card. Save and publish changes.
Log in as a non-admin user.
Go to the contributor dashboard and submit a translation suggestion for the card you added the image to ("Finding the place value of a number"). Make sure you copy the image in your translation.
In a terminal, open a shell to the oppia-dev-server Docker container and start an interactive Python session:
```
$ make shell.oppia-dev-server
$ docker exec -it oppia-dev-server /bin/sh
# python
```

View the images currently present:

>>> import scripts.start   # This sets up all the Python import paths correctly. May not be necessary.
>>> from core.platform import models
>>> storage_services = models.Registry.import_storage_services()
# The `abc` is the GCS bucket name, which you can put anything for.
>>> storage_services.listdir('abc', 'exploration_suggestions/25/assets')
# You may see slightly different image filenames depending on what image you use.
[EmulatorBlob(name=exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_micro.svg, content_type=image/svg+xml), EmulatorBlob(name=exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_compressed.svg, content_type=image/svg+xml), EmulatorBlob(name=exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314.svg, content_type=image/svg+xml)]
>>> storage_services.listdir('abc', 'exploration/25/assets')
# You may see slightly different image filenames depending on what image you use.
[EmulatorBlob(name=exploration/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_micro.svg, content_type=image/svg+xml), EmulatorBlob(name=exploration/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_compressed.svg, content_type=image/svg+xml), EmulatorBlob(name=exploration/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314.svg, content_type=image/svg+xml)]

Navigate to the contributor dashboard as the admin user and confirm that you can see the image:

onNewCommitPreMut.mov

Delete the images under exploration_suggestions/25/assets:

>>> storage_services.delete('abc', 'exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_micro.svg')
>>> storage_services.delete('abc', 'exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314_compressed.svg')
>>> storage_services.delete('abc', 'exploration_suggestions/25/assets/image/img_20241203_121203_tn09emjdm7_height_472_width_314.svg')
>>> storage_services.listdir('abc', 'exploration_suggestions/25/assets')
[]

Now we have a GCS state that the job can fix. On the contributor dashboard page, the admin user should see the problem described in #19110. Here is a screen recording showing how we can observe this problem, run the job introduced in this PR, and see that the problem has been resolved.

onNewCommitPostMut.mov

Further, we can use the Python shell to confirm that the image has been copied:

>>> storage_services.listdir('abc', 'exploration_suggestions/25/assets')
[EmulatorBlob(name=exploration_suggestions/25/assets/image/img_20241203_131426_nfvpcm6q2h_height_472_width_314.svg, content_type=image/svg+xml)]

PR Pointers

Never force push! If you do, your PR will be closed.
To reply to reviewers, follow these instructions: https://github.com/oppia/oppia/wiki/Make-a-pull-request#step-5-address-review-comments-until-all-reviewers-approve
Some e2e tests are flaky, and can fail for reasons unrelated to your PR. We are working on fixing this, but in the meantime, if you need to restart the tests, please check the "If your build fails" wiki page.
See the Code Owner's wiki page for what code owners will expect.

oppiabot · 2024-12-04T23:33:20Z

Hi @vojtechjelinek, @DubeySandeep, @kevintab95, PTAL at this PR, it modifies files in jobs or platform folders.
Also @U8NWXD, please make sure to fill in the server jobs form for the new job or feature to be tested on the backup server. This PR can be merged only after the test run is successful. Please refer to this guide for details.
Thanks!

oppiabot · 2024-12-04T23:33:30Z

Hi @U8NWXD please assign the required reviewer(s) for this PR. Thanks!

The as_stdout() and as_stderr() functions were annotated as only accepting str and int arguments, but they are designed (and frequently used) to process arbitrary input types. Therefore, they should actually accept inputs of type Any.

kevintab95

Leaving a review hold until this is run on the server.
/cc @Nik-09

oppiabot · 2024-12-09T05:11:58Z

Unassigning @kevintab95 since the review is done.

lkbhitesh07

Thanks @U8NWXD, LGTM!

oppiabot · 2024-12-12T01:38:35Z

Assigning @Vir-8 for code owner reviews. Thanks!

oppiabot · 2024-12-14T15:23:09Z

Unassigning @Vir-8 since they have already approved the PR.

U8NWXD · 2024-12-15T02:01:58Z

@kevintab95 the job has been submitted for testing

masterboy376 · 2024-12-17T15:45:21Z

Sorry @U8NWXD, I accidentally clicked the close button

Nik-09 · 2024-12-17T16:39:46Z

Sorry @U8NWXD, I accidentally clicked the close button

I will wait for the CI to pass completely, otherwise, the build for the release candidate will fail, after this, I will start the job run. Thanks.

U8NWXD · 2024-12-17T22:49:03Z

@Nik-09 FYI the CI checks have now passed. Thanks for handling the job testing!

Nik-09 · 2024-12-18T03:23:55Z

@Nik-09 FYI the CI checks have now passed. Thanks for handling the job testing!

Hi @U8NWXD I have started the process. Currently in the "Build release candidate" stage. Will keep you posted. Thanks.

Nik-09 · 2024-12-21T05:00:02Z

Hi @U8NWXD the audit job failed.

id: "2024-12-20_20_45_26-4950297230698149536"
project_id: "oppiaserver-backup-migration"
name: "beamapp-www-data-1221044519-974439-ly6sdhrg"
type_: JOB_TYPE_BATCH
environment {
user_agent {
fields {
key: "name"
value {
string_value: "Apache Beam Python 3.9 SDK"
}
}
fields {
key: "version"
value {
string_value: "2.43.0"
}
}
}
version {
fields {
key: "job_type"
value {
string_value: "FNAPI_BATCH"
}
}
fields {
key: "major"
value {
string_value: "8"
}
}
}
shuffle_mode: SERVICE_BASED
}
current_state: JOB_STATE_FAILED
current_state_time {
seconds: 1734756799
nanos: 159987000
}
create_time {
seconds: 1734756327
nanos: 191587000
}
client_request_id: "20241221044519975492-3606"
location: "us-central1"
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-close-shuffle26"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-UnKey_71.None"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "F76"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-shuffle-session64"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-open-shuffle18"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756343
nanos: 36000000
}
}
stage_states {
execution_stage_name: "F80"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756565
nanos: 313000000
}
}
stage_states {
execution_stage_name: "F74"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "F79"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756713
nanos: 897000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-open-shuffle10"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-shuffle-session17"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756343
nanos: 112000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-open-shuffle29"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "F81"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-shuffle-session9"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "FlushCache/CombineGlobally()/InjectDefault/View-python_side_input0-FlushCache/CombineGlobally()/InjectDefault"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-close-shuffle12"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-open-shuffle34"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756565
nanos: 606000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-shuffle-session28"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-shuffle-session23"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756714
nanos: 231000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-open-shuffle65"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "failure83"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756718
nanos: 400000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-close-shuffle31"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-shuffle-session33"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756565
nanos: 655000000
}
}
stage_states {
execution_stage_name: "F77"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-close-shuffle20"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756565
nanos: 443000000
}
}
stage_states {
execution_stage_name: "success82"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "start84"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756333
nanos: 864000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-close-shuffle36"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756713
nanos: 995000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-close-shuffle67"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "F75"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
stage_states {
execution_stage_name: "F78"
execution_stage_state: JOB_STATE_FAILED
current_state_time {
seconds: 1734756718
nanos: 324000000
}
}
stage_states {
execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-open-shuffle24"
execution_stage_state: JOB_STATE_DONE
current_state_time {
seconds: 1734756714
nanos: 184000000
}
}
stage_states {
execution_stage_name: "FlushCache/CombineGlobally()/InjectDefault/View-python_side_input0-FlushCache/CombineGlobally()/InjectDefault.out"
execution_stage_state: JOB_STATE_PENDING
current_state_time {
seconds: 1734756333
nanos: 756000000
}
}
job_metadata {
sdk_version {
version: "2.43.0"
version_display_name: "Apache Beam Python 3.9 SDK"
sdk_support_status: DEPRECATED
}
}
start_time {
seconds: 1734756327
nanos: 191587000
}

U8NWXD · 2024-12-21T20:48:32Z

As a note for the future, here are the logs with the formatting fixed:

id: "2024-12-20_20_45_26-4950297230698149536"
project_id: "oppiaserver-backup-migration"
name: "beamapp-www-data-1221044519-974439-ly6sdhrg"
type_: JOB_TYPE_BATCH
environment {
    user_agent {
        fields {
            key: "name"
            value {
                string_value: "Apache Beam Python 3.9 SDK"
            }
        }
        fields {
            key: "version"
            value {
                string_value: "2.43.0"
            }
        }
    }
    version {
        fields {
            key: "job_type"
            value {
                string_value: "FNAPI_BATCH"
            }
        }
        fields {
            key: "major"
            value {
                string_value: "8"
            }
        }
    }
    shuffle_mode: SERVICE_BASED
}
current_state: JOB_STATE_FAILED
current_state_time {
    seconds: 1734756799
    nanos: 159987000
}
create_time {
    seconds: 1734756327
    nanos: 191587000
}
client_request_id: "20241221044519975492-3606"
location: "us-central1"
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-close-shuffle26"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-UnKey_71.None"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "F76"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-shuffle-session64"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-open-shuffle18"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756343
        nanos: 36000000
    }
}
stage_states {
    execution_stage_name: "F80"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756565
        nanos: 313000000
    }
}
stage_states {
    execution_stage_name: "F74"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "F79"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756713
        nanos: 897000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-open-shuffle10"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-shuffle-session17"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756343
        nanos: 112000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-open-shuffle29"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "F81"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-shuffle-session9"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "FlushCache/CombineGlobally()/InjectDefault/View-python_side_input0-FlushCache/CombineGlobally()/InjectDefault"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_FlushCache-CombineGlobally-lambda-at-cache_io-py-49-CombinePerKey-GroupByKey_694-close-shuffle12"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-open-shuffle34"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756565
        nanos: 606000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-shuffle-session28"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-shuffle-session23"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756714
        nanos: 231000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-open-shuffle65"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "failure83"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756718
        nanos: 400000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-src-dst-str-src_exist-bool-dst_exist-bool-CoGrou_46-close-shuffle31"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-shuffle-session33"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756565
        nanos: 655000000
    }
}
stage_states {
    execution_stage_name: "F77"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Get-all-GeneralSuggestionModels-Reading-Query-kind-Genera_15-close-shuffle20"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756565
        nanos: 443000000
    }
}
stage_states {
    execution_stage_name: "success82"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "start84"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756333
        nanos: 864000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-by-suggestion-id-GroupByKey_23-close-shuffle36"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756713
        nanos: 995000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_PutResults-GroupIntoBatches-ParDo-_GroupIntoBatchesDoFn-_55-close-shuffle67"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "F75"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
stage_states {
    execution_stage_name: "F78"
    execution_stage_state: JOB_STATE_FAILED
    current_state_time {
        seconds: 1734756718
        nanos: 324000000
    }
}
stage_states {
    execution_stage_name: "ref_AppliedPTransform_Plan-copy-operations-Group-as-model_id-target_id-str-image_names-str-CoGroupBy_31-open-shuffle24"
    execution_stage_state: JOB_STATE_DONE
    current_state_time {
        seconds: 1734756714
        nanos: 184000000
    }
}
stage_states {
    execution_stage_name: "FlushCache/CombineGlobally()/InjectDefault/View-python_side_input0-FlushCache/CombineGlobally()/InjectDefault.out"
    execution_stage_state: JOB_STATE_PENDING
    current_state_time {
        seconds: 1734756333
        nanos: 756000000
    }
}
job_metadata {
    sdk_version {
        version: "2.43.0"
        version_display_name: "Apache Beam Python 3.9 SDK"
        sdk_support_status: DEPRECATED
    }
}
start_time {
    seconds: 1734756327
    nanos: 191587000
}

oppiabot · 2024-12-24T06:23:16Z

Hi @U8NWXD. Due to recent changes in the "develop" branch, this PR now has a merge conflict. Please follow this link if you need help resolving the conflict, so that the PR can be merged. Thanks!

oppiabot · 2024-12-24T06:23:19Z

Hi @U8NWXD. Due to recent changes in the "develop" branch, this PR now has a merge conflict. Please follow this link if you need help resolving the conflict, so that the PR can be merged. Thanks!

oppiabot · 2024-12-31T07:05:56Z

Hi @U8NWXD, I'm going to mark this PR as stale because it hasn't had any updates for 7 days. If no further activity occurs within 4 days, it will be automatically closed so that others can take up the issue.
If you are still working on this PR, please make a follow-up commit within 4 days (and submit it for review, if applicable). Please also let us know if you are stuck so we can help you!

Implement job to copy mis-placed translation images

522b872

oppiabot bot assigned DubeySandeep, kevintab95 and vojtechjelinek Dec 4, 2024

oppiabot bot added the PR: Affects datastore layer Labels to indicate a PR that changes the datastore layer. label Dec 4, 2024

oppiabot bot assigned U8NWXD Dec 4, 2024

U8NWXD unassigned vojtechjelinek, kevintab95 and DubeySandeep Dec 4, 2024

U8NWXD added 5 commits December 8, 2024 20:00

Implement job to copy mis-placed translation images

80c94f9

Fix type annotations in JobRunResult

2c9314e

The as_stdout() and as_stderr() functions were annotated as only accepting str and int arguments, but they are designed (and frequently used) to process arbitrary input types. Therefore, they should actually accept inputs of type Any.

Add tests

02c636b

Merge branch 'fix-19110' of github.com:U8NWXD/oppia into fix-19110

0219c12

Fix tests

fd26f37

U8NWXD marked this pull request as ready for review December 9, 2024 02:40

U8NWXD requested review from a team as code owners December 9, 2024 02:40

U8NWXD requested review from Vir-8 and lkbhitesh07 December 9, 2024 02:40

U8NWXD assigned vojtechjelinek, kevintab95, DubeySandeep and lkbhitesh07 Dec 9, 2024

seanlip unassigned vojtechjelinek and DubeySandeep Dec 9, 2024

kevintab95 requested changes Dec 9, 2024

View reviewed changes

oppiabot bot unassigned kevintab95 Dec 9, 2024

U8NWXD added 3 commits December 9, 2024 09:51

Fix job imports test

c5c8922

Separate debugging and audit jobs

f929aa1

Clean up validation logic

7d4ef07

lkbhitesh07 approved these changes Dec 12, 2024

View reviewed changes

lkbhitesh07 removed their assignment Dec 12, 2024

oppiabot bot assigned Vir-8 Dec 12, 2024

Merge branch 'develop' into fix-19110

ae64c53

U8NWXD removed their assignment Dec 12, 2024

Vir-8 approved these changes Dec 14, 2024

View reviewed changes

oppiabot bot unassigned Vir-8 Dec 14, 2024

masterboy376 closed this Dec 17, 2024

masterboy376 reopened this Dec 17, 2024

Nik-09 self-assigned this Dec 19, 2024

Nik-09 assigned U8NWXD and unassigned Nik-09 Dec 21, 2024

oppiabot bot added the PR: don't merge - HAS MERGE CONFLICTS label Dec 24, 2024

oppiabot bot added the stale label Dec 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #19110: Implement job to copy mis-placed translation images #21378

Fix #19110: Implement job to copy mis-placed translation images #21378

U8NWXD commented Dec 4, 2024 •

edited

Loading

oppiabot bot commented Dec 4, 2024

oppiabot bot commented Dec 4, 2024

kevintab95 left a comment

oppiabot bot commented Dec 9, 2024

lkbhitesh07 left a comment

oppiabot bot commented Dec 12, 2024

oppiabot bot commented Dec 14, 2024

U8NWXD commented Dec 15, 2024

masterboy376 commented Dec 17, 2024

Nik-09 commented Dec 17, 2024 •

edited

Loading

U8NWXD commented Dec 17, 2024

Nik-09 commented Dec 18, 2024

Nik-09 commented Dec 21, 2024

U8NWXD commented Dec 21, 2024 •

edited

Loading

oppiabot bot commented Dec 24, 2024

oppiabot bot commented Dec 24, 2024

oppiabot bot commented Dec 31, 2024

Fix #19110: Implement job to copy mis-placed translation images #21378

Are you sure you want to change the base?

Fix #19110: Implement job to copy mis-placed translation images #21378

Conversation

U8NWXD commented Dec 4, 2024 • edited Loading

Overview

Essential Checklist

Testing doc (for PRs with Beam jobs that modify production server data)

Proof that changes are correct

PR Pointers

oppiabot bot commented Dec 4, 2024

oppiabot bot commented Dec 4, 2024

kevintab95 left a comment

Choose a reason for hiding this comment

oppiabot bot commented Dec 9, 2024

lkbhitesh07 left a comment

Choose a reason for hiding this comment

oppiabot bot commented Dec 12, 2024

oppiabot bot commented Dec 14, 2024

U8NWXD commented Dec 15, 2024

masterboy376 commented Dec 17, 2024

Nik-09 commented Dec 17, 2024 • edited Loading

U8NWXD commented Dec 17, 2024

Nik-09 commented Dec 18, 2024

Nik-09 commented Dec 21, 2024

U8NWXD commented Dec 21, 2024 • edited Loading

oppiabot bot commented Dec 24, 2024

oppiabot bot commented Dec 24, 2024

oppiabot bot commented Dec 31, 2024

U8NWXD commented Dec 4, 2024 •

edited

Loading

Nik-09 commented Dec 17, 2024 •

edited

Loading

U8NWXD commented Dec 21, 2024 •

edited

Loading