Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Existing output directory #660

Draft
wants to merge 21 commits into
base: develop
Choose a base branch
from
Draft

Existing output directory #660

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
fb91f20
Tests to skip module (MCCD PSF) if output file exists
martinkilbinger Sep 5, 2023
ac061c0
Merge remote-tracking branch 'upstream/develop' into output_exists
martinkilbinger Sep 17, 2023
a93546e
n_smp for more jobs; N_EPOCH bugs; ngmix checking for existing output
martinkilbinger Sep 17, 2023
1ed596c
numbering scheme with re pattern: copied, not changed
martinkilbinger Sep 24, 2023
ba4aa19
numbering scheme with re pattern: copied, not changed
martinkilbinger Sep 24, 2023
776943c
removed galsim from job script message
martinkilbinger Sep 24, 2023
3216eb6
Fixed import typo
martinkilbinger Oct 16, 2023
eeadfbf
ngmix template
martinkilbinger Oct 18, 2023
ee90dc1
testing openmpi 5.0.0 on candide
martinkilbinger Oct 28, 2023
22c2f56
Testing MPI on candide; errors with process list
martinkilbinger Oct 31, 2023
27e9bbb
Updated MPI setting and candide job
martinkilbinger Oct 31, 2023
c380cd4
ngmix runner reset to develop
martinkilbinger Oct 31, 2023
13940f5
ngmix script reset to develop
martinkilbinger Oct 31, 2023
0bec318
mccd and pysap dependencies added back in to example
martinkilbinger Oct 31, 2023
691daf3
submit run added missing arg
martinkilbinger Oct 31, 2023
3712734
config mpi
martinkilbinger Oct 31, 2023
ddf4091
removed debug prints
martinkilbinger Nov 6, 2023
3fb5a25
mpi4py upgraded to 3.1.5
martinkilbinger Nov 6, 2023
6a76454
changed warning print output
martinkilbinger Nov 6, 2023
7ac5042
ngmix_runner checked out from develop
martinkilbinger Nov 6, 2023
bfcdb78
Added testing of mmap file existence
martinkilbinger Nov 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ngmix_runner checked out from develop
  • Loading branch information
@martinkilbinger
martinkilbinger committed Nov 6, 2023
commit 7ac5042dc5328ec25f95ce1a35d402bd0228f90b
38 changes: 13 additions & 25 deletions shapepipe/modules/ngmix_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@

"""

import os

from shapepipe.modules.module_decorator import module_runner
from shapepipe.modules.ngmix_package.ngmix import Ngmix

Expand Down Expand Up @@ -54,31 +52,21 @@ def ngmix_runner(
id_obj_min = config.getint(module_config_sec, 'ID_OBJ_MIN')
id_obj_max = config.getint(module_config_sec, 'ID_OBJ_MAX')

output_path = (
f"{run_dirs['output']}/ngmix{file_number_string}.fits"
# Initialise class instance
ngmix_inst = Ngmix(
input_file_list,
run_dirs['output'],
file_number_string,
zero_point,
pixel_scale,
f_wcs_path,
w_log,
id_obj_min=id_obj_min,
id_obj_max=id_obj_max,
)
if os.path.exists(output_path):
w_log.info(
f"output file {output_path} already exists, skipping"
)
else:
# Initialise class instance
w_log.info(
f"Processing data for output file {output_path}"
)
ngmix_inst = Ngmix(
input_file_list,
output_path,
zero_point,
pixel_scale,
f_wcs_path,
w_log,
id_obj_min=id_obj_min,
id_obj_max=id_obj_max,
)

# Process ngmix shape measurement and metacalibration
ngmix_inst.process()
# Process ngmix shape measurement and metacalibration
ngmix_inst.process()

# No return objects
return None, None
Loading