Bug fix to correct job log returining and other edge cases #683

ddimatos · 2023-03-23T20:45:12Z

SUMMARY

Main changes were to add back the joblog for failed job submissions and extend the modules tolerance to certain edge cases which are documented and results shown in each of the tickets.

These are grouped as one PR because they overlap in code and also are localized to one module.

Addresses:

[Bug] [zos_job_submit] Submitted local invalid JCL and returned with a stack trace #623
[Bug] zos_jobs_submit failed without any explanation when having a JCL Error #624
[Bug] [zos_job_submit] When a job that does not succeed for some conditions the job log is not returned #655
[Bug] [zos_job_submit] failed with wait_time_s parameter set to 0 #670
[Bug] [zos_job_submit] Jobs with a JCL error which get purged as a result end up with not helpful error #681
[Bug] [zos_job_submit] failed with wait_time_s parameter set to 0 #670
[Bug] [zos_job_submit] JCL submitted that results in a security exception returns no job log. #684
[Bug] [zos_job_submit] When TYPRUN=SCAN is added to a job the response uses the full wait_time_s #685

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

…g nearly always Signed-off-by: ddimatos <dimatos@gmail.com>

Signed-off-by: ddimatos <dimatos@gmail.com>

ddimatos · 2023-03-25T06:18:39Z

Log suffix: view/AC/job/zosAnsible/1531/console

…-log

plugins/modules/zos_job_submit.py

richp405

Other than a small grammatical issue on 685 of zos_job_submit, this looks fine.

Grammar change and correction.

* Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com>

* Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com>

* Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * update zoau version checker and add unit testing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Bugfix/769/1.6/zos copy does not overwrite permission on overwriten copy as comunity module (#790) * Change function behaivour, variables names and add test case to ensure result * Change test assertion to ensure wokrs as ansible module * Change test for the HSF * Add fragment * More readable test and better coments * add changelog fragment for zoau version checker bugfix (#800) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Add ansible-lint tooling added (#813) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: ac galaxy.yml meta/runtime.yml Changes to be committed: new file: .ansible-lint new file: ac modified: galaxy.yml modified: meta/runtime.yml * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Removing file brought it in by cherry pick Signed-off-by: ddimatos <dimatos@gmail.com> * Added missing keyword 'build_ignore' Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix for #807 - zos_copy ignores encoding for binary files (#810) * Updated normalization condition * Added test cases for bugfix * Added changelog fragment * Updated changelog fragment * Update zos_data_set member description created (#816) * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes the issue of parts of a vsam cluster remaining behind and allows user to correctly delete DS not in cat Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volume Signed-off-by: ddimatos <dimatos@gmail.com> * Unbound local var fix Signed-off-by: ddimatos <dimatos@gmail.com> * Lint corrections Signed-off-by: ddimatos <dimatos@gmail.com> * remove unused imports Signed-off-by: ddimatos <dimatos@gmail.com> * Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volumegit Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update grammar issue Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#814) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: deleted: tests/functional/modules/test_module_display.py modified: tests/functional/modules/test_zos_copy_func.py * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: modified: tests/functional/modules/test_zos_copy_func.py * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: modified: tests/functional/modules/test_zos_copy_func.py * changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment change Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Release tasks v1.6.0 (#829) * Galaxy for 1.6 Signed-off-by: ddimatos <dimatos@gmail.com> * Update ansible-core meta runtime Signed-off-by: ddimatos <dimatos@gmail.com> * Update collections private meta Signed-off-by: ddimatos <dimatos@gmail.com> * Missing trailing empty line Signed-off-by: ddimatos <dimatos@gmail.com> * Update readme with volume init info Signed-off-by: ddimatos <dimatos@gmail.com> * Update lint to match galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog summary Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog Signed-off-by: ddimatos <dimatos@gmail.com> * Minor manual CHANGELOG updates Signed-off-by: ddimatos <dimatos@gmail.com> * Relase notes updated Signed-off-by: ddimatos <dimatos@gmail.com> * Update build_ignore to skip importer_result.json Signed-off-by: ddimatos <dimatos@gmail.com> * update galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> * update galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up changelog fragements no longer need post releasing 1.6 Signed-off-by: ddimatos <dimatos@gmail.com> * delete older profile, not needed Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com>

* Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan…

* Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volum…

* Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy d…

* Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *non…

* Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository template updates and future proofing Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Formatting corrections for release notes Signed-off-by: ddimatos <dimatos@gmail.com> * Upate issue templates with newer version of software Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command supporting files (#789) * Update ac command supporting files Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Add recently changed module doc from prior commits Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Encode files recursively and test case for keep behavior. (#772) * Bring the jinja2 solution to dev and add test case * Add fragment * Solve problem z/OS 2.5 HFS * Declaration error solve * Need to check the validation with HFS * Ensure validating z/OS work with HFS * Change inecesary changes and fragments q * Return all test cases to normal * Return all test cases to normal * Create the local test case * Add local test case and change test case to be acurate * Get better cleanup of test-case * Update test_zos_data_set_func.py Equalize test mount func * Update ac to support a single test (#793) * Update ac to support a single test Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update test description Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Return the dynamically created destination attributes (#773) * First iteration to get dynamic values * Spaces and lines rectified * Add validation and extra variable to ensure consistency * Whitespaces * Change imports in test_zos_mount_func * Update test_zos_fetch_func imports * Update all imports for pipelines runs * Revert "Update all imports for pipelines runs" This reverts commit 1b370a2ba3c0001c316e0121ddab82ae7cc6d75d. Return one commit * Update data_set.py imports * Revert "Update data_set.py imports" This reverts commit 37561b0a12e04faaee8307a5541b71469dbe721d. * Update data_set imports * Update data_set imports * Update data_set imports * Restore import * Restore the imports * Add fragment * Solve a typo * Solve z/OS 2.5 HFS * Solve declaration error * Solve HFS and solution by now * Ensure HFS working with HFS * Better working on HFS testing problems * Change to cover many cases and add test * Modified changelog, corrected typos and shortemed file name * Delete 773-Return-the-dynamically-created-destintation-attributres.yaml * Update test_zos_data_set_func.py * Add documentation * Adjust spaces * Solve spaces in documentation * Solve problems on spaces in documentation * Adjust fragment and add validation for vsams * Better redaction to documentation * Solve spaces * Change documentation of code and collection * Change words in documentation --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Updated ac command to clean up the collections directory Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes the issue of parts of a vsam cluster remaining behind and allows user to correctly delete DS not in cat Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volume Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volumegit Signed-off-by: ddimatos <dimatos@gmail.com> * Unbound local var fix Signed-off-by: ddimatos <dimatos@gmail.com> * added changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Lint corrections Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog based on PR feedback Signed-off-by: ddimatos <dimatos@gmail.com> * Increase ansible supported version to 2.15 Signed-off-by: ddimatos <dimatos@gmail.com> * remove unused imports Signed-off-by: ddimatos <dimatos@gmail.com> * Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Change the line for the functional one (#805) * Add ansible-lint tooling added (#812) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> * add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * missing hyphen from command doc Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command with doc corrections Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 439 addf (#821) * initial changes to support F format * adding F option, added basic test looking for failure during ensure-present * added print_results to a failing uncatalog test. * adding more preint_result statements to track down cataloging issue * removed other print statements, added one back (cataloging is just plain finicky) * corrected volume name on new test * removed extra print statement from test code. Added Changelog fragment. * Expanded test case to try 1 of each record format creation. Added mention of 'F' into the documentation of record_format in dataset.py * Bugfix/769/mode option does not behave the same way that it does in the community module (#795) * First suggestion * Add files to be overwriten to the files to be changed * Add functionality to test case to ensure behaivour * Add test case for keep behaivour * Delete test repetition * Delete test case from other branch * Change test cases to ensure works as ansible module * Add fragment and change variable names for clarity * Get better test case and comments * Restore test --------- Co-authored-by: Demetri <dimatos@gmail.com> * bugfix/823/Return destination attributes had hardcoded type and record format (#824) * Add solution * Add fragment * Bufix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#806) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Changlog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> * remove verbosity from test Signed-off-by: ddimatos <dimatos@gmail.com> * correct indentation Signed-off-by: ddimatos <dimatos@gmail.com> * update changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Stagging v1.6.0 merge into dev (#832) * Update branch produc…

* Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository template updates and future proofing Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Formatting corrections for release notes Signed-off-by: ddimatos <dimatos@gmail.com> * Upate issue templates with newer version of software Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command supporting files (#789) * Update ac command supporting files Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Add recently changed module doc from prior commits Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Encode files recursively and test case for keep behavior. (#772) * Bring the jinja2 solution to dev and add test case * Add fragment * Solve problem z/OS 2.5 HFS * Declaration error solve * Need to check the validation with HFS * Ensure validating z/OS work with HFS * Change inecesary changes and fragments q * Return all test cases to normal * Return all test cases to normal * Create the local test case * Add local test case and change test case to be acurate * Get better cleanup of test-case * Update test_zos_data_set_func.py Equalize test mount func * Update ac to support a single test (#793) * Update ac to support a single test Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update test description Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Return the dynamically created destination attributes (#773) * First iteration to get dynamic values * Spaces and lines rectified * Add validation and extra variable to ensure consistency * Whitespaces * Change imports in test_zos_mount_func * Update test_zos_fetch_func imports * Update all imports for pipelines runs * Revert "Update all imports for pipelines runs" This reverts commit 1b370a2ba3c0001c316e0121ddab82ae7cc6d75d. Return one commit * Update data_set.py imports * Revert "Update data_set.py imports" This reverts commit 37561b0a12e04faaee8307a5541b71469dbe721d. * Update data_set imports * Update data_set imports * Update data_set imports * Restore import * Restore the imports * Add fragment * Solve a typo * Solve z/OS 2.5 HFS * Solve declaration error * Solve HFS and solution by now * Ensure HFS working with HFS * Better working on HFS testing problems * Change to cover many cases and add test * Modified changelog, corrected typos and shortemed file name * Delete 773-Return-the-dynamically-created-destintation-attributres.yaml * Update test_zos_data_set_func.py * Add documentation * Adjust spaces * Solve spaces in documentation * Solve problems on spaces in documentation * Adjust fragment and add validation for vsams * Better redaction to documentation * Solve spaces * Change documentation of code and collection * Change words in documentation --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Updated ac command to clean up the collections directory Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes the issue of parts of a vsam cluster remaining behind and allows user to correctly delete DS not in cat Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volume Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volumegit Signed-off-by: ddimatos <dimatos@gmail.com> * Unbound local var fix Signed-off-by: ddimatos <dimatos@gmail.com> * added changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Lint corrections Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog based on PR feedback Signed-off-by: ddimatos <dimatos@gmail.com> * Increase ansible supported version to 2.15 Signed-off-by: ddimatos <dimatos@gmail.com> * remove unused imports Signed-off-by: ddimatos <dimatos@gmail.com> * Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Change the line for the functional one (#805) * Add ansible-lint tooling added (#812) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> * add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * missing hyphen from command doc Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command with doc corrections Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 439 addf (#821) * initial changes to support F format * adding F option, added basic test looking for failure during ensure-present * added print_results to a failing uncatalog test. * adding more preint_result statements to track down cataloging issue * removed other print statements, added one back (cataloging is just plain finicky) * corrected volume name on new test * removed extra print statement from test code. Added Changelog fragment. * Expanded test case to try 1 of each record format creation. Added mention of 'F' into the documentation of record_format in dataset.py * Bugfix/769/mode option does not behave the same way that it does in the community module (#795) * First suggestion * Add files to be overwriten to the files to be changed * Add functionality to test case to ensure behaivour * Add test case for keep behaivour * Delete test repetition * Delete test case from other branch * Change test cases to ensure works as ansible module * Add fragment and change variable names for clarity * Get better test case and comments * Restore test --------- Co-authored-by: Demetri <dimatos@gmail.com> * bugfix/823/Return destination attributes had hardcoded type and record format (#824) * Add solution * Add fragment * Bufix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#806) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Changlog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> * remove verbosity from test Signed-off-by: ddimatos <dimatos@gmail.com> * correct indentation Signed-off-by: ddimatos <dimatos@gmail.com> * update changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Stagging v1.6.0 merge into dev (#832) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstrin…

* Merge Staging release v1.9.0 beta.1 into main (#1205) * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository template updates and future proofing Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Formatting corrections for release notes Signed-off-by: ddimatos <dimatos@gmail.com> * Upate issue templates with newer version of software Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command supporting files (#789) * Update ac command supporting files Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Add recently changed module doc from prior commits Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Encode files recursively and test case for keep behavior. (#772) * Bring the jinja2 solution to dev and add test case * Add fragment * Solve problem z/OS 2.5 HFS * Declaration error solve * Need to check the validation with HFS * Ensure validating z/OS work with HFS * Change inecesary changes and fragments q * Return all test cases to normal * Return all test cases to normal * Create the local test case * Add local test case and change test case to be acurate * Get better cleanup of test-case * Update test_zos_data_set_func.py Equalize test mount func * Update ac to support a single test (#793) * Update ac to support a single test Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update test description Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Return the dynamically created destination attributes (#773) * First iteration to get dynamic values * Spaces and lines rectified * Add validation and extra variable to ensure consistency * Whitespaces * Change imports in test_zos_mount_func * Update test_zos_fetch_func imports * Update all imports for pipelines runs * Revert "Update all imports for pipelines runs" This reverts commit 1b370a2ba3c0001c316e0121ddab82ae7cc6d75d. Return one commit * Update data_set.py imports * Revert "Update data_set.py imports" This reverts commit 37561b0a12e04faaee8307a5541b71469dbe721d. * Update data_set imports * Update data_set imports * Update data_set imports * Restore import * Restore the imports * Add fragment * Solve a typo * Solve z/OS 2.5 HFS * Solve declaration error * Solve HFS and solution by now * Ensure HFS working with HFS * Better working on HFS testing problems * Change to cover many cases and add test * Modified changelog, corrected typos and shortemed file name * Delete 773-Return-the-dynamically-created-destintation-attributres.yaml * Update test_zos_data_set_func.py * Add documentation * Adjust spaces * Solve spaces in documentation * Solve problems on spaces in documentation * Adjust fragment and add validation for vsams * Better redaction to documentation * Solve spaces * Change documentation of code and collection * Change words in documentation --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Updated ac command to clean up the collections directory Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes the issue of parts of a vsam cluster remaining behind and allows user to correctly delete DS not in cat Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volume Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volumegit Signed-off-by: ddimatos <dimatos@gmail.com> * Unbound local var fix Signed-off-by: ddimatos <dimatos@gmail.com> * added changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Lint corrections Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog based on PR feedback Signed-off-by: ddimatos <dimatos@gmail.com> * Increase ansible supported version to 2.15 Signed-off-by: ddimatos <dimatos@gmail.com> * remove unused imports Signed-off-by: ddimatos <dimatos@gmail.com> * Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Change the line for the functional one (#805) * Add ansible-lint tooling added (#812) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> * add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * missing hyphen from command doc Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command with doc corrections Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 439 addf (#821) * initial changes to support F format * adding F option, added basic test looking for failure during ensure-present * added print_results to a failing uncatalog test. * adding more preint_result statements to track down cataloging issue * removed other print statements, added one back (cataloging is just plain finicky) * corrected volume name on new test * removed extra print statement from test code. Added Changelog fragment. * Expanded test case to try 1 of each record format creation. Added mention of 'F' into the documentation of record_format in dataset.py * Bugfix/769/mode option does not behave the same way that it does in the community module (#795) * First suggestion * Add files to be overwriten to the files to be changed * Add functionality to test case to ensure behaivour * Add test case for keep behaivour * Delete test repetition * Delete test case from other branch * Change test cases to ensure works as ansible module * Add fragment and change variable names for clarity * Get better test case and comments * Restore test --------- Co-authored-by: Demetri <dimatos@gmail.com> * bugfix/823/Return destination attributes had hardcoded type and record format (#824) * Add solution * Add fragment * Bufix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#806) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Changlog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> * remove verbosity from test Signed-off-by: ddimatos <dimatos@gmail.com> * correct indentation Signed-off-by: ddimatos <dimatos@gmail.com> * update changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Stagging v1.6.0 merge into dev (#832) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_…

* Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Change the line for the functional one (#805) * Add ansible-lint tooling added (#812) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> * add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * missing hyphen from command doc Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command with doc corrections Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 439 addf (#821) * initial changes to support F format * adding F option, added basic test looking for failure during ensure-present * added print_results to a failing uncatalog test. * adding more preint_result statements to track down cataloging issue * removed other print statements, added one back (cataloging is just plain finicky) * corrected volume name on new test * removed extra print statement from test code. Added Changelog fragment. * Expanded test case to try 1 of each record format creation. Added mention of 'F' into the documentation of record_format in dataset.py * Bugfix/769/mode option does not behave the same way that it does in the community module (#795) * First suggestion * Add files to be overwriten to the files to be changed * Add functionality to test case to ensure behaivour * Add test case for keep behaivour * Delete test repetition * Delete test case from other branch * Change test cases to ensure works as ansible module * Add fragment and change variable names for clarity * Get better test case and comments * Restore test --------- Co-authored-by: Demetri <dimatos@gmail.com> * bugfix/823/Return destination attributes had hardcoded type and record format (#824) * Add solution * Add fragment * Bufix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#806) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Changlog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> * remove verbosity from test Signed-off-by: ddimatos <dimatos@gmail.com> * correct indentation Signed-off-by: ddimatos <dimatos@gmail.com> * update changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Stagging v1.6.0 merge into dev (#832) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * update zoau version checker and add unit testing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Bugfix/769/1.6/zos copy does not overwrite permission on overwriten copy as comunity module (#790) * Change function behaivour, variables names and add test case to ensure result * Change test assertion to ensure wokrs as ansible module * Change test for the HSF * Add fragment * More readable test and better coments * add changelog fragment for zoau version checker bugfix (#800) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Add ansible-lint tooling added (#813) * Add ansible-lint tooling Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: ac galaxy.yml meta/runtime.yml Changes to be committed: new file: .ansible-lint new file: ac modified: galaxy.yml modified: meta/runtime.yml * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Removing file brought it in by cherry pick Signed-off-by: ddimatos <dimatos@gmail.com> * Added missing keyword 'build_ignore' Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix for #807 - zos_copy ignores encoding for binary files (#810) * Updated normalization condition * Added test cases for bugfix * Added changelog fragment * Updated changelog fragment * Update zos_data_set member description created (#816) * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes the issue of parts of a vsam cluster remaining behind and allows user to correctly delete DS not in cat Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volume Signed-off-by: ddimatos <dimatos@gmail.com> * Unbound local var fix Signed-off-by: ddimatos <dimatos@gmail.com> * Lint corrections Signed-off-by: ddimatos <dimatos@gmail.com> * remove unused imports Signed-off-by: ddimatos <dimatos@gmail.com> * Added 2.16 ignore since our pipeline supports devel which is at this time 2.16 Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to explain data set deltion for given volumegit Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update grammar issue Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix: Fixes zos_copy and zos_fetch deprecation msg for using _play_context.verbosity (#814) * Add new test case for verbosity check Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_copy tests to support display.verbosity and nested encoding Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: deleted: tests/functional/modules/test_module_display.py modified: tests/functional/modules/test_zos_copy_func.py * Update test framewor to provide support for adhoc module calls Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_fetch plugin to use the display.verbosity Signed-off-by: ddimatos <dimatos@gmail.com> * Lint correction Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with verbosity Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: modified: tests/functional/modules/test_zos_copy_func.py * Change from shell to raw module usage Signed-off-by: ddimatos <dimatos@gmail.com> Conflicts: tests/functional/modules/test_zos_copy_func.py Changes to be committed: modified: tests/functional/modules/test_zos_copy_func.py * changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment change Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Release tasks v1.6.0 (#829) * Galaxy for 1.6 Signed-off-by: ddimatos <dimatos@gmail.com> * Update ansible-core meta runtime Signed-off-by: ddimatos <dimatos@gmail.com> * Update collections private meta Signed-off-by: ddimatos <dimatos@gmail.com> * Missing trailing empty line Signed-off-by: ddimatos <dimatos@gmail.com> * Update readme with volume init info Signed-off-by: ddimatos <dimatos@gmail.com> * Update lint to match galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog summary Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog Signed-off-by: ddimatos <dimatos@gmail.com> * Minor manual CHANGELOG updates Signed-off-by: ddimatos <dimatos@gmail.com> * Relase notes updated Signed-off-by: ddimatos <dimatos@gmail.com> * Update build_ignore to skip importer_result.json Signed-off-by: ddimatos <dimatos@gmail.com> * update galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> * update galaxy build_ignore Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up changelog fragements no longer need post releasing 1.6 Signed-off-by: ddimatos <dimatos@gmail.com> * delete older profile, not needed Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bug fix, zos_copy returns an error message when a concurrent copy fails (#794) * Test case to validate bug does not happen * First iteration for solutions * First proposal to validate_disposition * Remove unecesary test * Solvin unecesary code * Cover all cases with bug or false positive * Add test case to ensure behaviour * Get the better version of test case * Add fragment * Solve identation * Solve identation * Solve identation * Solve error in cleanup folders * Change function name * Change variables names * Solve wrote and write * Update changelog entry Signed-off-by: ddimatos <dimatos@gmail.com> * Better verbose and function name * Better message * Solve certification tests * Clearer and eficient version * continuation line over-indented solve * continuation line over-indented solve --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Enhancement/428/jinja2 template support (#667) * Added Jinja2 support for single files in zos_copy * Fixed cleanup of rendered templates * Added templating for directories * Fixed encoding conversion for directories (#616) While working on adding Jinja2 support for directories, I noticed that files that were in subdirectories would show garbage in a shell, which reminded me of issue #616. After implementing their suggestion, files now show the rendered templates. * Disabled rendering for remote sources * Enhanced exception handling for templates * Added encoding normalization for templates * Added templating tests to zos_copy * Added templating support to zos_job_submit * Fixed missing newline * Added failure when using templates with remote sources * Fixed encoding normalization * Fixed sanity tests issues * Added Jinja2 tests to zos_job_submit * Updated template test for zos_job_submit * Fixed template environment creation * Refactored template environment creation * Fixed initial size computation for local files * Fixed merge mistakes * Updated description for trim_blocks * Updated docs * Added test for Jinja2 loops * Added changelog fragment * Removed duplicated function * Changed try-except blocks used in action plugins * Updated template docs with variable precedence * Changed dictionary update in action plugin * Added another encoding for template tests * Fixed import errors in template.py * Fixed import error in Python 2.7 * Fixed bandit issue * Fixed template cleanup * Updated documentation --------- Co-authored-by: Demetri <dimatos@gmail.com> * Module zos_unarchive & zos_archive (#755) * Initial commit * Created template for zos_unarchive * Initial boiler plate for unarchive * Added unarchive handler selection * Added file/data set search * Adding AMATERSE unpacking * Added support for terse unarchive * Remove zos_archive module from the branch * Adding RECEIVE/XMIT support * Added temporary dataset removal * Adding RECEIVE as an mvscmd * Add RECEIVE using mvs_cmd * Add unpacked datasets display * Added display of unpacked targets support for multiple data sets * Added alias to options * Added include/ exclude options * Added include for xmit and terse * Modified include for all * Adding volume selection for SMS managed * Added list support for MVS archives * Removed unused var * Add force support for mvs data sets * Add archive listing for tar, bz2, and gz * Add unarchive all for tar, gz and bz2 * Added include/exclude support for tar, gz, bz2 * Add mvs_cmd amaterse * Modify mvs_cmd call * Add archive.py * Add latest zos_archive * Refactor mvs_cmd * Remove comments * Adding tests for zos_archive * Added unzip * Added arcroot logic and tarfile * Added changed logic * lastest changes * Multiple changes to zos_archive and zos_unarchive and its tests * Added support for pax * added list tests * Created action plugin for zos_unarchive * Add support for remove * Adding tests for exclusion list USS * Added tests * Add dest_data_set info * Adding multiple test cases and updated docs * Added test for mode * Removed unused stuff * Modified XMIT command * Added expanded paths support for mvs data sets * Added ugly multiple data set tests * Added various new tests for mvs data sets * Added new tests and default dest data set * Added default dest data set * Fixed pep8 issues * Added docs * Added docs * Fixed various sanity issues * removed abc.ABC * Added filters * modified fdict * polish test cases * Added tests * Added record length change * Fixed record length error * fix pylint issue * Add env cleaning in when terse or xmit fails * Moved locking tests below * Added tests for multiple data set in unarchive and modified test archive and unarchive names * Added tests for zos_unarchive * Added replace into zos_data_set invocation * Added replace * Added docs * Added * Added docs * Cleaned docs * Added permission change in zos_unarchive * Added mode support for USS files * Added is_binary option * Added zos_archive and zos_unarchive rst * Updated docs * Updated docs * Updated docs * Removed debug statements * fix pylint issues * Added get restored data sets * Removed replace flag * Added lock process removal * Removed print statements and blank lines * Removed print statements * Removed unused imports * added missing test * Completed uncompleted doc string * Fixed a bug that removed the source when not needed * Fixed pep8 issue * Added removal in correct place * fixed a bug that left behind tmp data sets * Added changes for deleting tmp data sets + pr code review comments * Added a notes and seealso section in the docs * Changed name to xmit_log_data_set * Added comments * Added comments about format * Added more description in terse_pack * Added mode description * Added description for tmp_hlq * Added description for remove * Added 'a' into some statements * Modified dest_state description * Updated option descriptions * Changed badzip error msg * changed adrdssu multiple data set error message * Added tso xmit error message * changed adrdssu multiple data set error message * modified description msg * Updated path to src and changed multiple docs * Fixed module doc issues * Removed unused error and modified error messages * Changed parameter result get * Updated docs in zos_unarchive * Updates docs * Updated docs * Modified mode * Changed src in action * Updated include docs * Updated include docs * Updated docs * Enhanced is_binary support * Fixed dest_data_set issue * Changed docs * Added changes to compute space primary * Added changes to compute space primary * Changed binary test case * Updated docs and test cases * Added xmit_log_data_set docs * Modified dest description * Updated docs * Updated docs * Updated docs * Updated docs * Updated docs * Updated docs * Updated docs * Updated docs * Updaded expanded sources * Updated docs * Updated docs * Updated docs * Updated docs and added dest_data_set * Added dest_data_set * Added primary_space * Fixed pep8 * updated dest_volumes comments * added xmit_log_data_set docs * Updated docs about mode * Added is binary comment * Removed is_binary * Removed is_binary from source * Updated targets list * Updated force docs * Updated the force docs * Updated exclude docs * Updated message for dest in zos_archive * Changed dest to required: true * Removed exclusion_patterns * Updated group in zos_archive * Corrected mode * Updated wording in owner * Updated src * Updated docs * Updated template * Fixed sanity issues * Updated PDS/PDSE to mayus * fixed mem creation * Fixed space type issue * Modifed test for des_data_set * fixed test * Improved dest_data_set * updated docs * updated dest_data_set type * Added latest docs * Removed emergency backup and auto-recovery features. (#896) * Removed emergency backup and auto-recovery features. Initial changelog fragment. * corrected changelog with PR # * Update changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> * removed 2 unused routines: restore_backup and erase_backup. --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Enhance/839/add field to job query (#841) * Initial change to manage 1.2.4 column changes for job listing * Improved documentation, code validated with playbooks. * Updated changelog fragment with PR * corrected duplicate entry in output documentation * Changes to comments and field names as per PR 841 Update to in-code documentation Expansion of changelog fragment * correction to datetime processing * fixing sample data * Enhancement/850/Redesining test cases to be clearer (#840) * Remove duplicates and unnecesary declaration from lineinfile test * Delete blockninfile repeat and unecesary cases and finishing lineinfile * Solve test do not pass * Summary USS test case lineinfile * Clean ds tests general * Finishing clear lineinfile * Lineinfile clean proposal * Clean lineinfile * Finishing the clean of lineinfile and blockinfile * Lineinfile USS fully cleaned * Lineinfile clean * Clean blocinfile test * Structure for all test cases * Add fragment * Finall version without dictionaries * Add expected to variables encoding test case and simplify names * Close lineinfile * Remove the unnecesary marks * Get better encoding tests * Get better encoding tests * Remove encodings of datasets * Add encoding for ds * Functional tso command test cases currently can not be run concurrently#880 (#895) * Remove all dependent test and summary in one * Add fragment * Solve multiple commands and text about the explanation of testing * Change variables to be accurate * Remove import do not used * Add comments * Solve typos and writting --------- Co-authored-by: ketankelkar <ktnklkr@gmail.com> * Update make file doc generation with pre and post scripts for a subset of modules. (#906) * Update make file and add scripts to correct doc gen Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc with missing definition of timestamp Signed-off-by: ddimatos <dimatos@gmail.com> * Correct doc to remove colon to prevent doc gen warning Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc base on minor module doc changes Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Enhance/911/Improve-job-query-performance (#911) * Initial change to manage 1.2.4 column changes for job listing * Improved documentation, code validated with playbooks. * Updated changelog fragment with PR * corrected duplicate entry in output documentation * Changes to comments and field names as per PR 841 Update to in-code documentation Expansion of changelog fragment * correction to datetime processing * fixing sample data * changed job to pass column options to disable the program_name column tested against fresh zoau build (881) * removed 'testing' comment * updated re-request call to use the kwargs function. note: this is for record-keeping... about to rebuild this section * Modified call chain in job:status to not pull dd's, making it faster added new 'don't get dd' variable in get_job_output internal * corrected pep8 issues (spaces and parentheses) * Addressing requested changes: eliminate double negative Added new changelog fragment, corrected link to PR Updated return documentation to show dependancy on zoau version * Correct grammar in changelog Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update Readme with new collection content Signed-off-by: ddimatos <dimatos@gmail.com> * Update copyright yrs Signed-off-by: ddimatos <dimatos@gmail.com> * Generated doc updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update module doc to clear the boolean value comes back as 1 or 0 Signed-off-by: ddimatos <dimatos@gmail.com> * Move a fragment that was outside the changlog folder Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment lint error corrections and summary added Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog generated Signed-off-by: ddimatos <dimatos@gmail.com> * release notes Signed-off-by: ddimatos <dimatos@gmail.com> * Delete changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> * Update test with string match Signed-off-by: ddimatos <dimatos@gmail.com> * Unused changed variable, found by flake8 Signed-off-by: ddimatos <dimatos@gmail.com> * Unused 'normalize_line_endings' functiion found by flake8 Signed-off-by: ddimatos <dimatos@gmail.com> * Correct flake8, import 'path' from line 18 shadowed by loop variable Signed-off-by: ddimatos <dimatos@gmail.com> * Correct comment starting at a new line Signed-off-by: ddimatos <dimatos@gmail.com> * Updated new script copyright year Signed-off-by: ddimatos <dimatos@gmail.com> * Fix release notes formatting Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/583/zos lineinfile does not behave community (#916) * First iteration * Clean test apf_func * Add test case to validate change in ZOAU 1.2.4 * Change test case for the new change * Change zos_job_query test accord to ZOAU 1.2.4 * Restore test as dev * Return test to originals * Return job_query as original * Add fragment * Add test case for DS and change fragments * Solve check of testing for DS * Change logic of tests * Deprecate debug=true in zos_blockinfile and set as_json=true (#904) * Change debug for as_json option * Add fragment --------- Co-authored-by: ketankelkar <ktnklkr@gmail.com> * Add test case to validate response come back complete (#918) * Add test case to validate response come back complete * Add fragment --------- Co-authored-by: ketankelkar <ktnklkr@gmail.com> * Remove conditional unnecessary (#934) * Remove conditional unecesary * Add fragment * Correct the conditional * v1.7.0 beta.2 into dev (#953) * Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> *…

* Added PR# to changelog tweaked description of new feature in both affected functions. * added changelog for ticket * Modified versions in bug issue template to avoid users picking a non-existing version Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Added zoau_api_version logic to check for 1.2.5 or later as a condition for wait_arg Added mention of this to documentation of interface * corrected pep8 errors * removed redundant changelog fragment * Changed Enhancements to minor_changes * push updated module doc Signed-off-by: ddimatos <dimatos@gmail.com> * Enabler/validate path join (#962) * Added real path fetch to base * Sec changes * Updated changelog and template * Added validation to zos_copy and zos_fetch Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Corrected positional argument * Added validation changes Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Removed f-string * Fixed path join for copy_to_file --------- Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Removed 'wait' as an option, and pass wait_arg=true to zoau * Added latest to allow member copy when disp=shr (#980) * Added latest to allow member copy when disp=shr * Added changelog fragment * Added new force option and test for locked data sets non VSAM * Fixed pep8 issue * Added new option force lock * Modified test case with new option * Added force option * Added doc and warning * Updated changelog fragment * Update 980-zos-copy-disp-shr.yml * Updated changelog fragment * Removed unused comments Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Added message * Added force_lock to all CopyHandlers * Modified test case * Changed use of dataset vs data set --------- Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Changed operator_action_query to wait=false time = 5 Renamed vague variable name to "use_wait_arg" Reflected changes and 1.2.5 dependancy in the changelog fragment * Enhancement/423/zos copy add data set member alias support (#1014) * add aliases option and enable text-based member copy w alias to an existing pds Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * disable alias included in data set member listing when collecting src members from pds Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch 'alias' option introduced in zoau1.2.5 to '-H' flag available in zoau1.2.4. also enable alias copying of executables Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * implement aliases for copy to/from USS, add guard rail for non-executable copy to USS with aliases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add exception handler for executable PDS copy, handle non-existent library pds for executable USS src, add error message for PDS copy attempt to USS file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up init functions, break up long lines Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * refactor executable member to member copy for alias work, this commit refactors some helpers which break a select few loadlib tests, but those will be refactored in upcoming comimts Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * refactor and expand test_copy_pds_loadlib_member_to_uss test case to copy to a new loadlib Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add aliases error raised check to text-based pds member copy to uss Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add aliases error raised check to text-based pds member copy to uss Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * parametrize cobol program to pass in custom output string, create helper method around running and validating loadlib pgms, refactor executable tests to use helper method, add helper method to create loadlib w multiple members, add test case for loadlib to loadlib copy w and w/o aliases. Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add test case for copying entire loadlib to uss dir and then to another loadlib. refactor other loadlib test case to reduce loc Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new pytest markers for aliases and loadlib test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in a sneak preview version of bug #920 addressed in PR #968 which adds LIBRARY as a valid value to the dest_data_set option Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * alter placement of aliases option to go after executable options Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add docs and examples for aliases option Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * minor tweaks to doc Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * addres santiy check issues Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * missed a sanity check issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve remaining merge conflicts Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 style issues Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup spacing issue in examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add updated rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add comments for explaning logic/code flow around full pds copy Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * New module zos_script (#961) * First version of the action plugin and module doc * Added remote execution * Fixed handling and cleanup of temp files * Fixed mode setting for scripts * Fixed undefined variable error in action plugin * Fixed chdir when running command * Added creates and removes options * Changed encoding validation * Filled out docs for the module * Added examples * Filled out RETURN block for module * Enhanced error messages * Generated docs for zos_script * Added first tests for zos_script * Reordered args * Added mode check to remote script test * Fixed template rendering * Fixed tests * Added tests for error handling and templates * Fixed a sanity error when returning a failure JSON * Updated ignore files * Updated module docs * Updated repository templates for issues * Fixed whitespace in docs * Updated tmp_path description * Updated notes in documentation * Removed use of local_charset * Removed private args This commit finishes the work needed to remove two sanity tests exceptions. * Fixed permissions for remote scripts * Updated module documentation * Updated documentation for tmp_path * main change to version checker for consolidation * Removed old version checker/shell call, added changelog entry * Changed location of duration value set, so it will always be populated. * found another edge case where duration was not being returned. * added tolerance to test on max_rc, where duration is not always returned. * Staging v1.7.0 merge to main (#1019) (#1023) * Galaxy 1.7 updates * Update meta runtime to support ansible-core 2.14 or later * Update ibm_zos_core_meta.yml with updated version * Update readme to align to supported ansible versions and new urls * Added additional sanity ignore files to the exclude list * Added additional sanity ignore files to the exclude list for ansible-lint. * Update copyright yrs for source files that were overlooked * Remove requirements from module doc, rely on offerings minimum requirements, also zoau 1.2.1 never was supported * Add changelog summary for 1.7 * Adding generated antsibull-changelog release changelog and artifacts * Remove v1.7.0_summary, its no longer needed * Update release notes for ac 1.7.0 * Remove unsupported collection versions requiring a version of zoau that is EOS --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> * Enabler/validate path join part 2 (#1029) * Added real path fetch to base * Sec changes * Updated changelog and template * Added validation to zos_copy and zos_fetch Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Corrected positional argument * Added validation changes Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Removed f-string * Fixed path join for copy_to_file * Added validation function to template * Added new files * Added changelog fragment --------- Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Add known issues doc for utf8 issues (#1035) * Add known issues doc for utf8 issues Signed-off-by: ddimatos <dimatos@gmail.com> * Updated changelog fragement to remove colon usage Signed-off-by: ddimatos <dimatos@gmail.com> * Corrected changelog errors Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * [Enhancement] [zos_copy] Add support for ASA control chars (#1028) * Added ASA support when copying from datasets with control chars * Added ASA support when copying from USS to sequential data sets * Added ASA support when copying from USS to PDS/E * Re-enabled copy from datasets to USS * Added copy from non-ASA data sets to seq ASA ones * Added copy from non-ASA data sets to partitioned ones * Added allocation of ASA destination data sets * Added first version of ASA compatibility validation * Added validations for asa_text * Added asa_text documentation * Fixed record_length issue when creating ASA data sets * Fixed record_length issue * Added asa_text example * Added first test * Added DBRM doc to zos_copy module (#1025) * Added DBRM doc to zos_copy module Signed-off-by: ddimatos <dimatos@gmail.com> * Removed an extra space from doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added more tests * Added changelog fragment * Updated RST files * Removed unused import * Fixed merge * Added another validation * Updated docs * Changed use of tsocmd for mvscmd * Fixed merge error * Fixed bug with VSAM sources * Fixed bug with VSAM destinations --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * Merge staging-v1.8.0-beta.1 back to dev (#1040) * Resolved merge conflicts from cherrypick * Delete changelogs * Fix sending a local archive into remote fails (#1045) * Added test * Added test for local archive and then remote unarchive * Added changelog * Removed format fixture * Ensure tempfile cleanup * Clean temporary data sets created during XMIT unarchive operation (#1049) * Added a temp cleanup * Added changelog * Modified changelog * Added removal of src if remote_src is False Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Modified changelog fragments --------- Signed-off-by: Fernando Flores <fernandofloresdev@gmail.com> * Enabler/1002/test_collections_on_ansible core 2_16 (#1053) * Add sanity ignore * Move dependencyfinder * Revert "Move dependencyfinder" This reverts commit 2bbbc5adffe94b32dd6d1af12f7c7cace93cca94. * Modify dependecyfinder * Modify dependecyfinder * Modify dependecyfinder * Add fragment * Modify fragment * [Enabler] [zos_mvs_raw] Remove Try, Except, Pass from code (#1051) * Added action inside exception to avoid pass * Added action inside exception to avoid pass * Added changelog * Update sanity test ignore (#1048) * Fixed Sanity ignore #6 and reduce use of Sanity issues 8-9 * Remove unused import * Remove ignore * Add fragment * Modify fragment * Change fragment * Update 1048-Update_sanity_tests_ignore.yml * Change ignore 2_dot_14 --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Update_sanity_ignore 2_16 (#1056) * Update sanity ignore 2_16 * Add fragment * Change fragment * [Enabler] [module_utils/dd_statement.py] Changed try except pass to except specific DatasetDeleteError exception (#1052) * Changed try except pass to except specific class * Added changelog * Update 1052-try-except-pass-dd-statement.yml * Update dd_statement.py updated copyright year * Removed run command use of subprocess from encode defaults infavor of using AnsibleModule run command (#1055) * Removed run command use of subprocess from encode defaults infavor of using ansible module * Added changelog * [1.9.0] zos_apf remove try expect pass to better exception handling (#1036) * Removed except pass * Added empty strings * Added changelog * Corrected changelog * Modified if statement to honor current behavior * Update 1036-apf-try-except.yml * Update 1036-apf-try-except.yml * [v1.9.0] Replace randint to fix "Standard pseudo-random generators ... " bandit warning (#1016) * Changed random member name generation to choices * Modified choices in module_utils/data_set and blockinfile test * Added changelog fragment * Remove randint import * Removed unused imports * 1043 bug title zos operator is passing wrong value to zoauopercmd (#1044) * corrected kwarg index value from 'wait_arg' to 'wait' Also corrected true/false issue in zoaq * Added and updated changelog. * update PR number in changelog fragment * changed test from \$ to \\$ to eliminate warning * added blocking test to maks sure minimum wait is reached in zoau>1.2.4.5 * removed the else condition from the blocking test, since it is not needed. * corrected tense grammer in changelog fragment * corrected capitalization of ZOAU in changelog fragment. * Bug 1041 zos submit job honor return output literally (#1058) * initial commit to pass return_output to job_output. * corrected fragment name to match branch * tweaked data set test to show result values if positive test fails * removed trace in zos_data_set, and added trace output to job_submit * removed extra text from functional testing. * put in correct PR number in changelog fragment. * changed trivial to minor_changes, added documentation to dd_scan in job:job_output. * Enabler/1024/remote_tmp for zos_script (#1060) * Changed tmp_path for Ansible's remote_tmp * Remove tmp_path from module's options * Update module documentation * Remove tmp_path test case * Update zos_script's RST file * Add changelog fragment * Updated module examples * [zos_copy] Files corrupted after second copy (#1064) * Initial change to replace shutil.copy * Added fix for corrupted directory copies * Added changelog fragment * Modified docstring and fixed copy_tree * Added punctiation * Added copystat * Added set mode for dirs * Update 1064-corruped-second-copy.yml * [Documentation] [zos_tso_command] Add REXX exec example (#1065) * Add REXX exec example * Add fragment * Update module documentation * Fix PR link * Reword example task name * Updated REXX example * Mvs to non existent mvs copy destination attrs match up (#1066) * Fixed error * Fix identation * Add fragment * Modify fragment * Modify fragment * Modify fragment * Bugfix/374/module zos mvs raw errors with long multi line quoted string in content field (#1057) * Add function of write content * Push easy soultion for two cases * Fix identation and more issues * Fix identation and more issues * Solve error of null * Add validation comments and separete the code * Add fragment * Modify logics * Return overthink * Add explanation for the user and change logic * Add explanation for the user and change logic * Change documentation * Change fragment * Better error message, better documentation and fragment * Get better mesages * Change the logic * Change documentation * Change logic * Add scape to # * Check failing * Check failing * Add valid scapes * Update zos_mvs_raw fragment and module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Remove tarfile.exctractall and zipfile.extractall in favor of individual member extraction for sanity (#1077) * Implemented extract_all function * Added changelog * Removed commented lines * Update ac tool version of pyyaml to avoid wheel issue Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fixed sanity issues with zos_mvs_raw (#1084) * [bugfix][v1.9.0][zos_unarchive]action plugin does not clean up remote temporary files after completion (#1073) * Push solution * Add fragment * Modify logic for fails and ensure works * Change bug * Chamge fragment * Chamge fragment * Add comma --------- Co-authored-by: Demetri <dimatos@gmail.com> * [v1.9.0] Bugfix/837/missing ZOAU imports (#1042) * Added new missing import handler to zos_operator * Added new import handler to job util and zos_job_query * Added ZOAU check to zos_operator * Fixed sanity test issue * Fixed Python 2.7 sanity issue * Add changelog fragment --------- Co-authored-by: Demetri <dimatos@gmail.com> * [Enhancement][zos_job_submit] Modify error messages to be clearer (#1074) * Improved error messages * Updated failure logic --------- Co-authored-by: Demetri <dimatos@gmail.com> * update link in managed_node doc (#1089) * update link in managed_node doc Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Updated docs regarding managed node Signed-off-by: ddimatos <dimatos@gmail.com> * Doc fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * First check of solution * Add clear solution * Return local * Add test ignores * Add ignore to 2.16 * Optimize the encoding * Remove unused import * Return encoded * Add encoding * Remove unused encoded * Modified logic to remove tmp files if they exist and not only if location is local * Added temp_file null definition * Update 1091-Update_undocumented_argument_and_import_exception.yml * Merge release v1.8.0 into dev branch (#1095) * Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gma…

* Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* …

* Optimize the encoding * Remove unused import * Return encoded * Add encoding * Remove unused encoded * Modified logic to remove tmp files if they exist and not only if location is local * Added temp_file null definition * Update 1091-Update_undocumented_argument_and_import_exception.yml * Merge release v1.8.0 into dev branch (#1095) * Staging v1.7.0 beta.1 (#915) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gm…

* fix undefined variable * Add changelog * Modified changelog * Change in fetch * Modify fetch * Fix fetch * Fix find and continue with fetch * Catalog a vsam * Catalog a vsam * Catalog a vsam * Add name to vsam * Add name to vsam * Add name to vsam * Add name to vsam * Delete and add names to pds with member * Remove other modules dependencies * Remove hardcoded content * Modified variable definitions * removed variable assignments * Changed IMSTESTL name * Add fragment * [Bugfix][v1.9.0]short job name sends back a value error with a full stack trace as the msg (#1078) * Add types to better_args_parser * Remove unused re import * Manage spaces * Manage documentation * Manage documentation * Modify regex * Modify regex * Add fragment * Test cases * Change bool to str * Solve documentation * Remove runtime error * Remove re * Test case for owner and job id and add to changelog * Cover all cases * [Enabler][zos_copy]Refactor calls to use new alias and execute options (#1163) * Refactor zos copy and test case fixed * Update ZOAU list to include v1.3.0 (#1166) * [v1.10.0][zos_lineinfile] Removed zos_copy dependency from test cases (#1152) * Changed zos_copy to echo * Modified test case * Added changelog * [v1.10.0][zos_fetch] Remove zos_copy dependency from test cases (#1165) * Removed zos_copy from zos_fetch test cases * Added trailing char * Initial commit * Updated changelog * removed old fragment * Update 1165-remove-zos-copy-dep-from-zos-fetch.yml * [v1.10.0][zos_encode] Remove zos_copy dependency from test cases (#1157) * Removed zos_copy dependency * Added changelog * [v1.10.0][zos_archive] Remove zos_copy depedency from test cases (#1156) * removed zos_copy from zos_archive tests * Added changelog * [v1.10.0][zos_blockinfile] Remove zos_copy from test cases (#1167) * Removed zos_copy from test cases * Added changelog * Added file creation * [v1.10.0][zos_lineinfile] Remove zos encode from zos lineinfile (#1179) * Started removing encoding from tests * WIP * WIP * Added conversion * Removed zos_encode * Added changelog * Removed unused code * [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo * zos_operator - ZOAU 1.3.0 migration (#1181) * adjust value of timeout param which the module measures in seconds to centiseconds for zoau v1.3.x+ migration Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename timeout param in helper functions to distinguish unit of measurement - timeout_s and timeout_c Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright year Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [Enabler] [job] ZOAU v1.3.0 migration for job.py (#1169) * Update calls to jobs * Update copyright years * Add changelog fragment * Rename changelog fragment * Fix references to None types * Update 1169-util-job-zoau-migration.yml --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/copy.py] Implement ZOAU 1.3 migration changes into module_utils/copy.py (#1187) * Replaced zoau datasets import * Updated changelog * Updated changelog * Modified copyright year * Update 1187-migrate-module-utils-copy.yml * [v1.10.0] [module_utils/dd_statement.py] Implement ZOAU 1.3 migration changes into module_utils/dd_statement.py (#1190) * Migrated module_utils/dd_statement * Added changelog * Update 1190-migrate-module_utils-dd_statement.yml * [v1.10.0] [module_utils/backup.py] Implement ZOAU 1.3 migration changes into module_utils/backup.py (#1188) * Changed datasets call to zoau 1.3 * Updated changelog * Corrected changelog location and tag * Corrected copyright dates * Removed wrong exception raise * Removed unused var * [v1.10.0][module_utils/data_set.py] Implement ZOAU 1.3 migration changes into module_utils/data_set.py (#1182) * Made changes to module utils * Added traceback import * Updated changelog * Changed year * Updated zoau import * Update 1182-migrate-module-utils-data-set.yml * Changed build zoau args to dataset type * [Enabler] [zos_copy] Remove zos_fetch call in loadlib test (#1184) * Remove zos_fetch call in loadlib test * Add changelog fragment * Change use of cp to dcp * Add delay to last zos_copy call * Change dcp call * Disable cleanup temporarily * Change tmp dir used * Change scp for sftp * Turn cleanup on once again * Removed print statement --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/encode.py] Implement ZOAU 1.3 migration changes into module_utils/encode.py (#1189) * Updated module_utils encode * Updated changelog * Update 1189-migrate-module_utils-encode.yml * Modified datasets.create call * Changed datasets.create call * [v1.10.0] [zos_gather_facts] ZOAU 1.3 migration - zos_gather_facts (#1196) * update module to leverage zoau python api for zinfo Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address pep8 issues Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update catch-all error message Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * [v1.10.0] [zos_backup_restore]Added choices for space type (#1200) * Added choices for space type * Added changelog * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 (#1194) * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 Added preferred volumes changes [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo Fix for empty volumes on test_config Added comment to remind uncomment test case in the future Add more validation Add more validation Add more validation Add to config volumes and remove upper case variable name * Added keyword into tests --------- Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> * Cherry picked 1.9.0 beta.1 into dev (#1207) * Merge Staging release v1.9.0 beta.1 into main (#1205) * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository templa…

* Fix fetch * Fix find and continue with fetch * Catalog a vsam * Catalog a vsam * Catalog a vsam * Add name to vsam * Add name to vsam * Add name to vsam * Add name to vsam * Delete and add names to pds with member * Remove other modules dependencies * Remove hardcoded content * Modified variable definitions * removed variable assignments * Changed IMSTESTL name * Add fragment * [Bugfix][v1.9.0]short job name sends back a value error with a full stack trace as the msg (#1078) * Add types to better_args_parser * Remove unused re import * Manage spaces * Manage documentation * Manage documentation * Modify regex * Modify regex * Add fragment * Test cases * Change bool to str * Solve documentation * Remove runtime error * Remove re * Test case for owner and job id and add to changelog * Cover all cases * [Enabler][zos_copy]Refactor calls to use new alias and execute options (#1163) * Refactor zos copy and test case fixed * Update ZOAU list to include v1.3.0 (#1166) * [v1.10.0][zos_lineinfile] Removed zos_copy dependency from test cases (#1152) * Changed zos_copy to echo * Modified test case * Added changelog * [v1.10.0][zos_fetch] Remove zos_copy dependency from test cases (#1165) * Removed zos_copy from zos_fetch test cases * Added trailing char * Initial commit * Updated changelog * removed old fragment * Update 1165-remove-zos-copy-dep-from-zos-fetch.yml * [v1.10.0][zos_encode] Remove zos_copy dependency from test cases (#1157) * Removed zos_copy dependency * Added changelog * [v1.10.0][zos_archive] Remove zos_copy depedency from test cases (#1156) * removed zos_copy from zos_archive tests * Added changelog * [v1.10.0][zos_blockinfile] Remove zos_copy from test cases (#1167) * Removed zos_copy from test cases * Added changelog * Added file creation * [v1.10.0][zos_lineinfile] Remove zos encode from zos lineinfile (#1179) * Started removing encoding from tests * WIP * WIP * Added conversion * Removed zos_encode * Added changelog * Removed unused code * [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo * zos_operator - ZOAU 1.3.0 migration (#1181) * adjust value of timeout param which the module measures in seconds to centiseconds for zoau v1.3.x+ migration Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename timeout param in helper functions to distinguish unit of measurement - timeout_s and timeout_c Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright year Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [Enabler] [job] ZOAU v1.3.0 migration for job.py (#1169) * Update calls to jobs * Update copyright years * Add changelog fragment * Rename changelog fragment * Fix references to None types * Update 1169-util-job-zoau-migration.yml --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/copy.py] Implement ZOAU 1.3 migration changes into module_utils/copy.py (#1187) * Replaced zoau datasets import * Updated changelog * Updated changelog * Modified copyright year * Update 1187-migrate-module-utils-copy.yml * [v1.10.0] [module_utils/dd_statement.py] Implement ZOAU 1.3 migration changes into module_utils/dd_statement.py (#1190) * Migrated module_utils/dd_statement * Added changelog * Update 1190-migrate-module_utils-dd_statement.yml * [v1.10.0] [module_utils/backup.py] Implement ZOAU 1.3 migration changes into module_utils/backup.py (#1188) * Changed datasets call to zoau 1.3 * Updated changelog * Corrected changelog location and tag * Corrected copyright dates * Removed wrong exception raise * Removed unused var * [v1.10.0][module_utils/data_set.py] Implement ZOAU 1.3 migration changes into module_utils/data_set.py (#1182) * Made changes to module utils * Added traceback import * Updated changelog * Changed year * Updated zoau import * Update 1182-migrate-module-utils-data-set.yml * Changed build zoau args to dataset type * [Enabler] [zos_copy] Remove zos_fetch call in loadlib test (#1184) * Remove zos_fetch call in loadlib test * Add changelog fragment * Change use of cp to dcp * Add delay to last zos_copy call * Change dcp call * Disable cleanup temporarily * Change tmp dir used * Change scp for sftp * Turn cleanup on once again * Removed print statement --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/encode.py] Implement ZOAU 1.3 migration changes into module_utils/encode.py (#1189) * Updated module_utils encode * Updated changelog * Update 1189-migrate-module_utils-encode.yml * Modified datasets.create call * Changed datasets.create call * [v1.10.0] [zos_gather_facts] ZOAU 1.3 migration - zos_gather_facts (#1196) * update module to leverage zoau python api for zinfo Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address pep8 issues Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update catch-all error message Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * [v1.10.0] [zos_backup_restore]Added choices for space type (#1200) * Added choices for space type * Added changelog * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 (#1194) * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 Added preferred volumes changes [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo Fix for empty volumes on test_config Added comment to remind uncomment test case in the future Add more validation Add more validation Add more validation Add to config volumes and remove upper case variable name * Added keyword into tests --------- Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> * Cherry picked 1.9.0 beta.1 into dev (#1207) * Merge Staging release v1.9.0 beta.1 into main (#1205) * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository template updates and future proofing Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Formatting corrections for release notes Signed-off-by: ddimatos <dimatos@gmail.com> * Upate issue templates with newer version of software Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac command supporting files (#789) * Update ac command supporting files Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_data_set module member description Signed-off-by: ddimatos <dimatos@gmail.com> * Add recently changed module doc from prior commits Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Adding fix for uncataloged vsam and non-vsam data sets Signed-off-by: ddimatos <dimatos@gmail.com> * Encode files recursively and test case for keep behavior. (#772) * Bring the jinja2 solution to dev and add test case * Add fragment * Solve problem z/OS 2.5 HFS * Declaration error solve * Need to check the validation with HFS * Ensure validating z/OS work with HFS * Change inecesary changes and fragments q * Return all test cases to normal * Return all test cases to normal * Create the local test case * Add local test case and change test case to be acurate * Get better cleanup of test-case * Update test_zos_data_set_func.py Equalize test mount func * Update ac to support a single test (#793) * Update ac to support a single test Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update test description Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Return the dynamically created destination attributes (#773) * First iteration to get dynamic values * Spaces and lines rectified * Add validation and extra variable to ensure consistency * Whitespaces * Change imports in test_zos_mount_func * Update test_zos_fetch_func imports * Update all imports for pipelines runs * Revert "Update all imports for pipelines runs" This reverts commit 1b370a2ba3c0001c316e0121ddab82ae7cc6d75d. Return one commit * Update data_set.py imports * Revert "Update data_set.py imports" This reverts commit 37561b0a12e04faaee8307a5541b71469dbe721d. * Update data_set imports * Update data_set imports * Update data_set imports * Restore import * Restore the imports * Add fragment * Solve a ty…

* Staging v1.12.0 beta.1 (#1771) * Fix fetch * Fix find and continue with fetch * Catalog a vsam * Catalog a vsam * Catalog a vsam * Add name to vsam * Add name to vsam * Add name to vsam * Add name to vsam * Delete and add names to pds with member * Remove other modules dependencies * Remove hardcoded content * Modified variable definitions * removed variable assignments * Changed IMSTESTL name * Add fragment * [Bugfix][v1.9.0]short job name sends back a value error with a full stack trace as the msg (#1078) * Add types to better_args_parser * Remove unused re import * Manage spaces * Manage documentation * Manage documentation * Modify regex * Modify regex * Add fragment * Test cases * Change bool to str * Solve documentation * Remove runtime error * Remove re * Test case for owner and job id and add to changelog * Cover all cases * [Enabler][zos_copy]Refactor calls to use new alias and execute options (#1163) * Refactor zos copy and test case fixed * Update ZOAU list to include v1.3.0 (#1166) * [v1.10.0][zos_lineinfile] Removed zos_copy dependency from test cases (#1152) * Changed zos_copy to echo * Modified test case * Added changelog * [v1.10.0][zos_fetch] Remove zos_copy dependency from test cases (#1165) * Removed zos_copy from zos_fetch test cases * Added trailing char * Initial commit * Updated changelog * removed old fragment * Update 1165-remove-zos-copy-dep-from-zos-fetch.yml * [v1.10.0][zos_encode] Remove zos_copy dependency from test cases (#1157) * Removed zos_copy dependency * Added changelog * [v1.10.0][zos_archive] Remove zos_copy depedency from test cases (#1156) * removed zos_copy from zos_archive tests * Added changelog * [v1.10.0][zos_blockinfile] Remove zos_copy from test cases (#1167) * Removed zos_copy from test cases * Added changelog * Added file creation * [v1.10.0][zos_lineinfile] Remove zos encode from zos lineinfile (#1179) * Started removing encoding from tests * WIP * WIP * Added conversion * Removed zos_encode * Added changelog * Removed unused code * [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo * zos_operator - ZOAU 1.3.0 migration (#1181) * adjust value of timeout param which the module measures in seconds to centiseconds for zoau v1.3.x+ migration Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename timeout param in helper functions to distinguish unit of measurement - timeout_s and timeout_c Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright year Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [Enabler] [job] ZOAU v1.3.0 migration for job.py (#1169) * Update calls to jobs * Update copyright years * Add changelog fragment * Rename changelog fragment * Fix references to None types * Update 1169-util-job-zoau-migration.yml --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/copy.py] Implement ZOAU 1.3 migration changes into module_utils/copy.py (#1187) * Replaced zoau datasets import * Updated changelog * Updated changelog * Modified copyright year * Update 1187-migrate-module-utils-copy.yml * [v1.10.0] [module_utils/dd_statement.py] Implement ZOAU 1.3 migration changes into module_utils/dd_statement.py (#1190) * Migrated module_utils/dd_statement * Added changelog * Update 1190-migrate-module_utils-dd_statement.yml * [v1.10.0] [module_utils/backup.py] Implement ZOAU 1.3 migration changes into module_utils/backup.py (#1188) * Changed datasets call to zoau 1.3 * Updated changelog * Corrected changelog location and tag * Corrected copyright dates * Removed wrong exception raise * Removed unused var * [v1.10.0][module_utils/data_set.py] Implement ZOAU 1.3 migration changes into module_utils/data_set.py (#1182) * Made changes to module utils * Added traceback import * Updated changelog * Changed year * Updated zoau import * Update 1182-migrate-module-utils-data-set.yml * Changed build zoau args to dataset type * [Enabler] [zos_copy] Remove zos_fetch call in loadlib test (#1184) * Remove zos_fetch call in loadlib test * Add changelog fragment * Change use of cp to dcp * Add delay to last zos_copy call * Change dcp call * Disable cleanup temporarily * Change tmp dir used * Change scp for sftp * Turn cleanup on once again * Removed print statement --------- Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * [v1.10.0] [module_utils/encode.py] Implement ZOAU 1.3 migration changes into module_utils/encode.py (#1189) * Updated module_utils encode * Updated changelog * Update 1189-migrate-module_utils-encode.yml * Modified datasets.create call * Changed datasets.create call * [v1.10.0] [zos_gather_facts] ZOAU 1.3 migration - zos_gather_facts (#1196) * update module to leverage zoau python api for zinfo Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address pep8 issues Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update catch-all error message Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * [v1.10.0] [zos_backup_restore]Added choices for space type (#1200) * Added choices for space type * Added changelog * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 (#1194) * Cherry picked removed hard coded content from staging-v1.9.0-beta.1 Added preferred volumes changes [v1.10.0] [zos_copy] Enhance zos_copy performance when copying multiple PDS members (#1183) * [v1.9.0] [zos_copy] Enhancement/764/copy members (#1176) * Changed member copy into a bulk call * Modified copy to members * Cleaned code * Removed hardcoded content from zos_copy tests * Added fix for uss files * Added distinction between uss and mvs * Added alias fix * Moved the copy section to below * Modified for seq test * Added fix for copy dest lock * Added msgs for debugging * Added final changes to member copy * Added copy for when seq to pdse * Add changelog * Added a line into docs * Modified doc * Modified doc * Update changelog * Created a new changelog * Corrected typo Fix for empty volumes on test_config Added comment to remind uncomment test case in the future Add more validation Add more validation Add more validation Add to config volumes and remove upper case variable name * Added keyword into tests --------- Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> * Cherry picked 1.9.0 beta.1 into dev (#1207) * Merge Staging release v1.9.0 beta.1 into main (#1205) * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge master to dev for 1.6.0 beta.1 (#763) * Update branch production branch Main with release v1.5.0 content (#756) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Correct run on sentence Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added change fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * bring in updates to zos_gather_facts RETURN docstring (#744) Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests Signed-off-by: ddimatos <dimatos@gmail.com> * Update release notes and changelog with full date formats Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove changelog fragments no longer needed (#754) Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * v1.6.0-beta.1 release (#759) * Added tagging of converted files * Updated tests for zos_encode * Added restoration of PDS/PDSE members when module fails zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work. * Fixed cleanup of dest when module fails * Removed exception used for debugging * Added pytest markers * solved pep8 issue * Added more information to error when restoration fails * Update make file with new features to manage the nodes (#563) * Update make file with new features to manage the nodes Signed-off-by: ddimatos <dimatos@gmail.com> * Correct typo Signed-off-by: ddimatos <dimatos@gmail.com> * updated makefile to allow for no password in the clear and added a new host Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a bug that failed to copy the cert to the managed node (#577) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Added apropiate error message in zos_lineinfile when src is not found * Better error message * Added fragment for changelog * Update git issue slection field (#593) * Update git issue slection field * Added some additional future versions * Added rule to ignore python 2.7 compile not supporting f strings * Corrected rule in ignore file * Update 584-zos_lineinfile-error-message.yml * Added missing fragments for issues 309 and 408 * update make file and related artifacts with recent zVM changes (#598) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Expand what is ingored for venv to all venv's (#613) Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Ansible-core versions with updated ignore file pass (#615) * Validated that ansible-core versions with updated ignore file pass locally Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> * remove redundant entry on last line Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> * Update make file to read exported var VENV else default to 'venv' (#614) * Update make file to read exported var VENV else default to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Correct the default from defaultValue to 'venv' Signed-off-by: ddimatos <dimatos@gmail.com> * Bugfix/558/zos copy backup dev (#609) * Create emergency backup only if force is set to False and dest exists * Added Changelog fragment * Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602) * Updated import in zos_fetch * Updated import in zos_job_submit * Updated import in module_utils/data_set * Updated import in module_utils/System * Updated import in zos_copy * Updated import in zos_fetch * Added changelog fragment * Update enhancement-518-text-converter-import.yml * Updates the makefile and related scripts for the latest mounts (#628) * Update makefile coments and use --ignore for pytest Signed-off-by: ddimatos <dimatos@gmail.com> * lexicographical order targets Signed-off-by: ddimatos <dimatos@gmail.com> * Update makefile and artifacts to support new mounts Signed-off-by: ddimatos <dimatos@gmail.com> * Removing encrypted versions of the scripts, no longer senstive content Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * update profile created for mount points Signed-off-by: ddimatos <dimatos@gmail.com> * Correct f-string usage to remain 2.7 compatible (#659) * Correct f-string usage to remain 2.7 compatible Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Updated shell scripts for development tooling Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Add issue template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Update collab template Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * Template updates Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * add udpated checkbox to collab template Signed-off-by: ddimatos <dimatos@gmail.com> * changed zos_job_query to allow multi/embedded wildcard job_values. job.py/_get_job_status now uses fnmatch to test for wildcarded job_names. added test looking for a wildcard job name. added basic changelog text as 'enhancement'. * expanded wildcard support to job_id as well as job_name corrected error-handling issue in query routine playbook tests complete * cleaned up pep8 issues * Bug fix to correct job log returining and other edge cases (#683) * Update zos_job_submit to handle some edge cases and return the job log nearly always Signed-off-by: ddimatos <dimatos@gmail.com> * Linting correction Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement for pr 683 Signed-off-by: ddimatos <dimatos@gmail.com> * Clean up comments in code Signed-off-by: ddimatos <dimatos@gmail.com> * Update restructured text for modules and changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Add support to check for security exception to job submit module Signed-off-by: ddimatos <dimatos@gmail.com> * Add additonal logic for jobs that use typerun=scan Signed-off-by: ddimatos <dimatos@gmail.com> * Update chnagelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update zos_job_submit.py Grammar change and correction. --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added uss_tag_encoding function * Fixing linter issues * removed extraneous comment on query, eliminated unused variable * responding to reviewer comments * Expanded query tests to run a hello jcl file to make sure we are searching for a job id that exists. * Expanded documentation in code to highlight asterisk availability. Added examples showing multi-asterisk search. * Corrected 2 documentation errors * Change to documentation text (indent on multi line string?) * Still trying to get documentation to pass * Looks like '---' was killing documentation block. * Update zos_blockinfile with complex examples (#727) * Update zos_blockinfile with complex examples Signed-off-by: ddimatos <dimatos@gmail.com> * Add changelog fragement Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Added additional D SYMBOLS example to zos_operator documentation (#730) * Update zos_operator with an additional example Signed-off-by: ddimatos <dimatos@gmail.com> * Added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * zos_gather_facts - add sample output to RETURN docstring (#722) * zos_gather_facts - add sample output to RETURN docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update generated module doc Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * 574find (#668) * change to allow find loop to continue when one particular data set element is not found This has been validated with a zos-check-find.yaml playbook * added functional test that includes non-existant pds entry, which should be skipped over * added changelog fragment. * zos_blockinfile , can quotes in content can be supported (#680) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * A needed space * All in structured * Comments solved * Better order --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Demetri <dimatos@gmail.com> * zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723) * Verify coomand of ZOAU support the doble quotes and get better output of message * Verify coomand of ZOAU support the doble quotes and get better output of message * Restore to the one tyme function solving TypeError * Test about cases with quotes supported * Solve comments * Comments in the changelog * Adjust test for working accord the text * Solve the dest functional mode set for the applied to destination directory * Identation and spaces * To work well * To work well * To work well * To work well * Changelogs added * Solved the fragment test and separte the cases --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * corrected job test case that wanted to extract job id. * changed call to zos_job_query in the functional test. * zos_data_set - add force parameter to enable member delete via disp shr (#718) * add force parameter to enable member delete via disp shr Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update link to PR instead of issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * fix minor mistake in func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update DOCstring for missed items highlighted in PR review Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years in data set module util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Update to doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * update doc, rst Signed-off-by: ddimatos <dimatos@gmail.com> * Update doc and examples Signed-off-by: ddimatos <dimatos@gmail.com> * Added handler for job not found edge cases (None not iterable errors) * corrected pep8 issue (bad indent) * removed tracking text from error/not found messages. * Update zos_job_query.py * Add and automate a load lib test case for module zos_copy (#640) * Modified if statement * Added changelog fragment * Corrected statements Corrected PR in changelog fix lint issue * Adding test case on the work * Added assertion to loadlib test Changed cobol src Added newlines * Correcting cobol src * Removed unnecessary comments * Removed deprint statements for debug * Update 601-copy-loadlib-member.yml * Reduce space primary * Make sure cobol is executable * corrected name * Corrected names * Adding debug statements * modified name in link step * Changing copy name * Removed print statements for debug * Removed previous changelog * Removed unused fragment * Removed test case * Add max_rc support for module zos_tso_command (#666) * rebuilding 565 * fixing pep8 issue * corrected not in code, added functional test to use maxrc * changing maxrc return codes as indicated * updated copy_rexx to incorporate max_rc * pep8 issue on an if statement * changed test to dump intermediate values temporarily * added max_rc to arg params to get it to pass through * removed extra output on test * Added changelog fragment. * Update 565-zos_tsocommand_maxrc.yml * PARTIAL correction based on reviewer's feedback. Will complete later today. * Partial push: change origrc to orig_rc and added documentation. Added experimental loop for multiline results. Changed error case to continue through the commands even after a failure (?!) * Changes based on review feedback: - rc is actual returned value - origrc/orig_rc removed - in a list of commands, first error (considering max_rc) stops the chain - while processing commands, a list of success/failures are kept for error state info - changed listds to a longer, more complete string in example * updated test case for max_rc to use longer, more complete listds command * Minor pep8 and yamllint changes * Updated string builder to use append and join, for efficiency. * Update changelog fragment to correspond to pull request number Signed-off-by: ddimatos <dimatos@gmail.com> * removed old fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Changed output of command-by-command to use max_rc instead of Max RC, for consistency. * clarifying failure message when user hasn't set a max_rc. * corrected date on copyright notice to include current year. * corrected copyright date on test case --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> * added changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * zos_copy forward port a test case for symbols on a volume #739 (#740) * added test case for symbols on a volume Signed-off-by: ddimatos <dimatos@gmail.com> * Changelog fragment added Signed-off-by: ddimatos <dimatos@gmail.com> * Update test case with added comments Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * Volume Initialization Module (#654) * Adding ZOS_INIT plugins The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py. * Create zos_ickdsf_command.rst Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection. * Update zos_ickdsf_command.py fixed to pass pep8 sanity test. * Update zos_ickdsf_command.py Edited zos_ickdsf_command.py in action folder to pass pep8 style check. * bugfix - undo hard-coded storagegroup param from command passed to mvs_raw Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update encoding and licensing info Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * edit descriptions, leaving in old to verify messaging with dfsms team Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to buildix Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * cleanup examples docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add return docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add some unit tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update dict names, some clean up Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update interface and docs with cleaner descriptions and define default values and required true/false where applicable Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove init param and bring out sub-options one level Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module from zos_ickdsf_command to zos_ickdsf_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add ignore missing gplv3 license warning from sanity testings to existing ignore files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * style edits to pass sanity tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add in-depth task name to first example docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add minimal params functional test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename rc param, update func test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add note about disabled verify_offline check Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add invalid volume_address func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * change param names, organize mvscmd return output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func tests for "verify_existing_volid" and "verify_no_data_sets_exist" Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add volid positive and negative func tests Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add mutual exclusion to no index/sms_managedw with func test, minor edits to comments * add func tests for index param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add func test for vtoc_tracks param Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * convert 'content' return param to list for prettier playbook output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add specific examples to docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add guard rail test with volser check set to ensure target volume is as expected Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * switch target test vol to user02, remove existing data sets in initial test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * start adding author names to docstring, edits of yaml for sanity test Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename module to zos_volume_init Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove unit test suite which was not kept up to date with code restructure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove old/outdated .rst file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * bring up to speed with v1.5.0-beta.1 which is currently more recent than dev Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add sanity test 2.15 file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * clean up trailing empty lines/space chars Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * more pep8 style Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * pull in more changes from latest dev branch Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * tweak tests/sanity/ignore-2.10.txt file Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * resolve pep8 line too long issue Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add changelog fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add backticks to other special chars in docs for consistency Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * on more formatting tweak to national chars in docstring Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add new param to support tmp_hlq Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * remove soft language ie beware, please; reduce line lengths, fix a spelling error Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> * Ported bugfixes for #664 and #678 from 1.5.0 (#743) * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Updated changelog fragment * Updated copyright information * Added comments to record length tests * Added more context to the record length normalization * Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731) * Add DISP=SHR by function force to the user * Add helper function for force cases only for EOF inserts * Test cases added * Just to check in the change case * All test cases for force in zos_lineinfile working properly * Test cases for force blockinfile * Fixes for style of code * One identation error * One identation error * One identation error * Documentation error * All test cases for line infile and block infile * Comments * Change the stdout to assert with expected of ds * Error of test case * Fragment added * Solve typo errors * Update to remove uncessary comment Signed-off-by: ddimatos <dimatos@gmail.com> * Update changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Update module with force comments Signed-off-by: ddimatos <dimatos@gmail.com> * Solve some test problems * Copyright year * Copyright year * Add cases for other ecodes * Add test for all encodings * Cover all encodes test cases with force * Blockinfile force added for other encodes and DS types --------- Signed-off-by: ddimatos <dimatos@gmail.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> * remove duplicate function (#753) * Update branch production branch Main with release v1.5.0 content (#756) (#758) * Bug fix to correct job log returining and other edge cases (#688) * Update zos_job_submit to handle some edge cases and return the job log nearly always * Linting correction * Add changelog fragement for pr 683 * Clean up comments in code * Update restructured text for modules and changelog fragment * Add support to check for security exception to job submit module * Add additonal logic for jobs that use typerun=scan * Update chnagelog fragment * Correct run on sentence --------- * Added additional `D SYMBOLS` example to zos_operator documentation (#729) * Update zos_operator with an additional example * Added change fragment --------- * Ported bugfixes for #664 and #678 * Updated file normalization for sequential datasets * Fixed encoding when a remote source is used * Added encoding normalization for record length (#664) This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well. * Refactored normalization for USS files * Updated line endings normalization in PDSEs * Updated create_temp_with_lf_endings * Added tests for line endings normalization * Added changelog fragment * Add copy lib member test case * Corrected changelog fragment * Add conflict resolution * Update 641-copy-loadlib-member.yml * zos_copy forward port a test case for symbols on a volume (#739) * Add a test case for symbols using an existing volume on our test system * Add changelog fragment * Update 739-zos_copy-volume-symbol-test.yml Update description to use symbols overs special chars. * Update test case with added comments --------- * bring in updates to zos_gather_facts RETURN docstring (#744) * Solve problem 619 about set mode on dest file (#746) * Solve problem 619 about set mode on dest file * Add fragment of 746 PR --------- * Merge release tasks for v1.5.0 into release branch (#752) * Updated metadata * Updated copyright information * Updated copyright in README * Generated changelog * Added previous versions to changelog * Changed a minor_change fragment to bugfix * Updated release notes * Update dependency finder to allow for all tests minus skip tests * Update release notes and changelog with full date formats --------- * remove changelog fragments no longer needed (#754) --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * update galaxy and meta/ files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update copyright years Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * update rst doc files Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add 1.6.0-beta.1 release summary fragment Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * generate changelog Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add release notes Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * add additional stylizing Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> * Delete fragements once changelog is generated Signed-off-by: ddimatos <dimatos@gmail.com> * update stylizing around *none* in zos_volume_init docs Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Oscar Fernando Flores Garcia <fernandofloresdev@gmail.com> Co-authored-by: Demetri <dimatos@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Update ibm_zos_core_meta.yml --------- Signed-off-by: ddimatos <dimatos@gmail.com> Signed-off-by: Ketan Kelkar <ktnklkr@gmail.com> Co-authored-by: Ivan Moreno <iamorenosoto@gmail.com> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> Co-authored-by: Rich Parker <richp405@gmail.com> Co-authored-by: ketankelkar <ktnklkr@gmail.com> Co-authored-by: André Marcel Gutiérrez Benítez <68956970+AndreMarcel99@users.noreply.github.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: trevor-glassey <Trevor.Glassey@ibm.com> Co-authored-by: Andre Marcel Gutierrez Benitez <andre@mbp-de-andre-2.gdl.mex.ibm.com> * Bugfix/619/mode set for files applied test case (#757) * Add test case for copy dest file * Add comments * Add test for folders * Adjust spaces * Changes for ensure consistency for all tests * Changes of name and clean creations --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> * Bugfix/381/failed when the job name was null or not found (#747) * Add the verbose for failed when job name was null or not found * Adjust message for what we can get * Whitespaces move * Add code from dev * Ecode utility as is in dev * Year for copyright * Case for having both the jod_id and job_name * Ecode utils functions not in my branch * Add final line ecode * Add fragment * Delete encode function two times, adjust job message and change the fragment * Change variable name for one more descriptive * Restore encode and change one word * Encode * bugfixes * Set up as dev * Better fragment --------- Co-authored-by: Andre Marcel Gutierrez Benitez <andre@MacBook-Pro-de-Andre-2.local> Co-authored-by: Fernando Flores <fernandofloresdev@gmail.com> * Bugfix/660/zos operator reported failure caused by unrelated error response messages (#762) * Add options * Add transparency on the response and test cases * Solve spaces * Add validation to append * Fragment Added * Adjust fail_json on non_zero response * Identation mistakes solved * Solve last idenation problem * Replace prior tooling (makefile) that aidded the development workflow with a new 'ac' command. (#766) * Make file mount script helper Signed-off-by: ddimatos <dimatos@gmail.com> * Comments to mount script Signed-off-by: ddimatos <dimatos@gmail.com> * Staged updated scripts for makefile usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update mount scripts for use with makefile Signed-off-by: ddimatos <dimatos@gmail.com> * updates to correct mounts and add function to mounts-datasets Signed-off-by: ddimatos <dimatos@gmail.com> * adding completed new ac command files for development Signed-off-by: ddimatos <dimatos@gmail.com> * update ignore to more specific with venv Signed-off-by: ddimatos <dimatos@gmail.com> * Correcting ignore to allow for venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * moved logic that checks for info.env to venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Adding changelog fragment Signed-off-by: ddimatos <dimatos@gmail.com> * Fix a path issue when calling venv.sh Signed-off-by: ddimatos <dimatos@gmail.com> * Fixes issue not being able to run all tests, fixes issue with content being written to collections folder Signed-off-by: ddimatos <dimatos@gmail.com> * Support zSH and update scp to fall back to legacy scp protocal Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Update ac with password usage Signed-off-by: ddimatos <dimatos@gmail.com> * Fix incorrect message and remove the cd's before and after ac-test Signed-off-by: ddimatos <dimatos@gmail.com> --------- Signed-off-by: ddimatos <dimatos@gmail.com> * 347 new query fields (#778) * changing job.py to return 7 more fields, and for zos_job_query to pass them through * corrected testing to pull all new values through this assumes zoau 1.2.3 and z/OS at least 2.4 need to test older zoau to make sure this will still work * Added zoau version testing import to job.py so it won't reach for non-existent members. * pep8 and lint required changes * changed test to see if it will pass unit testing * Modified test_zos_data_set_func to skip HFS test if zOS > 02.04 * changed OS test for hfs usage * corrected usage of 'hosts'... removed the definition in prior edit. * changing OS version checker * corrected string extraction for OS version checker * added delete shell to 196/197 (finally of cat/uncat test) removed success message from 830 (version test logic) * removed the mvscmdauth call, as it coincides with some new test failures. * added changed=false back into testing of job_query * correction of zos->zoau name in comments. * Missing fragment in PR 778 New query fields (#780) * added fragment for pr 778 * Added changelog fragment query new fields Added changelog fragment query new fields * Update 778-query-new-fields.yml * Update docs with ansible/ansible-core version, AAP and fix the dated git issue templates (#771) * Doc vesion updates Signed-off-by: ddimatos <dimatos@gmail.com> * Repository template updates and future proofing Signed-off-by: ddimatos <dimatos…

ddimatos added 8 commits March 23, 2023 13:22

Update zos_job_submit to handle some edge cases and return the job lo…

8a7c0e7

…g nearly always Signed-off-by: ddimatos <dimatos@gmail.com>

Linting correction

c53e0b4

Signed-off-by: ddimatos <dimatos@gmail.com>

Add changelog fragement for pr 683

cf733c9

Signed-off-by: ddimatos <dimatos@gmail.com>

Clean up comments in code

358a0ac

Signed-off-by: ddimatos <dimatos@gmail.com>

Update restructured text for modules and changelog fragment

74d28fd

Signed-off-by: ddimatos <dimatos@gmail.com>

Add support to check for security exception to job submit module

772d040

Signed-off-by: ddimatos <dimatos@gmail.com>

Add additonal logic for jobs that use typerun=scan

e9b85be

Signed-off-by: ddimatos <dimatos@gmail.com>

Update chnagelog fragment

e2cadff

Signed-off-by: ddimatos <dimatos@gmail.com>

ddimatos requested a review from richp405 March 25, 2023 06:13

ddimatos marked this pull request as ready for review March 25, 2023 06:13

Merge branch 'dev' into bugfix/623-624-655-670-681/zos_job_submit-job…

62a3c51

…-log

ddimatos self-assigned this Mar 25, 2023

richp405 reviewed Mar 27, 2023

View reviewed changes

plugins/modules/zos_job_submit.py Outdated Show resolved Hide resolved

richp405 approved these changes Mar 27, 2023

View reviewed changes

Update zos_job_submit.py

72570d3

Grammar change and correction.

ddimatos merged commit 7b93fec into dev Mar 27, 2023

ddimatos deleted the bugfix/623-624-655-670-681/zos_job_submit-job-log branch March 27, 2023 22:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug fix to correct job log returining and other edge cases #683

Bug fix to correct job log returining and other edge cases #683

ddimatos commented Mar 23, 2023 •

edited

Loading

ddimatos commented Mar 25, 2023

richp405 left a comment

Bug fix to correct job log returining and other edge cases #683

Bug fix to correct job log returining and other edge cases #683

Conversation

ddimatos commented Mar 23, 2023 • edited Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

ddimatos commented Mar 25, 2023

richp405 left a comment

Choose a reason for hiding this comment

ddimatos commented Mar 23, 2023 •

edited

Loading