Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handling FILLER Names #542

Closed
saikumare-a opened this issue Dec 10, 2022 · 8 comments
Closed

Handling FILLER Names #542

saikumare-a opened this issue Dec 10, 2022 · 8 comments
Labels
accepted Accepted for implementation enhancement New feature or request

Comments

@saikumare-a
Copy link

Background

cobrix is adding numbers as suffixes to FILLER column's if there are multiple filler columns in copybook

INPUT:
01 CUST-NAME PIC X.
01 FILLER PIC X.
01 CUST-ADDR PIC 100.
01 CUST-NO PIC X.
01 FILLER PIC X.

OUTPUT - column names
CUST_NAME
FILLER
CUST_ADDR
CUST_NO
FILLER2

##Problem
if there is a new filler gets added in between these 2 fillers, the previous "FILLER2" column becomes "FILLER3" column and this is impacting the downstream

INPUT:
01 CUST-NAME PIC X.
01 FILLER PIC X.
01 CUST-ADDR PIC 100.
01 FILLER PIC X.
01 CUST-NO PIC X.
01 FILLER PIC X.

OUTPUT - column names
CUST_NAME
FILLER
CUST_ADDR
FILLER2
CUST_NO
FILLER3

Feature

The option of having filler names with different suffixes and wont get changed based on new fillers addition could be very helpful.

optional parameter --> option("filler_suffixing","previous_column_name") --> default["seq_number"]

Proposed Solution [Optional]

Solution Ideas

  1. one approach could be adding previous column name as suffix like "FILLER_AFTER_{prev_column_name}". this would also have issue if there is a new column gets added between existing column and filler. but this could occur rare compared to previous number suffixing

Example [Optional]

OUTPUT - column names
CUST_NAME
FILLER_AFTER_CUST_NAME
CUST_ADDR
FILLER_AFTER_CUST_ADDR
CUST_NO
FILLER_AFTER_CUST_NO
@saikumare-a saikumare-a added the enhancement New feature or request label Dec 10, 2022
@yruslan
Copy link
Collaborator

yruslan commented Dec 12, 2022

Thank you for the feature request. i can see the issue it would solve. Could you help me understanding your use case more.

FILLERs are unnamed fields that are usually added as paddings and should not contain useful information. Cobrix initially was dropping all FILLERs. But we got many user requests to retain them.

If FILLERs in your case do contain useful information, and the way you name them is important, why would't you just name these fields in the copybook in the first place? If you replace your fillers with any other name - it will work exactly as you expect.

@saikumare-a
Copy link
Author

  1. we are receiving other important columns in FILLER groups ( scaling to many files and many sub columns in FILLERS)
  2. there are quarterly new columns additions to files

the data is from third party system ( they provide the copybook),

  1. so we dont have control on asking them to change
  2. we could handle that internally by changing in copybook, but due to frequent addition of columns, this approach is becoming problem for us

so, thought of handling in cobrix would be helpful and can benefit other cobrix users

@yruslan yruslan added the accepted Accepted for implementation label Dec 15, 2022
@yruslan
Copy link
Collaborator

yruslan commented Dec 15, 2022

Ok, I see how this can make your life easier. Adding to the backlog...

@saikumare-a
Copy link
Author

Hi @yruslan ,

any luck with thoughts on timeline for this enhancement. Thank you for the support !!

@yruslan
Copy link
Collaborator

yruslan commented Dec 21, 2022

Sorry, can't provide timelines. A ballpark estimate might be early next year

@saikumare-a
Copy link
Author

Thank you for the update.

yruslan added a commit that referenced this issue Dec 28, 2022
@yruslan
#542 Implement filler naming strategies and previous fields based str…
6b76518 
…ategy.
yruslan added a commit that referenced this issue Dec 28, 2022
@yruslan
#542 Implement filler naming strategies and previous fields based str…
bfe201c 
…ategy.
@saikumare-a
Copy link
Author

Hi @yruslan,

seems this enhancement is done, could you attach the latest jar snapshot so that i can test once and confirm.

Thank you for the support!!

@yruslan
Copy link
Collaborator

yruslan commented Jan 3, 2023

It would really help if you can build it. You just need JDK 1.8 and sbt (any version). Then, use one of commands from here: https://github.com/AbsaOSS/cobrix#creating-an-uber-jar

@yruslan yruslan closed this as completed Jan 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
accepted Accepted for implementation enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yruslan @saikumare-a