Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rearrange reader #327

Merged
merged 31 commits into from
Sep 4, 2020
Merged

Rearrange reader #327

merged 31 commits into from
Sep 4, 2020

Conversation

kinverarity1
Copy link
Owner

@kinverarity1 kinverarity1 commented May 3, 2020

This PR will finalise rearranging the reader code so that the following process is followed when reading a file:

  • 1. The file is iterated over line by line looking for ~ characters and it stores the location of these in the file and the contents of the line (i.e. the section title).
  • 2. The reader returns to each section. For those which are not data sections it seeks to the beginning of each one and parses it by iterating only over the lines which are in that section
  • 3. After all that, the parser returns to each data section and parses it, using the information obtaining from parsing the definition/curves section in step 2

Now all header sections are parsed fully before returning
to read data sections.

Broken - this is a work in progress.
@kinverarity1 kinverarity1 added the las3 stuff relating to LAS 3.0 label May 3, 2020
@kinverarity1 kinverarity1 self-assigned this May 3, 2020
@kinverarity1 kinverarity1 changed the base branch from master to las3-develop May 3, 2020 23:53
kinverarity1 and others added 13 commits May 5, 2020 07:36
- test_open_file.py::test_open_incorrect_filename - UnboundLocalError:
  local variable 'file_obj' referenced before assignment
  fix: assign file_obj as empty string before usage.
- test_read.py::test_v12_inf_uwi_leading_zero_value -
  lasio.exceptions.LASHeaderError: Line 18 (section ~P): "~C"
  fix: stop parsing section when we are at a new section.
Change the leading zero test files to be sample*.las plus the leading
zero special case.  Previously the leading zero test files had other
special cases in them that caused their tests to exercise unrelated code
paths.
…section-test

Rearrange-Reader Branch: Fix test_read::missing-a-section test
Rearrange-Reader: Base leading zero test files on sample.las
- Add tests/examples/3.0 dir.
- Add the CWLS's 3.0 example las file.
- Copy the example file to sample_3.0.las to standardize with
  1.2 and 2.0 sample las files.
- Create a tests/test_read_30.py with basic read test.
  However, the test is set to SKIP because it current fails on the
  rearrange-reader branch
…30-test

To rearrange-reader branch: add basic las 30 test
This enables tests/test_read.py::test_barebones_missing_all_sections
to pass, but may not be the complete or final solution.
…test

To rearrange-reader branch:  fix barebones test
kinverarity1 and others added 9 commits July 2, 2020 00:01
LAS 3.0 : Rearrange reader branch: Fix find_sections_in_file with text stream
The current tests check 1.2 version files with unknown sections.
Rearrange-Reader: Enable Unknown section tests to pass
- fixes tests/test_null_policy.py::test_null_policy_NULL_none
Rearrange Reader:  fix json and write tests
Merge to Rearrange-Reader: Add check if null should be replaced with nan
Parsing a data line with regexp_subs fixes common errors in the data
(~A) section. Adding the regexp_subs to inspect_data_section() fixes
the following tests by standardizing the line data resulting in
consistent date array lengths:
- test_null_policy_runon_replaced_1
- test_null_policy_runon_replaced_2
- test_null_policy_runon_ok_1
- test_null_policy_runon_ok_2
- test_write_sect_widths_12
@kinverarity1 kinverarity1 changed the base branch from las3-develop to master July 8, 2020 05:53
@kinverarity1 kinverarity1 marked this pull request as ready for review July 8, 2020 07:45
@dcslagel
Copy link
Collaborator

dcslagel commented Jul 8, 2020

Rebase findings:

  • I wasn't able to make a rebase branch that you could merge to rearrange-reader and then to master without conflicts. So not posting a rebase-branch pull-request
  • There are just 2 conflicts
    • tests/test_read.py (Take the master branch changes)
    • tests/test_read_30.py (Take the rearrange-reader changes)
  • Reviewing the code after rebasing everything looked okay.
  • You should see good post-rebase test-coverage results. This is what my local post-rebase results are:
    220 passed, 3 skipped
Name                       Stmts   Miss  Cover
----------------------------------------------
lasio/__init__.py             13      2    85%
lasio/convert_version.py      20     20     0%
lasio/defaults.py             11      0   100%
lasio/examples.py             42     10    76%
lasio/excel.py                88     34    61%
lasio/exceptions.py            6      0   100%
lasio/las.py                 376     55    85%
lasio/las_items.py           190     31    84%
lasio/las_version.py          34     22    35%
lasio/reader.py              437     84    81%
lasio/writer.py              160      9    94%
----------------------------------------------
TOTAL                       1377    267    81%

@dcslagel
Copy link
Collaborator

dcslagel commented Aug 2, 2020

Just checking on this pull-request. Is there additional tasks that should be done to enable this to merge?

@kinverarity1
Copy link
Owner Author

@dcslagel no thanks, all the hard work has been done now!! I just need some time to do it, should be this weekend.

@dcslagel
Copy link
Collaborator

dcslagel commented Aug 4, 2020

Okay, Great. :-) Yeah, I thought you might be busy but I wanted to check incase there was something we could unblock. :-)

@kinverarity1 kinverarity1 merged commit 9af056e into master Sep 4, 2020
@kinverarity1 kinverarity1 deleted the rearrange-reader branch May 13, 2022 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
las3 stuff relating to LAS 3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants