Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CR's being stripped from multi-line fields? #731

Closed
4 of 13 tasks
chrisjlocke opened this issue Aug 15, 2016 · 11 comments
Closed
4 of 13 tasks

CR's being stripped from multi-line fields? #731

chrisjlocke opened this issue Aug 15, 2016 · 11 comments
Assignees
Labels
bug Confirmed bugs or reports that are very likely to be bugs. response requested

Comments

@chrisjlocke
Copy link
Member

chrisjlocke commented Aug 15, 2016

Details for the issue

When entering multi-line data in DB4S, the data is changed so that CRLF is changed to LF.
Possibly.

Please see this screencast.
http://screencast-o-matic.com/watch/cDjo29iM8o

I enter 123 456 789 (split on 3 lines) from my Windows program. This loads and saves fine. DB4S shows this as 3 lines. If I add a further line, 000, my program now sees it as one string. HOWEVER, if I press the right arrow at a set interval (not easy to pick up as there is no audio!) you can see the cursor 'stick' where the new lines should be - the textbox is seeing a LF but being Windows, doesn't know what to do with it.

Exporting the field to a text file and viewing that in Notepad++, shows LFs, not CRLFs.
Exporting the field the first time, showed CRLFs.
Notepad++ sees it as a 'Unix' file.
Viewing the field in DB4S correctly shows the 4 lines. Copying that to the clipboard correctly shows CRLFs. Its as if the data is wrong, but DB4S 'corrects' it.

image

Additional testing. Correcting the text in DB4S mangles it further. A mix of CRLF and LF. However, additional saving DOES NOT mangle it further (ie, editing does not compound the issue)
image

Useful extra information

I'm opening this issue because:

  • DB4S is crashing
  • DB4S has a bug
  • DB4S needs a feature
  • DB4S has another problem

I'm using DB4S on:

  • Windows
  • Linux: ( distro: ___ )
  • Mac OS
  • Other: ___

I'm using DB4S version:

  • 3.9.0 beta1
  • 3.8.0
  • Other: ___

I have also:

@justinclift
Copy link
Member

Ahhh, that's interesting. As a quick thought, would you be ok to try the 3.9.0 beta1 for Windows?

A lot of the copy-paste code was changed around just before release, and it seemed to pass all of the (rather basic) testing we did on it.

I didn't specifically check for CR/LF characters being kept unchanged though, so this bug _might_ still be present.

@chrisjlocke
Copy link
Member Author

Just tried the beta with the same results.
http://screencast-o-matic.com/u/hnsz/stripped1

In further testing, I exported the field to a text file, converted it back to CRLF, imported it, confirmed all the fields in DB4S were showing OK, and exported again to show back to LF.
http://screencast-o-matic.com/u/hnsz/stripped2

Just to confirm, as someone will probably say, "the data is OK, but the exporting is borked", but my application doesn't like the data in the database, so the raw data is LF, not CRLF. I've viewed the database in binary format and also confirmed this.
http://screencast-o-matic.com/watch/cDjDQmiLuY

One last screencast. Promise. I added 0d codes to the field so that it was od 0a (I understand thats binary for CRLF). Switched to text and saw the World was good. Switched back to binary, and SWITCHEROOO! Clang! Thanks DB4S! 😉
http://screencast-o-matic.com/watch/cDjDQLiLuz

I entered 0e by purpose just to see if it was stripping characters anyway, but then added a random 0d on its own, and that was converted too.

Its a bit naughty its having to do any 'conversion' or data manipulation anyway, isn't it? Surely if I want LFs or CRLFs, it should display that, as it must be there for a purpose. For it to decide, "hey, you've got this data in your database. I'll just tweak that for you..... " seems a bit scary.

@justinclift
Copy link
Member

Thanks Chris. 😄

The original screencast URL wasn't working for me at first yesterday, but seems to be now. So, the new ones will probably be workable too.

I'll try them out in a bit, I just need to finish some other things first. 😄

@justinclift justinclift added the bug Confirmed bugs or reports that are very likely to be bugs. label Aug 16, 2016
@justinclift justinclift self-assigned this Aug 16, 2016
@justinclift
Copy link
Member

Reading through this bit again:

One last screencast. Promise. I added 0d codes to the field so that it was od 0a (I understand thats binary for CRLF). Switched to text and saw the World was good. Switched back to binary, and SWITCHEROOO! Clang! Thanks DB4S! 😉
http://screencast-o-matic.com/watch/cDjDQLiLuz

Yep, that's super clear. It's a side effect of the data being stored in one format for the hex editor, then being converted to plain text for display in the text editor. (and vice versa)

The central reason is these editor views have their own buffers... they can't share, and they require different data formats.

But, it may be possible to fix this. It'll need some thinking about, and probably some experimentation to get right.

Almost no chance of this bit being fixed for 3.9.0, but the 3.10.0 release is possibly feasible. 😄

@chrisjlocke
Copy link
Member Author

Can't the viewers just view 'as is' rather than 'converting'? There may be a reason (especially if HTML code) for CRLFs and LFs. Thats the nice thing with the binary view - WYSIRWYG! (...really what you get!)

@justinclift
Copy link
Member

justinclift commented Aug 17, 2016

It'd be fantastic if it was that simple. 😁

For the hex editing view, that's the closest thing to what you describe. It's a view into a literal array of bytes. No weirdness involved.

The text editor though is one of these. It has it's own internal buffer which seems to be a very text oriented data type managed by Qt, and not a byte array. The conversion problem seems to be biting when going between these two., as these two widgets can't just share data. 😦

Well, that's my understanding of it so far. It might turn out there are ways to accomplish it. Just not sure at the moment.

In the worst case scenario, we might be able to subclass the QtTextEdit box that's used, to include some non-corrupting import/export functions. No idea yet...

All help is welcome though, if you're keen to dig into it. 😁

@chrisjlocke
Copy link
Member Author

backs slowly away... 😉 I was following you up to "simple"....

@justinclift
Copy link
Member

Heh Heh Heh 😁

@prutz1311
Copy link
Contributor

Would it be a good idea to always pass to EditDialog::loadData(const QByteArray& data) only the data from the database cell instead of taking it from some other buffer when changing editing mode? Though that would introduce questions like how to deal with unsaved data, i.e. when Apply has not been pressed.

@mgrojo
Copy link
Member

mgrojo commented Apr 26, 2020

@chrisjlocke Is this solved with 3.12 alpha? Sorry, I haven't read it all. The new version uses QScintilla for the cell editor and should leave control characters as they are. This is supposed to be solved with #1793 (but, sorry, could not review it all the answers).

@mgrojo
Copy link
Member

mgrojo commented Aug 5, 2024

I guess we can close this, as it seems fixed by #1793 and we also have #2218 for missing pieces.

@mgrojo mgrojo closed this as completed Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Confirmed bugs or reports that are very likely to be bugs. response requested
Projects
None yet
Development

No branches or pull requests

4 participants