Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can not input non-latin characters in REPL #32

Closed
ghost opened this issue May 2, 2016 · 6 comments
Closed

Can not input non-latin characters in REPL #32

ghost opened this issue May 2, 2016 · 6 comments

Comments

@ghost
Copy link

ghost commented May 2, 2016

I have no idea if ChezScheme's REPL support unicode. Is there any good way to solve this problem now?

@dybvig
Copy link
Member

dybvig commented May 2, 2016

The expression editor uses 8-bit characters and 8-bit ncurses functions. It might not be too challenging to modify it to use wide characters and the wide ncursesw functions, though it might take more than cosmetic changes. If you disable the expression editor (`scheme --eedisable``) Unicode should already work in a terminal emulator that supports it.

@eraserhd
Copy link
Contributor

I've spent a bunch of time today reading code, and I think I have a straightforward way of doing this, but I wanted to check here first. There are three parts to it.

  1. Write characters to the console using wctombs, which will respect the current locale (and should output utf8 in anything modern).
  2. Read characters from the console, covering with mbrtowc() tracking an mbstate_t in static variables in s_ee_read_char() (the windows code does something similar), with care to respect the blocking flag. This will require handling a few different cases (like EWOULDBLOCK in the middle of a Unicode sequence).
  3. Make the key mapping in expeditor.ss allow characters over 255. (We could survive without this, but seems like a good idea.) ???

None of this handles Unicode character composition, but it would allow all precomposed characters, emoji, and what not. AFAICT, character composition would require retrofitting a significant about of math in expeditor.ss.

So, before I embark, is this a decent approach? Can you think of anything specific I'll need to address to get this code accepted?

@dybvig
Copy link
Member

dybvig commented Jul 18, 2016

Have you looked into using the "wide" version of ncurses (ncursesw)? If it works, it should simplify the input side; plus I'm not sure the non-wide ncurses will even deliver multibyte input characters.

@eraserhd
Copy link
Contributor

@dybvig I did, but I found this which suggested a different approach.

It looks like I have a bug to work out, but I'll push what I have, which is mostly working for review.

@dybvig
Copy link
Member

dybvig commented Jul 19, 2016

Thanks for the pointer to the comment. I had forgotten curses getch.required that we go full screen.

eraserhd added a commit to eraserhd/ChezScheme that referenced this issue Jul 21, 2016
… work

  except that combining characters are not treated correctly for
  line-wrapping.  this addresses github issue cisco#32 and part of issue cisco#81.
    c/expeditor.c, s/expeditor.ss
@dybvig
Copy link
Member

dybvig commented Oct 11, 2017

I believe this has since been fixed.

@dybvig dybvig closed this as completed Oct 11, 2017
spall pushed a commit to spall/ChezScheme that referenced this issue May 25, 2021
… work

  except that combining characters are not treated correctly for
  line-wrapping.  this addresses github issue cisco#32 and part of issue cisco#81.
    c/expeditor.c, s/expeditor.ss
mflatt pushed a commit to mflatt/ChezScheme that referenced this issue Oct 10, 2023
… work

  except that combining characters are not treated correctly for
  line-wrapping.  this addresses github issue cisco#32 and part of issue cisco#81.
    c/expeditor.c, s/expeditor.ss

Original commit: 87d4811
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants