Skip to content

Unable to position the cursor in the editor using braille routing keys in NVDA #27216

Closed
@bramd

Description

@bramd
  • VSCode Version: Code 1.12.2 (19222cd, 2017-05-10T13:20:36.315Z)
  • OS Version: Windows_NT ia32 10.0.15063

Steps to Reproduce:

  1. Open a file in the editor
  2. Ensure NVDA is in focus mode (for example by pressing space/enter on the editor)
  3. Use a cursor routing button on a connected braille display and click on a character in the text

Expected:

The cursor moves to the chosen position.

Actual:

Cursor doesn't move in the editor, but the cursor is displayed on the right position in braille in the textarea used for editor accessibility until it refreshes (e.g. by moving cursor using the keyboard).

Activity

added
accessibilityKeyboard, mouse, ARIA, vision, screen readers (non-specific) issues
on May 29, 2017
alexdima

alexdima commented on May 29, 2017

@alexdima
Member

@bramd Do you know if there is any way to simulate what NVDA does in this case without having a real braille display?

added
bugIssue identified by VS Code Team member as probable bug
on May 30, 2017
bramd

bramd commented on Jun 2, 2017

@bramd
Author

@alexandrudima I had a look in the NVDA source and it's not clear to me what NVDA does exactly. I guess it sends messages through the winapi or another API to instruct the textarea to move the cursor. I didn't catch any useful events with accevent and the default settings. Probably @jcsteh knows?

If you know another way to catch the events/messages being sent, I'm glad to provide a log.

jcsteh

jcsteh commented on Jun 5, 2017

@jcsteh

NVDA calls IAccessible2::setCaretOffset. I believe this is supposed to be reflected as a selectionChange DOM event. However, I think this may have only been implemented correctly in a fairly recent Chromium version, though I can't find a reference for this. Furthermore, MDN seems to suggest this is only supported for contentEditable in Chrome, not for input/textarea, though this could be outdated information. (I can't remember whether VS Code uses contentEditable or textarea, but vaguely recall textarea.)

zersiax

zersiax commented on Jun 12, 2017

@zersiax

After doing some testing, this most likely is a Chromium issue. I had a play with Postman running in a Chrome Stable instance and ran into the same issue, haven't tried any experimental builds though. Looking at the behavior of the braille display it seems the accessibility API registers the event and acts on it, but doesn't actually properly move the caret.

jcsteh

jcsteh commented on Jun 12, 2017

@jcsteh

@zersiax commented on 13 Jun 2017, 04:26 GMT+10:

After doing some testing, this most likely is a Chromium issue. I had a play with Postman running in a Chrome Stable instance and ran into the same issue

Don't know anything about Postman, but assuming it does custom management of the caret, even if Chrome does fire the selectionChange DOM event, Postman would need to support this event and move its internal idea of the caret accordingly.

I confirmed that VS Code uses textarea. I also confirmed that Chrome (59) does fire selectionChange for textarea:

data:text/html,<textarea>foo</textarea><script>document.addEventListener("selectionchange", function() {alert('selectionChange');});</script>

If I call IAccessibleText::setCaretOffset (as will happen when you press a cursor routing key on a braille display), I get the alert.

added this to the On Deck milestone on Jun 13, 2017
zersiax

zersiax commented on Jul 19, 2017

@zersiax

For braille users this is a bit of a productivity hog, do we know how we want to proceed on this one yet?

ushiushix

ushiushix commented on Jul 19, 2017

@ushiushix

I use braille a lot especially for programming. I tried to use Code as my primary editor for Windows, but this issue blocked me.
Capability of jumping the cursor to the touching position immediately is the biggest advantage of braille displays.

alexdima

alexdima commented on Aug 8, 2017

@alexdima
Member

I have briefly investigated listening to the browser event selectionchange. I received it all the time from the textarea, even when clicking around in VS Code or when alt+tabbing, so I cannot add some code that always resets the editor's cursor to the textarea caret when this event is received.

So I would need to write some filtering condition to figure out if this event is coming in due to the use of a Braille display and if we should react to it or not.

The problem I ran into is that I don't know at this time how to emulate/simulate a Braille display caret change i.e. as @jcsteh mentions, the IAccessible2::setCaretOffset call NVDA does, to try to discriminate the false selectionchange events from the ones we should react to.

I can poke around at building NVDA from source and try to get it to call IAccessible2::setCaretOffset or I could create a special version of VS Code where I log all the incoming selectionchange events and then ask you for a trace.

jcsteh

jcsteh commented on Aug 8, 2017

@jcsteh

13 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

accessibilityKeyboard, mouse, ARIA, vision, screen readers (non-specific) issuesbugIssue identified by VS Code Team member as probable bugverifiedVerification succeeded

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Unable to position the cursor in the editor using braille routing keys in NVDA · Issue #27216 · microsoft/vscode