Accessibility: NVDA reading wrong or duplicate content in editor

[zersiax]

• VSCode Version: Latest Insider
• OS Version: OS build 15063

Steps to Reproduce:

1. Open a file with code in it, the type does not matter.
2. Arrow through the file at a rate of about twice per second, while NVDA is running.

Observed Behavior:
Rather than speaking the focused line, NVDA will at times read the line or even several lines above it.
Interestingly, braille output appears to work well enough.

Expected Behavior:
NVDA should speak the correct line when navigating through code, for a lot of people using screen readers do not use braille displays to compensate.

Suspected cause:
The fact that braille works well and speech does not leads me to believe it takes too long for the focused line to be visible for NVDA. I don't know enough about the VS Code internals to comment further.

[Neurrone]

I can reproduce this on the latest stable as well with NVDA.

[zersiax]

I have heard from several NVDA users getting hit by this, some going as far as saying this bug renders the IDE impossible to use effectively

[Neurrone]

@zersiax yes, its impossible to use, notepad would be better because of this bug.

Its like having the screen not displaying lines as they properly appear in the file.

[alexdima]

The editor text accessibility is implemented by using a textarea (the same one we use for input events), where we mirror the buffer text and the selection. Until last month (when we updated to a newer Electron version), it was not possible for us to discern if a screen reader is attached or not, so at this point we always do the following:

• the textarea only gets a "page" of the buffer. e.g. if the cursor is in lines 1-10, only lines 1-10 are placed in the textarea. If the cursor is in lines 11-20, only lines 11-20 are placed in the textarea. We are doing this because when opening large files (more than 1 MB), Chromium stalls for seconds if we write large quantities of text to the textarea. This might be a cause of confusion for screen readers when going from line 10 to line 11, for example, where all of a sudden we move to the next page.

• we write to the textarea with the same code path that we paint the lines, i.e. at the next browser animation frame (i.e. not synchronously in the handling of the keyboard event). If two key presses occur before Chromium calls us back it is possible we will update the textarea only once. This might also be a cause of confusion for screen readers.

I will investigate to adopt the new available Electron API to detect when a screen reader is attached and attempt to remove these two possible causes.

[Neurrone]

@alexandrudima thanks. Probably a stupid question, but aren't there standard html controls that can be used for multi-line edits?

[alexdima]

@Neurrone The standard html control that is used for multi-line text is a <textarea>.

[alexdima]

I can confirm and reproduce the problem even in a file with 10 lines (i.e. the "paging" does not appear to be the root cause):

• I create a file with 10 lines, with the content:

Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

• I place the cursor at the beginning of the file and I press arrow down 9 times
• Sometimes, the same line is pronounced multiple times by NVDA. e.g. of pronounced text:

Line 1
Line 1
Line 2
Line 3
Line 5
Line 5
Line 6
Line 8
Line 9

I'll try next a change where we call setSelectionRange in the <textarea> synchronously from the keyboard event handler (instead of from the animation frame).

[alexdima]

Unfortunately, calling setSelectionRange synchronously from the keyboard event handler makes no difference.

[alexdima]

Also, this is something I can reproduce on Chrome v58 (via the standalone editor running from master and via the standalone editor from January at https://microsoft.github.io/monaco-editor/)

[Neurrone]

@jcsteh any ideas?

[alexdima]

Ok. I have spent around 3hrs reading through Chromium source code, looking at every setter or method on the <textarea>, and then I finally used AccEvent.exe and friends to try and understand what is so different between when moving the cursor in a regular textarea (which NVDA handles well) vs. moving the cursor in VS Code (where these glitches occur).

AccEvent.exe stopped working at one point (presumably I need to reboot), but I've found Chrome producing a whole lot more events when moving the cursor in VS Code compared to when moving the cursor in a regular textarea. When moving the cursor in a regular textarea, Chrome produces precisely one event, while when moving the cursor in VS Code, Chrome produces around 7.

One of those events appeared to contain the status bar text for "Ln 1, Col 1", etc. I think Chrome sends out events for every DOM change we do.

I have made some local changes to VS Code such that:

• the status bar line number, column is never rendered
• selection highlighting is off
• the cursor div is never rendered
• the line highlight is never rendered

This makes it that Chromium sends precisely one event that shows up in AccEvent.exe when pressing arrow down, similar to when pressing arrow down in a regular textarea... And now NVDA works flawlessly.

It is my belief that NVDA gets confused by all the events Chromium generates as we update various parts of the DOM on cursor move.

[alexdima]

Even having developer tools opened in Chrome where console log messages go (which presumably also sends some accessibility events) seems to cause the NVDA issue to reproduce.

[alexdima]

IMHO something is fundamentally broken in Chromium or in NVDA. Even when I change all our code on the key handling path in such a way that the only Accessibility Event coming out of Chromium matches the one Chromium sends from a native textarea, sometimes (although a lot less often, possibly every 30-40 lines) NVDA still skips a line or reads the same line twice. I believe this is timing related, as the same line, under the same conditions sometimes gets read twice, sometimes gets skipped, and sometimes gets read correctly.

I can also trigger the same behaviour in a native Chrome textarea, when I place a large ammount of text inside it (and by large I actually mean a small ammount e.g. 59 KB). Chromium freezes noticeably when moving the cursor in such a textarea, while NVDA happily reads the same character or the same line until at one point they seem to catch up with one another.

[alexdima]

I will look into using an <iframe> + content editable instead of a <textarea>.

[alexdima]

Unfortunately, using an <iframe> + content editable makes no difference, here is my theory of what goes wrong:

• NVDA possibly sees that arrow down or arrow left is pressed
• if Chromium is quick and emits a mysterious 11B event type fast enough, NVDA reads the line or character correctly.
• if Chromium is not fast enough, NVDA will read the same line or the same character twice and effectively skip a line or character when repeating the key press.

I can reproduce in Google Docs with Braille enabled and in a standalone HTML that has a <textarea> with 60 KB of text. I will file issues to both Chromium and NVDA, to get their input.

On our side, we can look into supporting two accessibility modes (like Google Docs does -- one where Braille is enabled and one where we use alerts for screen readers).

My iframe MVP changes are saved on https://github.com/alexandrudima/vscode/tree/iframe-io

[alexdima]

Upstream issue: https://bugs.chromium.org/p/chromium/issues/detail?id=725051

[zersiax]

@alexandrudima Good work so far. I saw you committed the changes you made with the textarea approach which should at least minimize this behavior, is that a solution we can roll with for now until the possible Chromium or NVDA issues that might be contributing to this get fixed?

[Neurrone]

@alexandrudima does having braille mode turned off make a difference? Thanks for investigating :)

[zersiax]

Also this might possibly have a similar root cause, but when you arrow through the file explorer you pretty consistently get the file you departed from followed by the file you arrive at. So, say you have a file readme.md followed by structure.md, when you arrow down from readme.me to get to structure.md, the following is read: "readme.md, structure.md, file explorer level 2". Sounds like we've getting too many accessibility events being fired there as well. This is markedly less annoying but does hamper productivity if you use the file browser a lot. I tend to use fuzzy search more myself but figured I'd report.

[alexdima]

Adding @bpasero for the explorer.

[Neurrone]

I've installed the add-on and the global plugin that bumps the wait time up to 200ms.

A possibly related problem: if a long line of text is wrapped across multiple lines, NVDA incorrectly reads the entire line when arrowing through. For example, this section is right at the top of the default settings.json file

// Controls auto save of dirty files. Accepted values:  'off', 'afterDelay', 'onFocusChange' (editor loses focus), 'onWindowChange' (window loses focus). If set to 'afterDelay', you can configure the delay in 'files.autoSaveDelay'.

NVDA reads the entire 3-line block when arrowing through each line.

Also when in the editor for files in readonly mode, pressing up arrow when on the first line unexpectedly moves you outside the editor. Down arrows will also sometimes move you outside the editor as well.

[bpasero]

I pushed a change for trees in VS Code to no longer produce duplicate events. Turns out we were changing the aria-label always when selection changes. Now we apply aria-label on all elements directly and do not update it based on selection. That seems to fix it for me.

Interestingly the list widget never had this bug and in there we are not updating aria-label based on selection. In fact, list widget never used aria-label but just relied on the screen reader computing the label properly (which seems to work OK).

@joaomoreno fyi I think in the SCM world we however lost the ability to read back additional meta information to screen readers when navigating through outgoing changes. I think we used to announce the type of change for each item in the tree.

[derekriemer]

These are well known chrome bugs.

…

On Wed, May 24, 2017 at 8:36 PM, Dickson Tan ***@***.***> wrote: I've installed the add-on and the global plugin that bumps the wait time up to 200ms. A possibly related problem: if a long line of text is wrapped across multiple lines, NVDA incorrectly reads the entire line when arrowing through. For example, this section is right at the top of the default settings.json file // Controls auto save of dirty files. Accepted values: 'off', 'afterDelay', 'onFocusChange' (editor loses focus), 'onWindowChange' (window loses focus). If set to 'afterDelay', you can configure the delay in 'files.autoSaveDelay'. NVDA reads the entire 3-line block when arrowing through each line. Also when in the editor for files in readonly mode, pressing up arrow when on the first line unexpectedly moves you outside the editor. Down arrows will also sometimes move you outside the editor as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26730 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFGivZx7xu6weuU8z4pKwsf5B3Die3bcks5r9OkTgaJpZM4NcLUm> .

-- Derek Riemer: Improving the world one byte at a time! - University of Colorado Boulder Department of computer science, 4th year undergraduate student. - Accessibility enthusiast. - Proud user of the NVDA screen reader. - Open source enthusiast. - Skier. Personal website <http://derekriemer.com>

[Neurrone]

That's terrible and quite surprising, since its been around for several years now. Guess I'll be sticking to Firefox for the foreseeable future, and will have to wait for upstream fixes.

[jcsteh]

@Neurrone commented on May 25, 2017, 12:35 PM GMT+10:

A possibly related problem: if a long line of text is wrapped across multiple lines, NVDA incorrectly reads the entire line when arrowing through. ...
Also when in the editor for files in readonly mode, pressing up arrow when on the first line unexpectedly moves you outside the editor. Down arrows will also sometimes move you outside the editor as well.

@derekriemer commented on May 25, 2017, 11:35 PM GMT+10:

These are well known chrome bugs.

On the contrary, I can't reproduce either of these issues in Chrome using the long line that @Neurrone provided, either with a textarea or a contentEditable div.
Test case (rename to .html)

[Neurrone]

@jcsteh perhaps the version of VS code I'm using (latest stable) uses an older version of chrome?

[alexdima]

Since this is becoming an uber-issue, let me try to sum up and split it into other issues where necessary:

Issue from @zersiax : Navigating with cursor keys in the editor results in NVDA reading the same character or the same line twice

Changes on our side: we have done a full review of all the stuff (stepping through the ~25 listeners) that executes in the cursor movement case and have reduced the time spent in our code to ~1-2ms. As part of this 2ms we also immediately move the cursor in the <textarea> such that Chromium will emit a cursor position change accessibility event sooner, not at the next animation frame. These ~2ms spent on our side sometimes result in the event being emitted within the 30ms NVDA timeout, sometimes not, but this is outside of our control.

Solution: The plugin provided by @derekriemer that increases NVDA's timeout to 200ms seems to completely resolve the issue. Furthermore, thanks to @jcsteh there is an open PR in NVDA where the default timeout is increased from 30ms to 100ms.

---

Issue from @zersiax : Navigating with the cursor keys in the explorer tree results in too many accessibility events and too many items are read

Solution: @bpasero has pushed a fix where the aria labels on tree elements are no longer modified in the selection change case.

---

Issues from @Neurrone :

Arrow down in word wrapped editor ends reading the entire line possibly multiple times: extracted to #27766 - I can reproduce

Up/Down arrow in a read only editor sometimes moves you outside the editor #27765 - I still need to reproduce and confirm.

---

I think those 4 issues sum up this thread, since we continue the issues from @Neurrone in two separate threads, I propose to close this one and celebrate our progress. ❤️

@derekriemer One last ask: Would it be possible for you to host the plugin that increases the timeout to 200ms on GitHub or something similar. I would like to document that we recommend folks install this plugin when working with NVDA and VS Code and I would like to link to it from our website and from our release notes. I wouldn't want this to end up causing too much traffic costs or something... Thank you !

[derekriemer]

Sure, Where should I contact you guys when I do? I don't mind it being hosted by my site, I have the bandwidth and can handle it, but github isn't my server admin, and is more likely to suffer no downtime. I'll post it later this week.

…

On Wed, May 31, 2017 at 11:59 AM, Alexandru Dima ***@***.***> wrote: Closed #26730 <#26730>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26730 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFGivUm2ziooG8T2Zx333MyambcaD_jIks5r_aqCgaJpZM4NcLUm> .

-- Derek Riemer: Improving the world one byte at a time! - University of Colorado Boulder Department of computer science, 4th year undergraduate student. - Accessibility enthusiast. - Proud user of the NVDA screen reader. - Open source enthusiast. - Skier. Personal website <http://derekriemer.com>

[alexdima]

@derekriemer That would be great! Another comment here would work great ❤️.

[Neurrone]

Using the latest insider build and the add-on, am experiencing inaccurate reporting of the current word when moving by word using ctrl+right/left arrow. Still see this issue as well with up/down arrows, also by character.

Are the changes made on VS Code's end not included in the insiders build yet?

[alexdima]

@Neurrone The latest insiders contains all the relevant changes. Let's start a new thread/issue with your detailed setup. Could you please include:

• do you have any other installed NVDA add-ons that might interfere? (I don't have sufficient knowledge on this side to tell if there might be some interference going on)
• do you experience this on a particular file, or on all files (even small files)?
• do you experience this around selecting large blocks of text or simply around pressing the arrow keys?
• what does Alt+F1 report when you encounter this problem?

[Neurrone]

I've restarted NVDA with all add-ons disabled and am still encountering this. I'm experimenting with the read-only settings file and the user settings file.

Am just pressing the left/right arrow keys (arrows, ctrl + left/right arrow to move by word, etc). Notice this especially when you go right/left, then change direction.

Not sure what alt+f1 says, since i can't find the message with NVDA.

[bramd]

Switch to browse mode (nvda+space) to read the message after pressing alt+f1. I think that message might need an ARIA live region to be read automatically.

[alexdima]

@Neurrone Let's continue in a fresh issue, #28306.
@bramd I have extracted your comment to #28308.

[derekriemer]

Hi: Sorry, I never published the plugin. As of the last NVDA version (2017.3), it now has a cursor timeout of 100 MS.