Vision Enhancement Providers

Learn how to develop vision enhancement providers for NVDA

Overview

Vision enhancement providers add visual highlighting and other visual aids to NVDA, helping low-vision users track the focus, navigator object, and other elements on screen. Providers can draw highlights, magnify content, or apply color filters.

Architecture

Base Class

All vision providers inherit from vision.providerBase.VisionEnhancementProvider, which provides:

• Event registration framework
• Settings management
• Lifecycle management (initialization/termination)
• Configuration persistence

Provider Settings

Providers also define a settings class inheriting from VisionEnhancementProviderSettings:

from vision.providerBase import VisionEnhancementProviderSettings
from autoSettingsUtils.driverSetting import BooleanDriverSetting

class NVDAHighlighterSettings(VisionEnhancementProviderSettings):
    @classmethod
    def getId(cls) -> str:
        """Unique ID for this provider (used in config file)."""
        return "NVDAHighlighter"
    
    @classmethod
    def getDisplayName(cls) -> str:
        """Name shown in NVDA's vision settings."""
        return _("NVDA Highlighter")
    
    def _get_supportedSettings(self) -> list:
        """List of configurable settings."""
        return [
            BooleanDriverSetting(
                "highlightFocus",
                _("Highlight focus"),
                defaultVal=True,
            ),
            BooleanDriverSetting(
                "highlightNavigator",
                _("Highlight navigator object"),
                defaultVal=True,
            ),
        ]

Creating a Basic Provider

Minimal Provider Structure

from vision.providerBase import VisionEnhancementProvider, VisionEnhancementProviderSettings
from vision.visionHandlerExtensionPoints import EventExtensionPoints

class ProviderSettings(VisionEnhancementProviderSettings):
    @classmethod
    def getId(cls) -> str:
        return "exampleProvider"
    
    @classmethod
    def getDisplayName(cls) -> str:
        return "Example Vision Provider"
    
    def _get_supportedSettings(self) -> list:
        return []

class VisionEnhancementProvider(VisionEnhancementProvider):
    @classmethod
    def getSettings(cls):
        return ProviderSettings()
    
    @classmethod
    def canStart(cls) -> bool:
        """Check if provider can be initialized."""
        return True
    
    def registerEventExtensionPoints(self, extensionPoints: EventExtensionPoints):
        """Register for NVDA events."""
        extensionPoints.post_focusChange.register(self.handleFocusChange)
    
    def handleFocusChange(self, obj, nextHandler):
        """Called when focus changes."""
        # Do something visual
        nextHandler()
    
    def terminate(self):
        """Clean up resources."""
        pass

Full Implementation: NVDA Highlighter

The built-in NVDA Highlighter is a complete example showing all key features:

Provider Initialization

import threading
import wx
from typing import Dict
from vision.providerBase import VisionEnhancementProvider
from vision.constants import Context
from locationHelper import RectLTRB
from colors import RGB
import winGDI

class HighlightStyle:
    """Style definition for highlights."""
    def __init__(self, color: RGB, width: int, style: int, margin: int):
        self.color = color
        self.width = width
        self.style = style  # DashStyle: Solid, Dash, etc.
        self.margin = margin

# Predefined styles
BLUE = RGB(0x03, 0x36, 0xFF)
PINK = RGB(0xFF, 0x02, 0x66)
YELLOW = RGB(0xFF, 0xDE, 0x03)

SOLID_BLUE = HighlightStyle(BLUE, 5, winGDI.DashStyleSolid, 5)
SOLID_PINK = HighlightStyle(PINK, 5, winGDI.DashStyleSolid, 5)
SOLID_YELLOW = HighlightStyle(YELLOW, 2, winGDI.DashStyleSolid, 2)
DASH_BLUE = HighlightStyle(BLUE, 5, winGDI.DashStyleDash, 5)

class NVDAHighlighter(VisionEnhancementProvider):
    def __init__(self):
        super().__init__()
        
        # Store rects for each context
        self.contextToRectMap: Dict[Context, RectLTRB] = {}
        
        # Store which contexts are enabled
        self.enabledContexts: set[Context] = set()
        
        # Create highlight window
        self._highlightWindow = HighlightWindow(self)
        
        # Load settings
        settings = self.getSettings()
        if settings.highlightFocus:
            self.enabledContexts.add(Context.FOCUS)
        if settings.highlightNavigator:
            self.enabledContexts.add(Context.NAVIGATOR)

Event Registration

def registerEventExtensionPoints(self, extensionPoints: EventExtensionPoints):
    """Register for vision-related events."""
    # Focus tracking
    extensionPoints.post_focusChange.register(self.handleFocusChange)
    
    # Navigator object tracking
    extensionPoints.post_reviewMove.register(self.handleReviewMove)
    
    # Browse mode caret tracking
    extensionPoints.post_browseModeMove.register(self.handleBrowseModeMove)
    
    # Mouse tracking
    extensionPoints.post_mouseMove.register(self.handleMouseMove)

def handleFocusChange(self, obj, nextHandler):
    """Update focus highlight when focus changes."""
    if Context.FOCUS in self.enabledContexts:
        try:
            location = obj.location
            if location:
                self.contextToRectMap[Context.FOCUS] = RectLTRB.fromCompatibleType(location)
                self._highlightWindow.refresh()
        except Exception:
            log.error("Error updating focus highlight", exc_info=True)
    
    nextHandler()

def handleReviewMove(self, obj, nextHandler):
    """Update navigator object highlight."""
    if Context.NAVIGATOR in self.enabledContexts:
        try:
            location = obj.location
            if location:
                self.contextToRectMap[Context.NAVIGATOR] = RectLTRB.fromCompatibleType(location)
                self._highlightWindow.refresh()
        except Exception:
            log.error("Error updating navigator highlight", exc_info=True)
    
    nextHandler()

Highlight Window Implementation

The highlight window is a transparent, topmost window that draws rectangles:

Window Creation

from windowUtils import CustomWindow
import winUser
import winBindings.gdi32
from winAPI.messageWindow import WindowMessage

class HighlightWindow(CustomWindow):
    className = "NVDAHighlighter"
    windowName = "NVDA Highlighter Window"
    transparency = 0xFF
    transparentColor = 0  # Black
    
    # Window style for non-interactive, always-on-top transparent window
    windowStyle = winUser.WS_POPUP | winUser.WS_DISABLED
    extendedWindowStyle = (
        winUser.WS_EX_TOPMOST |        # Always on top
        winUser.WS_EX_LAYERED |         # Supports transparency
        winUser.WS_EX_NOACTIVATE |      # Can't be activated
        winUser.WS_EX_TRANSPARENT |     # Mouse clicks pass through
        winUser.WS_EX_TOOLWINDOW        # Not in taskbar
    )
    
    def __init__(self, highlighter):
        super().__init__(
            windowName=self.windowName,
            windowStyle=self.windowStyle,
            extendedWindowStyle=self.extendedWindowStyle,
        )
        
        self.highlighterRef = weakref.ref(highlighter)
        self._prevContextRects: Dict[Context, RectLTRB] = {}
        
        # Set up layered window with transparency
        winUser.SetLayeredWindowAttributes(
            self.handle,
            self.transparentColor,
            self.transparency,
            winUser.LWA_ALPHA | winUser.LWA_COLORKEY,
        )
        
        # Size to cover all displays
        self.updateLocationForDisplays()

Drawing Highlights

def _paint(self):
    """Paint the highlight rectangles."""
    highlighter = self.highlighterRef()
    if not highlighter:
        return
    
    # Begin painting
    hdc, ps = winUser.beginPaint(self.handle)
    
    try:
        # Create GDI+ graphics object
        graphics = winGDI.GdipCreateFromHDC(hdc)
        
        # Set high quality rendering
        winGDI.GdipSetSmoothingMode(graphics, winGDI.SmoothingModeAntiAlias)
        
        # Get rectangles to draw
        contextRects = self._getDrawRects(highlighter)
        
        # Draw each context
        for context, rect in contextRects.items():
            style = self._getStyleForContext(context)
            self._drawRect(graphics, rect, style)
        
        # Clean up
        winGDI.GdipDeleteGraphics(graphics)
    
    finally:
        winUser.endPaint(self.handle, ps)

def _drawRect(self, graphics, rect: RectLTRB, style: HighlightStyle):
    """Draw a single highlight rectangle."""
    # Apply margin
    rect = RectLTRB(
        rect.left - style.margin,
        rect.top - style.margin,
        rect.right + style.margin,
        rect.bottom + style.margin,
    )
    
    # Create pen
    pen = winGDI.GdipCreatePen(
        style.color.toGdiPlusARGB(),
        style.width,
    )
    
    # Set pen style
    winGDI.GdipSetPenDashStyle(pen, style.style)
    
    # Draw rectangle
    winGDI.GdipDrawRectangle(
        graphics,
        pen,
        rect.left,
        rect.top,
        rect.width,
        rect.height,
    )
    
    # Clean up
    winGDI.GdipDeletePen(pen)

def _getStyleForContext(self, context: Context) -> HighlightStyle:
    """Get the highlight style for a context."""
    if context == Context.FOCUS:
        return SOLID_BLUE
    elif context == Context.NAVIGATOR:
        return SOLID_PINK
    elif context == Context.FOCUS_NAVIGATOR:
        return DASH_BLUE  # When focus and navigator overlap
    elif context == Context.BROWSEMODE:
        return SOLID_YELLOW
    else:
        return SOLID_BLUE

Advanced Features

Multi-Monitor Support

def updateLocationForDisplays(self):
    """Size window to cover all displays."""
    displays = [wx.Display(i).GetGeometry() for i in range(wx.Display.GetCount())]
    
    # Calculate total screen dimensions
    minX = min(d.x for d in displays)
    minY = min(d.y for d in displays)
    maxX = max(d.x + d.width for d in displays)
    maxY = max(d.y + d.height for d in displays)
    
    screenWidth = maxX - minX
    screenHeight = maxY - minY - 1  # -1 to avoid desktop shortcut issue
    
    # Position and size the window
    user32.SetWindowPos(
        self.handle,
        winUser.HWND_TOPMOST,
        minX, minY,
        screenWidth, screenHeight,
        winUser.SWP_NOACTIVATE,
    )

Efficient Repainting

def refresh(self):
    """Refresh only the regions that changed."""
    highlighter = self.highlighterRef()
    if not highlighter:
        return
    
    # Get current and previous rectangles
    currentRects = self._getDrawRects(highlighter)
    prevRects = self._prevContextRects
    
    # Calculate dirty regions
    dirtyRegions = set()
    
    # Rectangles that changed or disappeared
    for context in set(prevRects.keys()) | set(currentRects.keys()):
        prevRect = prevRects.get(context)
        currentRect = currentRects.get(context)
        
        if prevRect != currentRect:
            if prevRect:
                dirtyRegions.add(prevRect)
            if currentRect:
                dirtyRegions.add(currentRect)
    
    # Invalidate dirty regions
    for rect in dirtyRegions:
        winRect = rect.toWinRECT()
        user32.InvalidateRect(self.handle, byref(winRect), True)
    
    # Update cached rects
    self._prevContextRects = currentRects.copy()

Settings Integration

from autoSettingsUtils.driverSetting import BooleanDriverSetting

class NVDAHighlighterSettings(VisionEnhancementProviderSettings):
    def _get_supportedSettings(self):
        return [
            BooleanDriverSetting(
                "highlightFocus",
                _("Highlight &focus"),
                defaultVal=True,
            ),
            BooleanDriverSetting(
                "highlightNavigator",
                _("Highlight &navigator object"),
                defaultVal=True,
            ),
            BooleanDriverSetting(
                "highlightBrowseMode",
                _("Highlight &browse mode cursor"),
                defaultVal=True,
            ),
        ]
    
    # Property getters/setters
    highlightFocus: bool
    highlightNavigator: bool
    highlightBrowseMode: bool

Testing Your Provider

1. Create provider file

   Place your provider in source/visionEnhancementProviders/yourprovider.py

2. Restart NVDA

   Restart NVDA or reload plugins

3. Enable provider

   Open NVDA Settings > Vision and enable your provider

4. Test functionality

   • Verify visual effects appear correctly
   • Test with focus changes
   • Test with navigator object movement
   • Verify multi-monitor support
   • Test settings changes take effect

Best Practices

Performance

Keep drawing operations efficient - highlights update frequently during navigation

Thread Safety

All drawing must happen on the GUI thread. Use wx.CallAfter if needed

Resource Cleanup

Always release GDI/GDI+ resources and unregister from event extension points

Error Handling

Handle exceptions gracefully - don’t crash NVDA if drawing fails

Vision providers run continuously and can impact performance. Optimize drawing operations and avoid memory leaks.

Common Issues

Highlights Don’t Appear

• Verify window is created and sized correctly
• Check that layered window attributes are set
• Ensure transparency is configured properly
• Verify GDI+ initialization succeeded

Highlights in Wrong Position

• Check coordinate system (screen vs client coordinates)
• Verify multi-monitor positioning logic
• Ensure location rectangles are valid
• Account for DPI scaling if needed

High CPU Usage

• Implement dirty region tracking
• Don’t redraw on every event
• Batch updates when possible
• Use efficient graphics operations

Highlights Not Updating

• Verify event extension points are registered
• Check that nextHandler() is always called
• Ensure InvalidateRect() is called appropriately
• Test that event handlers aren’t silently failing

Reference

Key Modules

• vision.providerBase: Base provider classes
• vision.visionHandlerExtensionPoints: Event extension points
• vision.constants: Context types and constants
• locationHelper: Rectangle helpers (RectLTRB, RectLTWH)
• winGDI: GDI+ drawing functions
• windowUtils: Custom window creation

Context Types

Providers can highlight different contexts:

• Context.FOCUS: Currently focused object
• Context.NAVIGATOR: Navigator object (object review)
• Context.BROWSEMODE: Browse mode caret position
• Context.FOCUS_NAVIGATOR: Focus and navigator overlap

Extension Points

Available extension points in EventExtensionPoints:

• post_focusChange: After focus changes
• post_reviewMove: After navigator object moves
• post_browseModeMove: After browse mode caret moves
• post_mouseMove: After mouse moves
• post_caretMove: After caret moves in edit fields

Example Providers

Study these providers in source/visionEnhancementProviders/:

• NVDAHighlighter.py: Full-featured highlighting provider
• _exampleProvider_autoGui.py: Example with dynamic settings

Developer Guide

See the NVDA Developer Guide for complete plugin development information