Enhance Accessibility and Usability with “Smart” Macros
About Smart Macros
“Dumb” vs. “Smart” macros
A typical ("dumb") macro is activated when the user presses a hotkey, clicks an object, or types a code. I have coined the term smart macro to describe a macro that is activated either by the user or the user interface (UI), and performs tasks based on feedback that the macro receives from the user, the UI, or both. A smart macro mediates between the user and the computer, makes decisions, and acts on the basis of information it receives from two sources. In contrast, a dumb macro is always activated by the user, cannot be controlled once it has been activated (except, perhaps, to halt its execution), and neither detects nor acts upon information from the UI.
Smart macros have a modicum of built-in intelligence. This intelligence increases their flexibility, reliability, and accuracy. Properly-designed smart macros dramatically improve software accessibility and usability in ways that cannot be achieved by other means — other than rewriting the source code.
What smart macros do
Smart macros lessen the physical and cognitive effort of using a computer by removing or minimizing software barriers caused by programming errors, UI inconsistencies, inaccessible design, and overly-complex procedures.
Macros with built-in intelligence have four main uses:
Repair accessibility problems
A repair macro fixes an accessibility problem, usually within a specific window or program. For example, a hypothetical "Options" dialog box contains a text field labelled "Addresss" (with underlined accelerator "As"), but pressing Alt+A does not move focus to the field. The repair macro, triggered by Alt+A, moves the focus to the text field by sending keystrokes, moving and clicking the mouse, or a combination of the two. This hotkey is available only when the titlebar says "Options" so Alt+A works normally in other windows.
Fix usability problems
A macro that fixes a usability problem forces a non-standard application to work the ways that users expect. If, for example, "Saves" is F2 instead of Ctrl+S, a program-specific macro can be written to send F2 to the application when the user presses Ctrl+S. Because the macro is program-specific, the hotkey has no effect on other applications.
Regulate complex tasks
When automating a task to perform a series of actions, a dumb macro may play back faster than the computer can keep up. The only technique for a dumb macro to control information flow is to pause at critical moments. Macros with time delays tend to be unreliable because pause lengths vary and are unpredictable. This is especially true when a macro interacts with the Internet; network bottlenecks can easily throw off the timing of scripts that are regulated with time delays.
The smart macro's ability to process information from the UI allows for more reliable control. Rather than wait for preset time periods, a smart macro waits for specific events to occur, and tests that the events have occurred before continuing. Events that can be monitored include:
- A window gains (or loses) focus.
- A web page finishes loading.
- The mouse cursor changes shape.
- A control becomes visible, gains focus, or is enabled.
- A file or folder comes into existence.
- A specific pixel changes colour.
Testing for UI changes makes it possible to detect errors. For example, if a particular window does not appear within 10 seconds, the macro can stop, prompt the user for further instructions, or wait longer.
Activate automatically
An automatic macro activates when a specific change occurs in the user interface. Many of the events listed above can serve as triggers. Automatic macros can also activate at set times or at regular intervals.
Creating smart macros
Degrees of difficulty
One does not need to be a programmer to script smart macros, but knowledge of programming concepts is extremely helpful, especially when automating multi-step tasks. The learning curve may be steep when first attempting to control a complex process. An approach that works reliably in one context may fail in another.
Not all smart macros are difficult to script. The "dumbest" smart macros — and the easiest to create — are macros with scope. The scope of a macro is the context in which it works. The scope may be window-specific (works only in windows with certain text in the titlebar); program-specific (works only within certain programs); and global-except (works in all contexts with the exception of certain windows or programs). It is also possible to script a smart macro with global scope, which works in all contexts.
Features to look for in macro software
Any macro software product with these four features can be used to create smart macros:
- Scope.
- Conditional statements, e.g., IF - THEN - ELSE.
- Variables.
- Event triggers, e.g., activate when windows get focus.
At least three third-party products for Windows allow the development of smart macros:
AutoHotkey: autohotkey.com
Macro Express: macros.com
Macro Scheduler: mjtnet.com
Case Studies Involving Smart Macros
Joan
Joan's disability is cerebral palsy. She has good control over her left arm, and moves it slowly within a slightly restricted range of motion. She types with her left index finger and thumb.
Joan designs questionnaires and distributes them electronically to fellow employees. All company e-mail addresses are listed in an address book folder. When the address book opens, it is always set to its default folder, which is not the one Joan needs. Every time Joan opens the address book, she spends time and energy selecting the correct folder before beginning to choose recipients.
The solution was a macro that triggers automatically when the address book appears. The macro sends a sequence of keystrokes to select the appropriate folder and move keyboard input focus to the list of names.
Nur
Nur has a ten-year history of mouse-induced repetitive strain injuries (RSIs). Much of her job involves retrieving data from a database application. The old, character-based version was 100% keyboard-driven, and did not cause Nur significant problems. The new, web-based version lacks built-in keyboard support. Soon after the new version was introduced, she began to experience pain and weakness in her fingers, hands, wrists, arms, elbows, neck, shoulders, and back. To address her condition, the developers created a hack to improve keyboard access, but the technique, which involves typing unintuitive key sequences, is awkward to use and hard to remember.
The application is hierarchically organized, and users typically follow predictable paths through its screens. Two hotkey macros were scripted: the first, triggered by pressing PgDown, drills down to the next screen; the second, activated by pressing PgUp, returns to the previous screen. Both macros detect the name of the current screen and determine which screen to load next. These macros greatly simplify navigation, and enable Nur to work without painful RSI flair-ups.
Jacob
Jacob is an office worker who is blind. The Windows-based client tracking application he uses is not screen reader friendly. Many screens cannot be made accessible via his screen reader's scripting language. However, some database query results can be made accessible to Jacob by exporting them to a plain text file.
The procedure to export data and open the file is 15 steps. The entire process was automated with a macro that parcels out more than 50 keystrokes in discrete batches. At each step, the macro detects changes to the shape of the mouse cursor; when the shape changes from the hourglass to the arrow, the step is complete and the next step begins. The length of time to execute each step varies because the application draws data from servers in different parts of the country. Some steps execute in a fraction of a second; others take 30 seconds or longer. At each step, the macro waits only as long as necessary before continuing.
The penultimate step can only be done by clicking an "Update" icon and watching for text to appear in a field. If the text does not appear, the step must be repeated. Every three seconds, the macro clicks the icon, moves the cursor to the field, and monitors the colour of a single pixel. When its colour changes from grey to black, the step is complete, and the final step, opening the text file, is executed.
This smart macro (and two similar ones) enable Jacob to perform essential job duties that he would not otherwise be able to do. Depending on network traffic, the macro takes 45 to 90 seconds to complete all 15 steps. (His co-workers take three to five minutes to perform the same tasks using keyboard and mouse.)
Conclusion
The ability to script intelligent macros is an essential skill for assistive technologists who accommodate computer users with disabilities. Smart macros make it possible for individuals with a wide range of abilities to operate software that is difficult or impossible to use. Smart macros simplify access to information because the macros themselves assume some of the physical and cognitive effort required to drive software.