Create Apps That Use Voice Control On Mac

‎APPControl Voice Edition is an air traffic controller game with voice recognition. You guide the aircraft on your radar screen by talking to your iPad using real ATC clearances like: - 'American 13, Climb and maintain 4 thousand feet' - 'Iberia 56, Turn left heading 2 7 0' - 'Continental 78. Oct 07, 2019  For the best performance when using Voice Control with a Mac notebook computer and an external display, keep your notebook lid open or use an external microphone. All audio processing for Voice Control happens on your device, so your personal data is always kept private. Use Voice Control on your iPhone or iPod touch.

With iPad, you can take your ideas anywhere. Switch Control lets you navigate onscreen keyboards with a single tap. Siri lets you tackle everyday tasks with just your words. And with AssistiveTouch, you can turn a pinch into a tap or customize other gestures to fit your needs.

Voice ControlTell your iPad exactly what you want to do.

Voice Control lets you fully control your iPad using only your voice.1 A comprehensive set of commands makes it easy to quickly open and interact with your favorite apps. And you can navigate virtually any app through numbered labels alongside clickable items or by superimposing a grid to precisely select, zoom, and drag.

Voice ControlA more efficient way to write and edit.

Voice Control provides the latest advances in audio-to-text transcription. And with rich text editing commands, making corrections or formatting changes is incredibly efficient. You can say things like “Replace ‘I’m almost there’ with ‘I just arrived,’” or “Move up two lines. Select previous word. Capitalize that” — and your iPad does it. If you want to correct or change a word or emoji, simply ask and you’ll get a list of suggested replacements. And Voice Control understands contextual cues, so you can seamlessly transition between text dictation and commands. For example, say “Happy Birthday. Tap send.” in Messages, and Voice Control sends “Happy Birthday” — just as you intended. You can also say “Delete that,” and it knows to delete what you just wrote.

Switch ControlTap and pinch without tapping and pinching.

Built directly into iOS and iPadOS, Switch Control is a powerful accessibility technology for anyone with extensive physical motor limitations. Switch Control allows you to navigate sequentially through onscreen items and perform specific actions using a variety of Bluetooth-enabled switch hardware. It also has features like expanded word prediction and a media playback menu to make it easier to use. And Switch Control is customizable for both beginners and advanced users — you can simplify existing actions or create your own.

Platform SwitchingControl your Mac with your iPad.

Switch Control supports Platform Switching, which allows you to use a single device to operate any other devices you have synced with your iCloud account. So you can control your Mac directly from your iPhone or iPad, without having to set up your switches on each new device.2

SiriStreamline the things you do every day.

Siri helps you with the things you do every day.3 All you have to do is ask. Say something like “Tell Lucy I’m running late” or “Remind me to make reservations for Saturday.” And with Siri Shortcuts, you can cue custom, multi-action commands by voicing a preferred phrase. If you’re leaving work, you could say “Heading Home” to simultaneously alert a family member, get directions in Maps, and adjust your thermostat. Siri is also integrated with Switch Control, so you can use it even when “Hey Siri” isn’t active.

DictationYou say it. iPad types it.

Create apps that use voice control on mac download

Dictation lets you talk wherever you would type. Tap the microphone button on the keyboard, say what you want to write, and your iPad converts your words (and numbers and characters) into text. So it’s easy to type an email, note, or web address — without typing at all.

AssistiveTouchTurn a pinch into a tap.

iPad is remarkably intuitive and easy to use. AssistiveTouch lets you adapt the Multi-Touch screen to your physical needs. So if you need to change some gestures, like pinch, you can make them accessible with just a tap of a finger. You can create your own gestures and even customize the layout of the AssistiveTouch menu. And if you want an alternative to pressing the Home button, you can activate it with an onscreen tap. Gestures like rotate and shake are available even when your iPad is mounted on a wheelchair. iOS and iPadOS devices also support a number of third-party assistive devices that help you interact with your iPhone, iPad, and iPod touch. And now Mouse support in AssistiveTouch lets you connect a Bluetooth mouse to mimic finger touch for navigation.

Touch AccommodationsMake touch interactions work for you.

With Touch Accommodations you can adjust how the screen responds to your touch. Control how long you touch before it’s recognized or whether repeat touches are ignored. So you can put your finger down anywhere on the screen and move to the item you want without mistakenly performing an action.

Predictive TextLet iPad finish what you started.

Tap to choose the perfect suggestion for your next word — Predictive Text adjusts results based on who you’re talking to and what you’ve said before.4

Keyboard ShortcutsShortcuts for commonly used words or phrases.

If there’s a word or phrase you frequently use, create a custom shortcut — and iPad will type it out for you. For example, “appt” can expand to “appointment” or “cyl” to “Call you later.” Keyboard shortcuts make it easier and faster to type your name, email address, home address, or any other text that you type often.

Hardware Keyboard SupportSimplify keystrokes and adjust sensitivity.

When using a physical keyboard, you can turn on Sticky Keys to combine your keystrokes, allowing you to press one key at a time instead of pressing them together. Or use Slow Keys to adjust the amount of time between when a key is pressed and when it is activated — so your iPad processes only the keystrokes you mean to make.

Third-Party Keyboard SupportUse the keyboard that’s right for you.

Swipe rather than type, or go old school with the classic keyboard layout. Developers can format keyboards for use within their own apps. And now that new keyboards are available, you can choose your favorite input method or layout systemwide.

Resources

Create Apps That Use Voice Control On Mac Laptop

Support

User Guides

Voice Control White Paper (PDF)
Learn more with the iPad User Guide

Developer Resources

Resources to help developers make their apps more accessible.

Learn more about Apple Accessibility for Developers

Intro

If you’re new to the Mac, learning and mastering the VoiceOver screenreader can seem daunting at first. In an attempt to streamline the search for essential getting started information across the AppleVis website, I will provide a series of tips, along with links to more comprehensive guides and podcasts, organized by heading and subheading.

From personal experience, I have found that one of the most effective ways to learn a new screenreader is to read documentation with another more familiar one. For example, you could start reading this guide on iOS, Android or Windows, gradually try various things on the Mac, and finally, see how you do relying on VoiceOver in addition to or in place of your other screenreader.

Terminology

The Mac refers to Apple’s line of personal computers.

Mac laptops range from the low-end MacBook Air to the high-end MacBook Pro. Mac desktops include the Mac Mini, iMac, iMac Pro, and Mac Pro.

For both laptops and desktops, specific product families are differentiated by features, performance, and price. For information and buying advice on Mac models at any given time, check out the MacRumors Buyer’s guide.

MacOS is the operating system that runs on all Macs. It was previously known as OS X, and mac OS X before that. Its major versions, released annually, are named after California landmarks, like Catalina or Mojave. To minimize confusion in this guide, and because VoiceOver doesn’t change a whole lot from version to version, I will cover concepts in the latest public release.

Create Apps That Use Voice Control On Mac

The Apple keyboard

As modifier keys on a keyboard can vary greatly, there is no way for me to know your keyboard’s precise configuration. For this reason, I will be discussing locations of keys on the standard Apple keyboard, the keyboard that comes built-in on Mac laptops and is included externally with Mac desktops.

At the bottom row from left to right are the Function, Control, Option, and Command keys, followed by the space bar. To the right of the space bar is another Command key, another Option key, and the arrow keys.

On Macs without a Touch Bar, on the top row from left to right is the Escape key, followed by twelve function keys. The Touch Bar is a small strip that holds dynamic touch controls for whatever app is in focus, in lieu of standard function keys. For information on how to use the Touch Bar and the implications for VoiceOver, check out this review of the 2016 MacBook Pro, the first Mac to include this feature. Although the review is several years old, the pertinent information is still accurate.

Setup

First things first, when you first attempt to set up a computer, you need to know how to turn it on.

If you’re setting up a laptop, simply opening the lid should cause it to power on; although at first there is no audible indication that this is occurring. If you’re setting up a desktop, plug it in and turn on any accessories like a wireless keyboard and mouse, and press the power button, which is typically the only tactile button on the case.

If you have a desktop that came with a wireless keyboard and mouse that are switched on, the computer should automatically recognize them and proceed with the boot process.

Once your Mac has booted for the first time, you will hear a voice that, in English says something like, “To use English for the main language, press the return key.” As stated, press the return key, the rectangular key above the right Shift key on Apple keyboards, to use English or use the up and down arrow keys to select a different language. After a few seconds, you will be told how to turn on VoiceOver or start the quick start tutorial, which I would recommend if you’re new to the Mac.

From there, Setup Assistant guides you through connecting to Wi-Fi, signing in with your Apple ID, creating your user account, and configuring basic settings. If you skip the tutorial, the following tips should help you get on your feet.

General

Like other screenreaders on desktop operating systems, VoiceOver is designed to be operated primarily from the keyboard, with a combination of VoiceOver commands and universal system and app keyboard shortcuts.

VoiceOver commands are denoted by the VoiceOver modifier, which is either the control and option keys or the caps lock key. This is referred to as, “VO,” for short. Therefore, if you are instructed, for example, to press VO Space, hold down the control and option keys, or the caps lock key, and press the space bar.

Move around the screen using VO left and right arrows, and activate, or, “Click,” items with VO Space. In some cases, the tab key will take you to certain elements, but this behavior is not totally consistent across the OS.

To jump to the top of a window, useful for reading instructions in the Setup Assistant and other wizards, press VO Function Left Arrow. To jump to the Bottom, press VO Function Right arrow.

Finally, if you’re trying to identify a particular VoiceOver command, either press VO H twice to access the Commands Help menu, or VO K to access Keyboard Help, where keys can be pressed without having any effect on the system.

The desktop

The desktop is the first point you’re placed after logging in. It is a place where any file can be dumped and by default, is where external storage devices are mounted for quick access. To get to the desktop from anywhere in macOS, press VO Shift D.

The menu bar

The menu bar contains useful commands for apps, as well as global tasks like restarting or shutting down. From anywhere in macOS, access the menu bar by pressing VO M.

Create Apps That Use Voice Control On Macbook

You are initially focused on the Apple Menu, located on the left. This menu is always shown, and contains commands that you can access from anywhere. The menu immediately to the right is the name of the current app in focus. For example, if you are working in Mail, the menu to the right of the Apple menu will be named, “Mail.” The rest of the menus depend on what app you have in focus, but are often file, edit, view, etc. Use the left and right arrow keys to navigate between menus, and the up and down arrow keys to cycle through the available options.

These menus include commands for the app in focus, as well as the commands’ respective keyboard shortcuts. This is how the menu bar can become very useful as you gain proficiency on the Mac, as keyboard shortcuts in apps can greatly reduce the steps needed to complete basic tasks.

To exit the menu bar at any time, press the Escape key, or Function Tab if your Mac's keyboard doesn't have one.

In addition to common system and app commands, the menu bar is also used to display status information about your Mac, such as Wi-Fi connection and battery information, accessed by pressing VO M twice. Navigate the menu headings with VO left or right arrow, and reveal the menu options by pressing VO Space. Press the Escape key, or Function Tab if your Mac's keyboard doesn't have one, to exit a menu.

The dock

The dock contains apps that you can access quickly from anywhere in macOS. Access the dock by pressing VO D and navigate between apps using the left and right arrow keys. Open apps using VO Space.

Additional options for an app can be accessed from the contextual menu by focusing on it and pressing VO Shift M, the Mac equivalent of a right-click.

Exit the dock at any time by pressing the Escape key, or Function Tab if your Mac's keyboard doesn't have one.

Interacting

By default, VoiceOver organizes interface elements hierarchically, meaning there can be elements within other elements. This is indicated by a slight pop sound when focusing on such an element.

While this behavior is not totally consistent across the OS, element types that typically require interaction include toolbars, groups, and scroll areas, among others. To interact, press VO Shift down arrow. To stop interacting, press VO Shift up arrow.

macOS keyboard shortcuts

While VoiceOver includes commands to complete some core tasks, some of the most useful and ubiquitous commands are universal macOS keyboard shortcuts. They vary from app to app, but generally follow a basic format. Below is a list of the most common shortcuts, and you can find a more comprehensive list in this guide.

  • new: Command N
  • open: Command O
  • close current window: Command W
  • quit current app: Command Q
  • print: Command P
  • copy: Command C
  • paste: Command V
  • access preferences for the app in focus: Command Comma
  • cycle between open apps: Command tab

Some immediate post setup tips

Updating your software

Periodically, Apple releases updates to macOS and bundled first-party apps. As updates may have been released since your Mac was packaged at the factory, it is a good idea to check for updates once initial setup is complete.

To do this, choose Apple > System Preferences and click software update. If updates are available, click update now.

Note: you may need to restart your computer to complete installation of some updates.

To update bundled apps, choose Apple > AppStore. Once you’ve passed any sign-in or welcome dialogs, choose Store > Updates, or press Command 8, to check for updates. Click update all to download and install all updates.

Configuring VoiceOver to start automatically at login

Note: if you prefer an audio demonstration, there is an AppleVis podcast that demonstrates this process.

By default, VoiceOver does not start automatically at the login prompt. To change this, follow these steps.

  1. Choose Apple > System Preferences and click Users & Groups.
  2. Hit the, “Click the lock to make changes,” button near the bottom of the window and enter your password.
  3. Interact with the, “Users, groups and login options,” list, and select login options.
  4. Stop interacting and click accessibility options.
  5. Interact with the table and select the VoiceOver checkbox.
  6. Click apply.

If you find that you cannot use standard VoiceOver navigation at the login prompt, you may have FileVault disk encryption turned on. To turn it off and get a more conventional login prompt, follow these steps.

  1. Open System Preferences and click Security & Privacy.
  2. Hit the, “Click the lock to make changes,” button near the bottom of the window and authenticate with either your password, Touch ID, or Apple Watch.
  3. Select the, “FileVault,” tab and click turn off FileVault.

Note: when this is turned off, the contents of your disk will be decrypted.

Customizing VoiceOver

VoiceOver settings can be customized using VoiceOver utility, accessed from the Utilities folder or by pressing VO F 8, or VO Function 8 if you're using a Mac with a Touch Bar.

Settings are grouped into categories like general, verbosity, speech, etc. Use the up and down arrow keys to find a category in the table and navigate using VO right arrow to explore the options.

In addition, some VoiceOver settings can be changed by VoiceOver commands from anywhere in macOS. For example, to change speech options, press VO Command Shift left or right arrow to view the available settings, and adjust the values using VO Command Shift up or down arrow. Also, some verbosity settings can be changed by pressing VO V and using the left and right arrow keys to view the settings, and the up and down arrow keys to adjust their values.

Along with the standard ways of navigating macOS, there are some additional customizations that can improve your workflow.

Quick Nav

You may have noticed, as you press the left and right arrow keys, that VoiceOver announces, “Quick Nav on,” and, “Quick Nav off,” which can alter VoiceOver behavior and thus be incredibly frustrating.

Quick Nav allows you to use the arrow keys as if the VoiceOver modifier was held down. For example, a press of the right arrow when not in a textfield is treated like VO Right Arrow, and thus moves VoiceOver focus right. You can also use Quick Nav commands to navigate the VoiceOver rotor, discussed later. If you use iOS, these commands will feel very familiar to you.

If you don’t use Quick Nav, you can prevent it from being inadvertently toggled from the keyboard by going to VoiceOver Utility > Commanders > Quick Nav and deselecting the, “Allow toggling of Quick Nav using left and right arrow keys,” checkbox.

Trackpad Commander

If you use iOS, you’re probably well accustomed to using touch gestures to navigate with VoiceOver. With Trackpad Commander, many of these gestures can be performed on your laptop’s internal trackpad or an Apple Magic Trackpad connected via bluetooth.

To enable Trackpad Commander, hold down the VoiceOver modifier and rotate two fingers clockwise on the trackpad; counterclockwise to disable it. VoiceOver will play a sound and announce when Trackpad Commander has been turned on or off.

From here, flick with one finger left or right to move, double tap to activate, flick right with two fingers to interact, and flick left with two fingers to stop interacting. These are just a few commands. For a full list, check out this guide.

Additional settings and custom gestures can be configured in VoiceOver Utility > Commanders > Trackpad.

Keyboard Commander

Keyboard Commander allows you to configure custom keyboard shortcuts to perform VoiceOver commands, run scripts, and open apps, eliminating the need to locate them manually and thus greatly streamlining your workflow. Commands can be configured in VoiceOver Utility > Commanders > Keyboard Commander.

By default, commands for opening Mail and Safari and running scripts for announcing the date and time and the number of unread Mail messages are included. For more information, check out this AppleVis podcast.

The rotor

Similar to iOS, the VoiceOver rotor is used on macOS to navigate by different levels of granularity; characters, words, lines, etc. When browsing webpages, the rotor also allows you to navigate by links, headings, tables, and other web element types.

The rotor can be used in several ways, with or without Quick Nav, or with Trackpad Commander.

To adjust the rotor with Quick Nav off, press VO Command left or right arrow to move between lists, and VO Command up or down arrow to navigate the available items. Pressing VO U will present a rotor that solely contains web element types like links, headings, and tables, which can be navigated using the arrow keys.

To adjust the rotor with Quick Nav on, press up arrow with the left or right arrow to move between lists, and use up or down arrow to navigate the available items. Activate items by pressing the up and down arrow keys together.

Finally, if you have Trackpad Commander on, rotate two fingers clockwise or counterclockwise to move between lists, and flick up or down with one finger to navigate the available items. Activate items by double tapping with one finger.

The Finder

Finder is the Mac’s file manager, and is constantly open. Think of it as the equivalent of File Explorer on Windows. It is where you create, organize and delete files and folders, view external storage devices, and manage and sync iOS devices. For information on how to use Finder, check out this guide.

Spotlight

Spotlight lets you locate files and other data on your Mac. It is located third from right in the menu extras, accessed by pressing VO M twice. It can also be accessed directly by pressing Command Space.

Type your search in the field and use the up and down arrow keys to navigate the results. If what you’re looking for is not in the list, select, “Show all,” and a Finder window with all the results will open. Settings like the keyboard shortcut, result categories to include, and locations to exclude can be changed in System Preferences > Spotlight.

Siri

Like on iOS, you can use Siri on your Mac to find information and complete basic tasks, such as checking weather, calendar, and contacts, opening apps, searching the web, sending emails and texts, etc. However, Siri on macOS does not currently support any third-party app integration.

Siri can be accessed by pressing and holding Command Space, or by saying, “Hey Siri,” on Macs with a T2 security chip. These Macs include the MacBook Air, (2018 and later) MacBook Pro (2018 and later) Mac Mini, (2018 and later) and iMac Pro, (2017).

Siri settings, like the voice and keyboard shortcut, can be changed in System preferences > Siri.

For more information and an audio demonstration, check out this AppleVis podcast.

Word processing

While the purpose of this section is not to cover one app, there are some concepts central to VoiceOver and macOS that can help you get started with basic word processing.

Your Mac comes with two main word processors, TextEdit, a basic document creation tool, and Pages, Apple’s answer to Microsoft Word. For information and a demonstration of some of Pages’ functions, check out this AppleVis podcast.

The following commands should work in any app used for creating and editing documents.

Create Apps That Use Voice Control On Macbook Pro

  • move by character: left or right arrow keys, hold down the Shift key to select characters the curser passes
  • move by word: Option left or right arrow keys, hold down the Shift key to select words the curser passes
  • move by line: up or down arrow keys, hold down the Shift key to select lines the curser passes
  • top of document: Command up arrow
  • bottom of document: Command down arrow
  • check spelling: Command semicolon
  • reveal full spell and grammar check window: Command colon

Tip: any time you’re in a textfield, you can dictate text by pressing the Function key twice and speaking after you hear a ding sound. Press the Function key once when you’re done, and the text should be inserted. The first time you do this, you’ll be asked if you want to use enhanced dictation, which downloads dictation resources locally to your computer, eliminating the need to send recordings to Apple’s servers for processing.

Web Browsing

When navigating websites, VoiceOver contains them in groups announced as, “Web content.” While this is generally associated with web browsing, “Web content,” groups are also used to display conversations in Messages and results in Dictionary.

When interacting with these groups, press VO left and right arrows to move by paragraphs and web elements, or use the rotor for more granular navigation. In addition, the following commands can help you brows by specific element types. For any of these commands, hold down the Shift key while performing the command to find the previous occurrence of that element type.

  • find next heading: VO Command H
  • find next link: VO Command L
  • find next table: VO Command T
  • find next list: VO Command X
  • find next frame: VO Command F

Create Apps That Use Voice Control On Mac Download

For more information on how to use Safari, the Mac’s built-in web browser, check out this guide.

Conclusion and additional resources

Although this guide touches on numerous topics, there is a wealth of information on AppleVis and elsewhere. More information is available in your Mac’s built-in help and the AppleVis forum, and below are a few links to some more potentially useful resources.

If you have any other suggestions or want something clarified, sound off in the comments.