The Revolutionary New iPhone

Over the past year, AccessWorld has reported on significant improvements in accessibility that Apple has made to its Mac and iPod line of products, and the American Foundation for the Blind presented Apple with a 2009 Access Award for these accomplishments. With the release of the revolutionary new iPhone 3G S, Apple has done it again, this time in an even bigger way. Apple's VoiceOver screen reader, which has been part of the Mac computer operating system for several years, is now part of the new iPhone, and Apple has developed an ingenious "gesture" technology for interacting with the iPhone's touch-screen interface. Moreover, the accessibility comes at the same price our sighted friends and colleagues pay, so there's no so-called blindness tax. This article describes how this new interface works and evaluates how well it can access the many applications included on the iPhone 3G S.

Description of the iPhone 3G S

Available in the United States with service from AT&T, the iPhone 3G S is available with two levels of onboard memory. The one with 16 GB of memory costs $199, and the 32 GB model costs $299. You have to have a data plan with the iPhone, so your monthly service costs will be higher than that for a simple voice plan. Check with AT&T for the various service plans. The iPhone measures 4.5 by 2.4 by 0.48 inches and weighs 4.8 ounces. Its touch screen features a high-contrast color display measuring 3.5 inches diagonally.

The touch screen is the key to controlling the iPhone, but it also has some physical buttons. The Home button, which is used to bring up the iPhone's home screen, is a round, slightly concave button at the bottom center of the iPhone, just below the active area of the touch screen. The Screen Lock button, on the right side of the top panel, is a long, skinny button used to lock the screen so you do not inadvertently interact with the touch screen while it is in your pocket. On the top left panel is a switch to turn the Vibrate Alert on, and just below that switch is a Rocker button for adjusting volume. There are other physical features of the iPhone that are not buttons. These features include a standard 3.5-mm headphone jack, which is on the left side of the top panel, and the earphone, which is on the top of the touch screen. Also, the bottom panel features the speaker on the left side and the phone microphone on the right side. Between them is Apple's standard iPod/iPhone jack, used to connect the iPhone to a wall outlet or your computer for charging or syncing data. An Apple USB cable is included to connect your iPhone to your computer, and the cable conveniently connects to a standard two-pronged electrical plug included with your iPhone, allowing you to charge it from an electrical outlet. A set of ear buds is also included, and it has on its cord the same three-button control that comes with the iPod Shuffle Third Generation, another iPod that has VoiceOver features. In addition to activating some of the iPod applications' (apps) features, the ear bud's control button answers and ends calls and has a built-in microphone for phone calls.

In addition, the iPhone 3G S has three built-in electronic sensors. The proximity sensor knows when your face is next to the iPhone and automatically sets the sound output to come from the earphone if your face is next to it or from the speaker if it is not. The accelerometer senses when you have turned your iPhone sideways and automatically adjusts the display to a landscape orientation. It is also used in the iPod app's "shake to shuffle" feature, allowing you simply to shake your iPhone to play your songs in a shuffled, random order. The magnetometer is used in the Compass app to find true north.

The iPhone Home Screen

The iPhone's home screen features icons for the various apps that come loaded on the iPhone, as well as several status indication icons. The status icons along the top edge of the touch screen indicate such things as your signal strength, service provider, wifi connection strength, the current time, and battery strength. Below these status icons and taking up most of the rest of the screen are icons for all the various apps that come with the iPhone, such as messages, calendar, camera, and iTunes. I discuss the apps further later in this article. Along the bottom of the home screen are icons for the four main apps you will use most: Phone, Mail, the Safari Web browser, and the iPod app.

Caption: The iPhone's home screen

VoiceOver and Gestures

Apple's VoiceOver screen reader and its accompanying gesture technology are how a person who is visually impaired accesses the many features of the iPhone. Sighted people also use gestures on the iPhone, but VoiceOver provides a unique set of gestures that allow a person who is blind or has low vision to use the phone. You pay nothing extra for VoiceOver, and it is built into all iPhones, not just the ones a person who is visually impaired would use. You can turn VoiceOver on when performing the initial setup and registration tasks for your iPhone. You do so using the iTunes software with the iPhone connected to a Mac or PC, and the process is entirely accessible on both the Mac and a PC using a screen reader or screen magnifier. VoiceOver can also be turned on using the settings on the phone itself, but sighted assistance is needed to do so. VoiceOver features an easy-to-understand speech synthesizer with a female voice, and you can adjust the rate of speech as well as the typing echo as you enter characters. It speaks in 23 different languages, with multiple dialects for some languages.

You may now be thinking to yourself, "OK, it speaks, but how do I control what it speaks if I have to use a flat touch screen with no buttons?" This is where VoiceOver's gestures come into play. If you know where an icon or button appears visually on the screen, you can just touch that position on the screen, and the button or icon will be highlighted and its name will be spoken. You can then "double tap" anywhere on the screen to activate the button or launch the application indicated by the icon. For example, I know that the iPod icon is on the bottom right-hand corner of the home screen, so I tap that area, hear the word "iPod" spoken, and then double tap anywhere on the screen to launch the iPod app. If you do not know where a button or icon is on the screen, you can drag your finger or thumb around the screen to hear the various elements on the screen and to learn their positions. You can also "flick" your finger left or right on the screen to move from element to element. For example, I know the Messages icon is near the top left-hand corner of the screen and the rest of the icons are in a grid layout on the screen. I can move my finger around that area until I hear the word "Messages" and then just flick my finger from left to right anywhere on the screen to move to the next icon. Continuous flicks move me through all the icons in the grid in a left-to-right, top-to-bottom manner. Flicking right to left moves you back through the grid in the reverse direction. When I hear the name of the icon I want, I can launch the app by double tapping (two quick taps with a finger) anywhere on the screen.

Here is a brief description of several other gestures for controlling the iPhone.

• two-finger tap: stops VoiceOver from speaking what it is currently saying.
• two-finger double tap: answers and ends phone calls or starts and stops music.
• two-finger flick down: reads from your current focus to the bottom of the screen or to the end of a Web page.
• two-finger flick up: reads from the top of the screen to the bottom.
• three-finger double tap: turns VoiceOver speech on or off.
• three-finger triple tap: turns the "screen curtain" on or off, allowing you to hide the display from sighted onlookers.
• three-finger flick: moves you from page to page in apps or web pages.
• split tap: used as an alternative to the double tap, and is described further in the section on typing on the iPhone.
• one-finger flick up and down: used to adjust some settings and to move by elements selected by the rotor.

The Rotor

The rotor is a gesture that warrants a section of its own, because it provides additional ways to navigate through information on the screen. It is like a virtual knob you turn to select the way you navigate through text and other elements on the screen. You turn your thumb and index finger on the glass just like you would grip and turn the volume dial on your stereo. As you "turn" this "knob," you hear the names of the various elements by which you can move. When you hear the element that you want, you flick up and down to move by that element. For example, if you are using the Notes app, which is a simple word processor, the rotor moves between two choices: words and characters. You then flick up or down to move forward or backward by word or by character. When you use the Safari Web browser, the rotor can choose between many more elements, including headers, form elements, links, nonvisited links, visited links, images, words, and characters. It all depends on which elements appear on the page you are currently viewing.

Typing and Entering Phone Numbers

When you type or enter a phone number on your iPhone, a virtual QWERTY keyboard or 3 by 4 numeric keypad appears on the screen. You can use several of the gestures described earlier to use these virtual keyboards. You can highlight a character by dragging your finger or thumb around the screen or tapping where you think a character is, or you can find a character and flick around until the actual character you want is highlighted. Then, you double tap to enter the character. However, I have found the split-tap method to be the most efficient for me. Split tapping is an alternative to double tapping, using one finger to highlight a character or icon on the screen and then tapping with another finger anywhere on the screen. I use my thumbs and quickly move my right thumb around the screen to highlight the character I want and tap with my left thumb to enter the character. Entering information on an iPhone is definitely something that requires some practice.

Other Accessibility Features

The iPhone includes other built-in accessibility features. It has a setting for a white-on-black display for those who prefer reverse contrast and a Zoom feature that allows for up to 5X magnification (which is discussed further in the section on low vision accessibility). It has a Mono Audio feature that sets it to play both channels of the stereo input in both sides of a headset in case you have lost hearing in one ear. The iPhone also has a Voice Command feature allowing you to use your voice to place calls or to play music.

Built-in Apps

In addition to VoiceOver, I counted 21 other apps that come with the iPhone and all of their features are accessible. This is a list of the apps as they appear on the home screen: Messages, Calendar, Photos, Camera, YouTube, Stocks, Maps, Weather, Voice Memos, Notes, Clock (including a world clock, alarm, stopwatch, and timer), Calculator, Settings, iTunes, App Store, Compass, Contacts, Mail, Safari, and iPod. Oh yes, there is also a Phone app.

I have had a great time testing and learning to use these apps since I got my iPhone in June, so much so that my wife has begun calling herself an iPhone widow. I do not have room to describe all the apps here, but I will mention my experiences with some of them.

Accessing and entering data in the Calendar and Contacts apps was easy to do. However, I have found it much easier to enter my appointments and contacts on a Mac or PC and sync them to my iPhone. Making calls directly from the contacts app is easy to do. However, there is a minor bug that sometimes occurs when editing a contact on the iPhone. If you are inserting or deleting a character in the middle of a person's name or phone number, it sometimes inserts or deletes at the end of the name or number. Apple is aware of this bug, and I hope it will be fixed in the next software update.

Caption: The author syncing playlist to his iPhone

I travel a good bit for my work, and the Weather app is easy to use to enter the city I am going to and find a quick five-day forecast. The Maps app is also handy for getting directions when travelling by car, and it provides walking directions. The Notes app is good for jotting down a quick note, but I found using the Voice Memos app is faster than tapping out a note. You can also send a voice memo as an e-mail to your friends.

The iTunes app is used to access the iTunes Store, and it is easy to use to search for and buy music, audio books, movies, and television shows. It does, however, sometimes lose focus when downloading a long file, such as a movie or television show, so I usually put it down and wait for the file to finish downloading. The built-in iPod is an excellent app, and everything is accessible, including the Search feature. It is easy to use for listening to music and reading books from audible.com. I am admittedly no judge of musical fidelity, but the ear buds sound great. However, I prefer using a Bluetooth headset for added convenience.

I also use the Mail app and the Safari Web browser a significant amount when I travel or am away from my computer. The Mail app is a great way to check my e-mail on the way to work. We did find a bug that makes it difficult to gain focus on the proper form fields when you initially set it up to access your e-mail account, but it is a one-time-only problem. Also, if you are using a POP or IMAP account, iTunes will configure your e-mail automatically. Safari is a great tool to Google an important fact, especially when I am arguing about sports history with my poker buddies. The rotor is a great way to navigate a Web site quickly, especially one that has been properly designed. However, using the rotor to navigate by header initially did not work on some Web pages. It appears Apple fixed this bug in a later software release.

The Phone app is probably the most difficult to get used to, and if you borrowed my iPhone to make a quick call without ever having used one before, you would probably find it frustrating. Even after I used it for about a month, I found it to be somewhat difficult to dial a call in a noisy environment or in a stressful situation where speed is important. The ear buds can help you to hear VoiceOver's output better in such situations. Also, it could be the way I am tapping, but I have found ending a call with the two-finger double-tap method to be inconsistent. However, pressing the Screen Lock button or the middle button on the ear bud controller always successfully ends a call.

One tricky task to accomplish on the iPhone with VoiceOver is using interactive voicemail answering systems many businesses use where you have to press 1 for sales and 2 for customer service and the like. As soon as the call is placed, you have to activate the virtual keypad quickly by double tapping the keypad icon and then press the correct digit to access the department you want. You must do so amid conflicting speech coming from the other end of the call, and you have to do it before the system times out. This may sound like a daunting task, but there are a couple of techniques to get the job done. First, if you know ahead of time the digit you need to press to access, such as 2 for customer service, you can program it into the number you are dialing by placing a comma followed by a space and then a 2 in the phone number form field. You can also memorize exactly where on the screen the keypad icon appears and double tap that spot as soon as the call is connected. You will also want to use the speakerphone or ear buds or a wireless Bluetooth earpiece, because the touch screen is not active when you hold the phone to your face. Perhaps Apple could include a setting in the next software update to allow for the keypad to appear automatically when a call is placed.

Third-Party Applications

As you may have gleamed from Apple's television commercials for the iPhone featuring the tag line, "There's an App for That," literally thousands of third-party apps are available through the Apple app store, ranging from diabetes management to online gaming. Many are free, and most cost only a dollar or two, but a tiny fraction are pricier. I tested only a handful, but you can learn about more than 100 accessible apps at the following link: http://www.lioncourt.com/voiceover-compatible-iphone-applications.

Here is a list of five apps I tried with a brief description of their function and accessibility.

• iFitness: This application is an extensive fitness-management software with several built-in workout routines as well as the ability to create custom workouts. It has a database of thousands of exercises in dozens of categories and gives detailed text descriptions of each. I have not tried all the features, but all that I have tried are accessible.
• Kindle: This is Amazon's Kindle book-reading software for use on the iPhone. I bet you have already guessed it is totally incompatible with VoiceOver.
• Wonder Radio: This app lets you search for and play thousands of radio stations from across the country, and it is mostly accessible except for a couple of unlabeled buttons. You can also browse categories of stations, and I find it great for listening to sports events.
• Shazam: This is a cool app that identifies the song and artist of any song that you may hear playing. Just hold your iPhone near the speaker, and in about a minute, it displays the name of the song and the artist, along with buttons to do things like buy the song, watch a YouTube video of the artist, and read a biography of the artist. It is completely accessible.
• Docs to Go: This app is used to send Microsoft Word documents and Excel spread sheets to your phone for viewing and editing. Unfortunately, this is an inaccessible app.
• The Lexmark Accessibility Solution: Although this is actually just a Web interface to a Lexmark multifunction copy machine, not an app, it is completely accessible to use with VoiceOver. It is a unique accessible interface for accessing the many copy, scan, fax, and e-mail functions of a Lexmark machine that has been installed on your network. You can read more about it in my other article in this issue of AccessWorld, Lexmark Marks a Path Toward Accessibility: A New Solution for their Multifunctional Document Centers.

There are tons of games and other fun stuff in the app store, but there are also some serious apps, and the iPhone has the potential to be the platform for many apps that could positively affect the lives of people with disabilities. There is already an augmentative communication app that helps people with speech-related disabilities to communicate, and there are several diabetes-management apps. Although the diabetes apps are currently only for manually entering data regarding your diabetes regimen, there is real potential for wirelessly communicating with and controlling medical devices, such as blood glucose meters or insulin pumps. In fact, some manufacturers of insulin pumps are already working on wireless connectivity to cell phones.

Battery Issues

I did some informal battery testing on the iPhone and found that it takes about two hours to charge from no battery power to a full charge when using an electrical outlet and about three hours when connected to a computer. It takes longer if you are actively using the phone while charging the battery.

The life of your battery's charge depends on what you are doing with the iPhone. When I played media on the iPhone, such as a song, audio book, or movie, I got about 8½ hours of battery life, regardless of the type of media.

In standby, with the screen locked and VoiceOver running but not in use, I got about 1½ days of battery life, compared to a standby battery life of about 5 days with VoiceOver turned off, so VoiceOver does eat up significant battery life, even when not in use. Although you can turn Voiceover's speech off with a three-finger double-tap command, VoiceOver is still actually running, and battery life is not increased. There is currently no gesture command to start or stop VoiceOver completely. However, you can purchase from Apple extra battery packs and battery-based chargers to extend your usage. You can also extend your battery life by turning your iPhone off completely by pressing and holding the Screen Lock key for about 5 seconds.

Low Vision Accessibility

In addition to VoiceOver, the iPhone has other features that can accommodate people with low vision. One is the high-contrast display itself. Its 94% contrast ratio is the highest we have ever measured in our AFB TECH Optics lab. You can learn more about our optics lab and contrast measurements in Lee Huffman's article in the July 2009 issue of AccessWorld, Combating the Small Visual Display Invasion: AFB Works to Set a Display Quality Standard. The "White on Black" feature can assist those who prefer a reverse display, and the "giant" font available for e-mail displays your messages in a 14-point font when in landscape orientation, which, when shown on the high-contrast display, can accommodate people with mild vision loss. The iPhone's Zoom feature allows you to magnify the entire screen up to 500%. There are gestures to use to adjust the magnification and to pan around the screen.

We have so far done only limited testing on the Zoom feature with Lee Huffman, AFB TECH's low vision tech expert. Lee's initial thoughts are that it could be useful for a person who may need just a little bit of magnification, but perhaps not so much for a person who needs a great deal of magnification. The problem with magnifying a small screen display is that the more you magnify the screen, the less information fits on the screen, and you have to do a lot of panning around to view all the screen information. Lee found it difficult to learn quickly how to use the Zoom feature for panning around the screen, efficiently and effectively, and he sometimes experienced what he called an "old- style window-blind effect" when panning. When he would pan in one direction and then lift his finger, it would quickly flip back to its original position like an old window blind. He also mentioned the panning would sometimes move in a somewhat jerky manner. Lee said he prefers using the VoiceOver feature, rather than Zoom, because it was much easier to learn to use. However, he was quick to caution that he was able to test Zoom only for a couple of hours, and a person may be able to practice and get used to using Zoom more effectively and efficiently if he or she really wants or needs to.

One note of interest is that you cannot use Zoom and VoiceOver at the same time. Many people with low vision like to use speech and magnification simultaneously. However, the two apps use some of the same gestures, and it may be difficult to add the gestures to make it possible to use them together.

Documentation

The iPhone's documentation is available in accessible PDF at http://www.apple.com/accessibility/iphone/vision.html. You can also find an accessible html version at http://help.apple.com/iphone/voiceover. The manual is also included on the phone itself, bookmarked in the Safari Web browser app.

The Bottom Line

We do not often use the term "revolutionary" in AccessWorld, but it does apply here. Apple's unique interface for accessing a flat touch-screen interface is not only important for accessing the seemingly unlimited apps that are available for the iPhone, but it proves the point that it can be done. This gives us hope of finding solutions for the many inaccessible touch screens many of us face in our daily lives. The third-party apps open up a myriad of possibilities, and Apple has done a great deal of work to make it possible for developers to design their apps to be compatible with Voiceover. They have created accessible Application Programming Interfaces (APIs) in iPhone OS 3.0, and the iPhone Software Developer Kit (SDK) costs only $99. You can learn more at the iPhone Dev Center: Resources for Making Your iPhone Application Accessible. Perhaps our community can approach these third-party vendors to work with us on the accessibility of their apps.

This is certainly a brand-new concept in accessibility, and there may be a learning curve for many people. However, most blind people I have spoken with about their use of the new iPhone have reported being up and running quickly. I definitely learned to use it quickly, but it is my job to do so, and I do not think the iPhone left my hands in the first two days. Some people may not get it as easily, so the rehabilitation and education professions may need to develop some training systems for using this new iPhone.

One great thing about the iPhone is it is a software-driven device, and Apple has shown a real desire to keep developing more accessibility. By the time you read this article, Apple will have already been at work fixing some of the minor issues with the iPhone I have pointed out in this article (see the Addendum at the end of this article for information on a later software update that adds to the accessibility of iPhone functionality). Apple has been working on significant improvements to the functionality of VoiceOver on the Mac, and it was included in the recent release of the Mac's new Snow Leopard operating system, to be reviewed later in AccessWorld.

New iPods

Apple has also released two new accessible iPods: the iPod Touch Third Generation and the iPod Nano Fifth Generation. The iPod Touch Third Generation includes VoiceOver and the same gesture technology described earlier for the iPhone, making it completely accessible. Basically, it has the same apps as the iPhone but without the phone. It also has a feature that allows you to learn and practice using gestures. The Nano Fifth Generation is the same size and shape as the iPod Fourth Generation, which was previously reviewed in AccessWorld, but it now has a built-in speaker, FM radio, and video camera.

Addendum

As we were about to post this article, Apple released the software version 3.1 update for the iPhone, and we want to report briefly on the significant improvements in accessibility provided by this update.

• Triple-click home: You can now press the home button three times quickly to toggle VoiceOver, Zoom, or White on Black on or off. You can also use the settings to choose which of these accessibility features you want to toggle with the triple-click Home command. This is handy if you want to share your iPhone with a sighted person or to save on batteries. Also, thanks to Mike Calvo and Serotek's Serotalk Podcast 23, we learned that this feature makes it possible to use the Hue Vue color identifier app that is available at the App Store. You can find the podcast at: http://serotalk.com
• Cut, copy, and paste: You can now use cut, copy, and paste while in the Notes application. The rotor gesture now has a choice for edit, and you can flick up and down to choose to select the current word or to select all. You can then choose cut or copy and paste the selection somewhere else. For example, you can paste a phone number or e-mail address you have jotted down into the phone app or e-mail app. You can even copy and paste a phone number that uses alphanumeric digits, such as 1-800-DOMINOS. Also, when pasting into an edit field, you can double tap to toggle the insertion point between the beginning and end of the field.
• Undo: You can now shake your iPhone. Flick left or right to choose the action to undo, and then double tap. For example, you can undo your edits to a note.
• Additional rotor settings: In addition to the Edit feature mentioned earlier, there are now additional rotor settings that you can use on web pages. The Static Text setting allows you to flick up and down to move quickly to blocks of static text, text that is not a link, form field, or other page element. The Zoom setting allows you to flick up and down to increase the magnification of the page.
• Navigating a list index: Some web pages have lists that have an alphabetical index along the right side, and you can now conveniently move through the index with VoiceOver. It is a great way to scroll quickly through your songs and artists on the iPod app, especially if you have hundreds or thousands of songs loaded. You simply touch the list index directly to select it and flick up or down to move through the index. Or you can select the list index and double tap and slide your finger up or down to move through the index quickly.
• Opening links in mail messages: In Mail, you can now double tap and hold down on a link. The address is displayed, and you can choose to open the link in Safari or copy the link address to the clipboard.
• Editing videos and voice memos: You can use VoiceOver gestures to trim the beginning or end of videos and Voice Memo recordings. It allows you to choose the amount of time, in seconds, that you want to trim. Also, when you pause a video or memo, it tells you how far you have progressed as a percentage. With this information, you can do the math and be very accurate with your trimming.
• PDF file support: VoiceOver can now read PDF (portable document format) documents that are attached to e-mail messages. You simply double tap on the button for the attachment, and it appears and VoiceOver begins reading it. Of course, the PDF has to be accessible. Just as with Adobe Reader on your computer, image-only documents do not work, and documents with bizarre layouts are difficult to comprehend. Another limitation is that it reads the whole document straight through from beginning to end, and if you stop, you cannot start where you left off. You can read by word or by character, but there are no other navigation options.
• Voice control using a Bluetooth headset: You can now enter spoken Voice Control commands through a Bluetooth headset in addition to the ear buds and built-in microphone.
• Saving photo and video e-mail attachments: You can now save photo and video attachments to your camera roll albums. You just touch and hold the image attachment and tap Save Image.
• Compass: VoiceOver now automatically announces the compass heading when there is a change in direction of 5 degrees or more.
• Battery life: The battery life between charges has also been significantly improved with this software update.

This Product Evaluation was funded by the Teubert Foundation, Huntington, WV.