D3.1 User requirements report with observations of difficulties users are experiencing
Donegan, M., Oosthuizen, L., Bates, R., Daunys, G., Hansen, J.P., Joos, M., Majaranta, P. and Signorile, I. (2005) D3.1 User requirements report with observations of difficulties users are experiencing. Communication by Gaze Interaction (COGAIN), IST-2003-511598: Deliverable 3.1. Available in PDF format at http://www.cogain.org/results/reports/COGAIN-D3.1.pdf
Mick Donegan (ACE, contact person, email: [forename(dot)surname(dot)cogain(dot)org])
Lisa Oosthuizen (ACE)
Richard Bates (DMU)
Gintautas Daunys (SU)
John Paulin Hansen (ITU)
Markus Joos (TU DRESDEN)
Päivi Majaranta (UTA)
Isabella Signorile (POLITO)
1 Executive Summary
From the literature and data collected it seems that, at present, eye control can only be used effectively to meet a limited range of user requirements. Furthermore, it can only be used effectively by a limited number of people with disabilities who might benefit greatly from it. To address these issues, a number of recommendations are made in this document for consideration by COGAIN partners.
1.1 End-users eye control hardware requirements
It is recommended that a good starting point would be to:
- Measure how effectively the eye control technology available can meet the needs of the full range of users who might benefit from it.
To achieve this aim, it is recommended that WP3 (User Involvement) should:
- Trial as many specialist Eye Control systems as possible[#_ftn1 ].
This will provide an opportunity to:
- Feed back to Eye Control System developers how effectively their technology is meeting the needs of the full range of existing and potential users[#_ftn2 ], and
- Make observations and suggestions relating to any potential modifications to their systems and/or software that might make it more accessible and/or more effective for more users[#_ftn3 ].
- As the above information is acquired, to enable users to make an informed choice of which hardware to consider for their eye control needs, it is recommended that:
- WP3 should add the information gathered from the above investigations to the WP5 catalogue of currently available eye trackers.
The emphasis of the information provided by WP3 should be specifically related to usability issues related to the requirements of end-users with disabilities, e.g. environmental control, portability issues, mounting issues, 'choice of output methods', 'range of access methods', etc[#_ftn4 ].
1.2 End-users' eye control software requirements
Features of the wide range of assistive software already being successfully used via a range of access methods in addition to eye control include the following: resizable cells and grids; a range of input methods; a wide choice of output methods; a choice of symbols or text output; a wide choice of text styles and colours; a range of speech facilities; a choice of languages, etc. As a result, it is recommended that the following issues be investigated with the involvement of the users themselves.
- Of the wide range of specialist (non-eye control) software that is already successfully being used by many people with disabilities for access and communication, find out which can be adapted effectively for eye control (e.g. The Grid, SAW).
This will enable COGAIN partners to:
- Make a comparison of how effectively both the existing range of software specifically designed for eye control and the adapted specialist software compare in terms of their efficacy with eye control systems.
As a result, on behalf of the Users, WP3 will be able to:
- Recommend modifications that could be made to the current range of software that can (or could) be used for eye control, so that it meets as many of the needs of as many existing and potential users as possible.
As the above information is acquired, to enable users to make an informed choice of which software to use for eye control, it is recommended that:
- A matrix should be set up on the COGAIN website relating to features of different software that can (or could be) used for eye control.
The comparison would be based on features such as those described above, such as 'choice of output methods', 'range of access methods', 'range of multi-modal access', etc[#_ftn5 ].
2 Introduction - What is the purpose of this User Requirements document?
2.1 Putting the user at the centre
"All technology operates within a context, and in designing products it is important to look at that context in addition to the technology itself. From this perspective, all technology is seen as being part of a wider system which must be designed correctly if that technology is to function appropriately."
USERfitIntroduction, p. 19, ECSC-EC-EAEC, Brussels-Luxembourg 1996.
Over the 5 years duration of the COGAIN Project, partners will be involved in developing technology that will (a) accommodate the specific individual needs and difficulties of users with disabilities and (b) meet as many of their overall requirements of Access Technology as possible. To help to put the next 5 years work into a context, therefore, this document will focus on providing information that will assist COGAIN partners in putting the real needs of end users at the centre of their research and development work.
Figure 2.1. Adapted from 'Diagram of Usability in a wider context', p. 15, ECSC-EC-EAEC, Brussels-Luxembourg 1996.
The aims of the COGAIN Project fit very well with the European Commission's TIDE 'USERfit' ideology (see Figure 2.1). Essentially, this promotes an approach that is:
- System Oriented
A user-centred approach is one, which argues that it is the end user's requirements, rather than technological capabilities, which drives product development. The logical starting point, then, is to take the time to understand the user population in some detail and understand what they need from products before going too far down the route of deciding on specific solutions. Too often, design is driven by technical feasibility, which can lead to a poor match to user's needs. A user-centred approach, on the other hand, is keen to ensure that the proposed solutions:
- Have real value for end users
- Are matched to user capabilities
- Are fit for the purpose for which they were designed
For these reasons, COGAIN is ensuring that potential users of Eye Control Technology should be involved as frequently and extensively as possible. This is why the ACE Centre, as specialists in the field of Access Technology, have been assigned with the responsibility of leading the process of facilitating the representation of user's views. This is a central element of the COGAIN Project.
In order to support COGAIN partners in making decisions about which user requirements to meet and how to meet them, it is important to put eye control into the context of disability and access technology as a whole. This document, therefore, sets out to consider:
- Who is currently able to use Eye Control Technology? (Chapter 3)
- Which new users need to be able to use Eye Control Technology who can't already? (Chapter 4)
- What are the potential alternatives to Eye Control? (Chapter 5)
- Usability issues - what are the potential benefits of Eye Control compared to other forms of access? (Chapter 6)
- What sorts of activities are people with disabilities currently using access technology as whole - not just eye control - for? (Chapter 7)
- If they can use eye control, what sorts of applications are people with disabilities currently using eye control for and how effectively? (Chapter 8)
With this information, it is intended that COGAIN partners will have the background knowledge required to make pragmatic decisions about which types of applications and which accessing issues to take into account in their research and development work under COGAIN. It is acknowledged that this can only be done within the context of a range of practical, reasonable considerations, for example:
- The quality and effectiveness of eye control hardware and software is, of course, dependent on variables such as the amount of development time and the budget available.
- Some users will have more limited requirements of this technology than others, so it is also reasonable to acknowledge that not all users will need to have a top-level eye control system with top-level assistive software.
Whichever decisions COGAIN partners make in meeting user requirements, it is important that users are closely involved as part of a dynamic, iterative process. It is, indeed, the responsibility of the ACE Centre as leaders of WP3 to ensure that the evolving and emerging needs and requirements of users are taken into account on an ongoing basis.
For this reason, this User Requirements Document should be regarded as a working document and one that evolves as the COGAIN Project progresses. Eye Control Technology is a developing field so many of those whose opinions have been sought in relation to this document will not yet have had the opportunity to base their opinions on direct experience. As a result, some of their expectations and requirements might not actually be achievable under COGAIN. As the project, progresses and those involved become more familiar with what is and is not possible through Eye Control Technology, their expectations of it will, of course, be shaped by events. It is important, therefore, that this document captures these evolving requirements. For this reason, it will be periodically reviewed and revised.
[Back to Contents]s]]
2.2 Data collection methods and background information
Because of the relatively small numbers of existing users of eye control systems and their wide geographical spread, it was anticipated that, within the timescale of this document, the numbers from whom information could be gathered and of whom observations could be made would be small. Similarly, the number of non-users with whom it would be possible to trial eye control systems with would also be small. For this reason, the approach to data collection involved a qualitative rather than quantitative approach to with a view to gaining an insight into the range of issues involved for consideration by COGAIN partners, rather than attempting to gather information of any quantitative significance, at this stage. The data collected, therefore, is intended to augment and enrich the information provided by the available literature.
Information from those supporting people with disabilities
- The views of parents and professionals (e.g. teacher, carer, health professional, assistive technology specialist) who support a people who have a disability were gathered through questionnaires[#_ftn6 ].
Information from end users
- Information from those people who have a disability and who are (or have been) regular users of eye control systems was gathered through observation and questionnaires[#_ftn7 ].
- Information from those people who have a disability and who do not use an eye control system (or have had only a little experience with one) was gathered through questionnaires and informal interviews.
- To gain an insight into the issues involved in calibration and utilisation of eye control systems with users who are not currently using eye control, a number of user trials[#_ftn8 ] were carried out with end users not currently using eye control.
COGAIN WP3 partners have written the background literature sections, collaboratively, in Chapters 3-8 inclusive.
An essential purpose of this document is to ensure that COGAIN partners are provided with information relating to:
- The wide range of user requirements that exist.
- The wide range of difficulties that need to be overcome in order to meet these requirements.
- The wide range of potential benefits of this technology in comparison with other access technology.
- The wide range of potential beneficiaries who are not yet able to access this technology.
With this information, it is intended that COGAIN partners will be able to share a common understanding of the range of user-centred considerations necessary to promote well-informed, pragmatic decisions about the hardware and software they develop.
3 Who is currently able to use eye control technology?
3.1 What the literature says
Current eye tracking technologies can be divided into the following categories: electro-oculography (EOG), scleral contact lens/search coil, video-oculography (VOG) or photo-oculography (POG), and video-based combined pupil/corneal reflection techniques (Duchowski, 2003). Not all of them are practical for interactive applications. For example, systems that use contact lenses are not practical or convenient for interactive applications, even though they can be very accurate for psychological or physiological studies, for example. Certain EOG systems are impractical for everyday use because they require electrodes to be placed around the eye to measure the skins electric potential differences. However, there are EOG systems aimed for augmentative and alternative communication, e.g. the EagleEyes system (Gips et al., 1993), Figure 3.1.
The most popular and suitable systems for use in interactive systems are video-based. Video-based combined pupil/corneal reflection techniques provide the Point Of Regard (POR) measurement: the system can calculate the direction of gaze (Duchowski, 2003). This requires taking into account the head movements in addition and relative to the eye movements. At least two reference points are required for the gaze point calculation (sometimes called the glint-pupil vector). By measuring the corneal reflection(s) relative to the centre of the pupil, the system can compensate for a certain degree of head movement.
Before the system can calculate the direction of gaze, it must be calibrated. This is done by showing a few (e.g. 9) points on the screen and asking the person to look at the points, one at a time. If the person is not able to direct his or her gaze and focus on the point (e.g. due to eye tremor), the accuracy of the calibration suffers or may become totally impossible. The duration needed to look at each calibration point may vary. For example, the LC Eyegazes calibration procedure simply waits for a good fixation before moving to the next calibration point (LC Eyegaze, 2001).
The corneal reflections are typically from an infrared (IR) or near infrared (NIR-LED) light source. Because IR is invisible to the eye, it does not distract the user. If the light source is located on-axis with the camera and the eye, it causes a so-called bright pupil effect. The light reflects directly back from the eye to the camera (similar to the red eyes effect in photographs taken with flash). If the light source is located off-axis, it the camera sees a dark pupil. There are variations in how well each eye tracking system can track the user's pupil. The size of the pupil has an affect on how well the system is able to track the persons eye. According to Nguyen et al. (2002), there are fairly big differences in the infrared bright pupil response of human eyes. The ethnic background has on effect on how the light is reflected from the retina. On the other hand, in techniques based on the dark pupil affect the colour of the iris matters; the contrast between the iris and the pupil must be distinguishable from the video image.
The camera must have an unobstructed view to the persons eye (especially the pupil) to be able to track it accurately. Eyelids or eyelashes may partially cover the pupil. The frames of the user's glasses may also obstruct the view to the eye, and the frames or the lenses may cause extra reflections. When contact lenses are used, the reflection is obtained from the surface of the contacts, instead of the cornea itself. Small, hard contact lenses may sometimes cause problems, if the lenses move around considerably on the cornea (LC Eyegaze, 2001).
For all the above-mentioned reasons, the calibration may fail even if the person is able-bodied and has normal vision. Indeed, Goldberg and Wichansky (2003) estimate that up to 20% of randomly selected subjects fail to get a good calibration. Eye tracker manufacturers typically report higher success rates. Lower success rates in real life conditions may be caused by ambient light or reflections from the environment. The user may not be sitting still enough for the system to be able to track him/her. Most systems also have an optimal distance and location concerning the positioning of the person in from of the eye-tracking device.
The problems are exacerbated when people who have severe involuntary head or eye movements try to use the systems. The person may also be lying on his or her back, or sit in a divergent position, e.g. her head rotated sideways. The current eye tracking systems have some ways of trying to cope with these problems. Most manufacturers provide accessories for positioning the eye-tracking device (and the computer screen) so that the distance and the angle of the tracker are adjustable (e.g. by using an adjustable arm). If the camera loses the eye, some of the systems automatically start searching for it. For example, the Metrovision VISIOBOARD systems camera automatically moves and scans the surroundings, trying to relocate the 'lost' eye. Whilst this can be very effective to 're-locate' the eye of a user who has moved away from the VISIOBOARD and then returns to it, it is not designed to work at the speed necessary to accommodate the head movement of a user who has ongoing, 'jerky' involuntary head movement.
The Tobii eye-tracking device tracks both eyes and is able to temporarily continue with one eyes data if the other eye is lost (Tobii, 2003). Whilst all current eye trackers have problems in tracking the eye(s) if the users eyes have a tremor or if the user has involuntary head movements, initial trials by The ACE Centre have suggested that the extent to which each system can 'cope' can vary considerably. No comparisons between the different systems in terms of their efficacy in this respect are available. For this reason, it is important for COGAIN to compare different systems with users who experience these difficulties.
If the calibration fails for the user, some systems can be used with a default calibration, and a special filtering of eye movements can be applied if the user has eye movement disorders (Charlier et al., 1997). For example, in the Metrovision VISIOBOARD system allows to use 3 x 2 matrix for people for whom the calibration fails. Naturally the accuracy of the measured point of gaze is very rough in such cases, and the objects on the screen must be large, e.g. only 6-9 large buttons visible on the screen at a time. If only a few buttons are visible at a time, normal on-screen keyboards cannot be used. The characters or commands must then be organized hierarchically. This makes the use of the system slow, because several selections are needed for one command. By using predictive algorithms, the system can speed up the selection process (Frey et al., 1990; Hansen et al., 2003b).
Figure 3.4. Two-level buttons (EagleEyes Staggered Speech application).
Whilst it is acknowledged that the use of larger targets can be slower because it might necessitate two 'hits' instead of one, it must be remembered that larger targets can make the difference between a user being able to use an eye-tracking device effectively and not using it at all.
Figure 3.5. Paul (left picture) was able to use a grid with smaller cells, whereas larger cells were more appropriate for Michael (right).
Making the cells in a grid larger can make the difference between accessing a target with two or three hits and not being able to access it at all (see Figure 3.5).
When compared with scanning, even two or three 'hits' to achieve a required target might be far less tiring, quicker and more efficient for some users than scanning. (Refer to Ahmar's case study in Chapter 6.2).
However, if everything else fails, the eyes might also be used as simple one or two-way switches and the focus can be shifted from one item to another by using a method called scanning (ten Kate et al., 1979). For example, the H.K. EyeCan VisionKey system uses a sequential scanning and requires only coarse vertical eye movements for selection (Kahn et al., 1999).
It must be noted, however, that, in practice, scanning can be considerably slower, and therefore considerably more frustrating than direct eye control for the user so that, given a choice, it is likely that the vast majority of users would choose eye control as opposed to eye blink.
Most eye tracker manufacturers report that the spatial accuracy of the system is about from 0.5 to 1 degree. Even if the calibration succeeds and the measured point of gaze is fairly accurate right after the calibration, the tracking may fail after a while because of the deterioration of a calibration that occurs over time; the calibration drifts. The drift may be caused by the change of pupil size, the eyes may become dry, or a considerable change of the angle, position and distance of the persons eyes in relation to the eye-tracking device. Systems that track both eyes cope with the drift better, because most drift effects are inversely symmetrical between the eyes (Tobii, 2003). There are also algorithms that can dynamically correct the drift (Stampe and Reingold, 1995). The accuracy of the calibration is checked on every successful selection and can then be gradually adjusted over time. The Metrovision VISIOBOARD also allows recalibration on any part of the screen. When the user fixates on any place of the screen for predefined period of time (longer that the normal duration/dwell time needed for a single or double click), a calibration point appears on that spot.
If eye gaze is used to control a mouse, not even 0.5-1 degree accuracy is enough for selecting the small elements in the standard graphical environment. The accuracy problems can be at least partly overcome by using special methods for selecting the tiny objects on the screen. For example, The Quick Glance, VISIOBOARD and ERICA use zooming, screen magnifiers and fish-eye views to select tiny icons and menus on the screen (Lankford, 2000; Rasmusson et al., 1999)
The eye-tracking devices can be table-mounted (remote) or head-mounted (worn on the head). If the device is used for interactive applications, heavy head-mounted trackers that must be tightly attached to the users head are not convenient. There are, however, lightweight head-mounted eye-tracking devices that can be used for a prolonged period of time without too much inconvenience. For example, the VisionKey by H.K. EyeCan is mounted on a pair of standard frames. If the eye-tracking system is used for social communication, the user is likely to want to use it outdoors, on the move, as well as indoors. While systems like the LC Eyegaze can be mounted to a wheelchair, and move around with the user, they are not designed for outdoor use.
With the Quick Glance II system, as with other control systems, the user's eye must be kept within in the camera's field of view. This is about 6 by 6 cm for models 2B and 2S. With the 2SH model, however, the area of 'Allowable Head Movement' is about 10 x 10 cm for model.
This model, therefore, would be the one to try for someone with involuntary head movement because they would stand a better chance of keeping maintaining the camera's view of their eye movement.
As with all eye control systems, however, the only way to evaluate their efficacy is to carry out trials with a range of users to find out how they work, in practice. For this reason, an important aim of COGAIN, on behalf of end-users, is to collaborate with them in order to provide information to assist them in making well-informed choices. This information will relate to the wide range of eye control systems available in terms of usability issues, the software that is already being used for eye control and the software that could potentially be used for eye control. In this way, we can provide support to end users and those who support them in answering such questions as this, from the questionnaire of a parent of a child who she would like to consider eye control for:
"How do we find out about the suitability of different products?"
It is anticipated that Work package 3, in collaboration with COGAIN partners, will help to provide some answers. [Back to Contents]
3.2 Information from stakeholders
One of the aims of COGAIN Work Package 3 (User Requirements) is to trial software applications developed under COGAIN. To do this, there is a need to find out where the existing users are. Establishing direct contact with those who are already using Eye Control technology in any significant numbers is not a quick and easy process but one that will evolve over time. It will necessitate, for example, raising awareness and gaining the trust and support of a range of manufacturers, suppliers, charities, etc. Publicity for COGAIN, too, through magazine and journal articles, etc. is an essential part of the awareness-raising process and this process is already well underway throughout the COGAIN Project. It is anticipated that the impact of these efforts, in terms of establishing direct contact with a community of existing eye-control users, will be felt increasingly as the project progresses. In the meantime, though the numbers of existing long-term eye control users we have established direct contact with is small, what we have learnt from them is both rich and informative. It provides a revealing insight into what can currently be achieved and what might be achieved in the future.
Contact has already been established with a very severely disabled eye control user called Keith. The information acquired from him provides a valuable insight into the impact of eye control technology on his quality of life:
Case Study - Keith
Keith has ALS. He is completely paralysed and is only able to move his eyes. He can no longer move his head at all. He cannot blink. This means that he experiences great difficulty and discomfort and it is necessary to keep clearing his eyes in order to prevent a film building up on them.
He has been using an LC technologies Eyegaze system for 2 years and he controls it from an upright position in his wheelchair. He uses the system for about 12 hours daily. He runs his laptop through the LC system.
He would also like to be able to use the Eyegaze to take control over his environment "so I could be more independentto change TV channels, turn lights on and answer phone". He says he could do this, but he just has not purchased that system.
Whilst he has not timed the number of words per minute he achieves, but the 'eye response control' (or 'dwell select') is set to 0.20 seconds, which suggests an extremely proficient user. In the experience of The ACE Centre, achieving such a speed with any other pointer method than eye control (e.g. a headmouse) would be very difficult indeed. Eye-writing is quicker for him than when he was able to use his fingers to type: "I am faster with my eyes than I ever was when my fingers used to work".
He regards the process of eye-control of the computer as essential to his quality of life ('I would have no desire to live without this eyegaze system') and uses it for a range of activities. He uses it for social communication, writing, emailing and access to the Internet. Eye-writing is his "only way of communicating". It enables him to "still be a part of other people's lives. Plus, I can still give advice and help others." Through emailing, he keeps in daily contact with people: "It gives me an outlet to feel like I can still make a difference on somebody's life". The Internet is his "only way of keeping up with what's going on in the outside world".
If it were possible, he feels that wheelchair control using his eyes would be very beneficial, as it would provide him with "freedom from always having to ask others for help".
At present, whilst certain systems have features that are intended to accommodate certain accessing difficulties, in practice, their effectiveness will vary for different users. Because of the lack of comparative information, it is important that, under COGAIN, the efficacy of different systems with different types of users is compared in order to provide them with the information they need to make a well-informed choice. Similarly, the way in which the software interface is designed (e.g. magnification, larger targets, etc.) offers opportunities to enable more users with accessing difficulties (e.g. those with involuntary head and eye movement) to access this technology effectively. A range of hardware and software will need to be trialled under COGAIN for comparison in relation the efficacy of its use for users with a wide range of needs.
Eye tracking systems aimed for people with disabilities include:
- EagleEyes (http://www.bc.edu/schools/csom/eagleeyes)
- Eye Response Technologies ERICA (http://www.eyeresponse.com)
- EyeTech Digital Systems Quick Glance (http://www.eyetechds.com)
- H.K. EyeCan VisioKey (http://www.eyecan.ca)
- LC Technologies Eyegaze (http://www.eyegaze.com)
- Metrovision VISIOBOARD (http://www.metrovision.fr/)
- TechnoWorks TE-9100 Nursing System (http://www.t-works.co.jp/page011.html)
- Tobii Technologies MyTobii (http://www.tobii.com/).
It is important that as many such potentially beneficial systems (and appropriate software) are trialled by a cross-section of users and potential users as possible and the information gathered made available to all users and potential users.
4 Who is currently unable to use eye control technology?
At present, Eye Control Technology finds it difficult to cope with users who have certain physical or visual difficulties. These include, for example, those with involuntary head movement (e.g. those who have athetoid cerebral palsy) or involuntary eye movement (e.g. due to nystagmus). This chapter sets out to explore the wide range of difficulties and abilities of users with disabilities, who wish to use this technology but who, at present, are unable to do so effectively. It will consider the reasons for this and indicate the technical difficulties that need to be overcome in order to include this currently excluded group.
4.1 What the literature says
Eye tracking for disabled people who have various physical difficulties resulting from, for example, Amyotrophic Lateral Sclerosis (ALS), head Injury, cerebral palsy, Multiple Sclerosis, Muscular Dystrophy, Spinal Muscular Atrophy, Werdnig-Hoffman syndrome, Rett syndrome, Spinal Cord Injuries resulting in Quadriplegia and "locked-in syndrome" may have difficulties using eye control technologies with respect to two possible sources of influence.
The first one is tracking the eyes themselves in terms of reliability and validity, i.e. in order to further process eye tracking data. First of all this data must reflect a) the true gaze position and b) the true temporal characteristics of the eye movement in question. The ability to reliably track users eyes is a prerequisite for the second problem: the eye tracking software has to interpret the data and respond to the user's intention in an appropriate way. From another point of view, the first problem is one of the eye-tracking hardware and the second a problem of the application software using the data from the eye tracking hardware. Influences on the hardware are closely associated with the physical abilities of the users. It is worth noting that that these physical abilities may also have an impact on certain on eye tracking in healthy subjects, but some of those listed in the next section are more prevalent in users with the aforementioned diseases. Since literature on the impact of physical abilities on eye tracking is sparse, most of the information given below came from practitioners with extensive experience.
The following physical conditions may make successful eye tracking difficult[#_ftn9 ]:
4.1.1 Difficulties with eye control
To have full control over commercially available eye control (as opposed to eye-blink), systems users must be able to look up, down, left and right. They must be able to direct their gaze on all areas of a computer screen. At a typical distance of a user from the screen this means that a user must be able to move their eyes approximately 25 of visual angle horizontally and 19 of visual angle vertically. In addition, they must be able to focus on a spot for at least 500ms in order for most eye typing systems to work properly. There are certain common eye movement problems that make a satisfactory calibration difficult to achieve:
This is a constant involuntary horizontal, vertical, or rotary movement of the eyeballs. It may lead to the inability of users to fix their gaze long enough to make selections. The problem here is that simply reducing the fixation dwell time threshold can increase the risk of involuntarily making selections (see Midas Touch problem below).
This visual difficulty (also called heterotropia or tropia) results in an inability to attain or maintain binocular vision due to some type of muscle imbalance in one eye, causing that eye to turn in, out, up, or down relative to the other eye. Strabismus can be intermittent (occurring sometimes), constant (occurring all the time), and/or alternating (occurring sometimes with one eye and sometimes with the other eye, whether intermittently or constantly). The problem for eye tracking arising from alternating strabismus is that for monocular systems it is impossible to make sure that the tracked eye is the one not deviating from the line of sight. One solution to this problem is putting a partial eye patch over the nasal side of the eye not being observed by the camera that often solves this tracking problem. Since only the un-patched eye can see the screen, it will continuously focus on the screen. By applying only a nasal-side patch to the other eye, the user will retain peripheral vision on that side.
Visual acuity issues:
Several common vision problems may affect a user's ability to see visual elements clearly on an eye tracking system monitor. These include the following: A) Inadequate Visual acuity: The user must be able to see visual elements on the screen with adequate acuity. If, prior to an injury or the onset of an illness a user wore glasses, he may need corrective lenses to operate an eye tracking system. If a user is over 40 years old and has not had his vision checked recently, he might need reading glasses in order to see the screen clearly. In most cases, eye tracking works well with glasses. The calibration procedure should accommodate for the refractive properties of most lenses. Hard-line bifocals can be a problem if the lens boundary splits the image of the pupil, making it difficult for a system's image processing software to determine the pupil centre accurately. Graded bifocals, however, typically do not interfere with eye tracking. If users wear contact lenses that cover all or most of the cornea eye tracking generally works well. In this case, corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can interfere, if the lens moves around considerably on the cornea and causes the corneal reflection to move across the discontinuity between the contact lens and the cornea. B) Diplopia (double vision): Diplopia may be the result of an injury to the brain, or a side effect of many commonly prescribed medications, and may make it difficult for the user to fix his gaze on a given point (see strabismus). C) Blurred vision: Another occurrence associated with some brain injuries, as well as a side effect of medications (see below). Blurred vision of screen elements decreases the accuracy of eye fixations. D) Cataracts (clouding of the lens of the eye): If a cataract has formed on the portion of the lens that covers the pupil, it may prevent light from passing through the pupil to reflect off the retina. Without a good retinal reflection, eye-tracking systems based on the bright pupil method cannot accurately predict the user's eye fixations. The clouded lens may also make it difficult for a user to see text on the screen clearly. Surgical removal of the cataracts will normally solve the problem and make the use of eye tracking possible. E) Homonymous hemianopsia (blindness or defective vision in the right or left halves of the visual fields of both eyes): This may make calibration almost impossible if the user cannot see calibration points on one side of the screen.
4.1.2 Ability to maintain a position in front of the Eyegaze monitor
It is generally easiest to run an eye tracking system from an upright, seated position, with the head centred in front of the Eyegaze monitor. However, the eye tracking system should be able to be operated from a semi-reclined position if necessary.
4.1.3 Medication side effects that affect eye tracking
Many commonly prescribed medications have potential side effects that can make it difficult to operate an eye tracking system. Anticonvulsants (seizure drugs) can cause nystagmus, blurred vision, diplopia, dizziness, drowsiness, headache and confusion. Some antidepressants can cause blurred vision and mydriasis (abnormally dilated pupil). Some drugs commonly used to decrease muscle spasms (e.g. Baclofen), can cause dizziness, drowsiness, headache, disorientation, blurred vision and mydriasis. Mydriasis can be severe enough to block eye tracking. If the retinal reflection is extremely bright, and the corneal reflection is sitting on top of a big, bright pupil, the corneal reflection may be indistinguishable and therefore unreadable by the computer.
In addition to these physical prerequisites, which can make tracking of users eyes impossible, there is the second problem of how to interpret the gaze data of an eye tracking system. Jacob (1995) pointedly termed this difficulty the Midas Touch Problem: The System has to differentiate attentive saccades with an intended goal of communication from the low-level eye movements that are just random or provoked by external stimulation. In order to answer this question one needs a model of eye movement control in relation to information processing tasks. First of all a very basic function of eye movements is search. If a potentially important object or feature has been discovered it should be perceptually described. Such identification is only hardly possible without eye movements (see also Findlay and Gilchrist, 2003). After identification, the information can be semantically categorized and some of the information may also receive additional self-refential interpretation. Only at this point, the information may be considered to be ready for a possible communication. The crucial parameter to distinguish different levels of processing and thus different states of intention is fixation duration. In many experiments (e.g. Velichkovsky, Pomplun and Rieser, 1996) a threshold value in the order of 450-500ms has proved as a practical solution to the Midas Touch Problem. In terms of eye-tracking-based communication systems, the eye-tracking data should hence be temporarily filtered to avoid unintended selections.
4.2 Information from stakeholders
Under the COGAIN project, it is anticipated that an effective methodology for evaluating the usability of a range of systems and software will be developed in relation to the wide range of user's abilities and requirements. However, to gain a small insight into how well eye control technology could cope with certain kinds of physical and visual difficulties, a small number of brief user trials was carried out. The aim was simply to find out how good a calibration (if any) could be achieved with a small number of users who had visual difficulties, difficulties with head control, or both. In all, there were 10 users involved in these 'introductory' user trials. They included children and adults whose range of difficulties included head injury, stroke, cerebral palsy and athetoid cerebral palsy. With several of them, along with their physical difficulties, they had an associated visual difficulty, e.g. divergent squint, nystagmus.
Our (very limited at this stage) user trials suggested that, to a degree, certain eye control technology is already able to cope with certain users who have certain difficulties with head and/or eye control. In turn, even if a calibration has been successfully carried out, eye control might only be effectively be used up to a certain level of accuracy. Two examples follow:
Case study - Claire
Claire has athetoid cerebral palsy, which means it is very difficult for her to control her movements. She is very bright, literate and well motivated. She uses a special joystick to access the computer. Using the joystick and a range of specialised on-screen grids designed within SAW, she is able to use the joystick effectively and accurately to control the computer.
Nonetheless, the method is very time consuming and involves a great deal of physical effort for her because, with her particular condition, there is a great deal of involuntary movement whenever she tries to carry out a manual task. Just reaching out in order to grasp the joystick handle in the first place is, in itself, very difficult, with her hand and arm sometimes 'overshooting' the target. It is not just hand movement that has this effect. Even if Claire just tries to speak, this also triggers off a range of involuntary movements and this, too, can be tiring for her. On the other hand, when there are no physical demands on Claire, she has learnt to sit reasonably still with comparatively little involuntary movement.
When a calibration was tried with Claire, using a Tobii eye control system, the results were encouraging. The system did manage to cope with a certain amount of involuntary head movement, whether forwards, backwards or sideways. When she tried 'eye-writing', using 'dwell-select', despite the cognitive load of concentrating on eye-typing using an unfamiliar grid layout she still managed to remain comparatively still. Nonetheless, the targets had to remain reasonably large to maintain accuracy, using the type of grid shown in Figure 4.3.
Figure 4.4. Claire using two 'hits' to select a 'y' to write the word 'party'.
This grid was simply 'experimental', with no special facilities such as prediction, etc. However, Claire's successful access to the cells she wished to select gave the following, encouraging indications:
- The Tobii system was able to achieve a calibration that was accurate enough for this user who had a certain amount of involuntary head movement to accurately select a letter she required for eye-typing with two hits, e.g. selecting a cell with the letters a-f and then selecting the actual letter she wanted.
- Even without specialist or customised software, there were already strong indications that eye control might have the potential to be (a) quicker (b) less tiring and, because of the sharp reduction in involuntary head movement, more comfortable for Claire.
Case study - Michael
Michael is in his early 40's and has a wife and three boys. He had a severe stroke about 2 years ago and now, after rehabilitation, is back home. He cannot speak but communicates by looking at letters on an E-tran frame[#_ftn10 ].
Figure 4.5. Michael using the E-tran frame to communicate with his wife, Wendy.
Before his stroke, Michael was very active and enjoyed a wide range of leisure pursuits. Despite the stroke, he remains a very intelligent man with an excellent sense of humour. He would like to be able to access the computer quickly and efficiently in order to communicate socially and assist with his wife's business. However, at present, his only form of access to technology is via switches. This he finds very slow and frustrating and would very much like to use eye control as a quicker and easier method, if at all possible.
Because of the stroke, Michael has a certain amount of difficulty with head control. In addition, he has nystagmus, which means that he cannot fix his gaze in the same way that most other people can. Both of Michael's eyes have a significant amount of involuntary side-to-side movement (nystagmus) to become more severe when he is tired.
A Tobii system was tried with Michael. With this particular system, if there are any specific areas that require re-calibration, they can be selected and recalibrated individually.
The grid used was 'experimental', with no special facilities such as prediction, etc. The first time it was tried, in the morning, the Tobii system was able to produce a calibration that was accurate enough for Michael to be able to select a letter he required for eye-typing with two hits, e.g. selecting a cell with the letters A-F and then selecting the actual letter he wanted. Unfortunately, when it was tried in the afternoon, when Michael's involuntary eye movement was greater, finding a calibration that would enable him to reliably access the same size of cells proved very difficult. The reason for this can be clearly seen in the screenshots below:
Figure 4.9. 'Before and after' - the calibration on the left was good enough for Michael to eye-write using a large grid
but with a later calibration (right) he was unable to use the same grid effectively.
Two other eye gaze systems were tried with Michael on separate occasions, an eye analysis system under development by QinetiQ, and the Quick Glance II SH. Despite three specialists from QinetiQ working with Michael for several hours, it proved impossible to achieve a satisfactory calibration. The Quick Glance, too, was not particularly successful. Even though a calibration could be achieved, albeit with difficulty, the level of accuracy was very poor indeed - only good enough, for example, to very roughly target each area of a 2 x 2 full-screen grid. Even a 2 x 2 grid, however, could not be used functionally, because the mouse movement was not sufficiently 'dampened' or filtered. For example, it was impossible to achieve any kind of dwell select because the pointer was jumping about so much.
The user trial with Michael gave the following indications:
At present, whichever eye control system of those available was tried, it would be difficult to achieve an effective calibration for Michael when his nystagmus is at its most severe.
Even if a successful calibration can be achieved, account must be taken of the fact that the nature of Michael's visual difficulties fluctuates and a successful calibration at one moment in time might not work effectively for him on another occasion.
On the occasion when Michael did achieve a successful calibration, the significant increase in speed that he was able to achieve in comparison with switches, combined with his greater comfort and satisfaction that the Tobii gave him, meant that he was extremely enthusiastic about eye control as an access method.
His enthusiasm emphasised the importance of the need for developers to try to accommodate people like Michael by taking his kinds of accessing difficulties (involuntary head movement and fluctuating visual difficulties) into account.
As described above, the questionnaire response from existing users of eye control was small. However, it was interesting to find a long-term user with athetoid cerebral palsy amongst those who are already an eye-control system successfully. This is encouraging, as many of those with this condition have a certain amount of involuntary head movement. Obviously, the visual and physical abilities and difficulties of any person with any given condition is different form any other person, whether they have the same condition or not. As a result, the amount of involuntary movement that results from having athetoid cerebral palsy, for example, will vary considerably. Therefore, if developers wish to meet the needs of any group of users with complex physical or visual difficulties, therefore, they will need to ensure that their systems are flexible enough to accommodate a wide range of differences.
It is important for COGAIN partners to remember that, despite the benefits that some people with disabilities are already enjoying though the use of eye control technology, many of those who would gain most benefit from its use are still excluded from using it. It is hoped that, by working closely with these potential users, many more of them will be able to use eye control technology by the time the project is completed.
5 What potential alternatives to eye control are there?
This chapter describes a selection of access methods that might be considered as alternatives to eye control to provide a context for the use of Eye Control Technology within the wider fields of Access Technology and Disability. Without such background information, eye control technology could be applied to meet a user's needs inappropriately when alternative access technology might be better.
Eye Control Technology is just one of many ways in which even the most disabled users can access the computer. It is very important, therefore, that those who are involved with it have a clear understanding of where it stands in relation to the very wide range of hardware and software available to the disabled community. Without such an understanding, there is a danger of going to all the trouble and expense of implementing the use of Eye Control Technology with a disabled person when perhaps a cheaper, simpler, more reliable or indeed more effective control method could be provided. It is not within the remit of this document to provide information on every single access device, every single piece of Access Technology and every single piece of access software. These amount to many thousands. However, it is important that COGAIN partners are aware of the categories and purposes of access methods and devices that might provide alternatives to Eye Control Technology. An overview of the range of alternatives to Eye Control Technology is now provided in order to provide a clearer understanding of the issues involved in considering the potential benefits and limitations of Eye Control Technology in relation to other types of Access Technology.
5.1 What the literature says
There are many alternative ways in which assistive technology devices can support people with disabilities. Most of these require a controlled physical movement. Some people might use a part of their body such as a fist, finger or foot. Some might use the blink of an eye or head movement. Obviously this needs to be a deliberate movement and not accidental, such as a tremor. A key issue is that the input device must be activated with a minimum of effort in relation to the result achieved, without causing any kind of discomfort.
Computer input devices for people with disabilities are classified by different features. According to Shein et al. (1992), there are three main groups:
- Pointing devices
In turn, there are 3 ways in which these input devices can be operated:
- Mechanical interaction
- Optical interaction
- Electrical interaction
Mechanical interaction requires the user to make a single, deliberate movement, to activate a switch, e.g. by applying direct pressure or blinking or by breath. With 'optical interaction', interaction between the user and input device occurs by means of light waves in visible or infrared form. With electrical interaction, devices use acquired electrical signals directly from the users skin or tissues.
Also it is pointed out (Shein et al., 1992) that proper seating and positioning that provides stabilization and support for the person with a physical disability. They emphasize that it is the foundation for promoting effective interaction with a computer.
5.1.1 Devices with direct mechanical interaction
Firstly, there are switches that require direct mechanical interaction. A wide range of switches is available. Their shape, size and the pressure required to activate them depends on the requirements and abilities of the user. Some, for example, provide tactile or auditory feedback. These are a few examples from a huge range of choices:
- Pillow Switch, which is suitable for activating with the jaw
- Floor/Foot Switch
- Mercury switch that is activated by tilting 5 degrees from horizontal position
- Grasp Switch
The last one is designed to be activated with a gentle squeeze or pinch. The user must be capable of a well-controlled grasping movement and able to release the grip within a short period of time. One of the special facilities of the switch is that, if it is held for over 2 seconds, then the switch will latch or lock on. If the user does not have a controlled grasping movement, therefore, then the Grasp Switch is not the appropriate switch for them. Below is an example of a jaw-activated switch. Here movements of the jaw are used to activate the switch (Fig. 5.1). The switch also has a headset for auditory feedback.
Figure 5.1. Jaw activated switch.
1, 2 earpieces, 3 head set support, 4 chin strap, 5 adjustable securing means,
6 oval chin cup, 7 chin bar, 8 fastening means, 9 chin bar electrode, 10 conductive wire.
An example of a dual switch is a Sip/Puff switch that is activated by inhalation/ exhalation. The Origin Instruments Corporation Sip/Puff Switch (Fig 5.2) is a head-mounted accessory used to activate a two-position switch by a simple sip or puff. It consists of a head frame with attached mouth tube and a switch box connected to the head frame by a second plastic tube. Sips and puffs are converted to switch closures inside the Switch Box.
In some cases, it is not necessary to develop a new input device to enable a user to interact with the computer. One alternative is to provide the user with a special device that acts as an interface between them and a mainstream input device. For example, for users who find it difficult to control their hands but who can move their heads effectively, a special helmet can be worn. Attached to the helmet is a 'wand' with which they can access an ordinary keyboard. Alternatively, instead of attaching an enabling device to the user, a device can be attached to the mainstream device itself to make it more accessible. For example, a keyguard can be attached to an ordinary keyboard. This can enable those who have a tremor or other difficulties when trying to access a keyboard to use the holes in the keyguard to accurately guide their fingers to the target keys.
Joysticks manipulated by hand, feet, chin, tongue are used to control the cursor on screen. For example, the "Tonguepoint" system (IBM Trackpoint III Trademark) is comprised of a pressure-sensitive joystick fastened to a mouthpiece that can be operated by the user's tongue (Salem and Zhai, 1997). The joystick provides cursor-control, while two switches, a bite switch and a manual switch, allow the user to use the left/right click buttons.
5.1.2 Devices with optical interaction
Optical systems rely on measurements of reflected or emitted light. These systems inevitably have two components: light sources and optical sensors. The light sources might be passive objects that reflect ambient light or active devices that emit internally generated light. Examples of active light sources include light-emitting diodes (LEDs), lasers, or simple light bulbs. Optical sensors can be either discrete devices (photodiodes, phototransistors and other), or image sensors (CCD, CMOS).
Lenses and apertures can be used to project images onto the sensor, indicating the angle to the source. The intensity of light reaching the sensor can be used to estimate the distance to the source. Filters can be added to selectively admit or reject certain wavelengths of light. For example, a sensor system might use infrared light sources in conjunction with filters that only admit infrared light, effectively providing a light channel separate from the ambient visible light. Examples include an eye blink detector or the Headmouse from Origin Instruments
The technology to detect a deliberate blinking action is based on the different amount of reflection from an opened and closed eye. The detection bracket houses both the IR emitting diode and corresponding IR photodetector (phototransistor) (Fig. 5.3) and can be clamped onto the frame of a normal pair of glasses (Shaw et al., 1990). The emitters and detectors (one on each side of the glasses) are focused at the same spot on the sclera, at the lateral corner of the eye. Since the eyelid is both more absorbent and less reflective than the sclera, the IR beam will be less strongly detected when the lid is closed. Thus, a clear threshold can be established between the "open" and "closed" states of the eyelid.
By setting up a 'dwell' time that exceeds the users normal blink time, the system can be set to be activated only when the user makes a deliberate 'wink'. In this way, the eye-blink becomes a single switch and can perform the same function as a mechanical switch pressed with the hand or any other part of the body. Making different dwell times perform different functions can extend the functionality of the eye-blink switch. For example, deliberately closing the eye for 0.5 seconds would produce a different result on the computer to a 'blink' of 1.0 second.
Figure 5.4. Optical blink detector. (left) basic concept, (right) detectors on bracket (Shaw et al., 1990)
For those who have good head control, there are various commercial mouse alternatives. Some systems use infrared emitters that are attached to the users glasses, headband, or cap. Other systems place the transmitter over the monitor and use an infrared reflector that is attached to the users forehead or glasses (see Fig.5.4). The users head movements control the mouse cursor on the screen. Mouse clicks are generated with a physical switch or a software interface. Evans et al. (2000) recently described a head-mounted infrared-emitting control system that is a relative pointing device and acts like a joystick rather than a mouse. Chen et al. (1999) developed a system that contains an infrared transmitter, mounted onto the users eyeglasses, a set of infrared receiving modules that substitute the keys of a computer keyboard, and a tongue-touch panel to activate the infrared beam.
From the commercial sector, the Headmouse uses a infrared to turn head movement into cursor control. Mouse functions can be controlled by dwelling over a key for a set period of time (dwell-select). Alternatively, an external switch or switches can be set up to emulate the mouse button(s), such as a. Sip/Puff Switch.
It must be acknowledged that helmets, electrodes, goggles, and mouthsticks can be uncomfortable to wear or use. Commercial head-mounted devices can often not be adjusted to fit a childs head. However, some users, in particular young children, dislike to be touched on their face and might object to any devices being attached to their heads. Non-mechanical switches, such as the Infrared Jelly Bean switch from AbleNet, Inc. can be operated by moving any part of the body through an infrared beam.
There is a range of software available that enables a camera to be used for tracking used to track body movements such as head movement. The 'CameraMouse' is software that enables for hands-free control of a computer using a video camera to track body movements, (head, for example), and convert those movements into cursor movements on a computer screen. An on-screen toolbar can allow the user to emulate all of the mouse controls.
The CameraMouse works with all standard software and requires no wires, dots, infrared beams, or anything else to be worn on the head. It tracks a body featurefor example, the tip of the nosewith a video camera and uses the detected motion of the feature to directly control the mouse pointer on a computer. The CameraMouse system currently involves two computers that are linked togethera 'vision computer' and a 'user computer'. The vision computer executes the visual tracking algorithm and sends the position of the tracked feature to the user computer. The user computer interprets the received signals and runs any application software the user wishes to use. CameraMouse will work with most USB Cameras that utilize the CCD image sensor. The Logitech QuickCam Pro 4000, 3000, Orbit and Intel Pro Video PC USB cameras have been lab tested and work well. A single-computer version of the system ha also been developed.
5.1.3 Electrical interaction
Another approach is the Brain-Computer Interface (BCI[#_ftn11 ]) that uses electroencephalographic (EEG) waves originating in the brain. Users interact with physical devices through nothing more than the voluntary control of their own mental activity. Between the simple EEG and the extremely invasive direct recording of neurons, a researcher might reasonably consider using another established brain-imaging technique (such as magnetoencephalography, functional magnetic resonance imaging, and emission tomography). Nevertheless, all such techniques require sophisticated devices that can be operated only in specially designed medical facilities.
One of the major limitations of BCI systems is the high potential for electromyographic (EMG) contamination. Any muscle movement on the head or neck can produce "noise" contamination from the corresponding EMG signal. From an application standpoint, this can result in difficulties for the user, especially if they have a movement-related disorder such as cerebral palsy.
The EMG/EEG-based Human-Computer Interaction system is an alternative to eye tracking systems and when combined with an on-screen keyboard is fully operational without the user having to initiate a physical movement. The system transforms biosignals into controls for two-dimensional cursor movement. In contrast to eye tracking systems, the HCI system has the potential to be relatively inexpensive and portable and eliminates some of the concerns related to eye tracking systems, such as "dwell time", user training, and loss of calibration.
Current approaches to EEG-based communication can be divided into two groups, those that use time-domain EEG components and those that use frequency-domain components. Frequency-domain methods use spectral analysis and focus on specific frequencies at specific scalp locations. The BCI developed at the Wadsworth Center uses mu (812 Hz) and/or beta (1825 Hz) rhythms recorded over sensorimotor cortex to control a cursor on a video screen. In the simplest case, the amplitude of a single spectral band at a single location on the scalp is used to determine one-dimensional (1-D) cursor movements. The user learns to control this amplitude. Cursor movements are calculated 10 times per second by an empirically derived linear equation. In offline analysis, data from the most recent sessions are used to determine the best location and frequency band for cursor control for the next sessions.
Figure 5.8. Placement of electrodes in a cap for EEG measurement (Millan, 2003).
The 'Cyberlink Interface' is a commercially available system that enables hands-free control of computers and electrical devices. The Cyberlink system detects brain and body signals by the sensors in the headband.
Figure 5.9. The Cyberlink system detects brain and body signals by the sensors in the headband.
These are amplified, digitized and transmitted to the computer in order to control a mouse or an interactive video game, navigate a business application, use a web browser, control almost any Windows application, play musical synthesizers, activate peripheral devices, adjust environmental controls, etc. It has been the experience of the ACE Centre that not all users experience success with system and those who do would find it hard to match the speed of an effective eye control system.
5.2 Information from stakeholders
In the questionnaire, potential eye control users were asked how they felt eye control would compare with their existing access method, in general. Where an opinion was expressed (6 out of 8) they felt that eye control would be 'better' or 'much better.' Their reasons given included:
- 'His eye pointing is very good so I would hope that it would be easier and quicker.'
- 'Less effort, greater choice, more control.'
- 'Her head control is good but becomes unreliable when she is trying to press a switch.'
- 'From the limited testing I have donepotential for increasing my access speed, especially when my body gets tired.'
However, there were certain reservations expressed. These included:
- Potential difficulties in ensuring the correct positioning of the eye control device.
- Outdoor use.
They were also asked specifically about the potential benefits of writing using eye control. Of the 8 potential eye control users who expressed an opinion, 2 thought it would be 'beneficial', 4 thought it would be 'very beneficial' and one thought it would be 'not very beneficial' (he already accesses the computer effectively with mouse and keyboard). Their reasons were related to enabling greater choice, ease of use, independence and speed. Comments included:
- It would allow her to have complete flexibility over what she wants to say.
- It is likely to be easier, quicker and more accurate.
Similarly, positive comments were made about using eye control for all 'standard' PC applications, such as Internet access, email, leisure software, etc. Against these potential benefits, however, there were reservations expressed by the some of the professionals in relation to reliability and/or safety issues when it comes to either environmental control or powered mobility.
As described earlier, facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to control their environment. Of the 3 eye control users, one did not comment. Another, who already uses eye control for their environment, feels it is 'beneficial.' The other, who does not have eye control over their environment, feels it would be 'very beneficial.' Their reasons included the following:
- "I'm doing things on my own"
Of the 8 users of other forms of access technology, 4 thought it would be 'very beneficial', 2 'not beneficial', one was able to use standard technology already and the other did not comment. Reasons for considering it to be beneficial included:
- Increased independence, privacy.
- 'He has more control over his eyes than his hands so it is likely to be easier, quicker and more accurate. It would increase the options available to him.'
Their reservations related to potential difficulties with reliability and reduced portability.
- 'It would be a lot easier but reliability would be a big concern. My independence is very important and I simply would not trust it. My environmental control system is extremely portable. I cannot see an eyegaze system ever being as portable.'
Of the 5 professionals, 4 thought that the facility to control the environment using eye control is 'very beneficial' and the other 1 said they thought it was 'quite beneficial.' Their potential benefits included: control of the TV, the ability to keeping up with events, allow the user to be 'wireless' and enable them to call their caregiver, if required.
At present, as far as we are aware, powered mobility using eye control is not an option that is commercially available. However, the views of respondents on its potential use were extremely informative. Of the 3 eye control users, one did not comment, one feels it would be 'beneficial', the other 'very beneficial.' Their reasons included the following:
- "Freedom from always asking others for help"
- "I am not moving around a lot in my chair"
Of the 8 users of other forms of access technology, 2 thought it would be 'very beneficial', one 'not very beneficial', 3 'not beneficial', one was 'unsure' and one did not comment. Reasons for considering it to be beneficial included increased independence. Their reservations related to safety issues, for example, problems with looking at the computer and direction at the same time, problems due to a visual impairment, problems with outdoor use.
Of the 5 professionals, 4 thought that the facility to control a wheelchair using eye control would be 'not beneficial' and the other 1 said they did not know. Their reasons for it not being beneficial included safety issues, as 'the eyes need to be used all the time for various purposes', or errors could be made if the user 'slid down the chair' or was 'under stress'
The questionnaire responses, then, remind us that there are a range of issues to be considered in relation to choosing an access device, including reliability, safety, ease of use and comfort. When users could access other, more established devices for mobility and environmental control, there was a preference for these 'non-eye control' methods, such as switch access, because they were regarded as more 'safe' and reliable alternatives. Two case studies follow which provide examples of the way in which users select different access methods depending on which activity they wish to carry out.
Case Study - Julius
Julius is a young man who was seriously injured as a result of an accident in Nï¿½rburg in Germany, in June 2002.
He had a spinal injury at C2/3 and some brain damage to the lower brain/brain stem (full extent unclear). He had a massive 'open book fracture' to the pelvis requiring metal plates and an abdominal cage to stabilise. For the first two months of his recovery, he was either unconscious or semiconscious. After three months, he returned to the UK. He then spent some time in the Spinal Injuries section, Aylesbury before going home in September 2002.
Julius now has a full time job working from home, using the computer and, intellectually, Julius is as able as he always has been. He operates an adapted electric wheelchair, using thumb control. At present, producing speech is not an easy process for him, making voice recognition inappropriate at this stage. He has a limited range of neck movement due to the metal pins used to fuse the break, but he does have very good head control.
Though it is difficult for him to move his arms, he is able to hold a mouse in one hand and operate the mouse buttons. He has visual difficulties in one eye that make it necessary to wear a strong prismatic lens over the right eye, which consequently appears opaque. This made the Tobii system, currently designed for use with both eyes simultaneously, difficult for him to use successfully. For Julius, whilst he has tried eye control and a headmouse as alternatives, his preference is for a 'CameraMouse' system. One important reason is that the system offers him a high level of accuracy in directly accessing all of the Windows applications he needs to use. Combined with a special on-screen keyboard, with built-in prediction, 'Skeleton Keys', Julius is quick and accurate, throughout the day, and able to access all the functions available to every other computer user. He describes himself as able to work at the speed of a "medium paced" user.
Figure 5.11. Even though 'Skeleton Keys' has small targets,
Julius is able to control it quickly and effectively using the CameraMouse.
Key factors that determine his proficiency on the system are the ability to press both buttons on the mouse quickly and selectively and what he describes as "the powerful functionality of Skeleton Keys" with which he can carry out all of the functions he would be able to on an ordinary keyboard. By comparison, at present, an eye control system would have very little to offer him.
- For Julius' computer access needs, an eye control system, no matter how good, would be unlikely to be more quick or accurate. With his CameraMouse, he is able to quickly and accurately select targets as small as the 'Minimise' and 'Close Window' buttons with ease, as well as all of the keys on his small on-screen QWERTY keyboard.
- Even a 'mid-price-range' eye control system would be far more expensive than his CameraMouse system.
- The CameraMouse can be used out of doors as well as indoors more reliably than an eye control system.
Case Study - Paul
Paul is a young man who has cerebral palsy. He uses a powered wheelchair for mobility that he controls, albeit with difficulty, using a centrally mounted joystick. When The ACE Centre first assessed him for access technology, 9 years ago, he was able to use an ordinary keyboard and mouse. However, since then, his physical abilities have changed. The speed at which Paul has been able to move his fingers and arms has slowed down, his range of movement has decreased and the amount of pressure he is able to apply has steadily reduced. For this reason, it has been necessary for The ACE Centre to review Paul's progress and recommend any changes necessary to the access technology Paul uses on an ongoing basis.
In terms of access devices, Paul initially used an ordinary mouse and keyboard, but then, when his range of movement and strength decreased, he needed to use a 'mini-joystick' with an 'on-screen' keyboard and predictor. Now, because he can no longer apply enough pressure to the mini-joystick, he uses a SmartNav headmouse, because he still has a reasonable range of head movement.
He combines the headmouse with a 'dwell-click' utility, 'Dragger 32' that enables him to carry out all of the functions that can be achieved with the buttons of an ordinary mouse.
Initially, he kept the same on-screen keyboard and predictor as before, which was helpful because his predictor's vocabulary had gradually evolved to become increasingly personalised over several years. About a year ago, Paul decided to change to Dasher as his preferred method of text entry. With this, he estimates that he is up to about four times quicker than with his grid-based text entry system.
Apart from having an on-screen method of text entry, however, the headmouse, combined with Dragger, is used to perform exactly the same functions as an ordinary mouse. As a result, Paul has quick and efficient access to the full range of Windows software.
At present, Paul is using his headmouse very effectively. However, from past experience, it is considered possible that head movement might become increasingly difficult and tiring for him. For this reason, it is important for him to consider eye control as his next method of accessing and controlling the computer.
There is a range of access devices available to for people with disabilities to use as alternatives to eye control, including switches, pointer-based interfaces (e.g. trackerballs, joysticks, headmice) and electrical interaction (e.g. brain-computer interface and muscle EMG). The factors involved in choosing an appropriate access device (or devices) are complex and depend on a range of issues, including safety, reliability, independence, ease of use, etc. For certain other people with disabilities, however, eye control is their only method of independent control of technology. It is important that, for these people especially, efforts are made to explore the full range of applications used by people with disabilities that can increase their independence and improve their quality of life, including social communication, powered mobility and environmental control.
6 Choosing between Eye Control and other access methods - The concept of Usability
6.1 What the literature says
There is a wide and expanding range of user groups who may benefit from eye or head based pointing direct interaction with an interface. These user groups range from people with no disabilities who may have hands occupied with other tasks and wish to point with head or eye (Jacob, 1995), or similarly people who may have some reluctance, difficulty or discomfort moving their hands or arms (in Europe alone, 6.6% of the working population (aged 16 to 64) suffer from some form of arm or hand disability or related problem(EUROSTAT, 2002)), to people who have little, if any, bodily movement (Chapman, 1991) (such as Amyotrophic Lateral Sclerosis or Motor Neurone Disease which causes, in later stages, a form of locked-in syndrome nearly 350,000 people suffer from these disabilities worldwide (International Alliance of ALS/MND Associations)). Between these extremes lie diverse ranges of motor disabled user groups who may benefit to a greater or lesser extent from using their eyes or head to interact with an interface. These include any disabilities that cause paralysis or impairment of motor function at a high level on the body. Examples include cerebral palsy, brain injury resulting in locked-in syndrome, multiple sclerosis, musculoskeletal diseases, polio, Parkinsonism and injuries to the cervical spinal column causing tetraplegia (currently there are between 30000 and 40000 people in the UK alone with tetraplegia or paraplegia of varying levels of injury[#_ftn12 ]).
As the level of motor disability increases, so the number of possible usable computer input devices, or input modalities (such as eye, head or hand pointing), decreases dramatically, with the majority of input devices becoming unusable once hand function is lost. As the level of motor disability approaches neck level only a range of single switch devices, some unusual and limited bandwidth[#_ftn13 ] devices such as brain activity and muscle EMG, speech and head and eye movement could be usable with sufficient bandwidth to give interaction for these users (Bates, 2002).
Of these available modalities, it is vitally important to accurately assess which is most usable for the person relying on these limited modalities for their everyday communication needs. It is not sufficient to simply observe that the user is able to generate meaningful input or communication to a system with a given choice of modality it is quite possible that this choice of modality is not the best suited to the user. The user may struggle with the modality, or experience considerable workload and undergo considerable exertion to accomplish tasks. It is also possible that the user can only use this given modality as this is all that is available to them, in this case the user must make do with the modality, but efforts could be made to make the operation of the modality more usable.
To assess what modalities and methods are most usable for a user, a definition of what constitutes usable and usability is required, together with suitable methods of measuring usability in relation to eye based and other control modalities. Knowledge of what available modalities in what configurations would give the most usable eye, or other modality, control for the user is vital for those using these systems and also for those helping others to use these systems.
A widely accepted definition of usability in the context of computer applications is the Degree to which specified users can achieve specified goals in a particular environment with effectiveness, efficiency and satisfaction (ISO DIS 9241-11). From this the notion of effectiveness refers to the completeness and accuracy with which goals can be achieved. The notion of objective efficiency refers to the effort or resources necessary to achieve these goals (Figure 6.1) and subjective satisfaction refers to the subjective assessment by the user of factors such as workload, comfort, and ease of use whilst achieving these goals (Table 6.1).
Figure 6.1. Possible eye control efficiency metrics (Bevan et al., 1995).
Satisfaction assessment areas and factors
Ease of use
Accuracy of pointing
Speed of pointing
Accuracy of selection
Speed of selection
Ease of system control
Table 6.1 Possible eye control satisfaction metrics (Bates, 2005 (Thesis); ISO, 1998; Smith, 1996; Douglas et al., 1999).
This idea of usability is well suited to examining the use of eye controlled pointing devices by motor-impaired users of computer-based systems. Specified users can refer to the level and type of motor impairment of the users. Specified goals can refer to activities associated with the operational aims of the user, for example using common software (such as word processing, web usage, email) as well as operation of the equipment associated with the pointing device itself (such as calibration). The particular environment refers to the physical environment in which the system will be used as well as the support and social context available for use (the former includes use of wheelchairs or beds, while the latter includes the provision of human helpers). These metrics all help to set and define the scenarios in which the user is operating. The definition then goes on to state the success and satisfaction of the user when using a given modality in the defined scenario.
Here it should be noted that measuring and understanding effectiveness and efficiency are more critical issues from the point of view of the motor-impaired user than the able-bodied user. Motor-impaired users may need to trade off apparently efficient ways of working which are physically demanding, against less efficient methods, which are less demanding but enable longer periods of use. Users may often have a finite amount of energy that has to be measured out over the tasks they wish to achieve. For example, using a communication aid should be sustainable throughout a day even if communication is slow, rather than only be usable for a short period of rapid communication. However there are balances here, where modalities may be chosen for their high effectiveness for short periods (such as playing a game or driving a wheelchair), will fallback modalities available for more sustained usage (such as communication). Thus measuring the satisfaction of a modality is a vital component in understanding the cost and sustainability of modality choices in terms of the available energy quota and the nature of tasks undertaken.
6.1.2 Tools currently in use for measuring the usability of eye control
Reviewing previous work on assessing eye based pointing devices finds that tools for measuring the usability of eye based control fall into two areas; abstract target acquisition tests (for example, MacKenzie, 1992; MacKenzie, 1991; MacKenzie and Buxton, 1992; Accot and Zhai, 1997; Sibert and Jacob, 2000; Douglas et al.,1999; Murata, 1991; Istance and Howarth, 1993; Bates, 1999; Radwin et al., 1990), and simulated real world interaction sequences on user interfaces (for example, Istance et al., 1996a; Jacob, 1993; Hansen et al., 2004; Majaranta et al., 2004). To define these two areas, abstract target acquisition test tools are based on presenting the user with a sequence of targets of varying size and spatial separation on an otherwise blank screen, and asking the user to rapidly point to targets in turn (Figure 6.2). Typically, the data collected from these experiments is sparse, with the time taken to select targets and the number of errors being recorded as usability or performance metrics.
Figure 6.2. Example abstract target acquisition test.[#_ftn14 ]
In contrast, simulated real world test tools are typically based on the user performing a small set of tasks or interaction sequences on either a real environment, or a simulated and simplified version of a real environment (Figure 6.3). Often these tests only assess one type of control, such as typing on an on-screen keyboard, that occurs in a real environment. The data from these experiments is usually determined by the nature of the assessment task, for example, words per minute for a typing task, but other metrics such as cursor paths, eye scan paths or user subjective reaction to the test are often recorded, giving a richer data set for usability assessment. The rationale behind these potentially complex real world tests is that, although frequently time consuming and laborious to conduct, the true usability of a device cannot be known unless that device is actually tested on such a real world complex environment. Hence, these tools will be better suited to usability assessment than tools or techniques aimed at simply measuring device performance.
Figure 6.3. Example Real World typing test[#_ftn15 ].
Reviewing literature, there appears to be no standard or commonly accepted test for assessing real world usability on an environment for any pointing device. Typically real world evaluation is designed to test or assess a particular element of interaction with specific interest, rather than the full range of interaction that is possible on an environment. This is acceptable if the user only ever wishes to perform these limited tasks, but what if the user wishes to do new or different tasks will the usability of the modality be the same as before?
In addition, the factors that are assessed and quantified vary due to the task undertaken, rather than using a common methodology. Examining previous work conducted on head and eye based pointing found a range of different test scenarios: A brief, with only a small number of tasks, but wide ranging assessment of eye-based interaction with text entry, text editing, application and menu manipulation and limited internet browsing was found (Istance et al., 1996b). In this work, usability metrics were the text entry rate in number of characters per minute, together with task times and task error rates. Another attempt at a range of assessment scenarios for eye based pointing involved typing on a full-screen keyboard, typing on an environmental control with full screen keys, and playing a simple game; with metrics of simple success or failure of the tasks (Chapman, 1991).
Metrics of task time and task error rate were used for eye-based selection and manipulation of ship representations on a military interface (Jacob, 1993; Sibert and Jakob, 2000), for drop-down menus (Byrne et al., 1999), manipulation of a simplified icon and menu based interface (Salvucci and Anderson, 2000).
Examining text entry, metrics for these studies were typing rate and subjective like or dislike of the overall system (Stampe and Reingold, 1995), typing rate, error count, task time, gaze scan paths of the eye on the interface and subjective like or dislike of the system (Majaranta et al., 2004), and typing rate and user subjective qualification of typing efficiency and satisfaction with the system (Hansen et al., 2004).
Task times and error rates are commonly used for eye based pointing usability studies, together with task success or failure, and task typing rates. Although adequate, with a modality or device that has a shorter task completion time and a lower error rate (higher quality of interaction) during the task almost certainly being more suitable for the task than a device with a longer task time and higher error rate, these metrics are quite crude and do not offer great insight into the detailed usability of a modality or device. In addition, perhaps a device has a shorter task time but higher error rate than another device with a longer task time but lower error rate which device is most suitable for the task?
In addition, from these findings it is evident that there is no commonly used usability assessment method for eye based pointing. Common metrics between differing works were found, with task time, task typing rate and task error rate used, however these depended highly on the nature of the tasks performed and hence could not allow comparison between devices and modalities and differing tasks unless the tasks were identical. However, the typing rate was common to papers assessing keyboards, and this could be used to compare differing eye based pointing devices if the same keyboards and text entry tasks were used, or if the same device was used and differing keyboards assessed for their efficacy. However, this is limited to assessing a single task type, not the diverse range of tasks a user may wish to perform in everyday life.
The objective usability metrics found are often supported by subjective metrics of user reaction to the modality or device. This is essential as, to better gain a full understanding of the usability of a modality, it is regarded as not adequate to simply measure the objective performance of a modality without also assessing the subjective reaction of the user when using the modality (Bevan et al., 1991, 1995). Perhaps a modality performed well objectively, with low task times and error rates, but the user worked hard to control the system, or the modality was uncomfortable to use. Would this modality be more suitable to the task than a modality that objectively performed less well but required less work from the user, or was more comfortable to use?
It is suggested from this study that perhaps what users require is a method of assessing the usability of modalities in a more standardised objective and subjective way, with a common measurement format that can be used to compare results across both modalities and usage scenarios. Looking back to the definitions of usability with respect to eye based pointing (Figures 6.1 and 6.2), such a usability assessment scheme should be based on these definitions of objective efficiency and subjective satisfaction.
6.1.3 Usability measurement and the usability of eye control
An attractive element in the usage of the notion of usability, as defined previously (Figures 6.1 and 6.2) is the derivation of metrics for efficiency, effectiveness and satisfaction to enable these quantities to be measured for a given pointing device (used by specified users for specified goals in particular environments). This can provide a basis for comparing pointing device usability to be able to recommend which device, or combination of device options, to use for a given situation. It also enables modifications or improvements proposed for devices to be assessed and quantified in terms of their impact on usability.
An example of the effectiveness of this approach can be shown for the evaluation of the usability of eye-based control. Typically eye control is regarded as inaccurate and difficult. However, by performing a detailed evaluation of eye control with both objective and subjective metrics of usability it can be shown that eye control can be effective for some users.
One study has been found that used this approach to show that eye based pointing can be an effective method of control for some motor-impaired users (Bates, 2005 thesis). Here objective efficiency was measured as described (Figure 6.1) with measured quantities of quality and quantity of goal achievement. Quality of goal achievement per task is measured as a deviation from a perfect score of 5. A number of pointing and selection actions between the start and finish of a task are classed as errors. As the quality of interaction during the task is degraded by the weighted counts of the error types, so the rating is reduced until either the task is completed or the quality rating is reduced to 1, at which point the current task is regarded as failed. In this measuring scheme, users are required to complete all of the tasks set, which effectively removes variability in the quantity of goal achievement. The task efficiency metric shown on the right hand of the diagram (Figure 6.1) can be used to compare device performance and to enable the separate components of efficiency to be examined in detail. These comparisons are complimented by measures of satisfaction (Table 6.1).
This approach has been used to quantify differences between head-based and eye-based pointing using a standard set of real-world tasks. The task sets are designed to include proportions of interaction objects and interaction techniques that are representative of typical word processing and web browsing tasks. Selection of targets on screen was made by two common methods used for eye control: by continued pointing at the same object (dwell) or by using a command (click), from separate device such as a switch, controlled by another modality (e.g. sip-puff).
The overall comparisons of device efficiency and satisfaction are shown in the figures below (Figures 6.4 and 6.4). The results for both of the eye-based modalities (selection by dwell and selection by click) were significantly worse in terms of efficiency and satisfaction than for the head-based modalities (Figures 6.4 and 6.4). Both the head and the eye pointing modalities were worse than the hand held mouse modality.
Figure 6.4. Device efficiency.
Figure 6.5. Device satisfaction (Low Workload and High Comfort and Ease of Use for highest Satisfaction).
The usefulness of this usability approach was that it now allowed close examination of the individual elements that constitute the objective and subjective metrics. Examining the quality metric in particular (Figure 6.6) showed that making control corrections to the pointer position (Factor 3, Figure 6.6), particularly when under eye control, accounted for a large proportion of the non-productive time in the trials and was the most influential factor on eye modality usability. Not surprisingly, this proportion increased for the smaller interface targets. If this could be reduced then the eye modality usability might be improve significantly. Examining the individual satisfaction ratings (Figure 6.5) showed that the eye modality required more workload than the head and hand modalities, and was less comfortable to use. The eye modality in dwell selection mode had low ease of use, but the eye modality in click selection mode was as easy to use as the head modality.
Figure 6.6. Composition of device task quality.
6.1.4 Usability variation in particular environments
A method of improving the usability of the eye control modality was found by effectively removing the control corrections to the pointer position for eye control. This was implemented by using a screen magnification system that temporarily increased target objects under the eye control pointer. This is in effect slightly modifying the particular environment that eye control is using. This zoom improvement thus greatly increased the efficiency of eye control to parity with head based control for both the dwell and click modalities, even when the same facility was available with head based control (Figure 6.7).
Figure 6.7. Modified device efficiency.
Hence (Figure 6.7), the objective usability of eye control was improved to equal head based objective usability. This benefit was also shown for subjective usability (Table 6.2) with the satisfaction rating for both head and eye modalities increased due to the improvement, although the eye modality was still less satisfying than the head and hand modalities.
Cost / benefit of enhancement and subjective device satisfaction
Satisfaction per device(1-7 rating)
Change due to zoom enhancement
Table 6.2. Modified device satisfaction.
6.1.5 Usability variation in specified users
A second change to eye control was now attempted by changing the specified users of eye based control. Highly experienced users (15-30 hours of use), medium experience (6-8 hours of use), and low experience users (1-2 hours of use) of the eye modality were chosen and their results examined (Table 6.3).
User Experience and device efficiency
Efficiency (%) by subject experience'(Low, Medium, High)
Head Dwell Zoom
Eye Dwell Zoom
Head Click Zoom
Eye Click Zoom
Table 6.3. User experience and efficiency.
This variation in specified users for both the original eye control modality and the eye modality with the modified particular environment showed that by changing the users specified (by experience or training) now made the eye control modality more efficient than the head modality (Table 6.2, right hand column in bold), with the eye control modality approaching the hand modality efficiency.
In summary, usability analysis based on objective efficiency and subjective satisfaction showed that eye control can be as effective as head based control provided the specified users and particular environments of eye control are well defined and optimised. It can show the individual elements and characteristics of eye control that limit its performance, and show how usability criteria can be used to define specific alterations in operational characteristics, environments and arrangements for users that optimise the usability of eye control.
6.2 Information from stakeholders
As described above, 'Usability' is generally defined as the "effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments" (ISO DIS 9241-11). Two case studies follow where users are making clear choices and decisions in relation to usability issues:
Case study - Sarah
Sarah has a high spinal lesion that, for her, means that she cannot move her body below the neck and has to rely on a ventilator to breathe. She has very good head control, however, and is able to use a mouthstick very effectively for a range of activities that she enjoys.
For example, using her mouthstick, Sarah is able to make friendship bracelets by weaving them and she can also use a mobile phone to send text messages.
She can also access the computer using the mouth-stick. She uses the mouth-stick to 'push' the mouse around the mat with great accuracy, accurately enough, indeed, to be able to design websites, a skill she recently acquired during a specialised two-year training course. For Sarah, this method of using the mouth-stick to control the mouse keyboard gives her the high level of accuracy she requires for web design. However, this method requires a great deal of effort and difficulty, due to the need to transfer her gaze from mouse to keyboard to screen and back again. For this reason, she is extremely interested in using eye control for whichever computer applications she can. Having trialled it over several days, eye control, Sarah feels, potentially offers her a more direct, comfortable and quick method of carrying out certain tasks on the computer - even more so than the SmartNav headmouse, which she sometimes uses. Eye control, in her opinion, is more 'natural' and requires less physical effort than either the mouthstick or headmouse.
With the Quick Glance II SH, in fact, Sarah is able to achieve a high enough level of accuracy to carry out a range of computer-related activities that she enjoys, such as computer games, web browsing, etc. However, at least for the present, she prefers to continue to use the mouthstick for web design, due to her requirement for a higher level of accuracy with her web design application.
For Sarah, in relation to usability issues, she finds eye control more 'effective, efficient and satisfying' for certain 'low accuracy level' applications whereas she prefers to use a mouthstick when pinpoint accuracy is required.
Case Study - Ahmar
Ahmar is a young man with cerebral palsy who uses an Origin Instruments 'Headmouse Extreme' to access the computer.
Figure 6.10. Ahmar wearing a reflective 'dot' on his forehead with which he controls
the 'Headmouse Extreme' on the top of the computer.
He uses two foot-switches to carry out the same tasks as the left and right mouse button. With this combination of headmouse and switches, therefore, he is able to accurately achieve whatever he wishes to, in terms of mouse control. However, he cannot access a real computer keyboard so he uses an on-screen keyboard, SofType, for this purpose, with prediction.
Figure 6.11. The SofType on-screen keyboard (with prediction) that Ahmar uses to write using a headmouse.
With SofType, he is able to directly access all of the letters of the on-screen keyboard directly and accurately, with 'one hit'. For this reason, it was not anticipated that eye control would have anything to offer Ahmar. However, even though he had tried eye control as an access method for a relatively brief period of time, Ahmar was extremely clear and decisive about his views on this access method compared to the headmouse. Ahmar felt that when accuracy was required, e.g. for computer games, he would still prefer to use the headmouse.
However, for word-processing, he would have no hesitation in switching to eye control, if available. This view was surprising, as Ahmar has been a successful user of his SofType/Headmouse combination for many years and, because it was only a trial layout, the on-screen keyboard he tested out for eye control was nowhere near as 'efficient' a keyboard layout as SofType.
His reason for choosing eye control technology for 'eye-writing', though, was simple. When using the headmouse for computer access, because of his cerebral palsy, Ahmar had to work at 'stiffening' his whole body in order to keep his head still enough to achieve the level of accuracy he required in order to access the onscreen keyboard. With eye control, however, using the Tobii system, it was different. This particular eye control system was able to 'ignore' Ahmar's involuntary head movement and track the movement of his eyes alone. As a result, Ahmar was able to relax his whole body. With this access method, he felt, regardless of whether it was quicker or not, he would be able to 'eye-write' for far longer, far more comfortably. Therefore, he felt he would gain far more 'user-satisfaction', overall, when using this method for writing.
- Given a free choice, Ahmar feels that he would find eye control more 'efficient, effective and satisfying' for writing on the computer. On the other hand, he prefers to use the headmouse for those applications where pinpoint accuracy is required. On these occasions he is prepared to 'trade off' greater accuracy for less comfort.
The extent to which an access method and/or application is 'efficient, effective and satisfying' to use is a complex and dynamic issue. Users' preferences for which access method they prefer to use is dependent on a range of factors, including the level of accuracy required for a specific application and their physical comfort when using it. For this reason, it must be remembered that users also need to be able to make a choice, depending on the application they wish to use. For example:
- Sarah prefers to use the mouthstick for web design because, whist it's not her most as efficient as eye control for this particular application, her most important requirement is the effectiveness of her eye control, in terms of accuracy.
On the other hand, for activities when pinpoint accuracy is not required and effectiveness and satisfaction (ie. comfort) are more important, use eye control might be preferred. For example:
- For Internet access and Games such as Solitaire, Sarah prefers to use eye control.
- Ahmar prefers to use 'eye-writing', rather than the headmouse to produce text, when the targets are large enough for him to access easily. He can relax his whole body and does not need to 'stiffen' in order to control his head movement, something he has to do in order to accurately control a headmouse to produce text.
Whilst the concept of choice is important, at the same time it must be noted that some users with disabilities do not have a choice. Clearly, from some of the questionnaire returns, for some people with complex accessing difficulties, the choice has been made for them. Some users with ALS and MND have been provided with 'eye-blink' systems without ever having had the chance to try an eye control system. For these users, then, it is eye-blink or nothing. In COGAIN Annex 1 (2004) - "Description of Work", it has been described how partners 'have already shown that current eye-writing rates can be increased from 1-5 words per minute (which is common in many present day AAC systems) to 10-25 words per minute'. Though it is encouraging that, at least, that the users of eye blink systems have some method of access to the computer, it is a responsibility of COGAIN partners to make both users and those supporting them aware of the significant potential advantages of eye control over many other currently used access methods.
A clear distinction needs to be made between the concepts of 'usable' and 'usability'. Many of those assessing and supporting those with complex disabilities are satisfied when they reach a point when they find an access method that enables their client to 'use' their technology. However, it is clear from the views of users like Ahmar and Sarah that it is necessary to take into account a range of factors relating to a wide range of usability issues.
7 What can people with disabilities currently achieve with access technology overall?
For many years now, research and development relating to Eye Control Technology has placed great emphasis on 'eye-writing' (or 'eye-typing'). Whilst very important for many people with disabilities, it is only one of a wide range of applications that are important for them to use. For this reason, this chapter provides examples of user requirements that are already being met by 'non-eye control' applications but not commonly found, as yet, in Eye Control applications. For example, as well as 'eye-typing', many of these are also designed to meet users' needs for social communication, environmental control and powered mobility.
It is intended that this information will help to (a) inform COGAIN partners' decisions about the features and facilities to include in their own Research and Development and (b) provide an opportunity to consider existing software, which might potentially be adapted for use with eye control. It is hoped that this will help to ensure that partners do not waste time 're-inventing the wheel' and give them the opportunity to apply their energies to capitalising on the hardware and software tools that are already available.
7.1 What the literature says
Access technology can enable people with disabilities to accomplish daily living tasks, assist them in communication, education, work or recreational activities. In essence, it can help them to achieve greater independence and enhance their quality of life.
Individuals with disabilities are often unable to achieve certain daily living tasks independently, so they must rely on others for assistance with eating, mobility, reading, and writing. This situation can lead to feelings of frustration and unhappiness for the individual.
Fortunately, technology offers a range of opportunities to support disabled people in achieving maintain independence in certain key areas, using assistive devices.
7.1.1 Social Communication and Writing using Assistive Technology
Often the biggest problem for a disabled is the communication to express himself, to have relationship with relatives, friends, colleagues, school friends with the world around. Communication is crucial to establishing an effective relationship, but without self-respect it's difficult to express concern for the other person.
With some good assistive devices, the user can establish relationships with the other persons: he can write letters, he can ask questions, he can report a need, he can meet friends, etc., so the person can play an active part in these situations.
There is a wide range of people with communication difficulties, for example those with cerebral palsy, MND (Motor Neurone Disease), Head Injury and ALS (Amyotrophic Lateral Sclerosis). The way in which a particular form of disability can affect their ability to communicate varies considerably. People with ALS, for example, can become unable to control muscles voluntary. This can impact on their writing skills and oral communication. The result is that they might find it impossible to communicate either through the written or spoken word. When communication difficulties are this severe, a communication device - whether 'high-tech' or 'low-tech' - can be an essential tool.
The E-tran frame can be used with both symbols and text. However, to use an E-tran frame with letters obviously requires the user to be literate. A wide range of 'high-tech' devices and software exist to enable the literate user with a communication difficulty to express themselves. These can be accessed by a range of methods (see Chapter 5) such as switch(es) and pointing device (e.g. joystick, Headmouse, CameraMouse and eye control). The Lightwriter is a text-based device that is light enough to be carried around by ambulant users.
Devices that enable literate users to communicate socially frequently involve a PC-based system, with speech output. For example, the Cameleon is a device that is PC-based and, with a program such as EZ keys it can be used for a range of communication and daily living activities.
For people with communication difficulties who move around in a wheelchair, communication aids need to be positioned on a tray or attached to a special mounting arm, so that the user can communicate wherever they happen to be. The process of communicating using techniques other than speech is called Alternative and Augmentative Communication (AAC). AAC involves a strategies set to enhance a person's ability to communicate. For users who cannot communicate using written language, or prefer to use an alternative, a symbol-based system might be more appropriate. Many symbol users might use a grid-based communication system, both in 'low-tech' form, e.g. a communication book with symbols, or high-tech, on a speech-output communication aid. The screenshot below shows the word 'tomato' being selected and spoken out using an alternative access device, e.g. switch or pointing device.
COMUNICA is communication software with voice output that uses PCS symbols. COMUNICA has 4.800 different symbols to create personal dynamic communication tables. It is a very powerful communication device for disabled people who either cannot or prefer not to use speech or written language. Actually, its not possible use COMUNICA with an eye control system. This is because it is a 'dedicated device that only runs its own software and not PC software. This is an important issue as, for people to need to use eye-control as an access method, the software they use would obviously need to be PC-based to be able to be run on a PC-based eye control. Nonetheless, there is a range of Symbol-based software that could be used with eye-control, such as the Grid, SAW, Speaking Dynamically Pro, etc.
7.1.2 Software for written communication
For written communication on a PC, a grid-based system is often used. This might involve a virtual keyboard that can be easily modified for the user. For example, the key size, the keyboard layout, the gap between the keys, etc. can be changed to suit the user. A range of devices can be used to access on-screen keyboards, including switch access and pointing devices, including joysticks, headmice, etc. Importantly, these keyboards can also be accessed using eye-control.
SofType is an example of a PC-based on-screen keyboard that can be accessed by any pointing device, including eye control, if the user and eye control system are sufficiently accurate.
Figure 7.5. SofType onscreen keyboard.
SofType has a word prediction system, to allow a reduction in the number of keys that the user must press, and thus increase the writing speed. The most important features of a good assistive system, of course, is that it allows the user to do things more easily, faster and without effort.
Today young people are accustomed to communicate by SMS, it's a quick way to send and receive greetings, invitations, news, etc. If a young person with severe physical difficulties wants to send SMS messages, but is unable to use a mobile telephone, QualiSMS can solve this problem. This is a software application that allows the user to send SMS using the PC, with their own specialised access device. With QualiSMS it is possible to send and manage SMS, personalise the folders and create a personal address book. However, the difference is that the user does not need to have a mobile phone.
Figure 7.6. QualiSMS application for sending text messages via a PC.
7.1.3 Controlling the Environment using Assistive Technology
Technology is available to enable people with disabilities to control their environment, whether it is a single room, the whole house or any other environment. For example, if the users want to control the TV, activate an alarm, turn the light on and off, open and close the curtains, answer or make phone calls, unlock or lock a door or set the temperature it is important to be able to do this independently, if possible.
Some of these environmental control systems are 'stand alone' devices that are not PC-based. These include systems like 'Steeper' or 'Possum'. With these, users with disabilities can control their environment wherever they are in the house, using their preferred access method, e.g. switch access.
As well as 'stand alone' devices, there are many PC-based systems that enable users to control their environment. Several of these, in fact, are an integral part of the users' communication software package, such as EZ Keys, QualiLife or The Grid. It is important to note that, in principle, such environmental software could be used effectively using an eye control system.
Figure 7.8. Examples from The Grids environmental control grids, which could potentially be adapted for eye control.
7.1.4 Powered Mobility using Assistive Technology
A range of assistive technology can be used to enable people with mobility impairments to gain opportunities for independent mobility. Many people in wheelchairs are able to control the wheelchair using a joystick. However, some people are unable to control the wheelchair with a joystick. For these, a number of alternative devices can be used. For a switch user, for example, switches can be positioned within the headrest to enable the switch user to have full, independent controlled mobility. Using a special interface, the switch user can independently change from controlling a wheelchair mounted PC to controlling their wheelchair and back again, using the same switch/switches (See 'Case study - Stewart' in 7.2 below)
7.1.5 Leisure activities using Assistive Technology
For a person with a physical disability, that might spend their time in a wheelchair or confined to a bed, it is important to have the opportunity use technology for leisure activities. Technology can enable the user to play games, whether by themselves or with others, and also to express themselves creatively, e.g. with Design or Music programmes. A range of PC games are available commercially that can be used by people with disabilities, without being modified. For example, if a person has good pointer control, whether with a headmouse, cameramouse, etc. Many 'pointer controlled board games, for example, can be played without modification in this way. In addition, specialised versions of games are also available for a wide range of access methods, including switches, such as QualiCHESS.
Figure 7.9. QualiCHESS application.
Importantly, many PC computer games can be played using eye control, without modification, if the user and their system are sufficiently accurate.
In summary, people with disabilities want to be able to communicate, play games, be independent, have mobility, communicate socially (whether in the home, the office, on the move, etc.) like everyone else. The range of access technology currently available does not always enable all users with disabilities to realise these objectives. COGAIN provides an excellent opportunity to discover the extent to which eye control technology can reduce the shortfall.
7.2 Information from stakeholders
The questionnaire returns provided an indication of the range of software and applications that can be used both with eye control and with other access technology. The eye control users who responded use their systems for Word-processing, social communication (e.g. Speaking Dynamically Pro), emailing, Internet, Instant messaging, games. The users of other access methods who responded use their systems for Word-processing (1 Dasher user), social communication (e.g. EZ Keys) emailing, Internet, Instant messaging, games, activating toys, powered mobility, environmental control. The non-eye control users who found their current access methods very difficult felt that use eye control would enable them to:
- Access 'Emails, Internet, communication - present methods are incredibly slow.'
- 'Indicate choice', have 'some degree of independence.'
- 'Would like to make it easier for him to access all the available technology - access is exhausting for him to do very much.'
For both eye control and non-eye control users, their reasons for wanting to use additional applications related primarily to the need for independence, but also included privacy and to reduce the shortfall between potential and performance. The existing users of eye control systems are clearly very positive and enthusiastic about what they are able to achieve with their eye control systems, overall. The following comments were made in response to a question relating to any additional requirements they might have of eye control technology, over and above they are achieving already.
- 'Environmental control' (This eye control user acknowledges that this could be done via eye control but it hasn't been purchased yet).
- 'Move from <social communication application> into <eye control application> myself.'
- 'Play more video games', 'turn the volume up and down.'
- 'If anything, I would like it to be more portable.'
From all of the respondents with disabilities, additional comments relating to their requirements of eye control technology, other than those covered in the questionnaire, included:
- Portability - 'Portable and connected to a laptop, so that the user was not tied to a PC indoors.'
- Control of positioning - 'Operation of an indoor static chair for changes in position. Changing the position of a profiling bed.'
- 'All aspects of daily living.'
From the professionals, suggestions relating to their additional requirements of eye control technology that they felt were not being provided at the moment, included
- Leisure - 'Playing PC games.'
- 'Arithmetic for children.'
- 'Alarms and assistance devices.'
Respondents were invited to make any further comments they wished relating to issues that had not been covered in the questionnaire. One respondent emphasised the need for information, which is one of the areas that are, indeed, already being addressed under COGAIN.
- 'How do we find out about the suitability of different products?
From some of the professionals, there were several comments relating to the practical issues involved in using eye control technology with confidence with all users for all of the applications they wish to use and wherever they wished to use them. These included:
- The need to overcome potential technical problems, including: conflicts between Windows and the eye control system software; involuntary movement; problems caused by natural and certain artificial lighting; robustness; snagging cables; difficulties in the setting up and calibration process; the need for gradual implementation to avoid eye strain; training and practice for the user - 'Its not a natural way of looking.' Finally, they felt there was a need for the user to be able to calibrate and re-calibrate independently - 'as and when the user wishes.' (It is acknowledged, of course, that certain systems already meet at least some of the above requirements.)
Case Study - Stewart
Stewart's case study provides an indication of the range of user needs that can be met through access technology overall. Stewart has athetoid cerebral palsy.
This means that it is difficult for him to make any controlled movement. Attempts at a deliberate movement can result in a certain amount of involuntary movement. For this reason, if Stewart wishes to keep still enough to control the head switch he uses, he 'fixes' himself by pressing his arms against the side of his wheelchair and pressing his head firmly against his headrest. Stewart uses head-switches to control a wide range of the technology he needs for leisure and daily living, using an integrated system.
For example, Stewart is able to control the movement of his electric wheelchair using head switches. He uses his switches to move independently from communication and environmental control to wheelchair control - and back again - using a DX device positioned at the side of his Cameleon (below):
Stewart's speech can only be understood by familiar listeners. For this reason, Stewart sometimes chooses to communicate using a speech output device, a 'Cameleon'. This is, in effect, a tablet PC with speech output, which is positioned on his wheelchair. With a communication program called 'EZ Keys', Stewart is able to express himself clearly, even though, because he uses a switch, communication in this way does take time and effort for him. Stewart also uses the Cameleon for writing, using the EZ keys scanning keyboard with a predictor. With this, he can write directly into a word processor, email application, etc.
For environmental control, Stewart also uses switches positioned in his headrest. He is able to independently switch from communication/writing on the Cameleon, using the EZ Keys software.
Figure 7.12. Stewart using EZ Keys on his 'Cameleon' PC-based speech output device for communication.
Stuart was one of the users we involved in the brief user trials, testing out an eye control application during a single session. When Stuart used eye-control for writing with the Tobii system, he used it very quickly and efficiently and was soon able to use it far more quickly and comfortably than his current switch access method. He was extremely pleased and impressed by this new, quick and efficient method of computer access. However, despite this, there is no way in which eye control can replace switch access in relation to his overall assistive technology requirements at present. For example:
- Stewart can use his switch to control his wheelchair. Eye control systems cannot be used for wheelchair control, at present.
- Whilst the Tobii system can be set up for environmental control, Stewart would have to go to whichever room the Tobii was in, in order to control his environment. His Cameleon, on the other hand, is attached to his wheelchair and, with this, his environmental control software can be used anywhere in the house. It is acknowledged that some eye control systems can be attached to a wheelchair and used for environmental control. However, at present, there are no commercially available eye control systems that enable a user to move around the house independently in order to control the whole of their environment.
- Using his switch and the Cameleon attached to his wheelchair, Stewart can communicate wherever he happens to be, whether in the house, at the shops, at the pub, etc. At the moment, whilst the Tobii can be set up for use with social communication software, it cannot be attached to the wheelchair and cannot be used outside, in daylight.
It is clear that, at this moment in time, eye control technology can be used to access and control a range of applications that provide enormous enjoyment and independence to many users of this technology. One application that is not commercially available yet, however, is eye control for powered mobility. In addition, whilst certain developers, such as LC Technologies, have included environmental control as a focus for their eye control applications, this is an area that has not been widely addressed in research and development work. In general, the primary focus for much of the research and development that has taken place over the last two decades at least has been 'eye-writing or 'eye-typing'. However, as this chapter has described, there is a wide range of applications that users with disabilities need to use and are already able to use successfully with other kinds of access technology. These include not only environmental control and powered mobility but also social communication and a wide range of games/leisure activities.
Furthermore, it is not only is it important for COGAIN to focus on increasing the range of applications accessible through eye control but also to focus on the quality of access to them, too, in terms us usability issues such as comfort, safety, efficacy, etc. Questionnaire responses from existing users of eye control systems provided brief but telling examples indicating the need to pay attention to such details. In response to asking whether the eye control users had any additional requirements they might have of eye control technology, over and above they are achieving already, one said that they would like to move from their social communication application into their eye control application (for writing, etc.) themselves. Clearly, the facility to move from application to another, so easily achieved by Stewart with his switch access (as illustrated in the case study), independently, was sadly lacking for the eye control user.
Another said they would like to 'Play more video games'. At present, while existing systems will enable users to access a certain number of leisure applications, this is another area that is seriously under-exploited and one that is ripe for COGAIN to focus upon.
Finally, the comment from the eye control user who said 'If anything I would like to make it more portable' underpinned the importance of trying to make this technology as portable and effective in as many situations and settings, as possible. Along with plans to increase in the number of eye control applications that COGAIN plans to focus on, the ability to control these applications in the widest possible range of settings goes hand in hand.
However good an environmental control system a user has, there are only limited benefits if they cannot independently move about within their whole environment. However good a social communication program they have, there are limited benefits if they can only communicate in a limited number of locations.
8 What can people with disabilities currently achieve using eye control technology specifically - and how well?
8.1 What the literature says
What, according to the literature, are users currently able to achieve successfully with eye control?
The literature on eye gaze control has reported on several areas in which this control mode has been used: type-to-talk, environmental control and leisure activities.
8.1.1 Non-electronic gaze communication
The most common use of eye control is for type-to-talk systems. Actually, gaze typing has been possible for more than 20 years (see Majaranta and Rï¿½ihï¿½, 2002 for an overview). AAC Gaze communication without computers has an even longer history. Users would gaze at an alphabet printed on a board (see Grossens and Crain, 1987, for an overview). Using an e-tran frame, characters or symbols can be displayed on a transparent sheet of plastic in a matrix, and the communication partner holds the sheet in front of the user. By maintaining eye contact through the sheet, it is easier for the communication partner to estimate at which character the user is looking (Drinker and Kropoff, 1981).
In the 1980s a non-electronic eye type was developed at Delft University of Technology (ten Kate et al., 1980, ten Kate et al., 1983-84). A mirror-prism communicator consisted of a letter board containing prisms and angled mirrors that reflected the users eye positions in a straight line from the area (s)he was looking at. Two versions were developed: one with a circular arrangement of letters and one with a column arrangement.
A freely available version of a gaze typing board is in regular use among ALS patients. It consists of a matrix arrangement of letters written on a piece of paper (Figure 8.1.). First, the communication partner asks the user which row the letter is located in: First?second?.....third?... and when the target row is mentioned, the user looks upwards. Then the communication partner asks if it is, for example, m...n...o...p...q...r? and again the user looks upward when the right letter gets mentioned. The communication partner can make a guess in order to complete a word or sentence, once the user has begun spelling. If this guess is correct, the user can look upward to confirm. If it is wrong, a downward look tells the communication partner to go on. Some prefer frowning instead of looking up and down (Jeppesen et al., 2004).
The advantage of this system is that it may be used everywhere and it costs nothing. The disadvantage is that it takes time for the user and communication partner to master it (especially without the paper in front of them). It does require mental concentration from either partner or eye contact between them. From our observations, typing speed varies. In some cases, when the user expresses daily needs in a well-known context it may be several words per minute. If irregular words and/or original thoughts are to be expressed, it may be just a few words per minute.
8.1.2 Computer based gaze communication
High-end eye trackers are available on the market at high prices. They have some obvious advantages in that they may provide the user with pixel-precision in pointing by gaze under optimal conditions. For instance, Tobii Technologies offers a Plug-and-Play eye tracking system. It can be installed on a computer within a few minutes without any requirements or complex hardware besides the tracker itself, which is integrated into a 17 TFT display. Tracking is performed automatically so there is no need for a dedicated operator. The tolerance to head movements is high (20x15x15 cm horizontal/vertical/depth) at 50 cm distance from the screen. The user can leave the system, and when (s)he comes back, tracking is resumed instantly. The cost of the system is approximately 20,000 (as of February 2005).
On-screen (soft) keyboards (e.g., Point for Windows, Wivik and others) are well suited in combination with these high precision trackers, and they have been used for decades within the Augmentative and Alternative Communication (AAC) community. Character sets can be arranged in a traditional QWERTY order, in alphabetic order or according to frequency-of-use. Acceleration features include predictions of the most likely next characters and words or access to a dictionary of words related to a specific context (such as dinnertime or shopping).
8.1.3 Environmental control
Besides type-to-talk, gaze interaction has been used for environmental control of lighting, appliances, television etc. Some of the more expensive systems have sufficient precision to allow control of a mouse-pointer in a Windows-environment. The screen resolution may have to be at a rather low level (e.g. 600 x 400 pixels), and larger-than-normal icon size may have to be applied, but in general it works. A zooming principle that works with normal size icons and at a higher resolution (e.g. 1024 x 768) has also been invented by one of the manufacturers (Lankford, 2000).
There are only a few reports on gaze control of wheelchairs (e.g. Roberts et al., 1999). Matsumo et al. (2001) achieved a precision of 3 degrees with their wheelchair mounted tracking system. They were inspired by the fact that a person often looks in the direction of a next move when walking or driving, so this may be a natural and intuitive way of controlling a wheelchair. However, they decided not to utilize gaze direction but to use tracking of face direction instead, because gaze cannot be reliably controlled by intention in a dynamic environment where, for instance, other people walk around. Canzler and Kraiss (2004) report on problems with highly variable lighting conditions (e.g. sunlight and neon lights shining directly into the camera pointing upwards at the users face). Vibrations from the rolling wheelchair also complicated tracking by their experience.
It seems like there may be a couple of reasons why there are no reports on successful use of what might otherwise seem like an obvious control for a wheelchair: First, the weight of gaze trackers and their sensitivity to sunlight may have prevented mobile and outdoor use. Secondly, while it can be annoying to do an unintended selection on a PC, it may cause a serious accident if it happens while driving. Finally, a lack of standard input protocols for wheelchairs may also have prevented gaze-tracking suppliers from supporting this control mode.
As an alternative to video-based tracking of gaze, EOG tracking may overcome some of the problems associated with analysis of poor images (see e.g. Barea et al, 2002). However, the danger of unintended eye movements causing dangerous driving still persists, unless some kind of activation/ deactivation clutch is included in the control. Fitting the electrodes for EOG recording on the users head is not a trivial exercise. It may not be desirable from an aesthetic point of view to have electrodes placed on the face while driving around (see Figure 8.2). Jordan (2000) warns strongly against ignoring the socio-pleasure when doing inclusive design: The results of such attitudes can be seen in some of the hideous looking products that are designed to meet disabled peoples physiological needs (p. 918). This will . draw attention to the users disability and, because they embody aesthetics that few would accept given a choice, immediately label the user as disabled and unempowered, Jordan says.
Figure 8.2. Electrodes placed on a user's face for EOG gaze tracking. The electrodes may draw more attention to the user than (s)he may like.
8.1.4 Entertainment and work
Advanced gaze tracking systems provide dwell-time selections of icons, menus and links. Users can operate standard browsers, some popular games (e.g. Solitaire) and a range of normal Windows software. In combination with virtual on-screen keyboards, text and numbers may be entered in most standard software. While it is obviously a great advantage to have control of a normal PC, the cost of the high precision systems and the restrictions they put on the mobility of the user are still an issue of some concern.
Web browsing is a very rich source of information and a leisure activity. Unfortunately, web browsing can be quite a challenge if people do not have good fine motor control. Most links and buttons on web pages are small in size, and if they are to be accessed through step-scanning it may take a long time to hit the target. Hitting the tiny scrolling buttons can be particularly annoying without some kind of zooming tool and entering data in text fields is not a trivial task either. Cascades of pop-up windows may invade the browser if they are not properly blocked. We have not been able to find any descriptions of a web browser designed for gaze control in the literature. However, using a two-monitor set-up with a normal browser window in one of the monitors, and the control functions (back, forward, etc.) in the other monitor, seems like a durable design task.
Hornof and Sato (2004) have developed a system, EyeMusic for direct control of electronic music compositions using a gaze tracker and established composition software. One of their main motivations was to make it possible for people with physical disabilities who interact by gaze to enjoy new opportunities for musical expression. During composition, the primary melody is derived from the horizontal dimension and counterpoint is derived from the vertical dimension. The resulting music produces a mysterious and lyrical ambiance (p. 188), they believe. Music production is often used in therapy to communicate emotional, non-verbalized feelings. We imagine that some people with ALS, for example, would appreciate having this means expression available to cope with the strong emotional changes the progress of their decease may cause.
Hornof (Hornof et al, 2004; Hornof and Cavender, 2005) has also been involved in development of EyeDraw that enables children with severe motor impairments to draw pictures with their eyes. Several able bodied children have used the system and the very first time they tried the system some of them were able to produce simple shapes and configurations that could be associated to the real object. Hornof and Cavender (2005) report on an evaluation of EyeDraw with disabled users. They identified two major problems: the lack of an eye image on the screen, and EyeDraw not being in the gaze tracking systems main eye-controlled menu. The last problem would require Hornof and Cavender to modify the gaze communication software but they are reluctant to risk introducing bugs. Nonetheless, it is clearly important to make EyeDraw and other software for this population accessible within their current eye-controlled environment, they state. Besides providing the personal enjoyment of drawing, systems like this may also be useful for sketching during communication.
Some computer games may work well with gaze control. LC Technology, producer of a gaze tracking system, state on their homepage: Identify the threat, acquire the target, move the scene right or left all with your eyes! Or place the bet, pull the handle, or deal yourself three cards. The door is just opening on the potential for using eye tracking technology in a broad range of gaming venues. (http://www.eyegaze.com/SOLUTIONS.htm#video - confirmed 2005-02-22). The two computer games that they include in their system are Paddle games and Score four.
Other games that work well with head tracking systems are also likely to work with gaze control systems.
A Game Accessibility Special Interest Group works under The International Game Developers Association (IGDA) (http://www.igda.org/accessibility/ - confirmed 2005-02-22). One of their aims is to improve hardware support for miscellaneous special devices that may be used as an alternative to joystick control. A gaze controller would be an obvious candidate to work within this group. One representative from a US company (Brain Actuated Technologies) that manufactures an EEG/EMG/EOG control device is included in the special interest group, and COGAIN might consider joining it as well. The head mouse producer Natural Point runs a forum for discussions of headmouse gaming: http://forums.naturalpoint.com/cgi-bin/ultimatebb.cgi. They also have a rather long list of games that they recommend for headmouse interaction, including flight simulators and racing games. When investigating games that would work well with gaze control, it may be a good starting point to look into the experiences gained from headmouse gaming.
How effective is gaze technology for users compared with other access methods e.g. in terms of speed, comfort, fatigue, etc.
While it is unlikely that a gaze controlled AAC system will ever achieve communication rates comparable to unimpeded speech (+ 150 wpm, cf. Table 8.1), as even a highly trained touch-typist using QWERTY is unable to keep up with ordinary conversation, the long term goal is certainly to achieve an input rate that is comparable to the QWERTY keyboard for a text composition task, as this is usually sufficient to partake in on-line conversations (chatting).
Speed of communication
150-250 wpm (Fould, 1980)
65 wpm (Matias et al., 1993)
Stylus tapping on screen keyboard
9-30 wpm (MacKenzie and Soukoreff 2002)
30 wpm (Wiklund et al., 1987)
25-30 wpm (Pierpont, 1997)
Palm top with Graffiti
20 wpm (Zhai et al., 2000)
Multi Tap on 12-key phone pads
8 wpm (James and Reischel, 2001)
12-key phone pads with T9 disambiguation
20 wpm (James and Reischel, 2001)
Various combinations of AAC systems
2-8 wpm (Beukelman and Mirenda, 1992)
Gaze typing with full on-screen keyboard
7 wpm (Spaepen et al., 1996; Majaranta and Rï¿½ihï¿½, 2002)
Gaze typing with restricted keyboard
6 wpm (Hansen et. al., 2004)
Gaze writing with Dasher
25 wpm (Ward and MacKay, 2002)
Table 8.1. Typical range of human communication, words per minute (wpm) equals 5 characters per minute.
Present day eye gaze communication is efficient at a level that compares to text input methods on mobile devices (PDA and mobile phones), and one particular system, Dasher, shows promise for reaching the level of Morse code and cursive handwriting, c.f. Table 8.1.
Experimental investigations of gaze-based single selections have often found them to be faster than mouse selections (e.g., Sibert and Jacob, 2000). Once the target is located, the pointer is already there. However, when gaze-based selections are used for more involved tasks such as typing or switch selections, this superiority has not been manifest. The speed of gaze selections has often been measured to be very similar to that of hand (mouse) selections (e.g., Calhoun, 1986; Miyoshi and Murata, 2001), but exhibiting a higher error rate (e.g., Ohno, 1998; Hansen et al., 2003). The productivity of previous gaze typing systems using on-screen keyboards has been relatively low, compared with other input modalities. For example Instance, Spinner and Howarth (1996) reported subjects to produce as little as one word per minute (wpm = 5 characters, including space) when entering their name and address, mainly because they spent much time correcting entry errors (e.g. typing the same character twice). Experienced users of "Erica, an early gaze typing system developed by Hutchinson et al. (1989), could produce a full page in 85 minutes, which is approximately 6 wpm, assuming a page contains 2400 characters. Spaepen et al. (1996) found performance to be approximately 7 wpm, and Stampe and Reingold (1995) obtained similar results on their gaze typing system.
Speed is not the only objective to consider when designing a user-friendly gaze communication system. Hansen et al. (2001) referred to additional user requirements for a system to be satisfying. The system should be easy to install, maintain and update. It should consist of standard consumer hardware components that can be replaced immediately when something breaks down. Calibrations should be performed easily and quickly. Tracking should be sufficiently robust to allow for mobile use with occasional changes in light conditions, use of glasses, and minor changes in head position. Prolonged use should not cause fatigue or cause the eyes to dry out. Ideally, the price of the system should not be prohibitively high. Finally, the system should not make the disabled person look awkward. For instance, members of an ALS user group have told us that they would prefer not to wear any kind of peculiar equipment on their head, and that the tracking hardware should either look familiar or be invisible.
When comparing gaze interaction to other access methods used for selection by pointing, several issues should be considered. First of all, gaze may be the only viable interaction form for some users, e.g. in a full locked-in situation. Secondly, gaze interaction may be one of several possible input methods used during the day, to ease the burden of muscles in the neck that are used for head based interaction, to example. Finally, comparisons of gaze interaction with other forms have often been done with previous generations of tracking technology that have now improved considerably. For this reason, the comparisons cited below should not be considered as representative of performance potentials of future systems.
Spaepen et al. (1996) compared gaze dwell time typing to a chin-operated trackerball. After 90 minutes of training, gaze typing was a bit faster than trackerball typing (35 characters per minute as against 30 characters per minute, similar to 7 wpm and 6 wpm, respectively) for able-bodied students. Subjects with multiple sclerosis achieved typing speeds of 5 wpm and 2 wpm, respectively. Error rates were higher for gaze interaction compared to chin (2.4 % versus 1.0%, data only provided for able-bodied subjects). Saepan et al. (1996) proposed that the higher speed with the gaze typing system might have encouraged the subjects to proceed at the expense of accuracy. The low typing speed for the disabled group using a chin track ball was presumably related to a high jaw fatigue reported by this group, especially in the beginning of the experiment. Saepan et al. (1996) concluded the power of gaze interaction to be the minimal motor ability required.
In a recent experiment Hansen et al. (2004) compared the speed of text-production of 12 Danish and 15 Japanese non-disabled subjects. They were writing either by hand-controlled mouse, head-tracking (the "NaturalPoint system) and gaze tracking (Quick Glance system) on a Danish and Japanese on-screen keyboard with very large buttons (only 12 buttons on a full screen). The hand-controlled mouse was the fastest input on day two for both interfaces, 7.5 wpm and 16.1 cpm respectively, whereas the head input yielded 6.1 wpm and 12.3 cpm respectively on day two. Gaze input was found to be 6.3 wpm and 11.4 cpm respectively on the second day. The difference between head and gaze input was not significant in any of the experiments by Hansen et al. (2004).
Accuracy was measured by the number of sentences that either were erroneous or had been corrected by the subjects in percentages of all sentences typed. Errors were very rare in the Japanese experiment for all three input devices. Only 3 % of the sentences typed by hand (mouse) were erroneous or had been corrected, whereas 5 % of the head-typed sentences and 6 % of the gaze-typed sentences had errors or corrections. The Danish subjects were much less accurate. They made errors in 14 % of the sentences for hand (mouse), 14 % for head and 28 % for gaze. In this experiment, subjective scale ratings of the efficiency found the users to be most pleased with the hand (mouse) input. The satisfaction with gaze input was lower than for mouse and head, while the efficiency ratings of head and gaze were almost identical.
8.1.5 What improvements need to be made to existing eye control technology and applications?
First area of possible improvement: Reducing the Price
Eye tracking systems available for people with special needs cost more than some people can afford (often more than 5000 ), and authorities that would normally supply citizens with communication aids may be reluctant to invest this high amount of money if the final effectiveness is uncertain. In a recent paper, Jakob and Karn (2004) explained why eye trackers are so costly:
(The) Eye tracker industry only sell tens or hundreds of systems in a year, which makes it difficult to invest in the large engineering effort that would be required to develop a really good, inexpensive unit. But without such a unit, the market will continue to be limited to tens or hundreds per year a chicken and egg problem.
Therefore, the first area of improvement is definitely to increase the market for gaze trackers by improving them and by informing the authorities about their potential. This is one of the main objectives of COGAIN.
Low-cost eye tracking has been investigated for some time by, for example, Hansen et al. (2002) and Corno et al. (2002). Several students are picking up the challenge to design a low-cost tracker (see e.g. Kreutz, 2003). With the rapid improvement in tracking hardware (video boards, digital camera and CPUs) a break-through may happen within the next few years, even without high investments in engineering efforts.
Second area of possible improvement: Reliability and robustness of tracking
According to Jakob and Karn (2004), 10 to 20 % of all subjects cannot be tracked reliably for continuous interaction. While this may differ from system to system, the robustness of most systems leaves much to be desired, they claim.
Detection of objects can be a difficult task to perform, and in particular detection of the human eye is difficult, as the contrast between eye and skin may be poor. The appearance of the pupil is heavily influenced by occlusions from the eyelids and will be totally covered for every eye blink. The effects of occlusion and illumination changes are also related to the ethnic origin of the user. Both Caucasian and Asian people have facial features that may make iris tracking difficult. The eyelids of Asians are generally close together and may thus result in less boundary information. On the other hand, the nasal bone and superciliary arch of Caucasians are usually more pronounced and therefore casts more shadows on the eye. In addition, the distance between the two eyes also varies across ethnic backgrounds.
Once the eye is located, the iris is a prominent and reliable feature within the eye region because of its high contrast. Eye tracking often includes information about the history of previous positions, whether the eye is open or closed, reflection properties and other information describing the appearance of the eye. In particular, information about previous positions may save computations and reduce the amount of false detections. In addition, although the appearance of the eye of a single user may change dramatically over time, tracking a known persons eye with known dynamics is expected to be more reliable and fast than tracking someone with unknown characteristics. Prior information of this kind could for example be obtained by collecting sufficient statistics to learn the possible appearances and the dynamics over time for a particular user (Blake and Isard, 1998).
Methods used for eye detection and tracking rely on different assumptions on the image data and in general two classes of approaches exist. One common and effective class of eye tracker exploits active illumination from infrared (IR) light emitters. Through novel synchronization schemes and by using the reflective properties of the pupil when exposed to near infrared light (dark or bright pupil effects) the eye can be detected and tracked effectively and reliably. In addition to controlling the light conditions, IR also plays an important role for some gaze estimation methods. Efforts are made to improve eye tracking under various light conditions. Sun light and glasses can seriously disturb the reflective properties of IR light. Methods using IR can therefore be less reliable in these situations and some researchers have suggested new approaches to these problems (e.g. Zhu et al., 2002).
Other approaches avoid the use of active illumination, but rely solely on natural light (e.g. Hansen and Pece, 2005). This is for two reasons: Standard, affordable components of off-the-shelf camera equipment may then be used, and problems with sunlight and glasses may (partly) be overcome. Natural light sources, however, make the problem of detection much harder as fewer assumptions on the image data can be made.
In summary, several new approaches are currently being taken within computer vision research projects that eye tracking may benefit from. The "challenging case of eye tracking is attaining the interest of several research groups. Eye tracking has become a recurring session theme in various computer vision conferences (e.g. Computer Vision and Pattern Recognition (CVPR), International Conference on Computer Vision (ICCV), International Conference on Automatic Face and Gesture Recognition (FG)). Special journal issues on this topic have been planned (e.g. "Computer Vision and Image Understanding, to appear April 2005) and books addressing the problems are forthcoming (e.g. "Physics of the automatic target recognition, Springer-Verlag, in press). It is fair, therefore, to believe that basic research within the field will mature in the years to come and new tracking methods will be invented.
Third area of improvement: Accelerate gaze selections and reduce user errors
Jakob (1991) identified The Midas Touch usability problem for gaze-based interactivity, namely that selections can happen unintentionally, simply because the user is studying the interface. He also noticed that it could be difficult for some people to stare at will in order to do a dwell-time selection. Naturally, the eyes are moved whenever a piece of information has been noticed and a decision to act has been taken. But if this is done before the end of the dwell time, the selection is cancelled. These two problems may explain why gaze selection often falls short on usability in more demanding tasks.
There are several ways to accelerate gaze selection and/or to reduce time-consuming error correction:
- Using word or character predictions to minimize search time for target locations (e.g., Ward and MacKay, 2002)
- Reducing or eliminating the dwell time for each selection (e.g. Salvucci, 1999; Ohno, 1998)
- Using task models to interpret inaccurate input (e.g., Salvucci and Anderson, 2000).
- Designing keys especially for gaze operation (e.g., Majaranta et al., 2003a, 2003b).
- Extensive use of trivial undo functions (e.g., Jacob, 1991).
- Increasing tolerance of noise on the gaze input by using large and/or well-separated selection areas (e.g., Hansen et al., 2003; Miniotas et al., 2003).
Fourth area of improvement: Allow for multi-modal in- and out-put on gaze controlled systems
Ideally, the user should be able to use the same system through all stages of their disability. Many ALS patients, for instance, have little or no previous experience with computers and are quite busy adapting to the severity of their situation; keyboards may be the only input form that they are familiar with. Several communication partner s must be able to help the user complete letters, edit text and use other functions in the program without having to learn an unfamiliar interaction method. Limited time resources among the specialists responsible for introducing and configuring the system mean that the duration and quality of user training is often severely limited.
The progression through the stages of disabilities such as ALS is gradual, and the fatigue factor often makes it necessary for the user to switch to a less efficient input method during the day.
They may become familiar with the system by using a well-known keyboard. When they lose the ability to use their fingers, they may begin to use a mouse, trackerball or joystick. Head tracking may then be the most convenient control that they master when they lose the ability to move their hands and arms. Finally, gaze may be the only function left that can be controlled at will. So a comprehensive system should support input by either a keyboard, mouse/joystick or tracking sensors/cameras.
Hand pointing may be an option to consider for gaze-controlled systems, in addition to keyboard, mouse, head- and gaze input. When the communication partner needs to operate the system, we imagine that (s)he could do so simply by moving a pointing finger in front of the camera that is normally used for head- or gaze-tracking, and thereby control the mouse pointer position. The keyboard is often put aside when it is not needed anyway, we have observed. Even if the keyboard is ready at hand, reaching it from a position aside or behind the user seems less comfortable than simply pointing with a finger in the field of the camera.
Combining voice input with gaze selections could also be a preference for some disabled users, even when it is slurred. For instance, a simple rrrhmmm-sound could command that the system should, for example, return to the main menu or initiate a re-calibration.
Personalized voice output: AAC has been pioneering the field of synthetic speech. Most of the old synthetic voices available were very robotic and monotonous, yet understandable, e.g. to blind people. New speech synthesis technology makes it possible to recreate human voices at a very convincing level compared to normal speech. In future type-to-talk systems, we imagine that some users might record their own pronunciations of their most frequently used words (e.g. before the ALS progresses to the stage of voice impairments or before a laryngectomy patient gets the throat removed). The communication system should then remind and assist the user in recording his or her most important words. A digitised, personalized recording of frequently used words may be used in combination with a pronunciation of unknown words by conventional synthetic speech. Future development of speech technology may present us with a synthetic voice that can be attuned to a personal voice simply by "listening to old samples of it taken from e.g. private video recordings.
Near-range audio communication with communication partners: By observing users of gaze controlled type-to-talk systems in their homes we have identified a need for the synthetic voice output to be transmitted across rooms in a house. For instance, the communication partner may be in the kitchen while the user sits in front of his computer in the living room. At this distance voice output can be difficult to hear, even when played loudly. Also, the communication partner would like to reply without going back to the living room. Therefore, we suggest including an option in the next generation of gaze communication systems that would transmit the synthetic voice output via a wireless network or Bluetooth connection to a headset worn by the communication partner. The communication partner may then speak back at a normal level of voice in the headsets microphone, transmitting the reply to the speakers of the computer in the living room.
Pointing at video images: Hansen et al. (2001) imagined a user-scenario, year 2010, in which a disabled person points with his gaze at an object on a video recording of the contents of his bag, in order to get a bypassing person to fetch it for him. Dahl, an ALS patient, once suggested having a video camera attached to his wheelchair. He would like it to be mounted in a normal eye height position to provide him a normal (standing) view, and he would like to be able to control zooming and movements of the camera by moving his eyes on a monitor on his wheelchair. This, he imagined, could become an efficient pointing stick for him. For instance, he would drive around a self-service table and zoom in on the food that he would like the communication partner s to give him (Henning Dahl, personal communication, 2000).
Fifth area of improvement: Increase mobility
Some of the present gaze tracking systems are quite heavy (up to 10 kilos) and only meant for indoor use at a fixed location. The user has to be seated in front of it and head movements are constrained. Low-weight laptop computers can be placed on a wheelchair table in front of the user. They may even be taken to bed. Some gaze tracking systems therefore offer a lightweight laptop model (e.g. Quick Glance, LC Technologies and Tobii).
However, even with the size and weight of laptop computers decreasing, the system still has to be positioned in front of the user. Consequently, it occupies some space and may not be convenient e.g. at a dinner table. Some people, who use a text-to-type system, have full mobility. They are definitely more obstructed by the constant need to have a computer monitor in front of them than they would be if the system were completely wearable.
Recently, GazeTalk software has been used with a low weight VGA monitor that can be clipped on to a normal pair of glasses. The monitor (model CO 3 from Microoptical) has 640 x 480 pixel resolution and 24-bit colour (60 Hz), see Figure 8.3. Field of view is approximately 16 degrees horizontally.
Figure 8.3. A researcher from ITU typing with GazeTalk by mouse on a micro-display. Typing speed was 12.8 wpm.
Average typing speed with a hand-controlled mouse across 90 sentences yielded 12.8 wpm and 8.6 wpm for two medium-trained subjects. This performance is compatible with the one found among experimental subjects who had a full-size monitor available, - in fact, one of the micro-typing subjects was just as fast as the fastest subject in the experiment reported by Hansen et al. (2004). This was regarded as a very promising observation, indicating that continuous typing on wearable micro-displays may be feasible and can be as efficient as typing on a standard size monitor.
The advantage of gaze tracking on head mounted displays is the freedom of orientation and position. Heynen and Snuggs (1999) report that the portability of the VisionKey head mounted gaze-tracking system is highly valued. Mounting of the tracking camera and display on the head itself eliminates the need to compensate for head movements. Also, the short distance between the eye and the display may provide a noticeable reflection in the cornea of the screen image, which can be used for system self-calibrating purposes or to eliminate vibration noise. As discussed earlier, however, the disadvantage is that these systems may look awkward, at least until it becomes common practice for users to wear such a display in front of a pair of glasses.
8.2 Information from stakeholders
Even though, in terms of long-term users, the number of questionnaire responses and observations was small, they were nonetheless very informative. They provided an indication of the range of activities that are already being achieved through certain eye-control systems and those users would like to achieve.
The range of activities carried out by those users from whom information was gathered included:
- Internet Access
- Social Communication
- Writing (i.e. not for emailing or social communication, e.g. for personal pleasure)
- Instant Messaging
- Playing Games
Those requirements, which users are not currently achieving through eye control but would like to, included:
- Powered mobility
- Environmental control
- Play more video games
- Move independently from one application to another (e.g. from Speaking Dynamically Pro to Writing software)
This is not to say that these activities cannot be achieved with certain, appropriately set-up eye control systems. Nor is it suggested that the list of requirements is in any way comprehensive. However, it provides an important indication of some of the things that certain existing users would like to be able to achieve through eye control but cannot.
In addition to 'eye-writing', eye control can already be used for a variety of applications: environmental control, powered mobility and leisure applications, such as games, music and drawing. In terms of its effectiveness for writing and communication, eye control is already able to produce results of up to 25wpm.
One of COGAIN's aims is that eye control should work more effectively for more people. As a result, it is anticipated that more eye control units can potentially be sold. In turn, it is anticipated that the price per unit will come down. As well as reducing its price, it is planned that amongst anticipated improvements in eye control technology will be greater reliability and accuracy.
Multi-modal input is also essential to enable more users to access the technology more efficiently. In addition, multi-modal input will enable the same user to continue to use the same software and applications whilst, at the same time, changing their access method(s) depending on changes in their physical capabilities, whether during a single day (e.g. if they become tired) or over a period of time (e.g. with a progressive condition). With multi-modal input, for example, a user can change from switch access to eye control and still keep the same social communication package, with all of their personalised vocabulary, whichever access method they use.
Practical considerations must also be taken into account, such as portability of the systems (some weigh up to 10 Kilos), as well as their appearance. How do users feel, for instance, about the appearance of the device? Do they feel comfortable with it, or are they too embarrassed to use it? In response to the questionnaires, the users, themselves, expressed a need to access as many applications as possible as effectively as possible. Finally, it will be an important goal of COGAIN to enable as many of these aims to be realised as possible whilst, at the same time, taking into account such practical considerations as safety and reliability.
9 Discussion and Recommendations
Eye control is a comparatively recent development as a method of access to technology for people with disabilities. From the literature and data collected, it seems that, at present, eye control can only be used effectively to meet a limited range of what users with disabilities require of it. Furthermore, it can only be used effectively by a limited number of people with disabilities (see diagram below).
Figure 9.1. Illustration of the current limited use of eye control technology
This chapter, therefore, will discuss the key issues that have arisen from this document relating to what users require of Eye Control Technology that they cannot already achieve. To address these issues, a number of recommendations are made for consideration by COGAIN partners.
9.1 Issues relating to users' eye control hardware requirements
Whilst many of those considered in this document already successfully use Access Technology to achieve many, if not all, of the things they wish to without using Eye Control, their expectation is that it could provide a more effective, efficient and satisfying form of access. Their current method(s) of access to technology might be slow and/or effortful for them, whereas Eye Control Technology might offer a much more direct and efficient form of access, at least for some of the applications they wish to use. Given the existing situation, where many eye control systems are unable to accommodate certain users with certain physical and/or visual difficulties, the starting point for COGAIN, on behalf of these currently excluded users is to make every effort to make it accessible to them. It is recommended that a good starting point would be to:
- Measure how effectively the eye control technology available can meet the needs of the full range of users who might benefit from it.
There are currently two broad categories of users - those who can use eye control effectively and those who cannot. From the questionnaires and observations made, they can be described as follows:
- Those who have good (or 'mainstream') eye control and who, at least in principle, should be able to use existing Eye Control Technology effectively.
- Those whose eye or head control is sufficiently different to the mainstream that currently available Eye Control Technology is unable to provide an effective calibration.
One of COGAIN's greatest challenges and, if successful, greatest achievements will be to help to make Eye Control Technology available and accessible to as many of those who cannot currently use it effectively as possible. This is particularly important because the alternative methods of access to technology might be extremely slow, painful, harmful, or even impossible for them. There are many thousands of people who fall into this category. They range from some of those with conditions such as athetoid cerebral palsy who might have strong, but uncontrolled, body and head movements to certain people with head injuries, who might find any form of movement, including eye movement, difficult to either initiate or control.
Both prior to and during the COGAIN Project, The ACE Centre has already been collaborating with Tobii Technology in order to make their eye control system accommodate the accessing difficulties of even more users with disabilities even more effectively.
All eye control systems are different though, of course, and some systems will be more effective for certain users than others. As a result, in order to meet the user requirements of as many people with disabilities as possible, one of COGAIN's aims should be to:
- Trial as many specialist eye control systems as possible [#_ftn16 ].
This will provide an opportunity to:
- Feed back to eye control system developers how effective their technology is in meeting the needs of the full range of existing and potential users [#_ftn17 ]
- Make observations and suggestions relating to any potential modifications to their systems and/or software that might make it more accessible and/or more effective for more users [#_ftn18 ].
9.2 Issues relating to users' eye control software requirements
Though it is important that developers of Eye Control hardware systems are supported in allowing as many potential users to use their technology as effectively as possible, it is nonetheless crucial that the assistive software that is used with their systems meets as many users requirements as possible. Features of the wide range of assistive software already used via a range of access methods include the following:
9.2.1 A wide choice of on-screen software interfaces
Figure 9.2. A well-established and well-designed framework program like Clicker can provide a wide range of choice
to users in relation to their preferences regarding text, symbol or pictures.
As wide a variety of on-screen software interfaces need to be offered as possible, to enable the range of choice and the likelihood of matching the system to the user's needs and abilities. Whichever software is used in order to enable users to interact with their eye control system their interface software needs to be as flexible as possible. In relation to a grid-based system, for example, the grids need to be resizable, for example. However, a grid-based system is not the only method of interfacing with the computer. For example, two eye-writing systems that can perform the same task but could not be more different from a user's perspective are 'Qwikwriter' and 'Dasher'.
Figure 9.3. Whilst both QwikWriter and Dasher can be used for eye-writing, they could not be
more different from a user's perspective.
Whilst both applications are similar in that the user can eye-write without the need to 'dwell-click, blink or press a switch, 'Qwikwriter', on the one hand, offers the user a virtually static interface, whereas Dasher is, of course, extremely dynamic.
9.2.2 A range of input methods
Users require eye control software that can accommodate a wide range of input methods, in addition to eye control. This is in order to either (a) enable users to change from one method to another if and when their condition changes, (b) spread the physical load, if required or (c) enable multi-modal access. One reason for accommodating multi-modal access, for example is that, whilst many users will have no choice but to use dwell-select or blink as their mouse-button equivalent, others would wish to use combinations of access methods, such as eye control + voice, eye control + switch, etc. A recent example of potential modifications to a pointer-access program so that it can also be used for switch access is provided by MacKay, Ball and Donegan (2004).
9.2.3 A wide choice of output methods
Much of the software that is written for eye-gaze systems is restricted to 'eye-writing' using text only. However, it is clear from the range of needs and abilities of users with disabilities that they require much more from their eye control applications. They require, for example:
A choice of symbols or text output
Many people who would benefit from eye control communicate using symbols, as opposed to text. For some, this is necessary because they do not have a high level of literacy. For others this is through choice, as they regard it as a quicker and more efficient form of communication, e.g. using Minspeak.
Figure 9.5. A Minspeak application enabling the user to combine symbols on a 'fixed overlay'
to be able to produce thousands of words and phrases.
Indeed, many Minspeak users are able to produce text quicker than users with a 'traditional' text plus predictor application. For this reason, users need to have a choice of whether they use symbols or text in their eye-control grids, whether for writing and/or social communication.
A wide choice of text styles and colours
Even if a user is literate, they may find it preferable or even essential to have their text output presented in a specific way. For this reason, there needs to be a full range of text output, including a choice of fonts, choice of font sizes and choice of foreground and background colours.
Figure 9.6. With Write OutLoud as the word processor you can choose whichever foreground and background colour you wish,
along with an extensive choice of auditory support.
Whether literate or not, some users may find it preferable or even essential to have symbols or pictures displayed instead of text. They will, in turn, require a choice of size, colour, background, etc. for their on-screen symbol displays.
A range of speech facilities
If using eye control for writing or generating symbols, the user needs to have the option of auditory feedback. With 'letter by letter' and/or 'word by word' auditory feedback, for example, for many users, this option would reduce or even make it unnecessary to check what is being written. Without the need to check what has been written visually this would (a) make the production of text quicker and (b) overcome any difficulties related to the so-called 'Midas Touch'.
If using eye control for social communication, a range of speech facilities would be essential. For example, users need to have the facility to choose options relating to, for example, the speed of spoken output, sex and age of the synthesised voice, depending on their personal preference.
9.2.4 A choice of languages
Clearly, there is a need for the eye control software to support as many users, globally, as possible. Therefore, there is a need to provide an option to use as many languages as possible. Dasher, for example, is available for use with over 80 languages.
9.2.5 Summary of software requirements
If we look at a 'typical' eye-writing program in relation to a typical example of a widely available PC-based assistive technology application such as 'The Grid', it is interesting to consider how they compare in terms of the desirable features described above.
Typical Eye-writing program
'The Grid' - a widely available 'framework' assistive technology program
Resizable cells and grids
A range of access methods
A choice of symbols or text output
Text, pictures and symbols
A wide choice of text output styles and colours
One font, one size
Range of font sizes and styles
A range of speech facilities
Limited speech output
Full auditory support - letter by letter, word by word, sentence, etc.
Choice of languages
Table 9.1. Matrix showing a selection of features of a typical 'eye-writing' program
with a widely available 'framework' assistive technology program, 'The Grid'.
These facilities are associated with written and social communication only. Of course, there is a wide range of additional facilities available within programmes like 'The Grid' that users also require, such as the ability to change independently from one application to the next, environmental control, etc. Because so many of these desirable features are already available in existing software, it could be argued that, if we truly wish to meet the full range of user requirements, then it might be easier to adapt an existing piece of software to eye-control than to create a new one. An example of this is the collaborative work of The Inference Group, at Cambridge University and The ACE Centre, Oxford, with the 'DECO: Dasher Eye Control Optimisation Project' (Donegan and MacKay, 2004). Over a period of two years, the two organisations have been working, in consultation with users, to produce an 'enhanced' version of Dasher that is better suited to eye control. Enhanced features include hands-free starting and stopping, colour coding and a range of speech output facilities[#_ftn19 ].
Whilst the development of an eye control version of Dasher through the DECO project has involved collaboration between the developers from The Inference Group, Cambridge, Assistive Technology specialists from The ACE Centre and users, this type of collaboration seems to be the exception rather than the rule. In general, the involvement of users with disabilities has not always been extensively employed in eye control software research and development work. There is a shortage of information, therefore, relating to such questions as:
- How well does the existing range of specialist Eye Control software (e.g. GazeTalk and UKO II) compare with the wide range of currently available 'framework' and specialist software used for access and communication (e.g. The Grid, Intercomm, SAW) currently used for pointer and switch access by many people with disabilities already?
- What improvements/ modifications need to be made to the current range of Eye Control software to meet as many needs of as many existing and potential users as possible?
To following recommendation is suggested in order to find answers to these questions:
- Measure how effectively current software applications are meeting the full range of eye control requirements of users with disabilities.
The User Trials that are an integral part of the COGAIN Project are critical, therefore, to find out this sort of information and should involve the following:
- Make a comparison of the various types of software that are currently or potentially available for eye control. This will include, for example, a comparison of the features of both the kinds of assistive software currently used with a range of access methods (SAW, the Grid, Clicker) with specially written eye-control software[#_ftn20 ].
For example, the WWAAC Internet Browser, is already in use by a wide range of people using different access methods, and can be controlled using other applications such as 'SAW' (from the ACE Centre).
Figure 9.8. The WWAAC browser: shown here with a simple on-screen interface designed with SAW (ACE Centre)
to enable users with physical and /or communication difficulties to access the Internet more effectively.
There is no reason why a powerful interface design tool such as SAW could not be used to design any sort of symbol or text-based user interface.
- To enable users to make an informed choice of which software to use for eye control a matrix should be set up on the COGAIN website relating to features of different software that can (or could be) used for eye control. The comparison could be based on features such as those described above, such as 'choice of output methods', 'range of access methods', range of multi-modal access, etc [#_ftn21 ].
As a result of collating and publishing this information, users and those supporting them will be able to help to 'steer' the work of researchers and developers both inside and outside the COGAIN project.
9.3 Issues relating to safety and reliability
It is essential that Health and safety issues are uppermost in the minds of COGAIN partners when working with end users:
Obviously, users require the technology they use to be both safe and reliable. In terms of the amount of infrared exposure, there are clear guidelines on the level of infra-red output for commercially available products.
- It is recommended that only equipment which meets the legal safety guidelines on infrared output is used in user trials.
Positioning and mounting issues
- When members of the COGAIN partnership carry out user trials involving equipment that the user is not already using, it is recommended that they involve a specialist in the positioning and mounting of the equipment.
This might be a Physiotherapist, Occupational Therapist, Clinical Scientist, or similar specialist. This is a highly complex area where the experience of specialists in positioning and mounting is critical to comfort and safety.
- It is important to ensure that the responsibility for setting up the equipment and its subsequent use is fully covered against any unforeseen adverse effects that might result from the intervention.
Of course, where a user already has an eye control system in place, and the intervention only involves trialling software, then the issues described do not apply. Nonetheless, COGAIN partners must make sure that they have appropriate 'public liability' insurance cover in place.
It is very important to ensure that COGAIN partners have a clear and shared understanding of the terminology we use, both (a) in relation to the Eye Control Technology itself and (b) those for whom it is intended. For this reason, a Glossary of Terms will be provided that will reduce the likelihood of any misunderstandings. It is intended that the Glossary will help to (a) assist clarity of thought and purpose among partners from a technical perspective and (b) ensure that users are involved in a dignified and appropriate way.
'Usability' in design is about producing products and systems that perform their function effectively, efficiently, safely and comfortably. (use diagram page 15)
An infrared device that tracks head movement to control the computer, e.g. via an on-screen pointer.
A camera that tracks the movement of the head or a feature on the face to control the computer, e.g. via an on-screen pointer.
Those parties who will be influenced in a development can be said to have a stake in its use, and can be termed stakeholders. Apart from the end-users, of course, there are a range of secondary users, including formal carers, family members, service providers, whose needs must also be taken into account.
User characteristics will affect users' abilities to use a system and will influence the form that a product or system takes, e.g. nystagmus or involuntary head move.
Design that takes, as its starting point, the needs of the user, rather than something that is bolted on later.
The process of gathering information through 'User Analysis' and 'Activity Analysis'.
What is known about the characteristics of users.
The activities that users need to perform.
The process by which text is produced using eye control.
A system with which a user can independently move from one from of assistive technology to another independently, e.g. between the following assistive technology: environmental control, social communication, writing, wheelchair control
Software that enables an end-user to access the computer with more efficiency, effectiveness or satisfaction.
AbleNet, Inc. homepage. Available at http://www.ablenetinc.com.
Accot, J. and Zhai, S. (1997) Beyond Fitts Law: Models for Trajectory-Based HCI Tasks, Proceedings of CHI'97, pp.295302.
Barea, R., Boquete, L., Mazo, M. and Lopez, E. (2002) System for assisted mobility using eye movements based on electro-oculography. IEEE Trans Neural Syst Rehab Eng., 2002, Dec 10(4), pp.20918.
Bates, R. (1999) Multimodal Eye-based Interaction for Zoomed Target Selection on a Standard Graphical User Interface. Proceedings of Interact'99, vol. II, British Computer Society, pp.78.
Bates, R. (2002) A Computer Input Device Selection Methodology for Users with High-Level Spinal Cord Injuries. Proceedings of the 1st Cambridge Workshop on Universal Access and Assistive Technology (CWUAAT), Trinity Hall, University of Cambridge 25th-27th March 2002.
Bates, R. (2005) Doctoral Thesis Enhancing the Performance of Eye and Head Mice: A Validated Assessment Methodology and Investigation into the Performance of Eye and Head Based Assistive Technology Pointing Devices. De Montfort University 2005.
Beukelman, D.R. and Mirenda, P. (1992) Augmentative and Alternative Communication: Management of Severe Communication Disorders in Children and Adults. Paul H. Brooke Publishing Co, Baltimore.
Bevan, N., Kirakowski, J. and Maissel, J. (1991) What is usability?. Human Aspects in Computing: Design and Use of Interactive Systems and Work with Terminals, Elsevier Science.
Bevan, N., Rengger, R., Macleod, M., Bowden, R., Drynan, A., Blayney, M. and Cooper, D. (1995) The Performance Measurement Handbook Version 3, National Physical Laboratory, Teddington, Middlesex, United Kingdom, TW11 0LW, NPLUS/PMH/v3.0/Dec 95.
Blake, A. and Isard. M. (1998) Active Contours: The Application of Techniques from Graphics, Vision, Control Theory and Statistics to Visual Tracking of Shapes in Motion. Springer-Verlag.
Byrne, M.D., Anderson, J.A., Douglass, S. and Matessa, M. (1999) Eye tracking the visual search of click-down menus. Proceedings of CHI'99 Conference on Human Factors in Computing Systems, NewYork: ACM, pp.402409.
Calhoun, Gloris L. (1986) Use of eye control to select switches. Proceedings of the Human Factors Society, 30th Annual Meeting, pp.154158.
CameraMouse, Inc. homepage. Available at http://www.cameramouse.com.
Canzler, U. and Kraiss, K.F. (2004) Person-Adaptive Facial Feature Analysis for an Advanced Wheelchair User-Interface. In Paul Drews (Eds.), Conference on Mechatronics & Robotics 2004, Volume Part III, September 13-15, Aachen, Sascha Eysoldt Verlag, ISBN 3-938153-50-X, pp.871876.
Chapman, J. E. (1991) The Use of Eye-Operated Computer System in Locked-In Syndrome. Proceedings of the Sixth Annual International Conference on Technology and Persons with Disabilities (CSUN91), Los Angeles, CA.
Charlier J., Buquet, C., Dubus, F., Hugeux, J.P. and Degroc, B. (1997) VISIOBOARD: A new gaze command system for handicapped subjects. Medical and Biological Engineering and Computing, 1997, 35, 416 supplement. D90.OS1.03.
Chen, Y.L., Tang, F. T., Chang, W.H., Wong, M.K., Shih, Y.Y. and Kuo, T.S. (1999) The new design of an infrared-controlled human-computer interface for the disabled. IEEE Trans. Rehab. Eng., vol. 7, Dec. 1999, pp.474481.
COGAIN Annex 1 (2004). Communication by Gaze Interaction: COGAIN - 511598 - Annex 1, p.6.
Corno, F., Farinetti, L. and Signorile, I.(2002) A Cost-Effective Solution for Eye-Gaze Assistive Technology. ICME2002: IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland.
Donegan, M. and MacKay, D.J.C. (2004) DECO: The Dasher Eye Control Optimisation Project. Proceedings of the Eleventh Biennial International Conference of International Society for Alternative and Augmentative Communication (ISAAC 2004) Natal, Brazil.
Douglas, S.A., Kirkpatrick, A.E. and MacKenzie, I.S. (1999) Testing Pointing Device Performance and use Assessment with the ISO9241, Part 9 Standard. Proceedings of CHI99, ACM Press.
Drinker, P.A. and Kropoff, S. (1981) Eye-link for non-vocal communication: Direct selection by eye contact. Paper presented at the 4th Annual Conference on Rehabilitation Engineering, Washington, DC.
Duchowski, A.T. (2003) Eye Tracking Methodology: Theory & Practice, Springer-Verlag, London, UK, 2003.
EUROSTAT (2002). http://europa.eu.int/comm/eurostat, 2002.
Evans, D.G., Drew, R. and Blenkhorn, P. (2000) Controlling Mouse Pointer Position Using an Infrared Head-Operated Joystick. IEEE Trans. Rehab. Eng., vol. 8, no. 1, pp.107117.
Findlay, J. M. and Gilchrist, I. D. (2003) Active vision: The psychology of looking and seeing. Oxford University Press, Oxford.
Fould, R. (1980) Communication rates for nonspeech expression as a function of manual tasks. Proceedings of the International Conference on Rehabilitation Engineering (RESNA), pp.8387.
Frey, L.A, White, K.P. Jr. and Hutchinson, T.E. (1990) Eye-Gaze Word Processing, IEEE Transactions on Systems, Man, and Cybernetics 20 (4), pp.944950.
Gips, J., Olivieri, C.P. and Tecce, J.J. (1993) Direct control of the computer through electrodes placed around the eyes. In Smith, M.J. and Salvendy, G. (eds.) Human-Computer Interaction: Applications and Case Studies, Elsevier, Amsterdam, pp.630635.
Goldberg, J.H. and Wichansky, A.M. (2003) Eye tracking in usability evaluation: A practitioner's guide. In Hyn, J., Radach, R. and Deubel, H. (eds.) The Mind's Eyes: Cognitive and Applied Aspects of Eye Movements. Oxford, Elsevier Science..
Grossens, C.A. and Crain, S.S. (1987) Overview of Nonelectronic Eye-Gaze Communication Techniques. AAC Augmentative and Alternative Communication, 3(2), 77- 89.
Hansen, D.W., Hansen, J.P., Nielsen, M., Johansen, A.S. and Stegmann, M.B. (2002) Eye Typing using Markov and Active Appearance Models. IEEE Workshop on Applications on Computer Vision, pp.132136.
Hansen, D.W. and Pece, A.E.C. (2005) Eye tracking in the wild. Computer Vision and Image Understanding, 98(1), April 2005, pp.155 181.
Hansen, J.P., Johansen, A.S., Hansen, D.W., Itoh, K., and Mashino, S. (2003a) Language Technology in a Predictive, Restricted On-screen Keyboard with Ambiguous Layout for Severely Disabled People. EACL 2003 Workshop on Language Modeling for Text Entry Methods, April 13, 2003, Budapest, Hungary.
Hansen, J.P., Johansen, A. S., Hansen, D. W., Itoh, K. and Mashino, S. (2003b). Command Without a Click: Dwell Time Typing by Mouse and Gaze Selections. Rauterberg, M. et al. (Eds.) Human-Computer Interaction INTERACT03. IOS Press, pp.121128.
Hansen, J.P., Torning, K., Johansen, A.S., Itoh, K. and Aoki, H. (2004) Gaze typing compared with input by head and hand. Proceedings of the Eye Tracking Research & Applications Conference (ETRA 2004), ACM Press.
Hansen, J.P., Hansen, D.W., Johansen, A. S. (2001) Bringing Gaze-based Interaction Back to Basics. Proceedings of Universal Access in Human-Computer Interaction (UAHCI 2001), New Orleans, Louisiana.
Heynen, J. and Snuggs, G. (1999) Eye Controlled Computing: The VisionKey Experience. Paper presented at CSUN 99. Available at: http://www.eyecan.ca/pages/vkexperience.htm (confirmed 2005-02-19).
Hornof, A.J. and Cavender, A. (2005, to appear). EyeDraw: Enabling children with severe motor impairments to draw with their eyes. Proceedings of ACM CHI 2005: Conference on Human Factors in Computing Systems, New York: ACM.
Hornof, A. J., Cavender, A., and Hoselton, R. (2004) EyeDraw: A system for drawing pictures with the eyes. Extended Abstracts of ACM CHI 2004: Conference on Human Factors in Computing Systems, New York: ACM, pp.1251125.
Hornof, A., and Sato, L. (2004). EyeMusic: Making Music with the Eyes. Proceedings of the 2004 Conference on New Interfaces for Musical Expression (NIME04), Hamamatsu, Japan, June 3-5, pp.185188.
Hutchinson, T.E., White, K.P., Martin, W.N., Reichert, K.C. and Frey, L.A. (1989) Human-Computer Interaction Using Eye-Gaze Input, IEEE Transactions on Systems, Man, and Cybernetics 19 (6), pp.15271534.
International Alliance of ALS/MND (Amyotrophic Lateral Sclerosis and Motor Neurone Disease) Associations. http://www.alsmndalliance.org.
ISO (1998) ISO/TC 159/SC4/WG3 N147: Ergonomic requirements for office work with visual display terminals (VDTs) - Part 9 - Requirements for non-keyboard input devices. International Organisation for Standardisation, May 25, 1998.
Istance, H.O.I. and Howarth, P. (1993) Performance Modelling and Investigation of User Performance using an Eyetracker as an Input Device. Polish-English Meeting on Information Systems, Technical University of Bialystock, Poland
Istance, H.O.I., Spinner, C. and Howarth, P.A. (1996a) Eye-based control of standard GUI software. Proceedings of People and Computers XI HCI '96, Springer
Istance, H.O.I., Spinner, C. and Howarth, P.A. (1996b) Providing Motor-Impaired users with Access to Standard Graphical User Interfaces (GUI) Software via Eye-Based Interaction. Proceedings of the 1st European Conference on Disability, Virtual Reality and Associated Technologies (ECDVRAT96), Maidenhead, UK.
Jacob, R. J. K. (1991). The use of eye movements in human-computer interaction techniques: what you look at is what you get. ACM Transactions on Information Systems (TOIS) 9, pp.152169.
Jacob, R.J.K. (1993) Eye-gaze computer interfaces: What you look at is what you get. IEEE Computer 26, 7 (July 1993), pp.6567.
Jacob, R.J.K. (1995). Eye tracking in advanced interface design. In Barfield, W. and Furness, T. (Eds.) Advanced interface design and virtual environments. New York, Oxford University Press.
Jacob, R.J.K. and Karn, K.S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. (Section commentary). In Hyn, J., Radach, R. and Deubel, H. (eds.) The Mind's Eyes: Cognitive and Applied Aspects of Eye Movements. Oxford, Elsevier Science. Available at http://www.cs.tufts.edu/~jacob/papers/ecem.pdf.
James, C.L. and Reischel, K.M. (2001) Text input for mobile devices: Comparing model predictions to actual performance. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2001. New York: ACM, 2001, pp.365371.
Jeppesen, B.B., Andersen, J., Grnnegaard, A., Hauge, M., Kirkegaard, J. and Seelen, B. (2004) Speaking with the eyes. DVD (in English) available from http://www.thi-fyn.dk.
Jordan, P.W. (2000) Inclusive Design: An Holistic Approach. Proceedings of the IEA 2000/HFES 2000 Congress, pp. 917920.
Kahn, D.A., Heynen, J. and Snuggs, G.L. (1999) Eye-Controlled Computing: The VisionKey Experience. Proceedings of the Fourteenth International Conference on Technology and Persons with Disabilities (CSUN99), Los Angeles, CA.
ten Kate, J.H., Donker-Duyvis, J. and LePoole, J.B. (1983-84) Optical Communication Aids in Rehabilitation. Medical Progress Through Technology, 10(2), pp.109123.
ten Kate, J.H., Frietman, E. E., Stoel, F. J. and Williems, W. (1980) Eye-controlled Communication Aids. Medical Progress Through Technology, 8(1), pp.121.
ten Kate, J.H., Frietman, E.E.E., Willems, W., Ter Haar Romeny, B.M. and Tenkink, E. (1979) Eye-Switch Controlled Communication Aids. Proceedings of the 12th International Conference on Medical & Biological Engineering, Jerusalem, Israel.
Kreutz, K. (2003) Infrared Gaze Treacking in Matlab. Barchelor student thesis. The University of Queensland. Available from innovexpo.itee.uq.edu.au confirmed 2005-02-22
Lankford, C. (2000) Effective Eye-Gaze Input Into Windows. Proceedings of the Eye Tracking Research & Applications Symposium 2000 (ETRA'00), Palm Beach Gardens, FL, ACM Press, pp.2327.
LC Eyegaze (2001) The Eyegaze Development System. A Tool for Eyetracking Applications. LC Technologies, Inc. August 2001.
MacKay, D.J.C., Ball, C.J. and Donegan, M. (2004) Efficient communication with one or two buttons. Proceedings of Maximum Entropy and Bayesian Methods, 2004
MacKenzie, I.S. (1991) Fitts' law as a performance model in human-computer interaction. Doctoral dissertation, University of Toronto.
MacKenzie, I.S. (1992) Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7, pp.91139.
MacKenzie, I.S., and Buxton, W. (1992) Extending Fitts Law to Two-Dimensional Tasks. Proceedings of CHI 1992.
MacKenzie, I.S. and Soukoreff, R.W. (2002) Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction, 17, pp.147198.
Majaranta, P., Aula, A. and Rï¿½ihï¿½, K.J. (2004) Effects of feedback on eye typing with a short dwell time Proceedings of the Eye tracking research & applications symposium on Eye tracking research & applications, San Antonio, Texas, pp.139146.
Majaranta, P., MacKenzie, I.S., A. Aula, and Rih, K.-J. (2003a) Auditory and visual feedback during eye typing. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems CHI 2003. New York: ACM, pp.766767.
Majaranta, P., MacKenzie, I.S. and Rih, K.-J. (2003b) Using motion to guide the focus of gaze during eye typing. Abstracts of ECEM12 12th European Conference on Eye Movements, Dundee, Scotland, August 2003.
Majaranta, P. and Rih, K.J. (2002) Twenty Years of Eye Typing: Systems and Design Issues. Proceedings of the Symposium on ETRA 2002: Eye Tracking Research & Applications Symposium 2002, New Orleans, Louisiana, pp.15 22.
Matias, E., MacKenzie, I.S. and Buxton, W. (1993) Half-QWERTY: A one-handed keyboard facilitating skill transfer from QWERTY. Proceedings of the ACM Conference on Human Factors in Computing Systems INTERCHI 93. New York: ACM, pp. 8894.
Matsumoto, Y., Ino, T. and Ogasawara, T. (2001) Development of Intelligent Wheelchair System with Face and Gaze Based Interface, Proceedings of 10th IEEE Int. Workshop on Robot and Human Communication (ROMAN 2001), pp.262267.
Millan, J.R. (2003) Adaptive Brain Interfaces. Communications of the ACM, vol.46, no. 3, pp.7580.
Miniotas, D., Spakov, O. and Evreinov, G. (2003) Symbol Creator: an alternative Eye-based Text Entry Technique with Low Demand for Screen Space. Proceedings of INTERACT 2003, September 2003, pp.137143.
Miyoshi, T. and Murata, A. (2001) Input Device Using Eye Tracker in Human-Computer Interaction. IEEE International Workshop on Robot and Human Interactive Communication, pp.580585.
Murata, A. (1991) An experimental evaluation of mouse trackball joystick joycard lightpen trackball and touchscreen for pointing - basic study on human interface design. In Bullinger, H. J. (ed.) Human aspects in computing:design and use of interactive systems and work with terminals. Elsevier.
Nguyen, K., Wagner, C., Koons, D. and Flickner, M. (2002) Differences in the infrared bright pupil response of human eyes. Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA 2002), New Orleans, LA. ACM Press: New York, pp.133138.
Ohno, T. (1998) Features of Eye Gaze Interface for Selection Tasks. The Third Asia Pacific Computer Human Interaction - APCHI'98, IEEE Computer Society, pp.176182.
Origin Instruments Corporation homepage. Available at http://www.orin.com.
Pierpont, W.G. (1997) The Art & Skill of Radio Telegraphy. N0HFF, 3rd Revised Edition (2002) Available at http://www.qsl.net/n9bor/n0hff.htm.
Rasmusson, D., Chappell, R. and Trego, M. (1999) Quick Glance: Eye-tracking Access to the Windows95 Operating Environment. Proceedings of the Fourteenth International Conference on Technology and Persons with Disabilities (CSUN99), Los Angeles, CA
Radwin, R.G., Vanderheiden, G.C. and Lin, M.L. (1990) A method for evaluating head-controlled computer input devices using Fitts' law. Human Factors, Aug;32(4):423-38.
Salem, C. and Zhai, S. (1997) An Isometric Tongue Pointing Device. Proceedings of CHI 97. Available at http://www.acm.org/sigchi/chi97/proceedings/tech-note/cs.htm.
Salvucci, D.D. (1999) Inferring intent in eye-based interfaces: tracing eye movements with process models. Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, pp.254261, May 15-20, 1999, Pittsburgh, Pennsylvania, United States.
Salvucci, D.D. and Anderson, J.R. (2000) Intelligent gaze-added interfaces. Proceedings of CHI 2000, 2, ACM Press, pp.273280.
Shaw, R., Crisman, E., Loomis, A. and Laszewski, Z. (1990) The Eye Wink Control Interface: Using The Computer To Provide The Severely Disabled With Increased Flexibility And Comfort. Third Annual IEEE Symposium on Computer-Based Medical Systems, Chapel Hill, NC, USA, 1990.
Shein, G.F. (1997) Towards Task Transparency in Alternative Computer Access: Selection of Text Through Switch-Based Scanning. Ph.D. Thesis, Dept. of Industrial Engineering, University of Toronto.
Shein, G.F., Treviranus, J., Brownlow, N.D., Milner, M. and Parnes, P. (1992) An overview of human-computer interaction techniques for people with physical disabilities. Int. Journal of Industrial Ergonomics, vol. 9, issue 2, February 1992, pp.171181.
Sibert, L.E. and Jacob, R.J.K. (2000) Evaluation of eye gaze interaction. Proceedings of CHI 2000, 2, ACM Press, pp.281288.
Smith, W.J. (1996) ISO and ANSI Ergonomic Standards for Computer Products: A Guide to Implementation and Compliance. Prentice Hall, New York.
Spaepen, A., Edwards, K., Verheust, M. Van Mele, I., Wauters, F. Schampheleer, W., Ketelaer, P. Jacobs, R., Hendrickx, E. and van Steenberghe, D. (1996). Access design for all: a study on learning curves during training on devices for textual input to telecommunications equipment for the motor impaired. ICCHP '96: Proceedings of the 5th International conference on Computers helping people with special needs. Part I, pp.277282.
Stampe, D.M. and Reingold, E.M. (1995) Selection by Looking: A Novel Computer Interface and its Application to Psychological Research. In Findlay, J.M., Kentridge, R.W. and Walker, R. (Eds.) Eye Movement research: Mechanisms, processes and applications. Elsevier.
Tobii (2003). User Manual. Tobii eye-tracker and ClearView analysis software. Tobii Technology AB, December 2003.
Velichkovsky, B.M., Pomplun, M. and Rieser. H. (1996). Attention and communication: Eye-movement-based research paradigms. In Zangemeister, W.H., Stiel, S. and Freksa, C. (Eds.) Visual attention and cognition. Amsterdam/New York: Elsevier Science Publisher.
Ward, D. J. and MacKay, D. J. C. (2002) Fast hands-free writing by gaze direction. Nature, vol. 418, number 6900, p.838.
Wiklund, M.E., Dumas, J.S. and Hoffman, L.R. (1987) Optimizing a portable terminal keyboard for combined one-handed and two-handed use. Proceedings of the Human Factors Society -- 31st Annual Meeting, Santa Monica, CA: Human Factors Society, pp.585589.
Zhai, S., Hunter, M., and Smith, B. A. (2000). The Metropolis keyboard: An exploration of quantitative techniques for virtual keyboard design. Proceedings of the UIST 2000 Symposium on User Interface Software and Technology.
Zhu, Z., Fujimura, K. and Ji, Q. (2002). Real-time eye detection and tracking under various light conditions. Symposium on ETRA 2002: Eye Tracking Research Applications Symposium, New Orleans, Louisiana, pp.139144.
The appendices are available in PDF format:
[#_ftnref1 ] It is anticipated that this will be carried out in collaboration with other Work packages, e.g. WP5: 'Eye Tracker Development'.
[#_ftnref2 ] It is anticipated that this will be carried out in collaboration with other Work packages.
[#_ftnref3 ] It is anticipated that this will be carried out in collaboration with other Work packages.
[#_ftnref4 ] It is anticipated that this will be carried out in partnership with WP4: 'Tool development'.
[#_ftnref5 ] It is anticipated that this will be carried out in partnership with WP4: 'Tool development'.
[#_ftnref8 ] These user trials involved trying to achieve as successful a calibration as possible with a selection of users with complex difficulties and subsequently using the calibration to gain an indication of how effectively they might be able to use a selection of eye-writing applications.
[#_ftnref9 ] Text adapted from http://www.eyegaze.com/2Products/Disability/Medicaldata.html (courtesy of Dixon and Nancy Cleveland).
[#_ftnref10 ] The E-tran frame is a deceptively simple system of communication that utilises eye-pointing. The transparent frame is held between the non-speaking person and their conversation partner. The central window enables full eye contact. The non-speaking person is first asked to look at the block of letters that contains the letter first letter of the word they wish to communicate. They then look at the coloured spot that corresponds with the colour of the letter. The conversation partner may be able to guess the word, or the non-speaking person can carry on to the next letter. In this way, complex conversations can be held.
[#_ftnref11 ] The BCI research was presented at the Joint Research Centre of the European Commision in Ispra, Italy (Millan, 2003).
[#_ftnref12 ] Spinal Injuries Association Company Limited, Newpoint House, 76 St James Lane, London.
[#_ftnref13 ] Where bandwidth may be defined as the amount of information communicated to the interface per unit time by the modality. For example, a switch generates low bandwidth binary information, a desktop mouse higher bandwidth x ,y positional information.
[#_ftnref14 ] Target acquisition test from Accot and Zhai, 1997
[#_ftnref15 ] Text editor and keyboard from Majaranta et al., 2004
[#_ftnref16 ] It is anticipated that this will be carried out in partnership with WP5: 'Eye tracker development'
[#_ftnref17 ] It is anticipated that this will be carried out in partnership with WP5: 'Eye tracker development'
[#_ftnref18 ] It is anticipated that this will be carried out in partnership with WP5: 'Eye tracker development'
[#_ftnref19 ] Speech facilities planned to be fully implemented by Summer 2005
[#_ftnref20 ] It is anticipated that this will be carried out in partnership with WP4: 'Tool development'
[#_ftnref21 ] It is anticipated that this will be carried out in partnership with WP4: 'Tool development'
This wiki code has been generated with I love wiki, an HTML to wiki syntax converter