logo braillenet

Access to Mathematics by Blind Students

retour colloque

Arthur I. Karshmer and Chris Bledsoe
nformation Technology Program
University of South Florida
Lakeland, FL USA

  1. Introduction

  2. Reading and writing mathematics is fundamentally different than reading and writing text. While Braille is adequate for the representation of text, it is not up to the task of representing mathematics. The two basic reasons for this are:

    1. Linearity

    2. Text is linear in nature while mathematical equations are two dimensional. What you have been reading in this text is a good example of this problem. In contrast, examine the following relatively simple equation

      formule simple

      Figure 1: A relatively simple equation

      One will immediately notice that the equation contains a superscript and a fraction - both being two dimensional in nature. The equation could have been written in a linear form, for example:

      a = sqrt(((x super 2) - y) / z)

      For this relatively simple equation, a linear representation is adequate for reading to a blind user. But, with any increase in complexity it becomes apparent that linear representations are no longer useful.

    3. Character Set

    Text can be generally represented in a somewhat limited number of characters which normally include upper and lowercase letters, the 10 digits, various punctuation marks and a small set of special characters. Equations on the other hand can contain all of the normal text characters plus a large number of special characters.

    Normal 6-dot Braille can intrinsically represent 64 unique characters. Through the use of special "escape" sequences Braille can support a much larger set of characters. But this comes at some cost. This means that the basic characters that can be represented in Braille can have different meaning in different contexts. While there is effectively no limit to the number of characters one can represent in the fashion, the reading and understanding of these special sequences becomes very involved. The more new characters one needs, the more escape sequences one requires. So, for example the letter "a" can have different meanings in different contexts. It could be an "a," an "A", a "1," and so on. This makes for difficult reading and/or writing of mathematical equations in Braille.
    Now, add to the mix the multi-dimensional nature of all but the simplest equations, and the escape sequences required to represent them. It becomes clear that Braille math representation is very difficult given the character set that can be represented in 6-dot Braille.

    In spite of these limitations, numerous Braille mathematical notations have been developed over the years, primarily based on 6-dot Braille. In the United States the Braille math standard is named after it inventor, Dr. Abraham Nemeth. Other countries have their own standards and some countries have more than one. These representations use the tricks described above and are therefore not easy to learn.

    Add to these problems the fact that the vast majority of mathematics teachers don't know, and don't care to learn math Braille notations and you can easily see the problems associated with learning math if you are severely visually impaired or blind.

    Finally, while text can be recorded by sighted readers in the form of talking books, this technique does not work well with transmitting mathematical equations (Gillan, et al. - 2001). Having a human reader at your side does help, but is an impractical solution on several levels.

    Other techniques have been tried to solve these basic problems, and these approaches will be presented in the remainder of this paper. While not an exhaustive review of other techniques, we present several examples of methods reported in the literature.

  3. General Approaches

    Most of the work reported in developing techniques to help visually impaired students deal with mathematics fall within the following general categories :

    While more exotic approaches have been postulated and are possible in the future, we will examine work that falls into the basic categories listed above.

  4. Tactile Representations

    We have already discussed the use of traditional 6-dot Braille and its limitations. Now we turn to other tactile methods. First and foremost amongst these other tactile methods is 8-dot Braille representation. By using 8 dots, we extend our basic character set from 64 to 256 characters. Additionally, most 8-dot systems attempt to have some sort of graphics associated with the dot pattern (Weber - 1994; Gardner - 2001).

    In an effort to alleviate the problem of requiring teachers to learn Braille, there have been projects that allow a teacher to prepare instructional materials in familiar electronic ways and then have their work automatically translated into Braille. Three such projects are, 1) the ASTER project (Raman - 1994), 2) the LABRADOOR project (Miesenberger, et al. - 1998) and 3) the MAVIS project (Karshmer, et al. - 1999). Other markup languages have also been used to accomplish the same end as seen in the MATHS project which used SGML (Edwards - 2001) and the MAVIS project which now uses XML (Karshmer, et al. - 2001), By using standard markup languages, these projects offer teachers powerful tools to create instructional materials and make them usable by the blind. Another feature of many of these projects is their ability to generate refreshable Braille output.

    Tactile output is also very helpful for the blind math student in other ways. For example, sighted students can easily examine the graphical output of a sine function, but for the blind this is not an easily achieved option. The ultimate goal here would be the development .of a high resolution refreshable tactile device that could present images to be touched by the user. There are several industrial efforts to create such a device that would be inexpensive enough to be practical, but it hasn't been accomplished yet. In the meantime, there are other devices that can prepare the equivalent tactile sheet on paper or paper-like substances, One of the more promising of these devices is the Tiger Printer (Walsh and Gardner - 2001).

  5. Audio Aids

    Audio has been one of the most popular and successful output media for the blind student over the years and is employed in a wide variety of computer-based device interfaces. Certainly, Jaws for Windows (Freedom Scientific - 2001) is a necessary tool for any blind computer user. But this tool is primarily designed for the general user interface and is not well suited for more technical data interfaces.

    The problems associated with the automated reading of equations have been the subject of several research projects over the years. Professor Abraham Nemeth not only defined the U.S. standard for math Braille representation, but also developed a simple spoken structure for reading equations (Nemeth - 1996). An interesting overview of the problem of reading math (Hayes - 1996) offers an overview of techniques developed to that date. Finally, the MAVIS project carried out a series of psychological experiments to better understand the process and its inherent problems (Gillan et al., - 2001).

    Two early projects in the delivery of technical information to the blind student through audio output were ASTER (Raman - 1994) and Talking Emacs (York and Karshmer - 1991). ASTER reads complex equations to the listener with special sonification to highlight parts of the equation while the Talking Emacs project reads C code to the user providing descriptive information about the code in addition to the basic program material. Given that the project was based on the Emacs editor work was planned to enhance the editor's base Lisp code to also read equations.

    More sophisticated equation reading were key to both the MATHS and MAVIS projects. Additional features such as the use of prosody and highly flexible equation browsers to allow the user to navigate through complex equations. The MATHS system also uses non-speech sounds to give an overview of the equation. MAVIS (see Figure 2), on the other hand uses a hierarchical structure to group sub sections of an equation into meaningful "chunks," for browsing. Both MAVIS and MATHS also offer refreshable Braille output and specialized video output for low vision users.

    Figure 2 - The MAVIS Math Genie

  6. Tonal Representations

    Reading equations is not the whole problem associated with mathematics and blind students. The output of a function, for example, can also be a multi-dimensional structure. These forms of mathematical structure are even more difficult to describe in words than an equation.

    Sonification of graphs has been the subject of several interesting research projects over the years (Mansur - 1975; Kennel - 1996). In theses systems, musical tones are used to represent the shape of graphical object. Both two- and three-dimensional objects have been sonified with a these techniques.

    The use of strictly tonal representations have had limited success especially when used for complex graphs. Hearing alone seems to lack the information carrying capability to describe complex objects. While not a complete solution, sonification can be used with other non-visual output media to describe graphs.

  7. Haptic Devices

    The development of high resolution refreshable dot grids would be the best way to represent both text and non-text data. Unfortunately, such devices are still way beyond an affordable price. Other devices like the Tiger printer can produce relatively inexpensive raised dot paper output, but they are not dynamic in nature once they are embossed.

    Another solution to this problem can be found in haptic or forced feedback devices. Such devices can work in either two- or three-dimensional space, and under computer control allow the user the "feel" shapes (Yu, et al. - 2000; Kennedy, et al., 1992; Sjostrom - 1999). Again, high-quality haptic devices are reasonably expensive, but new less expensive devices are now coming on the market. Prices can range from US$ 6,000 down to US$ 100. Naturally, the less expensive devices don't offer the resolution of their more expensive counterparts.

  8. Integrated Approaches

    While all of the approaches to delivering math education to blind students have a great deal of merit, the truly successful ones offer a number of interface techniques. In the domain of reading equations, for example, the MATHS and MAVIS projects used multiple output media to accomplish their tasks. These included speech, sound, Braille and structured navigational techniques. In the case of MAVIS research from math reading experiments laid the groundwork for less ambiguous equation reading and the ability to speak equations into the system by the user.

    Figure 3 - Math Braille Translation

    Another example of integrated approaches is seen in the combination of haptic and sonic devices for describing graphical output (Grabowski and Barner - 1998; Sahyun and Gardner - 1998 In these projects, forced feedback devices are coupled with sonifcation techniques to deliver a high quality rendering of the graphical output. An interesting project from the University of Bologna in Italy includes both haptic and sonic output to describe objects, but also incorporates vibrators to represent colors in the form of a tactile RGB display (Bonivento - 1999).

    Finally, a new project at the University of South Florida, with partners in New Mexico, Texas and Austria, is integrating several input and output media to build a tool to translate a variety of national math Braille representations (see Figure 3). The tool will allow blind mathematicians in different countries to use tools developed in other projects to exchange mathematical works with blind colleagues in other countries. Here the math browser developed in the MAVIS project is integrated into a tool that translates different math Braille notations through standard markup languages to other national math Braille notations.

  9. Conclusions

    Making mathematics accessible to the blind is a challenging and difficult process. The computer and its range of output devices has become the foundation of numerous projects that have brought this goal closer to a reality. With I/O devices such as high-quality speech, musical tones, refreshable Braille, haptic feedback and high reliability speech input, new and effective tools will soon be on the market. Other research into direct neural connectivity, will in the future, make the picture even brighter.

  10. Acknowledgements

    Quelques-uns des travaux décrits dans ce papier ont reçu des fonds de la Fondation Nationale des Sciences des Etats-Unis. et du Département de l'Education, de l'Institut National de la Recherche sur les handicaps et la Réadaptation. ..

  11. Références

Some of the work reported in this paper was funded under grants from the U.S. National Science Foundation (HRD 9800209 & HRD 9906130) and the U.S. Department of Education, National Institute of Disabilities and Rehabilitation Research (H133G010046).

Bonivento, C., Virtual Reality for the Visually Impaired: VIDET, www.lar.deis.unibo.it/activities/videt

Edwards, A.,D., www.cs.york.ac.uk/maths

Freedom Scientific, Inc., www.freedomscientific.com

Gardner, John, http://dots.physics.orst.edu/dotsplus.html, 2001

Gillan, D., Pennington, E., Pazuchanics, S., Karshmer, A., and Pontelli, E., "Reading Complex Structures," The Proceedings of the 45th Human Factors and Ergonomics Conference, Minneapolis/St. Paul, October, 2001.

Grabowski, N.A., and Barner, K., "Data Visualization Methods fro the Blind Using Force Feedback and Sonification," SPIE Conference on Telemanipulator and Telepresence Technologies, Boston, 1998

Hayes, Brian, "Speaking Mathematics," The American Scientist, March-April, 1996

Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C., Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999

Karshmer, A.I., Gupta, G., Pontelli, E., Miesenberger, K., and H-F Guo, "The Development of a Tool to Enhance Communications Between Blind and Sighted Mathematicians, Students and Teachers: A Global Translation Appliance," Proceedings of the HCI Conference, New Orleans, 2001

Kennel, A.R., "AudioGraf: A Diagram Reader for the Blind," ASSETS 96 Conference, 1996

Mansur, D.,L., "Graphs in Sound: A Numerical Data Analysis Method for the Blind,," Masters Thesis, University of California at Davis, 1975

Miesenberger, K., Batusic, M, and Stoeger, B., LABRADOOR: LaTeX-to-Braille-Door, http://www.snv.jussieu.fr/inova/ntevh/labradoor.htm, 1998

Nemeth, A., Teaching Mathematics as a Blind Person, Speech given to the Mathematical Association of America, Orlando, Florida, January, 1996.

Raman, T.V., http://www.cs.cornell.edu/Info/People/raman/aster/aster-toplevel.html, 1994

Sahyun, S., and Gardner, J., Audio and Haptic Access to Math and Science - Audio Graphs, Triangle, MathPlus Toolbox and Tiger Printer," 15th IFIP World Congress, Vienna, 1998.

Sjostrom, C., and Rassmus-Grohn, K., "The sense of touch provides new interaction techniques for disabled people," Technology and Disability, Volume 10, Number 1, IOS Press, 1999

Walsh, P., and Gardner, J., TIGER, A NEW AGE OF TACTILE TEXT AND GRAPHICS, CSUN 2001, www.csun.edu/cod/conf2001/proceedings/0128walsh.html

Weber, Gerhardt, www.rit.edu/~easi/itd/itdv01n3/weber.html,1994

York, B.W. & Karshmer, A.I., "Tools to Support Blind Programmers," appearing in Visual Programming Environments, edited by E.P. Glinert, IEEE Computer Society Press (1991).

back to proceeding