Reading and writing mathematics is fundamentally different than reading and writing text. While Braille is adequate for the representation of text, it is not up to the task of representing mathematics. The two basic reasons for this are:
Text is linear in nature while mathematical equations are two dimensional. What you have been reading in this text is a good example of this problem. In contrast, examine the following relatively simple equation
One will immediately notice that the equation contains a superscript and a fraction - both being two dimensional in nature. The equation could have been written in a linear form, for example:
a = sqrt(((x super 2) - y) / z)
For this relatively simple equation, a linear representation is adequate
for reading to a blind user. But, with any increase in complexity it becomes
apparent that linear representations are no longer useful.
Text can be generally represented in a somewhat limited number of characters which normally include upper and lowercase letters, the 10 digits, various punctuation marks and a small set of special characters. Equations on the other hand can contain all of the normal text characters plus a large number of special characters.
Normal 6-dot Braille can intrinsically represent 64 unique characters. Through
the use of special "escape" sequences Braille can support a much
larger set of characters. But this comes at some cost. This means that the
basic characters that can be represented in Braille can have different meaning
in different contexts. While there is effectively no limit to the number of
characters one can represent in the fashion, the reading and understanding
of these special sequences becomes very involved. The more new characters
one needs, the more escape sequences one requires. So, for example the letter
"a" can have different meanings in different contexts. It could
be an "a," an "A", a "1," and so on. This makes
for difficult reading and/or writing of mathematical equations in Braille.
Now, add to the mix the multi-dimensional nature of all but the simplest equations, and the escape sequences required to represent them. It becomes clear that Braille math representation is very difficult given the character set that can be represented in 6-dot Braille.
In spite of these limitations, numerous Braille mathematical notations have been developed over the years, primarily based on 6-dot Braille. In the United States the Braille math standard is named after it inventor, Dr. Abraham Nemeth. Other countries have their own standards and some countries have more than one. These representations use the tricks described above and are therefore not easy to learn.
Add to these problems the fact that the vast majority of mathematics teachers don't know, and don't care to learn math Braille notations and you can easily see the problems associated with learning math if you are severely visually impaired or blind.
Finally, while text can be recorded by sighted readers in the form of talking books, this technique does not work well with transmitting mathematical equations (Gillan, et al. - 2001). Having a human reader at your side does help, but is an impractical solution on several levels.
Other techniques have been tried to solve these basic problems, and these
approaches will be presented in the remainder of this paper. While not an
exhaustive review of other techniques, we present several examples of methods
reported in the literature.
Most of the work reported in developing techniques to help visually impaired students deal with mathematics fall within the following general categories :
Tactile as in Braille and other raised representations.
Audio aids that read equations to the student with tools to help in the reading process.
Tonal representations of equations and graphs.
Haptic or forced feedback devices that represent shapes of objects and curves.
While more exotic approaches have been postulated and are possible in the
future, we will examine work that falls into the basic categories listed
We have already discussed the use of traditional 6-dot Braille and its limitations. Now we turn to other tactile methods. First and foremost amongst these other tactile methods is 8-dot Braille representation. By using 8 dots, we extend our basic character set from 64 to 256 characters. Additionally, most 8-dot systems attempt to have some sort of graphics associated with the dot pattern (Weber - 1994; Gardner - 2001).
In an effort to alleviate the problem of requiring teachers to learn Braille, there have been projects that allow a teacher to prepare instructional materials in familiar electronic ways and then have their work automatically translated into Braille. Three such projects are, 1) the ASTER project (Raman - 1994), 2) the LABRADOOR project (Miesenberger, et al. - 1998) and 3) the MAVIS project (Karshmer, et al. - 1999). Other markup languages have also been used to accomplish the same end as seen in the MATHS project which used SGML (Edwards - 2001) and the MAVIS project which now uses XML (Karshmer, et al. - 2001), By using standard markup languages, these projects offer teachers powerful tools to create instructional materials and make them usable by the blind. Another feature of many of these projects is their ability to generate refreshable Braille output.
Tactile output is also very helpful for the blind math student in other
ways. For example, sighted students can easily examine the graphical output
of a sine function, but for the blind this is not an easily achieved option.
The ultimate goal here would be the development .of a high resolution refreshable
tactile device that could present images to be touched by the user. There
are several industrial efforts to create such a device that would be inexpensive
enough to be practical, but it hasn't been accomplished yet. In the meantime,
there are other devices that can prepare the equivalent tactile sheet on
paper or paper-like substances, One of the more promising of these devices
is the Tiger Printer (Walsh and Gardner - 2001).
Audio has been one of the most popular and successful output media for the blind student over the years and is employed in a wide variety of computer-based device interfaces. Certainly, Jaws for Windows (Freedom Scientific - 2001) is a necessary tool for any blind computer user. But this tool is primarily designed for the general user interface and is not well suited for more technical data interfaces.
The problems associated with the automated reading of equations have been the subject of several research projects over the years. Professor Abraham Nemeth not only defined the U.S. standard for math Braille representation, but also developed a simple spoken structure for reading equations (Nemeth - 1996). An interesting overview of the problem of reading math (Hayes - 1996) offers an overview of techniques developed to that date. Finally, the MAVIS project carried out a series of psychological experiments to better understand the process and its inherent problems (Gillan et al., - 2001).
Two early projects in the delivery of technical information to the blind student through audio output were ASTER (Raman - 1994) and Talking Emacs (York and Karshmer - 1991). ASTER reads complex equations to the listener with special sonification to highlight parts of the equation while the Talking Emacs project reads C code to the user providing descriptive information about the code in addition to the basic program material. Given that the project was based on the Emacs editor work was planned to enhance the editor's base Lisp code to also read equations.
More sophisticated equation reading were key to both the MATHS and MAVIS
projects. Additional features such as the use of prosody and highly flexible
equation browsers to allow the user to navigate through complex equations.
The MATHS system also uses non-speech sounds to give an overview of the
equation. MAVIS (see Figure 2), on the other hand uses a hierarchical structure
to group sub sections of an equation into meaningful "chunks,"
for browsing. Both MAVIS and MATHS also offer refreshable Braille output
and specialized video output for low vision users.
Reading equations is not the whole problem associated with mathematics and blind students. The output of a function, for example, can also be a multi-dimensional structure. These forms of mathematical structure are even more difficult to describe in words than an equation.
Sonification of graphs has been the subject of several interesting research projects over the years (Mansur - 1975; Kennel - 1996). In theses systems, musical tones are used to represent the shape of graphical object. Both two- and three-dimensional objects have been sonified with a these techniques.
The use of strictly tonal representations have had limited success especially
when used for complex graphs. Hearing alone seems to lack the information
carrying capability to describe complex objects. While not a complete solution,
sonification can be used with other non-visual output media to describe
The development of high resolution refreshable dot grids would be the best way to represent both text and non-text data. Unfortunately, such devices are still way beyond an affordable price. Other devices like the Tiger printer can produce relatively inexpensive raised dot paper output, but they are not dynamic in nature once they are embossed.
Another solution to this problem can be found in haptic or forced feedback
devices. Such devices can work in either two- or three-dimensional space,
and under computer control allow the user the "feel" shapes (Yu,
et al. - 2000; Kennedy, et al., 1992; Sjostrom - 1999). Again, high-quality
haptic devices are reasonably expensive, but new less expensive devices
are now coming on the market. Prices can range from US$ 6,000 down to US$
100. Naturally, the less expensive devices don't offer the resolution of
their more expensive counterparts.
While all of the approaches to delivering math education to blind students have a great deal of merit, the truly successful ones offer a number of interface techniques. In the domain of reading equations, for example, the MATHS and MAVIS projects used multiple output media to accomplish their tasks. These included speech, sound, Braille and structured navigational techniques. In the case of MAVIS research from math reading experiments laid the groundwork for less ambiguous equation reading and the ability to speak equations into the system by the user.
Another example of integrated approaches is seen in the combination of haptic and sonic devices for describing graphical output (Grabowski and Barner - 1998; Sahyun and Gardner - 1998 In these projects, forced feedback devices are coupled with sonifcation techniques to deliver a high quality rendering of the graphical output. An interesting project from the University of Bologna in Italy includes both haptic and sonic output to describe objects, but also incorporates vibrators to represent colors in the form of a tactile RGB display (Bonivento - 1999).
Finally, a new project at the University of South Florida, with partners
in New Mexico, Texas and Austria, is integrating several input and output
media to build a tool to translate a variety of national math Braille representations
(see Figure 3). The tool will allow blind mathematicians in different countries
to use tools developed in other projects to exchange mathematical works
with blind colleagues in other countries. Here the math browser developed
in the MAVIS project is integrated into a tool that translates different
math Braille notations through standard markup languages to other national
math Braille notations.
Making mathematics accessible to the blind is a challenging and difficult process. The computer and its range of output devices has become the foundation of numerous projects that have brought this goal closer to a reality. With I/O devices such as high-quality speech, musical tones, refreshable Braille, haptic feedback and high reliability speech input, new and effective tools will soon be on the market. Other research into direct neural connectivity, will in the future, make the picture even brighter.
Quelques-uns des travaux décrits dans ce papier ont reçu des fonds de la Fondation Nationale des Sciences des Etats-Unis. et du Département de l'Education, de l'Institut National de la Recherche sur les handicaps et la Réadaptation. ..
Some of the work reported in this paper was funded under grants from the U.S. National Science Foundation (HRD 9800209 & HRD 9906130) and the U.S. Department of Education, National Institute of Disabilities and Rehabilitation Research (H133G010046).
Bonivento, C., Virtual Reality for the Visually Impaired: VIDET, www.lar.deis.unibo.it/activities/videt
Edwards, A.,D., www.cs.york.ac.uk/maths
Freedom Scientific, Inc., www.freedomscientific.com
Gardner, John, http://dots.physics.orst.edu/dotsplus.html, 2001
Gillan, D., Pennington, E., Pazuchanics, S., Karshmer, A., and Pontelli, E., "Reading Complex Structures," The Proceedings of the 45th Human Factors and Ergonomics Conference, Minneapolis/St. Paul, October, 2001.
Grabowski, N.A., and Barner, K., "Data Visualization Methods fro the Blind Using Force Feedback and Sonification," SPIE Conference on Telemanipulator and Telepresence Technologies, Boston, 1998
Hayes, Brian, "Speaking Mathematics," The American Scientist, March-April, 1996
Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C., Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999
Karshmer, A.I., Gupta, G., Pontelli, E., Miesenberger, K., and H-F Guo, "The Development of a Tool to Enhance Communications Between Blind and Sighted Mathematicians, Students and Teachers: A Global Translation Appliance," Proceedings of the HCI Conference, New Orleans, 2001
Kennel, A.R., "AudioGraf: A Diagram Reader for the Blind," ASSETS 96 Conference, 1996
Mansur, D.,L., "Graphs in Sound: A Numerical Data Analysis Method for the Blind,," Masters Thesis, University of California at Davis, 1975
Miesenberger, K., Batusic, M, and Stoeger, B., LABRADOOR: LaTeX-to-Braille-Door, http://www.snv.jussieu.fr/inova/ntevh/labradoor.htm, 1998
Nemeth, A., Teaching Mathematics as a Blind Person, Speech given to the Mathematical Association of America, Orlando, Florida, January, 1996.
Raman, T.V., http://www.cs.cornell.edu/Info/People/raman/aster/aster-toplevel.html, 1994
Sahyun, S., and Gardner, J., Audio and Haptic Access to Math and Science - Audio Graphs, Triangle, MathPlus Toolbox and Tiger Printer," 15th IFIP World Congress, Vienna, 1998.
Sjostrom, C., and Rassmus-Grohn, K., "The sense of touch provides new interaction techniques for disabled people," Technology and Disability, Volume 10, Number 1, IOS Press, 1999
Walsh, P., and Gardner, J., TIGER, A NEW AGE OF TACTILE TEXT AND GRAPHICS, CSUN 2001, www.csun.edu/cod/conf2001/proceedings/0128walsh.html
Weber, Gerhardt, www.rit.edu/~easi/itd/itdv01n3/weber.html,1994
York, B.W. & Karshmer, A.I., "Tools to Support Blind Programmers,"
appearing in Visual Programming Environments, edited by E.P. Glinert, IEEE
Computer Society Press (1991).
back to proceeding