Helen PETRIE, Chetz COLWELL, Univ. HERTFORDSHIRE (United Kingdom)
Filip EVENEPOEL Univ. LEUVEN (Belgium)
The World Wide Web (WWW) currently holds great potential for people with disabilities, particularly visual disabilities, but it also poses many problems of access for them. The potential is that the WWW can give direct access to enormous amounts of information, without the need to be mediated by time-consuming and expensive processes (e.g. being transcribed into Braille or read onto audio-tape). However, particularly for people with visual disabilities, the highly visual nature of the WWW poses many problems of access. This paper will discuss the tools available to web authors to assist them in developing web pages which are accessible to everyone. In particular we will focus on the Web Page Accessibility Self-Evaluation Test and an evaluation which we have conducted of that test.
The first type of tool to be considered are guidelines which provide web authors with information about what constitutes accessible pages and how to create them. A number of sets of guidelines relating to web page accessibility have been produced, for example by the Trace Centre [7] in the U.S.A. and the EU-funded HARMONY Project [5]. The World Wide Web Consortium (W3C, see [13]) has now initiated a project concerned with accessibility issues, the Web Accessibility Initiative (WAI, see [9]) which has taken over responsibility for developing a comprehensive set of guidelines for web authors [11] from the Trace Centre. The WAI is also developing a checklist [10] which will encapsulate the main information from the guidelines and allow web authors to check whether their pages meet the accessibility criteria. We are conducting an evaluation of these new guidelines and the checklist which is reported in greater detail elsewhere [3].
The Center for Applied Special Technology (CAST, see [2]) have developed an automatic web page validator, Bobby [1], which checks for both browser compatibility and accessibility problems. Any public web page can be submitted to Bobby, which will very promptly display the page indicating points where any compatibility or accessibility problems exist. Bobby will also rate each page from four stars (very accessible) to one star (not very accessible). Pages which achieve four stars are eligible to display the "Bobby Approved" symbol of accessibility. Clicking on the points indicating problems will display an additional page which will indicate the underlying code which is producing the problem, but unfortunately will not make suggestions on how to improve the situation. The other limitation with Bobby is that it will not always detect accessibility problems. We submitted six pages which clearly had accessibility problems: three received four stars, one received three stars, one two stars and only one received one star. Reasons for why Bobby fails to detect some accessibility problems will be explored in the paper. In conclusion, the Bobby page validator is a useful tool for web authors, but should not be the only method used to check the accessibility of a page.
The final type of tool to be considered are non-automatic tests which web authors must conduct themselves. The Public Service Commission of Canada [6] has developed the Web Page Accessibility Self-Evaluation Test (WPASET, see [12]), a 27 item test which web authors can use to rate their pages. Each question is worth between one and six points, depending on the accessibility issue involved. Some questions require identification of specific HTML features (e.g. Does the page use frames?), others require a knowledge of the capability of different browsers (e.g. avoid browser specific tags), others require the web author to make a global judgment about their own page (e.g. use a consistent design). The last type of question seems particularly problematic, as authors may not be the best people to judge their own pages on somewhat subjective questions. A number of inconsistencies in the scoring of the test were also identified. For example, the question on providing alternative descriptions of graphics for visually disabled users is worth four points, whereas the corresponding question of providing alternative descriptions of audio clips for hearing disabled users is only worth three points. Why are the needs of one user group worth more than the needs of another? It is also not clear how pages should be scored if particular questions are irrelevant (e.g. if they contain no graphics or audio clips). Web pages which receive a total score of more than 80% are deemed to be accessible, although it is not clear why this particular score was chosen.
We asked a group of 21 students of human-computer interaction to use the WPASET to assess the accessibility of three web sites. Although they were not the authors of the web pages involved, they had access to the source code of all the pages, and this was thought to be an interesting test of whether the WPASET could be used easily and reliably. Reliability of the resulting overall scores was found to be low. All 21 students assessed the University of Hertfordshire Department of Psychology site [8] with WPASET scores ranging from 48% to 78% (mean score = 65.9%, standard deviation = 8.3). 17 students assessed the Daily Telegraph web site [4] with an even wider dispersion of scores: 42% to 93% (mean score = 68.5%, standard deviation = 13.7). A number of problems concerning the ease of use of the test, the interpretation of the questions and problems with the reliability of the scores will be discussed. Although the idea of the WPASET is a very good one, a number of improvements in the test might result in an easier to use and more reliable test.
The World Wide Web (WWW) currently holds great potential for people with disabilities, particularly visual disabilities, but it also poses many problems of access for them. The potential is that the WWW can give direct access to enormous amounts of information, without the need to be mediated by time-consuming and expensive processes (for example by being transcribed into Braille or read onto audio-tape). However, particularly for people with visual disabilities, the highly visual nature of the WWW poses many problems of access. Tools which are available to web page authors to assist them in developing pages which are accessible to all web users include: guidelines for web authors on how to produce accessible pages; checklists which allow authors to check whether their pages meet accessibility criteria; automatic validators, programs which check whether pages meet accessibility criteria; and non-automatic tests with which authors themselves evaluate their pages on accessibility criteria. In particular, this paper focuses on the Web Page Accessibility Self-Evaluation Test and reports on an evaluation which we have conducted of that test.
2.1 Guidelines and checklists for web page authors
The first type of tool to be considered are guidelines which provide web page authors with information about what constitutes accessible pages and how to create them. A number of sets of guidelines relating to web page accessibility have been produced, for example by the Trace Centre [6] in the U.S.A. and the EU-funded HARMONY Project [4]. The World Wide Web Consortium (W3C, see [11]) has now initiated a project concerned with accessibility issues, the Web Accessibility Initiative (WAI, see [7]) which has taken over responsibility for developing the Trace Centre guidelines into a comprehensive set of guidelines for web authors [9]. The WAI is also developing a checklist [8] which will encapsulate the main information from the guidelines and allow web page authors to check whether their pages meet accessibility criteria. The first two authors are conducting an evaluation of these new guidelines and the checklist which will reported in greater detail elsewhere [3].
2.2 Bobby: a tool for validating web pages for accessibility
The Centre for Applied Special Technology (CAST, see [2]) has developed an automatic web page validator, Bobby [1], which checks web pages for both browser compatibility and accessibility problems. Any public web page can be submitted to Bobby, which will very promptly display the page, indicating where any compatibility/accessibility problems exist. Bobby will also rate each page from four stars (very accessible) to one star (not very accessible). Pages which achieve four stars are eligible to display the "Bobby Approved" symbol of accessibility. Selecting a particular problem will display an additional page which will indicate the underlying code which is producing the problem, but unfortunately this additional description does not make suggestions for how to improve the situation, which undoubtedly would be very useful to web page authors. Another problem with Bobby is that it will not always detect accessibility problems. We submitted five pages which we knew to have accessibility problems: three received four stars, one received three stars, one two stars and only one received one star (see Table 1). In conclusion, the Bobby page validator is a useful tool for web page authors, but should not be the only method used to check the accessibility of a page, as it is not infallible. (N.B. Since writing this paper, a new version of Bobby will be released. An up-dated evaluation of Bobby will be provided at the conference.)
Table 1: Bobby ratings for a number of web sites with known accessibility problems
Web site |
url (http:// ....) |
Accessibility problem/s of particular interest |
Bobby rating |
| British Broadcasting Corporation | www.bbc.co.uk | frames | 4 stars |
| Cognitive and Psychological Sciences | dasun2.essex.ac.uk | graphical links without alt text | 4 stars |
| Web Site Evaluation | www.soc.staffs.ac.uk | frames and user-input form | 3 stars |
| New Scientist Magazine | www.newscientist.com | use of table tag for page layout | 2 stars |
| PC Magazine | www.zdnet.com/pcmag | frames, images (inc. links) without alt text | 1 star |
2.3 The Web Page Accessibility Self-Evaluation Test (WPASET): a tool for web page authors to evaluate their own pages
The final type of tool to be considered are non-automatic tests, which page authors must conduct themselves. The Public Service Commission of Canada [5] has developed the Web Page Accessibility Self-Evaluation Test (WPASET, see [10]), a 27 item test which authors can use to rate their pages. Each question is worth between 1 and 6 points, depending on the particular aspect of accessibility.
Some questions require identification of specific HTML features (e.g. Does the page use frames? - Q3), others a knowledge of the capability of different browsers (e.g. avoid browser specific tags - Q9), others require the author to make a global judgement about their own page (e.g. use a consistent design - Q1). The last type of question seems particularly problematic, as the author may not be the best person to judge their own pages on questions such as this.
Pages which receive a total score of more than 80% on the WPASET are deemed to be accessible; pages which score between 60 and 79% are deemed to be "fairly accessible"; those which score between 40 and 59% are "partially accessible"; those which score between 20 and 39% require "much improvement"; and those which score less than 20% are "highly inaccessible to many persons". It is not clear why these particular bands were chosen for the scoring.
.
We asked a group of 21 students of human-computer interaction to use the WPASET to assess the accessibility of 3 Web sites, from a selection of 6 possible sites. Although they were not the authors of the pages involved (which is the expectation with the WPASET), they were learning about the Web and its underlying hypertext markup language (HTML) as part of their course and they did have access to the source code of all the pages. We therefore thought that this exercise would constitute an interesting evaluation of whether the WPASET could be used easily and reliably by Web page authors.
For this evaluation, the WPASET was scored such that if students decided that a particular question was not applicable to that page, it was removed entirely from the scoring procedure, so the final percentage score was of the total number of relevant questions. Reliability of the resulting overall scores was found to be low. Table 2 shows the mean WPASET scores and other statistics for the 4 sites which were assessed by 10 or more students. From this table it can be seen that all 4 sites obtained mean WPASET scores in the "fairly accessible" band, but that for each site the range of scores given by individual students was very high.
Table 2: Mean WPASET scores and range of scores for four web sites
Site |
url (http:// ...) |
Mean WPASET score |
Range of scores |
Daily Telegraph |
www.telegraph.co.uk |
68.50% |
42 - 93% |
UH Psychology |
www.phoenix.herts.ac.uk |
65.9 |
48 - 78 |
Hypertext '97 |
journals.ecs.soton.ac.uk |
65.1 |
44 -100 |
PC Magazine |
www.zdnet.com/pcmag |
61.5 |
51 - 71 |
More detailed analysis of the use of the WPASET revealed two different types of problems with its use:
(1) some questions produced a lot of disagreement as to whether they were applicable or not. For example, a question on whether all browsers would be able to read images as links or read the alternate text within the image tag (Q2) produced such disagreement. For the University of Hertfordshire (UH) UH Psychology site, 12 students thought this was not applicable, whereas 9 thought it was applicable. Presumably this confusion arose because students were not confident in their knowledge of what different browsers would do with the particular page. Expecting web page authors to be conversant with all the different browsers, including those for disabled people, in addition to HTML, certainly places a considerable burden on them.
(2) Other questions produced agreement as to whether they were applicable, but disagreement as to the level of score which was appropriate for a particular page. For example, on the question concerning the layout of text-based links (Q19), for the UH Psychology site, all students thought this question was applicable. However, 6 students gave a maximum score of 3, 9 gave a score of 2 and 6 gave a score of 0.
Interestingly, what appeared to be the most subjective question, that on design consistency (Q1) produced a high level of agreement, on both applicability and how to score the pages. It may be that in this instance the fact that the students were not the actual page authors was useful, as it allowed them to make a global judgement of design consistency more from the point of view of real end-users.
Other problems with the usability of the WPASET were encountered by the students. The questions were not arranged in a logical sequence or grouped by topic, which they found frustrating. Wording of the questions was often not clear, and often students wanted more examples of the problems being discussed and the appropriate solutions which might be used.
A number of inconsistencies in the scoring of the test were also identified. For example the question on providing alternative descriptions of graphics for visually disabled users is worth 4 points, whereas the corresponding question of providing alternative descriptions of audio clips for hearing disabled users worth 3 points. Why are the needs of one user group worth more than the needs of another? It is also not clear how pages should be scored if particular questions are irrelevant (e.g. if the pages do not contain graphics or audio clips).
In spite of these shortcomings, the idea of a self-administered test to evaluate one’s own web pages is a sensible one, and an improved version of the WPASET would be a useful component of a set of tools available for web page authors to assist them in creating accessible web pages.
It is vital that the WWW is made as accessible as possible to users with different disabilities. Currently users with visual disabilities are particularly disadvantaged in using the WWW, which is frustrating, given its great potential to access information. However, as we have illustrated, a range of tools to assist web page authors in developing accessible pages exist. It is important that these tools are subject to evaluations and improvements, so that the task of web authors in making accessible pages is as easy as possible.
We would like to thank the 1997 BSc in Cognitive Sciences third year students who participated in the evaluation of the WPASET for their assistance.
(all urls correct on 1 May 1998)
[1] Bobby home page, available at: www.cast.org/bobby/
[2] Center for Applied Special Technology home page, available at: www.cast.org/
[3] Colwell, C. and Petrie, H. (1998). Evaluation of guidelines for designing accessible world wide web pages. Paper at this conference.
[4] Harmony Project home page, available at: www.esat.kuleuvne.ac.be/teo/Resproj/HARMONY/harmony.htm
[5] Public Service Commission of Canada home page, available at: www.psc-cfp.gc.ca
[6] Trace Centre, available at: trace.wisc.edu/
[7] Web Accessibility Initiative home page, available at: http://www.w3.org/WAI/
[8] Web Accessibility Initiative Checklist for Page Authors, available at: http://www.w3.org/WAI/GL/authorcl.htm
[9] Web Accessibility Initiative Page Author Guidelines, available at: http://www.w3.org/WAI/GL/author.htm
[10] Web Page Accessibility Self-Evaluation Test, available at: www.psc-cfp.gc.ca/dmd/access/testver1.htm
[11] World Wide Web home page, available at: http://www.w3.org/
J. Engelen et ali: "COPERNICUS NR 806 Final report", Katholieke Universiteit Leuven, April 1997.
Helen PETRIE, Chetz COLWELL, Univ. HERTFORDSHIRE (United Kingdom)
Filip EVENEPOEL Univ. LEUVEN (Belgium)
Le World Wide Web (WWW) présente actuellement un immense potentiel pour les personnes handicapées, en particulier les handicapés visuels, mais il leur pose également de nombreux problèmes d'accès. Ce potentiel est que le Web peut donner accès à d'énormes quantités d'informations sans avoir à passer par des processus longs et coûteux (par exemple la transcription en braille ou la lecture sur cassette audio). Cependant, et en particulier pour les handicapés visuels, la nature très visuelle du Web pose de nombreux problèmes d'accès. Cet article traitera des outils dont les auteurs de pages Web disposent pour mettre au point des pages Web accessibles à tous. Nous nous intéresserons particulièrement au Test d'Auto-Evaluation de l'Accessibilité des Pages Web et à une évaluation de ce test à laquelle nous avons procédé.
Le premier type d'outil à examiner, ce sont les directives qui expliquent aux auteurs ce que sont des pages accessibles et la manière de les créer. Un certain nombre de directives relatives à l'accessibilité des pages Web ont été réalisées, par exemple par le Trace Centre [7] aux Etats-Unis et par le projet HARMONY financé par l'Union Européenne [5]. Le World Wide Web Consortium (W3C, voir [13]) a maintenant mis en route un projet portant sur les questions d'accessibilité, l'Initiative pour l'Accessibilité du Web (WAI, voir [9]) qui a pris la relève du Trace Centre afin de mettre au point une série de directives détaillées pour les auteurs de pages Web [11]. Le WAI est également en train de mettre au point une liste à cocher [10] qui récapitulera les principaux points des directives et permettra aux auteurs de vérifier si leurs pages répondent aux critères d'accessibilité. Nous procédons à une évaluation de ces nouvelles directives et de la liste à cocher, qui fera l'objet d'un rapport détaillé ailleurs [3].
Le Centre de Technologie Spécialisée Appliquée (CAST, voir [2]) a mis au point un validateur automatique de pages Web, Bobby [1], qui vérifie à la fois la compatibilité avec les navigateurs et l'accessibilité. N'importe quelle page Web publique peut être soumise à Bobby, qui affichera très rapidement la page Web en indiquant les endroits présentant d'éventuels problèmes de compatibilité ou d'accessibilité. Bobby donnera également une note à chaque page, de quatre étoiles (très accessible) à une étoile (pas très accessible). Les pages obtenant quatre étoiles ont droit au label d'accessibilité "Approuvé par Bobby". Cliquer sur les points posant des problèmes fera s'afficher une page supplémentaire indiquant le code sous-jacent qui cause le problème, mais ne proposera malheureusement pas de solutions pour améliorer la situation. L'autre limitation de Bobby est qu'il ne détecte pas toujours les problèmes d'accessibilité. Nous lui avons soumis six pages qui posaient d'évidents problèmes d'accessibilité : trois d'entre elles ont obtenu quatre étoiles, une en a obtenu trois, une autre deux, et une seule n'a obtenu qu'une étoile. Nous examinerons dans notre article les raisons pour lesquelles Bobby ne détecte pas certains problèmes d'accessibilité. Pour conclure, le validateur de pages Bobby est un outil pratique pour les auteurs de pages Web, mais ne devrait pas être la seule méthode employée pour vérifier l'accessibilité d'une page.
Le dernier type d'outil à examiner, ce sont les tests non automatiques auxquels les auteurs de pages Web doivent procéder eux-mêmes. La Commission canadienne du Service Public [6] a mis au point le Test d'Auto-Evaluation de l'Accessibilité des Pages Web (WPASET, voir [12]), un test en 27 points que les auteurs peuvent utiliser pour noter leurs pages Web. Chaque question vaut entre un et six points selon le problème d'accessibilité concerné. Certaines questions nécessitent l'identification de caractéristiques HTML particulières (par exemple, la page utilise-t-elle des bordures ?), d'autres nécessitent une connaissance des capacités des différents navigateurs (par exemple, éviter les étiquettes spécifiques aux navigateurs), d'autres enfin demandent que l'auteur émette un jugement global sur sa propre page Web (par exemple l'utilisation d'une présentation cohérente). Le dernier type de question semble particulièrement problématique, puisque les auteurs ne sont peut-être pas les mieux placés pour juger leurs propres pages sur des questions plutôt subjectives. Nous avons également relevé un certain nombre d'irrégularités dans le système de notation du test. Par exemple, la question sur la fourniture de descriptions complémentaires aux graphiques pour les utilisateurs déficients visuels vaut quatre points, tandis que la question correspondante sur la fourniture d'alternatives descriptives aux clips audio pour les utilisateurs déficients auditifs ne vaut que trois points. Pourquoi les besoins d'un groupe d'utilisateurs valent-ils plus de points que ceux d'un autre groupe ? Il n'est pas clair non plus comment la notation s'appliquerait aux pages non concernées par certaines questions (par exemple celles qui ne contiennent pas de graphiques ou de clips audio). Les pages Web obtenant plus de 80% des points sont considérées comme accessibles, bien que les raisons du choix de ce pourcentage et non d'un autre ne soient pas claires.
Nous avons demandé à un groupe de 21 étudiants spécialisés dans l'interaction homme-ordinateur d'utiliser le WPASET pour évaluer l'accessibilité de trois sites Web. Bien que n'étant pas eux-mêmes les auteurs des pages Web en question, ils avaient accès au code source de toutes les pages, et nous avons pensé que cela serait un moyen intéressant de tester la facilité d'utilisation et la fiabilité du WPASET. La fiabilité des résultats généraux obtenus s'est avérée faible. Les 21 étudiants ont tous attribué au site du département de psychologie de l'Université de Hertfordshire [8] des notes au WPASET allant de 48% à 78% (note moyenne = 65,9%, écart standard = 8,3). 17 étudiants ont évalué le site Web du Daily Telegraph avec une dispersion de la notation encore plus importante : entre 42% et 93% (note moyenne = 68,5%, écart standard = 13,7). Nous traiterons d'un certain nombre de problèmes concernant la facilité d'utilisation du test, l'interprétation des questions et les problèmes de fiabilité des résultats. Bien que l'idée du WPASET soit excellente, un certain nombre d'améliorations permettraient d'obtenir un test plus facile à utiliser et plus fiable.
[Actes]