Math & Science Working Symposium May 12 & 13, 1994 Conference Update -- George Kerscher, RFB This update is to let you know about the developments of the conference. There has been a very good response by those initially invited. Many people have informed me about other researchers and developers that should be invited and these invitations have gone out. Some people have not responded as of yet, but we are trying to determine who will be attending. Update Table of Contents Part 1 - Technical Considerations Part 2 - Future Considerations Part 3 - Thoughts as to an Agenda Part 4 - List of confirmed persons attending Part 5 - Conference and Accommodation Confirmation Part 6 - Attachments **** Part 1 Technical Considerations **** I want to set a high, computer science, technical tone for the conference. A traditional conference has presenters, and everybody leaves the meeting aware of significant work in the field; the CSUN conference is a fine example of this activity. The Math & Science Working Symposium will build on the presentations of CSUN, but I assure you that we intend this conference to be a working meeting intended to launch the development of new systems with many groups working together. We want to face the challenge of access to math and science together, capitalising on everybody's strengths. Dr. T.V. Raman's work at Cornell has demonstrated "Audio System for Technical Readings" (AsTeR). The data structures he developed are rich and powerful. He represents math in a n-ary tree. Each node may have up to six additional dimensions, which may be a n-ary tree itself. This system allows the user to have the links spoken interactively. Following these links allows the user to hear the math in ways never experienced before. We all must congratulate Dr. Raman. At the end of this update is an article from Siam and the abstract of Dr. Raman's thesis. In addition, RFB would be pleased to send you a cassette tape demonstrating his work. RFB will make this tape available to you, but there is one major reservation. AsTeR is a completely interactive system. The user controls how and what is to be spoken. Much of the power comes from this interaction. Of course, the cassette recording "freezes" this spoken math and, you must understand, that the interaction is lost. We know the limitations of recordings, but we feel it would be helpful to have been exposed to Raman's work as background to the conference. (Send E-mail to cbfb_gwk@selway.umt.edu or call Vicki at 406/728-7201 to request the two track stereo recording.) We believe that the types of data structures defined in Raman's thesis can enable accurate transformations. Braille in any form, Dotsplus, correct visual representations and spoken math can all be extracted from the tree. A software program can be developed that provides the underlying structure to modules that produce simultaneous outputs. We want this conference to explore ways to extend Raman's work. We want the technical computer science oriented researchers, to contribute to the discussions that are sure to arise. We are looking for expertise necessary to make AsTeR and related systems easily available on diverse computer platforms. Some people have indicated that they want to observe and that their backgrounds will not allow them to participate in the technical discussions. For those lawmakers, teachers, and program directors in the field who want to observe this conference we have made provisions for them to observe Friday afternoon. We also will make the conference proceedings available upon request. **** Part 2 - Future Considerations **** We want to investigate the feasibility of a flexible interactive reading and writing system for math that incorporates braille, correct visual layout and spoken math. Such a system should support new developments such as a tactile refreshable tablet. Investigation in this direction should concern itself with practical matters of electronic file manufacturing, availability, copyright issues, and delivery systems. This direction may serve to bring a wide variety of groups together in a development effort. **** Part 3 - Thoughts as to an Agenda **** Thursday Introductions Several minutes with each researcher explaining their affiliations and brief summary of their work. (3 minutes each) AsTeR Presentation Dr. Raman will explain AsTeR Extending Raman's Work Panel discussion Dr. Rich Cox, Chris Brooks, George Kerscher, Steve Edwards, Dr. Raman. This discussion will focus on: technical considerations, problems, kinds of activity, skills, and try to put dimensions around what is to be done Lunch The Big Picture Round Table All researchers and developers will be asked to share in how their work: overlaps, compliments, extends and ties in with extending access to math and science. Individuals should be prepared to answer detailed questions from the participants. We are very interested to hear about the activities of the European Maths project. Friday Brainstorming Round table What is worth doing? How should it be done? What would I like to do? Next steps! Lunch Observers Those observers now joining the conference at this point will be introduced. Remarks by Ritchie Geisel, President of RFB Summary Description of our conclusion of what needs to be done. Raman and selected others Follow up discussion Organization development, action items. **** Part 4 - List of confirmed persons attending **** Dr. T.V. Raman, DEC Research Analyst Dr. Caryn Navy, Raised Dot Computing Mr. David Holladay, Raised Dot Computing Mr. Joseph Sullivan, Duxbury Systems, Inc. Dr. Norberto Salinas, University of Kansas Prof. John Gardner, Oregon State University Dr. Abraham Nemeth, Professor Emeritus, University of Detroit Dr. James Thatcher, IBM Special Needs Dr. Albert Blank, The College of Staten Island Dr. Michael E. Kress, The College of Staten Island Dr. Karen Luxton Gourgey, Center for the Visually Impaired, Baruch College, City University of New York Dr. William B. Woolf, American Mathematical Society Ms. Barbara Freeze, Canadian National Institute for the Blind Mr. Paul Clarke, Canadian National Institute for the Blind Mr. Venkatesh R. Chari, Technology For Independence, Inc. Mr. John Cookson, National Library Service for the Blind and Physically Handicapped, Library of Congress Dr. William Barry, Oregon State University Dr. Gerhard Weber, University of Stuttgart Mr. Lar Kaufman, Polymedia Services Ms. Lynne M. Schmelz, Harvard University Dr. Charles Halperin-Hamu, InfoDesign Corporation Prof. Norman Coombs, Rochester Institute of Technology RFB Participating Staff Ritchie Geisel, RFB President Christopher Brooks, RFB VP Information Technology John Kelly, RFB VP Library Services Dr. Richard Cox, RFB Technical Advisory Committee Mr. Robert Finger, RFB Technical Advisory Committee Mr. James I. Magid, RFB Technical Advisory Committee George Kerscher, RFB Director R&D (RFB R&D team) Steve Edwards, RFB Research Analyst (RFB R&D team) Bill Robinson, RFB R&D team Linden Clarke, RFB E-Text Mfg. team Sarah Johns, RFB E-Text Mfg. team Rhona Ethe', RFB IT Management team Cindi Krnac, RFB IT support **** Part 5 Conference and Accommodation Confirmation **** Recording for the Blind 20 Roszel Road Princeton, NJ 08540 Conference and Accommodation Confirmation "Math and Science Working Symposium" May 12 & 13, 1994 Guest: Name: Dir.ir. J. J. Engelen Title: Professor of Applied Electronics and Optics Company: Katholieke Universiteit Leuven Street Address: Kardinaal Mercierlaan 94 City, State, Zip: B-3030 Leuven - Heverlee BELGIUM Phone: +(32) 16 22 09 31 Toll Free: Fax: E-mail: fheda02@cc1.kuleuven.ac.be Conference Location: Castles's Room, Recording for the Blind, 20 Roszel Road, Princeton, New Jersey 08540 Presenters are responsible for providing their own handouts. Hotel Information: Novotel Princeton 100 Independence Way Princeton, NJ 08540 Phone: 609-520-1200 Fax: 609-520-0594 Single or Double Room: Preferred Accommodation (Non Smoking, etc.): Room Rate (May 11 & 12 only): $39.00/night Arrival Date: Departure Date: Confirmation #: Vicki Please communicate any changes regarding the above conference information or accommodations to Vicki Norbury. If you have not confirmed your lodging, please notify Vicki Norbury by April 15, 1994. Vicki Lee Norbury, Adm. Asst. Recording for the Blind Research and Product Development 127 N. Higgins Avenue, Suite 202 P. O. Box 7068 Missoula, MT 59807 Phone: 406-728-7201 Fax: 406-728-6331 InterNet: cbfb_gwk@selway.umt.edu **** Part 6 Attachments **** The following is the abstract of Dr. Raman's thesis Topic: Audio System for Technical Readings Time: 13:15 Date: 17 January, 1994 Place: 5130 Upson ------------------------------------------------------------------- The advent of electronic documents makes information available in more than its visual form ---electronic information can now be display-independent. We describe a computing system, AsTeR, that {\em audio formats\/} electronic documents to produce audio documents. AsTeR can speak both literary texts and highly technical documents (presently in La)TeX) that contain complex mathematics. Visual communication is characterized by the eye's ability to actively access parts of a two-dimensional display. The reader is active, while the display is passive. This active-passive role is reversed by the temporal nature of oral communication: information flows actively past a passive listener. This prohibits multiple views ---it is impossible to first obtain a high-level view and then "look" at details. These shortcomings become severe when presenting complex mathematics orally. Audio formatting, which renders information structure in a manner attuned to an auditory display, overcomes these problems. AsTeR is interactive, and the ability to browse information structure and obtain multiple views enables {\em active\ /} listening. --Raman ******** The following article appeared in the March 1994 issue of SIAM News, the monthly news journal of the Society for Applied and Industrial Mathematics, pages 7-16. Reprinted with permission from the publisher. Copyright 1994 by the Society for Industrial and Applied Mathematics, all rights reserved. THESE INNOVATIVE TOOLS HAVE EYES FOR MATHEMATICS By Rena F. Bloom The visually impaired will be finding it easier to approximate the way sighted people read and write equations and other mathematical language, thanks to remarkable tools recently developed by three scientists. Two of the three are visually impaired themselves; their advances are a result of extraordinary ingenuity and persistence born partly out of their own frustration. It took a great deal of effort, for instance, for T.V. Raman to pursue a mathematical education. Born in India, he was partially sighted until the age of 14, enough to learn to read and write, but after that he was dependent on readers, recordings, and Braille. However, neither Braille textbooks nor the standard Braille mathematics codes were available to him, and pre-recorded textbooks were scarce. By 1988, the last year he studied in India, 13 readers were each spending three hours a week recording texts for Raman. Although talking computers (which read text in a synthesized voice) had been developed by that time, he did not have access to them, and when he began to study computing for his master's degree at the Indian Institute of Technology in Bombay, someone had to stand behind him and tell him what was on the visual display. Audio System for Technical Readings (AsTeR) The lack of adaptive equipment in India spurred Raman's decision to leave his homeland in 1989 and join the PhD program in applied mathematics at Cornell University. There he acquired his first guide dog, Aster, and his first talking computer. Both, he says, have enriched his life. Now Raman is poised to enrich the lives of other visually impaired scientists. For his PhD dissertation, he has developed AsTeR, a computing system that converts electronic documents written in TeX or LaTeX into an audio format through a speech synthesizer. One way AsTeR can be used is to produce -- electronically and speedily -- audiotapes comparable to those made by skilled readers of technical material, such as those who read for the Princeton-based Recording for the Blind (RFB). The rigorous control over sound available through the voice synthesizer permits such innovations as the use of a higher pitch to indicate superscripts and a lower pitch for subscripts. In fact, one advantage of AsTeR-generated tapes is "absolute consistency" in the enunciation of mathematical expressions, says Chuck Romine, a mathematician at Oak Ridge National Laboratory and one of the 30 or so at ORNL who are regular RFB volunteers. However, AsTeR's greatest value lies in the interactive features it offers, Raman believes. Because the system is programmed to recognize the markup language of the TeX family, it can comprehend "the logical structure of a document," Raman says, and convert aspects of that structure to an audio format by applying "rendering rules." AsTeR thus enables users to take an active role that is for the most part denied to listeners who are tied to the linearity of an audiotape and whose participation is limited to pausing, rewinding, or forwarding the tape. Like sighted readers who rely on what Raman calls "multiple views" to aid comprehension, AsTeR users can browse headings to obtain an overview of the content of a text, skip from equation to equation, mark a passage to be returned to later, skip passages or sections, call up a prior reference, and choose the level of detail they want. AsTeR's cross-referencing capability is particularly useful for mathematical material. "If a proof such as 'By theorem 2.1 and lemma 3.5 we get equation 8 and hence the result' looks abstruse in print, it sounds meaningless in audio," Raman says. In his system, when a numbered theorem or equation is encountered, AsTeR allows the user the option of labeling it with a name (for example, "Fermat's theorem"). Later, when a numbered cross-reference comes up, AsTeR uses the name instead of the number. In addition, pressing a predetermined key will recall the theorem or equation in full detail. AsTeR also enables users to preview an equation's "top-level" structure. When sighted readers see a complex equation, Raman explains, they may first "view it as an equation with a double summation on the left-hand side and a double integral on the right-hand side, and only then attempt to read the equation in full detail." Actually, most visual mathematical notation represents an attempt to increase the reader's understanding by "grouping subexpressions together in a meaningful manner," he says. To compensate for the relentless linearity of audio renderings, Raman has built into AsTeR a feature that recognizes and labels the top-level structure and groupings in expressions. When this feature is activated, the top-level description of the equation is spoken first, using such grouping labels as "lower constraint 1," before the full details of each group are spoken (Figure 1). While a skilled reader of equations will provide some description when recording texts by saying, for example, "the fraction whose numerator is . . . ," most such readers use a basic left-to-right formula rather than AsTeR's top-down rendering, according to Chuck Romine. Romine is in a good position to make comparisons. To test his system, Raman sent six printed pages of complex mathematics (including the equation in Figure 1) to Recording for the Blind. Romine spent more than an hour recording the material under RFB's careful, closely monitored procedure. Sometime later, he had the chance to review an AsTeR-generated tape of the same material that had taken 15 minutes to produce. He calls the system "an amazing tool." RFB, also impressed, is making Raman's work part of a major effort to improve accessibility to mathematics and the sciences. Making Braille Work for Mathematics Physicist John A. Gardner of Oregon State University is another scientist who remembers what it was like to be sighted. He has also come up with a revolutionary tool, in this case a refinement of the Braille system, to make mathematics more accessible to the visually impaired. When he lost his eyesight about five years ago as a complication of surgery, Gardner searched for the resources to enable him to continue his teaching and National Science Foundation-funded research on high- temperature superconducting materials. He discovered that talking computers were useful for keeping up with research in his field, but he found Braille, even special Braille mathematics codes, complicated and basically unsuited to the mathematical expressions that are the language of science. (In Braille, for instance, the nine digits are the same as the first nine letters of the alphabet and must be distinguished from them by being preceded by a special "number" character.) Not unsurprisingly, there appeared to be few advanced mathematical and science texts in Braille. Determined to keep up with his research and to make the language of science accessible to others who share his disability, Gardner invented Dotsplus, a modified Braille tactile system that, like AsTeR, introduces a spatial format into a heretofore linear method. Dotsplus retains the common visual mathematical symbols, such as plus, minus, summation, and integral signs, enlarging them and printing them as raised images. An expanded Braille dot system (using eight-dot "cells" instead of the standard six-dot ones) is used for letters and numbers, eliminating the need for special symbols for numbers or capitalization. The placement of characters on the page is the same as it would be in print for the sighted, with subscripts below and superscripts above, and numerators of fractions placed above denominators, thus enabling the Braille user to approximate the sighted reader's multidimensional view of equations (see Figure 1, right, page 7). Needed: A Better Tactile Printer Finding a printer that can make raised tactile images other than dots is the biggest stumbling block in making Dotsplus widely available, says Gardner's OSU physicist colleague, William Barry. Braille printers basically work by punching holes through paper and can't reproduce the lines and other graphic images integral to Dotsplus. Currently Gardner and Barry print Dotsplus on a wax jet printer that is no longer manufactured. They hope that commercial companies such as those attending a conference this month on technology for the disabled at California State University, Northridge, will take up the challenge of creating a suitable printer for Dotsplus. The technology that reduces challenges to the visually impaired need not be dazzlingly electronic. For years, those who used manual Braille writers were forced to write from right to left, because the stylus produced its raised dots on the back of the paper. Even worse, according to Raman, who recalls trying to take notes with these devices, was that the user could not "read" what had been written without removing the paper and turning it over. Now, Larry Hawk, an ORNL engineer who is also a Recording for the Blind volunteer, has invented, along with colleague Joe Turner, a new hand-held Braille writer that eliminates these problems (see illustration at left). Still, as Raman, Gardner, and Barry emphasize in a joint paper to be presented at the CSUN conference, one aspect of modern technology has been crucial to their advances. Precise mathematical markup languages like TeX have enabled computers to translate mathematical expressions into the tactile symbols of Dotsplus as well as render them through AsTeR's talking computer. Last month Raman completed his PhD work at Cornell and left for Cambridge, Massachusetts, where he has accepted a research position with Digital Equipment Corporation. Gardner is on sabbatical leave, pursuing his research at the University of Konstanz in Germany. Both hope that their advances will soon open new possibilities for the visually impaired everywhere who are trying to do mathematics. Raman (raman@cs.cornell.edu) has made his thesis available from ftp.cs.cornell.edu in directory /pub/raman as aster-thesis.dvi and aster-thesis.ps. It will also soon be available from RFB. Gardner (gardner@zircon.physics. orst.edu) and Barry (barryw@ucs.orst.edu) can answer questions about Dotsplus. Direct Manual Braille Slates are available from the American Institute for Life Technologies in Malibu, California (fax: (818) 889-9826). Caption for Figure 1: Figure 1. Faa DeBruno's formula for the generalization of the chain rule to the nth derivative, as "labeled" in T.V. Raman's AsTeR system (left) and rendered in the Braille Dotsplus system invented by physicist John Gardner (right). If so instructed, AsTeR would first summarize the equation: "Nth derivative with respect to x of w equals summation over 0 less than or equal to j less than or equal to n summation over lower constraint 1 of jth derviative of u with respect to x times the fraction numerator 1 over denominator 1." Then the details of the labeled parts would be spoken: "Where lower constraint 1 is . . . , numerator 1 is . . . , denominator 1 is . . . ." In the formula as represented in Dotsplus (at twice the size shown here), mathematical symbols as well as dots are raised images, and placement on the page approximates what a sighted reader would see. Caption for photo: The Direct Manual Braille Slate, invented by ORNL's Larry Hawk. The paper is placed between the guide plate and the slate with its bed of pins. A hollow stylus produces dots on the top of the paper, enabling users to write from left to right and to read what they have written. Caption for photo: T.V. Raman with his two "stars": Aster, his guide dog, and the computing equipment that he uses for AsTeR, the system that renders TeX documents in an audio format.