Hi, please

“Empathy, not sympathy”: DIY, custom-made assistive technology.

Early 2009, NY based artist and programmer Zach Lieberman and a group formed by members of Free Art and Technology (FAT), OpenFrameworks and the Graffiti Research Lab were contacted to help develop drawing software. The hard part? The goal was helping Tony Quan (aka Tempt1), L.A. based graffiti artist, to draw again after 5 years of paralysis caused by ALS.

The team worked on a project based on an eye-tracking system, an adaptive technology device for measuring eye positions and eye movement; which allows people with limited or no bodily movement to use their eyes as controllers instead of the hands. They then developed a drawing program to be used with the eye-tracking system just for Tony’s needs, and the EyeWriter was born.

TemptOne wearingh the EyeWriter --- a tag from the 90s --- his first EyeTag

How it works

What Lieberman’s software does is interpret the movement of the eye’s pupil as the cursor on the screen of a program for drawing. The cursor moves as the pupil gazes over the screen, guiding it over the buttons and commands to select tools and draw. The movements are recorded by a single camera mounted on a pair of glasses, with infrared LEDs that illuminate the eye and create a dark pupil effect. The software reads the image from the camera and interprets the pupil as a black dot positioned on a grid, which can move as a mouse cursor that’s been guided with the eye instead of with the hand.

This program allows you to draw, manipulate and style a tag designed on the screen. Instead of point and click, it uses a time-based interface so that the “click” effect is triggered by focusing the eye on a position for a few seconds. This way the user selects/deselects commands and tools, and initiates/finishes the traces on the grid.

The program guides the user through the process of creating a tag, taking Tag projected on Kyoto's City Hallhim step by step through tracing the letters, their size, stroke, shadowing, coloring, and special effects. The final tags are saved into an FTP to be projected on a surface; only the tag isn’t just a static, finalized image being cast on the wall. Instead, the software repeats the process of how the tag was drawn, giving the effect of it being “live painted” on the wall.


Getting a lot of attention

TemptOne's electronic tag

Tony has been able to do some amazing tag work on the EyeWriter. The designs are very complex and beautiful, and it is getting a lot of attention within the graffiti community beyond LA and the US. Since his first EyeTag on August 2009, his work is being uploaded directly from his Eyewriter to here. The project has been presented at the BLK River Festival in Vienna, CREAM International Festival for Arts and Media in Yokohama, Japan; the Nuart festival in Norway, was completely re-fabricated at the Bombay TechFest 2009, and it just won the interactive category at Design Museum “designs of the year” awards.

DIY assistive technology.

The Eyewriter links two very different types of new media: art, and assistive technology (AT)– technology used by individuals with disabilities in order to perform functions that might otherwise be difficult or impossible, i.e. a wheelchair, a hearing aid, a screen reader or an artificial lung. In general, AT is very expensive and is linked to a costly process that includes highly specialized physicians, hospitalization and health insurance.

The Eyewriter offers an affordable alternative of DIY hardware and software that is open source. The price difference is huge: regular eyetracking systems go for no less than $7,000, like the Arrington Research Monocular Nystamus Laptop System, a device that consists of almost the same elements as the EW and that costs $7,998 . The Eyewriter can be built for less than $50, depending on the quality of the material for the headset and the place where you buy the stuff.

Commercial tracker: $7,988 vs. EyeWriter: less than $50

Who is working at it right now?

Zach's class at Parsons working on their EW

The Ewewriter is an ongoing project under development. Zach’s students at Parson’s Collaboration Studios (collab.eyewriter.org) are working on it for the whole term, bringing different ideas and upgrades.

Just recently, one of the class members managed to control the computer’s cursor on the screen with the EW. This means a lot since it brings the EW a step forward to being useful for existing apps beyond the drawing program.

The main question: can it influence the field of assistive technology?

The answer is an absolute yes. The Eyewriter is gaining attention of AT developers, sellers and users alike. A lot of networking is going on around the project:

  • The developing team at eyewriter.org has been contacted by several hundreds of interested people.
  • Most of them are people that would like to use it for a loved one who is suffering from ASL or other kind of paralysis.
  • They have also been contacted by several ASL foundations.
  • There are a lot of people who would like to cooperate with the project, either by jumping in the design and programming, or with economic support.
  • The Veteran’s Authority also contacted Zach interested in the project, and he’ll be meeting with them next April in DC.
  • Mark Surabian, AT expert who has the only internet café for disabled people in NYC also contacted them and visited the class at Parsons.They’re also meeting with the people at the “Open Prosthetics Project” at Duke

    Robotagger uses the same language

    University.

  • The “Graffiti Markup Language” that was created for the Eyewriter has now been used to develop the Robotagger, a robotic arm that can reproduce tags with marker directly on a surface.

What are the EyeWriter’s limitations?

“I want an eyetracker that works just like a mouse, or at least like my headmouse – able to move freely between any programs, navigating buttons, text, sliders, keyframe rubberbanding, continuous controller data, all WITHOUT modifying the applications. I love my tools and I’m going to miss them, no matter what…”

-NuJack, comic and multimedia artist.

The project is receiving a lot of feedback: through TemptOne’s use, the student’s work, and the suggestions from outside people that have contacted Zach after trying to develop their own.

-The main challenge for the EyeWriter is that, until now, you can’t get it to control other software; it can only be used with the drawing program that was designed for. This means that it can’t substitute a headmouse, –which is what NuJack uses for now, since he has full movement of his head– and that can navigate you through any kind of application. Using the eyes for complex programs (such as Photoshop) is almost impossible—the EW needs larger buttons in order to adapt to the eye’s jittery movements, never as precise as the hand’s.

-There are a lot of people interested in the project, but as Zack Lieberman explained, it is very hard to manage a community that is very diverse in what it has to offer, the level of expertise and is also geographically disperse.

-Even though the software can be understood by a tech expert, it needs to be made even easier and more user friendly, not as intimidating.

What comes next?

-Improve both the hardware and software. The headset is designed for Tony, who can’t move his head. The next step is to adapt it so that people who can move their heads can also use it. This means finding a way to calibrate both the pupil and the head movement.

-Actively work with a ALS patient in NYC. That will help the development since Tony is in LA and it makes it so much harder to get immediate feedback.

-DIY Eyewriter kits—once the device is substantially improved, the developers would like to put together kits that contain all the necessary materials to build the EW. That would allow for a much lower unit price since the materials would be bought in bulk and they would be of proven quality.

Conclusions

“When Mark Surabian visited the class, he said something I personally found very touching, that the key to assistive tech is “empathy not sympathy”

–Zach Lieberman

That is the main challenge with the EyeWriter: to truly understand what the patient needs in order to develop assistive technology that effectively covers the necessity. It has been very successful for TemptOne’s tagging needs because it was designed exactly for him. As NuJack puts it, “when they went to make TEMPT1′s graffiti rig, they were building it from the ground up…that’s why it works!” Zach agrees with that: every case of paralysis is different and every patient has different communication needs.

The long-term goal is to create a professional/social network of software developers, hardware hackers and ALS patients from around the world who are using local materials and open source research to creatively connect and develop eyetracking systems. The idea is to make the EW code so accessible and the hardware so easy to build that each person can adapt it to their patient’s needs.

“Without the proper team in place i fear endless frustration. it may well be that in a year’s time i will be corresponding with you using a (limited) eyetracker, reduced to relating solely through text, my multimedia empire nothing but a fond memory.”

-NuJack

Zach demonstrates the EW in his Brooklyn office

Creative Commons License
“Empathy, not sympathy”: DIY, custom-made assistive technology. by Jimena Lara is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Based on a work at cultureandcommunication.org.

Similar Posts:

6 Comments

  1. mushon 08:27, Mar 9th, 10

    Great post!
    It is exciting to see how art and assistive technology, two very different practices work together in this context. Suspicion is not misplaced in questioning for how long would artists be committed to this mostly unglorified, heavy, complex, and insurance driven field of AT. I think it is fair to believe the art world will not be as excited about the 3rd 4th or 27th Eyewriters while for the AT world it will only start to shine when it is beyond the highly individualized hardware/software proof of concept that is this version 1.

    That’s where open source kicks in. I think open sourcing this project is the key for these sorts of cross-pollination. I don’t think Zach should devote his life to the EyeWriter even if that’s what is required for a tool like this to reach its full AT potential. Zach is an artist, and as an artist he is a serial prototyper, that’s what he does best and that’s what he should continue doing. Zach is also great about passing what he knows on, so with good documentation of hardware and software, there is no reason why this tool may not be developed further by others who have the capacity and interest to develop it for years to the point it can reach its full potential.

    The burden for artists to document their work and to make their tools accessible is new, and often does not directly align with the wider culture of art making, which tends to be more proprietary and to alleviate the individual, the master, the genius over the group, the hobbyists, the students. A new generation of new media artists that are closely related to the dynamics and work ethics of free and open source software are changing this culture. For them documentation, remixing and art as a toolkit, as a prototype that can and should be reworked is the real thrill. I believe this new approach is making art more relevant than ever. It makes art not only more approachable to a wider audience, it actually extends its power as an agent of change in the world. A power that art often claim to have but only rarely truly posses.

  2. we have the technology 14:16, Mar 9th, 10

    “empathy not sympathy” – i like

    complete hardware guide over at openEyes, basically the same thing. http://thirtysixthspan.com/openEyes

    openEyes those go a step fwd by adding a 2nd cam for AR and traditional eye tracking usage.
    also they do some funky stuff to their sensor (the camera manufacture later took their idea and now sell cameras with “detachable” sensor for flexible mounting)

    but their software is matlab based and the project seems abandoned…

  3. Jimena 23:16, Mar 9th, 10

    Welcome! thanks for the comment.

    I hadn’t heard about openEyes and it does look pretty similar. It is a bit more expensive, though, the 2 cameras do add up… although if webcams can be used the price could be lowered down quite a bit.

    Correct me if I’m wrong, but I think the main difference is in the software: The EyeWriter’s software is a drawing app, and by what I could tell from the openEyes project, the software is intended for basic gaze tracking, but it would still need an additional app for word processing, drawing, browsing, etc…

    I like the hardware, the flexible mounting seems confortable and adaptable. Also, it looks pretty sturdy… The EyeWriter was designed for an ALS patient without head movement; i’m not sure that openEyes was intended for a paralysis patient. Do you know?

    Thanks for your comment!

  4. we have the technology 01:27, Mar 10th, 10

    eye gazing\gaze tracking\eye watching(?!) is used for many things.
    openeyes uses two cameras to to embed the line of sight on reality. but the other cam can be ditched.
    as for software – you can use wikipedia.org/wiki/Dasher
    for wordtyping (word choosing?). and a mouse emulator is a fairly straightFwd thing to do, that’s all a basic user needs.

    when i was younger i worked as a vid editor to a retired Doctor with eye cataract and a passion for video art. it always pissed me of the prices he had to pay for all kinds of contraptions the medical insurance didn’t cover and made his life easier (or bearable).

    there is not reason this stuff should cost 7000$… open source it

  5. Ryan 13:26, Mar 10th, 10

    This was a wonderful and informative post, not to mention the topic itself. As you know, this resonated with me from my own personal experience in rehabilitative therapy with spinal cord injuries and stroke patients and seeing how artistic expression through groundbreaking AT. I thought the even more interesting point was how a product like this was open sourcing the hardware and software. I wonder how many artists or more so rehab centers would be interested in this type of AT for quadriplegic patients or even stroke patients.

    Awesome job again.

  6. nadine 09:31, Apr 23rd, 10

    Have a look at this article:
    New way to guide a car: With your eyes, not hands
    http://hosted.ap.org/dynamic/stories/E/EU_GERMANY_EYE_CAR?SITE=IADES&SECTION=HOME&TEMPLATE=DEFAULT

    What I ask myself: surely, this company will patent their eye mouvement tracking technology. Could they- in theory- sue afterwards the open license of the EyeWriter? Like Apple sued HTC (Android= open software)?