vi-describe

Human connection can make art accessible.

VI-Describe drew inspiration from an art class experience, where one of our teammates observed a classmate asking friends to describe a piece of art. The diverse interpretations highlighted the unique, personal responses art evokes, something often lost in traditional alt-text descriptions or audio descriptors. This observation formed the crux of our mission: how might we bring the emotional and interpretive aspects of art to visually impaired individuals?

Team: Ian Ng, Ting Lin, Manh Dao, Sean Mori

Try the demo on Github. See the full project devpost.

Date

February 2023

Duration

36 hours

Type

Design, Development

Achievement

Treehacks 2023 Accessibility Track Entry


Brainstorming

To address the design challenge of communicating human responses to art we brainstormed features using guiding questions that included:

  • How do we elicit diverse responses to art?

  • How can we present those responses in a meaningful way to someone who cannot perceive the artwork visually?

We drew inspiration from apps like Be My Eyes while reviewing accessibility guidelines. We were motivated by the sentiment expressed by one user:

“I don’t just want to know what’s in the image. I want to understand how it makes someone feel.”

Development

We began with sketches and wireframes on Figma, focusing on:

  • A simple submission flow: Users could upload an image and request interpretations in just a few clicks.

  • Community-driven responses: Volunteers could submit their thoughts through a structured form, prompting creative and emotional responses.

We moved quickly into the high-fidelity phase due to being under time constraints during the hackathon.

As both a designer and developer, I contributed to both the visual design and technical implementation of vi-describe:

  • Frontend: I styled and implemented core components like the gallery view and response submission pages using React Native.

  • Backend: I integrated the app with Supabase, handling data storage for images and user-submitted responses.

Collaboration in a high-velocity, small-team environment posed challenges, particularly with code synchronization. For example. The info.plist file, which was ignored by default in Git, was accidentally deleted and reinstated multiple times. These issues required frequent branch reconciliations, which slowed development but taught us valuable lessons in communication and version control.

Accessibility and Ethics

From the outset, we prioritized accessibility in our design process, adhering to Web Content Accessibility Guidelines (WCAG) 2.0. Key features included:

  • Simple Gestures: Navigation relied on taps and clicks, avoiding complex gestures like dragging or multi-touch.

  • Touch Spacing: Buttons were spaced to minimize errors for users with motor impairments.

  • Crowdsourced Descriptions: Every image was paired with text descriptions reflecting the emotional and interpretive responses of the community.

However, the app’s reliance on text-based responses revealed a fundamental flaw: we neglected to prioritiSe audio accessibility. Moving forward, we would incorporate features like:

  • Text-to-Speech: Allowing users to hear responses instead of reading them.

  • Voice Submissions: Enabling volunteers to describe artworks verbally.

This shift would align the app more closely with the needs of visually impaired users, ensuring true accessibility.

Team Reflections

One of the most valuable lessons we learned as a team was the importance of deep user research. While we were proud of the app’s design and functionality, the gap between our intentions and the real needs of visually impaired users highlighted the need for iterative testing and collaboration with the target audience.

While designing vi-describe, our solution contained an inherent limitation: how can a visually impaired person access these human responses to visual art if they are presented visually, on a screen? Unlike apps like Be My Eyes, which provide real-time audio assistance, our approach relied on text and visuals to communicate the responses—a mismatch which was crucial to use of the app, but which we did not think to prioritise in our 36 hours.

“I think this is cool, but it depends on how much control I have. It would be better if I could hear the responses myself.”