The Future of Multi-modal Mobile Communication

Skills I used: exploratory research, extreme ideation, focused research, synthesis, extended storytelling, sketching, Photoshop, InDesign

Summary: For a self-directed independent study project under the guidance of Anind Dey and Haakon Faste, I created a storybook that tells the tale of a young couple progressing through their relationship with an imagined mobile device, the “sense phone.” This device uses multi-modal experiences to facilitate effective communication and meaningful personal interactions.

Download the full book by clicking on the panel below [Download size: 1.7 MB]


Exploratory Research on Applied Multi-modality
The initial prompt and goal of my independent study was to explore and design compelling uses for multi-modal feedback on mobile devices. I began my research with a lit review on multi-modality, prototyping, and virtual reality experiences, and then brainstormed ways to utilize multi-modal feedback to enhance the experience of technology.

I identified and interviewed four extreme users to spark even more ideas: a writer who uses a typewriter, an “extreme e-mailer” who sends 200+ e-mails per day, a professional dancer, and a professional massage therapist. These people were chosen because of their unique relationship to sensual experiences, technology, and communication.

Extreme e-mailer.
Extreme e-mailer.
A writer who enjoys using a typewriter.

Focused Research on Close Relationships
Many of the emerging ideas were ways to facilitate communication in close relationships. I decided to take a more focused research approach on young couples.

First, I came up with more ideas by framing them within the context of specific needs in close relationships. I made several “How might we…?” statements and then uncovered new opportunities and ideas.

Ideating based on needs.
Ideas were brainstormed to support the needs of understanding, stability, trust, affection, respect, and excitement.

I then held individual research sessions with participants who were in long-term relationships or married. The sessions consisted of the following methods:

1. Semi-structured interview about communication behavior with a significant other, with an emphasis on expressing emotion through technology.
2. Sorting activity with participant to determine his or her preferred modes of communication (text, phone call, social networking site, etc.)
3. “I like” and “I wish” activity for each mode of communication the participant reported using.
4. Questions about sharing media with significant others, as well as other meaningful media experiences such as having a “song.”
5. Drawing activity which prompted participants to say something to their significant other by drawing anything on a blank piece of paper.

Sorting activity
A research participant sorts his preferred methods of communication.

Telling a Story
The focused research efforts produced a large quantity of design ideas that specifically facilitated communication in close relationships. I decided to tell a cohesive story of one imagined couple that could benefit from the presence of these features throughout their relationship.

I translated the design ideas into a narrative arc by grouping them according to their relevance to various stages of intimacy and seriousness in close relationships. Then, I planned out each panel text and description of the panel visual for each page in the storybook.

Pages of the storybook were sketched with pen on paper, scanned into digital format, painted in PhotoShop, and assembled into a final book in InDesign.