Shaping the future of touchless gestures through machine learning


UX/UI Designer


Sketch, InVision, Illustator, InDesign 


August - December 2017 (5 months)


Sophie Howe, Audun Bjørnerud Mo, Viktorija Bezsazna, Christopher Kim, Osman Ali Ansari, Afiny Akdemir, Gillian Wu 

Xesto is a machine learning startup that provides developers with a cloud-based device agnostic platform to recognize and record the gestures they want in their applications, as well as bypassing the process of creating hardware compatibility.


During my time at Xesto, I had the privilege of work with an diverse group of people that continuously challenged me and stimulated my growth as designer. Over the course of 5 months, I led the re-design of their marketing site and assisted in the creation of their API portal, Wave.


At its current state, touchless user interface development is very costly, time consuming and tedious. Up to 35% of total development time is allocated to creating and implementing gestural recognition solutions which are tethered to specific hardware. 



There were two main projects I tackled while I was working at Xesto, each of them came with a different set of challenges. For the marketing site, I started by asking myself questions on how I can improve the usability and experience of it. 

  1.  Why are people currently having trouble understanding what Xesto does?
  2. How can I enhance the marketing website so that it provides a clear and concise message so that potential users can understand?

 The biggest challenge I had with creating the Wave API Portal was starting from scratch. It was multi-facetted and more complex then the marketing site, so I needed to ask myself a different set of questions.

  1. How can I develop a strong user experience for the Wave API portal without losing out on some of the important features?
  2. How will a developer go through the portal versus a project manager?

The first step of my research was to interview the CEO and CTO of the company. I wanted to understand what kind of problems they were currently experiencing with their marketing site, as well as the short and long-term goals of the company. Some of the qualitative questions I asked included:

  • What specific areas of your current site do you feel are successful and why? 
  • What are the current challenges users are experiencing with your product?
  • What are your business goals? (i.e. Increased sales, marketing/branding recognition, etc)
  • What are the actions users should take when coming to your site? Describe the user’s flow.

After it was determined that the marketing site lacked clarity due to the copy, I spent a full day with the Xesto team to hash out how we can clarify the messaging for the brand.

Product goals

Short term goals

  • Marketing & brand awareness
  • Launching open (monetized) Beta of Wave AI for VR and desktop applications 

Long term goals

  • Long term goal: Increased users
  • API Portal device agnostic
  •  Collect largest gesture ML database through iPhone X application
  •  Trace AI: a revolutionary new way for machine learning models to self-improve 

04_Xesto_Landing page
Identifying the User

Xesto has 3 target groups, developers, project managers and enterprises. 


Developers who spend long hours working on touchless gesture models. Hobbyists that are interested in touchless user interfaces and enjoy tinkering and playing around with new products.


Developers would use Xesto to create quick interactions of touchless gestures.


The API portal would be used in the working environment at the office or at home.


The users will need a computer and a leap motion. (In the future, it will work on multiple platforms such as the iPhone X).

Site Structure

The site structure of the Xesto API portal is designed to be as simple and straightforward as possible.

API Portal

Low-Fidelity Wireframe

From the site map, I created sketches to figure out the layout and flow of the API Portal.

Gesture Library 01
Gesture Library 02
Train Gesture

API Portal Outcome

One of the biggest challenges of this project was coming up with a platform that was unique in nature. For two and a half months, I helped map out the user flow and iterate the first design sprint of the platform. One thing it is important to note is that this is not the finalized product and the virtual hand models (designed by Facebook) are used for mockup purposes only. 

Project Library

The project library holds all the training sets a user has recorded.

v copy
v copy 4

Working with Xesto had given me the opportunity to apply what I learned from my summer internship at Half Hunter to the marketing site and API Portal. As the only designer on the team, I was often challenged and asked to articulate my design decisions. There was many times where I bounced concepts back and fourth between various members of my team. Working at an early stage startup meant that I was juggling multiple roles at the same time and that everything I did had to be fast pace. Looking back, I would have liked to be able to focus more of my time conducting research and field studies with Xesto’s target audience.

Other Work

VaxUX / UI Design

Re-designing Cetaris.comUX / UI Design

PortrProduct Design

XestoProduct Design

Ethereal MagazineBrand Identity

Boston Children's MuseumBrand Identity