Computing devices play a more significant role in our lives than ever before, so it is more important than ever for them to understand who we are as people on a deep level. But many of today’s user interfaces do not have such an understanding: rather, their designs are based on developers’ intuitions alone. This often leads to mismatches between how useful computing systems promise to be and how useful they are in practice.

In this talk, I will show how analyzing and even modeling human behavior can unlock insights that help resolve such mismatches, resulting in systems that are significantly more useful than what they would otherwise be. I will discuss four results that I have worked on, making it possible for users to (1) interact with objects by looking at them and (2) type on smartphones more quickly and with far fewer errors, and for systems to (3) recognize individual game players from their controller inputs and personalize the game for them, and (4) support people who are blind as they play the same types of racing games as sighted players, with similar speed and sense of control.

Brian A. Smith is a Ph.D. candidate in Computer Science at Columbia University, where he is a member of the Computer Vision Laboratory and the Computer Graphics and User Interfaces Laboratory. His research is in human–computer interaction, accessibility, and game design, and focuses on analyzing human behavior to make computing more useful. He has been awarded the NDSEG Fellowship, an NSF IGERT data science traineeship, and Columbia Engineering’s Extraordinary Teaching Assistant Award. He received his MS and BS in Computer Science from Columbia University.