Next generating drawing systems in VR

Mayra Barrera shares her graduate student journey and her exciting research projects on drawing in VR.

What are you working on in your graduate studies?

I explore how people draw in virtual reality to inform the creation of user interfaces that allow everyone to create accurate shapes and precise strokes. I aim for my work to support stroke expressiveness while avoiding the interruption of the user’s creative process.

What are your favourite projects?

Multiplanes is a VR drawing system that supports the flexibility of freehand drawing and the ability to draw accurate shapes in 3D. The project started as a research internship at Adobe Research with Paul Asente, Jingwan Lu and Byungmoon Kim. The philosophy of the system is to make the process of drawing as easy as possible to allow the user to focus on creating new things. This is important, because drawing in 3D creates new challenges that don’t exist in 2D. For example, navigation in the virtual environment imposes higher demands on spatial cognition, and the lack of a surface imposes high demands on the user’s motor skills and depth perception. Multiplanes overcomes these challenges by generating snapping planes and beautification trigger points based on the controller’s pose, and previous and current strokes. The beautification trigger points serve to guide the user to reach specific positions in space and are based on geometrical relationships of previous strokes. The system also beautifies the user’s strokes based on the most probable intended shape while the user is drawing them. In turn, these features allow users to focus less on doing precise hand movements and more on the content they are creating.

Smart3DGuides is a project that was born out of the premise of helping users draw more accurate without interfering with their actions. Therefore, I had to find a way to provide the users with enough visual information upon which they can identify errors in their strokes. Based on this, I designed Smart3DGuides, a set of non-constraining visual guides, to help users improve the shape-likeness and stroke precision of VR freehand sketching, without eliminating the expressiveness of their hand movements. As I explained before, drawing in 3D is harder than in 2D. Smart3DGuides provides users with additional depth cues and orientation indicators to help address these issues. The user interface automatically shows visual guidance inside the virtual environment to help users avoid mistakes when planning a stroke in VR. In turn, this allows users to fix errors before starting a stroke proactively. Smart3DGuides works in synchrony with the user, since it is based on the user’s current view direction, controller’s pose, and previously drawn strokes. I work on this project with my supervisor Wolfgang Stuerzlinger and with Paul Asente, from Adobe Research.

What motivated you to pursue graduate school?

My first encounters with the tech industry were as a software developer in Mexico, sadly most of my work revolved around implementing someone else’s technology. To change that, I decided to go to graduate school because I felt that there I could create new technology. I also wanted to work on topics I found interesting. Right now, as a Ph.D. student, I’m not only doing that, but I am also exploring the forefront of knowledge about how humans use VR.

Why did you choose the School of Interactive Arts & Technology over other programs?

My research has taken me across the world, and the journey started in Australia where I did my master’s degree. During that time, I remember that I was constantly referencing the work of Dr. Wolfgang Stuerzlinger and that sparked my curiosity in his research.

When I was looking for places to do a Ph.D., I wanted to find a space that not only had world-class research but the means to do so. I decided to look for a supervisor who has made breakthroughs in HCI and whose laboratory had a presence in the field. Luckily for me, Wolfgang was looking for new students, and now I get to do research under his supervision.

What advice would you give to those who are deciding if graduate school is right for them?

I think that you need to be clear about your goals. You also need to understand that doing research is a long and stressful journey and that you will face unexpected problems along the way. Grad school is about knowing how and what to prioritize, i.e. prioritizing internships if you want industry, or teaching for academia. Finally, the best advice I have is to talk to your future supervisor and see if you are compatible. Try also talking to their students before joining a lab, as they will be the people you will work with for the next few years.

How did you cope with the transition into grad school?

I decided to start with a master’s degree as an experiment to see if I liked doing research and if I was able to cope with everything it involved (i.e. moving countries, lots of work and stress, etc.). Nonetheless, once I finished it, I knew that becoming a researcher was the path I wanted to follow. Having said that, I consider grad school like any other intense full-time job; I work 40 hours a week, five days week and I take three weeks of holidays per year, as an important part of grad school is knowing when to take a break.

What are your future aspirations?

I’m currently approaching the end of my Ph.D. studies, and my next steps involve looking for postdoc positions in Human-Computer Interaction laboratories and continuing my journey towards industry research.

Down the road, I see myself working in an industry research lab. That has been one of my main goals since I started my Ph.D. and I’ve found that through the tech industry I can make an impact on society by helping create new technologies. In my case, by designing user interfaces that allow everyone to experience drawing in 3D environments. For the future direction of my investigation, I want to continue working on getting a better understanding of the perceptual and cognitive limitations of humans that arise when working in virtual environments to create better user interfaces.

If I think about the future of the user interfaces I designed during my Ph.D. studies, I see them becoming an integral part of VR drawing software. I see my work getting traction from an applied research perspective, but more importantly for allowing a bigger part of the express their ideas via VR drawing.