After publishing Typography Insight for HoloLens and experimenting with type in augmented reality space, I started thinking about bringing my app to the virtual reality world. I have been playing with HTC Vive for some time, but this is my first time creating an app for Vive. I would like to share my experience in bringing the app from HoloLens to Vive.
About Typography Insight for VR
Type is beautiful in 3D space. Typography Insight encourages you to experience, layout and to play with type in 3D virtual reality space. It will help you understand the characteristics of type in VR. The app is composed of three feature areas.
Compose typographic layout in 3D space with various fonts, sizes, and colors. You can use it for designing architectural signage or constructing an experimental typographic structure in space. It supports multiple backgrounds including deep space, a Paris street and grid space. With physics feature, you can play with floating type with zero gravity.
Learn about basic terminology and the detail elements of typefaces.
Explore historically important typefaces in 3D timeline view. From old style Garamond, to modern Helvetica you can scroll through the evolution of type in 3D space.
Frameworks for building VR Apps
To start the project, the first thing I did was research for the tutorials and available frameworks. I knew that I could use Unity to build VR apps but I was looking for something similar to Holographic Academy tutorials and HoloToolkit that helped me build mixed reality experiences on HoloLens. VR Dev School’s tutorial was very helpful to understand the basics of SteamVR and the mechanism for the controllers. For the framework, I have been looking into NewtonVR and VRTK. Both of them had great sample scenes with detailed implementation examples. I chose VRTK as my base framework to build Typography Insight since it covered various topics that I needed such as laser pointer and controller interactions.
Re-thinking interactions with 6DOF controllers
In Type Playground, you can play with type and create typographic compositions in three-dimensional space. Using VRTK’s scripts with Vive’s 6DOF controllers, I was able to interact with text objects directly. Therefore, I didn’t have to use the gaze cursor. With this input change, I made some updates to the menu system.
- Many UI buttons that I had in the HoloLens project were no longer needed. For example, I could remove the buttons for the different manipulation modes (move, rotate-x, rotate y) since I can grab and manipulate text objects directly with the controller.
- Body locked floating menu bar was no longer needed. Instead, I have moved the buttons from this menu bar to the top of the left controller so that users can easily access anytime they need them.
- I moved the edit/duplicate/delete buttons to the top of each text object for faster actions.
Menu buttons before & after
Floating tool bar UI in HoloLens version designed for Gaze input and Air tap Gesture
Tools menu on the left controller in VIVE VR version designed for the laser pointer input and trigger
Type with Physics is Fun!
Since you can grab type with the controllers, they feel much closer to you. You can observe the detailed shape of the type in gigantic scale at close distance. (You can literally ‘eat’ your favorite type!) In addition to increased physicality, Unity’s physics engine is opening up a whole new way of experiencing type. In the zero-gravity environment, you can play with type that floats around and collides with each other. If you don’t like a layout, you can throw them away into space :)
Environment and Floor
In HoloLens, real environment was the background of the app. However, In VR, I had to create my own environment in the virtual world. Skybox is an element in Unity that you can use as an environmental background by assigning 360 photo images or solid colors. For the default experience, I created a 3D grid environment with image texture that aligns well with the topic typography. In addition to this grid-based environment background, I added several photo environments that make the experience feel like augmented reality. You can play with typographic layout in deep space or on the streets of Paris.
Another interesting element in VR is the floor plane. Since Skybox is a huge spherical or cube object that surrounds you, you still feel floating in the space. This could be an uncomfortable experience for users. To help the user feel grounded, I added the floor plane with opaque color. I tried translucent colors with different opacity on this floor plane but being able to see the deep space below my feet felt uncomfortable.
Grabbing Text with Pointer
Manipulating text objects with laser pointer
For easier type manipulation and menu interaction, I added a laser pointer. This allows users to easily grab and layout type at a distance.
Reusing assets and scripts
Even though I had to replace a lot of scripts/APIs, I was still able to reuse many assets and components. For example, for the menu button elements, I reused the prefabs from HoloLens project as-is without modifying them. The script for the 3D timeline scroll layout is another example that I reused the code from my HoloLens project. This made me feel comfortable when I started the project.
Reusing assets and prefabs
As the first version, it is missing many elements that I think are crucial for playing with type. These are some of the components that I would like to work on:
- * Multiple slots for save/load to help people explore various typographic compositions.
- * Reference type ramp to help designers and developers understand the type size and distance.
- * Sharing with community.
- * Gravity level control — make type fly or fall.
It is truly an exciting moment for us designers and developers. I can’t wait to see how people start exploring typography in a three-dimensional canvas!