Projects

Below are a few CS-related projects I work/worked on that I think are interesting.

NounTown: VR Language Learning

NounTown is a Virtual Reality (VR) game aimed at foreign language acquisition. Research shows that VR can benefit language learning by leveraging immersion and embodied cognition. NounTown immerses a user in an interactive town that comes to life as you speak, listen, read, and interact with objects in a target foreign language.

A Steam Early Access release will be launched in November or December 2021. Currently NounTown is in closed beta for our KickStarter backers.

More information can be found at Noun.town.

Robotics with Virtual Reality

While autonomous robots are a goal in robotics, currently there are many applications that require robots to be tele-operated. When tele-operating a robot, VR can enable a deeper understanding of the robot’s environment by immersing the user in the robot’s environment. VR can allow the user camera’s pose to change in the virtual environment to better inspect any part of the environment. VR can be especially useful for interpreting the scale of the robot’s environment.

VR can also be useful in the human-robot interaction domain, where information about a robot’s task and/or intention should be conveyed to the human.

I created a basic prototype of combining robotics and VR by visualizing the point clouds from a mobile robot’s RGB-D camera.

ROS# repository: https://github.com/siemens/ros-sharp
Shader credit: https://github.com/UM-ARM-Lab/unity_victor_teleop/blob/master/Assets/Kinect/DepthGeometryPointSprites.shader

Axis Football

Axis Football is an indie simulation American football game that has a yearly ship cycle on Steam.

I did not create Axis Football, but I worked on Axis Football for about 1 year during the 2017 and 2018 release cycles. I started as a tester and moved to a developer role early on.

The development team had 4 programmers when I worked on the game. Due to the small development team, I had to learn the codebase quickly and my work was not limited to one topic. I spent the most time working on:

  • Player AI (avoiding defenders, reaching pass position, etc.)
  • Pop-up stats GUI
  • Music management

Latest release trailer:

2017 Release: https://store.steampowered.com/app/554750/Axis_Football_2017/

2018 Release: https://store.steampowered.com/app/932980/Axis_Football_2018/

Kanji Practice with Virtual Reality

I tried to learn Japanese for a while, and I found the rote repetition of learning Kanji to be very dry so I tried to think of ways to improve this part of the learning process. I also wanted to learn some VR development so I thought of two ways that VR could improve this process:

  1. Immersing oneself in a virtual environment that mimics where you might speak the language in real life, e.g., Tokyo. I used a Unity asset (link below) to place the user inside a ward of Tokyo

  2. Incorporating manipulation and haptic feedback into the flash card process. Giving Kanji symbols presence in the environment and allowing the user to manipulate them can bring in more avenues to form memories.

I made this small project to be a prototype of this idea. I used Blender to create the 3D models. My headset was an Oculus Rift S, and I used the Unity XR Interaction Toolkit as a VR software framework.

City asset: https://assetstore.unity.com/packages/3d/environments/urban/japanese-otaku-city-20359