• Goal: Tracking objects in motion usually requires cameras to be synchronized, which most consumer cameras are not capable of, and even research cameras like PointGreys are very expensive or their cheaper alternatives like PiCams are hard to receive images from. This project tries to deal with all of these problems at once by using cheap $7-10 cameras that are "synchronized" with smart threading instead. 2 of them are attached to the dynamic manipulators themselves to deal with occlusion, which requires ViveTrackers as well.

• Current progress: We can track Vive tech in Hololens space and the entire pipeline is complete, with some naive assumptions being made that should eventually be corrected and some interpolation needs to be done. There is also some delay that needs be be dealt with to have better realtime performance

• Technology that I'm using: Unity with C# & Vuforia for marker-tracking, UE4 w/ Blueprint for ViveTrackers & pooling, Hololens/Unity/Vuforia/C# for application.

• Documentation for replicability & source code will be provided after publication.

Short ISMAR paper/poster (more succinct; less technical detail)

Longer, older paper draft that's not as well-written but has more technical detail and pipeline figures

Short class presentation (pending the ISMAR presentation)

• Co-investigators: Hao Jiang, Andrei State, and Andrei Illie

• Goal: We hypothesize that having a 3D view of the surgery area through an AR display such as the Hololens will improve accuracy of movement, since the user has the additional stereoscopic depth cues to aid them in localizing the workspace.

• Current progress: We are able to get a basic laparoscopic training scenario to work in the Hololens, with limitations in tracking and calibration that we are working on. I personally focus on the realtime simulation and user study scenario after the tracking data is received.

• Technology that I'm using: Unity with C# for the realtime implementation. Hololens as the headset. Blender to build all 3d-printed models. OpenCV+Vuforia for tracking (Hao is using OpenCV to track the cube accurately while I use Vuforia for world calibration)

• Attached are pictures of a theoretical portable laparoscopic training apparatus and some videos of our process (more or less ordered by most recent to oldest)