Carolina Scientific, Fall 2015

An old article I wrote about how VR is showing potential but lacks necessary funding. The state of the industry has significantly changed since then...

Carolina Scientific, Spring 2016

An old article I wrote about Gary Bishop's work on accessible technologies and Maze Day, a day on which kids with various disabilities come to the UNC CS department and test new technology.

Chabra, R., Ilie, A., Rewkowski, N., Cha, Y. W., & Fuchs, H. (2017, March). Optimizing placement of commodity depth cameras for known 3D dynamic scene capture. In Virtual Reality (VR), 2017 IEEE (pp. 157-166). IEEE 

This work proposes and evaluates an algorithm deciding the optimal placement of an array of Kinect sensors to achieve the best possible 3D reconstruction of a dynamic scene

 I worked on the 3D animated bodies for the nurse, doctor, and patient that were used as synthetic validation for the proposed optimization algorithm with Blender and Unity

Wilson, J., Sterling, A., Rewkowski, N., & Lin, M. C. (2017). Glass half full: sound synthesis for fluid–structure coupling using added mass operator. The Visual Computer, 33(6-8), 1039-1048

This work uses physically-based modeling to synthesize the sounds of objects with various amounts and types of liquids, allowing for musical notes to be played with only synthetic audio

I worked on the realtime UE4 implementation and UX design of the demo

website: http://gamma.cs.unc.edu/GlassHalfFull/ 

Rungta, A., Schissler, C., Rewkowski, N., Mehra, R., & Manocha, D. (2018). Diffraction Kernels for Interactive Sound Propagation in Dynamic Environments. IEEE transactions on visualization and computer graphics, 24(4), 1613-1622.

 This work proposes a precomputation algorithm allowing for diffraction to be used in realtime audio scenarios through a technique involving diffraction kernels, which evaluate an audio obstacle from all possible angles, taking advantage of symmetry and other characteristics

I built all 3 non-realtime demos and modified Oculus demos to work with our improved audio engine (UE4), most figures, and video

website: http://gamma.cs.unc.edu/diffractionkernel/ 

Rungta, A., Rewkowski, N., Schissler, C., Robinson, P., Mehra, R., & Manocha, D. (2018, August). Effects of virtual acoustics on target-word identification performance in multi-talker environments. In Proceedings of the 15th ACM Symposium on Applied Perception (p. 16). ACM.

 This work evaluates the reproducibility of the cocktail party effect in synthetic environments and conducts a user study establishing limits of the ability to hear specific sounds in a crowd

I worked on study design and 3D user study environment (Unity), figures, and video

website: http://gamma.cs.unc.edu/ckp/ 

Cha, Y. W., Price, T., Wei, Z., Lu, X., Rewkowski, N., Chabra, R., ... & Ilie, A. (2018). Towards Fully Mobile 3D Face, Body, and Environment Capture Using Only Head-worn Cameras. IEEE transactions on visualization and computer graphics.

This work proposes and implements a headworn camera rig capable of reproducing the wearer’s bodily actions, including appendage movement and facial expressions

I worked on the realtime implementation of the system output as an Alembic animation in UE4

Sterling, A., Rewkowski, N., Klatzky, R. & Lin, M. (2019). Audio-Material Reconstruction for Virtualized Reality Using a Probabilistic Damping Model. IEEE transactions on visualization and computer graphics.

I built the realtime implementation in UE4, including all demos. Also made video

Rungta, A., Rewkowski, N., Klatzky, R. & Lin, M. (2019). P-Reverb: Perceptual Characterization of Early and Late Reflections for Auditory Displays. In Virtual Reality (VR), 2019 IEEE. IEEE.

I worked on in-house reverberation engine integration and realtime system made in Unity (and figures). I also have tools made in Unreal 4 and Blender that get valid sample points used for computing reverb parameters

Bhattacharya, U., Rewkowski, N., Guhan, P., Williams, N. L., Mittal, T., Bera, A., & Manocha, D. (2020). Generating Emotive Gaits for Virtual Agents Using Affect-Based Autoregression. arXiv preprint arXiv:2010.01615. 

• ISMAR 2020

Rigged the characters to work with the generated animations, rendered most of the 3D videos, made the realtime AR implementation with the HoloLens

Bhattacharya, Uttaran, et al. "Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents." arXiv preprint arXiv:2101.11101 (2021).

• IEEE VR 2021

Rigged the characters to work with the generated animations, rendered most of the 3D videos, output synthetic results on rigged characters

Bhattacharya, Uttaran, et al. "Speech2affectivegestures: Synthesizing co-speech gestures with generative adversarial affective expression learning." Proceedings of the 29th ACM International Conference on Multimedia. 2021.

• ACM Multimedia 2021

Rigged the characters to work with the generated animations, rendered most of the 3D videos, output synthetic results on rigged characters

Wilson, Justin, Nicholas Rewkowski, and Ming C. Lin. "Audio-Visual Depth and Material Estimation for Robot Navigation." 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022.

• IROS 2022

Made the realtime demo, helped with data collection, helped with some of the mesh postprocessing, made the SteamAudio-based synthetic data collector

Chen, H., & Fuchs, H. (2017, June). Supporting free walking in a large virtual environment: imperceptible redirected walking with an immersive distractor. In Proceedings of the Computer Graphics International Conference (p. 22). ACM.

Helped with user study and video

Pratapa, S., Krajcevski, P., & Manocha, D. (2017, February). MPTC: video rendering for virtual screens using compressed textures. In Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (p. 14). ACM.

Made video

website: http://gamma.cs.unc.edu/MPTC/