The New Yorker Animated Objects Augmented Reality

Overview

This was a collaboration between Dreamboat, Condé Nast, and The New Yorker. What if you could point your phone at everyday objects around you and they were annotated with a short comedic caption in the style of The New Yorker Cartoons? This AR Experience within The New Yorker App does exactly that! Here’s how it works.

Contribution

I collaborated with Dreamboat to setup up the core functionality of finding annotated image data sets, training a machine learning model that recognized 40 selected everyday objects, and optimizing this model to run on iOS within The New Yorker App. After implementing the training algorithm in a Jupyter Notebook, I migrated the system to cloud servers to reduce training time from days (on local hardware) to minutes (on cloud servers).