AIF Research Spotlight I Generative Landmarks

Director of Research at AI Foundation, Gaurav Bharaj, and vision research engineer, David Ferman, recently published a new poster for Eurographics 2021 on how their unique approach of generating landmarks can improve overall tracking and consistency. From this latest research, it will help in furthering the creation of future digital humans.

Director of Research at AI Foundation, Gaurav Bharaj, and vision research engineer, David Ferman, recently published a new poster for Eurographics 2021 on how their unique approach of generating landmarks can improve overall tracking and consistency. From this latest research, it will help in furthering the creation of future digital humans.

We propose a general purpose approach to detect landmarks with improved temporal consistency, and personalization. Most sparse landmark detection methods rely on laborious, manually labeled landmarks, where inconsistency in annotations over a temporal volume leads to sub-optimal landmark learning. Further, high-quality landmarks with personalization is often hard to achieve. We pose landmark detection as an image translation problem. We capture two sets of unpaired marked (with paint) and unmarked videos. We then use a generative adversarial network and cyclic consistency to predict deformations of landmark templates that simulate markers on unmarked images until these images are indistinguishable from ground-truth marked images. Our novel method does not rely on manually labeled priors, is temporally consistent, and image class agnostic – face, and hand landmarks detection examples are shown.

Ref: Generative Landmarks.  Eurographics 2021 – Posters (2021) | DOI: 10.2312/egp.20211029