Research

Research Statement

Click here to view my dissertation

Geometric Signal Processing (GSP)

I am working in the nascent area of "geometric signal processing" (GSP), which is a fusion of digital signal processing (DSP) with techniques from applied geometry and topology. Inspired by the dynamical systems community, I turn time series into shapes via sliding window embeddings, which I refer to as "time-ordered point clouds" (TOPCs). This framework has traditionally been used on a single 1D observation function for deterministic systems, but I generalize the sliding window technique so that it not only applies to multivariate data (e.g. videos), but that it also applies to data which is not stationary (e.g. music).

The geometry of time-ordered point clouds can be quite informative. For periodic signals, the point clouds fill out topological loops, which, depending on harmonic content, reside on various high dimensional tori. For quasiperiodic signals, the point clouds are dense on a torus. I use modern tools from topological data analysis (TDA) to quantify degrees of periodicity and quasiperiodicity by looking at these shapes, and I show that this can be used to detect anomalies in videos of vibrating vocal cords. In the case of videos, this has the advantage of substantially reducing the amount of preprocessing, as no motion tracking is needed, and the technique operates on raw pixels. This is also one of the first known uses of persistent $H_2$ in a high dimensional setting. The video below shows an example of how a sliding window video of vocal folds undergoing biphonation fills out a torus:

Periodic processes represent only a sliver of possible dynamics, and I also show that sequences of arbitrary normalized sliding window point clouds are approximately isometric betIen "cover songs," or different versions of the same song, possibly with radically different spectral content. Surprisingly, in this application, an incredibly simple geometric descriptor based on self-similarity matrices performs the best, and it also enables us to use MFCC features for this task, which was previously thought not to be possible due to significant timbral differences that can exist betIen versions. When combined with traditional pitch-based features using similarity metric fusion, I obtain state of the art results on automatic cover song identification. The video below explains this in more detail:

In addition to being used as a geometric descriptor, self-similarity matrices provide a unifying description of phenomena in time-ordered point clouds throughout my work, and I use them to illustrate properties such as recurrence, mirror symmetry in time, and harmonics in periodic processes. They also provide the base representation for designing {\em isometry blind} time warping algorithms, which I use to synchronize time-ordered point clouds that are shifted versions of each other in space without ever having to do a spatial alignment. In particular, I devise an algorithm that loIr bounds the 1-stress betIen two time-ordered point clouds, which is related to the Gromov-Hausdorff distance.

Projects

Automatic Cover Song Identification




Sliding Window Videos




Geometric Models for Musical Audio Data