Audio Restoration/Preservation
Recently, my research has branched into a new area for me where we are using signal processing and analog circuit modeling to design new tools for audio restoration and preservation. My Ph.D. student Ben Thompson has been the driving force behind this new research direction and has made some amazing contributions to the field in the short amount of time that we’ve been working in this area. This work has also been aided by collaborations with Prof. Darren Mueller at the Eastman School of Music, my other Ph.D. student Melissa Chen, and many fruitful conversations with restoration engineers and archivists at the Library of Congress, New York Public Library, and the British Library. Some of our most recent contributions are summarized below.
Playback Equalization
Since the inception of electrical recording for phonograph records in 1924, records have been intentionally cut with a non-uniform frequency response to maximize the information density on a disc and to improve the signal-to-noise ratio. To reproduce a nominally flat signal within the available bandwidth, the effects of this cutting curve must be undone by applying an inverse curve on playback. Until 1953, with the introduction of what has become known as the RIAA curve, the playback curve required for any particular disc could vary by record company and over time. As a consequence, anyone seeking to hear or restore the information on a disc must have access to equipment that is capable of implementing multiple playback equalizations. This correction may be accomplished with either analog hardware or digital processing. The digital approach has the advantages of reduced cost and expanded versatility, but requires a transformation from continuous time, where the original curves are defined, to discrete time. This transformation inevitably comes with some deviations from the continuous-time response near the Nyquist frequency. There are many established methods for discretizing continuous-time filters, and these vary in performance, computational cost, and inherent latency. Our work in this area is intended as a resource for anyone developing systems for digital playback equalization or similar applications that require approximating the response of a continuous-time filter digitally. Ben developed an awesome plugin based on his work that you can purchase here…mention my name, it may be worth a discount, or may cost you more money depending on how many revisions of his thesis I ask him to write.
Virtual Analog
In addition to the degradation of the medium itself, another issue in the world of audio preservation/restoration is the degradation of the playback equipment. Many hardware devices are out of production and use obsolete parts that are difficult to find and maintain - especially for smaller institutions. We’ve looked into some digital modeling of vintage hardware or hardware processes to try address these issues. A significant part of this work was the creation of new datasets to allow researchers to evaluate their models more accurately.
Our first paper in this area was on the Solid State Logic (SSL) FX G-384 Bus Compressor, which was generously donated to the University of Rochester by the estate of Ric Ocasek. The quantity that defines the behavior of a dynamic range compressor is the time-varying gain applied to the signal as a function of the input level. However, models of these devices are typically evaluated using proxy metrics because isolating the gain reduction signal from the audio input–output data included in existing datasets creates an ill-conditioned inverse problem. It is unclear how accurately these metrics describe the behavior the model is tasked with emulating, particularly as waveform-based metrics can be influenced by secondary effects introduced by analog processing and capture, even when those effects are inaudible. We investigate a method of evaluation in which the gain-reduction signal produced by a model is measured directly against a gain-reduction control voltage signal produced by the hardware. To evaluate the efficacy of this metric as a learning objective, a gray-box model is trained using loss computed directly over the gain control signals alongside two models trained using common proxy losses. The models trained using proxy losses did not achieve parity with models trained directly on the gain control signal when evaluated with respect to the underlying control trajectory, and the waveform-domain metrics assigned similar errors to models that were clearly separated by the direct metric. To facilitate further exploration of this method of evaluation, we created a Solid State Logic bus compressor dataset that includes the gain control voltage signal captured alongside the audio output. A more detailed dive into the electronics of this particular unit is in preparation.
Above: SSL Logic FX G384 bus compressor measured for dataset. The wire test leads are connected to the control input of the left audio voltage-controlled amplifier (VCA). Right: Dynamics processor error metrics. The top panel shows the measured linear gain signal, g[k], from a hardware SSL compressor alongside the gain produced by a software model, G[k], for the same input. The second panel shows the absolute error between the linear gain signals, |g[k]−G[k]|. The third panel shows the absolute error between the output waveforms, |y[k]−Y[k]|. The bot-tom panel shows the MSTE error between the output waveforms, where each frame’s contribution is distributed uniformly over the duration of that frame. The frame sizes used in the MSTE calculation are 8, 16, 32, and 64 samples, with 75% overlap. All error signals are normalized by their respective maxima to facilitate comparison.
Noise Reduction and Bandwidth Extension
Restoration of historical recordings is typically framed as a rigid mapping from a degraded input to a single clean output, enforcing irreversible decisions about what counts as damage and inseparably hiding synthesized content within the final mix. Because archival aesthetics are highly subjective, we argue that restoration tools should empower users with interactive control rather than imposing a single predetermined outcome. To achieve this, we introduce RESTORE, a framework that reformulates historical restoration as a six-source semantic decomposition. By expanding a pretrained HTDemucs backbone, a single forward pass disentangles the mixture into vocals, music, broadband hiss, impulsive transients, and an unmodeled residual, while jointly synthesizing a high-frequency extension. Users steer the restoration by simply remixing stem gains, ensuring generative content remains isolated and auditable. Evaluated across diverse genres, RESTORE improves audio quality and FAD over the baselines while delivering genuine aesthetic steerability at 50× real time on a single GPU. The training data for RESTORE was developed by introducing known degradations such as pops, clicks, noise, and equalization to wide-band modern recordings. The model was then tested on real vintage recordings borrowed from the Great78 project. Real-time demonstrations of RESTORE can be found here.
Above: In RESTORE, a degraded historical mixture is processed by a dual-domain generator with a cross-domain transformer bottleneck, then decoded and split into six semantically targeted stems. All stems are supervised via L1 and STFT losses with a mixture consistency penalty enforcing the sum back to the input. A discriminator is applied only to the high-frequency extension stem. The restored waveform is reconstructed by summing desired stems with independent, user-defined gains.