Equivariant Differentially Private Deep Learning: Why DP-SGD Needs Sparser Models

Florian A. Hölzl (Institute for Artifical Intelligence in Medicine, Technical University of Munich), Daniel Rueckert (Institute for Artifical Intelligence in Medicine, Technical University of Munich), Georgios Kaissis (Institute for Artifical Intelligence in Medicine, Technical University of Munich)

Published Jan 30, 2023 · arXiv:2301.13104

cs.CVcs.CRcs.LG

Abstract

This paper proposes using sparse model design through Equivariant CNN (ECNN) architectures to improve the performance and efficiency of deep learning models trained with Differentially Private Stochastic Gradient Descent (DP-SGD), achieving state-of-the-art results on image classification benchmarks with substantially lower computational requirements.

arXiv abstract PDF