Universally Instance-Optimal Mechanisms for Private Statistical Estimation
AuthorsHilal Asi, John C. Duchi‡, Saminul Haque‡, Zewei Li†, Feng Ruan†
AuthorsHilal Asi, John C. Duchi‡, Saminul Haque‡, Zewei Li†, Feng Ruan†
We consider the problem of instance-optimal statistical estimation under the constraint of differential privacy where mechanisms must adapt to the difficulty of the input dataset. We prove a new instance specific lower bound using a new divergence and show it characterizes the local minimax optimal rates for private statistical estimation. We propose two new mechanisms that are universally instance-optimal for general estimation problems up to logarithmic factors. Our first mechanism, the total variation mechanism, builds on the exponential mechanism with stable approximations of the total variation distance, and is universally instance-optimal in the high privacy regime . Our second mechanism, the T-mechanism, is based on the T-estimator framework (Birge´, 2006) using the clipped log likelihood ratio as a stable test: it attains instance-optimal rates for any up to logarithmic factors. Finally, we study the implications of our results to robust statistical estimation, and show that our algorithms are universally optimal for this problem, characterizing the optimal minimax rates for robust statistical estimation.
† Northwestern University
‡ Stanford University
November 21, 2024research area Privacyconference NeurIPS
Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation...
July 23, 2024research area Privacyconference TPDP
Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation...