Skip to content
Live
Apple Machine Learning Research·· 7 days ago

Understanding Private Evolution as Learning-Augmented Clustering

Understanding Private Evolution as Learning-Augmented Clustering

Brief overview

Private Evolution (PE) is a differential privacy synthetic data generation algorithm. The authors redefine it as generative model-enhanced Wasserstein learning and prove that combining generative models that can capture real distributions can significantly improve performance boundaries, such as the complexity of generating samples in low-dimensional spaces depending on the intrinsic dimension rather than the environmental dimension. Experiments show that the newly proposed geometry-aware version of PE has provable convergence on simple aggregation instances and can improve recall compared to standard baseline algorithms.

Article

AuthorsAudra McMillan, Kunal Talwar, Felix Zhou†**

Private Evolution (PE) is a differentially private algorithm for synthetic data generation. While it can be viewed as a Wasserstein learning algorithm, it performs much better in practice than worst-case Wasserstein analyses would predict. We recast PE as generative model-augmented Wasserstein learning. We show theoretically that when we take into account the use of a generative model that is able to capture something about the true distribution, then we can obtain much better performance bounds. For example, if the generator gives samples in the same low-dimensional space as the distribution, then sample complexity depends on intrinsic, not ambient, dimension. We also show that standard variants of PE can fail to converge on simple well-clustered instances, and propose a new geometry-aware version of PE with provable convergence on such instances. Experimentally, we show that our new algorithm is competitive with standard baselines and can improve recall.

  • † Yale University
  • ** Work done while at Apple

Related readings and updates.

Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation…

Read more

Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation…

Read more

Source: Apple Machine Learning Research · machinelearning.apple.com