Paper List

1. Differential Privacy Characterization

• 1.1 DP

• 1.2 RDP

• 1.3 Final Round RDP

• 1.4 Final Round Langevin

• 1.5 DP Sampling

• 1.6 Convergence Analysis

2. Differential Privacy Estimation

1. Differential Privacy Characterization

1.1 Differential Privacy (DP)

• Dwork C, Rothblum G N, Vadhan S. Boosting and differential privacy. IEEE 51st Annual Symposium on Foundations of Computer Science (FOCS), 2010.

• Bassily R, Smith A, Thakurta A. Private empirical risk minimization: Efficient algorithms and tight error bounds. IEEE 55th Annual Symposium on Foundations of Computer Science, 2014.

• Kairouz P, Oh S, Viswanath P. The composition theorem for differential privacy. International Conference on Machine Learning (ICML), 2015.

• Abadi M, Chu A, Goodfellow I, et al. Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016.

1.2 Rényi Differential Privacy (RDP)

• Van Erven T, Harremos P. Rényi divergence and Kullback-Leibler divergence. IEEE Transactions on Information Theory (TIT), 2014.

• Mironov I. Rényi differential privacy. IEEE 30th Computer Security Foundations Symposium (CSF), 2017.

1.3 Final Round RDP

• Feldman V, Mironov I, Talwar K, et al. Privacy amplification by iteration. IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), 2018.

• Altschuler J, Talwar K. Privacy of noisy stochastic gradient descent: More iterations without more privacy loss. Advances in Neural Information Processing Systems (NIPS), 2022.

• Altschuler J M, Bok J, Talwar K. On the privacy of noisy stochastic gradient descent for convex optimization. SIAM Journal on Computing, 2024.

• Kong W, Ribero M. Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite losses. arXiv preprint arXiv:2407.05237, 2024.

• Chien E, Li P. Convergent privacy loss of noisy-sgd without convexity and smoothness. arXiv preprint arXiv:2410.01068, 2024.

1.4 Final Round Langevin

• Chourasia R, Ye J, Shokri R. Differential privacy dynamics of Langevin diffusion and noisy gradient descent. Advances in Neural Information Processing Systems (NIPS), 2021.

• Ryffel T, Bach F, Pointcheval D. Differential privacy guarantees for stochastic gradient Langevin dynamics. arXiv preprint arXiv:2201.11980, 2022.

• Ye J, Shokri R. Differentially private learning needs hidden state (or much faster convergence). Advances in Neural Information Processing Systems (NIPS), 2022.

• Asoodeh S, Diaz M. Privacy loss of noisy stochastic gradient descent might converge even for non-convex losses. arXiv preprint arXiv:2305.09903, 2023.

1.5 DP Sampling

• Beimel A, Brenner H, Kasiviswanathan S P, et al. Bounds on the sample complexity for private learning and private data release. Machine Learning, 2014.

• Bun M, Dwork C, Rothblum G N, et al. Composable and versatile privacy via truncated cdp. Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, 2018.

• Mironov I, Talwar K, Zhang L. Rényi differential privacy of the sampled Gaussian mechanism. arXiv preprint arXiv:1908.10530, 2019.

• Balle B, Barthe G, Gaboardi M. Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in Neural Information Processing Systems (NIPS), 2018.

• Wang Y X, Balle B, Kasiviswanathan S P. Subsampled Rényi differential privacy and analytical moments accountant. The 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.

• Zhu Y, Wang Y X. Poission subsampled Rényi differential privacy. International Conference on Machine Learning (ICML), 2019.

• Steinke T. Composition of differential privacy & privacy amplification by subsampling. arXiv preprint arXiv:2210.00597, 2022.

1.6 Convergence Analysis

• Zhang J, He T, Sra S, et al. Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv preprint arXiv:1905.11881, 2019.

• Song S, Thakkar O, Thakurta A. Characterizing private clipped gradient descent on convex generalized linear problems. arXiv preprint arXiv:2006.06783, 2020.

• Fang H, Li X, Fan C, et al. Improved convergence of differential private sgd with gradient clipping. International Conference on Learning Representations (ICLR), 2023.

• Koloskova A, Hendrikx H, Stich S U. Revisiting gradient clipping: Stochastic bias and tight convergence guarantees. International Conference on Machine Learning (ICML), 2023.

2. Differential Privacy Estimation

• Steinke T, Nasr M, Jagielski M. Privacy auditing with one (1) training run. Advances in Neural Information Processing Systems (NIPS), 2024.

• Andrew G, Kairouz P, Oh S, et al. One-shot empirical privacy estimation for federated learning. arXiv preprint arXiv:2302.03098, 2023.