Authors: Agus Hartoyo, Dominika Ciupek, Maciej Malawski, and Alessandro Crimi
This study explores the task of data reconstruction from machine learning models via inverse estimation and Bayesian inference, with the goal of recovering the original dataset solely based on the trained model. We introduce a novel theoretical framework that investigates the factors affecting the data reconstruction quality. Specifically, we derive expressions that quantify how variations in key variables influence the divergence between true and estimated posteriors by examining the concurrent behavior of their partial derivatives with respect to independent variables. This derivative-based approach establishes theoretical correlations between the variables, demonstrating that the fidelity of the recovered data is governed by two primary factors: (1) the accuracy of the assumed prior, and (2) the accuracy of the machine learning model. Empirical results across multiple benchmark datasets and machine learning algorithms corroborate these theoretical predictions, reinforcing the validity and robustness of our theoretical framework. Practically, our data reconstruction method enables the creation of synthetic models that closely replicate the performance of the original models. This work contributes to advancing the theoretical understanding and practical techniques for data reconstruction and model introspection within the context of machine learning.