Unlearning in Tabular-to-Hypergraph Learning via Selective Distillation

Abstract

Machine unlearning requires efficiently removing the influence of specified information (rows/columns/values) from a deployed model to satisfy privacy and regulatory constraints. Converting tables into hypergraphs can capture high-order relations such as “same column”, “same value”, and multi-column interactions, but unlearning for hypergraph-based models is often implemented via costly retraining or gradient/Hessian-based approximations, and becomes especially challenging for column- and value-level deletion due to broad structural dependencies and limited verifiability. We propose HERMES, a fundamentally different unlearning paradigm for tabular-to-hypergraph modeling based on selective knowledge transfer rather than parameter rollback or second-order correction. HERMES freezes the original trained model as a teacher and trains a student as the released unlearned model: on the retained set, the student learns from the teacher through structure-augmented distillation and consistency regularization to preserve utility; on the forget set, the student is driven toward maximum-entropy predictions and explicitly pushed away from the teacher via anti-distillation divergence, actively erasing memorized behaviors that can be exploited by membership inference. For column/value deletion, HERMES introduces counterfactual invariance by replacing deleted attributes/values and enforcing prediction consistency, preventing the student from relying on prohibited information even through hypergraph message-passing “detours”. The framework requires neither graph partitioning nor second-order information, and can be implemented as a lightweight post-processing stage that completes in a few epochs while offering practical verification signals based on forget-set entropy, teacher–student disagreement, and membership inference risk.