Inexact Generalized Gauss–Newton–CG for Binary Cross-Entropy Minimization - Mohammad Jamhuri; Silvi Puspita Sari; Siti Amiroch; Juhari Juhari; Vivi Aida Fitria (Universitas Islam Negeri Maulana Malik Ibrahim Malang, 2025)
Binary cross-entropy (BCE) minimization is a standard objective in probabilistic binary classification, yet practical training pipelines often rely on first-order methods whose performance can be sensitive to step-size choices and may require many iterations to reach low-loss solutions. This paper studies an inexact curvature-based solver that combines a (generalized) Gauss–Newton approximation with conjugate gradient (CG) inner iterations for minimizing the regularized BCE objective in full-batch logistic regression. At each outer iteration, the method computes a descent direction by approximately solving a damped Gauss–Newton system in a matrix-free manner via repeated products with X and X⊤, and terminates CG according to a relative-residual inexactness rule. Numerical experiments on three benchmark datasets show that the proposed Inexact GGN–CG can substantially reduce the number of outer iterations on smaller numerical data, while remaining competitive in predictive performance, and can improve both validation and test mean BCE on larger mixed-type data after one-hot encoding. In particular, on Adult Census Income the method achieves lower test mean BCE (0.3176 ± 0.0044) and higher F1-score (0.6623 ± 0.0066) than Adam and gradient descent under the same regularization-selection protocol, at the cost of additional CG work. These results highlight how damping and inexactness jointly govern the trade-off between curvature-solve effort, wall-clock time, and achieved BCE values in deterministic logistic-regression training.