Paper Detail
Matan Schliserman, Gon Buzaglo, Itay Evron, Daniel Soudry
We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets. This result generalizes prior analyses restricted to either stationary (single-task) deep models or continual linear models. We show that global convergence generally fails, even for simple models linear in data but nonlinear in parameters. Nevertheless, by leveraging results from nonconvex projection theory, we identify regularity properties of homogeneous deep networks that guarantee local linear convergence under random and cyclic task sequences. Finally, we extend our analysis to continual regression, unifying the framework for homogeneous models.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@article{schliserman2026convergence,
title = {Convergence of Continual Learning in Homogeneous Deep Networks},
author = {Matan Schliserman and Gon Buzaglo and Itay Evron and Daniel Soudry},
year = {2026},
abstract = {We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets. This result generalizes prior analyses restricted to either stationary (single-task) deep models or continual linear models. We show that global convergence generally fails, even for simple models linear in data but nonlinear in parameters. Nevertheless, by leveraging results from nonconvex projection theory, we identify regularity properties of homogeneous deep netw},
url = {https://arxiv.org/abs/2606.30559},
keywords = {cs.LG, math.NA, math.OC, stat.ML},
eprint = {2606.30559},
archiveprefix = {arXiv},
}
{}