Paper Detail

Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning

Nazanin Mirhosseini

arxiv Score 16.0

Published 2026-10-12 · First seen 2026-10-14

Research Track A

Abstract

We propose a continuous learning formulation for $T$ non-forgetting tasks with the spherical parameter space $\mathbb{S}^{d-1}(\sqrt{d})$ in asymptotic regime $d\rightarrow \infty$. Each task $i$ has an optimum $\bmΘ_i$, minimizing its corresponding loss function $R_i(\bm{w})$ on sphere. We reduce each task $i$ to an acceptable loss region modeled by a geodesic ball $B_{\bmΘ_i}(r_i(d))$ on $\mathbb{S}^{d-1}(\sqrt{d})$ centered at $\bmΘ_i$ with radius $r_i(d)$. To model task dependencies (similarities), task optima $\bmΘ_i$ are distributed by a Neyman-Scott clustered point process $Φ$. Cluster centers $\{\bm{Y}_j\}_{j=1}^{N_p}$ form a homogeneous Poisson point process on sphere, while intra-cluster task optima follow a von Mises-Fisher distribution with mean $\bm{Y}_j$ and concentration $κ$. Each task $t\in \{1, ..., T\}$ learning algorithm is considered to be a channel $Q_{t}(\mathrm{d}\bm{w}_t | \bm{w}_{t-1}, \mathcal{K}_{t-1}(\bmθ_{1:t-1}), \bmθ_t)$, where $\bm{W}_t$ is the learned parameter of task $t$ and $\mathcal{K}_{t-1}(\bmΘ_{1:t-1})=\bigcap_{i=1}^{t-1}B_{\bmΘ_i}(r_i(d))$. Since in continual learning, the learner does not need to reach $\bmΘ_t$ exactly to successfully learn task $t$, to avoid forgetting as well as reliable learning, it only needs for the learned parameter vector $\bm{W}_{t}$ to enter $B_{\bmΘ_t}(r_t(d)) \cap \mathcal{K}_{t-1}(\bmΘ_{1:t-1})$. Using this model, we derive an upper bound on $T$ ensuring $\mathbb{P}[\mathcal{K}_{T}(\bmΘ_{1:T}) \neq \emptyset] \rightarrow 1$ as $d\rightarrow \infty$ and $κ/d \rightarrow α$, where $α>0$ is a constant. Finally, for $T$ tasks sampled uniformly from $Φ$, we derive an upper bound for total information theoretic capacity under the non-forgetting constraint for each task.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{mirhosseini2026effect,
  title = {Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning},
  author = {Nazanin Mirhosseini},
  year = {2026},
  abstract = {We propose a continuous learning formulation for \$T\$ non-forgetting tasks with the spherical parameter space \$\textbackslash{}mathbb\{S\}\textasciicircum{}\{d-1\}(\textbackslash{}sqrt\{d\})\$ in asymptotic regime \$d\textbackslash{}rightarrow \textbackslash{}infty\$. Each task \$i\$ has an optimum \$\textbackslash{}bmΘ\_i\$, minimizing its corresponding loss function \$R\_i(\textbackslash{}bm\{w\})\$ on sphere. We reduce each task \$i\$ to an acceptable loss region modeled by a geodesic ball \$B\_\{\textbackslash{}bmΘ\_i\}(r\_i(d))\$ on \$\textbackslash{}mathbb\{S\}\textasciicircum{}\{d-1\}(\textbackslash{}sqrt\{d\})\$ centered at \$\textbackslash{}bmΘ\_i\$ with radius \$r\_i(d)\$. To model task dependencies (similar},
  url = {https://arxiv.org/abs/2610.17453},
  keywords = {cs.LG, cs.IT, stat.ML},
  eprint = {2610.17453},
  archiveprefix = {arXiv},
}

Metadata

{}