Paper Detail
Nazanin Mirhosseini
We propose a continuous learning formulation for $T$ non-forgetting tasks with the spherical parameter space $\mathbb{S}^{d-1}(\sqrt{d})$ in asymptotic regime $d\rightarrow \infty$. Each task $i$ has an optimum $\bmΘ_i$, minimizing its corresponding loss function $R_i(\bm{w})$ on sphere. We reduce each task $i$ to an acceptable loss region modeled by a geodesic ball $B_{\bmΘ_i}(r_i(d))$ on $\mathbb{S}^{d-1}(\sqrt{d})$ centered at $\bmΘ_i$ with radius $r_i(d)$. To model task dependencies (similarities), task optima $\bmΘ_i$ are distributed by a Neyman-Scott clustered point process $Φ$. Cluster centers $\{\bm{Y}_j\}_{j=1}^{N_p}$ form a homogeneous Poisson point process on sphere, while intra-cluster task optima follow a von Mises-Fisher distribution with mean $\bm{Y}_j$ and concentration $κ$. Each task $t\in \{1, ..., T\}$ learning algorithm is considered to be a channel $Q_{t}(\mathrm{d}\bm{w}_t | \bm{w}_{t-1}, \mathcal{K}_{t-1}(\bmθ_{1:t-1}), \bmθ_t)$, where $\bm{W}_t$ is the learned parameter of task $t$ and $\mathcal{K}_{t-1}(\bmΘ_{1:t-1})=\bigcap_{i=1}^{t-1}B_{\bmΘ_i}(r_i(d))$. Since in continual learning, the learner does not need to reach $\bmΘ_t$ exactly to successfully learn task $t$, to avoid forgetting as well as reliable learning, it only needs for the learned parameter vector $\bm{W}_{t}$ to enter $B_{\bmΘ_t}(r_t(d)) \cap \mathcal{K}_{t-1}(\bmΘ_{1:t-1})$. Using this model, we derive an upper bound on $T$ ensuring $\mathbb{P}[\mathcal{K}_{T}(\bmΘ_{1:T}) \neq \emptyset] \rightarrow 1$ as $d\rightarrow \infty$ and $κ/d \rightarrow α$, where $α>0$ is a constant. Finally, for $T$ tasks sampled uniformly from $Φ$, we derive an upper bound for total information theoretic capacity under the non-forgetting constraint for each task.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@article{mirhosseini2026effect,
title = {Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning},
author = {Nazanin Mirhosseini},
year = {2026},
abstract = {We propose a continuous learning formulation for \$T\$ non-forgetting tasks with the spherical parameter space \$\textbackslash{}mathbb\{S\}\textasciicircum{}\{d-1\}(\textbackslash{}sqrt\{d\})\$ in asymptotic regime \$d\textbackslash{}rightarrow \textbackslash{}infty\$. Each task \$i\$ has an optimum \$\textbackslash{}bmΘ\_i\$, minimizing its corresponding loss function \$R\_i(\textbackslash{}bm\{w\})\$ on sphere. We reduce each task \$i\$ to an acceptable loss region modeled by a geodesic ball \$B\_\{\textbackslash{}bmΘ\_i\}(r\_i(d))\$ on \$\textbackslash{}mathbb\{S\}\textasciicircum{}\{d-1\}(\textbackslash{}sqrt\{d\})\$ centered at \$\textbackslash{}bmΘ\_i\$ with radius \$r\_i(d)\$. To model task dependencies (similar},
url = {https://arxiv.org/abs/2610.17453},
keywords = {cs.LG, cs.IT, stat.ML},
eprint = {2610.17453},
archiveprefix = {arXiv},
}
{}