C-RASP length-generalization conjecture and sample-size bounds

Determine asymptotically tight worst-case sample-size bounds for length generalization by transformers whose solutions are expressible in the fragments C-RASP+\mathsf{C\text{-}RASP}_{+} and C-RASP1\mathsf{C\text{-}RASP}_{1}. In particular, decide whether the previously known double-exponential upper bounds are tight, or whether substantially smaller bounds suffice.

References

Primary source

arXiv

Progress summary

Refreshed
Claimed progress

A recent paper claims much tighter limits for how far these sequence models generalize, but the full conjecture and sample-size question remain open.

The problem concerns length generalization and sample-size bounds for C-RASP\mathsf{C\text{-}RASP} and equivalent transformers. A March 2026 paper claims the unrestricted computability question has a negative answer, while a September 2026 paper claims substantially sharper quantitative bounds.

Known results

  • One-layer C-RASP\mathsf{C\text{-}RASP}: lengths O(T2)O(T^{2}) suffice when parameter precision is bounded by TT (June 2025).
  • Two-layer C-RASP\mathsf{C\text{-}RASP} with at most KK heads: upper bound O(TO(K))O(T^{O(K)}) (June 2025).
  • The unrestricted problem was claimed noncomputable, even at depth two; C-RASP+\mathsf{C\text{-}RASP}_{+} was claimed to have exponential upper and lower bounds, while sample-size questions remained open (March 2026).
  • Learning a narrow teacher implementing a C-RASP\mathsf{C\text{-}RASP} program was given sample bound N=O ⁣(1εLdlog⁡Q)N=\mathcal{O}\!\left(\frac{1}{\varepsilon}Ld\log Q\right) under stated assumptions (July 2026).

September 2026 quantitative-bound claim

On September 8, 2026, Length Generalization for Transformers via Compression reported an exponentially smaller bound than the prior double-exponential estimate and claimed to reconcile conflicting experiments. It does not establish every form of the original C-RASP\mathsf{C\text{-}RASP} hypothesis, and the claim has no independent verification in the retrieved sources.

Current status (as of September 2026): Restricted bounds are known, while the general computability claim and sharper quantitative bounds remain unverified and the full sample-size question remains open.

Sources

Solutions 0

No solutions have been posted yet.