90 problems

Minimax lower-bound conjecture. For distributionally robust reinforcement learning, the minimax lower bound on the number of samples is still

In tabular reinforcement learning with planning horizon $H$, consider any algorithm seeking an $\epsilon$-optimal policy when the total reward is bounded by $1$. Jiang et al.'s con…

Boundedness of the proxy approximation. There exists a constant $C>0$ such that, for a sufficiently fine state-space discretization,

Let $\mathcal{A}\to\mathsf{WfMd}$ be a Lie algebroid of controlled intervention modes over a typed Markdown workflow space, with anchor $\rho:\mathcal{A}\to T\mathsf{WfMd}$. For co…

The TD learning recursion is governed by a Markov process, with eligibility-trace parameter $\lambda$; for $\lambda=0$, the asymptotic bias limit exists and is given by the bias fo…

Let $S\geq 2$ and $A\geq 2$ be integers, let $D\geq 4\left\lceil\log_A S\right\rceil$, and consider horizon-$T$ algorithms for MDPs with $S$ states, $A$ actions, and diameter at mo…

A verifiable task is one in which a judge model can enforce the legality of player moves and score their plausibility, thereby providing a continuous reward signal for search optim…

The paper studies model-free off-policy reinforcement learning in continuous time with general function approximation. Its analysis uses an ellipticity condition, which induces a H…

Persistent-bias conjecture. This behavior arises from the algorithm’s online learning nature and/or the coupled dynamics of the actor and critic networks, which induce a more persi…

Regularizer-dependent gating conjecture. Different gating rules will emerge from proximal gradients arising from different regularizers.

In continuous-time reinforcement learning, let the dynamics be described by a controlled diffusion and let the Hamiltonian denote the local operator combining the drift, diffusion,…

Let $\lambda_{\texttt{min}}$ denote the minimum visitation probability in the asynchronous Q-learning setting described above. Visitation-probability dependence conjecture. Some de…

Extension conjecture. The insights gained from this analysis could be extended to general controlled Markov diffusions beyond the fine-tuning setting.

Consider a sequential decision process, or Markov decision process, with state $s$, action set $\Delta(\mathcal{A}(s))$, successor state $s'$, discount factor $\gamma$, and an empo…

Let $F$ and $G$ be the continuous self-maps defining the model-based and robot value equations, with positive inverse-temperature parameters $\beta_h$ and $\beta_r$. Since $F$ and…

Stratified latent-space conjecture. Distinct strata in the latent space correspond to different state-action trajectories, with increases in local dimension occurring when the agen…

Minimum attention is defined by the control-change functional … Here $u$ is a control varying over state space $\Omega$ and time interval $[0,T]$, and the functional penalizes chan…

Convergence-rate conjecture. The linear rate of convergence in the estimate referred to as, obtained for uncontrolled volatility, does not hold in the controlled-volatility case, f…

Ergodicity conjecture. The strong ergodicity assumption could be weakened.

Conjecture on improving $\alpha$. A smaller function $\alpha$ may be sufficient for the cited inequalities to hold, so that the resulting improvement could be carried over to the p…

Wider-regime conjecture. A similar lower bound should hold in the wider regime $\nu_0^{-1}\lesssim n\lesssim|\mathcal{S}|$.

Uniformly mixing complexity conjecture. Theorem should also imply an optimal complexity of $\widetilde{O}(SA\frac{\tau_{\mathrm{unif}}}{\varepsilon^2})$ for this setting by an anal…

The ergodic Avellaneda–Stoikov market-making model is assumed to describe market behaviour up to an unknown parameter, and the optimal control derived from maximum-likelihood estim…

Let $\lambda$ denote the temperature parameter controlling the weight on exploration, and consider the learned option prices obtained from the stopping and control procedures. For…

Let VRCQ denote the variance-reduced cascade Q-learning algorithm for estimating the optimal Q-function in a discounted Markov decision process, and let $\mathcal{U}$ be its action…