29 problems

Extension of the asymptotic bias formula to general eligibility traces
arXiv@arxiv
The TD learning recursion is governed by a Markov process, with eligibility-trace parameter $\lambda$ ; for $\lambda=0$ , the asymptotic bias limit exists and is given by the bias fo…
Conjectureopenreinforcement-learning stochastic-approximation temporal-difference-learning
1
Steady-state convergence and moment conjecture for general-objective SGD
arXiv@arxiv
Let $f:cmathbb{R}ctocmathbb{R}$ be convex and of class $C^h(cmathbb{R})$ for an even integer $h>0$ , with unique minimizer $x^*$ , satisfying $\lim_{x\to x^*}f^{(k)}(x)=0$ for…
Conjectureopensteady-state-analysis stochastic-approximation stochastic-gradient-descent
0
Chen's fundamental-limit conjecture for stochastic-gradient identification
arXiv@arxiv
Chen's conjecture. The established sufficient condition $\alpha\leq 1/3$ for strong consistency of the stochastic-gradient identification algorithm is inherently conservative; the…
Conjectureopenadaptive-control stochastic-approximation system-identification
1
Validity of the framework for broader classes of moving boundaries
arXiv@arxiv
Let $T(j,\alpha_t)$ be the moving boundary used by the textsc{StageSampling} procedure, which adaptively samples at a design point and stops when the accumulated statistic satisfie…
Conjectureopenroot-finding sequential-analysis stochastic-approximation
0
Reduction of the online-estimator limit set
arXiv@arxiv
Let $n$ be the number of agents, let $[n]=\{1,\ldots,n\}$ , and let $\boldsymbol{\pi}=(\pi_1,\ldots,\pi_n)$ be the vector of unreliability parameters. Let…
Conjectureopendynamical-systems fact-checking stochastic-approximation
0
Conjecture on bias characterization for nonsmooth stochastic approximation operators
arXiv@arxiv
Bias characterization conjecture. The bias characterization result, including the conclusion that under non-degenerate noise the steady-state bias is of order…
Conjectureopennonsmooth-analysis reinforcement-learning stochastic-approximation
0
Applicability of the inductive proof technique to stochastic approximation
arXiv@arxiv
The authors study temporal-difference learning with linear function approximation under Markovian sampling and develop an inductive technique for proving that the iterates remain u…
Conjectureopenreinforcement-learning stochastic-approximation
0
Conjectured simplification and sharpening of robustness results for compressed TD learning
arXiv@arxiv
The paper studies inductive proofs for the effect of delays on iterative reinforcement-learning algorithms and compares them with robustness results for temporal-difference learnin…
Conjectureopendistributed-learning reinforcement-learning stochastic-approximation
0
Adaptive quantization for stochastic approximation with Lipschitz operators
arXiv@arxiv
A stochastic approximation problem has an underlying operator with some version of Lipschitzness when, for example, its operator is a gradient arising in smooth optimization. Adapt…
Conjectureopenoptimization quantization stochastic-approximation
0
Conjecture on non-diminishing stepsizes and asymptotic behavior in Adam-family methods
arXiv@arxiv
The Adam-family framework uses sequences of stepsizes $\{\eta_k\}$ and $\{\theta_k\}$ , and the sequence $\{y_k\}$ asymptotically approximates trajectories of a differential inclusi…
Conjectureopendeep-learning optimization stochastic-approximation
0
The conjecture that constant-gain techniques can estimate global minima faster than simulated annealing
arXiv@arxiv
Let the techniques developed in this paper be applied to optimization problems with objectives having multiple local minima, and let simulated annealing be used as a comparison met…
Conjectureopenoptimization simulated-annealing stochastic-approximation
0
Conjecture that the stochastic approximation convergence rate is not optimal
arXiv@arxiv
Consider the stochastic approximation setting above, with convergence-rate parameters $d$ , $d_{a,c}$ , and $\xi$ satisfying $\xi \in \left(0,\frac{2d}{3}\right]$ . The established ra…
Conjectureopenconvergence-rates stochastic-approximation stochastic-control
0
Convergence conjecture for average utilities under dual scheduling
arXiv@arxiv
Average-utility convergence conjecture. For every such $d$ and ${\bm \beta}({\bf H},{\bar{\bf U}})$ ,
Conjectureopenmarkov-chains stochastic-approximation wireless-scheduling
1
Sliding-mode implication conjecture for policy chattering in approximate reinforcement learning
arXiv@arxiv
Sliding-mode implication conjecture. The existence of a sliding mode solution in the DI must imply that policy chattering takes place almost surely.
Conjectureopendifferential-inclusions reinforcement-learning stochastic-approximation
1
Invariant-chain-transitive-set conjecture for the Gordon differential inclusion
arXiv@arxiv
Invariant-chain-transitive-set conjecture. The only invariant, internally chain transitive sets of the DI $$ are the singleton sets
Conjectureopendifferential-inclusions reinforcement-learning stochastic-approximation
1
Second-order moment conjecture for long-range dependent white noise
arXiv@arxiv
Let the stochastic approximation methods and the long-range dependent white noise processes considered in the paper be given, with convergence-rate exponent $\sigma$ and Hurst inde…
Conjectureopenlong-range-dependence stochastic-approximation streaming-data
0
Optimality of the leading-order term in the fixed-point solution
arXiv@arxiv
Let $s_n$ denote the solution to the fixed-point equation arising in the preceding bound, and let its leading-order term refer to the dominant asymptotic contribution as $n$ become…
Conjectureopenasymptotic-optimality reinforcement-learning stochastic-approximation
1
The sufficient boundary-utility condition for stochastic approximation of symmetric Nash equilibria
arXiv@arxiv
Boundary-utility conjecture. The following condition is sufficient: for all $\mathbf{p}\notin\mathcal{S}$ ,
Conjectureopennash-equilibrium queueing-games stochastic-approximation
0
The conjectured step-size condition for policy tracking with TD-learning
arXiv@arxiv
Let $\gamma_\alpha$ and $\gamma_\pi$ be the exponents governing the step-size and policy-update schedules, respectively. The stated convergence bound requires…
Conjectureopenreinforcement-learning stochastic-approximation
0
Nonconvergence conjecture for nonuniform Monte Carlo exploring starts
arXiv@arxiv
Consider the update rule … where $Gamma_t=gamma_t\operatorname{diag}\{p(1),p(2),\ldots,p(n)\}$ and each $p(i)$ is the probability of selecting state $i$ , with every state having no…
Conjectureopendynamic-programming reinforcement-learning stochastic-approximation
0
RaQL's independent risk estimation and Q-value updates reduce iterative bias
arXiv@arxiv
Risk-aware Q-learning (RaQL) uses an inner--outer loop structure in which the risk is estimated in an inner procedure and the $Q$ -values are updated in an outer procedure. RaQL bia…
Conjectureopenq-learning risk-aware-mdps stochastic-approximation
1
The computational advantage of RaQL over exact risk-aware dynamic programming
arXiv@arxiv
Risk-aware dynamic programming estimates risk at each iteration by solving an exact saddle-point optimization problem, while RaQL uses stochastic approximation with a stochastic ap…
Conjectureopendynamic-programming risk-aware-mdps stochastic-approximation
0
Conjecture on stability of Zap Q-learning under general conditions
arXiv@arxiv
Zap Q( $\lambda$ )-learning refers to the family of stochastic-approximation algorithms introduced in the paper for parameterized reinforcement-learning problems. The source asks for…
Conjectureopenreinforcement-learning stochastic-approximation
0
Conjecture on the true stochastic order of PBA error
arXiv@arxiv
Let $T_n$ be the total number of queries required by the first $n$ tests in the modified probabilistic bisection algorithm (PBA), and let the error be measured by the distance betw…
Conjectureopenprobabilistic-bisection root-finding stochastic-approximation
0
Convergence to equilibrium for strongly reinforced Pólya urns
arXiv@arxiv
Convergence to equilibrium. For any WARM with $\alpha>1$ , there exists a random vector $\vec{X}=(X_1,\dots,X_n)$ , supported on the set of linearly-stable and critical equilibria, s…
Conjectureopendynamical-systems polya-urns stochastic-approximation
0