^* indicates equal contribution. Advisees are underlined. ^† indicates equal advising. 🎉 indicates the works I like most.

• [arXiv 2024] Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise.
  Xiaochi Qian^*, Zixuan Xie^*, Xinyu Liu^*, Shangtong Zhang.

• [arXiv 2024] The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise.
  Shuze Liu, Shuhang Chen, Shangtong Zhang.

• [arXiv 2024] Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning.
  Jiuqi Wang^*, Ethan Blaser^*, Hadi Daneshmand, Shangtong Zhang.
  Best Paper Award at the ICML Workshop on In-Context Learning, 2024.

• [AAAI 2025] Efficient Multi-Policy Evaluation for Reinforcement Learning.
  Shuze Liu, Claire Chen, Shangtong Zhang.

• [arXiv 2024] CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening.
  Amar Kulkarni, Shangtong Zhang, Madhur Behl.

• [arXiv 2024] Efficient Policy Evaluation with Safety Constraint for Reinforcement Learning.
  Claire Chen^*, Shuze Liu^*, Shangtong Zhang.

• [arXiv 2024] Doubly Optimal Policy Evaluation for Reinforcement Learning.
  Shuze Liu, Claire Chen, Shangtong Zhang.

• [arXiv 2024] Almost Sure Convergence of Average Reward Temporal Difference Learning.
  Ethan Blaser, Shangtong Zhang.

• [arXiv 2024] Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features.
  Jiuqi Wang, Shangtong Zhang.

• [ICML 2024] Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design.
  Shuze Liu, Shangtong Zhang.

• [arXiv 2023] Revisiting a Design Choice in Gradient Temporal Difference Learning.
  Xiaochi Qian, Shangtong Zhang.

• [AAAI 2023] A New Challenge in Policy Evaluation.
  Shangtong Zhang.

• [ICML 2023] On the Convergence of SARSA with Linear Function Approximation.
  Shangtong Zhang, Remi Tachet des Combes, Romain Laroche.

• [arXiv 2023] StarCraft II Unplugged: Large Scale Offline Reinforcement Learning
  Michael Mathieu^*, Sherjil Ozair^*, Srivatsan Srinivasan^*, Caglar Gulcehre^*, Shangtong Zhang^*, Ray Jiang^*, Tom Le Paine^*, Richard Powell, Konrad Zolna, Julian Schrittwieser, David Choi, Petko Georgiev, Daniel Toyama, Aja Huang, Roman Ring, Igor Babuschkin, Timo Ewalds, Mahyar Bordbar, Sarah Henderson, Sergio Gomez Colmenarejo, Aaron van den Oord, Wojciech Marian Czarnecki, Nando de Freitas, Oriol Vinyals.

• [JMLR 2022] Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch.
  Shangtong Zhang, Remi Tachet des Combes^†, Romain Laroche^†.

• [JMLR 2022] Truncated Emphatic Temporal Difference Methods for Prediction and Control.
  Shangtong Zhang, Shimon Whiteson.

• [AAMAS 2022] A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms.
  Shangtong Zhang, Romain Laroche, Harm van Seijen, Shimon Whiteson, Remi Tachet des Combes.
  Oral Presentation

• [AAAI 2022] Learning Expected Emphatic Traces for Deep RL.
  Ray Jiang, Shangtong Zhang, Veronica Chelu, Adam White, Hado van Hasselt.

• [ICML 2021] Breaking the Deadly Triad with a Target Network.
  Shangtong Zhang, Hengshuai Yao, Shimon Whiteson.

• [ICML 2021] Average-Reward Off-Policy Policy Evaluation with Function Approximation.
  Shangtong Zhang^*, Yi Wan^*, Richard S. Sutton, Shimon Whiteson.

• [AAAI 2021] Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning.
  Shangtong Zhang, Bo Liu, Shimon Whiteson.

• [NeurIPS 2020] Learning Retrospective Knowledge with Reverse Reinforcement Learning.
  Shangtong Zhang, Vivek Veeriah, Shimon Whiteson.

• [ICML 2020] GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values.
  Shangtong Zhang, Bo Liu, Shimon Whiteson.

• [ICML 2020] Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation.
  Shangtong Zhang, Bo Liu, Hengshuai Yao, Shimon Whiteson.

• [AAMAS 2020] Deep Residual Reinforcement Learning.
  Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson.
  Best Paper Award.

• [AAAI 2020] Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards.
  Yuhang Song, Jianyi Wang, Thomas Lukasiewicz, Zhenghua Xu, Shangtong Zhang, Mai Xu.

• [NeurIPS 2019] Generalized Off-Policy Actor-Critic.
  Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson.

• [NeurIPS 2019] DAC: The Double Actor-Critic Architecture for Learning Options.
  Shangtong Zhang, Shimon Whiteson.

• [ICML 2019] Distributional Reinforcement Learning for Efficient Exploration.
  Borislav Mavrin, Shangtong Zhang, Hengshuai Yao, Linglong Kong, Kaiwen Wu, Yaoliang Yu.
  

• [AAAI 2019] ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search.
  Shangtong Zhang, Hao Chen, Hengshuai Yao.
  Spotlight Presentation

• [AAAI 2019] QUOTA: The Quantile Option Architecture for Reinforcement Learning.
  Shangtong Zhang, Borislav Mavrin, Linglong Kong, Bo Liu, Hengshuai Yao.
  Oral Presentation

• [JOSS 2018] MLPack 3: A Fast, Flexible Machine Learning Library.
  Ryan R Curtin, Marcus Edel, Mikhail Lozhnikov, Yannis Mentekidis, Sumedh Ghaisas, Shangtong Zhang.

• [ECML-PKDD 2017] Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks.
  Vivek Veeriah^*, Shangtong Zhang^*, Richard S. Sutton.

• [Deep RL Symposium, NIPS 2017] A Deeper Look at Experience Replay.
  Shangtong Zhang, Richard S. Sutton.

• [Deep RL Symposium, NIPS 2017] Comparing Deep Reinforcement Learning and Evolutionary Methods in Continuous Control.
  Shangtong Zhang, Osmar R. Zaiane.

• [Hierarchical RL Workshop, NIPS 2017] A Demon Control Architecture with Off-Policy Learning and Flexible Behavior Policy.
  Shangtong Zhang, Richard S. Sutton.

• [ICMR 2015] A Deep Neural Network for Modeling Music.
  Pengjing Zhang, Xiaoqing Zheng, Wenqiang Zhang, Siyan Li, Sheng Qian, Wenqi He, Shangtong Zhang, Ziyuan Wang.