IEEE Office 3 Park Avenue, 17th Floor New York, New York 10016-5997 U.S.A. Tel: +1 212 419 7900 Fax: +1 212 752 4929
2014 Best oral presentation and paper at the IEEE
Systems, Man, and Cybernetics Society Conference

The winning paper is: Jeffrey S. Campbell, Sidney N. Givigi, and Howard M. Schwartz, 
Multiple Model Q-Learning for Stochastic Reinforcement Delays, 2014 IEEE International
Conference on Systems, Man, and Cybernetics, October 5-8, 2014, San Diego, CA, USA


The main contribution of this work is a novel machine reinforcement learning algorithm for
problems where a Poissonian stochastic time delay is present in the agent's reinforcement
signal. Despite the presence of the reinforcement noise, the algorithm can craft a suitable
control policy for the agent's environment. The novel approach can deal with reinforcements
which may be received out of order in time or may even overlap, which was not previously
considered in the literature. The proposed algorithm is simulated and its performance is
compared to a standard Q-learning algorithm. Through simulation, the proposed method is
found to improve the performance of a learning agent in an environment with Poissonian-type
stochastically delayed rewards.


The Autonomous Robotics Research Group (ARRG) includes researchers in three
institutions, the Royal Military College of Canada (RMCC), Carleton University and Queen's
University. The main interest of the group is in machine learning, especially Reinforcement
Learning, and multiple robotics, from ground robots to Unmanned Aerial Vehicles (UAVs) and
Autonomous Underwater Vehicles (AUVs). It counts with state-of-the-art lab facilities located
at RMCC with more than a dozen ground robots and ten UAVs.


Second Lieutenant Jeffrey S. Campbell is a pilot in the Royal Canadian Air Force. He
completed his undergraduate degree in 2012 at the Royal Military College of Canada in
Kingston, Canada. While there, he specialized in robotic control and worked on quadrotor
unmanned aerial vehicles. In 2014 he received his master's degree in electrical engineering
from Carleton University in Ottawa, Canada. His work there focused on unsupervised machine
learning with applications in mobile robotics. Jeff is currently posted at Defence Research and
Development Canada to work on over-the-horizon radar projects.

Sidney N. Givigi received his B.Sc. in Computer Science and an M.A.Sc. in Electrical
Engineering from the Federal University of Esp$\acute{i}$rito Santo, Brazil. He also received
his Ph.D. in Electrical and Computer Engineering from Carleton University, Canada. In 2009,
he joined the Department of Electrical and Computer Engineering of the Royal Military
College of Canada (RMCC) as an Assistant Professor. Sidney's research interests are mainly
focused on autonomous systems, especially the decentralized control of multiple vehicles,
learning and adaptation of autonomous robots and modeling of complex systems with Game

Professor H.M. Schwartz received his B.Eng. degree from McGill University, Montreal,
Quebec and his M.S. degree and Ph.D. degree from M.I.T., Cambridge, Massachusetts.
His research interests include adaptive and intelligent control systems, robotics, system
modelling and system identification. His most recent research is in multi agent learning with
applications to teams of mobile robots.
Home President's Message Editorial Member Profile Conference Calendar Submission Guidelines Back Issues Society Information