UNIVERSITY OF HERTFORDSHIRE
	     COMPUTER SCIENCE RESEARCH COLLOQUIUM

			    presents


"Finite State Markov Decision Problems with Transfer Entropy Constraints"

		     Dr. Takashi Tanaka
	 (KTH Royal Institute of Technology, Sweden)

		  5 April 2017 (Wednesday)
			   1 - 2 pm


		  Hatfield, College Lane Campus
	           Lecture Theatre C400 

		  Everyone is Welcome to Attend

		  Refreshments will be available


Abstract:

  We consider finite state Markov decision problems (MDPs) in which a
weighted sum of the classical state-dependent cost and the transfer
entropy from the state random process to the control random process is
minimized. The transfer entropy is given an interpretation in networked
control systems as the rate at which the sensor data is fed back to
the controller. In non-equilibrium thermodynamics, the transfer entropy
provides a fundamental limitation of the efficiency of a thermal engine
operated far from equilibrium.
In this talk, we first show that an optimality condition for the decision
policy is obtained by dynamic programming. Then we discuss an iterative
forward-backward computational procedure similar to Arimoto-Blahut
algorithm to synthesize the optimal policy numerically. Convergence
of the algorithm is established. The proposed algorithm is applied to
an information-constrained navigation problem over a maze, whereby we
study how the price of information alters the optimal decision polices
qualitatively.

About the Speaker:

Takashi Tanaka received his B.S. degree from Tokyo University in 2006,
M.S. and Ph.D. degrees from University of Illinois at Urbana-Champaign
(UIUC) in 2009 and 2012, all in Aerospace Engineering. He was a
postdoctoral associate at the Laboratory for Information and Decision
Systems (LIDS) at Massachusetts Institute of Technology (MIT) from 2012
to 2015. Currently, he is a postdoctoral researcher in the School of
Electrical Engineering at KTH Royal Institute of Technology, Sweden. From
Fall 2017, he will be an Assistant Professor in the Department of
Aerospace Engineering and Engineering Mechanics at the University of
Texas at Austin. His research interests include control, optimization,
game theory, information theory, and their applications to distributed
decision-making problems.

---------------------------------------------------
Hertfordshire Computer Science Research Colloquium
http://cs-colloq.cs.herts.ac.uk