Schedule: Day – 3

Computing is undergoing a paradigm shift: treating programs as mechanical procedures that realize a mathematical specification is slowly being replaced by a host of statistical learning methods that exploit the power of data abundance, preternatural generalization abilities, and efficient hardware platforms. While this is perhaps an acceptable approach for programs that decide what new product to suggest to a potential customer, it is potentially a recipe for disasters when applied to the paradigm of safety-critical systems. The question this talk seeks to answer is: why can we not have the best of both worlds? Can we combine learning techniques (for example reinforcement learning (RL) and deep RL) with traditional techniques for the design of safety-critical systems? We will look at some recent work where we are able to perform reward shaping from high-level temporal logic specifications such that any off-the-shelf RL algorithm that maximizes the expected long-term returns is guaranteed to maximize the probability of satisfying a given temporal logic specification. This result lets us not worry about good hand-crafted rewards that guarantee desired and safe behaviour, but instead uses mathematically precise high-level specifications to automatically infer such rewards. We will also examine how logical specifications can be leveraged to drastically reduce the number of demonstrations required to learn control policies from user demonstrations. Again, the objective is to move away from inferring implied task objectives (and control policies) from expert demonstrations to using temporal logic to formally specify high-level task objectives and couple it with non-expert demonstrations to infer optimal control policies. The main agenda of this talk is to argue for a logic-based framework to provide formal safety arguments for autonomous systems without sacrificing the advances provided by learning.