Presented By O’Reilly and Intel AI
Put AI to work
April 10-11, 2018: Training
April 11-13, 2018: Tutorials & Conference
Beijing, CN

Building deep reinforcement learning applications on BigDL and Spark

This will be presented in English.

Arsenii Mustafin (Fudan University)
14:0014:40 Friday, April 13, 2018
实施人工智能 (Implementing AI), 英文讲话 (Presented in English)
Location: 多功能厅2(Function Room 2) Level: Intermediate
Secondary topics:  增强学习(Reinforcement Learning), 深度学习(Deep Learning)

必要预备知识 (Prerequisite Knowledge)

基本理解BigDL和Spark。
A basic understanding of BigDL and Spark

您将学到什么 (What you'll learn)

能理解如何用BigDL和Spark进行增强学习。
Understand how to do RL on BigDL and Spark

描述 (Description)

本讲话将用英语授课,同时会提供中文同声传译。中文版本摘要会在英文摘要下面给出。

Deep reinforcement learning is a thriving area—DeepMind’s AlphaGo, for instance, has drawn the attention of the entire world. But besides playing games, deep reinforcement learning (DRL) also has many practical applications in industry, such as autonomous driving, chatbots, financial investment, inventory management, and even recommendation systems. Although DRL applications are similar to supervised computer vision or natural language processing tasks, they are unique in many ways. For example, they have to interact with or explore the environment to obtain training samples along the optimization, and the method to improve the model is usually different from common supervised applications.

BigDL, a well-developed deep learning library on Spark, is handy for big data users but has been mostly used for supervised and unsupervised machine learning. Arsenii Mustafin shares his experience developing deep reinforcement learning applications on BigDL and Spark, discussing extensions particularly for DRL algorithms (DQN, PG, PPO, Actor-Critic, etc.). You’ll get tips on how to build a RL application for your own use case.


深度增强学习(Deep ReinforcementLearning,DRL)是当今AI战场里一个蓬勃发展的领域。DeepMind的AlphaGO是DRL一个非常成功的应用,获得了全世界的关注。除了能玩游戏,DRL还能在多个行业有很多应用,比如无人驾驶、对话机器人、金融投资、库存管理、甚至是推荐系统。虽然DRL的应用和监督学习型的计算机视觉或自然语言处理任务在某些方面有相同点,但它在很多方面都是非常独特的。例如,DRL的应用必须要通过和环境的交互(探索)来获得训练样本并优化,而且DRL用于改进模型的方法和通常的监督学习型应用也有所不同。

在本演讲里,我们会分享我们使用BigDL/Spark来构建深度增强学习应用的经验。BigDL是一个基于Spark的完善的深度学习库,对大数据用户而言非常用以上手。通常BigDL被用于监督和无监督的机器学习。不过我们特意为DRL算法(例如,DQN、PG、PPO、Actor-Critic等算法)进行了扩展,实现了经典的DRL算法,并用它们构建了DRL的应用,还进行了性能调优。我们很高兴来分享在做这些的过程中学到的东西。希望我们的经验能帮助听众学会如何自己为它们的业务构建一个增强学习的应用。

Photo of Arsenii Mustafin

Arsenii Mustafin

Fudan University

Arsenii Mustafin is a Russian PhD student at Fudan University, where he specializes in economic studies and data analysis.