Skip to 0 minutes and 14 seconds After the success of DeepMind, researchers start to explore other applications using deep reinforcement learning. Remember that DeepMind uses Atari games to demonstrate deep reinforcement learning. After that, researchers have started to use more powerful 3D engines, and applied virtual learning techniques to other computer vision applications, including image segmentation, indoor navigation, autonomous driving, and many others.
Skip to 0 minutes and 45 seconds There are two examples below: The UnrealCV used Unreal engine to render labelled object data for learning semantic segmentation; CAD2Real trained models in a virtual world created by AutoCAD, and successfully deployed on a real drone. Most important of all, researchers found that the models trained in virtual world can be deployed in real world. Let’s see some examples from Pan’s BMVC paper. The pictures on the left side are virtual images, and the pictures on the right side are real images, As we can see, the virtual images and real images are almost the same after semantic segmentation.
Skip to 1 minute and 28 seconds That shows how easily we can deploy models trained in virtual world to real world Since the virtual-to-real learning has a lot of potentials, big companies and researchers start to build virtual environments for training deep learning models. Microsoft creates Project Malmo and AirSim Project Malmo is based on Minecraft. which is a very flexible game that allows users to build their own world and create their own game rules. Facebook created House3D, HoME based on Matterport’s 3D scanning and reconstruction technology. DeepMind built DeepMind Lab based on Quake, an old 3D shooting engine (game) Intel created MINOS, which is also based on Matterport technology Unity created Unity AI using their own 3D engine and created tasks which are like OpenAI Gym’s.
Skip to 2 minutes and 25 seconds There are also many open-source environment from Academia. The most popular one is OpenAI gym. We also support OpenAI gym API for deep reinforcement learning. As we can see, there is a need for open-source real-world simulator. To fill this gap,
Skip to 2 minutes and 43 seconds we developed an open-source VR engine called VIVID: Virtual Environment for Visual Deep Learning. Our work won the best open source software award of ACM Multimedia 2018. VIVID is based on the Unreal engine.
What is the real world look like for drones? We can see from the slide. Prof. Lai will talk about applications using deep reinforcement learning.
Nowadays, big companies and researchers start developing many virtual environments Industry:
- Microsoft: Project Malmo (Minecraft), AirSim
- Facebook: House3D, HoME
- DeepMind: DeepMind Lab
- Intel: MINOS
- Unity: Unity AI
In Academia also there are a couple famous example.
- OpenAI Gym
- Deep GTA V (DeepDrive)
- AI2-THOR, Sim4CV, UnrealCV, Stanford 3D, VizDoom
From the example above, there seems a need for companies or researchers to develop or improve virtual environments more. Thus, an open-source real-world simulator becomes important. Prof. Lai will continue to introduce VIVID: Virtual Environment for Visual Deep Learning.