Spanning the reality gap between AI and Industry 4.0

Artificial intelligence has become a booming trend in the industrial sector these days, as automation and optimization continue to be the primary focus of the digital revolution. In this article, we will take a look at the state of the art computer vision techniques that have generated a lot of excitement in AI community in the last few years, and are considered to be industry-ready and are sure to have a significant and practical impact for industrial use cases. Some of these techniques demonstrate incremental yet incredible advancements in performance, surpassing human level performance and thus surpassing precision and reliability standards expected by most industries. The incredible advancement in basic computer vision tasks, such as image classification, have made it feasible to reliably combine multiple techniques to create new, compound techniques that enable entirely new use cases that have never been explored in industrial environments before. That being said, these new techniques have demonstrated that is it possible to obtain precision and reliability results comparable to those that would otherwise only be obtainable with specialized systems that are very hardware intensive. While there are practical difficulties and limitations in implementing these specialized systems and installing the required hardware, cameras are readily available, thereby dramatically broadening the scope of use cases. Computer vision systems powered by AI make it possible to leapfrog into an entirely new domain accelerating the progress towards Industry 4.0 and the true digitization and augmentation of the physical reality.

Object detection

Object detection is a computer vision technique that identifies and locates objects in images or videos. This is typically done by encompassing the objects with bounding labeled boxes. Object detection is a key technology behind self-driving cars, enabling them to recognize other cars or distinguish a pedestrian from a lamppost. It is also useful in a variety of applications such as industrial inspection, and robotic vision. Due to the ImageNet competition, there has been a 1.7× reduction in localization error (from 42.5% to 25.3%) between 2010 and 2014 alone. The video clip below shows the results of a real-time implementation of this technique for the detection of cars, people and other common objects found in a city that are relevant to the vision system of a self-driving car.

YOLO v3: An Incremental Improvement(Redmon et al. Apr 2018)

Keypoint detection and pose estimation

A key point is a feature that is considered as an interesting or important part of an image. They are spatial locations or points in the image that define what is interesting or what stands out in the image. The reason why key points are special is that it is possible to track the same key points in a modified image where the image, or the objects in the image, are subject to rotation, contraction/expansion or deformation.

Pose Estimation is a general problem in computer vision where the aim is to detect the position and orientation of an object. This usually means detecting keypoint locations of the object. This technique can be used to create a very accurate 2D/3D model that describes the position of the key points of the object, which can then be used to create a digital twin that can be updated in real time.

For example, in the problem of pose estimation common boxy household objects the corners can be detected to gain insight into the 3D position of the objects in the environment.

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects (Trembley et al. Sep 2018)

The same can be done for detecting human poses, where the key points on a human body such as shoulders, elbows, hands, knees, and feet are detected.

OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields (Cao et al. 18 Dec 2018)

Semantic segmentation (a.k.a. masking)

The next technique is known as semantic segmentation and it addresses one of the key problems in the field of computer vision as it intuitively separates objects in an image by clearly defining their boundary. Looking at the big picture, semantic segmentation paves the way towards complete scene understanding. This is incredibly useful because it gives a computer the ability to precisely identify the boundaries of different objects. The importance of scene understanding as a core computer vision problem is highlighted by the fact that an increasing number of applications nourish from the knowledge gained with semantic segmentation. In the self-driving car example shown below, it helps the car identify the exact position of the road and other objects.

Super slow motion

Video interpolation aims to generate intermediate frames between two consecutive frames. These artificially generated frames adhere indistinguishably to the same visual characteristics as the original footage. This technique is ideal for amplifying the capabilities of camera systems. Experimental results on several data sets demonstrate that the deep learning approach performs consistently better than existing methods. The results of this technique can be seen in the video clip below, where a smooth slow motion video was created by adding 7 intermediate frames between the original frames.

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation(Jiang et al. Jul 2018)