Recognising visual patterns, be it a person’s face or a human figure in a road scene, is instinctive to most humans. We’re not aware of the complex neural processes going on in our brains. Trying to teach a machine to do the same thing is incredibly difficult.
Nevertheless, it’s a goal well worth striving for, because, once trained, a machine can look closer and harder. It will never get tired, lazy, distracted or bored. It will carry on relentlessly where people may eventually falter.
For years, giving machines the intelligence to learn to do the things that come naturally to humans has been a holy grail of industrial automation.
Subscribe to PII and receive:
Our long-established bi-monthly digital magazine *Process Industry Informer*, featuring exclusive articles, news and updates across the Process Industries, plus commentary from our three industry experts:
Sean Moran
Gavin Smith
Dave Green
PLUS:
- Process Industry Update — our weekly e-newsletter
- PII Podcast
- Special messages from our advertising partners
Machine Vision is a measurement and inspection tool that is an obvious partner for automated processes because it is founded on teaching a machine to recognise optical features or patterns.
Traditional machine vision systems use “if-then” rules to compare images against a set of pre-determined geometrical or measurement parameters to answer specific questions. The system ‘sees’ a pattern, a colour, contrast, or measurement and compares it to what a ‘good’ product should look like.
But, for products or scenes where there are potentially an infinite number of variations, the expert advice has always been that machine vision had its limits. Despite the obvious need to redeploy people from repetitive tasks, in the past some products, processes or components have been just too varied to teach to a machine.
Removing Barriers to Automation
Deep learning is a branch of artificial intelligence technology that is now offering a way of removing these barriers. Deep Learning machine vision reproduces human responses to more complex visual data that can’t be predicted using these rule-based parameters to solve applications such as defect inspection, localising objects in the camera field-of-view, sorting based on visual appearance or identifying foreign objects in a food product.
Not only is it opening a route to automate more challenging processes, Deep Learning machine vision also promises to enable much greater production flexibility. That’s because it also offers the potential to retrain machines when new products are added, adapt when processes are changed, and respond to a higher variety of variation in items being produced at the same time; these are all key elements of Industry 4.0.
Crucially, Deep Learning technologies are offering the chance to shortcut the development time needed to set up complex optical inspections. That is an especially attractive prospect both to machine manufacturers and to end users because they can cut out tedious and lengthy programming time and costs, especially for more complex tasks.
Guided through specially-developed intuitive interfaces, engineers without specialist skills can more easily teach machines to classify complex or irregular-shaped shapes, patterns or features in a scene. This offers a tantalising prospect of automating machine vision tasks that have previously been too difficult, costly or time-consuming.
The foundations for progress have been laid by developments in vision sensor hardware. Vision sensors have also become intelligent, with processing power onboard so they can be programmed to run applications on the device, or through more localised, edge integrations.
At SICK, we wanted to go even further and make it as easy to download a ready-made “App” to set up and configure a programmable vision sensor as it is on a mobile phone. We have been able to draw on our experience of simplifying vision system applications with customers, and cutting out tedious and lengthy programming time, to launch our first Deep Learning software SensorApp. Called Intelligent Inspection, it operates directly onboard our InspectorP 600 2D programmable vision sensors.
Deep Learning Machine Vision by Examples
These advances are opening up an opportunity for manufacturers to revisit optical inspections that have previously defied automation. So, what kinds of process manufacturing applications would benefit from Deep Learning Machine Vision? That’s quite a difficult question to answer, because every situation is, by its very nature, quite different.
At SICK, our focus has been to simplify optical inspections of complex or irregular-shaped produce, products or packaging. Such processes might provide the ‘eyes’ for a quality check, to sort products or packs into categories, or to direct a robot.
A good example might be of naturally-grown produce, where each strawberry or apple is infinitely varied and reliably checking and grading by quality would be a task very challenging and time-consuming to set up with traditional systems.
In the timber industry, Deep Learning machine vision can be used to optimise the cutting process in a sawmill recognising the annual age rings and other features in the lumber, for example knots in the wood. The sawmill makes the best use of each log and avoids waste, while improving overall product quality.
In packaging machinery, Deep Learning is proving useful in label detection and verification. It is making Optical Character Reading (OCR) systems more flexible and adaptable where there is a wide degree of variation in the label print quality, rather than having to specify many different types of ‘S’, for example.
Inspector P620 Label Inspection application
Deep learning is being used use in a bottle cleaning process to distinguish new bottles from used ones in the infeed zone of the machine. The objective is to optimise water consumption of the machine which operates different programs for new and used bottles.
When it comes to Deep Learning, the demand is not for a universal solution. Rather, the focus is on a solution tailored to a specific case. SICK’s deep learning experts work closely with the customer’s process and quality experts. Their unique process expertise forms the basis of simulation training and the heart of subsequent deep learning algorithms in the sensor.
Deep neural networks are a multi-layered, mathematical algorithms, designed a little like a human brain, that compute vast, complex amounts of patterns and data. SICK’s cloud-based service hosts our own Deep Neural Networks to process the enormous quantity of information required for the vision inspections.
The good news is that all of that processing takes place in the background and the interface for users is remarkably simple given the complexity behind it. Indeed, the time needed to train a deep learning network can be as little as a few hours. The deep learning algorithms generated are placed on the sensor locally via the cloud, making them fail-safe and directly available on an intelligent camera.
How does setting up a Deep Learning system work in practice?
A Deep Learning system is taught by being shown many real-life variations of the same product. For this reason, some people call it an “example-based”, as opposed to a “rule-based” process.
Intelligent Inspection image collection tools collect example images of your product, package, or component in realistic production conditions, prompts teach the system to recognise pass and fail examples, which can then be uploaded to a cloud-based training service.
You can then apply further production images to evaluate and adjust the system. Once you are satisfied, download the custom-trained Deep Learning solution to the device, and the automated inference process will begin, with no further Cloud connection necessary.
Deep Learning is most suitable for products with many variations, such as naturally-grown produce
Unlike traditional vision systems, there is no need to select from the conventional toolbox of algorithms used to identify defects, such as pattern finding or edge detection. Deep Learning cameras can automatically detect, verify, classify and locate ‘trained’ objects, or features, by analysing the complete ‘taught-in’ image library.
However, because it is a seamless extension to the pre-installed Quality Inspection software, with SICK Intelligent Inspection, you can also combine, or extend, the functionality of the inspection further, using traditional machine vision tools, if you wish.
Start Simply and Be Prepared
Deep Learning is still not the ‘catch-all’ silver bullet for every application. Given the potential complexity, our advice would be the same as with conventional machine vision: Start simply. A common mistake is to think: “I want to do everything”. You still have to train the system to classify what is good and what is bad. Make sure you define the task clearly and know the desired outcomes: What are the specifics that I’m trying to detect?
Even with Deep Learning, it’s still important to filter out unwanted variations. Make the environment robust and repeatable, so the same result could be expected every time. That means making sure lighting and the way the machine presents the part to the camera are constants, for example.
The good news is that you now can dip your toes into Deep Learning to test the suitability of your product or, perhaps, extend a current application.
