Amazon’s multi-modal identification technology is designed to help robotic arms and robots at Amazon fulfillment centers identify and process products for faster delivery times.
Amazon has decided to upgrade its fulfillment process via a highly sophisticated Multi-model Identification system that aids the robotics arm in identifying packages without relying on the conventional package identification, barcodes, that have been around for over 50 years.
The reason for creating Amazon’s Multi-model Identification system
The manual identification of items on the conveyor belts did not produce good enough operational efficiency. The new algorithm-based system would ensure fewer slowdowns in the fulfillment centers, helping with operational efficiency and customer satisfaction.
Previously, fulfillment center employees would use barcodes to identify the item’s identity at several stages. Missing barcodes would become a problem, especially when tons of item catalogs need to be processed. According to Amazon’s press release, the company wanted to completely eliminate manual identification.
What is Amazon’s Multimodal Identification system?
Multimodal Identification or MMID extracts critical insights about a package, such as its dimensions and appearance, from its image to quickly process it.
Here’s how it works:
The system features an algorithm that matches an item with its photograph. So, instead of looking at the barcodes, robotic arms and robots featuring the MMID sensor match items with their images.
To make that happen, the science and robotics team at Amazon built a huge library of images by taking pictures of the items as they moved along the conveyor belt.
Each photograph was then translated into a machine-readable list of vectors and numbers. For example, the dimensions of an item were translated into vectors.
Then the team worked on the machine-learning algorithm to match the image vectors with the product vectors. Initially, the match rate was 75%, but it improved to 99% (after extensive scientific investments).
The new system makes the process fast and non-intrusive. Each item belongs to a tote that contains a few products. So instead of matching items against all catalogs, the algorithms only match an item with a single tote. Additionally, to avoid interruptions, the company recycles mismatched items back into the system to their correct locations, according to Doug Morrison, a Robotics AI applied scientist at Amazon.
While it is a sophisticated technology, initially it did face problems. During amazon’s Prime day promotion, Hundreds of Echo dots came in two colors, blue and gray. The algorithm could not distinguish between the two. The robotics team created a confidence score for each identification where a high score would indicate a high mismatch, and a low score would mean no action because the system is not sure about a particular item.
In the future, Amazon will be integrating MMID into its other operational components to further enhance the efficiency of its fulfillment centers.