Amazon wants to get rid of the barcode: an image-based system is more accurate

The barcode is the system that for almost 50 years has allowed logistics all over the world to identify a product, managing its entry into the warehouse and delivery to the end customer. After two years of development Amazon is starting to replace it with multimodal identification (MMID), which uses the recognition by the appearance and size of the article, and that has arrived at an accuracy of 99%.

The limit of the barcode is that it can get damaged and sometimes it can even be missing; moreover, to find it you have to handle the packaging until you locate it.

Tell me what you are like and I will tell you who you are

In the case of Amazon, at the moment, we are talking about the item’s barcode, not the one on the delivery package. It is therefore the code that must be scanned when the item enters Amazon’s sourcing and shipping centers (the warehouses) and later when it is picked for the order.

In order to dispense with the barcode and use the appearance and shape of the item as identification data, Amazon had to create a machine learning model based on images of products as they walked along the conveyor belts of its warehouses.

Each image was then translated into a list of numbers, or a vector. Item size has also become a vector. The researchers then developed machine learning algorithms to extract these vectors and match them with the corresponding vectors of the candidate items. At the beginning of the implementation of the system the match rate was between 75% and 80%.

Today, Amazon’s MMID has reached 99% accuracy because the system has been trained to recognize not one item among the millions available on Amazon, but only those among the few dozen in a fulfillment cart following an order.

A refinement of the accuracy rate came during a Prime Day, when several hundred Echo Dots, in two colors: gray and blue, were rolling out of the distribution center every hour. But the algorithm couldn’t tell them apart, because the only difference on the packaging was a blue or gray dot that confused the system.

Amazon has therefore introduced a “confidence score”, according to which a high value indicates a potential mismatch and is equivalent to saying “don’t let the tray pass”, while a low score is equivalent to “I’m not sure about this outcome, I don’t take action”.

In the future also for withdrawals

Amazon says the MMID could be integrated into other components of the fulfillment process, though there are hurdles. On a conveyor belt, the lighting and speed of the items are relatively controlled and constant. In the picking phase, on the other hand, if an operator picks up an item, there are many more variables to carry out the identification in hand. The employee’s hand can make it more difficult to identify the item depending on how he holds it.

The Amazon MMID system is currently in use in the Barcelona and Hamburg procurement centers, with its first trial taking place in the Szczecin, Poland plant.