Visual Search Application

A Product developed from 0 to 1: An Image Recognition App to detect misplaced garments in the warehouse.

Overview

The Visual Search Application was designed for the garment identification in the warehouse using image recognition technology. The application provided a list of styles which matched the picture of the garment with missing or damaged barcodes taken using the app. It enabled the warehouse teams to assign one of the styles in the list to garment.

Problem Statement

When a barcode fell off a garment at the distribution center (DC), the Research department had to follow a very inefficient process to hunt for the specific style-color on the DC floor. Additionally, some garments returned to the warehouse without barcodes, prompting the warehouse team's Research department to manually search for the corresponding style-color. They identified these garments using Inventory data in Netsuite and visually matched them, dedicating approximately 85% of their time to this task. With the future plans to integrate the business with dropship platforms, identifying garments received in the DC, which will be sourced from dropship, was expected to become cumbersome.

Solution

A tablet for the Warehouse team with the Visual Search application to take a picture of the garment with damaged or missing barcode..
An Image recognition algorithm to find a list of matching garments for the user to browse and select.
Feedback from the interaction in the app captured and recorded in the database.
A Tableau dashboard to measure track the performance and success of the application.

My Role

Created technical specifications to develop micro-services using the Image Recognition Algo for the Application.
Defined Scope and planned the first launch of the algorithm.
Identified the Labeled data and Features of the Images.
Reviewed App mockups and defined image validations and minimum image requirements to ensure optimum quality of the input images taken from the app.
Identified the Positive and Negative feedbacks to be recorded for analysis.
Defined metrics and dashboards to measure the success and accuracy of the Algorithm.
Analyzed the performance of the algorithm and created a backlog of features.
Collaborated with Data Science and Analytics, Data Engineering and Product teams.

Key Achievements

The warehouse team can efficiently assign styles to the garments with missing barcodes with a success rate of ~80%.
The image recognition algorithm can be leveraged to build a customer facing application to identify similar garments on the website

Challenges

After the initial launch, the success rate was very low and we were struggling to identify the reason behind it. Luckily, we had a good feedback system created and we were capturing images and logs for each interaction. This helped us debug that even when the picture was taken in portrait mode, internally the iOS system was rotating the image and saving it in landscape mode. It caused the feature extraction to capture incorrect parameters for matching.
Due to the lower success rate for the initial launch of the app we saw a drop in usage of the app by the warehouse team as they were spending more time on the app than the usual process. This was fixed by finding out low effort and high impact features and rolling out in shorter cycles. We were able to achieve around 50% increase in the success rate in less than a month.

Learnings

Data-Driven Decision Making: Implementing performance metrics and dashboards enabled us to monitor algorithm success and make informed decisions. Analyzing data helped identify areas for improvement and track progress over time.
Technical Precision: Precise technical specifications are crucial for the development of ML products like image recognition app. Clear definitions of labeled data, image features, feedback data and data availability are a must when defining such specifications.
Agile Adaptation: Overcoming challenges, such as low initial success rates, required agile thinking and quick adaptation. Implementing user feedback loops and prioritizing high-impact features allowed us to rapidly improve performance.