How to combat fashion infringement with image recognition

The following is part of the transcript of the webinar, featuring a conversation between Red Points’ Daniel Shapiro and Juan Galdeano.

Image Recognition versus Fashion Infringements

Today, we will be covering three areas – Look-alikes in the fashion industry, a challenging problem, employing image recognition to protect brands, a new solution for an old problem, and a brief demo to show you what it looks like in real life. So, the main topic this afternoon will be the look-alikes in the fashion industry, and what we do to solve that challenge. 

Defining the problem with fashion infringements

Sometimes one of the biggest challenges is getting on the same page as it can feel like there are many names for these infringements to take place. Different people use different terms. So, for the purpose of this webinar today, I thought we would discuss terms that we could use throughout this presentation that are the same. They may be different from what you use in your own brand or your own practice, but for today’s purposes we’ll use these particular definitions. 

So counterfeit or replica, those are products that use the trademark of your brand, that purport to be original or authentic and try to really fake out the consumer that it’s actually an original item. Look-alikes, or copycats, or knockoffs are generally a product that is designed to look just like the brand, but in often cases, they remove the logo or the trademark, so that when listing the product, it doesn’t seem to have your brand and title or doesn’t seem to have your brand on the product but looks confusingly similar for sure. And then lastly, the challenge of these infringing listings, in some cases, the brand, as I mentioned previously, isn’t in the title. So, one of the things that we find many people do in the online marketplace world is begin to remove the brand name in the title of the item, so that it becomes a circumvention technique used by counterfeiters to avoid detection. And today that has become one of the frustrating elements in companies that are designed to help brands solve counterfeits. And it’s one of the biggest challenges for brands today is actually finding those products when the name is missing.

The challenge of detecting fashion infringements online

One of the thought processes today why this is such a big challenge is, with the multiple marketplaces and social media platforms, there are billions of billions of listings out there. So, when people hide behind using the brand name or don’t use the brand name, searching that vast size of e-commerce can be extremely challenging. So, one of the things we want to talk about is this is one of the oldest problems in the challenge of keeping up with detection today and enforcement. Our options have really just been in the past, you know, keywords and coming up with new keywords to find out how counterfeiters are evading this detection. Today, we’re talking about how do we change the game? What do we do differently to make it more difficult for counterfeiters, and maybe in one instance, meet the pace of these counterfeiters by deploying all sorts of new technology to manage that. So today, we deploy things like machine learning to help generate new search terms and different pieces of artificial intelligence to keep us in pace.

Image recognition and the future of detecting fashion infringements

(…) We’re trying to talk about and trying to show you how we use machine learning techniques in order to simplify and automate the process of detection and validation of the infringement that we can find in the Internet. The technology that we’re going to see is about image recognition, image fingerprinting, image searching, and logo recognition. None of these techniques alone can solve the problem. So, we mix all of them in order to get to get good results. Imagine that this is the space that we are looking for. So every dot in this slide is just a listing, it could be a counterfeit, it could be a legit product, it could be whatever, but we have to find the infringers inside this space. So, the initial approach is we’re using a targeted search keyword. That’s very specific search keywords trying to be very, very accurate in the search. So, when we detect, we detect exactly the asset that we are looking for. But that have a problem. The matching result is high, so it’s very accurate, but we are missing a lot of infringement. 

So, how do we deal with this? We need to use broader search keywords, that means that we are going to put in the platform a lot of listings, tons of listing, what is the problem. In these days, we are talking about millions of listings. That is impossible to work in a manual work to validate or assure that the listing contains the asset that we are protecting. We introduce all of them in the system with some matching results. How do we get these matching results? Using different techniques, like image recognition. We train models to try to find out the object of the asset that we are protecting inside different images. If the score that we get from these images is high enough and we are confident to move forward, we can qualify in this lesson that this is the acid we’re looking for and then start with the validation process automatically. For example, this is image recognition fashion. We train, the model is using specific characteristics of the products, not even the product itself, the full product, maybe just part of the product. And we try more with different with different images, good images and bad images, and we get results where this three different images all tell me that the product is included in the in the listing. 

Juan, may I ask you just a quick question? Not to interrupt you, but I’ve had you, you showed me a lot of examples in the past where sometimes you will even find this in the background, it may not even be the picture in the front. Is that true sometimes?

[Juan Galdeano]  

Yeah, it’s true, because we don’t train with the full image, and we train with a lot of different images containing the bottle. We can train with, like you guys see, with the hand holding the bottle, with the person drinking from the bottle in a backpack, in a set of bottles, even in a banner or sections of the bottle, we tried to find out in this case, this part of the image we train models with this part because it’s the unique part for this article. So, every image that contains this will be validated as okay. Imagine that we have the same bottle but without this part. We are not going to detect this one because we train the model with this one. Usually it’s easier to do this. Imagine we can train models using the model set, the person, and not the product, because usually the person is hired by a brand to show the product. So, we can train with the face of the model. 

What is image fingerprinting and how does it work?

In image fingerprinting, the other technique, we are just finding exactly the same images. Imagine that we are talking about a couple of images. The infringer will usually use the same image as the original product. So, we can detect this image, even variations of the same image. For example, the first case is easy because it’s exactly the same way. The second clearly is not the image, so we’re going to remove this one. The third is the same product with different colors. For fingerprinting, this is still valid image, so we still move forward with this one. And even with a slight difference, like this one without the brand, you see that the logo is not here anymore, we still know that this is still the product, is the image of the product. So, we can validate this also. Even with collage of images or with cropped images, we can do it, or flipped images, that kind of things. Everything works with fingerprinting. But the difference between recognition and fingerprinting is in this case we are looking for exactly this image in fingerprinting, and in recognition we’re looking for specific parts. So, it’s wider, image recognition, than fingerprinting. 

How Logo Recognition or Brand Identification can detect infringements

The other technique that we use is logo recognition. Logo could be present in the image, or in the box, or even in the banner inside the advertised. So, we gather all the logos that are present in the images, and we score those logos too – about the size, about the quantity of time that appears in the image, the positions, all of this, and give a score. In this case, as you can see, the first thing that we did with the shoe. The logo is present. In the t-shirt, it’s present. But in the older one it’s not present. So, because of this, we complement all technologies, and we use logo recognition, image recognition, and image fingerprinting because, in some cases, one is valid and the other, no.

[Daniel Shapiro]  

Juan, may ask you one other question with logo recognition? You and I were having a conversation about sometimes we see counterfeit listings on certain marketplaces or social media sites starting to use maybe like a WhatsApp number or a Facebook logo because they want to maybe take you off the platform and move you on to a private Facebook to transact this counterfeit. Could we use this logo recognition to identify inside of a listing where someone is listing a product but has a Facebook or a WhatsApp, or a telephone number trying to take you off the platform to a private area to transact this good? Would you be able to find that with that logo recognition?

[Juan Galdeano]

Sure. Even we have two different approaches in local recognition. One is the one that I just explained. But the other one is what you said, because usually no legit provider or seller use WhatsApp or Instagram or Twitter accounts in the images. Only the counterfeiters do that because they want to hide from the standard text technologies, then text numbers about the telephone number or Twitter account or the Facebook account or the Instagram account. So, it’s another indicator that the product is probably a counterfeit or the replica or something if we found this kind of logo. And this logo allows us to use combining logo recognition with OCR to determine the number itself, and we gather that information to a stored information related to the seller, the Twitter account the same, WhatsApp or whatever. So, it’s a very, very valid usage of logo recognition, not only for brand’s logo, also for social media platforms or marketplaces, or whatever.

[Daniel Shapiro]  

Thank you, everyone. 

[Juan Galdeano] 

Thanks, everyone. Bye.