Visual Search Guide: Table of Contents

• Visual Search Definition
• Visual Search Statistics
• Visual Search for Retailers
• Visual Search Marketing: Brands Using Visual Search Today
• In the News
• Visual Search Acquisitions and Fundraising
• Pinterest Visual Search News
• Google Visual Search News
• Amazon Visual Search News
• Instagram/Facebook Visual Search News
• Quotes
• Charts and Graphs
• Trends
• Challenges
• Visual Search APIs
• Visual Search Engines
• Further Resources
• Get in touch!

What is Visual Search?

Visual search uses real-world images (screenshots, Internet images, or photographs) as the stimuli for online searches.

Modern visual search technology uses AI (artificial intelligence) to understand the content and context of these images and return a list of related results.

It has a host of applications in the eCommerce industry, particularly for fashion and home decor retailers. Visual search allows retailers to suggest thematically or stylistically related items to shoppers in a way they would struggle to do using a text query alone.

Pinterest, Google, and Amazon are the main visual search engines today. Microsoft has also developed impressive computer vision capabilities for its Bing search engine.

A host of retailers including ASOS, Wayfair, Neiman Marcus, Argos, and IKEA have all built proprietary visual search tools.

You can read a little more in the recent Raconteur article, ?What is visual search?? here.

Why is Visual Search Important?

Visual search has the potential to change the way we interact with the world around us.

Our culture is already dominated by the visual, so it seems natural that we would use an image to start a search.

After all, when we shop offline, we rarely start with text. Visual search brings that sense of visual discovery to the online world.

Moreover, we often want to find a new look, outfit, or theme, rather than a single object. Visual search technology helps to tie these items together based on aesthetic links in a way text has never been able to capture.

Visual search strategy for fashion. Fashion brands can now engage with users through mobile phone cameras.

Google, Amazon, Pinterest, Bing, and a host of retailers have all developed their own visual search engines and their collective weight will surely see the technology?s use take hold very soon.

As part of a broader trend, visual search can be packaged under ?sensory search? alongside the well-documented growth of voice search and the introduction of more tactile interactions with websites.

The technology is still in its infant stages, but recent trends reveal just how quickly it can advance. In particular, Pinterest?s visual search technology (known as Pinterest Lens) and Google?s visual search (also, imaginatively, called Lens), are leading the way.

So, I have compiled a list of particularly interesting, newsworthy, and/or impressive visual search facts and stats here.

I?ll keep updating it frequently and hopefully it will serve as a useful resource for those who wish to understand how visual search can affect the everyday lives of consumers. Or just for those who need a few visual search statistics for a presentation.

Visual Search Statistics

• 90% of information transmitted to the human brain is visual. MIT
• The human brain can identify images seen for as little as 13 milliseconds. MIT
• The image recognition market will grow to $25.65 billion by 2019. MarketsandMarkets
• 62% of millennials want visual search over any other new technology. Visenze
• 21% of advertisers believe visual search is the most important trend for their business in 2019. Marin Software
• 45% of retailers in the UK now use visual search. Tech HQ
• 36% of consumers have conducted a visual search. The Intent Lab
• 35% of marketers plan to optimise for visual search through 2020. Search Engine Journal
• 55% of consumers say Visual Search is instrumental in developing their style and taste. Pinterest
• By 2021, early adopter brands that redesign their websites to support visual and voice search will increase digital commerce revenue by 30%. Gartner
• Visual information is preferred over text by at least 50% of respondents in all categories except for electronics, household goods and wine and spirits. The Intent Lab
• 59% think visual information is more important than textual information across categories (vs. 41% who think textual information is more important). The Intent Lab
• When shopping online for clothing or furniture, more than 85% of respondents respectively put more importance on visual information than text information. The Intent Lab
• 20% of app users make use of visual search when the feature is available. GlobalData
• The Global Visual Search Market is estimated to surpass $14,727m by 2023 growing at CAGR +9% during the forecast period 2018?2023. Report Consultant

Google Lens Search Statistics

• Images are returned for 19% of search queries on Google. Moz
• Images are returned for 34% of search results. Econsultancy
• Google Lens can detect over 1 billion objects. TechSpot
• Google Shopping ads see budget increase of 33%, versus a rise of just 3% for text ads. Merkle report, Q3 2018
• Google Lens has been used over 1 billion times. Google

Pinterest Visual Search Statistics

• Brands can now target over 5,000 categories through visual search advertising on Pinterest Lens. Business Insider
• Image-based Pinterest Ads have an 8.5% conversion rate. Heap Analytics
• Pinterest projected to clear $1 billion in ad revenues per annum by 2020. eMarketer
• 21% of Pinterest users use traditional search less when they have an option to use visual search. Raymond James study
• There are over 600 million visual searches on Pinterest every month. Pinterest
• There are 3x as many visual searches on Pinterest as last year. (June 1, 2020) ? Pinterest
• 80% of Pinners (those who use the Lens technology every day) start with visual search when shopping vs. 58% of non-Pinners. Pinterest
• When shopping online for clothing or furniture, over 85% of respondents put more importance on visual information than text information.
• 49% of Pinners say they develop a better relationship with brands they love through visual search.
• 61% of consumers say visual search elevates their experience while in-store browsing.

Visual search statistics infographic

Visual Search Technology: Retailers

• ASOS Style Match allows shoppers to purchase using images. Engadget
• eBay debuts visual search technology. Business Insider
• Neimann Marcus reports strong quarter after developing visual search technology. ZDNet
• Ted Baker shoppable videos. Ted Baker
• Blippar uses visual search to help customers find their way. TechCrunch
• EasyJet launches visual search tool. Digiday
• Levi?s and Disney experiment with visual search on Snapchat. Mobile Marketer
• Walmart is testing an in-house visual search technology. TechCrunch
• Salesforce launches Einstein visual search for retailers. MarTech Today
• Syte launches new visual search tool for retailers. PR Newswire
• Houseology launches new visual search technology. Houseology
• CarStory launches first automotive visual search app: Business Wire
• Snapchat?s Lens Creative Partners program helps brands find AR filter creators. Marketing Land
• Farfetch launches a visual search app, allowing users to search for images they found on Instagram or Pinterest. Luxury Daily
• Marks and Spencer launches new mobile visual search technology. Essential Retail
• AI Startup Syte Helps Retailers Enable Visual Search Capabilities. PSFK
• Lifestyle adds new retail technologies, including voice and visual search. India Retail
• ViSenze brings visual search to Samsung smartphones. RLI
• Slyce debuts visual search for grocery stores. Fresh Plaza
• Argos launches AI-powered visual search to connect offline stores with online catalogue. Internet Retailing
• Lush demos visual search app and fresh ?digital packaging? at SXSW. Mobile Marketer
• LexSet debuts new visual search software. Furniture Today
• The brave new world of computer vision: What retailers need to know. RIS News
• 5 ways e-commerce brands can leverage the growing demand for visual search. Smart Insights
• Can Visual Search Help Retailers Find New Sales? PYMNTS
• Retailers wear Artificial Intelligence: Visual Search and the future of the shopping experience. Sales Layer
• Argos launches a visual search service for app users. Tech Advisor
• IKEA revamps app to allow shoppable visual search at home. Reuters
• Balmain launches range of virtual models for new campaign. Independent
• Walmart bets on shoppable video. Digiday
• Este Lauder, Sephora unveil shoppable AR makeup try-ons on Pinterest. Mobile Marketer
• YNAP aims to add virtual try-on through visual search ? Essential Retail
• Uniqlo Is Using Customers? Photos to Create Tailored Product Recs ? PSFK
• Pixyle.ai offers visual search, automatic product tagging, and similar recommendations ? Pixyle.ai
• Taxonomy ? the new must-have for fashion tech ? Vogue Business
• [PODCAST] Can Visual Search Help Retailers Find New Sales?

Visual Search Marketing: Brands Using Visual Search Today

• Forever 21 increases AOV by 20% through visual search. Retail Touch Points
• BooHoo enjoyed an 85 percent higher conversion rate for those using Camera Search, as compared to those who did not. Grit Daily
• Wayfair uses visual search to improve the customer experience. Forbes
• Walmart Turns Its iPhone App?s Barcode Scanner into an Augmented Reality Price Comparison Tool. Next Reality
• PropertyGuru launches its own ?Lens? visual search tool: Tech in Asia
• Tommy Hilfiger?s image recognition app drives new web traffic and engagement: The Current Daily
• IKEA launches a Pinterest-embedded catalogue. Biz Journals
• YouTube launches AR for cosmetics. Charged Retail
• French retailer Conforama launches visual search powered by Syte. Business Wire
• How Samsung is giving brands the edge with fashion visual search. The Drum

Visual Search in the News

• This AI can find Waldo. The Verge

• How visual and voice search are revitalizing SEO. Marketing Land
• Forget AR: Visual search will be the hottest e-commerce tech among millennials. Marketing Tech
• Visual search is on the verge of a breakthrough. Digiday
• Visual search will make a big impact in ecommerce. MediaPost
• Is the world ready for the era of visual search? The National
• 3 ?Vs? of the future ? voice, visual, and video ? take center stage at CES. The Drum
• PropertyGuru launches PropertyGuru Lens in Asia. Digital News Asia
• Visual search leader Slyce acquires shopping platform Allyke. PR Newswire
• Wescover is making the world a showroom. Lonny
• As visual search takes off, brands adapt to shifting demands. Street Fight
• Bing updates its visual search capabilities. Search Engine Journal
• Publishers can use visual search to reinvigorate their offering. Ad Age
• Visual search and local search: A match made in Mountain View. Street Fight
• Search is getting smarter. Think with Google
• The trillion-dollar visual search opportunity. Grit Daily
• [2019 STUDY] Who Has the Best Image Recognition Engine? Perficient
• [2019 Report] Gartner Optimistic About Visual Search As An Emerging Technology ? Media Post
• Snapchat launches new ?Scan? visual search tool. Digital Media Solutions
• Microsoft launches Custom Visual Search for retailers. MS Power User
• Visual Search Hits The Runway At Fashion Week. PYMNTS
• Slyce is piloting a visual search kiosk machine at the Home Depot in South Philly. Technical.ly
• Snapchat partners with PlantSnap and Dog Scanner to improve visual search. TechCrunch
• [PODCAST] eBay joins the visual search party. eMarketer

Visual Search Acquisitions and Fundraising

• Lonely Planet to buy visual search company Trill. Travel Weekly
• Visual Search Leader Slyce Acquires IP and Social-Media Monitoring Technology of Ditto Labs. PR Newswire
• Donde Search picks up $6 million to bring visual search to more retailers. TechCrunch
• Image recognition startup ViSenze raises $20M Series C. TechCrunch

Pinterest Visual Search News

• Pinterest Lens and Home Depot help people decorate a new home. Pinterest
• What makes Pinterest so shoppable? ClickZ
• Introducing the next wave of visual search and shopping. Pinterest
• Celebrating one year of Pinterest Lens. Pinterest
• Pinterest launches features for the visually impaired. Computer Weekly
• Pinterest introduces Product Pins to make visual search your own shopping assistant. Engadget
• Pinterest launches its Complete the Look visual search tool. Venture Beat
• [VIDEO] Total Retail interview with Pinterest?s Amy Vener. Total Retail
• [PODCAST] Pinterest CEO Ben Silbermann on the future of visual search. SoundCloud

Wired podcast: Pinterest CEO on the future of visual search.

• Pinterest fully automates Shop the Look. Venture Beat
• Pinterest launches Catalogues feature to make visual search more shoppable. WWD
• The battle for the top of the funnel: Pinterest?s long road to becoming an ecommerce platform. Digiday
• Pinterest adds Shop tab to visual serch results ? AdWeek
• Pinterest enters the online grocery foray ? National Retail Federation
• Pinterest files for IPO. Some stats from the application:

Google Lens Visual Search News

• Google introduces shoppable image ads. Search Engine Land
• When in nature, Google Lens does what the human brain can?t. Wired
• Copy-paste real world text with Google Lens. Android Central
• Cool things you can do with Google Lens. CNET
• The new PixelBook Pen performs a visual search when you circle an image. 9 to 5 Google
• Google Lens comes to the Pixel 3 smartphone. TechCrunch
• Google Lens comes to Google image search. Search Engine Land
• Google Lens, one year in: Google
• Google launches shoppable images in major move towards visual ecommerce: Google
• Google Assistant adds new, visual search results: Tech Times
• Google Lens to add new translation features and filters: Engadget
• Helpful new visual features in Google Lens: Google
• Google announces a host of new, visual features in search results: Marketing Land
• Google launches Glass Enterprise Edition 2: The Verge
• Here?s what Google?s bet on visual search means for advertisers: Ad Age
• Google Lens and visual search will change your life. AZ Central
• Google Images adds side panel and product details in web revamp. 9 to 5 Google
• Google Shopping tests a Google Lens shortcut and prepares for AR item previews and dark mode. XDA Developers
• Learn more about what you see on Google Images ? Google
• Google takes aim at Pinterest with new ?style ideas?. Social Media Today

• Google Lens announcements from I/O 2019:

Google Lens announcements presentation

• Google Lens due to launch new ?Education? feature to help kids with homework ? XDA Developers

Amazon Visual Search News

• Amazon partners with Getty to provide content for visual searches. TechCrunch
• Snap and Amazon working on a visual search product. Fortune
• Amazon launches SCOUT for visual search browsing. Digital Commerce 365
• Snapchat and Amazon launch shoppable image partnership. TechCrunch
• Amazon debuts Showroom visual search shopping. TechCrunch
• Amazon launches StyleSnap, a new visual search tool for high-end retailers. engadget
• Amazon wants to use AI to recommend you clothing ? again. MIT Technology Review
• Amazon discontinues the Echo Look, migrates visual search style recommendations to other Alexa apps. VentureBeat

Facebook and Instagram Visual Search News

• Facebook launches visual search in Marketplace. TechCrunch
• Instagram adds shoppable images. Ad Week
• Instagram will let you add alt text to images. Search Engine Journal
• Facebook buys visual search technology startup, GrokStyle. engadget
• Instagram may soon launch a Pinterest-esque ?collections? feature. Business Insider
• Instagram takes visual search ecommerce to the next level with one-click checkout. The Motley Fool
• Instagram is letting people buy a handful of brands from a select group of influencers. Bloomberg
• Social shopping is taking off, and Facebook is the biggest beneficiary. Motley Fool
• Powered by AI: Advancing product understanding and building new shopping experiences. Facebook

Visual Search Quotes

The future of search

?The future of search will be about pictures rather than keywords.?

? Pinterest CEO, Ben Silbermann

Visual search: object detection

?In the English language there?s something like 180,000 words, and we only use 3,000 to 5,000 of them. If you?re trying to do voice recognition, there?s a really small set of things you actually need to be able to recognize. Think about how many objects there are in the world, distinct objects, billions, and they all come in different shapes and sizes. So the problem of search in vision is just vastly larger than what we?ve seen with text or even with voice.?

? Clay Bavor, Google

Visual search: user behaviors

?By now people have the muscle memory for taking pictures of all sorts of things ? not just sunsets and selfies but the parking lot where you parked, business cards, books to read. That?s a massive behavior shift.?

? Aparna Chennapragada, vice president of product for AR, VR, and vision-based products at Google.

Visual search: breaking down images

?A field linguist has gone to visit a culture whose language is entirely different from our own. The linguist is trying to learn some words from a helpful native speaker, when a rabbit scurries by. The native speaker declares ?gavagai?, and the linguist is left to infer the meaning of this new word. The linguist is faced with an abundance of possible inferences, including that ?gavagai? refers to rabbits, animals, white things, that specific rabbit, or ?undetached parts of rabbits?. There is an infinity of possible inferences to be made. How are people able to choose the correct one??

? Deepmind

Amazon visual search

?An advantage of visual search is that it relies entirely on item appearance. There is no need for other data such as bar codes, QR codes, product name, or other product metadata.?

? Brent Rabowsky, Amazon Web Services

Visual search: ecommerce personalisation

?You?re going to see products in stock be increasingly personalized and relevant. When you encounter a product on Pinterest, it won?t just be any product, but something that matches your tastes, style and brands you have an affinity for. The challenge with ecommerce personalization is that it only works when you have massive amounts of engagement data, not just transaction data. Our huge advantage is that we have engagement data on products. Pins links to products and as an extension, we can take all the power of personalization and directly map it to products.?

? Tim Weingarten, Pinterest (ClickZ interview)

The evolution of search

?Being able to search the world around you is the next logical step?

? Brian Rakowski, VP Product Management, Google.

Why visual search will take off, according to Amazonhttps://www.bondcap.com/report/itr19/#view/86

This is a new way to shop, allowing customers to browse millions of items and quickly refine the selection based solely on visual attributes. It is perfect for shoppers who face two common dilemmas: ?I don?t know what I want, but I?ll know it when I see it? and ?I know what I want, but I don?t know what it?s called.? Customers simply like an image (by clicking or tapping the thumbs up) or dislike an image (by clicking or tapping the thumbs down) to get tailored recommendations in real time and find the products that best suit their needs. Amazon uses imagery from across its robust selection to extract thousands of visual attributes for showing customers a variety of items so they can select their preferences as they go. This innovative shopping experience is powered by Machine Learning. The result is a beautiful and inspirational image feed, which gives customers the ability to explore a wide range of products in a playful and personalized manner with just a few clicks. It is currently being tested on Amazon.com and in the Amazon App for home furnishings and women?s shoes.

? Amazon statement to CNBC about new visual search platform, Amazon Scout.

?Customers are increasingly using social media platforms, such as Instagram and Pinterest, as a source of inspiration so visual search has the potential to transform how we shop for the home.?

– Mark Steel, Digital Director, Argos

Visual Search Charts and Graphs

Visual Search Use, By Age Group

https://www.retail-insight-network.com/comment/rise-of-visual-search-2019/

Who Has the Best Image Recognition Engine?

https://www.perficientdigital.com/insights/our-research/image-recognition-accuracy-study

Pinterest Visual Search: How it Works

Created by author.

How interested are US internet users in using visual search retail technology?

Emarketer: https://www.emarketer.com/content/how-do-smart-speakers-stack-up-against-visual-based-retail-technologyhttps://www.bondcap.com/report/itr19/#view/80https://www.bondcap.com/report/itr19/#view/79

User preference for similar product results through visual search:

How people use visual search today:

Business Insider: https://www.businessinsider.com/target-to-bring-pinterest-lens-to-app-2017-9

Google Lens in action:

Google Lenshttps://www.bondcap.com/report/itr19/#view/81

62% of millennials would like to be able to search by image:

https://www.emarketer.com/content/visual-search-2018

Number of visual searches each month through Pinterest Lens:

Pinterest Lens: Most popular visual searches of 2017:

Source: https://newsroom.pinterest.com/en/post/celebrating-one-year-of-pinterest-lens

Retailer priorities in 2018:

https://www.businessinsider.com/ebay-debuts-visual-search-for-mobile-app-2017-10?IR=T

Visual search timeline:

Source: https://www.searchenginejournal.com/visual-search-images/213324/

Google Images is the second-largest ?search engine?:

https://sparktoro.com/blog/new-jumpshot-2018-data-where-searches-happen-on-the-web-google-amazon-facebook-beyond/

Pinterest conversion rates:

https://heapanalytics.com/blog/data-stories/4-key-facts-you-should-know-before-allocating-ad-spend

Pinterest engineering:

https://medium.com/@Pinterest_Engineering/building-lens-your-look-unifying-text-and-camera-search-1b2f3ef4e393https://medium.com/@Pinterest_Engineering/building-lens-your-look-unifying-text-and-camera-search-1b2f3ef4e393

Amazon Visual Search:

https://aws.amazon.com/blogs/machine-learning/visual-search-on-aws-part-1-engine-implementation-with-amazon-sagemaker/

Pinterest advertising revenues:

https://www.emarketer.com/content/pinterest-makes-roughly-7-per-userhttps://www.emarketer.com/content/pinterest-makes-roughly-7-per-userhttps://www.raconteur.net/technology/what-is-visual-search

Visual Search Trends

• Integration with chatbots: Visual search starts a conversation; a fact some retailers are exploiting to integrate a chatbot into the shopping experience. Inc
• Big retailers are on board: Target has been using Pinterest Lens for the past year. WSJ
• Pinterest is combining visual search with text search, which should increase its reach.
• Visual platforms like Instagram and Snapchat have a natural tie-in with visual search, if they can make the shopping experience as frictionless as possible.
• Brands (notably, Mastercard) are dropping text from their images altogether in favor of a design-driven expression of their identity.
• Google Search offers an increasingly visual experience, both through the presence of more images and also in the redesigned user interface. Basically, Google Images looks more and more like Pinterest.
• Retailers are developing their own visual search capabilities, rather than relying on search engines and social networks to act as the intermediary.
• Visual search technology (particularly Google Lens) is integrated into a lot more functions within smartphones now. This is a natural development; as it stands, there is something slightly jarring about having to open a specific app, click on a function, then take a picture to access visual search. The point of this technology is that it fits into how we perceive and interact with the world. We should expect to see the use of visual search become more natural and intuitive over time, with more features included within Lens.

Challenges Visual Search Must Overcome

• Reporting and attribution: It is not yet possible to segment search queries to see which used an image. This is a significant challenge that has deterred some brands, but we can also expect it to be resolved with SQRs and Search Console soon.
• User behaviors: People simply aren?t accustomed to searching with an image. It may take some time for this behavioral patterns to adapt, especially as the process of taking a picture before submitting it as a visual search is less than seamless.
• Paid search product: It will be difficult to monetize visual search in the same way as text search, as the targeting mechanism could not focus so directly on keywords. As the industry moves towards audience-based bidding, this hesitation may be addressed. TechCrunch

Visual Search Products and APIs

• Google Cloud Vision.
• IBM Watson Visual Recognition.
• Amazon Rekognition.
• Snap Tech Visual Search for Oracle Commerce Cloud.
• Einstein Vision.
• Bing Visual Search API.
• Clarifai.
• Amazon Web Services DeepLens
• Amazon Web Services visual search code.
• Syte
• nyris
• Pixolution
• Perficient Digital Smart Images Evaluator

Visual Search Engines

• Google Lens.

• Pinterest Lens.
• Bing Visual Search.

• CamFind.
• EasyJet

• Kiddle Visual Search Engine for Kids.

Visual Search Resources

• Pinterest engineering.
• Alibaba engineering: Visual search at Alibaba
• The future of visual search. Econsultancy
• Visual Search 2018: eMarketer.
• Drive conversions with visual search.
• Econsultancy guide to visual search (Download).
• Visual search on Amazon guide: Part 1
• Visual search on Amazon guide: Part 2
• How Snapchatters shop.

https://www.slideshare.net/ClarkBoyd/brightonseo-2018-september-28-how-to-optimize-for-visual-search

• How to optimize for visual search.

Get in touch!

If you need a visual search expert for an article you?re writing, or an event you?re hosting, get in touch via the details below:

Email: [email protected]

LinkedIn: https://www.linkedin.com/in/clark-boyd-digital/

Twitter: https://twitter.com/ClarkBoyd

Previous contributions to visual search content as a subject expert:WARC ? Visual Search Promises Visual Commerce: https://www.warc.com/newsandopinion/news/visual_search_promises_visual_commerce/40510

Interview – Clark Boyd on visual search: https://searchenginewatch.com/2018/06/13/sew-interview-clark-boyd-on-visual-search/

CBC Radio Interview – A New Way to Search: https://www.clarkboyd.digital/thinking/cbc-radio-clark-boyd-visual-search

Search is Getting Smarter ? Think with Google:https://www.thinkwithgoogle.com/intl/en-gb/advertising-channels/search/search-getting-smarter/

Visual Search and the Future of the Shopping Experience: https://blog.saleslayer.com/retailers-wear-artificial-intelligence-visual-search-and-the-future-of-the-shopping-experience

Visual Search: Everything You Need to Know [Live presentation]: https://slideslive.com/38911244/visual-search-everything-you-need-to-know

The Past, Present, and Future of Visual Search: https://medium.com/swlh/the-past-present-and-future-of-visual-search-9178f006a985

How to Optimize for Visual Search [Slide deck]: https://www.slideshare.net/ClarkBoyd/visual-search-how-it-works-and-how-to-optimize-for-pinterest-google-and-amazon