Last month, we announced Search by Image, which allows you to search using a picture instead of typing in words. Today, we’re giving you a look under the hood at how Search by Image uses computer vision technology to analyze your image, determine what it is, and return relevant results to you.
Computer vision technology is an active area of computer science research because it’s difficult for a computer to match a person’s ability to see and understand. Search by Image uses computer vision techniques to “see” what is in the image. Computer vision technology doesn’t look at the image filename or where the image came from -- rather, it looks at the content of the image itself to determine what that image is.
When you upload an image to Search by Image, the algorithms analyze the content of the image and break it down into smaller pieces called “features”. These features try to capture specific, distinct characteristics of the image - like textures, colors, and shapes. Features and their geometric configuration represent the computer’s understanding of what the image looks like.
These features are then sent to our backend servers and compared against the billions of images in our index to see if a good match exists. When the algorithm is very confident that it’s found a matching image, you’ll see a “best guess” of what your image is on the results page. Whether or not we have a best guess, you’ll also see results for images that are visually similar -- though they may not be related to your original image.
Check out this video below for an animated look at how Search by Image works.
Because our algorithm sees the world through the “features” that are extracted from images, those define what it can “see” well and what it can’t. We’re more likely to find a good match if your image query has a unique appearance, so landmarks like the Eiffel Tower work really well. Other things that lack distinctive features or a consistent shape, like a crumpled blanket or a puppy, don’t result in confident matches, but will return images which are visually similar in appearance. You can refine your results in those cases by giving the algorithm a hint. Add a word or two into the search box that describes the image, and the results may display better “Similar Images” results.
The results page summarizes a variety of information that we can match for your query image, including our best guess for the image, related web results, and images that are visually similar to the one you’ve uploaded.
0 comments:
Post a Comment