Google to use MUM to improve visual search, offer broader result fields

Google’s latest learnings in artificial intelligence will now be visible in search products, triggering what it thinks will be “new ways to search and explore information in more natural and intuitive ways”. As Google rolls out Multitask Unified Model, or MUM, users will start seeing newer ways to search visually, and will also be able to get a wider understanding of a topic they are searching for.

With the new visual search capabilities, said a new blog post by Senior Vice President Prabhakar Raghavan that coincides with Google’s annual Search On event, Google users will be able to just tap on the Lens icon while looking “at a picture of a shirt, and ask Google to find you the same pattern — but on another article of clothing, like socks”.

The post explained that this helps when the search is hard to describe accurately with words alone. You can also use this visual search to point at a particular part, the name of which you don’t know, and get a tutorial on how to use or fix it.

Liz Reid of Google Search explained that the new AI capabilities bring in three things. “One, what is really the question behind your query and how do we understand that; and to be able to do that in new ways and not just with text, voice or images. The second is helping you ask the next question when sometimes you don’t know what that should be. The third is just making it easy to explore information… the web is amazing with this vastness, but sometimes it can be a little bit overwhelming,” she said.

The blog said it will also become easier to explore and understand new topics with “Things to know”. For most topics, Google will use its learnings to shows the aspects it knows people are likely to look for at first. Google promises that soon “MUM will unlock deeper insights you might not have known to search for”. It will also start showing new features to refine and broaden searches and offer a newly designed, browsable results page to make it easier to get inspired.

In video, Google will start identifying related topics “with links to easily dig deeper and learn more”. The post said, “Using MUM, we can even show related topics that aren’t explicitly mentioned in the video, based on our advanced understanding of information in the video”.

Asked how Google will contextualise when the search is coming in a different language or from a location where the sensibilities will be different, Reid told “What MUM understands really well is the concepts, so it can take your query and sort of map it, but then actually connect it to related information that might be expressed differently in another language. The fact that it is cross-training with all of the different languages together also makes it easier.”

At the moment, MUM has visibility on 75 different languages and understands queries in all of these.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top