Cloud Search with Liam Cavanagh

Search is part of almost every application. Users search for movies to watch. Engineers search through terabytes of log messages to find exceptions. Drivers search through maps to find a destination. Search remains an unsolved problem, with lots of room for optimization.

Many search applications have been built Elasticsearch, an open source distributed search engine. Elasticsearch is the code that powers some search-as-a-service products offered by major cloud providers. After eight years of open source development, Elasticsearch is excellent at core search functionalities, such as indexing data, sharding, and serving queries.

With improved access to machine learning tools, search applications can advance in new and interesting ways. For example, an incoming search query can be sent to an API for natural language processing before being served by the search engine. A natural language processing API can derive additional meaning from the query, adding metadata to a search query. Machine learning can also be applied to better understand how people are searching across your search index, and to optimize the search index to incorporate those user preferences.

Liam Cavanagh is the principal program manager on Azure Search. He joins the show to talk about the architecture of a search index, how search queries are served by an index, and how machine learning APIs can be used to improve queries.