A Comprehensive Study of Elasticsearch

With the ever-increasing demand for data storage, querying and retrieving data from abundant data sources is a tedious and time-consuming task. Hence, we require a system for querying data that is highly available, has high capacity and can scale out easily without the need to add more hardware onto a single device. In the paper, we discuss one such heavy full-text search and analytics engine called Elasticsearch. Elasticsearch is designed to work with various types of data such as structured, unstructured, geospatial, graphical and numerical data. It was built on top of Lucene and has been improvised with better features. The power of Elasticsearch is amplified with the help of a number of technologies that provide a visualization platform, data processing pipeline, monitoring, machine learning, data shipping etc. They, together with Elasticsearch, are called the Elastic Stack (ELK Stack). Comparison of Elasticsearch with other recent search engine technologies such as Solr, Sphinx and Azure search is provided, which would help readers better understand which technology to choose. Elasticsearch is being used in a number of organizations today as a powerful search engine and has been preferred over databases like MongoDB for querying over stored data, both being JSON document oriented, distributed datastores. But Elasticsearch provides a better searching capability like full-text search unlike MongoDB which is only preferred for CRUD operations. Elasticsearch is also relatively very fast compared to its counterparts and comes with real-time search capabilities thereby having negligible latency, hence making it viable to analyze billions of documents within a few seconds. Besides that, it also has a high throughput, being able to search through and analyze a number of documents concurrently within a limited response time. Elasticsearch also deals with failure of any node of a cluster and loss of shards on it by replicating primary shards into a number of replica shards and distributing them across multiple nodes. This distributed nature of Elasticsearch makes it highly available and robust.