Summary
Elasticsearch is a distributed, RESTful search and analytics engine that stores, searches, and analyzes large volumes of data in near real time, forming the storage and query layer of the ELK Stack.
What is Elasticsearch?
Elasticsearch stores data as JSON documents in indices. Its inverted index structure enables sub-second full-text search across billions of records. Horizontal scaling is built in: adding nodes to a cluster automatically redistributes data and queries across the new capacity.
In the observability context, Elasticsearch stores logs, traces, and metrics forwarded by Logstash or Beats. Its aggregation framework supports powerful analytics — calculating error rates, percentile latencies, and top-N queries — that feed Kibana visualizations.
Elasticsearch is developed by Elastic and available as open source (with a dual SSPL/Elastic License) or as Elastic Cloud. The Elastic Common Schema (ECS) standardizes field names across log sources to simplify queries and dashboards.
Why is Elasticsearch relevant?
- Scale: Handles petabyte-scale log ingestion with horizontal sharding
- Speed: Near real-time indexing and sub-second query responses for operations teams
- Flexibility: Supports structured metrics, unstructured logs, and vector embeddings in one engine
- Ecosystem: Deep integration with Kibana, Logstash, and Beats reduces integration effort