ELK Engineer Interview Questions for Experienced (2027)

48 ELK engineer interview questions for experienced roles: shard sizing, ILM, data streams, Logstash queues, security and red-cluster scenarios.

Sep 16, 2025 - 17:30
Nov 7, 2026 - 19:27
 0  98
ELK Engineer Interview Questions for Experienced (2027)

ELK engineer interviews for experienced candidates test whether you can design, run and troubleshoot an Elasticsearch, Logstash and Kibana platform in production: shard and index design, lifecycle policies, ingest pipelines, cluster health, security and cost control. This guide has 48 questions with short, direct answers, grouped by level and followed by on-call scenarios.

It is written for engineers with 3 or more years of hands-on logging work who are moving into an ELK engineer, observability engineer or SRE role. If you are still learning the basics, start with our 80+ ELK Stack interview questions (the main question bank) or the most asked ELK questions for freshers. For pure troubleshooting drills, see the real-time ELK interview questions.

Key takeaways

  • Experienced rounds focus on design decisions: how many shards, which lifecycle phases, when to use data streams and how to size nodes.
  • Elastic documents five ILM phases: hot, warm, cold, frozen and delete. Know what each one does and what it costs.
  • Data streams are Elastic's recommended way to manage append-only time series data with ILM.
  • Interviewers expect you to read _cluster/health, _cat/shards and _cluster/allocation/explain output and explain a red or yellow cluster.
  • Logstash questions are about pipeline reliability: persistent queues, dead letter queues, pipeline-to-pipeline and back pressure.
  • Security is not optional: TLS between nodes, role-based access, API keys and audit logging come up in most senior rounds.

How this guide is organised

SectionQuestionsWhat it tests
Core engineering (recap for 2+ years)1 to 14Mappings, analyzers, shards, ingest options
Engineer level (3 to 5 years)15 to 32ILM, data streams, Logstash reliability, Kibana operations, security
Senior and lead33 to 40Capacity planning, multi-cluster design, upgrades, cost
Scenarios41 to 48Red cluster, ingest lag, mapping explosion, slow dashboards

Core engineering questions

1. What does each component of the Elastic Stack do in a logging platform?

Elasticsearch stores and searches the data, Logstash and ingest pipelines parse and enrich it, Beats or Elastic Agent collect it from hosts, and Kibana is the UI for search, dashboards, alerting and stack management. Many current deployments replace individual Beats with Elastic Agent managed centrally through Fleet.

2. What is the difference between a primary shard and a replica shard?

A primary shard holds the original copy of a slice of an index and handles writes first. A replica is a copy on a different node that gives high availability and extra read capacity. The number of primaries is fixed when an index is created (you change it with shrink, split or reindex); replicas can be changed at any time.

3. How do you choose the number of primary shards for a log index?

Work backwards from shard size. Elastic's sizing guidance suggests aiming for shards between roughly 10 GB and 50 GB. With rollover, you usually keep one or a few primaries per index and let rollover create a new index when it reaches a size or age limit, instead of guessing a large shard count up front.

4. What is a mapping, and why should you avoid relying on dynamic mapping in production?

A mapping defines field names and types. Dynamic mapping guesses types from the first document it sees, so a field that first arrives as a number and later as text causes rejected documents, and uncontrolled JSON keys can create thousands of fields. Production indices should use index templates with explicit mappings, and dynamic templates or dynamic: strict where needed.

5. What is the difference between text and keyword fields?

A text field is analyzed into tokens for full-text search. A keyword field is stored as one exact value and is used for filters, sorting and aggregations. Log fields such as host.name, http.response.status_code as a string, or service.name should be keyword. The message body is usually text.

6. What is an analyzer?

An analyzer is a chain of zero or more character filters, one tokenizer and zero or more token filters. It turns text into the terms stored in the inverted index. Experienced candidates should be able to explain why the same analyzer is normally used at index and search time, and how to test one with the _analyze API.

7. What is the Elastic Common Schema (ECS) and why use it?

ECS is Elastic's open specification for field names, such as source.ip, event.dataset and log.level. Using it means dashboards, detection rules and integrations work across sources without per-team field mapping, and correlation between logs, metrics and traces becomes easier.

8. When do you use an Elasticsearch ingest pipeline instead of Logstash?

Use ingest pipelines for parsing and enrichment that fits inside Elasticsearch processors (grok, dissect, date, geoip, rename, script). Use Logstash when you need many inputs and outputs, buffering, complex conditional routing, or sending the same event to several destinations. Elastic Agent integrations ship ingest pipelines, so many teams now run little or no Logstash.

9. What is the difference between grok and dissect?

Grok matches with regular expressions and handles variable formats but costs more CPU. Dissect splits on fixed delimiters, is faster and is the better choice when the log format is consistent. A common pattern is dissect first, then grok only on the field that still varies.

10. What does refresh interval control?

The refresh interval decides how often new documents become visible to search, by default every second on indices that are actively searched. Raising it, for example to 30 seconds, on heavy ingest indices reduces segment creation and improves indexing throughput at the cost of search freshness.

11. What is a node role and which roles matter in a logging cluster?

A role tells a node what work it does. Logging clusters typically use dedicated master-eligible nodes, data nodes split into tiers (data_hot, data_warm, data_cold, data_frozen), ingest nodes and sometimes coordinating-only nodes. Separating master-eligible nodes protects cluster stability during heavy load.

12. Why do you need at least three master-eligible nodes?

Master election needs a majority of voting nodes. With three, the cluster can lose one and still elect a master. With two, losing either one stops elections. Three small dedicated master nodes are the usual production baseline.

13. What is the difference between a query and a filter context?

Query context calculates a relevance score; filter context only answers yes or no and can be cached. Log searches are almost always filters on time range, service and level, so putting those clauses in filter inside a bool query is faster.

14. What is Kibana's role beyond dashboards?

Kibana also handles data views, Discover, alerting rules and connectors, Fleet management, index and ILM management, role management and dev tools. In engineer interviews you may be asked how you would export saved objects between spaces or environments, which Kibana supports through the saved objects export and import features.

Engineer level questions (3 to 5 years)

15. What are the ILM phases and what happens in each?

Elastic documents hot, warm, cold, frozen and delete. Hot takes writes and recent searches, warm holds read-mostly data that can be shrunk or force-merged, cold holds rarely searched data often on cheaper nodes or searchable snapshots, frozen keeps data in snapshot storage and searches it on demand, and delete removes it. Each phase has a minimum age and a set of actions.

PUT _ilm/policy/app-logs
{
  "policy": {
    "phases": {
      "hot":    { "actions": { "rollover": { "max_primary_shard_size": "50gb", "max_age": "1d" } } },
      "warm":   { "min_age": "7d",  "actions": { "forcemerge": { "max_num_segments": 1 } } },
      "delete": { "min_age": "30d", "actions": { "delete": {} } }
    }
  }
}

16. What is a data stream and why is it recommended for logs?

A data stream is a named resource over a set of hidden backing indices for append-only time series data. You write to one name, rollover creates new backing indices, and ILM manages them as one unit. Elastic's ILM documentation calls data streams the recommended option compared to managing indices individually.

17. What can you not do directly on a data stream?

Data streams are append-only. You cannot send a plain index request that updates an existing document by ID through the stream; you use _update_by_query or _delete_by_query, or target the backing index. Every document also needs an @timestamp field.

18. What is data stream lifecycle and how does it relate to ILM?

Data stream lifecycle is a simpler built-in option focused on retention and rollover. Elastic positions it as the lifecycle tool for Elastic Cloud Serverless, where ILM is not available. On self-managed and hosted deployments ILM is still available and gives more control over tiers and actions.

19. How do index templates and component templates work together?

Component templates hold reusable blocks of settings, mappings or aliases. A composable index template matches index patterns, sets a priority and lists the component templates to combine. This lets a platform team own a base ECS mapping while application teams add their own fields.

20. How does Logstash guarantee delivery if Elasticsearch is down?

By default Logstash uses an in-memory queue, so events in flight can be lost if Logstash crashes. Enabling the persistent queue (queue.type: persisted) writes events to disk and gives at-least-once delivery. Inputs that support acknowledgement, such as the Beats input, apply back pressure to senders when the queue fills.

# logstash.yml
queue.type: persisted
queue.max_bytes: 8gb
dead_letter_queue.enable: true

21. What is the Logstash dead letter queue for?

It stores events that the Elasticsearch output could not index, for example because of a mapping conflict, so they are not silently dropped. You can read them back with the dead_letter_queue input, fix the data and reprocess.

22. What are pipeline-to-pipeline communication patterns in Logstash?

Using the pipeline input and output, one Logstash instance can run separate pipelines that pass events to each other. Common patterns are distributor (one input routes to several pipelines by type), output isolator (each output gets its own queue so a slow destination does not block the others) and forked path.

23. How do you tune Logstash throughput?

Measure first with the monitoring API. Then adjust pipeline.workers (defaults to the number of CPU cores), pipeline.batch.size and the JVM heap, simplify expensive grok patterns and remove unneeded filters. If the bottleneck is Elasticsearch, adding Logstash workers will not help.

24. How do you secure communication in an Elastic cluster?

Enable TLS on the transport layer between nodes and on the HTTP layer for clients, enable authentication, and create roles with least privilege. Recent Elasticsearch versions configure security automatically on first start for new self-managed clusters. Use API keys or service accounts for agents and integrations instead of shared user passwords.

25. What is document-level and field-level security?

Document-level security restricts which documents a role can see through a query in the role definition. Field-level security restricts which fields are visible. They let you share one index between teams while hiding, for example, another team's logs or a field with personal data. Check your licence tier, because Elastic gates some security features by subscription.

26. How do you mask personal data before it is indexed?

Remove or hash it in the ingest path. In Logstash you can use the mutate or fingerprint filters; in ingest pipelines use remove, gsub, fingerprint or a redact processor where available. Masking at ingest is safer than relying on field-level security, because the data never reaches the index.

27. What is a snapshot repository and how do you use it?

A snapshot repository is a storage location (shared file system, S3, GCS, Azure Blob and others) where Elasticsearch writes incremental snapshots of indices and cluster state. Snapshot lifecycle management (SLM) schedules snapshots and retention. Snapshots are the supported backup method; copying data directories is not.

28. What are searchable snapshots?

Searchable snapshots let the cold and frozen tiers search data stored in a snapshot repository. The cold tier keeps a full local copy without replicas; the frozen tier caches only the parts that are searched. They cut storage cost for long retention, and they require an appropriate subscription level.

29. How do you monitor the Elastic Stack itself?

Elastic recommends sending monitoring data to a separate monitoring cluster, typically collected by Elastic Agent or Metricbeat, so you can still see what happened when the production cluster has problems. Watch heap usage, GC time, indexing and search latency, rejected thread pool tasks, disk watermarks and pending tasks.

30. What are disk watermarks?

Elasticsearch has low, high and flood-stage disk watermarks (85%, 90% and 95% by default). At low it stops allocating new shards to the node, at high it moves shards away, and at flood stage it puts a read-only block on indices with shards on that node. A sudden write failure with a read-only block usually means a full disk.

31. How do you write a Kibana alert for an error spike?

Create a rule (for example an Elasticsearch query rule or a log threshold rule) that counts documents with log.level: error per service over a short window, set a threshold, and attach a connector such as email, Slack, PagerDuty or a webhook. Good answers mention avoiding alert storms with grouping and a sensible check interval.

32. How do you handle multiline logs such as Java stack traces?

Join them at the collector, not later. Filebeat and the Elastic Agent filestream input support multiline settings based on a pattern, for example lines that do not start with a timestamp belong to the previous event. Joining at the source keeps one event per exception and avoids broken searches.

Senior and lead questions

33. How do you size a cluster for a given daily log volume?

Start with daily raw volume, multiply by expected index overhead and by one plus the replica count, then by retention per tier. Divide by usable disk per node after leaving headroom below the watermarks. Validate with a load test using real data, because compression and mapping choices change the ratio a lot.

InputWhy it matters
Daily ingest volume and peak rateDrives hot tier CPU and disk
Retention per tierDrives warm, cold and frozen storage
Replica countMultiplies storage on the hot and warm tiers
Query patternsDashboards over 30 days need different nodes from 15-minute searches

34. How many shards per node is too many?

Every shard costs heap and cluster state. Elasticsearch limits shards per non-frozen data node through cluster.max_shards_per_node (1,000 by default). In practice, rollover by size, fewer and larger shards, and deleting old data keep you far from the limit.

35. When would you run several clusters instead of one large one?

Split when teams need isolation, when regions have data residency rules, or when one workload (security analytics, for example) has very different performance needs. Cross-cluster search lets users query several clusters from one Kibana, and cross-cluster replication copies indices to another cluster for disaster recovery or locality.

36. How do you plan a major version upgrade?

Read the breaking changes, run the Upgrade Assistant in Kibana, fix deprecations, take a snapshot, and upgrade in Elastic's documented order. Rolling upgrades work within supported version paths; skipping majors usually needs an intermediate upgrade or a reindex. Test client libraries and Logstash or Agent versions for compatibility first.

37. How do you reduce the storage cost of a logging platform?

Drop fields nobody queries, avoid indexing large message bodies twice, use keyword only where needed, set realistic retention, move old data to cold or frozen tiers with searchable snapshots, and force-merge read-only indices. Logsdb index mode, where your version and licence support it, reduces storage for log data further.

38. What is your approach to multi-tenancy?

Choose between index per tenant, data stream per tenant with a namespace, or shared indices with document-level security. Kibana spaces separate saved objects. The decision depends on tenant count (many small tenants means too many shards if each gets its own index) and on how strict isolation must be.

39. How do you integrate ELK with traces and metrics?

Use ECS fields and put trace.id into log lines so Kibana can link logs to APM traces. Elastic APM agents and OpenTelemetry both send data the stack can ingest. Our guide to observability versus traditional monitoring covers why this correlation matters.

40. How would you compare ELK with a Prometheus and Loki stack in a design review?

ELK indexes log content for fast full-text search and rich aggregations, at a higher storage and compute cost. Loki indexes only labels and is cheaper for high volume when queries are mostly by label. Prometheus is for metrics, not logs. Many teams run Prometheus for metrics and choose between ELK and Loki for logs by search needs and budget. See our observability tools overview.

Scenario questions

41. The cluster is red. What do you do?

Red means at least one primary shard is unassigned. Run GET _cluster/health, then GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason, then GET _cluster/allocation/explain for the first unassigned primary. Common causes are a lost node, a full disk, or allocation filters that no node satisfies. Restore the node or disk space first; restore from snapshot only if the data is gone.

curl -s -u elastic "https://es01:9200/_cluster/allocation/explain?pretty"

42. Indexing suddenly fails with "index read-only / allow delete" errors.

The flood-stage disk watermark was reached. Free disk by deleting old indices or adding capacity. Recent versions remove the block automatically once usage falls below the high watermark; on older versions you remove index.blocks.read_only_allow_delete yourself. Then fix retention so it does not happen again.

43. Logs from one service are arriving 20 minutes late.

Find where the lag is. Compare @timestamp with event.ingested, check the agent's output metrics, Logstash queue depth and Elasticsearch write thread pool rejections. Typical causes are a slow grok pattern, a full persistent queue because Elasticsearch is rejecting writes, or the agent backing off after errors.

44. A developer added a JSON payload field and the index now has 4,000 fields.

That is a mapping explosion. Stop the growth by setting the payload field to flattened or object with enabled: false in the template, keep index.mapping.total_fields.limit at a sane value, and roll over so new data uses the fixed mapping. Agree a logging contract with the team so new fields are reviewed.

45. A Kibana dashboard over 90 days times out.

Check which visualisations hit the most shards and use high-cardinality terms aggregations. Options are shorter default time ranges, rollups or transforms that pre-aggregate daily summaries, using keyword fields for aggregations, and putting older data on a tier sized for these queries.

46. After a node restart, the cluster stays yellow for an hour.

Replicas are recovering. Check _cat/recovery, and confirm that index.unassigned.node_left.delayed_timeout and the recovery throttles are not set too low. For planned restarts, disable replica allocation first, restart, then re-enable it, as Elastic's rolling restart procedure describes.

47. Security asks you to prove who deleted an index.

That requires audit logging, which records security events such as authentication and access to indices. If it was enabled, search the audit log for the delete request. If it was not, say so and enable it, restrict delete privileges to a small role, and protect important indices with snapshots.

48. Costs doubled after a new team onboarded.

Break down storage and ingest by data stream or index pattern. Look for debug-level logs in production, duplicated fields, missing ILM policies and too many replicas. Set per-team retention in the policy, push debug logs to a short-lived stream, and show each team their usage.

How to prepare for an ELK engineer round

  1. Build a three-node cluster on containers, enable security, and break it on purpose: fill a disk, stop a node, create a mapping conflict.
  2. Write one ILM policy with rollover, warm and delete phases and attach it to a data stream through an index template.
  3. Run Logstash with a persistent queue and dead letter queue and watch what happens when Elasticsearch is unavailable.
  4. Practise explaining one design you built: volume, retention, shard count and why.

Our ELK Stack training covers these labs in the Kharadi, Pune classroom and live online. For the wider interview picture, the 200+ DevOps interview questions hub links every topic, and ELK for centralized logging explains the basics.

FAQ

What does an ELK engineer do?

An ELK engineer designs, runs and tunes Elasticsearch, Logstash, Kibana and the collectors that feed them. Day to day that means index templates, lifecycle policies, ingest pipelines, cluster health, access control, upgrades and helping teams search their logs.

How many shards should a log index have?

Size shards instead of picking a number. Elastic suggests shards of roughly 10 GB to 50 GB. With rollover on a data stream, one or a few primaries per backing index is common.

Is Logstash still needed if I use Elastic Agent?

Not always. Elastic Agent integrations ship ingest pipelines that parse common formats inside Elasticsearch. Logstash is still useful for complex routing, buffering and sending events to several destinations.

What are the ILM phases?

Hot, warm, cold, frozen and delete. Each phase has a minimum age and actions such as rollover, shrink, force-merge, searchable snapshot or delete.

Why is my Elasticsearch cluster red?

At least one primary shard is unassigned. Use the cluster allocation explain API to see why, which is often a missing node, full disk or an allocation rule no node can satisfy.

Which Elastic certification fits an ELK engineer?

Elastic offers an Elastic Certified Engineer exam. Check Elastic's own certification page for the current objectives and format before booking. Our ELK certification questions post covers exam-style practice.

References

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0
Mridul I am a passionate technology enthusiast with a strong focus on DevOps, Cloud Computing, and Cybersecurity. Through my blogs at DevOps Training Institute, I aim to simplify complex concepts and share practical insights for learners and professionals. My goal is to empower readers with knowledge, hands-on tips, and industry best practices to stay ahead in the ever-evolving world of DevOps.