50+ Splunk FAQs Asked in DevOps Interviews (2027)

52 Splunk FAQs for DevOps interviews: forwarders, HEC, indexes and buckets, SPL for errors and deploys, alerts, clustering and CI/CD log monitoring.

Sep 16, 2025 - 11:25
Nov 7, 2026 - 22:36
 0  57
50+ Splunk FAQs Asked in DevOps Interviews (2027)

Splunk is a platform that ingests machine data such as logs and metrics, indexes it, and lets you search it with Search Processing Language (SPL) to build alerts and dashboards. DevOps interviews focus on forwarders, the HTTP Event Collector, index retention, SPL for error rates and deployments, and clustering. These 52 FAQs cover them.

Splunk often sits next to metrics tools in a monitoring stack. For where logs, metrics and traces fit together, read our guide to observability versus traditional monitoring. Technical details below were checked against the Splunk Enterprise documentation; the docs version selector lists 10.6 as the newest Splunk Enterprise release at the time of writing.

This post is the FAQ format for DevOps engineers. Siblings take other cuts: the 75-question master list, most asked questions, real-time project questions, scenario questions, advanced questions and a preparation guide. For a log-search alternative, compare the ELK Stack course.

Forwarders compared

This is the most repeated comparison in Splunk interviews.

CapabilityUniversal forwarderHeavy forwarder
What it isSeparate lightweight download that only forwards dataFull Splunk Enterprise instance with some features reduced
Parses data into eventsNot in general, only in limited casesYes, and can send parsed data
Index or search locallyCannot search, index or alert on dataCan index locally if you enable it, but cannot do distributed search
Route by event contentNoYes, because the data is parsed
Typical useCollect from servers and containers and send onFilter, mask or route data before it reaches indexers

Key takeaways

  • Know the data path: input, forwarder, indexer (parsing and indexing), search head. Interviewers expect you to place each component.
  • The universal forwarder only forwards. A heavy forwarder is a full Splunk Enterprise instance that can parse, filter and route data.
  • Buckets age from hot to warm to cold to frozen, and frozen data is deleted by default unless you archive it. The default frozenTimePeriodInSecs is 188697600 seconds, about 6 years.
  • The HTTP Event Collector listens on port 8088 by default and lets CI jobs and apps send events with a token.
  • Prefer stats and tstats for aggregation. tstats works on indexed fields in tsidx files, so it is much faster than raw event searches.
  • In an indexer cluster, a manager node coordinates peer nodes, and replication and search factors decide how many copies exist and how many are searchable.

Splunk basics and architecture (questions 1 to 12)

1. What is Splunk used for in DevOps?

Centralised log and event search. DevOps teams use it to troubleshoot incidents, watch deployments, build dashboards and alerts, and investigate security events from one searchable store.

2. What are the main Splunk components?

Forwarders collect and send data, indexers parse and store it, and search heads run searches and host dashboards. In larger setups a manager node, deployment server and license manager are added.

3. What is the difference between Splunk Enterprise and Splunk Cloud Platform?

Splunk Enterprise is software you install and run yourself. Splunk Cloud Platform is the hosted service where Splunk operates the infrastructure for you.

4. What is an index?

A repository where Splunk stores event data. Each index lives in its own directory under $SPLUNK_HOME/var/lib/splunk and is split into buckets by age.

5. Which default fields does every event have?

Events carry fields such as host, source, sourcetype, index, _time and _raw. Getting sourcetype right makes parsing and searching reliable.

6. What is a sourcetype?

A label that tells Splunk the format of the data so it applies the right timestamp, line breaking and field extraction rules.

7. What is the difference between index time and search time?

Index-time processing, like timestamping and line breaking, happens when data is parsed and cannot be changed afterwards without reindexing. Search-time extraction applies when you search, so it is flexible.

8. Why prefer search-time field extraction?

It can be changed at any time without touching stored data, and it avoids bloating the index with extra indexed fields. Use index-time fields only for high-use, stable values.

9. How do bucket states work?

Data is written to hot buckets, rolls to warm, then to cold in a different location, then to frozen where it is deleted unless archived. Archived frozen data can be restored to a thawed state for search.

10. What is the retention default?

Data freezes when either limit is reached: frozenTimePeriodInSecs (default 188697600 seconds, about 6 years) or maxTotalDataSizeMB (default 500,000 MB per index). If the size limit is hit first, data freezes early.

[app_logs]
homePath   = $SPLUNK_DB/app_logs/db
coldPath   = $SPLUNK_DB/app_logs/colddb
thawedPath = $SPLUNK_DB/app_logs/thaweddb
frozenTimePeriodInSecs = 15552000
maxTotalDataSizeMB = 250000

11. Why is that retention risk worth mentioning?

Changing archiving settings can make the indexer delete old data without prompting. Always confirm the frozen path or archive script before shortening retention.

12. What is SmartStore?

An indexing feature that keeps data in a remote object store such as S3. With SmartStore the cold state does not ordinarily exist.

Data ingestion, forwarders and HEC (questions 13 to 24)

13. What is a universal forwarder?

A lightweight agent whose sole purpose is to forward data. It cannot search, index or alert on data and does not parse data except in limited cases. It also does not bundle Python.

14. What is a heavy forwarder?

A full Splunk Enterprise instance that can parse, filter, mask and route events, and index data locally if enabled. It has a bigger footprint than a universal forwarder.

15. When do you choose a heavy forwarder?

When you must analyse or change data before it leaves, or route events by content, such as sending PCI data to one index and the rest to another.

16. How do you configure a forwarder to read a log file?

Add a monitor stanza in inputs.conf with the path, index and sourcetype, and point outputs.conf at the indexers.

[monitor:///var/log/app/*.log]
index = app_logs
sourcetype = app:log
disabled = false
[tcpout]
defaultGroup = primary_indexers

[tcpout:primary_indexers]
server = idx1.example.com:9997, idx2.example.com:9997

17. Which port do indexers conventionally receive on?

Port 9997 is the conventional receiver port on indexers. Enable it with splunk enable listen 9997 or in the UI.

18. What is the HTTP Event Collector (HEC)?

A token-based HTTP endpoint for sending events straight to Splunk without a forwarder. It suits applications, CI jobs and serverless functions.

curl -k https://splunk.example.com:8088/services/collector/event \
  -H "Authorization: Splunk <hec-token>" \
  -d '{"event": "deploy finished", "sourcetype": "deploy", "index": "cicd"}'

19. What port does HEC use?

8088 by default, which you can change in the HEC global settings. Examples in the docs use HTTPS on that port.

20. How do you ship Kubernetes container logs to Splunk?

Run a collector as a DaemonSet that tails container logs and sends them to HEC or a forwarder, enriching events with namespace, pod and container metadata. Keep pod labels consistent so searches can group by service.

21. How do you stop sensitive data being indexed?

Mask or drop it before indexing with a heavy forwarder or indexer-side props and transforms, and never log secrets in the first place. Restrict index access by role.

22. What does load balancing across indexers mean?

Forwarders spread data across a list of indexers in outputs.conf, which improves ingest throughput and keeps sending if one indexer is down.

23. How do you manage hundreds of forwarders?

Use a deployment server to push configuration apps to forwarder clients by server class, or manage them through configuration management tooling such as Ansible.

24. Data is missing from searches. First checks?

Confirm the forwarder is running and the monitor path is correct, check the outputs connection, check the time range and timezone, and search the internal index for forwarder errors.

SPL for DevOps tasks (questions 25 to 38)

25. What is SPL?

Search Processing Language: a search followed by commands joined with pipes. Each command transforms the results of the previous one.

26. How do you count errors by service in the last 15 minutes?

Filter on index, time and level, then aggregate with stats.

index=app_logs sourcetype=app:log level=ERROR earliest=-15m
| stats count by service
| sort - count

27. What does the stats command do?

It calculates aggregate statistics such as count, average and sum over the result set, similar to SQL aggregation. Without a BY clause it returns a single row.

28. How do you chart an error rate over time?

Create a 1 or 0 field with eval, then average it per time bucket with timechart.

index=web sourcetype=access_combined earliest=-4h
| eval is_error=if(status>=500, 1, 0)
| timechart span=5m avg(is_error) AS error_rate

29. How would you chart deployment frequency?

Count successful deployment events per day from your CI/CD log index.

index=cicd sourcetype=deploy status=success earliest=-30d
| timechart span=1d count AS deployments

30. How would you compute mean time to restore from incident events?

Use stats to find the opening and closing time per incident, then calculate the difference.

index=incidents
| stats earliest(_time) AS opened latest(_time) AS closed by incident_id
| eval mttr_minutes=round((closed-opened)/60, 1)
| stats avg(mttr_minutes) AS avg_mttr

31. How do you extract a field on the fly?

With rex, using a regular expression with a named capture group. The max_match option controls how many matches are returned.

32. How do you enrich events with a CSV or KV lookup?

Use the lookup command to add fields, for example mapping a service to its owning team.

index=app_logs
| rex field=_raw "user=(?<user>\w+)"
| lookup service_owners service OUTPUT team
| stats count by team, user

33. When do you use tstats?

When you only need counts or aggregates over indexed fields. It queries tsidx files, including accelerated data models, and is far faster than scanning raw events.

| tstats count where index=web by host

34. What is the difference between stats, eventstats and streamstats?

stats collapses results into summary rows. eventstats adds the aggregate as a new field on every event while keeping the events. streamstats computes running statistics over events in order.

35. What does the transaction command do and what is its cost?

It groups related events into one transaction using options like maxspan, maxpause, startswith and endswith. It holds events in memory, so use stats with a common ID field where that is enough.

36. How do you speed up slow searches?

Narrow the time range, specify index and sourcetype early, filter before transforming, use tstats or summary indexing for repeated aggregates, and avoid wildcards at the start of a term.

37. How do you find the top values of a field?

Use the top command or stats count by field | sort - count | head 10.

38. How do you remove duplicates?

Use dedup on a field, keeping the first event for each value. Be sure your sort order matches the event you want to keep.

Alerts, dashboards, clustering and operations (questions 39 to 46)

39. What types of alerts exist?

Scheduled alerts run a saved search on a cron schedule, and real-time alerts run continuously. Prefer scheduled alerts for most work because real-time searches cost more.

40. How do you stop alert fatigue?

Alert on symptoms, set sensible thresholds and time windows, use throttling so one issue sends one notification, and route by service owner.

41. What is a dashboard best practice?

Keep one question per panel, use base searches and post-process searches where possible, add time and environment inputs, and avoid expensive real-time panels.

42. What is an indexer cluster?

A group of indexers (peer nodes) managed by a manager node that replicate data so you can lose a peer without losing searchable data. A search head runs searches across the peers.

43. What are replication factor and search factor?

The replication factor is how many copies of data the cluster keeps. The search factor is how many of those copies are searchable. With a search factor of at least 2, search continues with little interruption if a peer fails.

44. Why not make every copy searchable?

Searchable copies use much more storage than non-searchable ones. The trade-off is storage cost against recovery speed.

45. How many peers do you need?

At least as many peer nodes as the replication factor. Plan for more to allow for maintenance and failure.

46. How do you control who sees what data?

Use roles with index restrictions and search filters, split sensitive data into separate indexes and review capabilities granted to each role.

DevOps use cases (questions 47 to 52)

47. How do you monitor a CI/CD pipeline in Splunk?

Send build, test and deploy events to an index with consistent fields such as pipeline, stage, status and duration. Then chart failure rate, stage duration and deployment frequency.

48. How do you correlate a deployment with an error spike?

Overlay deployment events and error counts on one timechart, and use a shared service and version field so you can pivot from the spike to the release.

49. How do you troubleshoot an outage using Splunk?

Start from the alert time, check errors by service, find the first failing component, follow a request or trace ID across services, then confirm the fix by watching the error rate return to baseline.

50. How do you use Splunk with Kubernetes?

Collect container logs, events and metrics with a cluster collector, tag them with namespace, pod and node, and build dashboards for restarts, OOM kills and error rates per workload.

51. How do you handle high-volume log sources?

Filter noise at the edge, set a log level policy, route low-value data to a cheaper index with shorter retention, and watch license and storage use.

52. What should you practise before the interview?

Install a free trial instance, ingest a log file with a forwarder, send a test event through HEC, write the SPL above against your own data and set up one alert and one dashboard.

Frequently asked questions

What is the difference between a universal forwarder and a heavy forwarder?

A universal forwarder only forwards data and cannot search, index or alert. A heavy forwarder is a full Splunk Enterprise instance that can parse, filter and route data and index locally if enabled.

What port does the HTTP Event Collector use?

8088 by default, which can be changed in the HEC global settings.

How long does Splunk keep data by default?

The default frozenTimePeriodInSecs is 188697600 seconds, about 6 years, and the default maximum index size is 500,000 MB. Data freezes when either limit is reached, and frozen data is deleted unless archived.

What are the stages of a Splunk bucket?

Hot, warm, cold, frozen and thawed. Hot is actively written, frozen data is deleted unless archived, and thawed is archived data restored for search.

What is the difference between stats and tstats?

stats aggregates the results of a search. tstats queries indexed fields in tsidx files or accelerated data models directly, which is much faster for counts and summaries.

What are replication factor and search factor?

Replication factor is the number of copies the indexer cluster keeps. Search factor is the number of those copies that are searchable.

Which Splunk Enterprise version is current?

The Splunk docs version selector lists 10.6 as the newest release at the time of writing. Check the release notes for the version your team runs.

How should I prepare for a Splunk DevOps interview?

Ingest logs with a forwarder, send an event with HEC, write SPL for error rate, deployments and MTTR, then build a simple alert and dashboard on that data.

Official references

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0
Mridul I am a passionate technology enthusiast with a strong focus on DevOps, Cloud Computing, and Cybersecurity. Through my blogs at DevOps Training Institute, I aim to simplify complex concepts and share practical insights for learners and professionals. My goal is to empower readers with knowledge, hands-on tips, and industry best practices to stay ahead in the ever-evolving world of DevOps.