60+ Alertmanager Interview Questions: Config and K8s (2027)
62 Prometheus Alertmanager interview questions: routing, timers, inhibition, time intervals, templates, HA, Prometheus Operator CRDs and scenarios.
Prometheus Alertmanager interviews test whether you can turn a stream of firing alerts into the right notifications: routing trees, grouping timers, inhibition, silences, time intervals, receivers, templates, high availability and, increasingly, running it on Kubernetes with the Prometheus Operator. This page has 62 questions with direct answers, grouped by level and ending with scenarios, all checked against the Prometheus and Prometheus Operator documentation in November 2026.
This set focuses on configuration and Kubernetes. If you want the most frequently asked basics first, start with our most asked Alertmanager questions; the canonical Q&A page is Alertmanager interview questions and answers.
Key takeaways
- Prometheus evaluates alert rules; Alertmanager deduplicates, groups, routes, inhibits, silences and sends notifications.
- Know the route timer defaults:
group_wait30s,group_interval5m,repeat_interval4h. - Use
matchers,source_matchersandtarget_matchers; the oldermatchandmatch_reforms are deprecated. - For HA, run several instances clustered with the
--cluster.*flags and point Prometheus at all of them, not at a load balancer. - On Kubernetes, the operator's
AlertmanagerandAlertmanagerConfigresources replace hand-edited config files. - The latest release at the time of writing is v0.35.0 (9 November 2026), which includes breaking flag changes worth knowing.
How to use this page
| Your level | Focus on | Hands-on practice |
|---|---|---|
| Fresher | Questions 1 to 15 | Run Alertmanager in Docker, send a test alert, create a silence |
| 2 to 5 years | Questions 16 to 36 | Write a routing tree with child routes, inhibition and time intervals; test it with amtool |
| Kubernetes roles | Questions 37 to 48 | Deploy with the Prometheus Operator; add an AlertmanagerConfig per team namespace |
| Senior / SRE | Questions 49 to 56 and scenarios 57 to 62 | Run a three-node cluster, break a peer, and watch notification metrics |
Fresher questions (1 to 15)
1. What does Alertmanager do?
It receives alerts from Prometheus (or other clients), deduplicates and groups them, routes each group to a receiver such as Slack, email or PagerDuty, and applies silences and inhibition rules before sending notifications.
2. What is the split of responsibilities between Prometheus and Alertmanager?
Prometheus evaluates alerting rules and decides when an alert is firing. Alertmanager decides who is told, how and how often. Keeping the two apart lets many Prometheus servers share one notification layer.
3. Which port does Alertmanager listen on?
The web UI and API are on 9093 in the project's examples. Cluster gossip uses 9094 by default (--cluster.listen-address=0.0.0.0:9094), over both TCP and UDP.
4. How do you run Alertmanager quickly for a lab?
docker run --name alertmanager -d -p 127.0.0.1:9093:9093 quay.io/prometheus/alertmanager
This is the example from the project README. Open http://localhost:9093 for the UI.
5. What is grouping?
Grouping bundles alerts that share the labels listed in group_by into one notification. During an outage, 100 instance-down alerts for one cluster become one message instead of 100.
6. What is inhibition?
Inhibition mutes notifications for some alerts while another alert is firing. For example, a "cluster unreachable" alert can suppress the per-service alerts in that cluster.
7. What is a silence?
A time-limited mute for alerts that match a set of matchers. Silences are created in the UI, the API or with amtool, typically for planned maintenance.
8. Silence vs inhibition?
A silence is created manually and expires at a set time. Inhibition is a rule in the config file that applies automatically whenever a source alert is firing.
9. What is a receiver?
A named notification destination containing one or more integrations, such as slack_configs, email_configs, pagerduty_configs or webhook_configs.
10. What is the routing tree?
A tree of routes starting at the top-level route. Each alert enters at the root and travels down to the first matching child route; that route's receiver and timers apply.
11. Which matcher operators can you use?
=, !=, =~ and !~. The two regex operators are anchored at both ends, so wrap a pattern in .* if you want a partial match.
12. What does send_resolved do?
It controls whether a receiver is notified when an alert resolves. It defaults to true for most integrations, including PagerDuty, Opsgenie and webhooks, but false for email, Slack, Rocket.Chat and WeChat.
13. What is resolve_timeout?
A global setting, default 5m. Alertmanager uses it to mark an alert resolved when the alert does not include an end time.
14. How do you reload configuration without restarting?
Send POST /-/reload or a SIGHUP to the process. If the new file is invalid, the old configuration stays active.
15. What are /-/healthy and /-/ready?
Health and readiness endpoints. /-/healthy always returns 200 when the process is up; /-/ready returns 200 once Alertmanager can serve requests. Use them for liveness and readiness probes.
Configuration questions, 2 to 5 years (16 to 36)
16. What are the default route timers?
group_wait 30s, group_interval 5m and repeat_interval 4h.
17. Explain group_wait, group_interval and repeat_interval.
group_wait is how long a new group waits to collect more alerts before the first notification. group_interval is how long to wait before notifying about new alerts added to a group that has already notified. repeat_interval is how long to wait before resending a notification for a group that has not changed.
18. What does continue: true do on a route?
By default matching stops at the first child route that matches. With continue: true, the alert keeps being evaluated against later sibling routes, so it can go to several receivers, such as a team channel and an audit webhook.
19. What does group_by: ['...'] mean?
It groups by all labels, which effectively disables aggregation. Each unique label set gets its own notification. Use it only when a downstream system does its own grouping.
20. Which route settings are inherited?
receiver, group_by, group_wait, group_interval and repeat_interval are inherited from the parent when not set. mute_time_intervals and active_time_intervals are not inherited.
21. Why are match and match_re discouraged?
The docs mark them as deprecated in favour of matchers, which supports all four operators in one list and the newer matcher parser introduced in 0.27.
22. Write a routing tree that sends critical alerts to PagerDuty and everything else to Slack.
route:
receiver: slack-default
group_by: ['alertname', 'cluster']
routes:
- matchers:
- severity="critical"
receiver: pagerduty-oncall
receivers:
- name: slack-default
slack_configs:
- channel: '#alerts'
send_resolved: true
- name: pagerduty-oncall
pagerduty_configs:
- routing_key_file: /etc/alertmanager/secrets/pd-key
Setting send_resolved: true explicitly matters for Slack, where it defaults to false. Check the exact secret-file field names for each integration in the configuration reference for your version.
23. Write an inhibition rule.
inhibit_rules:
- source_matchers:
- alertname="ClusterDown"
target_matchers:
- severity=~"warning|critical"
equal: ['cluster']
While ClusterDown fires for a cluster, warning and critical alerts with the same cluster label are muted.
24. What does equal do in an inhibit rule, and what is the edge case?
It lists labels that must have the same value on source and target. A missing label and an empty label are treated as equal, so if neither alert has cluster, the rule still applies. Make sure the label exists on both.
25. Can an alert inhibit itself?
If an alert matches both the source and target matchers, it can. Design matchers so the source and target sets do not overlap, for example by keying on different alertname values.
26. What are time intervals?
Named schedules under the top-level time_intervals list. Each can specify times, weekdays, days_of_month, months, years and a location time zone.
27. mute_time_intervals vs active_time_intervals?
A route with mute_time_intervals sends nothing during those intervals. A route with active_time_intervals sends only during them. The root route cannot set either.
28. Write a business-hours interval for India.
time_intervals:
- name: office-hours
time_intervals:
- weekdays: ['monday:friday']
times:
- start_time: '09:00'
end_time: '18:00'
location: 'Asia/Kolkata'
location takes an IANA time zone name and defaults to UTC. The end time is exclusive.
29. How do you load custom notification templates?
List files under the top-level templates field. The last path component may be a wildcard, such as templates/*.tmpl.
30. Which fields are available in notification templates?
The data includes .Status, .Receiver, .GroupLabels, .CommonLabels, .CommonAnnotations, .ExternalURL, and the methods .Alerts.Firing and .Alerts.Resolved.
31. Write a Slack title template.
slack_configs:
- channel: '#alerts'
send_resolved: true
title: '[{{ .Status | toUpper }}] {{ .GroupLabels.alertname }} ({{ .Alerts.Firing | len }} firing)'
text: '{{ .CommonAnnotations.summary }}'
32. Which receiver integrations are built in?
The current reference lists Discord, email, Mattermost, Microsoft Teams (two config types), Jira, Opsgenie, PagerDuty, incident.io, Pushover, Rocket.Chat, Slack, Amazon SNS, Telegram, Splunk On-Call (VictorOps), Webex, generic webhooks and WeChat.
33. When would you use webhook_configs?
When the target has no built-in integration, or you need custom logic such as creating tickets, enriching alerts or forwarding to an internal event bus. The webhook receives the same data structure that templates see.
34. How do you validate a config and test routing?
amtool check-config alertmanager.yml
amtool config routes test --config.file=alertmanager.yml severity=critical team=payments
amtool config routes --config.file=alertmanager.yml
routes test prints which receiver an alert with those labels would reach; the README documents its --tree and --verify.receivers options. config routes prints the routing tree.
35. How do you create and expire a silence with amtool?
amtool --alertmanager.url=http://localhost:9093 silence add alertname=DiskFull instance=db1 \
--duration=2h --comment="Disk expansion" --author="ops"
amtool --alertmanager.url=http://localhost:9093 silence query
amtool --alertmanager.url=http://localhost:9093 silence expire <silence-id>
36. What are route labels?
Release 0.34.0 added optional templatable labels on routes. Labels are merged from parent to child and exposed to templates through routeLabels, so you can attach team or runbook metadata at the routing layer.
Kubernetes and Prometheus Operator questions (37 to 48)
37. How is Alertmanager deployed with the Prometheus Operator?
You create an Alertmanager custom resource (monitoring.coreos.com/v1) with settings such as replicas. The operator runs a StatefulSet and wires up clustering between replicas.
38. Where does the operator read the main configuration from?
By default from a Secret named alertmanager-<name> with the config under the key alertmanager.yaml. spec.configSecret can point to a different Secret name.
39. What is an AlertmanagerConfig?
A namespaced custom resource (monitoring.coreos.com/v1alpha1) holding routes, receivers, inhibit rules and time intervals in camelCase fields such as groupBy and webhookConfigs. The operator merges selected objects into the running configuration.
40. How does the operator pick which AlertmanagerConfig objects to use?
Through spec.alertmanagerConfigSelector (label selector on the objects) and spec.alertmanagerConfigNamespaceSelector (which namespaces to search). If the namespace selector is nil, only the Alertmanager's own namespace is checked.
41. What is spec.alertmanagerConfiguration?
A reference to one AlertmanagerConfig in the Alertmanager's namespace to use as the global configuration instead of a Secret. The operator docs note that in this mode the namespace label is not enforced for routes and inhibition rules.
42. Why do teams use per-namespace AlertmanagerConfig objects?
They let each team keep its own routes and receivers in its own namespace, while the platform team owns the main configuration and the selectors that decide which objects are merged. The operator docs note that when an AlertmanagerConfig is used as the global configuration, the namespace label is not enforced for routes and inhibition rules, so check how your operator version scopes namespaced objects before relying on it for isolation.
43. Write a minimal AlertmanagerConfig.
apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
name: payments-routing
namespace: payments
labels:
alertmanagerConfig: team
spec:
route:
receiver: payments-webhook
groupBy: ['alertname']
groupWait: 30s
repeatInterval: 4h
receivers:
- name: payments-webhook
webhookConfigs:
- url: 'http://alert-router.payments.svc:8080/hook'
44. How are alert rules defined with the operator?
As PrometheusRule resources (monitoring.coreos.com/v1) with rules under spec.groups[].rules[]. The Prometheus resource selects them with spec.ruleSelector and spec.ruleNamespaceSelector.
45. A new PrometheusRule is not loaded. What do you check first?
The selectors. The operator docs state that a nil ruleSelector picks up no rules, and that by default only the Prometheus object's own namespace is searched. Make the rule's labels and namespace match.
46. How does Prometheus find Alertmanager in the operator setup?
Through spec.alerting.alertmanagers on the Prometheus resource, listing the namespace, service name and port (for example web). Prometheus discovers all pods behind that service.
47. How do you select namespaces by name?
Use the automatically populated kubernetes.io/metadata.name label in a namespace selector.
48. What probes should an Alertmanager pod use?
Liveness on /-/healthy and readiness on /-/ready. If you deploy Alertmanager yourself rather than through an operator, add both probes to the pod spec.
Senior and operations questions (49 to 56)
49. How does Alertmanager high availability work?
Several instances form a cluster using --cluster.listen-address and --cluster.peer. Each instance receives every alert; silences and notification state are replicated between peers so that only one notification is sent per group.
50. Why should Prometheus not send alerts through a load balancer?
The project states that Alertmanager expects all alerts to be sent to all Alertmanagers. Configure Prometheus with the full list of instances so each one has the complete picture and deduplication works.
51. What network ports must be open between peers?
The cluster port, 9094 by default, on both TCP and UDP. If an instance has no RFC 6890 address with a default route, set --cluster.advertise-address.
52. How do you disable clustering for a single-node setup?
Set --cluster.listen-address to an empty string.
53. Which metrics show notification failures?
alertmanager_notifications_failed_total by integration. Since 0.34.0 its reason label separates authError and rateLimited from clientError, so update dashboards that matched only clientError.
54. What changed in v0.35.0 that could break an upgrade?
The auto-gomemlimit feature flag became the --auto-gomemlimit command-line flag and the old --enable-feature form was removed; the deprecated types Go package was removed; and event recorder outputs now need a name and use the events/v2 schema. Release 0.35.0 also serves a new UI at /ui/.
55. Why might a browser dashboard stop reading the Alertmanager API after an upgrade?
From 0.34.2, Alertmanager no longer sends CORS headers by default, so cross-origin calls to /api/v2 are blocked. The --web.cors.origin flag restores access for trusted origins.
56. How do you alert on Alertmanager itself?
Scrape it with Prometheus and alert on failed notifications, cluster peers dropping and config reload failures. Add an always-firing "watchdog" alert routed to an external dead man's switch, so you learn when the whole alerting path is down.
Scenario questions (57 to 62)
57. On-call receives the same alert three times from a three-node cluster. What is wrong?
The peers are probably not clustered. Check the UI or metrics for cluster members, confirm 9094 TCP and UDP are open between pods, and check --cluster.peer values. Also confirm Prometheus is sending to each instance directly, not through a load balancer.
58. A team says their critical alert never paged. How do you investigate?
Run amtool config routes test with the alert's labels to see which receiver it reaches. Check for an active silence, an inhibit rule whose equal labels are missing on both alerts, a mute time interval on the route, and failures in alertmanager_notifications_failed_total.
59. During a network outage Slack gets hundreds of messages. How do you fix it?
Group by fewer, broader labels such as cluster and alertname, add an inhibit rule so a network or cluster-level alert mutes service alerts, and check group_interval is not set too low.
60. Warning alerts wake people at night. What do you change?
Route warnings to a ticket or chat receiver with active_time_intervals set to office hours, and send only critical, user-impacting alerts to the paging receiver at all hours.
61. A team's AlertmanagerConfig is applied but has no effect. What do you check?
Whether its labels match alertmanagerConfigSelector, whether its namespace is selected by alertmanagerConfigNamespaceSelector, and the operator logs for validation errors. Then run amtool config routes against the generated configuration to see where the team's route landed in the tree.
62. Planned database maintenance tonight from 23:00 to 01:00. What do you set up?
A silence with matchers for the affected services, a start and end time, an author and a comment, created through amtool or the API so it is recorded. Keep alerts for unrelated systems unmuted, and expire the silence early if maintenance finishes sooner.
Preparation tips
Interviewers often ask you to draw the routing tree for a real team. Practise by writing one for three teams with different severities, then test it with amtool config routes test before the interview. Pair this page with our Prometheus alert rules questions, the Alertmanager study plan and our scenario-based Alertmanager questions. Our Prometheus and Grafana training covers Alertmanager setup on Kubernetes, in Kharadi, Pune or live online across India.
Frequently asked questions
What are the default Alertmanager group timers?
group_wait 30s, group_interval 5m and repeat_interval 4h.
What is the difference between a silence and an inhibition rule?
A silence is created manually for a fixed time. An inhibition rule is configured once and mutes target alerts automatically while a matching source alert fires.
How do you run Alertmanager in high availability?
Run several instances clustered with --cluster.listen-address and --cluster.peer, open port 9094 TCP and UDP between them, and list every instance in Prometheus.
What is AlertmanagerConfig in the Prometheus Operator?
A namespaced custom resource (monitoring.coreos.com/v1alpha1) that lets teams define routes and receivers, which the operator merges into the main Alertmanager configuration.
How do I test which receiver an alert will reach?
Run amtool config routes test --config.file=alertmanager.yml followed by the alert's labels.
Does Slack send resolved notifications by default?
No. send_resolved defaults to false for Slack and email; set it to true if you want resolve messages.
What is the latest Alertmanager version?
v0.35.0, released on 9 November 2026, at the time of writing. Check the GitHub releases page before an upgrade.
Are match and match_re still supported?
They are deprecated. Use matchers on routes and source_matchers/target_matchers on inhibit rules.
References
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0