Real-Time Log Anomaly Alerts for AWS Graviton EKS with Prometheus & Slack! 🚀🔥

Awesome! 🚀 Let’s set up Prometheus Alertmanager for real-time log anomaly alerts in your AWS Graviton-based EKS cluster. You’ll receive alerts on Slack, Email, or PagerDuty whenever critical log events occur! 🔥

---

🔹 Step 1: Install Alertmanager using Helm

helm upgrade --install alertmanager prometheus-community/kube-prometheus-stack \
  --namespace monitoring --set alertmanager.enabled=true

Verify Alertmanager is running:

kubectl get pods -n monitoring

---

🔹 Step 2: Configure Alertmanager for Slack Alerts

Create a ConfigMap for Alertmanager (alertmanager-config.yaml):

apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-config
  namespace: monitoring
data:
  alertmanager.yaml: |
    global:
      resolve_timeout: 5m

    receivers:
      - name: 'slack-notifications'
        slack_configs:
          - send_resolved: true
            channel: '#alerts'
            api_url: 'https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK'

    route:
      receiver: 'slack-notifications'
      group_wait: 10s
      group_interval: 5m
      repeat_interval: 3h

Replace YOUR/SLACK/WEBHOOK with your actual Slack webhook URL.

Apply the config:

kubectl apply -f alertmanager-config.yaml

---

🔹 Step 3: Restart Alertmanager to Apply Changes

kubectl delete pod -n monitoring -l app.kubernetes.io/name=alertmanager

Check if it’s running:

kubectl get pods -n monitoring

---

🔹 Step 4: Expose Alertmanager & Access Dashboard

Create an Ingress or LoadBalancer (alertmanager-service.yaml):

apiVersion: v1
kind: Service
metadata:
  name: alertmanager
  namespace: monitoring
spec:
  selector:
    app.kubernetes.io/name: alertmanager
  ports:
    - protocol: TCP
      port: 9093
      targetPort: 9093
  type: LoadBalancer

Apply the service:

kubectl apply -f alertmanager-service.yaml

Get the external Alertmanager URL:

kubectl get svc -n monitoring

Look for EXTERNAL-IP under the alertmanager service and open it in your browser.

---

🔹 Step 5: Configure Grafana to Use Alertmanager

1. Go to Grafana → Alerting → Notification Channels
2. Add a new notification channel
3. Select Alertmanager
4. Enter URL: http://alertmanager.monitoring.svc:9093
5. Save & Test

🎉 DONE! Your AWS Graviton EKS cluster now has real-time log anomaly alerts! 🚀