Troubleshooting

If your problem isn't here, check GitHub issues or ask in Discussions.

InstallCopy Anchor Link

Pods stay Pending or the control plane restartsCopy Anchor Link

The control plane waits for its stores for controlPlane.storeWait (2 minutes) and exits if they aren't reachable, so it restarts until they are. Check the embedded stores first:

kubectl -n kubehero-system get pods,pvc
kubectl -n kubehero-system logs deploy/kubehero-control-plane --tail=50

Pending stores usually mean no default StorageClass — set postgresql.storage.storageClass / clickhouse.storage.storageClass.

I can't sign in to the dashboardCopy Anchor Link

Sign-in takes a token. Read the admin token from the Secret:

kubectl -n kubehero-system get secret kubehero-control-plane \
  -o jsonpath='{.data.admin-token}' | base64 -d; echo

Pages show a "demo" badgeCopy Anchor Link

A signal whose store isn't configured serves labelled demo data. On a default install this clears once the stores are up. If you disabled the embedded stores, configure postgresql.external / clickhouse.external. In production set controlPlane.demoMode: "false" so a misconfiguration fails loudly instead.

CollectorCopy Anchor Link

kubectl -n kubehero-system get pods -l app.kubernetes.io/component=collector -o wide
kubectl -n kubehero-system logs ds/kubehero-collector --tail=50

Costs look N× too high (pre-0.3 collectors)Copy Anchor Link

Before v0.3 every collector reported every pod in the cluster. Upgrade the chart: v0.3 collectors report only their own node (NODE_NAME from the downward API). A collector without NODE_NAME logs a loud warning because it would double-count.

Node prices look wrongCopy Anchor Link

Each node sample records where its price came from: the kubehero.io/node-hourly-usd annotation, the pricing engine, or an estimate. Set the annotation for negotiated prices, and make sure the pricing engine is running (pricingEngine.enabled). GCP's catalog needs an API key (pricingEngine.gcpBillingSecret).

eBPF didn't attachCopy Anchor Link

kubehero_collector_ebpf_enabled == 1 and kubehero_collector_ebpf_attached == 0

The collector logs one line and keeps collecting everything else. Common causes:

• kernel older than 5.8 or cgroup v1 — check uname -r and stat -fc %T /sys/fs/cgroup (should print cgroup2fs) on the node;
• the platform forbids privileged pods (e.g. GKE Autopilot) — set collector.ebpf.enabled: false;
• a security module (SELinux, AppArmor, locked-down kernels) blocks BPF.

LogsCopy Anchor Link

• Nothing appears for old pods. collector.logs.startFrom: end (default) skips what was already on disk at the collector's first start; new lines flow. Use start to backfill.
• Lines are missing under load. The per-container rate limit dropped them — see kubehero_collector_log_lines_dropped_total; raise collector.logs.rateLimit.
• A namespace is missing. Check collector.logs.excludeNamespaces; the collector's own namespace is always excluded.
• A LogQL feature errors. See the support matrix — unwrap, quantile_over_time and the pattern parser aren't supported in v0.3.

ProfilesCopy Anchor Link

• No eBPF profiles. See eBPF didn't attach. Annotated pods are still scraped over pprof.
• Stacks full of [unknown] or truncated. User-space stacks need frame pointers: fine for Go and Rust; add -XX:+PreserveFramePointer for the JVM; rebuild C/C++ with frame pointers. JIT frames need perf maps — or use pprof / a Pyroscope SDK.
• A pod isn't scraped. Check the annotations (profiles.grafana.com/cpu.scrape: "true" plus the port) and that collector.profiling.scrape is on.

NetworkCopy Anchor Link

• The service map is empty. Flows need eBPF (above) and collector.ebpf.netflow: true; they appear after the first flush.
• Everything goes to one external address. A NAT gateway or proxy hides destinations; traffic is attributed to its address.
• Egress prices are off. Set controlPlane.pricing.netEgressUSDPerGB / netCrossZoneUSDPerGB to your rates.

Rightsizing doesn't applyCopy Anchor Link

Read the policy's status — every blocked change says why:

kubectl -n kubehero-system get rightsizingpolicy <name> -o jsonpath='{range .status.plannedChanges[*]}{.workload}{"\t"}{.outcome}{"\t"}{.reason}{"\n"}{end}'

Usual reasons: the policy isn't armed (Armed condition false), confidence is below minConfidence (less than a day or five days of data), maxChangePerDay reached, a rollout is in progress, a VPA owns the container, fewer than minReplicas replicas, or the workload is annotated kubehero.io/rightsizing: "disabled".

Alerts don't notifyCopy Anchor Link

• The five default rules are seeded without channels — add channels to them.
• Check the rule's pendingFor: pending alerts don't notify.
• Check silences: kubehero alerts silence list.
• Use Test in the rule editor, or kubehero alerts test, to see the evaluated series.

Grafana's Loki data source errorsCopy Anchor Link

• 401 — set the Authorization: Bearer <token> header on the data source.
• "multi-tenant queries are not supported" — X-Scope-OrgID contains |; use one cluster per data source.

MCP and the CLICopy Anchor Link

• unauthenticated — set KUBEHERO_ENDPOINT and KUBEHERO_TOKEN (or --endpoint / --token, or kubehero auth login).
• An HTTP MCP server isn't reachable from another machine — --http :8765 binds 127.0.0.1 only; pass an explicit host if you really mean to expose it.

The advisor uses the rules brainCopy Anchor Link

• Check the key: advisor.anthropic.existingSecret must name a Secret with the key in advisor.anthropic.secretKey (default api-key).
• Network policies may block egress to the Anthropic API.
• The advisor falls back to the rules brain on any API error, refusal or timeout — its logs say why.

Prometheus doesn't scrape KubeHeroCopy Anchor Link

The ServiceMonitor and PrometheusRule are matched by the prometheus.release label (default kube-prometheus-stack). Set it to your Prometheus release name, or disable label selection in kube-prometheus-stack (serviceMonitorSelectorNilUsesHelmValues: false, ruleSelectorNilUsesHelmValues: false).