Troubleshooting¶

Common issues and solutions for Pingora Gateway Controller.

Diagnostic Commands¶

Check Component Status¶

# Controller pods
kubectl get pods --namespace pingora-system \
  --selector app.kubernetes.io/name=pingora-gateway-controller

# Proxy pods
kubectl get pods --namespace pingora-system \
  --selector app.kubernetes.io/component=proxy

# All resources
kubectl get all --namespace pingora-system

View Logs¶

# Controller logs
kubectl logs --namespace pingora-system \
  --selector app.kubernetes.io/name=pingora-gateway-controller \
  --tail=100 --follow

# Proxy logs
kubectl logs --namespace pingora-system \
  --selector app.kubernetes.io/component=proxy \
  --tail=100 --follow

Check Events¶

kubectl get events --namespace pingora-system --sort-by='.lastTimestamp'

Common Issues¶

GatewayClass Not Accepted¶

Symptom: GatewayClass shows Accepted: False

kubectl get gatewayclass pingora

Causes:

Controller not running
Wrong controller name in GatewayClass

Solution:

# Verify controller is running
kubectl get pods --namespace pingora-system

# Check GatewayClass spec
kubectl get gatewayclass pingora --output yaml

# Controller name should match
controllerName: pingora.k8s.lex.la/gateway-controller

Gateway Not Programmed¶

Symptom: Gateway shows Programmed: False

Causes:

GatewayClass not accepted
PingoraConfig not found or invalid
Unable to connect to proxy

Solution:

# Check GatewayClass status
kubectl get gatewayclass pingora

# Check PingoraConfig
kubectl get pingoraconfig --output wide

# Verify proxy is reachable
kubectl exec -it deployment/pingora-gateway-controller \
  --namespace pingora-system -- \
  nc -zv pingora-gateway-controller-proxy 50051

HTTPRoute Not Accepted¶

Symptom: HTTPRoute shows Accepted: False

kubectl get httproute my-route --output yaml

Causes:

Gateway not found (wrong name or namespace)
GatewayClass not accepted
Backend service not found

Solution:

# Check parent reference
kubectl get httproute my-route --output jsonpath='{.spec.parentRefs}'

# Verify Gateway exists
kubectl get gateway --namespace pingora-system

# Check backend service
kubectl get service my-backend --namespace default

Cross-Namespace Reference Failed¶

Symptom: ResolvedRefs: False with reason RefNotPermitted

Causes:

ReferenceGrant missing
ReferenceGrant in wrong namespace
ReferenceGrant doesn't match source namespace

Solution:

# Check for ReferenceGrant in target namespace
kubectl get referencegrant --namespace target-namespace

# Verify ReferenceGrant allows source namespace
kubectl get referencegrant allow-grant --namespace target-namespace --output yaml

Routes Not Syncing¶

Symptom: Routes accepted but traffic not routing

Causes:

Proxy not receiving configuration
gRPC connection issues
Backend service has no endpoints

Solution:

# Check PingoraConfig status
kubectl get pingoraconfig --output yaml

# Look for sync errors in controller logs
kubectl logs --namespace pingora-system \
  --selector app.kubernetes.io/name=pingora-gateway-controller | grep -i error

# Verify backend has endpoints
kubectl get endpoints my-backend

Proxy Connection Refused¶

Symptom: Controller logs show gRPC connection errors

Causes:

Proxy pods not running
Wrong address in PingoraConfig
Network policy blocking traffic

Solution:

# Check proxy pods
kubectl get pods --namespace pingora-system \
  --selector app.kubernetes.io/component=proxy

# Verify PingoraConfig address
kubectl get pingoraconfig --output jsonpath='{.items[*].spec.address}'

# Test connectivity
kubectl exec -it deployment/pingora-gateway-controller \
  --namespace pingora-system -- \
  nc -zv pingora-gateway-controller-proxy.pingora-system.svc.cluster.local 50051

High Latency¶

Symptom: Slow request processing

Causes:

Proxy resource constraints
Backend service slow
Too many routes causing slow sync

Solution:

# Check proxy resource usage
kubectl top pods --namespace pingora-system \
  --selector app.kubernetes.io/component=proxy

# Check sync duration metrics
curl http://controller:8080/metrics | grep pingora_sync_duration

# Consider increasing proxy replicas/resources

Debug Mode¶

Enable debug logging for more detailed output:

# In Helm values
controller:
  logLevel: "debug"

proxy:
  logLevel: "debug"

Or patch the deployment:

kubectl set env deployment/pingora-gateway-controller \
  --namespace pingora-system \
  PINGORA_LOG_LEVEL=debug

Collecting Debug Information¶

For bug reports, collect:

# Version information
kubectl get deployment pingora-gateway-controller \
  --namespace pingora-system \
  --output jsonpath='{.spec.template.spec.containers[0].image}'

# Resource status
kubectl get gatewayclass,gateway,httproute,grpcroute,pingoraconfig \
  --all-namespaces --output yaml > resources.yaml

# Controller logs
kubectl logs --namespace pingora-system \
  --selector app.kubernetes.io/name=pingora-gateway-controller \
  --tail=1000 > controller.log

# Proxy logs
kubectl logs --namespace pingora-system \
  --selector app.kubernetes.io/component=proxy \
  --tail=1000 > proxy.log

# Events
kubectl get events --namespace pingora-system \
  --sort-by='.lastTimestamp' > events.txt

Getting Help¶

If issues persist:

Check GitHub Issues
Create a new issue with debug information
Include reproduction steps and expected behavior

Next Steps¶

Review Metrics for monitoring
Check Configuration for settings