K8sGPT - Giving Kubernetes Superpowers to Everyone

Overview

K8sGPT supports a wide range of AI providers, from cloud-based services to self-hosted solutions. Choose the provider that best fits your needs, whether you prioritize privacy, cost, performance, or specific model capabilities.

☁️ Cloud Providers

OpenAI

Default

GPT-3.5, GPT-4, and other OpenAI models. Most widely tested provider.

k8sgpt auth add --backend openai --model gpt-4o
# Or use GPT-3.5 for lower cost
k8sgpt auth add --backend openai --model gpt-3.5-turbo

API Key: Get from platform.openai.com

Azure OpenAI

OpenAI models hosted on Microsoft Azure. Ideal for enterprise deployments with Azure infrastructure.

k8sgpt auth add --backend azureopenai \
  --baseurl https://YOUR-ENDPOINT.openai.azure.com \
  --model YOUR-DEPLOYMENT-NAME

Note: Requires Azure OpenAI resource and deployment

Amazon Bedrock

Access foundation models from Amazon, Anthropic, Meta, and more through AWS Bedrock.

k8sgpt auth add --backend amazonbedrock \
  --model anthropic.claude-v2

Authentication: Uses AWS credentials from environment or IAM role

Amazon SageMaker

Deploy and use custom models or AWS-provided models via SageMaker endpoints.

k8sgpt auth add --backend amazonsagemaker \
  --model YOUR-ENDPOINT-NAME

Google Vertex AI

Enterprise AI platform with models like PaLM 2, Gemini, and more on Google Cloud.

k8sgpt auth add --backend googlevertexai \
  --model gemini-pro

Google Generative AI

Direct access to Google's Gemini models via the Generative AI API.

k8sgpt auth add --backend googlegenai \
  --model gemini-pro

Groq

Ultra-fast inference with Llama, Mixtral, and Gemma models on custom LPU hardware.

k8sgpt auth add --backend groq \
  --model llama-3.1-70b-versatile

API Key: Get from console.groq.com

Cohere

Enterprise-focused NLP with Command and Embed models.

k8sgpt auth add --backend cohere \
  --model command

OCI Generative AI

Oracle Cloud Infrastructure's generative AI service with various foundation models.

k8sgpt auth add --backend ocigenai \
  --model cohere.command

IBM WatsonX AI

IBM's enterprise AI platform with foundation models for business use cases.

k8sgpt auth add --backend watsonxai \
  --model ibm/granite-13b-chat-v2

Hugging Face

Access thousands of open-source models via Hugging Face's Inference API.

k8sgpt auth add --backend huggingface \
  --model meta-llama/Llama-2-70b-chat-hf

🏠 Self-Hosted / Local Providers

Privacy & Control: Self-hosted providers keep your data on your infrastructure and don't require external API calls.

Ollama

Local

Run local LLMs (Llama, Mistral, Gemma, etc.) on your machine. Perfect for air-gapped environments and data privacy.

# Install Ollama from ollama.ai
ollama pull llama3.2

# Configure K8sGPT
k8sgpt auth add --backend ollama \
  --model llama3.2 \
  --baseurl http://localhost:11434

📖 Learn more: Ollama Local LLM Tutorial

LocalAI

Local

OpenAI-compatible local AI server. Supports many models with GPU acceleration.

k8sgpt auth add --backend localai \
  --model ggml-gpt4all-j \
  --baseurl http://localhost:8080

Setup: See localai.io for installation

Custom REST Backend

Custom

Integrate any custom AI service that implements a simple REST API contract.

k8sgpt auth add --backend customrest \
  --baseurl http://your-service:8080/analyze

📖 Learn more: Custom REST Backend Tutorial

🛠️ Managing Providers

List Configured Providers

k8sgpt auth list

Remove a Provider

k8sgpt auth remove --backend openai

Update Provider Configuration

Remove and re-add with new settings:

k8sgpt auth remove --backend openai
k8sgpt auth add --backend openai --model gpt-4o

💡 Choosing a Provider

Best for Production

✅ OpenAI (most reliable, well-tested)
✅ Azure OpenAI (enterprise SLA)
✅ Google Vertex AI (GCP integration)

Best for Cost

💰 Groq (free tier, fast inference)
💰 Ollama (free, local)
💰 LocalAI (free, self-hosted)

Best for Privacy

🔒 Ollama (fully local)
🔒 LocalAI (self-hosted)
🔒 Custom REST (your infrastructure)

Best for Speed

⚡ Groq (LPU hardware)
⚡ GPT-3.5-turbo (fast responses)
⚡ Ollama (local, no network latency)

Need Help?

For detailed setup guides and troubleshooting, check the CLI Configuration page or join the community on Discord.

AI Providers