Welcome to Kubecost! This page provides commonly used product configurations and feature overviews to help get you up and running after the Kubecost product has been installed.
Configuration
- Setting up a cloud integration
- Product configuration at install-time
- Configuring metric storage
- Setting requests & limits
- Using an existing Prometheus installation
- Using an existing Grafana installation
- Using an existing node exporter installation
- Exposing Kubecost with an Ingress
- Deploying Kubecost without persistent volumes
Next Steps
- Measure cluster cost efficiency
- Cost monitoring best practices
- Understanding cost allocation metrics
There are many methods to setup Kubecost. A simple helm install will provide most functionality to understand what Kubecost can do. When you do not pass any values to the helm install, many of the customizations below are available in the settings tab.
By default, Kubecost will detect the cloud provider where it is installed and pull list prices for nodes, storage and LoadBalancers on Azure, AWS, and GCP.
While the basic helm install is useful for understanding the value Kubecost provides, most will want to deploy with an Infrastructure as code model. There are many methods to provide Kubecost with the necessary service accounts / privileges needed. We generally recommend using the various cloud-integration guides below.
By completing the cloud-integration with each provider, Kubecost is able to reconcile costs with your actual cloud bill to reflect enterprise discounts, spot market prices, commitment discounts, and more.
Cloud-integration also enables the ability to view Kubernetes cost metrics side-by-side with external cloud services cost, e.g. S3, BigQuery, Azure Database Services.
For Enterprise Subscriptions, Cloud-integration is only run on the Primary Cluster. Note that the file is a json array where multiple accounts and providers can be configured.
The remaining sections are optional and may be useful for specific use cases.
Product configuration at install-time
Kubecost has a number of product configuration options that you can specify at install time in order to minimize the number of settings changes required within the product UI. This makes it simple to redeploy Kubecost. These values can be configured under kubecostProductConfigs in our values.yaml. These parameters are passed to a ConfigMap that Kubecost detects and writes to its /var/configs.
The default Kubecost installation comes with a 32Gb persistent volume and a 15-day retention period for Prometheus metrics. This is enough space to retain data for ~300 pods, depending on your exact node and container count. See the Kubecost Helm chart configuration options to adjust both retention period and storage size.
To determine the appropriate disk size, you can use this formula to approximate:
needed_disk_space = retention_time_minutes * ingested_samples_per_minutes * bytes_per_sample
Where ingested samples can be measured as the average over a recent period, e.g. sum(avg_over_time(scrape_samples_post_metric_relabeling[24h])). On average, Prometheus uses around 1.5-2 bytes per sample. So ingesting 100k samples per minute and retaining for 15 days would demand around 40 GB. It’s recommended to add another 20-30% capacity for headroom and WAL. More info on disk sizing here.
Note: We do not recommend retaining greater than 30 days of data in Prometheus for larger clusters. For long-term data retention, contact us ([email protected]) about Kubecost with durable storage enabled.
Setting Requests & Limits
It's recommended that users set and/or update resource requests and limits before taking Kubecost into production at scale. These inputs can be configured in the Kubecost values.yaml for Kubecost modules + subcharts.
The exact recommended values for these parameters depend on the size of your cluster, availability requirements, and usage of the Kubecost product. Suggested values for each container can be found within Kubecost itself on the namespace page. More info on these recommendations is available here.
In practice, we recommend running Kubecost for up to 7 days on a production cluster and then tuning resource requests/limits based on resource consumption. Reach out any time to [email protected] if we can help give further guidance.
Using an existing node exporter
For teams already running node exporter on the default port, our bundled node exporter may remain in a Pending state. You can optionally use an existing node exporter DaemonSet by setting the prometheus.nodeExporter.enabled and prometheus.serviceAccounts.nodeExporter.create Kubecost helm chart config options to false. More configs options shown here. Note: this requires your existing node exporter endpoint to be visible from the namespace where Kubecost is installed.
Deploying Kubecost without persistent volumes
You may optionally pass the following Helm flags to install Kubecost and its bundled dependencies without any Persistent Volumes. Note any time the Prometheus server pod is restarted then all historical billing data will be lost unless Thanos or other long-term storage is enabled in the Kubecost product.
--set prometheus.alertmanager.persistentVolume.enabled=false
--set prometheus.pushgateway.persistentVolume.enabled=false
--set prometheus.server.persistentVolume.enabled=false
--set persistentVolume.enabled=false
Cost Optimization
To learn more about pod resource efficiency and cluster idle costs, visit this doc.