It’s inevitable that containerized infrastructure runs a certain amount of overhead capacity that is not used by underlying workloads. This “idle” usage has an associated cost (for the backing VMs and volumes) that needs to be accounted for. We have introduced the ability to distribute these idle container costs within Cloudability core analytics (reports and dashboards). While this is a capability that already exists in the dedicated Container Cost Allocation feature, adding it to core analytics can help users streamline chargeback processes and enables more flexible cost reporting. This distribution is done in the context of the Kubernetes constructs namespace and label.
This support is implemented through the introduction of a new metric category “Containers”, which makes available six new metrics. These metrics enable users to view separately the utilized, idle and fairshare costs either with a cash or amortized basis. For customers with custom pricing enabled these will all be represented as “adjusted” metrics.
Understanding the utilized, idle and fairshare cost metrics
The Container Cost Allocation feature evaluates resource utilization on each node and marries this to the billing data to calculate the cost of each cluster and to associate a direct, “utilized cost”, to K8s namespace and label values (this has been available within core reporting by using the namespace or a label dimension). With this enhancement you can also report on the idle cost via the new metric, which is allocated to the namespace and label values proportionally based on their direct cost contribution of each node. The Fairshare Cost is simply the Utilized Cost and Idle Cost combined - you could think of it as the total cost. Here are some considerations and reporting ideas to keep in mind with these new metrics:
Example Use CasesTo fully allocate the costs at the Namespace level use the Fairshare Cost metric. Try yourself with this report. For a different view customers might also find it handy to just report by Cluster Name or Resource ID - this can help identify where high idle rates are a problem.To visualize trends in utilized versus idle costs you can create multiple layers in a dashboard widget. You could do this for all clusters as shown below, or for specific clusters.Important: With this release, the new metrics are populated back to 1st May 2023. To populate additional historical data, reach out to your account team or support for the reprocess to be completed
@Andrew Midgley thanks, I'll review and revert if there are additional findings
apologies for the delayed response here @Prashant Kabadwal . Allocation to EBS volumes for Namespaces and Labels is fully available
Hi @Andrew Midgley please share the latest status and ETA for - 'In mid-June we'll be launching allocation to Namespace/labels for EBS volumes in our core reporting.'
@Andrew Midgley Support case is opened and with Apptio engineering. Let's see what they suggest.
It's possible you are looking at data that hasn't been fully backfilled. Probably best to work with your TAM/support to clarify the root cause @Prashant Kabadwal
Hi @Andrew Midgley 'If rolled up to the cluster level there shouldn't be a difference.' - this is the exact issue right now
Hi @Prashant Kabadwal , there will be a difference between these two metrics depending on what dimensions you are using in the report. For example, with Namespace, you should expect a difference because the fairshare cost has idle costs included. If rolled up to the cluster level there shouldn't be a difference. Also make sure to use data from May onwards or request a backfill.
thanks @Andrew Midgley
Another potential issue is difference in total fairshare cost(amortized) and cost(amortized) when using the report.
The numbers when viewed in Insights -> Containers matches with cost(amortized) in the report.
I want to understand reason for the difference. Am I inferring the data incorrectly or there is some issue.
Thanks for the feedback, you raise a good point @Prashant Kabadwal regarding EBS volumes. In mid-June we'll be launching allocation to Namespace/labels for EBS volumes in our core reporting. Stay tuned for that and other exciting launches related to container cost allocation.
Hi @Andrew Midgley
Thanks for sharing!
This feature is very promising. I'd request for a bug fix(or enhancement) since the costs for underlying EBS volumes isn't getting allocated to namespaces(or namespace labels) in the new reporting features, making chargebacks inaccurate.
I've shared the details here -> https://community.apptio.com/discussion/containers-idle-cost-distribution-in-core-analytics-feature-missing-ebs-cost-distribution#bm1658b8d6-745c-4b78-94c5-c4c1ebb57f2b