Exposing Costs with Infrastructure as Code

December 4, 2020 Claranet Limited

Liam Bennett

As a discipline, FinOps is often seen as a central function of an organisation – something managed by core operations teams and governed by finance. But that is only part of the picture.

The reality is that to be successful in implementing cost controls for cloud environments, there needs to be a cultural change in the engineering teams. They need to be responsible for and care deeply about the costs of their applications and have relevant tools to do so.  

One of the emerging techniques to support this integrates cost management and infrastructure as code to allow engineers to see directly the cost impact of their changes as they are making them.  

One the many benefits of using infrastructure as code is being able to preview the impact of any change before it’s made, e.g. Terraform plan, CloudFormation change sets, or ARM template What-Ifs. If it’s possible to see that infrastructure changes that you plan to make to your environment, then why not the cost impact of that change?  

Well first with Terraform Enterprise and now several open-source tools (terraform-cost-estimation, infracost, and terraform-cashier) we can start to realise that possibility.  

All these tools work on the same basic principles:  

  1. Get the current state of the infrastructure and estimate the cost  
  2. Get the desired state and estimate the cost  
  3. Show the changes in infrastructure and cost between the two  

(Ref: https://www.infracost.io/docs/)  

Being able to see, for example, that changing a machine type from a Standard_D2_v2 to a Standard_D2_v4 has a £50/month difference might influence your decision making or make you consider an alternative design.  

But this is all relative and doesn’t replace the need for implementing budgets. Budgets are a standard cost control mechanism often implemented as part of FinOps that will either notify based on a threshold or prevent further action upon that threshold. Allowing each team to understand the budget they have will be enable them to understand the context upon which their code change sits within that budget.  

For some teams, a code change of £50/month will be small, but for others very significant. Pushing this sort of cost insight alongside infrastructure as code tooling means that engineers will make a cost-driven decision before the change is made, rather than relying on the traditional post-deployment cost-analysis typical of most FinOps engagements.  

An additional step that will soon be possible (either with Terraform Sentinel, terraform-compliance, or other testing tools) will be to implement policies based on this cost impact. Perhaps you want to put a limit on the cost differential of any change to ensure that no one can spend the whole month’s budget in a single change. Alternatively, you may want to prevent the deployment of certain resources that you know have more cost-effective alternatives.  

Implementing this as part of the testing phase of the typical CI/CD pipeline ensures that costs, budgets, and other policy restrictions are managed as early in the lifecycle as possible. Recording the cost output as part of the pipeline execution (in the logs of the pipeline or as a comment on the pull-request) provides an audit trail of cost impact which can be mapped to the overall cost of changes as part of a typical monthly cost review.  

The tools currently in this space are still focused on PAYG pricing, rather than any other purchasing options. So for many organisations to date, this will only provide an indicative pricing or a guiding shape of cost impact rather than exact figures. Nevertheless, however, it will still show benefits to engineers, and we’ve started investigating this with our customers.  

Take a moment to review the tooling hyperlinked above. Then  speak to us about implementing it into your pipelines and interpreting the results as part of our ongoing FinOps service. It could have a significant beneficial impact on your business. 

Previous Article
Do you need a new cloud strategy?
Do you need a new cloud strategy?

This year we’ve all been disrupted by COVID-19, personally and professionally. The changes have been fast a...

Next Article
The Data Journey - Part Four: Prescriptive Analytics
The Data Journey - Part Four: Prescriptive Analytics

Every business has data. Some businesses can spin gold from their data but others merely collect it in the ...