Control the things you can control
The ability to iterate quickly and scale dynamically is invaluable for creating business value. This is one of the major motivations for shifting data workloads to the cloud.
With more and more companies moving to the cloud in the last ten years, there has been a shift from Capital Expenditures (CapEx) to Operational Expenditure (OpEx) in terms of IT expenditures.
While in the past large servers were budgeted and set up on premises, nowadays companies are shifting to the cloud where a “pay-as-you-go” pricing model is the standard. The latter has the benefit to scale on demand but generates much greater variance than the former. If managed properly, cloud costs can lead to significant savings as compared to on-premises installations. But if no governance is set up and developers are not well trained, it can lead to out-of-control bills. And as everybody knows “if something can go wrong, it will”. This is where FinOps enters the game with a set of best practices to manage your cost on the cloud.
This article will focus on how to control your FinOps with Infracost. As most companies have all their infrastructure deployed using Terraform, Infracost is a key player to be fully aware of the cost of new resources that are deployed. This article will touch a few words as well on usage-based costs that are harder to tame. Before digging into those topics, we start with a brief introduction of what FinOps is.
FinOps in a few words
The Cloud FinOps book offers the following definition of the term FinOps:
The term “FinOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between DevOps and Finance, resulting in an iterative data-driven management of infrastructure spending (i.e., lowering the unit economics of cloud) while simultaneously increasing the cost efficiency and, ultimately, the profitability of the cloud environment.
With FinOps, engineers need to be very well acquainted with the cost structure of cloud systems. Mastering this cost structure allows engineers to deploy the right mix of resources for each use case; for instance deploying a GKE cluster with different node pools (each one with different machine types and some in preemptible type in case the workloads are not for production). Cloud also offers fixed rate and it is the responsibility of the engineers to know when to switch from the “pay-as-you-go” pricing model to a fixed rate pricing model. This situation arises often with BigQuery where the “on-demand” pricing model becomes less and less interesting once your BigQuery monthly bill gets in the ten thousands of dollars. This is only one of many examples but such thinking process has to happen for each cloud component.
This article does not aim to define FinOps in detail and great resources are available either online or in books for this purpose. We recommend the following resources to become a FinOps guru:
FinOps is defined around six principles and we will see in the next section how Infracost checks those principles:
- Teams need to collaborate
- Everyone takes ownership for their cloud usage
- A centralized team drives FinOps
- Reports should be accessible and timely
- Decisions are driven by business value of cloud
- Take advantage of the variable cost model of the cloud.
For more details on those cornerstone principles, you can refer to this link.
Last but not least, FinOps is not the responsibility of an isolated team. Company needs to set up a cross-functional team known as a Cloud Cost Center of Excellence (CCoE). This team interacts with the rest of the business to manage the cloud strategy, governance, and best practices. that the rest of the organization can leverage to transform the business using the cloud. The following diagram (taken from the FinOps foundation wesite — link here) describes this collaboration perfectly and where the FinOps team lies. The FinOps team is also the main point of contact for communication and negotiations with the mainstream Cloud providers.