Cloud Data - FinOps is the crucial Ops

The daily hype is all around you.

From private to public cloud, multi-cloud, and even hybrid cloud, you're overrun with information telling you this is the path to your digital future. To complicate matters while you are contemplating these choices, you are expected to keep up your daily tasks of enhancing customer experiences and agile delivery of those applications.

Wrapped up in all this delivery and architectural infrastructure, there's a multitude of decisions around data to be considered when engaging with any cloud experience. There are regulatory and compliance pressures that force you to evaluate how we collect, process, and store our observability data. Understanding the pitfalls around the collection, maintenance, and storage of your cloud data can mean the difference between failure and success within your cloud strategy.

This series is based on a talk given previously in Dublin, Ireland and was brainstormed with my good friend Roel Hodzelmans. The reactions from the audience inspired me to share the concepts in this series.

The first article in this series provided an introduction to cloud and data, what that means in a cloud-native architecture beyond just storage. The second article in this series talked about the forgotten data that is often overlooked when planning for cloud-native solutions. This third and final article explores a new operations role that is going to be the most crucial one in your organization.

FinOps is the crucial Ops

Many of us are re-architecting our applications and development processes to be cloud native, our operations to be platform providers, and building a site reliability engineer (SRE) organization to close the feedback loop between the platform consumers and providers. We talk (hopefully) about data portability, having exit strategies and baselines. We talk about security and we talk about utilization.

The only thing that often gets overlooked is who owns the decisions being made around observability, data, and cloud providers. Naturally, the end responsibility lies with the CIO you might think, but often at the lower levels we're seeing developers, DevOps, architects, and others making decisions that have serious cost implications.

In the previous article, you saw that observability data growth is out of control in this quote shared here to refresh your memory. 

"It’s remarkable how common this situation is, where an organization is paying more for their observability data (typically metrics, logs, traces, and sometimes events), then they do for their production infrastructure."

Recently IDC (2022) published a cloud report in which they shared an amazing revelation showing that companies are taking this problem seriously. 

“By 2023, 80% of organizations using cloud services will establish a dedicated FinOps (financial operations) function to automate policy-driven observibility and optimization of cloud resources to maximize value.”

The action being taken is almost universal and is meant to resolve the cost issues by putting a financial owner in place to manage the decisions and processes in your organization.

This is a very important data point in this discussion and is almost historic with the speed at which this decision has been reached by a majority of organizations. You  can’t ignore this problem, that’s the message they are sending. It’s time to take action and decisive action by installing a single point of responsibility to corral the explosion of decisions impacting the costs of doing business in a cloud native world.

Cloud data pitfalls

The three pitfalls addressed in this series covered cloud and data, not just the storage issues but showed how observability data is often ignored in cloud native solutions. Finally, the role of FinOps is going to appear in your cloud native world in the very near future to address the ownership and process management to gain control of cloud native observability and data issues.

This series presents an overview of the original session presented live in Dublin and hopefully it has been able to share an awareness of the pitfalls that might be in your path regarding cloud native solutions and your data growth as a result.