Eric D. Schabell: O11y Architecture - Financial Payments Common Observability Elements

Monday, June 5, 2023

O11y Architecture - Financial Payments Common Observability Elements

Cloud native technology has been changing the way payment services are architected. In 2020 I presented a series with insights from real implementations adopting open source and cloud native technology to modernize payment services. 

The series consisted of six articles and covered architectural diagrams from logical, schematic, to detailed views of the various  use cases uncovered. 

The architectures presented were based on open source cloud native technologies, such as containers, microservices, and a Kubernetes based container platform. The major omission in this series was to avoid discussing any aspect of cloud native observability. This series will take a look at fixing that omission with an open source standards based cloud native observability platform that helps DevOps teams control the speed, scale, and complexity of a cloud native world for their financial payments architecture.

The part one introduction covered the baseline architecture, defined the payments project, and shared the planning for this series in adding observability to the logical and physical architectures. In this article we'll explore the logical diagram that captures the elements of a successful payments solution.

Background of generic architectures

Before diving in to the common elements, please understand that this is a collection of identified elements uncovered in multiple working implementations. These elements presented here are then the generic common architectural elements identified and raised up to a generic architecture.

The intent was to provide architectural guidance and not in depth technical details. The assumption is that you're smart and can figure out adopting it to your own architecture. You're more than capable of slotting in the technologies and components you've committed to in the past where applicable. The goal here is to describe generic components and outline a few specific cases enabling you to make the right choices when applying to your own architecture.

Feel free to comment or contact me directly with your feedback.

In a previous article, I've toured the generic architecture and outlined the common elements of the payments architecture. In this article I'll focus on introducing the logical view of only the component layers where I'm adding cloud native observability elements to the solution. 

Container platform

In the baseline article I previously covered all the common elements found in the financial payments architecture. In this section I want to focus on the new addition of cloud native observability components in the container platform logical view.

o11y collector container platform

Modern financial organizations are not only modernizing payments offerings with the use of cloud native technologies and containers, they are finding out that cloud native observability can be challenging for both their engineering and observability teams. They are not only building, testing, deploying, but also having to contend with cloud native complexity, data cardinality explosions, security, and more as they attempt to keep their solutions both running and cost effective.

Within this view of the container platform you find an addition, a Chronosphere Collector element, that ensures your microservices, applications, APIs, integration points, caching, and other services are providing their metrics and tracing data to your observability platform. 

Using open source standards and protocols from the CNCF projects, Prometheus and OpenTelemetry, it's collecting telemetry data, metrics data, and routing them to an external Chronosphere observability platform. This provides for a very easy transition for organizations that might have started their cloud native observability journey using open source projects and standards. 

In upcoming articles in this series I'll share the specific ways you can deploy and leverage the Chronosphere Collector element in a container environment. I'll also present several specific use cases and provide schematic diagrams that detail the physical architectures for those use cases. 

External systems

The elements found in the external systems capture the various regional or local needs for a payments solution. Many are not under the full hosted control of the financial organization, and this is where you find the SaaS solution for your observability needs. 

Many organizations start off their cloud native observability journey with do-it-yourself (DIY) solutions that, over time due to the payments solution success, grow into a resource burden in management, infrastructure, and observability data complexity. 

The managed Chronosphere observability platform allows you to host your cloud native observability needs, unplugging from your DIY infrastructure, and redirecting your open source standards ingestion of metrics and telemetry data. Your engineering and observability teams are still using the same open source tooling, query languages, and visualization that they are well versed in from their experiences with CNCF projects such as Prometheus and OpenTelemetry. 

Using the payments project

The architecture collection provided insights into all manner of use cases and industries researched between 2019 - 2022. The architectures each provide a collection of images for each diagram element as well as the entire project as a whole, for you to make use of as you see fit.

If we look at the financial payments project, you'll see a table of contents allowing you to jump directly to the topic or use case that interests you the most. You can also just scroll down through each section and explore at your leisure. 

Each section provides a short description covering what you see, how it's important to the specific payments topic listed, and a walk through the diagram(s) presented. You can download any of the images to make use in any way you like. At the bottom of the page you find a section titled Download diagrams

If you click on the Open Diagrams link, it will open all the available logical, schematic, and detailed diagrams in the diagram tooling we used to create them. This makes them readily available for any modifications you might see fit to make for use in your own architectures, so feel free to use and modify!

Finally, there is a free online beginners guide workshop available focused on using diagram tooling, please explore to learn tips and tricks from the experts.

Series overview

The following overview of this o11y architecture series on adding cloud native observability to your financial payments architecture can be found here:

Catch up on any articles you missed by following one of the links above. Next in this series, adding cloud native observability to a financial payments example.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.