Eric D. Schabell: PromCon EU 2022 - Centralized versus Decentralized Prometheus Metrics (slides)

Tuesday, November 8, 2022

PromCon EU 2022 - Centralized versus Decentralized Prometheus Metrics (slides)

Today I was on site at PromCon EU 2022 in Munich, Germany for the first of two days. 

The main event for Prometheus across Europe and the minds that are behind this open source metrics monitoring project gather to share and collaborate for two days.

he last in person edition was in 2019 and you can see what that was like in their video recap.

As they state on their website, this conference is to "...connect Prometheus users and developers from around the world in order to exchange knowledge, best practices, and experience gained around using Prometheus. We also want to collaborate to build a community and grow professional connections around systems and service monitoring."

I co-presented with Ales Koprivnikar on a session that DoorDash gave a few weeks ago at Prometheus Day in North America, a story about insights into metrics collection at a very large scale.

Below you will find the slides to our talk, and the abstract for completeness:

Centralized vs. decentralized - How Doordash collects Prometheus metrics

There are two primary approaches to scrape and collect metrics using Prometheus - using a centralized set of dedicated scrapers or decentralized scrapers that run as an agent. With centralized scraping, Prometheus is deployed as a central scraper to pull metrics from all discoverable endpoints and sometimes can be split across multiple centralized instances using a few different approaches. However, with a decentralized approach, Prometheus runs as an agent, in Kubernetes is deployed as a DaemonSet on each node in a cluster, and only collects metrics from the node it runs on. Each model has pros and cons - especially when operating at large scale - which can make it difficult when deciding which model to use.  

In this session, the speakers provide an overview of Prometheus metrics collection at DoorDash, where having highly reliable resources, easy endpoint discovery, and real-time insights is critical. They will share insights and best practices into DoorDash’s decision to implement a decentralized model by offering pros and cons of each approach. Leave with a better understanding of the “right” model for your use case(s).

Date: Tue, 8 Nov 2022
Time: 15:45 - 16:15 CET

Thanks to all who attended and for the interesting questions after our talk.