What is Prometheus, and what are the four obstacles to business adoption?

What is Prometheus and 4 challenges for enterprise adoption

Prometheus has become a basis for our cloud-native environment due to cloud migration. When we first started learning about Kubernetes, it became our flying cockpit. Nevertheless, if you don’t want a bad Kubernetes experience, there are a few things to keep in mind. This post will describe what Prometheus is, how it works, and what challenges you can face if you decide to use it in your business. If you’re unfamiliar with the name, let me clarify what Prometheus is before proceeding.

What is Prometheus?

Prometheus is an open-source monitoring and alerting toolkit that many businesses and organizations have widely adopted. Its popularity has risen due to the community’s vast number of Exporters. Also, the toolkit collects and saves metrics in a time-series database, which means that data is held alongside the timestamp it took and optional key-value combinations known as labels.

How Prometheus works

​The idea is straightforward. It will collect metrics from each target specified in the configuration file. External solutions that expose metrics in a Prometheus format are referred to as targets, and the targets are referred to as exporters in the Prometheus language.

An exporter’s metrics are an example of what can be exposed.

Prometheus includes all of the tools you’d expect for monitoring cloud-native systems, including:

1. Many exporters, developed by the community and vendors, enable us to collect data on Hardware (Netgear, Windows, IBM Z, etc.) 

• Table of Contents (MySQL, CouchDB, Oracle, etc.) 
• Messages (MQ, Kafka, MQTT, etc.) 
• Keeping things in storage (NetApp, Hadoop, Tivoli, etc.) 
• Others are (Gitlab, Jenkins, etc.) 

2. A Prometheus SDK that allows us to expose our custom metrics. 

3. Automation. All of Prometheus and Grafana’s configurations might be done automatically by: 

• Changing the appropriate configuration files: 

4. Add new scraping endpoints to Prometheus.yaml (exporters) 

5. Build graphs as code using Grafana’s JSON format using Alertmanager.yaml to define new alerts in Prometheus. 

Let’s take a closer look at how Prometheus monitors your technology now that we’ve learned about the features it offers. Two services support it:

1. The server Prometheus

• ​​It features a retrieval component that collects metrics from different sources (a solution exposing metrics in a Prometheus format is called an exporter) 
• We will use a storage component to store the metrics in a time-series database. 
• An HTTP server that serves as a user interface for developing our PromQL and API, Grafana, will use to display metrics in a dashboard. 

2. The manager of alerts

• An alert administrator is a tool for triggering Prometheus rules-based notifications.

Prometheus’s architecture in K8s

The majority of users are using a Helm Chart to deploy it, which will automatically deploy the following components in your Kubernetes cluster:

1. The Prometheus server is a server that was created by Prometheus (also known as The Prometheus Operator) 

2. The Alert Manager is a program that allows you to manage your 

3. Several exporters include:

• Metrics for Kube State 
• Exporter of nodes 
• CadvisorsGrafana 

As a result, these standard exporters give the level of detail required to assess the health of our Kubernetes cluster.

Let’s take a closer look at each exporter’s current activities:

• Will report all metrics about node status, node capacity, and Kube State metrics (CPU and memory) 

-Compliance with replica sets (desired/available/unavailable/updated replica status per deployment)

-Status of the pod (waiting, running, ready, etc.)

-Requests for resources and their limits

– Job and Cronjob Situation

• Each node in our cluster will expose hardware and OS metrics, which Node Exporter will report. 
• Advisor will gather information on the health of your containers. 

As a result, you can easily integrate your dashboard setting, notifications, and the deployment of additional exporters into your continuous delivery workflows with each exporter.

In Kubernetes, the Prometheus Operator additionally adds new resources to make scraping and alerting configuration much easier:

• Kubernetes
• sServiceMonitor
• sAlertmanager 

Also, all of the services that expose Prometheus metrics will be mapped by the ServiceMonitor. The ServiceMonitor features a label selector that allows you to choose services and the information needed to scrape the metrics from them:

• Port
• sInterval

What are the limitations?

Prometheus, however, has few limits that are relevant to businesses. It means that while bringing it to an organization, the owner is responsible for considering how to carry out these actions effectively:

• By default, security is not provided. 
• Access control: By default, everyone has access to all data. 
• Only vertical, not horizontal, scaling is possible. 
• Extra components are required for global visibility. 

Let’s take a closer look at many restrictions one by one:

Security: Not secure by default

Prometheus currently lacks native support for:

• TLS is a data encryption security mechanism.

We’re all aware that data can be susceptible to digital transformation. When it comes to security, most companies have pretty rigorous policies. Because these don’t support it, you’ll have to use the current workaround of delegating the SSL handshake to solutions like Ngninx.

Access Control: Everyone can see all data by default

The Prometheus operators will collect data from your cluster’s deployed and configured exporters. Anyone with access to the Prometheus UI can see your data. Applying filters to your Grafana dashboards allows you to restrict access to specific metrics. However, there are no options to manage privilege-based access to your metrics.

Scalability: Only vertical, not horizontal scaling

Prometheus is a fantastic solution, but it was not built to scale. The most significant drawback is that it is not meant to scale horizontally, so you won’t be able to load balance the burden by adding several Prometheus servers. Instead, it can only scale vertically, which means that if you want a bigger Prometheus server, you’ll need to add more resources.

There’s a reasonable probability that once deployed, you’ll:

1. Your company begins to use a large number of exporters to report:

• Metrics on health 
• The efficiency of your CI/CD process 
• Data on release management 
• Keep track of your operations tasks with ops metrics (I.e., backup) 

2. Custom metrics comprising business information and technical insights about the behavior of your application are valuable to your developers.

The number of services in Kubernetes will grow, and the metrics collected by Prometheus will grow as your company completes its digital transformation.

In Prometheus, “diseases” can be diagnosed by looking for problems when refreshing your Grafana dashboard. Prometheus’ memory use is proportional to the number of time-series data kept on the server.

Remember that a million measurements will need around 100 GB of RAM. So, if you’re not careful, Prometheus might gobble up most of your Kubernetes cluster’s resources.

Global visibility: Requires extra components.

To bypass the technical restrictions outlined above, we can handle our application burden with multiple small Kubernetes clusters rather than a few large clusters.

You’ll need to manage numerous Prometheus servers and Grafana if you’re in charge of multiple clusters, which poses the question of how we can ensure that our project owners and operators have access to data from various groups. Also, the short answer is to set up a single large Grafana server that connects to all of your Prometheus servers. While this adds to the complexity of maintaining our Prometheus architecture, it allows us to give our team members more visibility.

Solutions

So, how can we handle these limits without burdening our Kubernetes Operators with additional responsibilities? Numerous solutions, such as Thanos, will address scalability and security issues for Prometheus.

But there’s another issue that we all want to tackle when integrating observability in our environments: tying our metrics, logs, and traces to a context.

About Enteros

IT organizations routinely spend days and weeks troubleshooting production database performance issues across multitudes of critical business systems. Fast and reliable resolution of database performance problems by Enteros enables businesses to generate and save millions of direct revenue, minimize waste of employees’ productivity, reduce the number of licenses, servers, and cloud resources and maximize the productivity of the application, database, and IT operations teams.