Kubernetes Migration from Docker Compose

Docker Compose

If you’re unfamiliar with Docker Compose, it’s a framework that lets developers define container-based apps in a single YAML file. The Docker images utilized, exposed ports, dependencies, networking, and so on are all included in this specification. Docker Compose is a convenient tool that makes application deployment a breeze.

Preparing for the Docker to Kubernetes migration

Learning how Kubernetes differs from Docker Compose was the first challenge in transforming the project. Container-to-container communication is one of the most noticeable differences.

Containers in a Docker Compose environment run on a single host system. Docker Compose establishes a local network in which all containers are connected. Take a look at the sample below.

This block will construct a quoteServices container with the hostname quote-services and the port 8080. Any content within the local Docker Compose network can use http://quote-services:8080 to access it using this definition. Anything outside the local network needs to know the container’s IP address.

Kubernetes, on the other hand, is typically operated on numerous computers known as nodes thus it can’t just construct a local network for all the containers. I was worried before we started that this would result in many code modifications, but those fears were unjustified.

​​

Creating Kubernetes Migration YAML files

The easiest way to grasp the migration from Docker Compose to Kubernetes is to view a live demonstration of the process. Let’s take the quoteServices snippet above and change it to a Kubernetes-friendly format.

Remember that the Docker Compose block will be split into two parts: a Deployment and a Service.

As the name implies, the deployment tells Kubernetes most of what it needs to know about how to deploy the containers. This covers details such as what to call the containers, where to get the images from, how many containers to make, and so on.

As we previously noted, networking in Kubernetes differs from Docker Compose. The Service is what allows containers to communicate with one another.

This service description instructs Kubernetes to use quote-services as the hostname and port 8080 to access containers with a name = quoteServices as described under the selector. So, from within the Kubernetes application, you can access this Service at http://quote-services:8080. Because of the flexibility of defining services in this manner, we can maintain our URLs intact within our application and avoid making any changes due to networking considerations.

We turned a single Docker Compose file with around 24 blocks into roughly 20 separate files towards the end, most of which had a deployment and a service. This conversion was a crucial step in the migrating process. We utilized Kompose to build deployment and services files automatically at first to save time. We did, however, wind up editing all of the files after we recognized what we were doing. Using Kompose to create web pages is similar to using Word to create web pages. Sure, that works, but once you figure out what you’re doing, you’ll probably want to redo most of it because it adds a lot of useless tags.

Server Visibility agent

​It was very straightforward to incorporate the Server Visibility (SVM) agent. Docker Compose works on a single host, but Kubernetes Migration commonly uses several nodes that can be added or withdrawn dynamically.

In our Docker Compose approach, we deployed our SVM agent to a single container, which watched the host machine and the multiple containers. With Kubernetes, we’d have to run one of these containers on each cluster node. The easiest method to do this is to use a DaemonSet structure.

So one can see in the example above, a DaemonSet looks a lot like a Deployment. They are almost identical. The primary distinction is how they behave. A Deployment typically does not specify where the containers defined within it should be executed in the cluster, but it does limit the number of containers to construct. On the other hand, a DaemonSet will run a container on each node in the group. Because the number of nodes in a cluster might grow or shrink at any time, this is critical.

What works great

The work to transition Docker to Kubernetes adds some burden in development and operations, but there are clear benefits. I won’t go through all of the benefits here, but I will tell you about two of my favorites.

First and foremost, the Kubernetes monitoring dashboard is fantastic. It displays information on running containers, deployments, and services, among other things. You can also use the UI to update, add, or delete any of your definitions. So all I have to do when I make a change and create a new image is edit the image tag in the deployment definition. The old containers will be deleted, and new ones will be created with the updated title. It also offers any containers simple access to log files or a shell.

Dashboard for Kubernetes

Another benefit is that we no longer need to keep and manage the host machines on which our Docker Compose applications were executing. Containerizing apps are based on treating servers more like cattle than pets. While this is somewhat true, the Docker Compose host machines have evolved into new pets. We’ve had problems with host machines breaking down, requiring repair, running out of disk space, and so on. There are no more host computers with Kubernetes, and nodes in the cluster can be spun up and down at any time.

​​

Conclusion

I was worried about intra-application networking, deployment procedures, and adding extra layers to all of our operations before starting our Docker to Kubernetes conversion. True, we’ve increased the amount of configuration from a 300-line docker-compose.yaml file to over 1,000 lines divided across 20 files. However, this is primarily a one-time expense. We also had to rewrite some code, which was necessary in any case.

In exchange, we received all of the benefits of an accurate orchestration tool: scalability, increased container visibility, easier server management, and so on. The process will be much easier and faster when it comes time to move our next application, which won’t belong.

Enteros

About Enteros

Enteros offers a patented database performance management SaaS platform. It proactively identifies root causes of complex business-impacting database scalability and performance issues across a growing number of RDBMS, NoSQL, and machine learning database platforms.