Docker in production

For some time now I have been using Docker mainly because the facilities it provides to streamline the development process. However, after much thought and encouraged by my curiosity to learn, I decided to move one of my productive applications (real clients) completely to a Docker-based environment. Let me be clear, I am usually moving the application for several servers (every 2 or 3 months) and the truth is that the whole process of server provisioning can become exhausting especially after doing it 3 or 4 times 🙂

Docker is an extremely useful tool if you like to deliver a “Portable” application, knowing that in this way can be executed in another server without any complications. A simple application that doesn’t have a lot of execution nodes can use Docker perfectly but you would not be taking advantage of all the benefits of Docker. When you have a slightly more complex environment (as it usually happens in production) is when you see the real benefits of using Docker. To put a bit in context, the application I’m moving is a web scraper that obtains and generates information based on a set of properties that my clients select. if you notice that the description is very vague it is with all intention 🙂

To better show it here you have a diagram of each of the nodes that compose the application:

App Nodes

All green nodes are a different Docker container, where:

  1. Nginx Proxy: is basically the entry point to handle all HTTP requests through the port 80 and 443, and redirect them to a different Virtual Hosts.
  2. Fronted: developed in Django, is responsible for allowing me to manage users as well as the different configurations of the web scrapper.
  3. Worker Processes: they are several processes to scraping some web sites that are constantly running and work according to the configurations that each of my users has selected.
  4. Web Proxy Manager: developed in Node.js I use it to be able to proxy all our web requests. For more information and features you can see the details here: (https://luminati.io/)
  5. Web Database Administrator: is a web application, developed in PHP, for database administration. More information: (https://www.adminer.org/)
  6. Docker Web Administrator: I really like to have the ability to manage my containers and see their statistics from the web, mostly because I usually travel a lot and it’s much more simple to see it on the web than connecting by ssh to see the status of the containers.
  7. Log Collector: for the collection of the logs from containers, especially containers that are part of the core of my application I use Datadog (https://www.datadoghq.com/). If you’ve never used it, I strongly recommend it.
  8. Redis: well known for his excellent performance I use it to manage dynamic configurations and also to store cache data for my application. (https://redis.io/)

To make the process of configuring the dockers easier I use docker-compose, basically allows describing the services to each of nodes in just one file. For my production environment I am running all services in a single-host environment, but if you wanted to go around a multi-host you can always use docker swarm or Kubernetes.

An important detail that can not be left out is that the Mysql database is not in a container, sincerely the opinions regarding it are divided (https://stackoverflow.com/questions/48515460/is-it-recommended-to-use-database-as-a-container-in-production-environment). My modest experience after conducting a series of tests with and without containers show me that when i use containers for the database a decrease in the performance is observed, without being able to figure how exactly happen, my application had a lot of connection problems when the concurrence began to increase . Without going any further scientific answer to the problem , and after proving that without using the container this behavior did not happen, my recommendation is not to use containers for production databases.

Since I started using Docker in the production environment, exactly 29 days have passed at the time of writing this post and everything has been fine.

My containers

The actual workload of the server has not changed significantly, therefore I can not attest to performance improvements or vice versa. What is real is the easiest administration of the application from the update of my system using CI/CD to the administration of the logs. I believe that using Docker in production is a question based on the type of application environment needed rather than on the improvements that Docker will have to execute in itself. If you are thinking in to move your production environment to Docker you must be clear about why you are doing it and how much you would be really earning in the process, for me these were my key questions:

  1. How much will your administration work be facilitated if your entire execution environment is on Docker?
  2. Does your application have a good CI/CD process that allows you to get all the benefits of Docker?
  3. Is your log system Docker ready? You can read more here: (https://docs.docker.com/config/containers/logging/)
  4. Do you have a clear idea of how many containers will be running in production? Remember Dockers recommends a single process per container.
  5. What distribution are you thinking of using? Container distribution like RancherOS or more traditional distro like Ubuntu or Centos.

For me, moving the application environment to Docker was a real necessity, as well as the need to play around with Docker 🙂 In short, like almost always there is no right answer. The best for you would depend on many circumstances. But as a good friend says, if it works well do not touch it. In case you are thinking of moving to Docker, leave your comments here and let me know how it goes. If you already did tell us about your personal experience.

Here you have the docker-compose file with the description of all services I’m using right now:

https://gist.github.com/sdaechoes/aceb44d6fed0570e52440ac28bcf9076

A

Leave a Reply

Your email address will not be published. Required fields are marked *