Docker Engine: the Ultimate Deep Dive

This guide takes you "under the hood" of Docker. It explains the architecture of the Docker Engine, how to manage remote engines, and the Linux technologies (Namespaces & Cgroups) that make containerization possible.

What is Docker Engine?

Docker Engine is not just a single program; it is a modular client-server application with three major components:

1. Docker CLI (Client):

   • This is the command line tool you use (docker run, docker build).

   • It does not do the actual work. It just sends instructions.

2. REST API:

   • The interface (Translator) that allows the Client and Daemon to talk.

   • It follows standard HTTP patterns.

3. Docker Daemon (Server):

   • The background process (dockerd) running on the host.

   • This is the "Brain". It listens for API requests and manages images, containers, networks, and volumes.

Visual Flow

User (CLI) --> REST API --> Docker Daemon (Host)

Remote Docker Engine Management

By default, your Docker CLI talks to the Docker Daemon on the same machine (Localhost) via a Unix Socket (/var/run/docker.sock).

However, you can configure your Laptop's CLI to talk to a Remote Server's Daemon.

Command: Connect to Remote Engine

docker -H=remote-docker-engine:2375 run nginx

OR (using IP):

docker -H=10.123.2.1:2375 run nginx

Detailed Argument Breakdown

• -H (or --host): Critical Argument.

  • It tells the CLI: "Don't talk to the local daemon. Talk to this specific socket or address."

  • Format: tcp://<IP_ADDRESS>:<PORT>

• :2375: This is the default unencrypted port for the Docker Daemon API.

  • Security Note: In production, you should use port 2376 (TLS Encrypted) to prevent hackers from hijacking your engine.

Use Case: You are sitting in a cafe with your lightweight laptop (CLI only), but you are running heavy containers on a massive server in the cloud using the -H flag.

Under the Hood: Containerization

How does Docker isolate containers? It uses native Linux Kernel features. It doesn't use magic; it uses Namespaces.

A. Namespaces (isolation)

Namespaces trick the process into thinking it has its own dedicated computer. It limits what a process can SEE.

1. PID Namespace (Process ID):

   • Isolates processes. A process inside the container thinks it is PID 1 (The Boss), but outside it is just a normal child process (e.g., PID 4500).

2. Network Namespace:

   • Gives the container its own IP address, routing table, and localhost, separate from the host.

3. Mount Namespace:

   • Gives the container its own File System. It cannot see the Host's files unless explicitly mounted.

4. IPC Namespace (InterProcess Communication):

   • Prevents processes in one container from reading memory of processes in another container.

5. UTS Namespace:

   • Allows the container to have its own Hostname.

B. the Pid Matrix (host vs Container)

This is a classic interview concept.

Inside the Container (Child System):

• nginx thinks it is PID 1.

• It sees only itself.

On the Linux Host (Parent System):

• The nginx process is actually visible as PID 3456 (for example).

• It is just a child process of the Docker Daemon.

Resource Management: Control Groups (cgroups)

If Namespaces limit what a container can SEE, Control Groups (cgroups) limit what a container can USE.

Without cgroups, a single container could consume 100% of your CPU and crash your server.

A. Limiting Cpu

Command:

docker run --cpus=.5 ubuntu

• Flag: --cpus

• Value: .5 means "50% of ONE CPU core".

• Importance: Ensures this container never slows down other critical applications by eating all processing power.

B. Limiting Memory (ram)

Command:

docker run --memory=100m ubuntu