Docker Storage & Filesystem: the Complete Reference

This guide explains how Docker manages data, the internal file system structure, the layered architecture (with precise sizes), and how to persist data using Volumes and Bind Mounts.

The Docker File System

When you install Docker, it creates a specific directory structure on your Host machine to manage all data.

Location: /var/lib/docker

Inside this directory, you will find the following subdirectories:

• aufs (or overlay2): Stores the image layers and storage driver data.

• containers: Stores configuration files and logs for running containers.

• image: Stores metadata about images.

• volumes: Stores persistent data (This is where Docker volumes live).

Layered Architecture (image Building)

Docker images are built using a Layered Architecture. Each instruction in the Dockerfile creates a new layer.

Scenario A: Building the First Image

Command: docker build Dockerfile –t mmumshad/my-custom-app

The Dockerfile:

FROM Ubuntu
RUN apt-get update && apt-get -y install python
RUN pip install flask flask-mysql
COPY . /opt/source-code
ENTRYPOINT FLASK_APP=/opt/source-code/app.py flask run

The Layers & Sizes:

1. Layer 1 (Base Ubuntu): 120 MB

2. Layer 2 (apt packages): 306 MB

3. Layer 3 (pip packages): 6.3 MB

4. Layer 4 (Source code): 229 B

5. Layer 5 (Entrypoint): 0 B

Total Size: ~432 MB

Scenario B: Building the Second Image (optimization)

Command: docker build Dockerfile2 –t mmumshad/my-custom-app-2

The Dockerfile2:

FROM Ubuntu
RUN apt-get update && apt-get -y install python
RUN pip install flask flask-mysql
COPY app2.py /opt/source-code     # <--- Only this changed
ENTRYPOINT FLASK_APP=/opt/source-code/app2.py flask run

The Layers & Sizes (Caching Magic):

1. Layer 1: 0 MB (Cached - Reused from previous build)

2. Layer 2: 0 MB (Cached)

3. Layer 3: 0 MB (Cached)

4. Layer 4: 229 B (New Layer - Because source code changed)

5. Layer 5: 0 B (Update Entrypoint)

Key Takeaway: Docker is efficient. It only stores the difference (the new source code layer).

Container Layer vs. Image Layers

When you run a container (docker run), Docker adds a new layer on top.

The Structure:

1. Image Layers (Read-Only):

   • The bottom 5 layers we built above.

   • You cannot modify these directly.

2. Container Layer (Read-Write):

   • Layer 6: A thin, writable layer added on top.

   • Any file you create (temp.txt) or modify (app.py) is stored here.

Copy-On-Write (CoW): If you edit app.py (which is in the Read-Only Layer 4):

1. Docker finds app.py in Layer 4.

2. It copies it up to the Container Layer (Layer 6).

3. You edit the copy. The original file remains unchanged in the image.

Lifespan:

• If you delete the container -> The Container Layer (Layer 6) is destroyed.

• Data inside it is lost.

Persisting Data: Volumes

To prevent data loss (e.g., losing your MySQL database), we must map the container's data folder to the Host.

A. Data Volumes (managed by Docker)

This is the preferred way. Docker manages the storage location.

Create Volume:

docker volume create data_volume

(This creates a folder at /var/lib/docker/volumes/data_volume)

Run with Volume:

docker run -v data_volume:/var/lib/mysql mysql

• Syntax: -v volume_name:container_path

• Result: Data is saved in /var/lib/docker/volumes/data_volume. Even if you delete the container, the volume remains.

B. Bind Mounts (host Directory)

This maps a specific folder on your host machine (e.g., your project folder) to the container.

Run Command:

docker run -v /data/mysql:/var/lib/mysql mysql

• Syntax: -v /host/path:/container_path

• Use Case: Development (Editing code on your laptop and seeing changes in the container instantly).

C. the New Syntax (--mount)

This is the modern, verbose way to mount volumes.

Command:

docker run \
  --mount type=bind,source=/data/mysql,target=/var/lib/mysql \
  mysql

• type=bind: Use a host directory.

• type=volume: Use a Docker volume.

Storage Drivers

Docker uses Storage Drivers to manage the layered filesystem and the Copy-On-Write mechanism. The driver depends on your OS.

List of Drivers:

• Overlay2 (Current Default - Best performance)

• AUFS (Old default for Ubuntu)

• ZFS

• BTRFS

• Device Mapper

• Overlay