Docker Fundamentals

What is Docker?

Docker is an open-source platform that automates the deployment, scaling, and management of applications using containerization technology. Containers allow developers to package an application with all its dependencies into a standardized unit for software development. Docker has revolutionized how we build, ship, and run applications by providing a consistent environment across development, testing, and production.

Unlike virtual machines, containers share the host system's kernel, making them lightweight and fast. This means you can run many more containers on a given hardware combination than virtual machines. A typical server can run dozens or even hundreds of containers simultaneously, compared to just a handful of virtual machines.

The History of Docker

Docker was created by Solomon Hykes and released as an open-source project in March 2013. The technology was originally developed as an internal project at dotCloud, a Platform as a Service (PaaS) company. Docker quickly gained popularity due to its simplicity and the problems it solved for developers dealing with the "it works on my machine" problem.

In 2017, Docker Inc. donated the core container runtime (containerd) to the Cloud Native Computing Foundation (CNCF), establishing it as an industry standard. Today, Docker is the de facto standard for containerization, and its image format has become the foundation for container ecosystems including Kubernetes.

Docker Architecture

Docker uses a client-server architecture with the following key components:

Docker Daemon (dockerd)

The Docker daemon is the background service running on the host that manages Docker objects including images, containers, networks, and volumes. It listens for Docker API requests and manages Docker objects. The daemon can also communicate with other daemons to manage Docker services.

Docker Client

The Docker client is the primary way users interact with Docker. When you use commands like docker run, the client sends these commands to the Docker daemon, which executes them. The Docker client can communicate with local or remote Docker daemons.

Docker Registry

A Docker registry stores Docker images. Docker Hub is the default public registry that anyone can use. You can also run your own private registry using Docker Registry, Amazon ECR, Google Container Registry, Azure Container Registry, or other solutions. When you use docker pull or docker run commands, Docker pulls required images from your configured registry.

Docker Objects

• Images - Read-only templates used to create containers. Images are built from a series of layers, with each layer representing an instruction in the Dockerfile.
• Containers - Running instances of Docker images. Containers are isolated from each other and from the host, but you can control how isolated they are.
• Networks - Allow containers to communicate with each other and with the outside world. Docker provides several network drivers.
• Volumes - Persist data generated by containers. Volumes are stored on the host filesystem and managed by Docker.

Installing Docker

Installing Docker on Linux (Ubuntu/Debian)

Docker provides official installation scripts and packages for most Linux distributions. Here is the process for Ubuntu:

# Update package index
sudo apt-get update

# Install prerequisites
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg lsb-release

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Set up the stable repository
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list

# Install Docker Engine
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io

# Add your user to the docker group (optional, for running without sudo)
sudo usermod -aG docker $USER

Installing Docker on macOS

Docker Desktop for Mac provides an easy-to-use application that includes Docker Engine, Docker CLI, Docker Compose, and other tools. Download Docker Desktop from the official Docker website and follow the installation wizard. Docker Desktop uses a lightweight Linux VM to run containers on macOS.

Installing Docker on Windows

Docker Desktop for Windows is available for Windows 10/11 with WSL 2 (Windows Subsystem for Linux 2) or Hyper-V. WSL 2 is the recommended backend as it provides better performance and Linux compatibility. Download Docker Desktop from the official Docker website and ensure WSL 2 is enabled before installation.

Understanding Docker Images

What is a Docker Image?

A Docker image is a read-only template containing instructions for creating a Docker container. Images are composed of layers, where each layer represents a set of filesystem changes. When you modify an image, only the changed layers need to be rebuilt, making the build process efficient.

Image Layers

Each instruction in a Dockerfile creates a new layer. Layers are cached, so unchanged layers do not need to be rebuilt. This layer caching significantly speeds up build times. Understanding how layers work helps you write efficient Dockerfiles.

Base Images

Base images are the starting point for your Docker images. Common base images include:

• Alpine - A minimal Linux distribution (approximately 5MB), great for production
• Debian/Ubuntu - Full-featured Linux distributions with extensive package repositories
• Distroless - Google's minimal images containing only your application and runtime dependencies
• Scratch - An empty image, used for statically compiled binaries

Working with Docker Images

Searching for Images

# Search Docker Hub for images
docker search nginx

# Search with filters
docker search --filter is-official=true nginx

Pulling Images

# Pull the latest version
docker pull nginx

# Pull a specific version
docker pull nginx:1.25

# Pull from a different registry
docker pull gcr.io/google-containers/nginx:latest

Listing Images

# List all images
docker images

# List images with specific format
docker images --format "table {{.Repository}}	{{.Tag}}	{{.Size}}"

# List dangling images (unused image layers)
docker images -f dangling=true

Removing Images

# Remove a specific image
docker rmi nginx:latest

# Remove all dangling images
docker image prune

# Remove all unused images
docker image prune -a

# Force remove an image
docker rmi -f image_id

Working with Docker Containers

Running Containers

# Run a container in the foreground
docker run nginx

# Run in detached mode (background)
docker run -d nginx

# Run with a custom name
docker run -d --name my-nginx nginx

# Run with port mapping
docker run -d -p 8080:80 nginx

# Run with environment variables
docker run -d -e MYSQL_ROOT_PASSWORD=secret mysql

# Run with volume mounting
docker run -d -v /host/path:/container/path nginx

# Run with resource limits
docker run -d --memory="512m" --cpus="1.5" nginx

# Run interactively with a shell
docker run -it ubuntu bash

Managing Containers

# List running containers
docker ps

# List all containers (including stopped)
docker ps -a

# Stop a container
docker stop container_name

# Start a stopped container
docker start container_name

# Restart a container
docker restart container_name

# Remove a container
docker rm container_name

# Remove a running container (force)
docker rm -f container_name

# Remove all stopped containers
docker container prune

Inspecting Containers

# View container logs
docker logs container_name

# Follow logs in real-time
docker logs -f container_name

# View container resource usage
docker stats

# Inspect container details
docker inspect container_name

# View container processes
docker top container_name

# Execute command in running container
docker exec -it container_name bash

Dockerfile Deep Dive

A Dockerfile is a text document containing instructions to build a Docker image. Each instruction creates a layer in the image. Understanding Dockerfile instructions is essential for building efficient and secure container images.

Common Dockerfile Instructions

FROM

Specifies the base image. Every Dockerfile must start with FROM (except ARG before FROM).

FROM python:3.11-slim
FROM node:18-alpine
FROM ubuntu:22.04

WORKDIR

Sets the working directory for subsequent instructions.

WORKDIR /app

COPY and ADD

Copy files from the build context into the image. COPY is preferred; ADD has extra features like URL downloading and tar extraction.

COPY requirements.txt .
COPY . /app
ADD https://example.com/file.tar.gz /tmp/

RUN

Execute commands during the build process.

RUN apt-get update && apt-get install -y curl
RUN pip install --no-cache-dir -r requirements.txt

ENV

Set environment variables.

ENV PYTHONUNBUFFERED=1
ENV NODE_ENV=production

EXPOSE

Document which ports the container listens on. Does not actually publish ports.

EXPOSE 8080
EXPOSE 443/tcp

CMD and ENTRYPOINT

Define the default command to run. CMD can be overridden; ENTRYPOINT is the fixed executable.

CMD ["python", "app.py"]
ENTRYPOINT ["python", "app.py"]
ENTRYPOINT ["python"] 
CMD ["app.py"]

ARG

Define build-time variables.

ARG VERSION=latest
FROM python:${VERSION}

USER

Set the user for subsequent instructions and the running container.

RUN useradd -m appuser
USER appuser

Example Dockerfile for Python Application

# Use official Python runtime as base
FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# Create non-root user
RUN useradd --create-home --shell /bin/bash appuser

# Set work directory
WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY --chown=appuser:appuser . .

# Switch to non-root user
USER appuser

# Expose port
EXPOSE 8000

# Run the application
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "app:app"]

Multi-Stage Builds

Multi-stage builds allow you to use multiple FROM statements in your Dockerfile. This is useful for creating smaller production images by separating build-time dependencies from runtime dependencies.

Example Multi-Stage Build

# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

In this example, the final image only contains nginx and the built static files. All build tools (Node.js, npm, source code) are left behind in the builder stage, resulting in a much smaller image.

Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services, networks, and volumes. It simplifies the orchestration of multiple containers that work together.

Docker Compose File Structure

version: '3.8'

services:
  web:
    build: ./app
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgres://db:5432/myapp
    depends_on:
      - db
    networks:
      - backend
    volumes:
      - ./app:/app

  db:
    image: postgres:15
    environment:
      POSTGRES_DB: myapp
      POSTGRES_PASSWORD: secret
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - backend

  redis:
    image: redis:7-alpine
    networks:
      - backend

networks:
  backend:
    driver: bridge

volumes:
  postgres_data:

Docker Compose Commands

# Start all services
docker compose up

# Start in detached mode
docker compose up -d

# Stop all services
docker compose down

# Stop and remove volumes
docker compose down -v

# View logs
docker compose logs

# View logs for specific service
docker compose logs web

# Scale a service
docker compose up -d --scale web=3

# Rebuild images
docker compose build

# Execute command in a service
docker compose exec web bash

Docker Networking

Docker networking allows containers to communicate with each other and with the outside world. Docker provides several network drivers for different use cases.

Network Drivers

• bridge - Default network driver. Containers on the same bridge network can communicate.
• host - Removes network isolation. Container uses the host's network directly.
• none - Disables networking for the container.
• overlay - Enables multi-host networking for Docker Swarm services.
• macvlan - Assigns a MAC address to a container, making it appear as a physical device.

Network Commands

# List networks
docker network ls

# Create a network
docker network create my-network

# Connect container to network
docker network connect my-network container_name

# Disconnect container from network
docker network disconnect my-network container_name

# Inspect a network
docker network inspect my-network

# Remove a network
docker network rm my-network

Docker Volumes

Volumes are the preferred way to persist data generated by Docker containers. Unlike bind mounts, volumes are managed by Docker and work on both Linux and Windows containers.

Volume Types

• Named Volumes - Docker manages the storage location. Best for production.
• Bind Mounts - Map a host directory to a container directory. Good for development.
• tmpfs Mounts - Store data in the host's memory. Data is not persisted.

Volume Commands

# Create a volume
docker volume create my-volume

# List volumes
docker volume ls

# Inspect a volume
docker volume inspect my-volume

# Remove a volume
docker volume rm my-volume

# Remove all unused volumes
docker volume prune

# Use a volume in a container
docker run -d -v my-volume:/data nginx

# Use a bind mount
docker run -d -v /host/path:/container/path nginx

Docker Security Best Practices

Image Security

• Use official or verified images from trusted sources
• Regularly scan images for vulnerabilities using tools like Trivy, Snyk, or Docker Scout
• Keep base images updated to get security patches
• Use minimal base images (Alpine, Distroless) to reduce attack surface
• Sign images using Docker Content Trust

Container Security

• Run containers as non-root users whenever possible
• Use read-only file systems when the container does not need to write
• Limit container resources (CPU, memory) to prevent resource exhaustion
• Do not run containers in privileged mode unless absolutely necessary
• Use security profiles like AppArmor or seccomp

Secret Management

• Never store secrets in images or environment variables in Dockerfiles
• Use Docker Secrets for Swarm deployments
• Use secret management tools like HashiCorp Vault
• Mount secrets as files rather than environment variables when possible

Docker Best Practices

Dockerfile Best Practices

• Use specific image tags, not 'latest', for reproducible builds
• Order instructions from least to most frequently changing for better caching
• Combine RUN commands to reduce layers
• Use .dockerignore to exclude unnecessary files from build context
• Use multi-stage builds to create smaller production images
• Clean up in the same layer as installation to avoid layer bloat

Performance Optimization

• Use Alpine or Distroless base images to minimize image size
• Remove package manager caches in the same layer as installation
• Use build cache effectively by ordering Dockerfile instructions properly
• Use BuildKit for faster and more efficient builds

Troubleshooting Common Issues

Container Fails to Start

# Check container logs
docker logs container_name

# Check container exit code
docker inspect container_name --format='{{.State.ExitCode}}'

# Run container interactively to debug
docker run -it image_name bash

Networking Issues

# Check container IP address
docker inspect container_name | grep IPAddress

# Check if port is exposed
docker port container_name

# Test connectivity from host
curl http://localhost:8080

Storage Issues

# Check disk usage
docker system df

# Clean up unused resources
docker system prune

# Clean up everything including volumes
docker system prune -a --volumes

Conclusion

Docker has transformed how developers build, ship, and run applications. By understanding Docker's architecture, mastering Dockerfile creation, and following best practices, you can create efficient, secure, and portable containerized applications. The skills you learn with Docker form the foundation for working with container orchestration platforms like Kubernetes.

# Pull an image from Docker Hub
docker pull nginx:latest

# List all images
docker images

# Run a container
docker run -d -p 8080:80 --name my-nginx nginx:latest

# List running containers
docker ps

# Stop a container
docker stop my-nginx

# Remove a container
docker rm my-nginx

# Remove an image
docker rmi nginx:latest

# Use an official Python runtime as base
FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# Set work directory
WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy project
COPY . .

# Expose port
EXPOSE 8000

# Run the application
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]