From 75f1d831396a7901a4546963d7bf1f30b2a676c8 Mon Sep 17 00:00:00 2001 From: Sotiris Anagnostopoulos Date: Wed, 8 Oct 2025 00:56:45 +0300 Subject: [PATCH] first commit --- README.md | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 202 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..6b23368 --- /dev/null +++ b/README.md @@ -0,0 +1,202 @@ +# docker-nfsd + +--- + +### Table of Contents +1. [Overview](#1-overview) +2. [Purpose](#2-purpose) +3. [Architecture](#3-architecture) +4. [How it Works](#4-how-it-works) +5. [Why it Was Created](#5-why-it-was-created) +6. [Limitations and Notes](#6-limitations-and-notes) +7. [Installation](#7-installation) +8. [Usage](#8-usage) +9. [Design Philosophy](#9-design-philosophy) +10. [License](#10-license) + +--- + +## 1. Overview + +`docker-nfsd` is a small daemon that allows Docker and Docker Swarm to mount +NFS shares directly through the kernel. +It implements the [Docker VolumeDriver API](https://docs.docker.com/engine/extend/plugins_volume/) +over a UNIX socket and performs real mounts using the standard +`mount(2)` and `umount(2)` syscalls. + +The goal is simple: let Docker request a mount, and let the kernel do the work. + +--- + +## 2. Purpose + +Docker supports external "volume drivers" to manage persistent storage. +In theory, this should make it possible to mount any remote filesystem. +In practice, on Swarm, **distributed storage that “just works” does not exist**. + +Even in 2025, Docker provides **no stable driver for NFS, Ceph, or S3** — +which is absurd, considering NFS has existed longer than Docker itself. + +Existing NFS “drivers” are mostly containerized plugins written in Go or Python, +each with their own runtime, namespaces, and orchestration layers. +They fail on Swarm because mounts are host-level operations that cannot be done +inside a container in any reliable or consistent way. + +`docker-nfsd` was written to solve that: +a native, host-level daemon that performs kernel mounts directly, without tricks. + +--- + +## 3. Architecture + +``` + +┌──────────────────────────────┐ +│ Docker Daemon │ +│ (Engine / SwarmKit) │ +└──────────────┬───────────────┘ +│ HTTP over UNIX socket +▼ +┌──────────────────────────────┐ +│ docker-nfsd │ +│ Implements VolumeDriver API │ +│ - Receives JSON requests │ +│ - Calls mount(2)/umount(2) │ +│ - Exposes mountpoints │ +└──────────────┬───────────────┘ +│ Kernel syscalls +▼ +┌──────────────────────────────┐ +│ Linux Kernel │ +│ NFS client (v4.1) │ +└──────────────────────────────┘ + +```` + +--- + +## 4. How it Works + +1. **Socket registration** + The daemon listens on `/run/docker/plugins/docker-nfsd.sock`. + +2. **Docker interaction** + When Docker needs a volume, it connects to that socket and issues + JSON-encoded requests (`/Plugin.Activate`, `/VolumeDriver.Mount`, etc.). + +3. **Volume creation and exposure** + `docker-nfsd` creates a dedicated mount directory under + `/var/lib/docker-volumes-nfsd//` + and performs a real kernel mount using: + ```c + mount(server:path, target, "nfs4", MS_MGC_VAL, "nfsvers=4.1,rw,noatime,soft"); + ``` + + Once mounted, Docker bind-mounts that directory into the container. + The container never sees NFS directly — only the mounted directory. + +4. **Unmount and cleanup** + When the container stops, Docker calls `/VolumeDriver.Unmount`. + The daemon executes `umount(2)` and releases the directory. + + At no point does this involve FUSE, RPC daemons, or helper binaries. + All logic happens through kernel syscalls. + +--- + +## 5. Why it Was Created + +While setting up Swarm clusters with shared storage, we encountered +the following hard limitations: + +* **Docker provides no reliable host-level storage driver** for NFS or Ceph. +* The so-called “official” NFS drivers run *inside containers*, which + cannot perform kernel-level mounts on the host. +* Other solutions rely on heavyweight sidecars or Go daemons + that introduce complexity without solving the actual problem. + +Our requirement was simple: + +* Persistent shared volumes across Swarm nodes. +* No extra layers of abstraction. +* A driver that survives restarts and behaves like any normal service. + +So we wrote `docker-nfsd` — a 100 KB C daemon that does exactly that. + +--- + +## 6. Limitations and Notes + +* **Privileges:** Requires root privileges only as a normal system daemon. + (No different than `sshd`, `nfsd`, or `dockerd` itself.) +* **NFS Versions:** Uses NFSv4 by default, but older versions are supported + by the kernel and can be negotiated automatically if desired. +* **Concurrency:** Each mount request is independent. + The daemon can handle multiple volumes concurrently — one mount per request, + not one per host. Docker may issue many simultaneous mounts, and it will work. +* **Scope:** Designed for Linux systems using Docker Swarm or standalone Docker. + (If you’re running Swarm on Windows, you’re on your own — and probably deserve it.) + +--- + +## 7. Installation + +### Build and install + +```bash +apt install build-essential libmicrohttpd-dev +make +sudo make install +sudo systemctl enable --now docker-nfsd +``` + +This installs the binary under `/usr/local/sbin/docker-nfsd` +and registers a systemd service unit. + +--- + +## 8. Usage + +Example: + +```bash +docker volume create -d nfsd \ + --opt server=127.0.0.1 \ + --opt path=/exports/data \ + myvolume + +docker run --rm -v myvolume:/mnt alpine df -h /mnt +``` + +`docker-nfsd` will: + +* create `/var/lib/docker-volumes-nfsd/myvolume` +* mount `127.0.0.1:/exports/data` there +* return the mountpoint to Docker + +From Docker’s perspective, it’s a normal persistent volume. + +--- + +## 9. Design Philosophy + +* Written in **pure C** for transparency and performance. +* Uses only **libmicrohttpd** and **syscalls**. +* Does one job, and does it predictably. +* Follows the same principle as every proper Unix daemon: + + > “Start once, listen quietly, do your work, and stay out of the way.” + +--- + +## 10. License + +GPLv2 with the Affero clause (free as in freedom, and free as in beer). + +Use it, modify it, improve it, or ignore it. +Just don’t rewrite it in Go. + +--- + +### — Sotiris from Greece +