Posts about devops(1)

Page 1 of 1
Cool pic

Garage

An S3 object store so reliable you can run it outside datacenters

Most web apps eventually need to handle file uploads — profile pictures, attachments, assets. The instinct is to just write them to disk, but that breaks the moment your app restarts or scales.

Object Storage

Object storage (also called blob storage) is a way of storing data where each piece of data is kept as an independent object rather than inside a traditional file hierarchy or fixed disk blocks. Each object contains the data itself, some metadata describing it, and a unique identifier used to retrieve it.

My Use Case

Sometimes apps run on infrastructure where disk persistence isn’t reliable. That’s the case with Next.js apps, where local filesystem storage is ephemeral — files can disappear after restarts, redeploys, or when the app moves between instances. I needed an object storage solution that matches the production environment.

Why Garage

Garage is a lightweight, self-hosted S3-compatible store that gives you that — without handing your data to a third party. The project is not even hosted in GitHub but in deuxfleurs, which is the collective/organization behind Garage.

NOTE

deuxfleurs (aka 🌼🌼) is a French non-profit that builds decentralized, self-hosted infrastructure tools. Garage was originally built to power their own infrastructure and then open-sourced.

Using something like Garage (or MinIO) makes sense if:

  • You want full control over where data lives — no third party
  • You’re building for a client with data residency requirements
  • You’re deploying the whole stack on your own infra (VPS, bare metal) and want storage there too, not on AWS
  • You want prod to also be self-hosted — in which case Garage in dev mirrors prod exactly

Set Up a Garage Container

You can get familiar with several ways of getting Garage up and running in their quick start. We’ll be using a Docker container, so let’s add the following to our docker-compose.yml file:

services:
# Other services
garage:
image: dxflrs/garage:b72b090a097c8ee2711c8fb065d250ed68dcd0bf
container_name: garage_s3
ports:
- '3900:3900'
- '3901:3901'
environment:
- GARAGE_ALLOW_WORLD_READABLE=1
volumes:
- ./garage_config/garage.toml:/etc/garage.toml
- ./garage_metadata:/var/lib/garage/meta
- ./garage_data:/var/lib/garage/data

Garage Configuration

According to the aforementioned quick start, we need to create a configuration file:

# metadata_dir = "/tmp/meta"
# data_dir = "/tmp/data"
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1
rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
# rpc_secret = "$(openssl rand -hex 32)"
rpc_secret = "b89a46551da109ca0ecac670ae2dff0482a1ab98aa3ec28a96f1c65a656dee34"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"
[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage.localhost"
index = "index.html"
[k2v_api]
api_bind_addr = "[::]:3904"
[admin]
api_bind_addr = "[::]:3903"
# admin_token = "$(openssl rand -base64 32)"
admin_token = "zNZmDBE9Tiqs8CaUmN90Loo/63T+c32x0xC8M0Rjk8U="
# metrics_token = "$(openssl rand -base64 32)"
metrics_token = "69VTAfJSVuaog72X8oxcReRFAQKJJelF4BJzQgeXjvk="

IMPORTANT

This file is mounted as a volume in our Docker setup; the default location where garage looks up this file within the container is /etc/garage.toml.

To get the values for some configuration options, we have to run in the terminal the following commands:

  • openssl rand -hex 32 and paste the output in rpc_secret
  • openssl rand -base64 32 and paste the output in admin_token
  • openssl rand -base64 32 and paste the output in metrics_token

Creating a Cluster Layout

Creating a cluster layout for a Garage deployment means:

  1. Informing Garage of the disk space available on each node of the cluster (-c)
  2. As well as the name of the zone (e.g. datacenter), each machine is located in (-z).

Copy Node ID Prefix

Let’s start Garage with docker compose up garage, and check the status with:

Terminal window
docker exec -it garage_s3 /garage status
INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
INFO garage_net::netapp: Connection established to 2778c9ed427e0ca2
==== HEALTHY NODES ====
ID Hostname Address Tags Zone Capacity DataAvail
2778c9ed427e0ca2 0d45c890ad78 127.0.0.1:3901 NO ROLE ASSIGNED

Then let’s copy the prefix of the node, e.g. 2778 from 2778c9ed427e0ca2, and paste it in <NODE_ID_PREFIX>:

Terminal window
docker exec -it garage_s3 /garage layout assign -z dc1 -c 1G <NODE_ID_PREFIX>

Then apply the layout:

Terminal window
docker exec -it garage_s3 /garage layout apply --version 1

Which should output:

==== COMPUTATION OF A NEW PARTITION ASSIGNATION ====
Partitions are replicated 1 times on at least 1 distinct zones.
Optimal partition size: 3.9 MB
Usable capacity / total cluster capacity: 1000.0 MB / 1000.0 MB (100.0 %)
Effective capacity (replication factor 1): 1000.0 MB
dc1 Tags Partitions Capacity Usable capacity
2778c9ed427e0ca2 256 (256 new) 1000.0 MB 1000.0 MB (100.0%)
TOTAL 256 (256 unique) 1000.0 MB 1000.0 MB (100.0%)
New cluster layout with updated role assignment has been applied in cluster.
Data will now be moved around between nodes accordingly.

Create Bucket

Create bucket, which is the place where our image files will live:

Terminal window
docker exec -it garage_s3 /garage bucket create app-uploads

NOTE

You should get the output: Bucket app-uploads was created.

Add the bucket name to the .env file:

S3_BUCKET=app-uploads

Create Key

Let’s create the key that we’ll use to access the bucket we just created:

Terminal window
docker exec -it garage_s3 /garage key create next-demo-local

Which should output the keys:

Key name: next-demo-local
Key ID: GKbccef323a6f4dedd82f85d59
Secret key: 5ed99b90d1f7f0bd766755cfc3a9daad89ed747fb058dd66c502e544a3ed8f3c
Can create buckets: false
Key-specific bucket aliases:
Authorized buckets:

And paste these values in our .env file:

S3_ACCESS_KEY=GKbccef323a6f4dedd82f85d59
S3_SECRET_KEY=5ed99b90d1f7f0bd766755cfc3a9daad89ed747fb058dd66c502e544a3ed8f3c

Associate Key and Bucket

We need to grant the key access to the bucket:

Terminal window
docker exec -it garage_s3 /garage bucket allow --read --write --owner app-uploads --key next-demo-local

The command above should output something like:

New permissions for GKbccef323a6f4dedd82f85d59 on app-uploads: read true, write true, owner true.

It’s a good idea to verify with this:

Terminal window
docker exec -it garage_s3 /garage key info next-demo-local

Which should output:

Key name: next-demo-local
Key ID: GKbccef323a6f4dedd82f85d59
Secret key: (redacted)
Can create buckets: false
Key-specific bucket aliases:
Authorized buckets:
RWO app-uploads dfc275695b62a17e

NOTE

Note that Can create buckets is false, which means that this key cannot create new buckets, but it can still fully use the buckets explicitly granted to it, in this case app-uploads

Serving Files

Garage in this setup does not allow anonymous object reads, so a raw object URL like http://localhost:3900/app-uploads/events/... cannot be rendered directly in the browser.

NOTE

Whas is an Anonymous Object Read

Imagine you paste http://localhost:3900/app-uploads/events/a.png in browser:

  • Browser sends no S3 auth key/signature.
  • If storage still returns file, that’s an anonymous read. In our setup, Garage rejects that (AccessDenied), so objects are private by default. That’s why our app has to fetch files with credentials server-side (or use signed URLs).

To solve this, we expose a Next.js route handler at app/api/files/[...key]/route.ts. How it works:

  1. The events API returns Mongo data and sets image to a local app URL (for example: /api/files/events/my-image.png).
  2. The browser requests that local URL.
  3. The file route reconstructs the object key from the path (events/my-image.png).
  4. The route fetches the object from Garage using server-side credentials (s3Client + GetObjectCommand).
  5. The route streams bytes back with Content-Type, so the browser can display the image normally.

Why we need it:

  • Keeps Garage bucket private (no anonymous public reads).
  • Browser never needs storage credentials.
  • Works the same way in dev and prod: app API mediates access to object storage.