Blue zen garden stones and raked sand
ライフ ・ バランス

code flows like water balance found in empty lines the terminal waits

Featured Posts

Cool Pic

Microsandbox

Caging the Autocomplete Beast

I wasn’t very happy with my current sandboxing solution (E2B), so decided to give a try to MicroSandbox, and was glad I switched.

What are sandboxes for?

We’re entering an era where AI agents don’t just suggest code, they write it and execute it. They also install packages, run scripts, inspect files, call CLIs, start dev servers, and sometimes touch code we barely understand yet.

That changes the infrastructure we need around development.

If an agent is going to run arbitrary commands, it needs a safe place to do it. Not my laptop. Not my production machine. Not a long-lived environment full of secrets. It needs a disposable, isolated computer where mistakes are contained and experiments are cheap.

That’s what sandboxes are for.

A good sandbox gives each agent its own environment: a filesystem, a shell, network boundaries, resource limits, and a way to run and inspect code without trusting it too much. For AI coding tools, this is becoming table stakes.

The Microsandbox CLI

Before using the Microsandbox SDK, it’s worth getting familiar with what Microsandbox can do through the Microsandbox CLI.

In this post we’ll look into how to:

  • Install the msb CLI.
  • Create a persistent Node-based sandbox.
  • Execute shell commands inside the sandbox.
  • Install runtime tools and project dependencies.
  • Create and run an Astro app inside the sandbox.
  • Expose the Astro dev server through a mapped local port.
  • Inspect, stop, restart, and remove the sandbox.

Installation

We can install the CLI globally, by downloading and running the following script:

Terminal window
curl -fsSL https://install.microsandbox.dev | sh

WARNING

microsandbox requires Linux with KVM enabled, or macOS with Apple Silicon (M-series chip). Both use hardware virtualization.

After the installation, you’ll have to source ~/.zshrc, then verify the installation:

Terminal window
msb --version
msb 0.4.6

Using the msb CLI

In this section we’ll demo how to create a sandbox named astro-demo, which is nothing but a cointainerized Astro app, running in a Microsandbox.

1. Create A Persistent Node Sandbox

Let’s use the msb CLI to create a named sandbox from the node:22.22.3-slim Docker image and publish Astro’s default dev port:

Terminal window
msb run \
--name astro-demo \
--detach \
--replace \
-m 2G \
-c 2 \
-p 4324:4321 \
node:22.22.3-slim

Several things going on in the command above:

  • The --name flag will name our persistent sandbox as astro-demo.
  • The --detach flag to run the sandbox in the background and returns the terminal inmediately.
  • The --replace flag will recreate the sandbox if it already exists.
  • The -m 2G flag sets 2GB of memory.
  • The -c 2 flag sets 2 virtual CPUs.
  • The -p 4324:4321 is mapping ports from host 4324 -> sandbox 4321.
  • We’re using the node:22.22.3-slim Docker image, which gives us Node.js preinstalled, based on Debian slim. The slim variant is smaller than the full Node image, but still supports apt-get, so we can install tools like git and curl.

2. Prepare The Runtime

Now that our sandbox is running, we can install tools into it:

Terminal window
msb exec astro-demo -- sh -c "apt-get update && apt-get install -y git curl"

Enable pnpm as our package manager:

Terminal window
msb exec astro-demo -- corepack enable
msb exec astro-demo -- corepack prepare pnpm@latest --activate

Verify that everything was successfully installed by checking their versions:

Terminal window
msb exec astro-demo -- node --version
msb exec astro-demo -- pnpm --version
msb exec astro-demo -- git --version

3. Create An Astro App

Let’s create the app inside /home/workspace:

Terminal window
msb exec astro-demo -- sh -c "mkdir -p /home/workspace && cd /home/workspace && pnpm create astro@latest . --template basics --no-install --yes"

The command above will create without installing dependencies. Let’s install them:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && pnpm install --dangerously-allow-all-builds"

TIP

The --dangerously-allow-all-builds flag is for pnpm install. It allows packages with build/postinstall scripts to run during installation.

We use it because generated frontend apps often depend on packages that need install-time build steps. Without it, pnpm may skip those scripts and later commands can fail with ERR_PNPM_IGNORED_BUILDS.

Prefer running it in a separate install step:

Terminal window
pnpm install --dangerously-allow-all-builds

instead of passing it through pnpm create, where it may not be forwarded reliably.

If pnpm create astro prompts unexpectedly, use npm instead:

Terminal window
msb exec astro-demo -- sh -c "mkdir -p /home/workspace && cd /home/workspace && npm create astro@latest . -- --template basics --install --yes"

4. Start The Astro Dev Server

Ok, let’s start the Astro dev server on all interfaces so the published port can reach it:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && pnpm dev --host 0.0.0.0 --port 4321"

The command above also starts the sandbox if it was stopped. Now the sandboxed app should be available at https://localhost:4324 👍

TIP

You can start/stop the sandbox at anytime with:

Terminal window
msb stop astro-demo # Stop
msb start astro-demo # Start

Then inspect the logs:

Terminal window
msb logs astro-demo

NOTE

To start the Astro dev server in the background, we need to background the host-side msb exec process:

Terminal window
nohup msb exec astro-demo -- sh -c "cd /home/workspace && pnpm dev --host 0.0.0.0 --port 4321" </dev/null >/dev/null 2>&1 &
disown

Oof, lot to unpack:

  • nohup keeps msb exec alive after shell hangups.
  • </dev/null detaches stdin.
  • >/dev/null 2>&1 detaches output
  • & backgrounds the host msb exec process (outside the "")
  • disown removes it from shell job control.

To stop the background process, find it with:

Terminal window
ps aux | grep "msb exec astro-demo"

Then kill the matching PID:

Terminal window
kill <PID>

It makes things quite difficult if you’re not used to this level of shell-fu, so nothing wrong if you prefer to run it in the foreground, and open new terminal tabs.

5. Run Commands Inside The Sandbox

Using msb we can inspect files within the sandbox:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && ls -la"

Read package metadata:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && cat package.json"

Run a build:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && pnpm build"

6. Test Environment Variables

Run a one-off command with an env var:

Terminal window
msb exec astro-demo -- sh -c "DEMO_ENV=hello node -e 'console.log(process.env.DEMO_ENV)'"

For app runtime env vars, start the dev server with env in the shell command:

Terminal window
msb exec astro-demo -- sh -c "cd /home/workspace && PUBLIC_DEMO_VALUE=hello pnpm dev --host 0.0.0.0 > /tmp/astro.log 2>&1 &"

7. Inspect Sandbox State

The msb CLI offers a bunch of commands for managing sandboxes. We can list them:

Terminal window
msb ls

Show the ones that are running:

Terminal window
msb ps

Inspect a particular sandbox:

Terminal window
msb inspect astro-demo

Show its metrics:

Terminal window
msb metrics astro-demo

Show its logs:

Terminal window
msb logs astro-demo

8. Stop And Resume

To start (or restart) an existing sandbox:

Terminal window
msb start astro-demo

To stop it:

Terminal window
msb stop astro-demo

9. Clean Up

Whenever we are done with a sandbox, we can stop it:

Terminal window
msb stop astro-demo

And remove it:

Terminal window
msb rm astro-demo

Huzzah!

We’ve played a bit with the CLI, now go have fun with the SDK.

Cool pic

A Generic Set

Generics, concurrency and data structures in Go

In computer science, a set is an abstract data type that can store distinct values, without any particular order. In this article we’ll take a look at how to implement this basic data structure using generics and a bit of concurrency.

Before Generics

Before generics were a thing in Go, people used maps to create custom sets when needed:

existingTitles := map[string]bool{}
existingTitles["Getting Started with Go"] = true
existingTitles["Understanding Goroutines"] = true
existingTitles["Getting Started with Go"] = true
if existingTitles["Getting Started with Go"] {
fmt.Println("Title already exists.")
}

The type of the value isn’t important, since we only care about the keys. Using bool is convenient because it lets us check for membership directly in a conditional:

if validCategories[category] {
// whatevs...
}

When you access a map with a key that doesn’t exist, Go returns the zero value for the map’s value type, i.e. false. That way the condition evaluates to false whenever category isn’t in the set.

TIP

If memory usage is a concern, you can use an empty struct as the map value type:

existingTitles := map[string]struct{}{}

Note that I’ve also initialized the map with {}. Without it, the declaration is invalid. If that’s too many braces for you, use:

existingTitles := make(map[string]struct{})

And if you’re only declaring the variable, you could instead write:

var existingTitles map[string]struct{}

Using struct{} for set values is the idiomatic approach in Go because it occupies zero bytes.

This pattern works well and is widely used in Go, but it’s really just using a map to simulate a set. Before generics, you’d need a separate set implementation for every element type — one for strings, one for ints, and so on. With generics, we write it once and let the type parameter do the rest.

A Generic Set

The general syntax for a generic type declaration in Go is:

type TypeName[T Constraint] UnderlyingType

Where:

  • TypeName is the name of the generic type.
  • T is the type parameter (a placeholder for a concrete type).
  • Constraint specifies which types T is allowed to be.
  • UnderlyingType is the actual type definition that uses T.

Our generic Set will take a single type parameter, E, which represents the type of elements stored in the set:

type Set[E comparable] map[E]struct{}

Let’s break it down:

  • E is the type parameter. You can think of it as a placeholder for the actual element type, such as string, int, or User.
  • comparable is the type constraint: It restricts E to types that can be compared with == and !=.

NOTE

Go maps use equality operators to determine whether a key already exists. When you look up a key or insert one, the map compares it with existing keys. Because of this, map keys must be comparable.

A Generic Function

Just like types, generic functions can also have type parameters. The general syntax is:

func FunctionName[T Constraint](parameters) ReturnType {
// ...
}

Where:

  • FunctionName is, well, you guessed it.
  • T is the type parameter.
  • Constraint specifies which types T is allowed to be.

The type parameter can be used in the function’s parameters, return type, or both.

Since we can’t use a map until it’s initialised, let’s provide a constructor function, which will be generic as well:

func NewSet[E comparable]() Set[E] {
return Set[E]{}
}

When we call this function, we need to specify the concrete element type. For example, to create a Set of strings:

names := NewSet[string]()

Note that a generic set is not a complete type until we instantiate it with a concrete type. Each instantiation gives us a different set type; we write the set implementation once, but each set still keeps its own element type.

Adding Elements

Now let’s write a generic method to add values to the set. We’ll make it variadic, so that it can add one or more elements at a time:

func (s Set[E]) Add(values ...E) {
for _, v := range values {
s[v] = struct{}{}
}
}

Since our set is backed by a map, adding an element means assigning that value as a key. The map value is struct{}{} because we don’t need to store any extra data; the presence of the key is enough. For example:

Making Add variadic gives us a more flexible API. We can still add a single element, but we can also add multiple elements in one call:

names := NewSet[string]()
names.Add("Alice")
names.Add("Bob", "Charlie", "Frankie")
fmt.Println(names) // map[Alice:{} Bob:{} Frankie: {} Charlie: {}]

Note how the values are empty structs, {}. That doesn’t look nice, let’s fix it in the next section.

Listing Set Members

Printing the set reveals its underlying map representation, which isn’t ideal if we simply want to inspect its contents. Let’s add a method that returns all the elements in the set as a slice. Because sets are inherently unordered, the order of the elements is undefined.

func (s Set[E]) Members() []E {
result := make([]E, 0, len(s))
for v := range s {
result = append(result, v)
}
return result
}

Let’s take a look now:

names := NewSet[string]()
names.Add("Alice")
names.Add("Bob")
names.Add("Frankie")
fmt.Println(names.Members()) // [Alice Bob Charlie]

WARNING

The order of the elements is not guaranteed and may vary between runs!

String Method

Let’s also implement the String method so that our set prints more nicely:

func (s Set[E]) String() string {
return fmt.Sprintf("%v", s.Members())
}

With this method in place, we no longer need to call Members explicitly when printing a set:

fmt.Println(s) // [2 3 1]

You may notice that the elements appear in a different order each time you run the program, and that’s perfectly fine. By definition, a set is an unordered collection of distinct values. Since our implementation is backed by a map, it naturally inherits the map’s lack of ordering guarantees.

Checking for Membership

A set is primarily useful because it lets us answer a simple question efficiently:

Does this value belong to the set?

For sets, we don’t care about the value at all—we only care whether the key is present:

func (s Set[E]) Contains(v E) bool {
_, ok := s[v]
return ok
}

NOTE

Since our Set is backed by a map, we can take advantage of Go’s comma ok idiom. A map index expression can return two values:

  • The value associated with the key.
  • A boolean indicating whether the key exists in the map.

The blank identifier (_) discards the map value, while ok tells us whether v is a member of the set. For example:

names := NewSet[string]()
names.Add("Alice")
names.Add("Bob")
fmt.Println(names.Contains("Alice")) // true
fmt.Println(names.Contains("Charlie")) // false

Because map lookups are highly optimized, checking for membership in a set is typically an O(1) operation.

Union of Two Sets

A common operation on sets is the union of two sets: a new set containing all the elements that appear in either set.

union set

One way to implement this is to create a new set containing the members of the first set, and then add the members of the second set:

func (s Set[E]) Union(s2 Set[E]) Set[E] {
result := NewSet(s.Members()...)
result.Add(s2.Members()...)
return result
}

As you can see, we’re making use of our existing API to implement the Union:

  • NewSet to create the new set.
  • Add variadic to merge the second set without the need of a loop.

Let’s try it with two small sets of integers:

s1 := NewSet(1, 2, 3)
s2 := NewSet(3, 4, 5)
fmt.Println(s1.Union(s2)) // [1 2 3 4 5]

The resulting set contains every element from both sets, but only once. Even though both s1 and s2 contain 3, it appears only once in the union because sets cannot contain duplicate elements.

Intersection of Two Sets

Let’s also implement intersection. The intersection of two sets is a new set containing only the elements that both sets have in common.

union set

Unlike Union, we can’t simply combine the members of both sets. Instead, we need to examine each element of one set and check whether it also exists in the other:

func (s Set[E]) Intersection(s2 Set[E]) Set[E] {
result := NewSet[E]()
for _, v := range s.Members() {
if s2.Contains(v) {
result.Add(v)
}
}
return result
}

In other words, we iterate over the members of s, and whenever an element is also present in s2, we add it to the result. Let’s try it:

s1 := NewSet(1, 2, 3)
s2 := NewSet(3, 4, 5)
fmt.Println(s1.Intersection(s2)) // [3]

As expected, the only element shared by both sets is 3, so the intersection contains just that value.

Concurrency Safety

Maps in Go are not safe for concurrent use. If multiple goroutines read from and write to the same map simultaneously, the program will panic. To make our Set concurrency-safe, we can embed a sync.RWMutex and lock appropriately on every operation.

A Read-Write Mutex

Let’s update our Set type with an RWMutex:

type Set[E comparable] struct {
data map[E]struct{}
mutex *sync.RWMutex
}

A sync.RWMutex distinguishes between reads and writes:

  • Any number of goroutines can hold a read lock simultaneously. — Write locks are exclusive: when a writer holds it, no readers or other writers can proceed.

This makes it a good fit for a set, where reads (like Contains) are frequent and writes (like Add) are less so.

IMPORTANT

We’ll store a pointer to the mutex rather than the mutex value itself. If we stored it by value, copying the Set would copy the mutex too — and two independent mutexes no longer protect the same data. Since it never makes sense to copy a mutex, defining the mutex field as a pointer prevents this from happening accidentally.

Before generics, duplicating data structures across types made sophistication impractical — nobody wants to maintain N versions of the same complex code. So corners got cut. Generics remove that disincentive, making it worthwhile to build data structures that are actually good.

Constructor

The constructor looks much the same as before, with two additions: it initializes the mutex with &sync.RWMutex{} and the underlying map explicitly, then returns a pointer to the Set struct rather than the struct itself — consistent with storing the mutex by pointer.

func NewSet[E comparable](vals ...E) *Set[E] {
s := Set[E]{
mutex: &sync.RWMutex{},
data: map[E]struct{}{},
}
for _, v := range vals {
s.data[v] = struct{}{}
}
return &s
}

Add

The behaviour of Add is the same as before, with one important difference: it must acquire a write lock before touching the map. This prevents any other goroutine from reading or writing while the update is in progress.

func (s *Set[E]) Add(vals ...E) {
s.mutex.Lock()
defer s.mutex.Unlock()
for _, v := range vals {
s.data[v] = struct{}{}
}
}

defer s.mutex.Unlock() ensures the lock is always released when Add returns, even if something panics mid-loop.

Members

Members doesn’t modify the map, so a write lock would be overkill. A read lock is enough — it allows other goroutines to read concurrently, while still blocking any writer that tries to modify the map at the same time.

func (s *Set[E]) Members() []E {
s.mutex.RLock()
defer s.mutex.RUnlock()
result := make([]E, 0, len(s.data))
for v := range s.data {
result = append(result, v)
}
return result
}

Contains

Contains is also read-only, so the same strategy applies.

func (s *Set[E]) Contains(v E) bool {
s.mutex.RLock()
defer s.mutex.RUnlock()
_, ok := s.data[v]
return ok
}

Putting It to the Test

To verify that our is concurrently safe, we need to exercise the Set concurrently — at least one goroutine writing, another reading, and enough repetitions that a data race has a fair chance to surface. A single read and write might get lucky; a thousand of each is harder to fluke.

s := NewSet(1, 2, 3)
var wg sync.WaitGroup
wg.Add(1)
go func() {
for i := 0; i < 1000; i++ {
s.Add(i)
}
wg.Done()
}()
for i := 0; i < 1000; i++ {
_ = s.Members()
}
wg.Wait()
fmt.Println("Huzzah!") // Huzzah!

If the locking is correct, our program won’t panic, and we should see the output:

Huzzah!

Let’s see what happens if we comment out the locking code in Add and run the same test:

func (s *Set[E]) Add(vals ...E) {
// s.mutex.Lock()
// defer s.mutex.Unlock()
// ...
}

If you run the program, you’d most probably see something like this:

fatal error: concurrent map iteration and map write
goroutine 1 [running]:
...

The Go runtime detected a concurrent read in Members and a write in Add, and since it can no longer guarantee the integrity of the data, it terminates the program immediately.

Intersection: Taming Mutex Churn

We could implement Intersection the same way as before — reusing Members, Contains, and Add — and it would be concurrency-safe without any extra work, since those methods already handle their own locking.

But there’s an efficiency concern. Since we call s2.Contains — which locks and unlocks the mutex — for every member of s, large sets will produce a lot of mutex churn. Mutexes are fast, but they aren’t free, and repeatedly acquiring and releasing a lock in a tight loop adds up.

A better approach is to lock s2 once, do all the membership checks while the lock is held, then release it:

func (s *Set[E]) Intersection(s2 *Set[E]) *Set[E] {
result := NewSet[E]()
s2.mutex.RLock()
defer s2.mutex.RUnlock()
for _, v := range s.Members() {
if _, ok := s2.data[v]; ok {
result.Add(v)
}
}
return result
}

One read lock on s2 for the entire operation, rather than one per element.

NOTE

It’s a trade-off, though. Holding a read lock on s2 for the entire operation avoids mutex churn, but it also means blocking any writer that wants to modify s2 for the full duration of the intersection. For large sets, that could be a long time.

Depending on the application, more granular locking — acquiring and releasing the lock for each individual Contains check — might actually be preferable. Less contention per operation, at the cost of more locking overhead overall. It just depends on whether your bottleneck is mutex churn or lock contention.

Unsafe Set

Not every application needs concurrency safety. If your set is only ever accessed from a single goroutine, the mutex overhead is pure waste. In that case, we can go back to the simpler implementation from the start of this article — same methods, no locking.

Rather than maintaining two entirely separate types, we can offer both through two constructors:

  • NewSet for the concurrency safe default.
  • NewUnsafeSet for when performance matters more than safety.

Users get the right tool for the job, and we only write the implementation once.

Types

The idea is to split the two concerns into two types:

  • UnsafeSet is the raw, lock-free implementation — just a map, same as we started with.
  • Set wraps it, adding a mutex to make it safe for concurrent use:
type UnsafeSet[E comparable] map[E]struct{}
type Set[E comparable] struct {
data UnsafeSet[E]
mutex *sync.RWMutex
}

Constructors

Each type gets its own constructor:

  • NewUnsafeSet just initialises the map and populates it.
  • NewSet builds on top of it, wrapping the UnsafeSet in a Set and initialising the mutex:
func NewUnsafeSet[E comparable](vals ...E) UnsafeSet[E] {
s := UnsafeSet[E]{}
for _, v := range vals {
s[v] = struct{}{}
}
return s
}
func NewSet[E comparable](vals ...E) *Set[E] {
return &Set[E]{
data: NewUnsafeSet[E](vals...),
mutex: &sync.RWMutex{},
}
}

Adding Items

UnsafeSet.Add is the same as before — iterate and assign. Set.Add wraps it with a write lock, ensuring no other goroutine can read or write the underlying map while the update is in progress:

func (s UnsafeSet[E]) Add(vals ...E) {
for _, v := range vals {
s[v] = struct{}{}
}
}
func (s *Set[E]) Add(vals ...E) {
s.mutex.Lock()
defer s.mutex.Unlock()
s.data.Add(vals...)
}

Listing Members

UnsafeSet.Members is the same implementation we wrote earlier: create a slice, iterate over the map keys, and append each element. Set.Members adds a read lock around that same logic:

func (s UnsafeSet[E]) Members() []E {
result := make([]E, 0, len(s))
for v := range s {
result = append(result, v)
}
return result
}
func (s *Set[E]) Members() []E {
s.mutex.RLock()
defer s.mutex.RUnlock()
return s.data.Members()
}

Since Members only reads from the map, the safe version uses RLock instead of Lock.

Checking for Membership

UnsafeSet.Contains uses the comma-ok idiom directly on the map. Set.Contains wraps the same check in a read lock:

func (s UnsafeSet[E]) Contains(v E) bool {
_, ok := s[v]
return ok
}
func (s *Set[E]) Contains(v E) bool {
s.mutex.RLock()
defer s.mutex.RUnlock()
return s.data.Contains(v)
}

Since membership checks only read from the map, the safe version uses RLock.

String Method

UnsafeSet.String formats the set by reusing Members. Set.String does the same through the safe Members method:

func (s UnsafeSet[E]) String() string {
return fmt.Sprintf("%v", s.Members())
}
func (s *Set[E]) String() string {
return fmt.Sprintf("%v", s.Members())
}

Both versions print the set as a slice, and the element order is still undefined.

Union of Two Sets

UnsafeSet.Union creates a new unsafe set, copies the members of the first set, and then adds the members of the second set. Set.Union does the same through safe snapshots of both sets:

func (s UnsafeSet[E]) Union(s2 UnsafeSet[E]) UnsafeSet[E] {
result := NewUnsafeSet(s.Members()...)
result.Add(s2.Members()...)
return result
}
func (s *Set[E]) Union(s2 *Set[E]) *Set[E] {
result := NewSet(s.Members()...)
result.Add(s2.Members()...)
return result
}

The safe version uses Members on both sets, so each read is protected by its own read lock.

Intersection of Two Sets

UnsafeSet.Intersection creates a new unsafe set and adds only the values that also exist in the second set. Set.Intersection takes safe snapshots before doing the same work:

func (s UnsafeSet[E]) Intersection(s2 UnsafeSet[E]) UnsafeSet[E] {
result := NewUnsafeSet[E]()
for _, v := range s.Members() {
if s2.Contains(v) {
result.Add(v)
}
}
return result
}
func (s *Set[E]) Intersection(s2 *Set[E]) *Set[E] {
result := NewSet[E]()
for _, v := range s.Members() {
if s2.Contains(v) {
result.Add(v)
}
}
return result
}

The safe version delegates reads through Members and Contains, so each map access is protected by a read lock.

Deleting Items

UnsafeSet.Delete removes values directly from the map. Set.Delete wraps that same operation with a write lock:

func (s UnsafeSet[E]) Delete(vals ...E) {
for _, v := range vals {
delete(s, v)
}
}
func (s *Set[E]) Delete(vals ...E) {
s.mutex.Lock()
defer s.mutex.Unlock()
s.data.Delete(vals...)
}

Deleting a value that is not in the set is safe: Go’s built-in delete function does nothing when the key does not exist.

Set Length

UnsafeSet.Len returns the number of elements in the map. Set.Len wraps the same operation with a read lock:

func (s UnsafeSet[E]) Len() int {
return len(s)
}
func (s *Set[E]) Len() int {
s.mutex.RLock()
defer s.mutex.RUnlock()
return s.data.Len()
}

Since Len only reads the map, the safe version uses RLock.

Closing Thoughts

This article is not really about building the perfect set implementation. The set is just a useful example: small enough to understand, but rich enough to show how generics and concurrency fit together in Go.

NOTE

That’s not to say the implementation is complete. There’s plenty more we could add:

  • An Equals method
  • An IsSubset method
  • A Subtract method that removes all members that belong to some other set
  • A Difference function to find the members that two sets don’t have in common
  • A Make function to pre-allocate memory
  • Map, Reduce, and Filter operations on set elements

These are left as an exercise for the reader.

If you need a production-ready set package, two good options are:

Cool pic

Garage

An S3 object store so reliable you can run it outside datacenters

Most web apps eventually need to handle file uploads — profile pictures, attachments, assets. The instinct is to just write them to disk, but that breaks the moment your app restarts or scales.

Object Storage

Object storage (also called blob storage) is a way of storing data where each piece of data is kept as an independent object rather than inside a traditional file hierarchy or fixed disk blocks. Each object contains the data itself, some metadata describing it, and a unique identifier used to retrieve it.

My Use Case

Sometimes apps run on infrastructure where disk persistence isn’t reliable. That’s the case with Next.js apps, where local filesystem storage is ephemeral — files can disappear after restarts, redeploys, or when the app moves between instances. I needed an object storage solution that matches the production environment.

Why Garage

Garage is a lightweight, self-hosted S3-compatible store that gives you that — without handing your data to a third party. The project is not even hosted in GitHub but in deuxfleurs, which is the collective/organization behind Garage.

NOTE

deuxfleurs (aka 🌼🌼) is a French non-profit that builds decentralized, self-hosted infrastructure tools. Garage was originally built to power their own infrastructure and then open-sourced.

Using something like Garage (or MinIO) makes sense if:

  • You want full control over where data lives — no third party
  • You’re building for a client with data residency requirements
  • You’re deploying the whole stack on your own infra (VPS, bare metal) and want storage there too, not on AWS
  • You want prod to also be self-hosted — in which case Garage in dev mirrors prod exactly

Set Up a Garage Container

You can get familiar with several ways of getting Garage up and running in their quick start. We’ll be using a Docker container, so let’s add the following to our docker-compose.yml file:

services:
# Other services
garage:
image: dxflrs/garage:b72b090a097c8ee2711c8fb065d250ed68dcd0bf
container_name: garage_s3
ports:
- '3900:3900'
- '3901:3901'
environment:
- GARAGE_ALLOW_WORLD_READABLE=1
volumes:
- ./garage_config/garage.toml:/etc/garage.toml
- ./garage_metadata:/var/lib/garage/meta
- ./garage_data:/var/lib/garage/data

Garage Configuration

According to the aforementioned quick start, we need to create a configuration file:

# metadata_dir = "/tmp/meta"
# data_dir = "/tmp/data"
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1
rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
# rpc_secret = "$(openssl rand -hex 32)"
rpc_secret = "b89a46551da109ca0ecac670ae2dff0482a1ab98aa3ec28a96f1c65a656dee34"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"
[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage.localhost"
index = "index.html"
[k2v_api]
api_bind_addr = "[::]:3904"
[admin]
api_bind_addr = "[::]:3903"
# admin_token = "$(openssl rand -base64 32)"
admin_token = "zNZmDBE9Tiqs8CaUmN90Loo/63T+c32x0xC8M0Rjk8U="
# metrics_token = "$(openssl rand -base64 32)"
metrics_token = "69VTAfJSVuaog72X8oxcReRFAQKJJelF4BJzQgeXjvk="

IMPORTANT

This file is mounted as a volume in our Docker setup; the default location where garage looks up this file within the container is /etc/garage.toml.

To get the values for some configuration options, we have to run in the terminal the following commands:

  • openssl rand -hex 32 and paste the output in rpc_secret
  • openssl rand -base64 32 and paste the output in admin_token
  • openssl rand -base64 32 and paste the output in metrics_token

Creating a Cluster Layout

Creating a cluster layout for a Garage deployment means:

  1. Informing Garage of the disk space available on each node of the cluster (-c)
  2. As well as the name of the zone (e.g. datacenter), each machine is located in (-z).

Copy Node ID Prefix

Let’s start Garage with docker compose up garage, and check the status with:

Terminal window
docker exec -it garage_s3 /garage status
INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
INFO garage_net::netapp: Connection established to 2778c9ed427e0ca2
==== HEALTHY NODES ====
ID Hostname Address Tags Zone Capacity DataAvail
2778c9ed427e0ca2 0d45c890ad78 127.0.0.1:3901 NO ROLE ASSIGNED

Then let’s copy the prefix of the node, e.g. 2778 from 2778c9ed427e0ca2, and paste it in <NODE_ID_PREFIX>:

Terminal window
docker exec -it garage_s3 /garage layout assign -z dc1 -c 1G <NODE_ID_PREFIX>

Then apply the layout:

Terminal window
docker exec -it garage_s3 /garage layout apply --version 1

Which should output:

==== COMPUTATION OF A NEW PARTITION ASSIGNATION ====
Partitions are replicated 1 times on at least 1 distinct zones.
Optimal partition size: 3.9 MB
Usable capacity / total cluster capacity: 1000.0 MB / 1000.0 MB (100.0 %)
Effective capacity (replication factor 1): 1000.0 MB
dc1 Tags Partitions Capacity Usable capacity
2778c9ed427e0ca2 256 (256 new) 1000.0 MB 1000.0 MB (100.0%)
TOTAL 256 (256 unique) 1000.0 MB 1000.0 MB (100.0%)
New cluster layout with updated role assignment has been applied in cluster.
Data will now be moved around between nodes accordingly.

Create Bucket

Create bucket, which is the place where our image files will live:

Terminal window
docker exec -it garage_s3 /garage bucket create app-uploads

NOTE

You should get the output: Bucket app-uploads was created.

Add the bucket name to the .env file:

S3_BUCKET=app-uploads

Create Key

Let’s create the key that we’ll use to access the bucket we just created:

Terminal window
docker exec -it garage_s3 /garage key create next-demo-local

Which should output the keys:

Key name: next-demo-local
Key ID: GKbccef323a6f4dedd82f85d59
Secret key: 5ed99b90d1f7f0bd766755cfc3a9daad89ed747fb058dd66c502e544a3ed8f3c
Can create buckets: false
Key-specific bucket aliases:
Authorized buckets:

And paste these values in our .env file:

S3_ACCESS_KEY=GKbccef323a6f4dedd82f85d59
S3_SECRET_KEY=5ed99b90d1f7f0bd766755cfc3a9daad89ed747fb058dd66c502e544a3ed8f3c

Associate Key and Bucket

We need to grant the key access to the bucket:

Terminal window
docker exec -it garage_s3 /garage bucket allow --read --write --owner app-uploads --key next-demo-local

The command above should output something like:

New permissions for GKbccef323a6f4dedd82f85d59 on app-uploads: read true, write true, owner true.

It’s a good idea to verify with this:

Terminal window
docker exec -it garage_s3 /garage key info next-demo-local

Which should output:

Key name: next-demo-local
Key ID: GKbccef323a6f4dedd82f85d59
Secret key: (redacted)
Can create buckets: false
Key-specific bucket aliases:
Authorized buckets:
RWO app-uploads dfc275695b62a17e

NOTE

Note that Can create buckets is false, which means that this key cannot create new buckets, but it can still fully use the buckets explicitly granted to it, in this case app-uploads

Serving Files

Garage in this setup does not allow anonymous object reads, so a raw object URL like http://localhost:3900/app-uploads/events/... cannot be rendered directly in the browser.

NOTE

Whas is an Anonymous Object Read

Imagine you paste http://localhost:3900/app-uploads/events/a.png in browser:

  • Browser sends no S3 auth key/signature.
  • If storage still returns file, that’s an anonymous read. In our setup, Garage rejects that (AccessDenied), so objects are private by default. That’s why our app has to fetch files with credentials server-side (or use signed URLs).

To solve this, we expose a Next.js route handler at app/api/files/[...key]/route.ts. How it works:

  1. The events API returns Mongo data and sets image to a local app URL (for example: /api/files/events/my-image.png).
  2. The browser requests that local URL.
  3. The file route reconstructs the object key from the path (events/my-image.png).
  4. The route fetches the object from Garage using server-side credentials (s3Client + GetObjectCommand).
  5. The route streams bytes back with Content-Type, so the browser can display the image normally.

Why we need it:

  • Keeps Garage bucket private (no anonymous public reads).
  • Browser never needs storage credentials.
  • Works the same way in dev and prod: app API mediates access to object storage.
Cool pic

Pagefind

Search your site

Pagefind is a search library for static websites. It works by indexing the built static files of our site, and adding a JavaScript search API that allows the user to find content with code that runs client-side.

NOTE

Pagefind can be used with Hugo, Eleventy, Jekyll, Next, Astro, SvelteKit, or any other SSG. There’s even an Astro integration, but we’ll take care of it ourselves, from scratch (it’s more fun and educative).

Installing Pagefind

Pagefind is a static binary, written in Rust with no dynamic dependencies, so installing it would be as simple as:

  1. Downloading the release for our platform (currently supported on Windows, macOS, and Linux distributions).
  2. Place it somewhere in our PATH
  3. And voila!

TIP

The official Pagefind site contains excellent and comprehensive instructions for installing Pagefind

Since we are in Node.js, we can take advantage of the wrapper package for this runtime, and run:

npx pagefind --site "public"

Where public is whatever folder your SSG outputs the generated site, but I’m gonna go ahead and add it to my project dependencies so that I can integrate it in my package.json scripts:

npm i pagefind

Building the Index

As we’ve mentioned at the beginning, Pagefind indexes your site after it builds, so let’s build it:

npm run build

That should output our site’s build to a folder named dist. Now let’s use the pagefind command to index that folder:

pagefind --site dist

That will generate a folder named pagefind under our dist folder (so in dist/pagefind) that contains:

  • The indexed content.
  • The JavaScript that the user’s browser will need to search the indexed stuff.
  • Some UI scripts and stylesheets.

TIP

Probably a good idea to add a pagefind entry to your .gitignore; I added also another one for the dist folder.

Updating our build script

So far we need to use several commands to get Pagefind generating an index of the latest version our our site:

  1. Build the site itself: npm run build
  2. Generating the Pagefind index: pagefind --site dist

It’s probably a good idea to combine these two commands, so that when we deploy our site, whatever pipeline runs the npm run build, will also generate the Pagefind stuff. So let’s update the build script in our package.json file:

"scripts": {
"build": "astro build && pagefind --site dist"
}

Running npm run build now, would produce the usual output, plus the following:

[...]
dist/pagefind
[Building search indexes]
Total:
Indexed 1 language
Indexed 45 pages
Indexed 1285 words
Indexed 0 filters
Indexed 0 sorts

If we open the dist folder (where Astro outputs the build), we should see a pagefind folder with our indexed site.

CAUTION

In my case, that didn’t work due to the way my github workflow was written:

- name: Build with Astro
run: |
${{ steps.detect-package-manager.outputs.runner }} astro build \
--site "${{ steps.pages.outputs.origin }}" \
--base "${{ steps.pages.outputs.base_path }}"
working-directory: ${{ env.BUILD_PATH }}

As you can see, it runs the astro command directly, and not our npm run build. So if you don’t want to mess with your pipeline (no biggie, but hey), you can add the following to your build script:

"scripts": {
"pagefind": "pagefind --site dist && cp -r dist/pagefind public",
"build": "astro build && npm run pagefind"
}

That way, after every build, we end up with the pagefind folder in our public directory, and our GitHub pages deployment won’t have trouble finding it.

Bottom line, you can do it your way, but the whole setup needs the pagefind folder available in public folder.

IMPORTANT

Don’t forget to remove pagefind from your .gitignore file.

Adding the Component UI

Assuming that everytime we build our site, Pagefind indexes its content, we need a UI to search and see the results. Pagefind now recommends the Component UI, which gives us declarative web components for common search interfaces.

NOTE

The previous UI, based on pagefind-ui.js and new PagefindUI(...), still exists in the docs, but the Component UI is the recommended path now.

The nice thing about the new system is that the components follow WAI-ARIA best practices, handle visible and assistive text, and are designed to be hard to misuse.

Loading the Component UI

For the Component UI, we load these files:

<link href="/pagefind/pagefind-component-ui.css" rel="stylesheet" />
<script src="/pagefind/pagefind-component-ui.js" type="module"></script>

In Astro, I added them to my main layout and used import.meta.env.BASE_URL so the paths still work if the site ever lives under a subpath:

---
const pagefindPath = `${import.meta.env.BASE_URL}pagefind/`
---
<link href={`${pagefindPath}pagefind-component-ui.css`} rel="stylesheet" />
<script src={`${pagefindPath}pagefind-component-ui.js`} type="module"></script>

Selecting the Component

After loading the Component UI script and stylesheet, we can choose between a couple of components:

  • Modal search: good when you want a search dialog that opens from the navbar.
  • Searchbox: good when you want a search input with dropdown results.
  • Build your own: good when you want to assemble the input, summary, filters, and results yourself.

For my site, the modal is the best fit:

<pagefind-modal-trigger></pagefind-modal-trigger>
<pagefind-modal></pagefind-modal>

With the modal, Pagefind handles the dialog, focus management, keyboard navigation, and the Cmd+K / Ctrl+K shortcut for us.

Replacing the Custom Search Modal

Before, I had my own search button, my own overlay, and some JavaScript to open and close it. With the Component UI, the markup is enough:

---
const pagefindPath = `${import.meta.env.BASE_URL}pagefind/`
---
<div class="pagefind-search" data-pf-theme="dark">
<pagefind-config bundle-path={pagefindPath}></pagefind-config>
<pagefind-modal-trigger placeholder="Search"></pagefind-modal-trigger>
<pagefind-modal reset-on-close>
<pagefind-modal-header>
<pagefind-input></pagefind-input>
</pagefind-modal-header>
<pagefind-modal-body>
<pagefind-summary></pagefind-summary>
<pagefind-results show-images></pagefind-results>
</pagefind-modal-body>
<pagefind-modal-footer>
<pagefind-keyboard-hints></pagefind-keyboard-hints>
</pagefind-modal-footer>
</pagefind-modal>
</div>

The minimal version would be even smaller:

<pagefind-modal-trigger></pagefind-modal-trigger>
<pagefind-modal></pagefind-modal>

I used the longer version because I wanted to keep result images enabled:

<pagefind-results show-images></pagefind-results>

That’s the Component UI way to enable images in search results.

The pagefind-config component tells the UI where the generated Pagefind files live:

<pagefind-config bundle-path="/pagefind/"></pagefind-config>

In my Astro component, that value comes from BASE_URL:

<pagefind-config bundle-path={pagefindPath}></pagefind-config>

Keeping the Content Indexed

The UI is separate from the indexing markup. My post and project pages still mark their main content with data-pagefind-body:

<div class="post-content" data-pagefind-body>
<Content />
</div>

That tells Pagefind which part of the page should be searchable.

Result Images

For images in search results, I provide explicit Pagefind metadata in the page head:

<meta
data-pagefind-meta="image[content]"
content={post.data.coverImage.cropped.src}
/>
<meta
data-pagefind-meta="image_alt[content]"
content={post.data.coverImage.alt}
/>

One thing to keep in mind is that Pagefind indexes the built site. If images look broken while running the dev server, test with the production build instead:

Terminal window
npm run build
npm run preview

Fixing Result Titles

My search results showed the site title instead of the post title. The reason was subtle: Pagefind’s automatic title metadata comes from the first h1 on the page.

My logo used to be an h1:

<h1 class="anaglyph">生活のバランス</h1>

So Pagefind treated the logo as the page title. The fix was to make the logo a non-heading element:

<span class="anaglyph">生活のバランス</span>

Then I restored the old visual size with CSS:

.anaglyph {
display: inline-block;
font-size: 2em;
font-weight: 900;
line-height: 1.2;
}

Another good option is to explicitly mark the real page title for Pagefind:

<h1 data-pagefind-meta="title">{post.data.title}</h1>

That tells Pagefind exactly what title to use in search results.

Theming the New UI

The Component UI uses CSS custom properties. I scoped them to my search wrapper:

.pagefind-search {
--pf-text: var(--col-text-1);
--pf-text-secondary: var(--col-text-2);
--pf-text-muted: var(--col-text-2);
--pf-background: var(--col-bg);
--pf-border: var(--col-text-3);
--pf-border-focus: var(--col-text-2);
--pf-hover: var(--col-overlay);
--pf-outline-focus: var(--col-accent-1);
--pf-font: 'Mona Sans', sans-serif;
--pf-input-height: 2rem;
--pf-input-font-size: 16px;
--pf-border-radius: 0.3rem;
--pf-modal-backdrop: var(--col-overlay);
--pf-modal-max-width: min(800px, calc(100vw - 2rem));
--pf-modal-max-height: min(80dvh, 800px);
--pf-modal-top: 5rem;
}

This made the Pagefind modal fit the rest of the site’s dark theme.

Fixing Astro ClientRouter Navigation

One issue I ran into was related to Astro’s client-side navigation. The modal worked the first time, but after clicking a search result and navigating to another page, Cmd+K started behaving erratically.

The browser complained with something like:

Failed to execute 'showModal' on 'HTMLDialogElement':
The element is not in a Document.

The problem was that Astro swapped the page DOM, but Pagefind still had references to the old modal internally. So the next time I opened search, Pagefind sometimes tried to open a modal that was no longer in the document.

The fix was to clean up Pagefind before Astro swaps pages:

<script>
if (!document.documentElement.dataset.pagefindCleanupBound) {
document.documentElement.dataset.pagefindCleanupBound = 'true'
document.addEventListener('astro:before-swap', () => {
document.querySelectorAll('pagefind-modal').forEach((modal) => {
if ('close' in modal && typeof modal.close === 'function') {
modal.close()
}
})
const manager = window.PagefindComponents?.getInstanceManager?.()
if (!manager?.getInstanceNames || !manager?.removeInstance) return
for (const name of manager.getInstanceNames()) {
manager.removeInstance(name)
}
})
}
</script>

That closes any open Pagefind modal and clears Pagefind’s cached component instances before Astro swaps the page. The next page then creates fresh modal and trigger references.

Final Check

After adding the UI, rebuild the site:

Terminal window
npm run build

Pagefind should scan the built HTML, find the data-pagefind-body elements, and generate the index. After that, the search trigger should open the modal, Cmd+K / Ctrl+K should work, and the results should show the correct post titles.

Featured Projects

Zentry

Zentry Clone

A cool landing page

This time I decided to build a landing page that closely follows zentry. Doing this project I realize that having good graphic assets can push your design quite far, with less effort.

See it live here 👈

Used Technologies

Light theme Saas

Progress tracker

A light themed landing page

A landing page about a fictitious Saas, where I used responsive design (mobile, tablet and desktop) in a light theme. I added some parallax effects and attractive animations.

See it live here 👈

Used Technologies