← Blog
Mayank

Our open-source build is a compile target, not a branch

Niblr's public repo is generated from the private one on every release: hosted code deleted, seams swapped, a leak scan, then CI that migrates and builds it. How it works, and what broke first.

Niblr's open-source build is not a branch of the private repo and not a fork of it. It is generated from it. On every release tag an export deletes the hosted code, swaps thirteen seam files for single-store equivalents, regenerates the database types from the result, refuses to ship if a private identifier survives, and force-pushes one commit to a public repository. Here is why, and what went wrong on the way there.

What is actually in the public repo?

The single-store build. One store, no subdomains, no signup, no billing, no plan limits, no platform operator tools. It is MIT-licensed, it runs the same storefront, admin, checkout, invoicing, tax and page builder that merchants use on the hosted service, and it is at v1.0.4-oss today.

What is not in it: the multi-tenant hosting layer and everything that only makes sense when you are running stores for other people. That part is the paid product and it stays private. I want to be precise about this because "we are open source" is said loosely by a lot of commerce companies, including ones whose public repo is a client library. The honest version of the claim is: the build you can self-host is open, free, and the same code — and the machinery for hosting other people's stores is not.

Why generate it instead of just making the repo public?

Because the private repo contains two products, and one of them is the business.

The obvious alternative is a configuration flag: ship everything, and let an environment variable collapse the app to a single store. We ran exactly that for a while — NEXT_PUBLIC_SINGLE_TENANT=true, two dozen branch points spread over twenty-seven files. It works, and it is the wrong answer for an open-core product, for a reason that has nothing to do with licensing: a flag can be flipped back. If the multi-tenant code is present and one boolean stands between a reader and it, then the thing separating the free build from the paid one is a boolean. That is not a boundary, it is a speed bump.

So the flag is gone from the core code. There is no multi-tenant path in the public repo to switch on, because the code is not there to switch to.

How does the export know what is private?

One file. oss/manifest.json lists the paths that exist only in the hosted product, as globs. It is the single definition of "hosted", and three different things read it: the export that deletes those paths, a linter rule that stops anyone importing them, and a test that walks every import in the repo and fails if core code reaches into hosted code.

That last one matters more than it sounds. The rule is that core never imports hosted, and it is enforced on every pull request rather than remembered. When you try to write the import anyway, CI tells you to add an extension point instead.

Where core genuinely needs something the hosted product provides, it goes through a seam: a small file that re-exports the hosted implementation, and has a twin of the same shape that does the single-store thing. There are thirteen of them. The one that decides which store a request belongs to is the clearest example. Hosted resolves it from the request host, because a hosted request could be for any of thousands of stores. The public twin returns the one store, because there is only one. Core calls the same function either way and never learns which world it is in.

The pattern is the same one behind an earlier post about a rule that was enforced in code but not in the copy: if a boundary is real, something should fail when you cross it.

What stops something private slipping through?

Two things, and only one of them is the obvious one.

The obvious one is a scan. The export greps the generated tree for a list of identifiers that must never appear in it, and exits non-zero if it finds any. It runs on every pull request, not just at release, so the failure lands on the change that caused it.

The better one is subtler. The exported build regenerates its own database types from its own configuration, and then typechecks. The single-store configuration has no tenants table and no tenant column, so those types do not describe one — and any line of core code still reaching for a tenant simply stops compiling. It is not a rule I wrote and have to maintain. It is the type system noticing that a thing does not exist. Roughly two hundred query sites moved over to store-scoped helpers in one pass because this check told me, precisely and mechanically, which ones I had missed.

What broke on the way out?

The first tagged release took four attempts, and then the public repository's own CI failed anyway. That last failure is the one worth writing down.

The three failed attempts were unglamorous. The workflow did not pin a package manager version, so it never started. Then the build was killed for using more memory than the runner has. Then the same build, given a smaller memory ceiling, swapped until it passed the job time limit. That one I fixed by not building there at all — the exported tree needs about 6.6 GB to build and the runner has 7 GB, so the public repository builds it instead, on a bigger machine, which is a slightly absurd sentence to write but is genuinely the right split.

Then the release shipped, and the public repository ran its own CI for the first time. The export had been typechecking and running its 2,330 tests on every pull request for weeks, all green. The public run failed in under a minute: Cannot find module 'stripe'.

The export prunes dependencies that only the hosted product needs, and Stripe was on that list, because Niblr's own subscription billing runs on Stripe. But the merchant-facing Stripe gateway — the adapter a shop owner uses to take card payments with their own keys — is core, and it imports the same package. Pruning it broke the public build.

The reason nobody caught it is the part I keep thinking about. The generated tree was being checked inside the private repository, in a subdirectory. Node resolution walks up the filesystem looking for node_modules, found the private repo's, and quietly supplied the very package the export had just removed. Every check passed because the thing under test was still standing in the room it was supposed to have left.

The fix took one line. Getting to it cost three failed release runs and a red build in public: a check that runs inside the thing it is checking is not a check. Both workflows now copy the export out of the workspace and install it from its own lockfile before running anything against it. If the export forgets a dependency now, it fails on our machine rather than in public.

This is the same shape as every bug in that week where my tests passed and my code was wrong: the assertion was correct and the environment was lying.

What does CI actually prove?

The public repository ships with its own workflow, because a repository that cannot demonstrate it runs is a tarball with extra steps. On every push it typechecks, runs the unit suite and lints. Then it starts an empty Postgres, applies the single migration the export ships, builds the app, and runs a six-case browser smoke test against the production build: create the first user, save the store settings, add a category and a product, load the storefront, the catalogue, the product page and the cart, confirm the API is closed to anonymous readers, and check the 404.

That last set is the one I actually care about. It is the difference between "the code compiles" and "a stranger who clones this can open a store." If you want to do that, the self-hosting guide and the deployment guide are the two pages to read.

Can you send a pull request?

Yes, and I want to be straight about what happens to it, because this is the fair criticism of the whole approach and I would rather say it than have it said to me.

The public repository is generated. A pull request opened against it cannot be merged there in the normal way, because the next release would overwrite it. What happens instead is that the change is ported into the private repository, and it arrives in the public one on the next release tag, with the contributor credited.

That is worse than a normal open-source workflow in one specific way: an outside contributor cannot become a maintainer of the repository they are contributing to, and how fast their change lands depends on us rather than on them. I do not have a clever answer to that. It is the cost of the private repo being the source of truth, and the alternative — two repositories that drift — is worse for everyone, especially the person self-hosting.

What you do get is the thing that actually matters when you are deciding whether to depend on software: the code you would run is the code we run, you can read all of it, the licence cannot be revoked, and if we disappear you have a build that migrates and boots on your own hardware. That is checkable today rather than promised.

The repo is github.com/mcmayank/open-commerce-oss. The export script is in it too, which felt like the right way to end this.