Brut, the Brutal Router for Unix Tools

17 points by sstephenson


andyc

create programs that are highly portable, able to run on any contemporary Unix system with no runtime dependencies other than the basic utilities specified by POSIX.1-2024. (Of course, your programs may have their own runtime dependencies; that’s fine. But everything in Brut adheres to the POSIX constraint.) That means you can run your program on a wide range of hosts, from a Linux virtual machine, to a BSD server, to a Mac laptop, to a tiny single-board computer

I don't believe in "just use POSIX shell" ... Here is a recent article (not from me) that makes some good points:

I actually forgot about this example, and it's pretty bad:

$ sh -c "echo 'a\nb'"
a
b
$ bash -c "echo 'a\nb'"  # same program has a different meaning!!
a\nb

# POSIX doesn't help you !!!

i.e. busybox ash and dash are the default /bin/sh on Alpine and Debian respectively, and they differ a lot from bash, which is the /bin/sh on other distros (I think Red Hat based systems, although I am not sure. Maybe Arch)

Also I am not sure but on OS X did /bin/sh used to be bash 3 (which has many differences from bash 4 !!), and then it changed to zsh? Or do they have a different stable /bin/sh ? I suspect the former


If you really want to make shell portable in shell, you will get into crazy hacks like this: https://github.com/modernish/modernish

Modernish is a library for shell script programming which provides features like safer variable and command expansion, new language constructs for loop iteration, and much more.

i.e. It's not even that you can't use echo, but you have to turn shell into a different programming language. Look at modernish code -- it's a different language, that happens to run on many shells.


As far as Oils, I would say OSH makes a large subset of bash a reasonable portable target -- it is the most bash-compatible shell in the world: https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/spec/compat/TOP.html

And YSH is a different language, which is better done in native code, at the interpreter level, than in shell (as modernish does)

Verfeuil

I wish the intended goal was made a bit more clear. Is that a """framework""" to write shell scripts ?

spillybones

I think the end goal of this tool is not desirable (to me, at least). The point of grouping together commands like git add and git commit into a single binary is not just to allow you to type git add instead of git-add. The point is that the two subcommands share a ton of logic (parsing and operating on the git file structure). So, if you wanted to compile git-add and git-commit as separate binaries, you would be duplicating the same code for each subcommand. By my count, git has 178 subcommands and the git binary is 4.7MB. So assuming that most of that binary size would be shared by different subcommands, that would be over 800MB of duplicated binaries and a rather annoying build process.

hunger

So I have to use a shitty programming environment, have a mess of files on my hand that I somehow need to get into place, have to work around all the pecularities of basic posix utilities on different platforms, and in exchange I get slow execution speed with tons of processes spawned all over the place.

I am not sold on this one...

hunger

So I have to use a shitty programming environment, write tons of boilerplate code, have a mess of files on my hand that I somehow need to get into place, have to work around all the pecularities of "basic posix utilities" on different platforms, and in exchange I get slow execution speed with tons of processes spawned all over the place.

I am not sold on this one...

dutc

Thank you for sharing. I'm surprised by how little imagination we have with the possibilities of better shell scripting.

However, I am trying to figure out the motivation for this tool.

I suspect it is a consequence of shell scripting never having developed internal mechanisms for modularisation—e.g., whereas in Python, we may use conda or pip to install packages, no such standard package manager exists for shell (instead, we have our system package manager); whereas in Python, we may write, distribute, and import libraries to incrementalise the complexity of our work, in shell we write separate executables; whereas in Python we may subordinate our functionality to a larger system like a web framework, there really isn't any equivalent in shell, other than subordination to process execution mechanisms provided by the kernel or by systemd or its equivalent.

Since I strongly suspect this is the case, I am skeptical that we gain much through the unification of effort around a single shell scripting framework, which is what I think this project means in describing itself as a “router.” If effort could be so readily unified, then why not unify it around a programming language which provides these missing modularisation, structuring, and organisational tooling? Surely, we already have ways to distribute software to end users without needing a package manager or compiler available—e.g., there are statically linked binaries as well as a variety of dependency isolation and packing tricks like Python venv. If I am going to unify my effort around something that I want to deploy, which I want to work in some uniform fashion across systems, I might as well deploy a Janet or Zig executable; distributing a shell script (or a directory of shell scripts) doesn't really confer much advantage over this, and it comes at the cost of having to write “POSIX shell.”

(I am similarly skeptical that writing “POSIX shell” is something that anyone actually does—as the other comment illustrates, it's questionable whether this thing actually exists, and, even if it did, there are very references and tools that can affirm that I have succeeded in writing such a thing. More likely, I'll write a very constrained subset of mostly inoffensive Bash/Dash/Busybox Ash-like that bootstraps me into something more standardised that does most of the work… and wait for user complaints if this fails, at which time I may very well decline to support outliers.)

Instead, it seems like the usefulness of shell is precisely in its “messiness.” I currently have probably more than fifty thousand lines of hand-written shell across probably more than two hundred scripts that I use in my work. Some of these help with daily tasks; some for projects I will never revisit. It has taken me many years to develop an effective “style” for writing (zsh) shell scripts, since shell is, in generally, one of the most “under-opinionated” programming languages (i.e., it provides insufficient systematic guidance to users, so while it is mostly syntactically quite simple to write shell scripts, users are on their own to discover latent or emergent patterns and structures. As a result, irrespective of whether I can defend my zsh style, it probably comes across as extremely idiosyncratic…) I have some shell scripts that I think are written extremely well—to the same quality I could achieve in programming languages like Zig, Python, C++, Janet, &c.—and I have some shell scripts that are not. I might defend the latter by saying that those scripts did not demand the same exactitude or that they were written before I discovered systematic approaches, but it is precisely this unevenness that I think is core to shell scripting's success. It's “messy,” it's “uneven,” it's just kind of all over the place, and yet it's probably one of the most successful programming languages ever developed, if we measure by the totality of useful efforts accomplished in its manifestations. Terrible yet terrific.

I have no motivation to touch (much less rewrite) a working shell script, except to add new functionality. (Despite taking great pride in the quality of my work, conformance with an unmeasured or unmotivated standard of “elegance” has never been sufficient motivation for me to ever to anything.)

(It is notable that despite the large quantity of shell scripts that I have had to write, I have tried many times to develop common libraries and frameworks and have failed in every attempt. I have recently come to the conclusion that this organisational strategy is antithetical to effective shell scripting, because the resulting compositions are far too brittle, and the goal of shell scripting should be decomposable or disaggregated automation.)

As a result, I suspect any organisational effort that mandates, for example, precise directory structures for shell scripts are likely to be successful outside of specific operational niches (e.g., Pacman PKGBUILD, mkinitcpio, arch-boxes, &c. all have some framework-like organisation around what are effectively shell scripts, and these seem to be mostly successful efforts.) Instead, if the genesis of this framework is to address the shortcomings of shell—namely that it lacks the internal (organisational) mechanisms that we have come to expect from high-level programming languages—then it occurs to me that that is what should be addressed

There is surprising richness in this area. For example, Linux namespaces are extremely useful but do not (logically or mechanically) compose. As a result, there is space for a lot of creativity and ingenuity to devising a compositional mechanism for namespacing for shell scripts, and the availability of such a mechanism would dramatically improve the correctness of a lot of work while also simplifying the work itself.

Similarly, there seems to have been little progress in extending the piping metaphor. Of the author's other projects, I think dyad is actually more interesting and potentially more useful than brut, but I think it's still thinking too small.

Inspired by recent work in Zig, I have come to the conclusion that effective and useful tools inevitably find themselves composed in deeper and deeper ways. If I write a very useful shell script, it will get called from another shell script which is called from a subprocess.run in a Python script, which is triggered through a method on a class in a Python library, which is part of a Python CLI, which is called through a REST API, which is triggered by a shell script… I suspect the widespread use of LLMs will significantly amplify this compositional depth: their answer to nearly any question is “write more code.” In the case of shell scripts, I think our new compositional strategies need to address the deficiencies that arise from this extreme composition. For example, in brut, it seems we have shell scripts calling shell scripts calling shell scripts to accomplish some task. (After all, this is what we might expect from a “router.”) I don't think this is an effective direction for shell scripting for a variety of reasons (e.g., performance,) which is why I don't think a routing framework is likely to be particularly useful as an organisational strategy.

Ultimately, though, managing compositional complexity, designing for decomposition and disaggregation, finding new compositional strategies, and developing new compositional mechanisms is very important work.

Thank you to the author for sharing this work, and providing an opportunity to discuss these topics.

arcade

I think the literate programming style is very cool