Nine Reasons to Use OSH

But I think there are unavoidable tradeoffs at play. The same reasons shell languages are so productive are more or less the same reasons that make shell scripts hard to maintain and hard to be 100% correct. Some ideas in fish and zsh improve some stuff, but in the end, you either choose between the ergonomics of shells or the structure of other programming languages.

Yeah, I 100% agree with you here.

But the kind of problem I was referring to is more about “accidental complexity” than “inherent complexity”. There is no reason that there cannot be a Rust library that combines ergonomic process creation and ergonomic pipelines (with support for in-process pipeline stages!) and a bunch of other facilities for massaging I/O and OS resources, it just… apparently doesn’t exist.

However, the most interesting use of pipes is stuff such as tar c | ssh tar x. For this purpose, pipes are unbeatable, and I would not suggest anything else than a shell for those purposes in most cases.

Yup, this is the kind of task I often need pipelines for.

I fully agree that you don’t need pipelines solely for sed/grep/awk, because in a real programming language, you have real data manipulation tools to replace sed/grep/awk themselves. But even then, to replace sed/grep/awk, you’d want the ability to build true in-process pipelines!

To truly be able to replace sed/grep/awk with real data manipulation tools available to you as part of a real programming language, you’d want something like this instead (this is rough pseudocode, I don’t have nearly enough experience to design APIs on the fly, so please excuse any unsoundness or internal inconsistencies):

let hosts = [ "foo", "bar" ];
let uids = HashMap<(string, u32), usize>::new();
sh.data(&hosts)
    .xargs(|host| {
        cmd!(sh, "ssh", "root@{host}", "cat", "/etc/passwd")
            .stderr(|s| s.to_lines().map(|line| log_as_stderr!("ssh {host}: {line}")))
            .into_stdout()
            .to_lines()
            .map(|line| (host, PasswdEntry.parse(line)))
    )
    .filter(|host, entry| if (host == "somehost") { entry.uid >= 500 } else { entry.uid >= 1000 })
    .map(|host, entry| uids.entry((host, entry.uid)).and_modify(|nr| *nr += 1).or_insert(1))
    .run()?;

FWIW, I’d love to have something like the above in any language. Any ideas?

One thing that I don’t like about Fish (and the reason why I still use ZSH) is because its syntax is so different

That’s exactly the reason I switched from Fish back to Zsh. Even thin

Thanks!

I hope that when the OSH test framework is solidified, you may have a reason and ability to use it at work

OSH code, including the test framework and your tests, will still run under bash (that is done on purpose). So if you want reliability, and not relying on bash-isms [1], you can dip your toes in by using our test framework

Personally I was not very happy with existing test frameworks like BATS and the one that git uses

I know that not many people will take the time to test scripts under say PyPy and CPython, but it does happen, and IMO it is more warranted with bash.

Probably if we add profiling this will be more compelling – it’s something you can’t do with bash, and you don’t have to change any code. Many shell scripts are slow, e.g. forking too many processes in a loop

Recent discussion on that: https://lobste.rs/s/fl7ly9/traceboot_precise_lightweight_tracing

---

BTW I was going through the “project scope” and I came across the previous thread about “wait for it”

https://github.com/zombocom/wait_for_it

https://lobste.rs/s/qsaevf/semaphores_golang_gnu_make#c_eunaux

I am going to hit this myself, so I want to have an idiom for it in YSH … It’s a bit hard because

There are Three Kinds of Language Runtime (a blog post I want to write)

• synchronous or arbitrary - C / Python
• async - Go and node.js are both based around select()-style event loops
• shell - it’s actually based on a waitpid(-1) loop!
  • this is a Unix API problem: you need the self-pipe trick to simultaneously wait for descriptor events and process completion

Just wanted to say I didn’t forget about that … there are multiple possible solutions

---

[1] I keep discovering MORE after 9 years! e.g. with recent bash array work

I haven’t written anything in OSH/YSH but I’ve tried it out briefly and followed its development for a while. I’ve used nushell a bit. They are fascinating in how different their approaches are to development: nushell’s binary is 80mb+ to OSH/YSH’s 2mb; nushell leverages lots of existing rust libraries for functionality, and it feels like a larger group of contributors, while Oils is mostly @andyc and much more self-contained. nushell feels like a fun toolbox (including things that replace standard utils like find) glued together with a smart but quite different approach to the shell; Oils is like a surgical removal and replacement of Bash with something that has had most of its warts removed, and it gives you a nice upgrade path to a world with structured data and nicer syntax and quoting.

I’m sure someone more technical than me could write a nice comparison of technical/design decisions. I will mention my favorite thing about nushell, though, which is the seamless equivalence between function parameters and command arguments/flags (e.g. imagine if Typer was built into the language – no more confusing case+shift statements).

They are both exciting projects! I’m mostly waiting for their 1.0 releases though.

FWIW I added a FAQ here:

https://github.com/oils-for-unix/oils/wiki/YSH-FAQ#whats-the-difference-between-ysh-and-nushell

and BTW I chatted with a nushell developer recently, and they recognized/acknowledged the interior/exterior distinction

It’s a tradeoff, but personally for the stuff I do, “exterior” is generally more powerful because it’s closer to the OS. For Windows, the “interior” design might be more convenient, since Windows exposes a bunch of stuff as shared libraries, which the PowerShell VM can access, etc.

andyc will probably pop in to correct me if I get this wrong:

OSH and YSH are the same binary. Call it with one name and it acts that way.

Oils is a “project” – I would not refer to it as a “product”

Think of OSH and YSH like C and C++ … some people switched to C++ right away; some people switched after 20 years; some people never switched

You can also think of it like Rust/Zig and C – even on the most aggessive timelines, there’s still going to be C in those systems, probably long after we’re dead :-)

So it’s not either-or

It’s also valid to use OSH forever!

---

I’ll add a FAQ about the second question, since that has come up – generally speaking you should not have OSH and YSH code in the same process. You can technically do it, but it will be confusing. Consider when you have “mixed stacks” - OSH calling YSH calling OSH, etc.

But during migration, it may be a useful technique to have YSH shell out to OSH, and vice versa, just like you shell out to awk or whatever