I hate bash, but it's always the first thing I start scripting in when I need to glue some things together. I tend to go in the following order as complexity and a need for robustness increases.
1. Bash (or ysh, which I do prefer but isn't as available)
2. Python, using the `sh` package as needed. It's been great for upgrading shell scripts, and `sh` is really nice to use.
3. Rust, usually. That's my personal language of choice for "real" projects.
I have a number of projects at each stage actually, though most of the things I'd call a real "project" with it's own repo end up as Rust projects.
Perl excels at most of their list, though I understand why they aren't considering it. Just funny that you look at the list and one language in particular pops to mind.
In defense of bash, in spite of its many footguns... you do yourself a disservice if you refuse to learn bash, which is rock-solid stable and has basically ubiquitous deployment. SSH into any server or use any UNIX-like environment and you are instantly at home. While JS and Ruby ecosystems churn every year and mandate yet another round of dependency hell and keeping up with the latest and greatest in rubocop bikeshedding, bash just keeps on ticking throughout the years with little to no recurring maintenance burden.
That being said, if you are doing something other than just piping together processes and managing unstructured streams of flat plaintext, bash is the wrong tool for the job. bash is best used interactively, in the REPL, for having a back-and-forth conversation with the machine and interrogating its current state (filesystem, resource usage...), or that of other machines over the network (that is, using dig, curl, openssl, ping, mtr, etc), and processing streams of text. Even for lightly structured data, like if I am doing boring, garbage CRUDdy tedium on the web and need to figure out how to do some rudimentary transform of a JSON document, figure out which headers to send and which endpoints to hit, curl and jq it is - and there I am, reshaping the data in as interactive of an environment it gets.
The minute I need to write something for a non-interactive, end-user use case, with requirements for better ___domain modelling / structuring of data, testing, deployment, readability, maintainability, extensibility, etc. it is time to reach for a more modern language. But if you want to talk to the machine, virtually any (non-Windows) machine, interactively, bash is your lingua franca, or perhaps your Latin/Ancient Greek/Aramaic, esoteric and outdated as it is.
I also get the feeling that people just recoil at any level of complexity and need to hide behind their clean, simple abstractions these days. As a Haskell enthusiast, I am also susceptible to this tendency, but one needs to be at least capable of diving into the historical, messy work underneath the hood.
There is a bunch of bash that you should not learn. The differing coprocess syntax from korn is one particular bugbear. Still, if you are hacking on ReaR or other critical tools that sink deep into bash's slang, you don't have much choice.
The Debian/Ubuntu family uses the "Debian Almquist Shell" (dash) as the system shell (#!/bin/sh) for a number of reasons. It is very much not compatible with anything fancy from bash (or korn). The bash shell is available as #!/bin/bash, but that opens the door to a new set of problems (in that POSIX mode is not enabled).
In the embedded realm, busybox also implements the Almquist shell, but it does throw simple bash "slang" back in (but not this does not include arrays).
How often don't you have bash on a debian/ubuntu system though? #!/bin/bash and away you go. Every workplace I've been that uses debian just goes for bash in the shebang.
OpenWRT is undoubtedly using busybox. I actually saw the busybox copyright on a point-of-sale terminal at Old Navy a few years ago. The embedded realm is vast, and bash is too big for it, or intolerable because of gplv3.
Android uses mksh as the system shell. The more that you know of the korn dialects, the more comfortable you will be in scripting on Android. There is no native bash, as Android's userland is BSD. That realm is also vast.
Finally, Debian/Ubuntu does give you /bin/bash, but if you don't understand the need for "set -o posix" then the scripts that you wrote for CentOS will fail in many interesting ways, because you are carrying the baggage of bash's first decade, before the POSIX shell standard.
So yes, cutting your scripting away to pure POSIX.2 has considerable worth. It runs everywhere.
Whenever people look for "better shell scripting", they always run into the second-system effect. Trying to shove everything that wasn't in the first system, into the second system.
Bash shell scripting is already close to perfect for what it is intended. Where people always complain is one of three areas: 1) "footguns", 2) "not enough features", 3) "not a programming language", 4) inexperience.
1) is almost entirely due to it being a programming-like language bolted onto an interactive console terminal. Bash has a lot of changes over vanilla Bourne/Posix that make it less console-y and more programming-y. But if you take away the interactive-terminal-ness itself, then you don't need most of the Bashisms.
2) is almost always a mistake. Bash has pretty much all the features you need. But again, it's hampered by the interactive-terminal-ness; take that away, and you can do pretty much anything, with ease.
3) is because programmers don't understand what scripts are for. They are not for writing robust programs. They are for solving a small problem, in a specific, tailored way, with the least amount of effort and time. This means that you should be abandoning all of the programmer's best practices and rules. Throw out safety, reliability, consistency, verifiability, etc. What you end up with is a solution that works, with nothing extra.
4) comes when people simply don't know how to use Bash very well, which is 90% of the time. Ever since I became a programmer two decades ago, the great unwashed masses of techno-dweebs who call themselves "programmers" have been picking up complex and nuanced technology that they have never bother to formally learn, and then complain that it sucks and they need an alternative. When in reality, the problem is almost always either a) they didn't read the manual, b) they haven't practiced using it, or c) they're using the wrong tool anyway.
Doesn't matter if you disagree with me. We will be using the same shell scripts in 20 years. Know why? Because you can't make it any simpler, and it just works. Complain all you want about it not being perfect. You simply will not be able to make something better without simultaneously making it worse. Its killer feature is its lack of features and design. It is the anti-programming language. And boy do I love it.
"Bash shell scripting is already close to perfect for what it is intended."
It is very difficult to write a parser for any derivative of the POSIX shell. The lex/yacc utilities cannot be used; the lexer and parser must exchange state.
It would be better if the shell was an LR-parsed language with a yacc grammar. The yacc utility is itself in POSIX.2.
Here is a talk about an OCaml implementation of a parser for the POSIX shell:
And I don't really care. I'm super comfortable with one-liners in my bash shell. I can do useful stuff in bash scripts, but typically with a base script gets to more than a dozen or so lines, I find myself way more productive rewriting it in Python instead. Not because Python is "better" or because bash is "worse", but because _I'm_ better with Python and _I'm_ worse with bash. And I don't feel any urgent need to get better at bash, because the work I do means I pretty much always have Python available (at least everywhere I have bash available).
I know bash is way more capable than my kindergarten-grade use of it. And it'll remain in my toolkit and crontab for as long as unix-like OSen exist. But I'll be very unlikely to ever keep writing a bash script past a dozen or two lines ever again.
> Ever since I became a programmer two decades ago, the great unwashed masses of techno-dweebs who call themselves "programmers" have been picking up complex and nuanced technology that they have never bother to formally learn, and then complain that it sucks and they need an alternative. When in reality, the problem is almost always either a) they didn't read the manual, b) they haven't practiced using it
This kind of Dunning-Kruger-esque arrogance in software is intensely aggravating to me. Nobody presumes that you can master, say, a musical instrument, without intensive practice over a timespan of years; yet for some reason people can get plopped out the other end of a bootcamp and suddenly identify as world class programmers in 16 weeks without the requisite thousands of hours of practice with hands on keyboard, reading the manual, studying how the computer works, and actually solving technical problems.
I've been around a while. As in I've installed Linux from floppies kind of vintage.
I get new grads with great grades, who are decent junior devs that are productive in, say, Java and one or maybe two related things like SpringBoot or Hikari, but don't even know about simple/common things that sometimes baffle me.
They seem to think I'm some sort of wizard because I can quickly put together simple commands line pipelines using things like grep, awk, sed, sort, uniq, wc - and dig useful answers or insights out of log files or csv files or database exports, or even JSON by using jq as well.
It amazes me that some of them have never even heard of the existence of tools like that, or how easily you can pipe the output from one to the input of another and chain them up in such useful ways.
(At the same time, I'm fairly sure I have similar black holes in my knowledge and mental map of how things that other people specialise in work. :shrug: )
> It amazes me that some of them have never even heard of the existence of tools like that
I'm not a recent grad by any means. I have heard of those tools and even used them on occasion. The problem is "on occasion". There's no way I want to keep relearning awk every time it might be useful.
These days, though, LLMs are a saving grace. It's rare that I ask it to provide me with a command line to solve a problem and it gets it wrong. I still have the burden of verifying, but it's a happy medium between your expertise and my writing everything in Python.
On 4, my favorite example is hearing "do I quote this?" when trying to explain something. It's such a simple question, but quotes don't delimit strings like they do in other programming languages, they do something else entirely, so getting into it really reveals they have almost no clue how the language works.
It does, but it's so darn weird. Can we have another flavour of Powershell but with some other language hooked up to its dotnet underpinnings? Like I dunno, I would even take Lua over the weird syntax of PS.
PowerShell is sort of two languages, and it looks weird because they are often used together. But if you see them separate first, there isn't much there that is surprising. I do somewhat share your concern about it being built on dotnet, but one advantage is you have access to everything in dotnet, including building GUI apps (Windows only). Though, at that point the syntax does get kinda weird.
Indeed, it's very weird. It really took much more time than I expect to learn Powershell, even with many years of coding experience. But I'm totally happy now.
I'm fluent in bash but I have to say it is the jankiest, messiest language I've ever used. I can't believe another default hasn't totally obliterated it into obsolescence. Maybe because it is so ubiquitous people are afraid of all the work required to move. Wait, that's me.
It's not hard once you know the rules. shellcheck helps, but it's usually whiny about escaping variable substitution that are intended for outside of the current script such as for editing other files.
Its biggest flawed design decision is word splitting by default and Bash's lack of a standard library helper functions for arrays and associative arrays. These could also be implemented as bash native extensions without build-time modification with a user-provided programmatic builtin. I could be wrong, but zsh may be able to add compiled functionality at runtime dynamically as well.
It's been useful to me for slapping together solutions during ops emergencies. Luckily, I have usually had real programmers around to take the few bits of insight in those scripts to fix their code.
Same. I have started to rewrite my largest scripts in Ruby tho. These are the kind of scripts where performance began to be too rough and issues to ignore.
Been using nushell for the past couple years as my main shell environment and for any random scripts I need to use. It's everything I want out of a shell scripting language. It has even replaced "upgrading" to python in a lot of instances.
Its strange people of every generation, almost everyday keeps discovering the need for Perl.
Perl was made to solve precisely these kind of problems.
Just use Perl. Its fast, matured, has CPAN and is dedicated to maintaining backwards compatibility for years to come. Using anything else will not make all that much sense. You will be reinventing a whole universe(and badly from scratch). And achieve nothing at the end.
If LLMs know Perl well, that might make me agree with you more actually. Then the LLM can be the ersatz greybeard refreshing you with what the script actually does.
1. Bash (or ysh, which I do prefer but isn't as available)
2. Python, using the `sh` package as needed. It's been great for upgrading shell scripts, and `sh` is really nice to use.
3. Rust, usually. That's my personal language of choice for "real" projects.
I have a number of projects at each stage actually, though most of the things I'd call a real "project" with it's own repo end up as Rust projects.