I've seen this sentiment a lot here. "Once shell is >n lines, port to python". My experience has been different. Maybe half of the scripts I write are better off in python while the other half are exponentially longer in python than bash.
For example, anything to do with json can be done in 1 line of readable jq, while it could be 1, 5, or 20 lines in python depending on the problem.
I'd just like to put that out there because half of the time, the >n metric does not work for me at all. My shell scripts range from ~5-150 lines while python are 100+
Same. It’s mostly because if I have a shell script, while I’ll add some comments for tricky bits if needed, and maybe a `-h` function, that’s about it. In Python, though, I use the language’s features to make it as readable and safe as possible. Types, docstrings, argparse, etc. My thinking is that if I’m going to take the time to use a “proper” language, then I should make it bulletproof, otherwise I’d just stick with shell.
My personal decision matrix for when to switch to Python usually involves the relative comfort of the rest of my team in both languages, the likelihood that future maintenance or development of the script will be necessary, and whether or not I’m dealing with inputs may change (e.g. API responses, since sadly versioning isn’t always a guarantee).
> For example, anything to do with json can be done in 1 line of readable jq, while it could be 1, 5, or 20 lines in python depending on the problem.
I don't agree that there exists such a thing as "readable jq" to start with. It's very arcane and difficult to follow unless you live and breathe the thing (which I don't). Furthermore, jq may or may not be present on the system, whereas the json package is always there in Python. Finally, I don't think having more lines is bad. The question is, what do you get for the extra lines? Python might have 5 lines where bash has 1, but those 5 lines will be far easier to read and understand in the future. That's a very worthwhile trade-off in my opinion.
> It's very arcane and difficult to follow unless you live and breathe the thing
I used to think this before I actually read how it worked. If you know shell, jq is extremely easy to pick up. It acts the exact same way, but pipes JSON entities instead of bytes ("text") like shell does.
Like the Unix philosophy, every filter does exactly one thing very well. Like shell, you write it incrementally, one filter at a time.
Genuinely, I do not blame you for thinking it's complex. I have never seen a concise, correct explanation of how jq works that builds an intuitive understanding. I have a near-complete one, and it's on my todo list to eventually publish it.
Anyway, I don't mean to say more lines is always worse, but that it is worse about half the time. Python is certainly more readable, but I'd rather spend 60 seconds making a long pipeline than 10 minutes making it work in python.
Want to count lines in a file? wc -l. Compress a directory? tar -zcf. Send that compressed file somewhere? Pipe it to ssh. Each of those is an ordeal in python and it's around 10 keystrokes in shell.
The only thing bash is better at than Python is very short scripts, like 10ish lines. Everything else it sucks at, due to the horrible syntax and various bash footguns.
