Shell Proficiency

Written by Michael Anckaert
Published on
Posted in Commandline

In this article I'll give some useful tips on how to be more proficient with the shell. I've noticed that a lot of developers tend to gravitate towards using the commandline but don't make use of the full power of it. This articles will collect some useful tips, best practices and cool shortcuts to improve how you work on the commandline.

Basic use

I'm not going to waste any time explaining the basics of the shell / commandline to you here. The internet is filled with tutorials and screencasts of how to start using the commandline.

Some quick tips to make your life easier:

Commandline history

Every (modern) commandline/shell has a history. This means that you can press the up/down arrows to move over your history, this is commandline usage 101.

But an interesting feature is the keyboard shortcut CTRL-R, which allows you to search your shell history. You just press CTRL-R, and then type your search term. This can be anything you previously entered on the commandline (program name, filename, parameters) and your shell will show you the last matching command. Press CTRL-R again to go the next matching result. Press enter to execute the command.

Editing commands

Typing long commands / incantations on the commandline can be bit daunting. When you find yourself editing a lot of commands, it can be useful to edit the entire command in an editor. To do this, press CTRL-X CTRL-E while editing a command. Your favourite $EDITOR will open and you can edit the command with all the powers your editor gives you. Save and quit and you simply press enter to run the command you so carefully crafted.

Powerful utilities

A Linux based system has a number of powerful utilities such as grep and awk that can really make you do cool stuff. The great thing is that you link these small programs together to become much more awesome and powerful, a bit like the Power Rangers. You know, then all rangers grouped together in that huge big robot?

I won't start writing a large tutorial about grep or awk in this article. But here's a case I just encountered today.

I had a file containing a list of URL's that needed to be checked for the HTTP status code (find broking links to put it simply). So I had one file that contained about 1200 links and I had to get the HTTP status code for each link and output it to another file if the return code was 404. Sounds like a pretty hard problem but with the commandline it's actually pretty simple.

First I had to do was get the HTTP status code for a given URL. Luckily we have the curl utility that can output the HTTP header for a given url. If you run curl using the -I option, you're getting the HTTP headers. Let's also suppress any other output using the -s option:

$ curl -I -s $URL

HTTP/2 200 
content-type: text/html
content-length: 2507
date: Thu, 18 Feb 2021 14:28:32 GMT
last-modified: Wed, 03 Feb 2021 12:58:41 GMT
etag: "796d0e61ecc8e9b04c4af4718ce51fc1"
accept-ranges: bytes
server: AmazonS3
x-cache: Hit from cloudfront
via: 1.1 9099794184e0cace0bb57c49a112df52.cloudfront.net (CloudFront)
x-amz-cf-pop: BRU50-C1
x-amz-cf-id: 9yx88VHiuGZn-YDPsnKcckJOwA8_mtb28PUJG0rE4qY_Xx56fRXooA==
age: 191

Ok, looks good. But we only need the first line of output, which contains the HTTP status code. Let's pipe the output to the head program:

$ curl -I -s $URL | head -n 1

HTTP/2 200 

Wonderful! Now we need the '200' value from the output, as that is the HTTP status code. For this we can use the awk program and have it output the second column:

$ curl -I -s $URL | head -n 1 | awk '{print $2}'

200

We're almost there! Now we only need to loop over our entire file and output the status code for all the links in the file:

$ for URL in `cat list.txt`; do 
for> curl -I -s $URL | head -n 1 | awk '{print $2}'
for> done
200
200
301
200
404
200
200

If you extend your small shell script a bit further you'll have an automated link checker in no time!

Stepping up your game: ZSH

I know, picking your favourite shell is personal and there actually isn't anything objectively better about ZSH than for example Bash. And if there is, I'm sure someone will email me about it ;-) One of the reasons I like ZSH more than Bash is that there is (currently) a larger ecosystem of plugins, articles, themes and development around it than there is for Bash.

That being said: you can seriously enhance your productivity a bit if you switch to ZSH and use the Oh My Zsh framework. Oh My Zsh is a framework for ZSH that contains a huge number of plugins, functions and themes to make your commandline life a lot better.

Here are some of my favourites!

Themes to pimp your shell

I'm not one to fiddle with fancy themes a lot on my shells, but I do like a shell that is informative. The screenshot below shows my current shell theme and I like how it provides me with extra context when working in the shell. Just from a quick glance I can view the date & time of when I ran the last command, the current host I'm connected to, my working directory and what Git branch I'm on.

Shell Theme

Git plugin

Using the Git Plugin that is included in Oh My Zsh gives you a lot of small goodies that make working with Git just a bit... better?

The basics are the shortcuts that are included in the plugin. Typing gst instead of git status or gcm to quickly checkout the master branch all quickly become part of your muscle memory. Besides the basic git plugin, Oh My Zsh has a number of

More plugins

Be sure to check out the full plugin list to see all thta is offered by Oh My Zsh.

Extra tools

To really get the most of your commandline usage, I also recommend taking a look at: