Zsh notes

From raju

Sample scripts

Check number of arguments

% stuff check_arg_num.zsh
#! /usr/bin/env zsh
set -e
set -u
if [ "$#" -ne 1 ]; then
  echo "Usage: $0 foo" >&2
  exit 1
fi
foo=$1
echo $foo

where stuff is a custom script similar to cat but removes empty lines and comments.

Sample run without an argument

% zsh check_arg_num.zsh
Usage: check_arg_num.zsh foo

Sample run with an argument

% zsh check_arg_num.zsh something
something

Ref:- http://stackoverflow.com/questions/4341630/checking-for-the-correct-number-of-arguments

pass through

Here is an example of a pass through shell script that calls a python program located in the same directory.

% cat myapp
#! /bin/zsh -f
DIR="$(dirname "$(which "$0")")"
$DIR/myapp.py "$@"

I use this approach a lot. The idea is that all the heavy lifting is done by a script written in an appropriate language and the shell script just calls that. This way if we decide to use a different language to implement the feature or call a different script altogether, the end user will not notice a change.

printf examples

Let's say you are doing some analysis that takes a long time to complete and produces an output file in the end. In that case, we do not want to accidentally overwrite the output file if the script is rerun. One way to ensure this is to check if the file exists at the very beginning of the script and exit if that is the case.

ofile="analysis.out"
if [ -f $ofile ]; then
  printf "Error: output file %s already exists.\n\
Please remove it manually to rerun the analysis.\n" $ofile
  exit 1
fi

Sample output looks like

>./run_analysis.sh
Error: output file analysis.out already exists.
Please remove it manually to rerun the analysis.

tags | printf multiple lines, printf long strings

regex matching

% cat regex_match.sh
#! /bin/zsh

echo "foo123bar" | grep -qs "foo[0-9][0-9][0-9]bar"
if [ $? -eq 0 ]; then
   printf "Match found\n";
else
   printf "Match not found\n";
fi

echo "bar123foo" | grep -qs "foo[0-9][0-9][0-9]bar"
if [ $? -eq 0 ]; then
   printf "Match found\n";
else
   printf "Match not found\n";
fi

Sample usage

% ./regex_match.sh
Match found
Match not found

Ref:- http://stackoverflow.com/questions/8599628/shell-scripting-and-regular-expression

Using If else

Example 1

% cat using_if_else.sh
#! /usr/bin/env zsh

foo="kamaraju"
# foo="raju"
if [[ "X$foo" == "Xraju" ]] then
    printf "if stanza: Hi raju\n";
else
    printf "else stanza: Hi $foo\n";
fi

Example 2

% cat using_defs.sh
#! /usr/bin/env zsh

set -eu

def_kama=$1
def_raju=$2

if (($def_kama)) then
    print "kama is defined"
elif ((def_raju)) then
    print "raju is defined"
else
    print "neither kama nor raju is defined"
fi

% ./using_defs.sh 1 0
kama is defined

% ./using_defs.sh 0 1
raju is defined

% ./using_defs.sh 0 0
neither kama nor raju is defined

% ./using_defs.sh
./using_defs.sh:5: 1: parameter not set


Example 3: Start iceweasel if it has not already started

if pgrep -x iceweasel > /dev/null
then
    echo "Not starting iceweasel as it is already running."
else
    iceweasel &
fi

Check if a directory exists

% cat dir_exists_01.sh
#! /bin/zsh

set -eu

check_if_dir_exists()
{
    dirname=$1;
    if [ ! -d "$dirname" ]
    then
        echo "$dirname does not exist"
    else
        echo "$dirname exists"
    fi
}

check_if_dir_exists "/bin"
check_if_dir_exists "/kamaraju"
check_if_dir_exists "kama raju"

Sample run

% dir_exists_01.sh
/bin exists
/kamaraju does not exist
kama raju does not exist

Check if an environment variable is defined

parametric study using zsh script

Create a file

% cat ./parametric_study.sh
#! /bin/zsh
month=(
        "January"
        "February"
        "March"
        "April" "May"
        "June"
        "July"
        "August"
        "September"
        "October"
        "November"
        "December"
        )
for i in $month
do
    echo "month = " $i
done

Make it executable

% chmod +x parametric_study.sh

Run the script

% ./parametric_study.sh
month =  January
month =  February
month =  March
month =  April
month =  May
month =  June
month =  July
month =  August
month =  September
month =  October
month =  November
month =  December

The script can also be run as

% zsh ./parametric_study.sh

Tested using zsh 4.3.10.

Check if two files are different

% cat are_files_different.sh
#! /bin/zsh
cmp -s file_1 file_2 > /dev/null
cret=$?
if [ $cret -eq 0 ]; then
    print "files are identical"
elif [ $cret -eq 1 ]; then
    print "files are different"
else
    print "An error occurred when comparing files."
fi

star vs at

% cat star_vs_at.sh
#! /bin/zsh

bar() { echo $1:$2; }

foo_at() { bar "$@";}
foo_at_no_quotes() { bar $@; }
foo_star() { bar "$*"; }
foo_star_no_quotes() { bar $*; }

printf "--------------------------------------------------------------------------------\n"
foo_at "This is" a test
foo_at_no_quotes "This is" a test
foo_star "This is" a test
foo_star_no_quotes "This is" a test
printf "--------------------------------------------------------------------------------\n"

printf "--------------------------------------------------------------------------------\n"
foo_at This is "a test"
foo_at_no_quotes This is "a test"
foo_star This is "a test"
foo_star_no_quotes This is "a test"
printf "--------------------------------------------------------------------------------\n"

printf "--------------------------------------------------------------------------------\n"
foo_at This "is a" test
foo_at_no_quotes This "is a" test
foo_star This "is a" test
foo_star_no_quotes This "is a" test
printf "--------------------------------------------------------------------------------\n"

% ./star_vs_at.sh
--------------------------------------------------------------------------------
This is:a
This is:a
This is a test:
This is:a
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
This:is
This:is
This is a test:
This:is
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
This:is a
This:is a
This is a test:
This:is a
--------------------------------------------------------------------------------

grep history

Add the following to your ~/.zshrc

history_grep() {
    if [ -z "$*" ]
    then
        history 1
    else
        history 1 | egrep "$@"
    fi
}
alias hg=history_grep

Sample usage

% hg "\<diff Portfolio.C\>"
% history_grep "\<diff Portfolio.C\>"

Related links:

  • original version is from https://nixtricks.wordpress.com/2009/12/03/zsh-print-the-full-command-line-history/

echo commands

Use "set -x" to echo commands while the script is executing. Use "set +x" to stop echoing.

% cat ./echo_commands.zsh
#! /usr/bin/env zsh

echo "before set -x"
set -x
echo "after set -x"
set +x
echo "after set +x"

Sample run

% ./echo_commands.zsh
before set -x
+./echo_commands.zsh:5> echo 'after set -x'
after set -x
+./echo_commands.zsh:6> set +x
after set +x

backup a directory

The idea here is to cd into a given directory and then backup a directory in it. The path names are hard coded to make things simple.

#! /usr/bin/env zsh
set -e
topdir=/home/rajulocal/notes
bdir=projects

cd ${topdir}

asof=`date +'%Y%m%d_%H%M%S'`
ofile=${bdir}_${asof}.txz
tar cJf ${ofile} ${bdir}
if [ $? -eq 0 ]; then
    printf "backup stored in: ${topdir}/$ofile\n";
fi

Sample run:

% ~/bin/bkup_notes.sh
backup stored in: /home/rajulocal/notes/projects_20160103_224746.txz

Merge daily csv files into a single csv file

code snippets

Remove file extension

The following code moves files of type junk*.txt from directory x to directory x2. It also changes the extension of files to ".dat". The ":t" gives the base file name, ":r" removes the extension.

% for in in x/junk*.txt
do
odir=x2
out=$odir/${in:t:r}.dat
mv -i $in $out
done
`x/junk1.txt' -> `x2/junk1.dat'
`x/junk2.txt' -> `x2/junk2.dat'
`x/junk3.txt' -> `x2/junk3.dat'
`x/junk4.txt' -> `x2/junk4.dat'
`x/junk5.txt' -> `x2/junk5.dat'

Ref:

Remove file extension using sed

>echo "/path/to/foo.avro.gz" | sed -e 's/\.gz$//'
/path/to/foo.avro

def_notnow

Keep the code in the script for reference but do not execute it.

def_notnow=0
if [ $def_notnow == 1 ]; then
    # commands that should not to be executed
fi
# commands that should be be executed

template to exit after an error

#! /usr/bin/env zsh
set -e

Trouble shooting

process newline character

unset the PROMPTCR option if you are having trouble processing newline character from the output of shell commands

%ls -1
kama
raju
%ls -1 | tr '\n' ' '
%setopt NO_PROMPTCR
%ls -1 | tr '\n' ' '
kama raju %

So, with the PROMPTCR option set, the lines without newline character are overridden by the prompt. But when it is unset, those lines will be shown.

zshrc related

Search command line history using glob pattern matching

tags | reverse search

source one alias

Two possible approaches

  1. eval `grep "alias foo=" ~/.zshrc`
    
  2. grep "alias foo=" ~/.zshrc | source /dev/stdin
    

useful zsh functions

Get the nth last command

Sort the output of hdfs dfs -ls by time

Add the following lines to ~/.zshrc

# Use this function to sort the output of hdfs dfs -ls by time.
# hls "/path/to/hdfs/dirctory"
hls() {
    hdfs dfs -ls "$1" | sort -k6,7
}

To be sorted

Simple regression testing using zsh script

Change the default shell

Experiment with different shells

set up an alias for ls

show colors. Show timestamps in a machine readable format.

alias ls="ls --color=auto --time-style=\"+%Y%m%d_%H%M\""

% ls -al ~/.zshrc
-rw-r--r-- 1 rajulocal users 1048 20150118_1141 /home/rajulocal/.zshrc

useful external links

Multiline editing

  • Use Escape-Return to enter a new line while editing a multiline command

Ref:- page 99 in "From Bash to Z Shell: Conquering the Command Line" By Oliver Kiddle, Peter Stephenson, Jerry Peek

Multiline statements

#! /bin/env zsh
for fname in *.csv.gz
do
    # I am not showing the logic to construct $ofile here. Fix this later.
    (echo "COL1";
     gunzip -c $fname | cut -f1 -d',' | tail -n +2) > $ofile

Which shell am I using

difference between set -e and set -u

If you are using a variable before setting its value, then set -u will throw a message such as

file_name:line_num: var_name: parameter not set

and he program will continue.

If you use "set -e", the program will exit after the first error.

So, if you are using both (ex:- "set -ue") then program will immediately exit after the above message.

External links

My zshrc file