Zsh notes

From raju

Sample scripts

Check number of arguments

    % stuff check_arg_num.zsh
    #! /usr/bin/env zsh
    set -e
    set -u
    if [ "$#" -ne 1 ]; then
      echo "Usage: $0 foo" >&2
      exit 1
    fi
    foo=$1
    echo $foo
    

where stuff is a custom script similar to cat but removes empty lines and comments.

Sample run without an argument

    % zsh check_arg_num.zsh
    Usage: check_arg_num.zsh foo
    

Sample run with an argument

    % zsh check_arg_num.zsh something
    something
    

Ref:- http://stackoverflow.com/questions/4341630/checking-for-the-correct-number-of-arguments

pass through

Here is an example of a pass through shell script that calls a python program located in the same directory.

    % cat myapp
    #! /bin/zsh -f
    DIR="$(dirname "$(which "$0")")"
    $DIR/myapp.py "$@"
    

I use this approach a lot. The idea is that all the heavy lifting is done by a script written in an appropriate language and the shell script just calls that. This way if we decide to use a different language to implement the feature or call a different script altogether, the end user will not notice a change.

printf examples

Let's say you are doing some analysis that takes a long time to complete and produces an output file in the end. In that case, we do not want to accidentally overwrite the output file if the script is rerun. One way to ensure this is to check if the file exists at the very beginning of the script and exit if that is the case.

    ofile="analysis.out"
    if [ -f $ofile ]; then
      printf "Error: output file %s already exists.\n\
    Please remove it manually to rerun the analysis.\n" $ofile
      exit 1
    fi
    

Sample output looks like

    >./run_analysis.sh
    Error: output file analysis.out already exists.
    Please remove it manually to rerun the analysis.
    

tags | printf multiple lines, printf long strings

regex matching

% cat regex_match.sh
#! /bin/zsh

echo "foo123bar" | grep -qs "foo[0-9][0-9][0-9]bar"
if [ $? -eq 0 ]; then
   printf "Match found\n";
else
   printf "Match not found\n";
fi

echo "bar123foo" | grep -qs "foo[0-9][0-9][0-9]bar"
if [ $? -eq 0 ]; then
   printf "Match found\n";
else
   printf "Match not found\n";
fi

Sample usage

% ./regex_match.sh
Match found
Match not found

Ref:- http://stackoverflow.com/questions/8599628/shell-scripting-and-regular-expression

Using If else

Example 1

    % cat using_if_else.sh
    #! /usr/bin/env zsh
    
    foo="kamaraju"
    # foo="raju"
    if [[ "X$foo" == "Xraju" ]] then
        printf "if stanza: Hi raju\n";
    else
        printf "else stanza: Hi $foo\n";
    fi
    

Example 2

    % cat using_defs.sh
    #! /usr/bin/env zsh
    
    set -eu
    
    def_kama=$1
    def_raju=$2
    
    if (($def_kama)) then
        print "kama is defined"
    elif ((def_raju)) then
        print "raju is defined"
    else
        print "neither kama nor raju is defined"
    fi
    
    % ./using_defs.sh 1 0
    kama is defined
    
    % ./using_defs.sh 0 1
    raju is defined
    
    % ./using_defs.sh 0 0
    neither kama nor raju is defined
    
    % ./using_defs.sh
    ./using_defs.sh:5: 1: parameter not set
    


Example 3: Start iceweasel if it has not already started

    if pgrep -x iceweasel > /dev/null
    then
        echo "Not starting iceweasel as it is already running."
    else
        iceweasel &
    fi
    

Check if a directory exists

% cat dir_exists_01.sh
#! /bin/zsh

set -eu

check_if_dir_exists()
{
    dirname=$1;
    if [ ! -d "$dirname" ]
    then
        echo "$dirname does not exist"
    else
        echo "$dirname exists"
    fi
}

check_if_dir_exists "/bin"
check_if_dir_exists "/kamaraju"
check_if_dir_exists "kama raju"

Sample run

% dir_exists_01.sh
/bin exists
/kamaraju does not exist
kama raju does not exist

Check if an environment variable is defined

parametric study using zsh script

Create a file

% cat ./parametric_study.sh
#! /bin/zsh
month=(
        "January"
        "February"
        "March"
        "April" "May"
        "June"
        "July"
        "August"
        "September"
        "October"
        "November"
        "December"
        )
for i in $month
do
    echo "month = " $i
done

Make it executable

% chmod +x parametric_study.sh

Run the script

% ./parametric_study.sh
month =  January
month =  February
month =  March
month =  April
month =  May
month =  June
month =  July
month =  August
month =  September
month =  October
month =  November
month =  December

The script can also be run as

% zsh ./parametric_study.sh

Tested using zsh 4.3.10.

Check if two files are different

% cat are_files_different.sh
#! /bin/zsh
cmp -s file_1 file_2 > /dev/null
cret=$?
if [ $cret -eq 0 ]; then
    print "files are identical"
elif [ $cret -eq 1 ]; then
    print "files are different"
else
    print "An error occurred when comparing files."
fi

star vs at

% cat star_vs_at.sh
#! /bin/zsh

bar() { echo $1:$2; }

foo_at() { bar "$@";}
foo_at_no_quotes() { bar $@; }
foo_star() { bar "$*"; }
foo_star_no_quotes() { bar $*; }

printf "--------------------------------------------------------------------------------\n"
foo_at "This is" a test
foo_at_no_quotes "This is" a test
foo_star "This is" a test
foo_star_no_quotes "This is" a test
printf "--------------------------------------------------------------------------------\n"

printf "--------------------------------------------------------------------------------\n"
foo_at This is "a test"
foo_at_no_quotes This is "a test"
foo_star This is "a test"
foo_star_no_quotes This is "a test"
printf "--------------------------------------------------------------------------------\n"

printf "--------------------------------------------------------------------------------\n"
foo_at This "is a" test
foo_at_no_quotes This "is a" test
foo_star This "is a" test
foo_star_no_quotes This "is a" test
printf "--------------------------------------------------------------------------------\n"

% ./star_vs_at.sh
--------------------------------------------------------------------------------
This is:a
This is:a
This is a test:
This is:a
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
This:is
This:is
This is a test:
This:is
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
This:is a
This:is a
This is a test:
This:is a
--------------------------------------------------------------------------------

grep history

Add the following to your ~/.zshrc

history_grep() {
    if [ -z "$*" ]
    then
        history 1
    else
        history 1 | egrep "$@"
    fi
}
alias hg=history_grep

Sample usage

% hg "\<diff Portfolio.C\>"
% history_grep "\<diff Portfolio.C\>"

Related links:

  • original version is from https://nixtricks.wordpress.com/2009/12/03/zsh-print-the-full-command-line-history/

echo commands

Use "set -x" to echo commands while the script is executing. Use "set +x" to stop echoing.

% cat ./echo_commands.zsh
#! /usr/bin/env zsh

echo "before set -x"
set -x
echo "after set -x"
set +x
echo "after set +x"

Sample run

% ./echo_commands.zsh
before set -x
+./echo_commands.zsh:5> echo 'after set -x'
after set -x
+./echo_commands.zsh:6> set +x
after set +x

backup a directory

The idea here is to cd into a given directory and then backup a directory in it. The path names are hard coded to make things simple.

    #! /usr/bin/env zsh
    set -e
    topdir=/home/rajulocal/notes
    bdir=projects
    
    cd ${topdir}
    
    asof=`date +'%Y%m%d_%H%M%S'`
    ofile=${bdir}_${asof}.txz
    tar cJf ${ofile} ${bdir}
    if [ $? -eq 0 ]; then
        printf "backup stored in: ${topdir}/$ofile\n";
    fi
    

Sample run:

    % ~/bin/bkup_notes.sh
    backup stored in: /home/rajulocal/notes/projects_20160103_224746.txz
    

Merge daily csv files into a single csv file

code snippets

Remove file extension

The following code moves files of type junk*.txt from directory x to directory x2. It also changes the extension of files to ".dat". The ":t" gives the base file name, ":r" removes the extension.

% for in in x/junk*.txt
do
odir=x2
out=$odir/${in:t:r}.dat
mv -i $in $out
done
`x/junk1.txt' -> `x2/junk1.dat'
`x/junk2.txt' -> `x2/junk2.dat'
`x/junk3.txt' -> `x2/junk3.dat'
`x/junk4.txt' -> `x2/junk4.dat'
`x/junk5.txt' -> `x2/junk5.dat'

Ref:

Remove file extension using sed

    >echo "/path/to/foo.avro.gz" | sed -e 's/\.gz$//'
    /path/to/foo.avro
    

def_notnow

Keep the code in the script for reference but do not execute it.

def_notnow=0
if [ $def_notnow == 1 ]; then
    # commands that should not to be executed
fi
# commands that should be be executed

template to exit after an error

    #! /usr/bin/env zsh
    set -e
    

Trouble shooting

process newline character

unset the PROMPTCR option if you are having trouble processing newline character from the output of shell commands

%ls -1
kama
raju
%ls -1 | tr '\n' ' '
%setopt NO_PROMPTCR
%ls -1 | tr '\n' ' '
kama raju %

So, with the PROMPTCR option set, the lines without newline character are overridden by the prompt. But when it is unset, those lines will be shown.

zshrc related

Search command line history using glob pattern matching

source one alias

Two possible approaches

    1. eval `grep "alias foo=" ~/.zshrc`
    2. grep "alias foo=" ~/.zshrc | source /dev/stdin

useful zsh functions

Get the nth last command

Sort the output of hdfs dfs -ls by time

Add the following lines to ~/.zshrc

    # Use this function to sort the output of hdfs dfs -ls by time.
    # hls "/path/to/hdfs/dirctory"
    hls() {
        hdfs dfs -ls "$1" | sort -k6,7
    }
    

To be sorted

Simple regression testing using zsh script

change the default shell

use chsh. For example, to set the default shell to zsh.

rajulocal@hogwarts ~ % chsh -s /usr/bin/zsh 
Password:

set up an alias for ls

show colors. Show timestamps in a machine readable format.

alias ls="ls --color=auto --time-style=\"+%Y%m%d_%H%M\""

% ls -al ~/.zshrc
-rw-r--r-- 1 rajulocal users 1048 20150118_1141 /home/rajulocal/.zshrc

useful external links

Multiline editing

  • Use Escape-Return to enter a new line while editing a multiline command

Ref:- page 99 in "From Bash to Z Shell: Conquering the Command Line" By Oliver Kiddle, Peter Stephenson, Jerry Peek

Multiline statements

    #! /bin/env zsh
    for fname in *.csv.gz
    do
        # I am not showing the logic to construct $ofile here. Fix this later.
        (echo "COL1";
         gunzip -c $fname | cut -f1 -d',' | tail -n +2) > $ofile
    

Which shell am I using

difference between set -e and set -u

If you are using a variable before setting its value, then set -u will throw a message such as

    file_name:line_num: var_name: parameter not set
    

and he program will continue.

If you use "set -e", the program will exit after the first error.

So, if you are using both (ex:- "set -ue") then program will immediately exit after the above message.

External links

My zshrc file