Bash notes
From raju
Note:- This page will be migrated to http://www.kamaraju.xyz/dk/bash_notes. Once the migration is done, it will be deleted.
Scripts
Check for exact number of arguments
$cat check_arg_num.sh #! /usr/bin/env bash set -e set -u if [ "$#" -ne 1 ]; then echo "Usage: $0 foo" >&2 exit 1 fi foo=$1 echo $foo
Sample run without an argument
$./check_arg_num.sh Usage: ./check_arg_num.sh foo
Sample run with an argument
$./check_arg_num.sh something something
Snippet to echo multiline message:
if [ "$#" -ne 1 ]; then echo " Usage: $0 foo_date where foo_date is of the form YYYYMMDD" >&2 exit 1 fi foo_date="$1"
check that number of arguments are in a range
$ cat check_arg_range.sh #! /usr/bin/env bash set -e set -u if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then echo "Usage: $0 foo <bar>" >&2 echo "Number of arguments should be between 1 and 2" exit 1 fi foo=$1 bar=${2:-default} echo $foo echo $bar $ ./check_arg_range.sh Usage: ./check_arg_range.sh foo <bar> Number of arguments should be between 1 and 2 $ ./check_arg_range.sh valA valA default $ ./check_arg_range.sh valA valB valA valB $ ./check_arg_range.sh valA valB valC Usage: ./check_arg_range.sh foo <bar> Number of arguments should be between 1 and 2
file extension
To get the extension
echo ${filename##*.}
To get the last three characters in a filename
echo ${filename: -3}
Sample use case:
$ filename="d u m m y.b l a h" $ echo ${filename##*.} b l a h $ echo ${filename: -3} a h
characteristics of the input: Contains spaces in both file name and in extension, extension is not three characters.
full name, file name, file head, extension
for fullname in /path/to/*.csv do fname="${fullname##*/}" fhead="${fname%.*}" ext="${fname##*.}" echo $fullname $fname $fhead $ext done
Ref:- https://stackoverflow.com/questions/965053/extract-filename-and-extension-in-bash
replace string
$a="space_kama_raju_kama_raju_kama_raju" $echo $a space_kama_raju_kama_raju_kama_raju $echo ${a/kama/foo} space_foo_raju_kama_raju_kama_raju $echo ${a//kama/foo} space_foo_raju_foo_raju_foo_raju
looping on the command line
for i in foo.csv bar.csv; do echo "==>" $i "<=="; cat $i; echo ; done
default arguments
The idea here is to have some default values assigned to a variable if nothing is specified on the command line. If some values are specified on the command line, they will be used instead.
$cat default_args.sh #! /usr/bin/env bash if [ "$#" -eq 0 ]; then dirs=( "path1" "path2" ) else dirs="$@" fi for d in ${dirs[@]} do echo $d done $./default_args.sh path1 path2 $./default_args.sh foo bar foo bar
Used in git_status.sh to get the list of directories on which we want to run "git status" type of commands.
Demonstrates | check number of arguments in a bash script
code snippets
check if a variable is empty
if [ -z "${BASE}" ]; then # do something fi
check for two conditions inside if
This checks if BASE variable is empty or if LOCAL and BASE are the same.
if [ -z $BASE ] || [ $LOCAL = $BASE ]; then # do something fi
Check if a directory exists
if [ -d "$directory" ]; then echo "$directory exists" else echo "$directory does not exist" fi
To check if a directory does not exist, use
if [ ! -d "$directory" ]; then echo "$directory does not exist" fi
compare numbers
Sample code
# --autostash is available only if git version is >= 2.6 git_version=`git --version | cut -f3 -d' '` major_version=`echo $git_version | cut -f1 -d.` minor_version=`echo $git_version | cut -f2 -d.` if (( $major_version >= 2 )) && (( $minor_version >= 6 )); then use_autostash=1 else use_autostash=0 fi if [ $use_autostash == 1 ]; then git pull --rebase --autostash -v origin else git stash git pull --rebase -v origin git stash pop fi
To compare floating point numbers
$ [[ 2.5 > 2.10 ]] && echo 1 || echo 0 1 $ [[ 2.5 < 2.10 ]] && echo 1 || echo 0 0
It also works for comparing integers.
$ [[ 3 > 2 ]] && echo 1 || echo 0 1 $ [[ 3 < 2 ]] && echo 1 || echo 0 0
-gt, -lt etc., will not work for floating point numbers
$ [[ 2.5 -gt 2.10 ]] && echo 1 || echo 0 bash: [[: 2.5: syntax error: invalid arithmetic operator (error token is ".5") 0 $ [[ 2.5 -lt 2.10 ]] && echo 1 || echo 0 bash: [[: 2.5: syntax error: invalid arithmetic operator (error token is ".5") 0
But they work for integers
$ [[ 3 -gt 2 ]] && echo 1 || echo 0 1 $ [[ 3 -lt 2 ]] && echo 1 || echo 0 0
tested with
$ bash --version GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu) $ cat /etc/debian_version 9.4
cd into script directory
cd `dirname "$0"`
which can also be written as
cd "$(dirname "$0")"
source a script from current script's directory
source "$(dirname "$0")"/foo
join two strings to get a path
full_path="$part1/$part2"
check status of previous command
Sample use case
git rev-parse @ > /dev/null 2>&1 if [[ $? == 128 ]]; then echo "Not a git repository. Exiting." exit 1 fi
remove characters from the beginning and end
tags | remove first N characters, remove last N characters
To remove N characters from the beginning and M characters from the end of a string
${i:N:-M}
Example:
Prepare some data
mkdir -p ~/x cd ~/x mkdir -p a/.git b/.git c/d/.git
$ for i in $(find . -name .git); do > echo $i > done ./a/.git ./b/.git ./c/d/.git
To remove "./" from the beginning and "/.git" from the end
$ for i in $(find . -name .git); do > echo ${i:2:-5} > done a b c/d
using while loop to process the output of find
find blah blah | while read i; do echo $i done
is equivalent to
for i in $(find blah blah); do echo $i done
For example, to get all the git repos under a directory, we can either do
find . -name .git -type d -prune | while read d; do echo $d done
or
for d in $(find . -name .git -type d -prune); do echo $d done
output the current directory name in green
echo "$(tput setaf 2)$PWD$(tput sgr0)"
or more generically
green=$(tput setaf 2) reset=$(tput sgr0) echo "$red$PWD$reset"
Ref:- https://mywiki.wooledge.org/BashFAQ/037
echo "$@" echo "${@:2}" cd "$i"
parent directory
dir=/foo/bar parentdir="$(dirname "$dir")"
increment number in for loop
code snippet
i=0 for hash in $hashes do git checkout $hash -- $file mv $file m${i}_$file ((i++)) done
Ref:- https://stackoverflow.com/questions/20681210/incrementing-a-variable-inside-a-bash-loop
where I needed it:- https://github.com/KamarajuKusumanchi/rutils/blob/master/bin/xtract_hist_git
Scripting
Difference between $* and $@
formatting if and for statements
for f in *; do if [ -d "$f" ]; then # $f is a directory fi done
To do it on one line
for f in *; do if [ -d "$f" ]; then echo $f; fi done
Ref:- https://unix.stackexchange.com/a/86727/198064
sort after
ISO-8601 compliant dates
$ date +%Y-%m-%d 2017-10-30
$ date +%Y-%m-%dT%H:%M:%S%z 2017-10-30T17:44:13-0400
You can also achieve this by -I
$ date -I 2017-10-30 $ date -Iminutes 2017-10-30T17:47-04:00 $ date -Iseconds 2017-10-30T17:47:39-04:00
Equivalence between -Iseconds and -u
$date +%FT%T%z 2017-10-30T17:53:06-0400 $date -u +%FT%TZ 2017-10-30T21:53:06Z
Basically UTC time + offset printed by %z is equal to the local time.
Ref:-
- https://stackoverflow.com/questions/7216358/date-command-on-os-x-doesnt-have-iso-8601-i-option
- https://en.wikipedia.org/wiki/ISO_8601
- http://svnbook.red-bean.com/en/1.7/svn.tour.revs.specifiers.html#svn.tour.revs.dates
redirect both stdout and stderr to a file
foo.sh > out.txt 2>&1
To append both stdout and stderr to a file
foo.sh >> out.txt 2>&1
controlling history
Add the following to ~/.bashrc
# This is useful if you run more than one bash instance in parallel. Without # this option, the history file only contains the contents of the last exiting # shell. For more details, see # https://unix.stackexchange.com/questions/6501/why-would-anyone-not-set-histappend-in-bash shopt -s histappend # prevent duplicate entries of a single session from being saved to $HISTFILE. export HISTCONTROL=ignoreboth:erasedups export HISTSIZE=20000 # Up arrow will show commands from other shells as well export PROMPT_COMMAND="history -a;history -c;history -r;$PROMPT_COMMAND" # When a command is duplicated, store only the latest. # Adapted from # https://unix.stackexchange.com/questions/48713/how-can-i-remove-duplicates-in-my-bash-history-preserving-order tac $HISTFILE | awk '!x[$0]++' | tac > /tmp/tmpfile && "mv" -fv /tmp/tmpfile $HISTFILE wc -l $HISTFILE
What they do
* ignoreboth - does both ignorespace and ignoredups * erasedups - * ignorespace - ignore commands that start with spaces * ignoredups - ignore duplicates
* history -a # append history lines from this session to the history file. History file may contain history from other terminals not in this one so: * history -c # clear [in-memory] history list deleting all of the entries. * history -r # read the history file and append the contents to the history list instead.
redirect output and errors to /dev/null
Append this to the command
> /dev/null 2>&1
PS1
Some PS1 entries I like
# host@user:pwd # Use $PWD instead of \w so that the home directory is not abbreviated as ~ PS1="\h@\u:$PWD $"
Missing Features
- Currently, it is not possible to list files by modification time using tab completion. Use zsh instead.
Ref:- http://lists.gnu.org/archive/html/bug-bash/2014-12/msg00161.html
http://www.zsh.org/mla/users/2015/msg00078.html