Simple regression testing using zsh script
From raju
Task
I want to add new functionality to an existing script without introducing any "regressions", i.e. without breaking any existing functionality.
Solution
Here is a simple way to do it by using a zsh script.
% cat regr_test.zsh #! /bin/zsh # set -x set -u run() { local script=$1 local file=$2 local prefix=$3 local odir=$4 local id=$5 $script $file $prefix > $odir/${id}.txt } # using relative path names. But can use absolute path names if necessary. script_new=get_name.pl script_old=get_name_old.pl printf "performing regression test for\n" printf " $script_new vs. $script_old\n" # using relative path names. But can use absolute path names if necessary. odir_new=1 odir_old=2 mkdir -p $odir_new $odir_old printf "output is stored in $odir_new , $odir_old\n" test_arg=( "/usr/bin/ls" "$HOME/.vimrc" ) test_name=( "abs_path" "dot_file" ) prefix="RK_" ntest=${#test_arg}; for i in `seq 1 $ntest` do run $script_new ${test_arg[$i]} $prefix $odir_new ${test_name[$i]} run $script_old ${test_arg[$i]} $prefix $odir_old ${test_name[$i]} done diff -rq $odir_new $odir_old > /dev/null dret=$? if [ $dret -eq 0 ]; then printf "\nSuccess! No differences are found between legs $odir_new and $odir_old\n" else printf "\nFailed! Differences found between legs $odir_new and $odir_old\n" diff -rq $odir_new $odir_old fi
where get_name.pl and get_name_old.pl are new and old versions of a script that tries to add a prefix to a given file name.
% cat get_name.pl #! /usr/bin/perl -w use strict; use warnings; use File::Spec; # for rel2abs use File::Basename qw(dirname); my $file_arg=$ARGV[0]; my $prefix=$ARGV[1]; my $file_abs = File::Spec->rel2abs( $file_arg ); (my $base_name = $file_abs) =~ s!.*/!!; my $dir = dirname($file_abs); my $new_file = "$dir/$base_name$prefix"; print "$new_file\n";
% cat get_name_old.pl #! /usr/bin/perl -w use strict; use warnings; use File::Spec; # for rel2abs use File::Basename qw(dirname); my $file_arg=$ARGV[0]; my $prefix=$ARGV[1]; my $file_abs = File::Spec->rel2abs( $file_arg ); (my $base_name = $file_abs) =~ s!.*/!!; my $dir = dirname($file_abs); my $new_file = "$dir/$prefix$base_name"; print "$new_file\n";
Here, I purposefully introduced a bug in the new version of the script which appends the prefix instead of prepending.
% diff get_name.pl get_name_old.pl 15c15 < my $new_file = "$dir/$base_name$prefix"; --- > my $new_file = "$dir/$prefix$base_name";
The idea is to make sure that the regression test catches it.
First make all the scripts executable
% chmod +x get_name.pl get_name_old.pl regr_test.zsh
Run the regression test
% ./regr_test.zsh performing regression test for get_name.pl vs. get_name_old.pl output is stored in 1 , 2 Failed! Differences found between legs 1 and 2 Files 1/abs_path.txt and 2/abs_path.txt differ Files 1/dot_file.txt and 2/dot_file.txt differ
Here is the directory structure once the script is run.
% tree . |-- 1 | |-- abs_path.txt | `-- dot_file.txt |-- 2 | |-- abs_path.txt | `-- dot_file.txt |-- get_name_old.pl |-- get_name.pl `-- regr_test.zsh 2 directories, 7 files
Looking at the output files, it becomes obvious as to what went wrong
% head [12]/* ==> 1/abs_path.txt <== /usr/bin/lsRK_ ==> 1/dot_file.txt <== /home/rajulocal/.vimrcRK_ ==> 2/abs_path.txt <== /usr/bin/RK_ls ==> 2/dot_file.txt <== /home/rajulocal/RK_.vimrc
Once the bug is corrected, a successful run will look like
% ./regr_test.zsh performing regression test for get_name.pl vs. get_name_old.pl output is stored in 1 , 2 Success! No differences are found between legs 1 and 2
Side Effects
This page also demonstrates how to:
- pass arguments to functions in zsh shell scripts
Related links
- The File::Compare module is useful to compare files in perl.