Find usage

From raju

search all python files in a directory for a string

    find . -iname '*.py' -exec grep foo -inH --color {} +
    

Find recent files

    find dir -type f -mtime -1 -ls
    find . -iname '*.sw*' -atime -10 -type f
    

Do not show the starting directory in the output

By default, find shows the starting directory name in the output. For example

    $find x -iname '*elastic*'
    x/elastic_00.txt
    x/x1/elastic_01.txt
    x/x1/x2/elastic_02.txt
    

To remove it, use -printf '%P\n'

    $find x -iname '*elastic*' -printf '%P\n'
    elastic_00.txt
    x1/elastic_01.txt
    x1/x2/elastic_02.txt
    

To display only the file names without printing the subdirectories, use the '%f' modifier

    $find x -iname '*elastic*' -printf '%f\n'
    elastic_00.txt
    elastic_01.txt
    elastic_02.txt
    

Tested on GNU findutils 4.6.0

copy all but one file

To deploy all shell scripts and python scripts except the deploying script itself

    DEPLOY_DIR=$HOME/bin
    find . \( -iname '*.sh' -o -iname '*.py' \) \
        -not -name deploy.sh \
        -exec cp -vt $DEPLOY_DIR {} +
    

You can also use ! instead of -not

    DEPLOY_DIR=$HOME/bin
    find . \( -iname '*.sh' -o -iname '*.py' \) ! -name deploy.sh -exec cp -vt $DEPLOY_DIR {} +
    

Ref:- https://stackoverflow.com/questions/1313590/bash-copy-all-files-except-one

Find files containing two expressions

Search the C++ source code and header files to find files that have two keywords - ifstream and string

    find /path/to/dir \( -iname '*.C' -o -iname '*.H' \) -print0  | xargs -0 grep -lZ ifstream  | xargs -0 grep -l string
    

All intermediate grep statements should use the -Z option so that the output from grep can be passed to "xargs -0".

list files with two different extensions

To search for both .jpg and .png files

    find /path/to/dir \( -iname '*.jpg' -o -iname '*.png' \)
    

The expression within \( ... \) should be delimited by spaces on both sides.

To do the same in two directories

    find /path/to/dir1 /path/to/dir2 \( -iname '*.jpg' -o -iname '*.png' \)
    

list all files in a directory that do not end with a particular extension

=> find . -maxdepth 1 -not -iname "*.H"

find all git folders under a directory

find . -name .git -type d -prune

Ref:- http://stackoverflow.com/a/11998951/6305733

list a directory but exclude directories and vim swap files

find . -maxdepth 1 ! -type d ! -name "*.sw?"

grep files in a directory but exclude subdirectories, files matching a pattern

find . -maxdepth 1 ! -type d ! -name "*.sw?" ! -name "*_RK_????????_??????" ! -name "*_blame_????????_??????" -exec grep -inH --color key_word {} +

Did I ever call awk from a perl script?

rajulocal@hogwarts ~/work/rutils
 % find . -iname '*.pl' -exec grep awk -inH --color {} +
./perl/app_status_by_keyword.pl:48:    my $cmd = "dpkg -l \*$keyword\* | grep ^ii | awk '{print \$2}'";

List all the R programs I accessed in the last 60 days

% find . -iname '*.R' -atime -60 -type f

Find all the log files and delete them

First make sure that you are matching on the correct set of files

find . -iname '*.log'

Then use any of the following commands

find . -iname '*.log' -exec rm {} +
find . -iname '*.log' -print0 | xargs -0 rm
find . -iname '*.log' -delete

To delete interactively

find . -iname '*.log' -exec rm -i {} +

The exact command depends on the installation since some versions of find do not have -print0, -delete etc., In any case, try to avoid

find . -iname '*.log' -exec rm {} \;

which spawns an external process for each and every matched file.

The advantage of using "-exec rm" approach is that you can pass additional options to the rm command. For example

find . -iname '*.log' -exec rm -i {} \;

will delete the files interactively.

tags | remove files

delete output directories older than a month

    OUTPUT_ROOT=/foo/bar
    echo "Deleting directories older than a month in $OUTPUT_ROOT"
    
    # Check for directory existence before trying to remove the files.
    # Otherwise, in the offchance that the variable is empty, find will search the
    # current directory and will try to remove files there.
    if [ -d "$OUTPUT_ROOT" ]; then
        find "$OUTPUT_ROOT" -maxdepth 1 -type d -ctime +30 -exec rm -rfv {} +
    fi
    

Give write permission to the user for all top level directories under a directory

    find . -maxdepth 1 -type d -exec chmod u+w {} +
    

Aggregate data across all files modified in the last 10 minutes

 find . -mmin -10 -type f -print0 | xargs -0 tail -n +2 -q | sort | uniq

Find old files and move them to a directory

First make sure that you are matching on the correct set of files

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6

Then append it with the mv command as below

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6 -exec mv -ivt path/to/dest/dir {} +

Find files modified after a certain date

The code snippet below will find files modified on or after a certain date and copies them from an nfs directory to hdfs.

    doi="2017-04-24"
    # The -mtime in find is too coarse. If you have files modified 6 hours ago and
    # 8 hours ago, they can't be distinguished with "find -mtime". One way to work
    # around this is to create a temp file and look for files newer than that.
    
    tmp_file=`mktemp`
    touch --date $doi $tmp_file
    
    find /path/to/nfs/dir -iname '*.foo' \
    -newer $tmp_file -exec hadoop fs -put -f '{}' \
    /path/to/hdfs/dir \;
    
    rm -f $tmp_file
    

Find files newer than files in another directory

This will find all files in raw_dir that are newer than the files in processed_dir.

    find raw_dir -newer `ls processed_dir/*.txt -rt | tail -n1` -iname '*.txt'
    

gunzip files on nfs and copy to hadoop

    files_to_copy=`find $dir -maxdepth 1 -iname '*.avro.gz' -newer $tmp_file -print
    for i in $files_to_copy
    do
        unzipped_file=${tmp_nfs_dir}/`basename $i | sed -e 's/\.gz$//'`
        gunzip $i -c > $unzipped_file
        hadoop fs -put -f $unzipped_file $hdfs_dir
        echo "$unzipped_file -> $hdfs_dir"
        rm -f $unzipped_file
    done
    

tags | process the output of find

iterate over files from find where the files have spaces

Say we have some files with spaces in them

    bash-4.1$ ls -1
    foo (2).csv
    foo.csv
    

To iterate over them, change the IFS

    bash-4.1$ OLDIFS="$IFS"
    bash-4.1$ IFS=$'\n'
    bash-4.1$ a=`find . -iname '*.csv'`
    bash-4.1$ for i in $a; do echo $i; done
    ./foo.csv
    ./foo (2).csv
    bash-4.1$ IFS="$OLDIFS"
    

If you try to loop over the files without changing the IFS, it will break on the spaces.

    bash-4.1$ a=`find . -iname '*.csv'`
    bash-4.1$ for i in $a; do echo $i; done
    ./foo.csv
    ./foo
    (2).csv
    

Ref:- https://unix.stackexchange.com/a/9499/198064

find files with modified status information

    find / -type f -cmin -2880 -a -cmin +1440 -print | less
    

will find files whose status information was modified between 24 and 48 hours ago. If you renamed a file, that should affect the ctime. You can choose your numbers appropriately and focus the search by changing / to something more specific, or even adding -name \*.ext where ext is the known extension.

useful links