Find usage

From raju

find all python and shell scripts in a directory

search tags | regex py sh

    find . -regex ".*\.\(sh\|py\)$"


find files by extension and sort them by size

    find . -iname '*.py' -printf "%s %p\n" | sort -n | sed 's/^[0-9]* //'


    -printf "%s %p\n" - %s prints the file size in bytes; %p prints the file name.


search all python files in a directory for a string

    find . -iname '*.py' -exec grep foo -inH --color {} +

Find recent files

    find . -mtime -1 -type f
    find dir -type f -mtime -1 -ls
    find . -iname '*.sw*' -atime -10 -type f

Do not show the starting directory in the output

By default, find shows the starting directory name in the output. For example

    $find x -iname '*elastic*'

To remove it, use -printf '%P\n'

    $find x -iname '*elastic*' -printf '%P\n'

To display only the file names without printing the subdirectories, use the '%f' modifier

    $find x -iname '*elastic*' -printf '%f\n'

Tested on GNU findutils 4.6.0

copy all txt files in a directory to another

    find $SRC_DIR -maxdepth 1 -iname '*.txt' -exec cp -vt $DEST_DIR {} +

Note: The $DEST_DIR must exist. Otherwise it will throw an error.

The difference between this and "cp $SRC_DIR/*.txt $DEST_DIR" is that the latter throws an error if there are no txt files in $SRC_DIR. The find command simply completes even if there are no files.

Sample run

    $ mkdir -p x1/x2 x3
    $ touch x1/file1.txt x1/file1.pdf x1/file2.txt x1/x2/file3.txt
    ├── x1
    │   ├── file1.pdf
    │   ├── file1.txt
    │   ├── file2.txt
    │   └── x2
    │       └── file3.txt
    └── x3
    3 directories, 4 files
    $find x1 -maxdepth 1 -iname '*.txt' -exec cp -vt x3 {} +
    `x1/file1.txt' -> `x3/file1.txt'
    `x1/file2.txt' -> `x3/file2.txt'
    ├── x1
    │   ├── file1.pdf
    │   ├── file1.txt
    │   ├── file2.txt
    │   └── x2
    │       └── file3.txt
    └── x3
        ├── file1.txt
        └── file2.txt
    3 directories, 6 files

Without the -maxdepth option, all files underneath $SRC_DIR will be copied. The directory hierarchy is not preserved. For example, while files in x1 are copied to x3 so are the files in x1/x2 etc.,

    $ rm -rf x3
    $ mkdir x3
    $ find x1 -iname '*.txt' -exec cp -vt x3 {} +
    `x1/x2/file3.txt' -> `x3/file3.txt'
    `x1/file1.txt' -> `x3/file1.txt'
    `x1/file2.txt' -> `x3/file2.txt'
    $ tree
    ├── x1
    │   ├── file1.pdf
    │   ├── file1.txt
    │   ├── file2.txt
    │   └── x2
    │       └── file3.txt
    └── x3
        ├── file1.txt
        ├── file2.txt
        └── file3.txt
    3 directories, 7 files

copy all but one file

To deploy all shell scripts and python scripts except the deploying script itself

    find . \( -iname '*.sh' -o -iname '*.py' \) \
        -not -name \
        -exec cp -vt $DEPLOY_DIR {} +

You can also use ! instead of -not

    find . \( -iname '*.sh' -o -iname '*.py' \) ! -name -exec cp -vt $DEPLOY_DIR {} +


Find files containing two expressions

Search the C++ source code and header files to find files that have two keywords - ifstream and string

    find /path/to/dir \( -iname '*.C' -o -iname '*.H' \) -print0  | xargs -0 grep -lZ ifstream  | xargs -0 grep -l string

All intermediate grep statements should use the -Z option so that the output from grep can be passed to "xargs -0".

list files with two different extensions

To search for both .jpg and .png files

    find /path/to/dir \( -iname '*.jpg' -o -iname '*.png' \)

The expression within \( ... \) should be delimited by spaces on both sides.

To do the same in two directories

    find /path/to/dir1 /path/to/dir2 \( -iname '*.jpg' -o -iname '*.png' \)

list all files in a directory that do not end with a particular extension

=> find . -maxdepth 1 -not -iname "*.H"

find all git folders under a directory

find . -name .git -type d -prune


list a directory but exclude directories and vim swap files

find . -maxdepth 1 ! -type d ! -name "*.sw?"

grep files in a directory but exclude subdirectories, files matching a pattern

find . -maxdepth 1 ! -type d ! -name "*.sw?" ! -name "*_RK_????????_??????" ! -name "*_blame_????????_??????" -exec grep -inH --color key_word {} +

Did I ever call awk from a perl script?

rajulocal@hogwarts ~/work/rutils
 % find . -iname '*.pl' -exec grep awk -inH --color {} +
./perl/    my $cmd = "dpkg -l \*$keyword\* | grep ^ii | awk '{print \$2}'";

List all the R programs I accessed in the last 60 days

% find . -iname '*.R' -atime -60 -type f

Find all the log files and delete them

First make sure that you are matching on the correct set of files

find . -iname '*.log'

Then use any of the following commands

find . -iname '*.log' -exec rm {} +
find . -iname '*.log' -print0 | xargs -0 rm
find . -iname '*.log' -delete

To delete interactively

find . -iname '*.log' -exec rm -i {} +

The exact command depends on the installation since some versions of find do not have -print0, -delete etc., In any case, try to avoid

find . -iname '*.log' -exec rm {} \;

which spawns an external process for each and every matched file.

The advantage of using "-exec rm" approach is that you can pass additional options to the rm command. For example

find . -iname '*.log' -exec rm -i {} \;

will delete the files interactively.

tags | remove files

delete output directories older than a month

    echo "Deleting directories older than a month in $OUTPUT_ROOT"
    # Check for directory existence before trying to remove the files.
    # Otherwise, in the offchance that the variable is empty, find will search the
    # current directory and will try to remove files there.
    if [ -d "$OUTPUT_ROOT" ]; then
        find "$OUTPUT_ROOT" -maxdepth 1 -type d -ctime +30 -exec rm -rfv {} +

Give write permission to the user for all top level directories under a directory

    find . -maxdepth 1 -type d -exec chmod u+w {} +

List permissions of all shell scripts in a directory

    find . -iname "*.sh" -exec ls -l {} +

Make all the scripts in a directory executable

    find . -iname "*.sh" -exec chmod 755 {} +

Aggregate data across all files modified in the last 10 minutes

 find . -mmin -10 -type f -print0 | xargs -0 tail -n +2 -q | sort | uniq

Find old files and move them to a directory

First make sure that you are matching on the correct set of files

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6

Then append it with the mv command as below

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6 -exec mv -ivt path/to/dest/dir {} +

Find files modified after a certain date

The code snippet below will find files modified on or after a certain date and copies them from an nfs directory to hdfs.

    # The -mtime in find is too coarse. If you have files modified 6 hours ago and
    # 8 hours ago, they can't be distinguished with "find -mtime". One way to work
    # around this is to create a temp file and look for files newer than that.
    touch --date $doi $tmp_file
    find /path/to/nfs/dir -iname '*.foo' \
    -newer $tmp_file -exec hadoop fs -put -f '{}' \
    /path/to/hdfs/dir \;
    rm -f $tmp_file

Find files newer than files in another directory

This will find all files in raw_dir that are newer than the files in processed_dir.

    find raw_dir -newer `ls processed_dir/*.txt -rt | tail -n1` -iname '*.txt'

gunzip files on nfs and copy to hadoop

    files_to_copy=`find $dir -maxdepth 1 -iname '*.avro.gz' -newer $tmp_file -print
    for i in $files_to_copy
        unzipped_file=${tmp_nfs_dir}/`basename $i | sed -e 's/\.gz$//'`
        gunzip $i -c > $unzipped_file
        hadoop fs -put -f $unzipped_file $hdfs_dir
        echo "$unzipped_file -> $hdfs_dir"
        rm -f $unzipped_file

tags | process the output of find

iterate over files from find where the files have spaces

Say we have some files with spaces in them

    bash-4.1$ ls -1
    foo (2).csv

To iterate over them, change the IFS

    bash-4.1$ OLDIFS="$IFS"
    bash-4.1$ IFS=$'\n'
    bash-4.1$ a=`find . -iname '*.csv'`
    bash-4.1$ for i in $a; do echo $i; done
    ./foo (2).csv
    bash-4.1$ IFS="$OLDIFS"

If you try to loop over the files without changing the IFS, it will break on the spaces.

    bash-4.1$ a=`find . -iname '*.csv'`
    bash-4.1$ for i in $a; do echo $i; done


find files with modified status information

    find / -type f -cmin -2880 -a -cmin +1440 -print | less

will find files whose status information was modified between 24 and 48 hours ago. If you renamed a file, that should affect the ctime. You can choose your numbers appropriately and focus the search by changing / to something more specific, or even adding -name \*.ext where ext is the known extension.

search multiple directories

The usual path argument can take multiple directories if they are separated by space.

    find dir1 dir2 -maxdepth 1 -iname '*.py'

useful links