Find usage

From raju

find all python and shell scripts in a directory

search tags | regex py sh 

    find . -regex ".*\.\(sh\|py\)$"

Ref:- https://unix.stackexchange.com/questions/402843/how-to-add-execution-permission-x-to-all-scripts

find files by extension and sort them by size

    find . -iname '*.py' -printf "%s %p\n" | sort -n | sed 's/^[0-9]* //'

Explanation: 

    -printf "%s %p\n" - %s prints the file size in bytes; %p prints the file name.

Ref:- https://unix.stackexchange.com/questions/430381/order-all-files-by-size-using-find

search all python files in a directory for a string

    find . -iname '*.py' -exec grep foo -inH --color {} +

Find recent files

    find . -mtime -1 -type f
find dir -type f -mtime -1 -ls
find . -iname '*.sw*' -atime -10 -type f

Do not show the starting directory in the output

By default, find shows the starting directory name in the output. For example 

    $find x -iname '*elastic*'
x/elastic_00.txt
x/x1/elastic_01.txt
x/x1/x2/elastic_02.txt

To remove it, use -printf '%P\n' 

    $find x -iname '*elastic*' -printf '%P\n'
elastic_00.txt
x1/elastic_01.txt
x1/x2/elastic_02.txt

To display only the file names without printing the subdirectories, use the '%f' modifier 

    $find x -iname '*elastic*' -printf '%f\n'
elastic_00.txt
elastic_01.txt
elastic_02.txt

Tested on GNU findutils 4.6.0

copy all but one file

To deploy all shell scripts and python scripts except the deploying script itself 

    DEPLOY_DIR=$HOME/bin
find . \( -iname '*.sh' -o -iname '*.py' \) \
    -not -name deploy.sh \
    -exec cp -vt $DEPLOY_DIR {} +

You can also use ! instead of -not 

    DEPLOY_DIR=$HOME/bin
find . \( -iname '*.sh' -o -iname '*.py' \) ! -name deploy.sh -exec cp -vt $DEPLOY_DIR {} +

Ref:- https://stackoverflow.com/questions/1313590/bash-copy-all-files-except-one

Find files containing two expressions

Search the C++ source code and header files to find files that have two keywords - ifstream and string 

    find /path/to/dir \( -iname '*.C' -o -iname '*.H' \) -print0  | xargs -0 grep -lZ ifstream  | xargs -0 grep -l string

All intermediate grep statements should use the -Z option so that the output from grep can be passed to "xargs -0".

list files with two different extensions

To search for both .jpg and .png files 

    find /path/to/dir \( -iname '*.jpg' -o -iname '*.png' \)

The expression within \( ... \) should be delimited by spaces on both sides.

To do the same in two directories 

    find /path/to/dir1 /path/to/dir2 \( -iname '*.jpg' -o -iname '*.png' \)

list all files in a directory that do not end with a particular extension

=> find . -maxdepth 1 -not -iname "*.H"

find all git folders under a directory

find . -name .git -type d -prune

Ref:- http://stackoverflow.com/a/11998951/6305733

grep files in a directory but exclude subdirectories, files matching a pattern

find . -maxdepth 1 ! -type d ! -name "*.sw?" ! -name "*_RK_????????_??????" ! -name "*_blame_????????_??????" -exec grep -inH --color key_word {} +

Did I ever call awk from a perl script?

rajulocal@hogwarts ~/work/rutils
 % find . -iname '*.pl' -exec grep awk -inH --color {} +
./perl/app_status_by_keyword.pl:48:    my $cmd = "dpkg -l \*$keyword\* | grep ^ii | awk '{print \$2}'";

List all the R programs I accessed in the last 60 days

% find . -iname '*.R' -atime -60 -type f

Find all the log files and delete them

First make sure that you are matching on the correct set of files 

find . -iname '*.log'

Then use any of the following commands 

find . -iname '*.log' -exec rm {} +
find . -iname '*.log' -print0 | xargs -0 rm
find . -iname '*.log' -delete

To delete interactively 

find . -iname '*.log' -exec rm -i {} +

The exact command depends on the installation since some versions of find do not have -print0, -delete etc., In any case, try to avoid 

find . -iname '*.log' -exec rm {} \;

which spawns an external process for each and every matched file.

The advantage of using "-exec rm" approach is that you can pass additional options to the rm command. For example 

find . -iname '*.log' -exec rm -i {} \;

will delete the files interactively.

tags | remove files

delete output directories older than a month

    OUTPUT_ROOT=/foo/bar
echo "Deleting directories older than a month in $OUTPUT_ROOT"

# Check for directory existence before trying to remove the files.
# Otherwise, in the offchance that the variable is empty, find will search the
# current directory and will try to remove files there.
if [ -d "$OUTPUT_ROOT" ]; then
    find "$OUTPUT_ROOT" -maxdepth 1 -type d -ctime +30 -exec rm -rfv {} +
fi

Give write permission to the user for all top level directories under a directory

    find . -maxdepth 1 -type d -exec chmod u+w {} +

List permissions of all shell scripts in a directory

    find . -iname "*.sh" -exec ls -l {} +

Make all the scripts in a directory executable

    find . -iname "*.sh" -exec chmod 755 {} +

Aggregate data across all files modified in the last 10 minutes

 find . -mmin -10 -type f -print0 | xargs -0 tail -n +2 -q | sort | uniq

Find old files and move them to a directory

First make sure that you are matching on the correct set of files 

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6

Then append it with the mv command as below 

find path/to/source/dir \( -iname 'pattern1' -o -iname 'pattern2' \) -mtime +6 -exec mv -ivt path/to/dest/dir {} +

Find files modified after a certain date

The code snippet below will find files modified on or after a certain date and copies them from an nfs directory to hdfs. 

    doi="2017-04-24"
# The -mtime in find is too coarse. If you have files modified 6 hours ago and
# 8 hours ago, they can't be distinguished with "find -mtime". One way to work
# around this is to create a temp file and look for files newer than that.

tmp_file=`mktemp`
touch --date $doi $tmp_file

find /path/to/nfs/dir -iname '*.foo' \
-newer $tmp_file -exec hadoop fs -put -f '{}' \
/path/to/hdfs/dir \;

rm -f $tmp_file

Find files newer than files in another directory

This will find all files in raw_dir that are newer than the files in processed_dir. 

    find raw_dir -newer `ls processed_dir/*.txt -rt | tail -n1` -iname '*.txt'

gunzip files on nfs and copy to hadoop

iterate over files from find where the files have spaces

Say we have some files with spaces in them 

    bash-4.1$ ls -1
foo (2).csv
foo.csv

To iterate over them, change the IFS 

    bash-4.1$ OLDIFS="$IFS"
bash-4.1$ IFS=$'\n'
bash-4.1$ a=`find . -iname '*.csv'`
bash-4.1$ for i in $a; do echo $i; done
./foo.csv
./foo (2).csv
bash-4.1$ IFS="$OLDIFS"

If you try to loop over the files without changing the IFS, it will break on the spaces. 

    bash-4.1$ a=`find . -iname '*.csv'`
bash-4.1$ for i in $a; do echo $i; done
./foo.csv
./foo
(2).csv

Ref:- https://unix.stackexchange.com/a/9499/198064

find files with modified status information

    find / -type f -cmin -2880 -a -cmin +1440 -print | less

will find files whose status information was modified between 24 and 48 hours ago. If you renamed a file, that should affect the ctime. You can choose your numbers appropriately and focus the search by changing / to something more specific, or even adding -name \*.ext where ext is the known extension.

search multiple directories

The usual path argument can take multiple directories if they are separated by space. 

    find dir1 dir2 -maxdepth 1 -iname '*.py'

useful links

Retrieved from "https://raju.shoutwiki.com/w/index.php?title=Find_usage&oldid=5637"