Linux notes
From raju
dummy
head tail usage
head -n -5 FILE print everything but the last 5 lines head -n +5 FILE print only the first 5 lines tail -n -5 FILE print only the last 5 lines tail -n +5 FILE print from 5th line onwards
print last but one line
cat FILE | tail -n -2 | head -n +1
here documents
tags | EOF, heredocs
Example 1:
To remove leading zeros from otherwise integer values using command line and standard input
$cat << EOF | bc > 001 > 002 > 003 > EOF 1 2 3
Example 2:
Use 'EOF' to not expand the shell variables inside a here document.
$ cat << EOF > echo $HOME > EOF echo /home/raju $ cat << 'EOF' > echo $HOME > EOF echo $HOME
Tips
convert jpg to pdf
Change the default configuration of imagemagick. Has to be done only once.
sudo sed -i -e 's!<policy domain="coder" rights="none" pattern="PDF" />!<policy domain="coder" rights="read|write" pattern="PDF" />!g' /etc/ImageMagick-6/policy.xml
Then
convert foo.jpg foo.pdf
With the default policy.xml, I was getting
% convert first_day.jpg first_day.pdf convert-im6.q16: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408. % gs --version 9.27 % convert --version Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org Copyright: © 1999-2019 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC Modules OpenMP Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib % cat /etc/debian_version 10.4
Ref:- https://askubuntu.com/questions/1127260/imagemagick-convert-not-allowed
redirect standard output and error
To redirect standard output and error into a file
foo > log.txt 2>&1
useful HOWTOs and tutorials
Increase the Sound Level in a Video - http://blog.expat-it.net/index.cgi/Admin/commandLine/files/increase-sound-level-in-video.html
ffmpeg -i video-in.avi -vol 1536 -vcodec copy video-out.avi
Every 256 in vol increases sound by 100%. So in the above example, the sound will be increased by 600%.
wget tips
using wget to download a file via php script
wget http://www.vim.org/scripts/download_script.php?src_id=21680 -O NrrwRgn-0.32.vmb -q
Listing options in this order is more easy to remember (get the link, store it here and be quiet). Compare this to
wget -O csv-0.30.vmb -q http://vim.sourceforge.net/scripts/download_script.php?src_id=21684
where the thought process is not as smooth.
Ref:- https://raam.org/2007/using-wget-to-run-a-php-script/
execute external commands
To execute an external command in gdb
(gdb) help shell Execute the rest of the line as a shell command. With no arguments, run an inferior shell. (gdb) shell date Sun Jan 18 15:04:19 EST 2015
enter tab character in the shell
<CTRL-v><Tab>
will insert a tab character on the command-line.
find usage articles
Articles on using find
List directories by size
To sort the directories by disk usage
$ du -ah --max-depth=1 | sort -hr | head -n 10 1.3G . 143M ./agg_5 143M ./agg_4 143M ./agg_3 143M ./agg_2 143M ./agg_1 122M ./run_3 121M ./run_5 121M ./run_4 121M ./run_2
or
rajulocal@hogwarts ~ % du -a -h --max-depth=1 | sort -h | tail -n 10 225M ./compile_static_gdb_debian_way 228M ./.local 370M ./songs 501M ./youtube_videos 849M ./x 944M ./.cache 1.2G ./work 1.7G ./.adobe 8.0G ./software 15G .
tips available on the net
add a string to the end of each line
We can use awk to add a given string to the end of each line in a file. For example,
% cat ~/x/junk21.txt a an the they their
% cat ~/x/junk21.txt| awk '{print $0 " str .=$()-"}' a str .=$()- an str .=$()- the str .=$()- they str .=$()- their str .=$()-
search the symbols
nm DebuggableExecutable -C
check if the DISPLAY variable is set correctly
Use "xset q" and check the error code. For example if it is working
% xset q <output suppressed> % echo $? 0
If it is not working
% xset q xset: unable to open display "" % echo $? 1
What type of OS am I running?
Use OSTYPE environment variable to check the type of operating system.
% printenv | grep OS ... OSTYPE=linux-gnu ...
or simply
% printenv OSTYPE linux-gnu
top usage
- Run in batch mode
top -c -u [username] -b -n 1
-c -u [username] is optional -b is for batch mode -n 1 is to stop after one iteration
- track the memory usage of a process over time
top -b -p [pid] < /dev/null >& top.out &
Look at the VIRT field.
updatedb and locate
- To generate the database
updatedb -U <dir1> -l 0 -o <dir1.db> updatedb -U <dir2> -l 0 -o <dir2.db>
- To search the database
locate -d <dir1.db> -d <dir2.db> -e -i <expr>
Looks like the updatedb options have changed?
mkdir -p ~/.locate cd ~/.locate updatedb --netpaths='dir1' --output=dir1.db
tested using | updatedb 4.6.0
Open a pdf at a specific page
To open at the 10th page of a pdf. Note that this is not same as opening it at page 10 by label.
acroread /a page=10 file.pdf evince -i 10 file.pdf okular -p 10 file.pdf
To open at page 10 by label.
evince -p 10 file.pdf
cat show filename
print the type of file system
df -T /path/to/dir
This can tell if the directory is a local directory (ex:- ext4 filesytem) or a remote mount (ex:- nfs).
create a file of specific size with random ascii data
Method 1:
base64 /dev/urandom | head -c 1024 > file.txt
Replace 1024 with the number of bytes. You can also use 1k, 1m, 1g (or 1K, 1M, 1G) etc., as arguments to create files of 1 kilo byte (i.e. 1024 bytes), mega byte and giga byte respectively.
Method 2:
tr -dc A-Za-z0-9 < /dev/urandom | head -c 1024 > file.txt
create a zip file from list of files
zip foo.zip bar.txt baz.txt
To unzip
unzip foo.zip
Note that this will not delete the foo.zip.
shell redirect before the command
$ >fruits echo apple $ cat fruits apple $ >>fruits echo grape $ cat fruits apple grape
Ref:-
- I first came across this trick in https://devblogs.microsoft.com/oldnewthing/20190514-00/?p=102493 . This trick is necessary to avoid unwanted spaces, or to avoid conflicts with lines that end with a digit.
recursively count number of lines in all files under a directory
wc `find`
Ref:- https://stackoverflow.com/a/8263707/6305733
external links
- https://www.rootusers.com/13-simple-xz-examples/ - {pros: "shows common use cases, clearly explained", cons : "too many ads"}
append output of two commands
Sample commands:
( command1 ; command2 ; command3 ) | cat { command1 ; command2 ; command3 ; } > outfile.txt { command1 & command2; } { command1 & command2; } > new_file { command1 & command2; } > STDOUT_file 2> STDERR_file
Ref:- https://unix.stackexchange.com/questions/64736/combine-the-output-of-two-commands-in-bash
echo multiple lines
Can be with cat and a here document
$ cat << EOF > line1 > line2 > EOF line1 line2
commonly used date time stamps
time stamp for log files
$ foo > bar_`date +'%Y%m%d_%H%M%S'`
stores the output of foo into a file called bar_YYYYmmDDHHMMSS
Standards
XDG
- XDG Base Directory Specification - https://specifications.freedesktop.org/basedir-spec/latest/
Troubleshooting
slow file listing
tl;dr
Setting LS_COLORS as
export LS_COLORS='ex=00:su=00:sg=00:ca=00:'
will speed up the listing of files in directories with lots of files.