| Author | (all..) |
| Maintainers | Jim Meyering |
| Paul Eggert | |
| Pádraig Brady | |
| Released | 2003 |
| Source | coreutils.git |
| Homepage | GNU Coreutils |
The GNU Core Utilities are the basic file, shell and text manipulation utilities
of the GNU operating system. These are the core utilities which are expected to
exist on every operating system. This package is the union of the GNU
fileutils, shellutils, and textutils packages.
| CMD | MAN | DESCRIPTION |
| [ | (1) | bash built-in commands, see bash(1) |
| arch | (1) | print machine hardware name (same as uname -m) |
| b2sum | (1) | compute and check BLAKE2 message digest |
| base32 | (1) | base32 encode/decode data and print to standard output |
| base64 | (1) | base64 encode/decode data and print to standard output |
| basename | (3) | parse pathname components |
| basename | (1) | strip directory and suffix from filenames |
| basename | (3p) | return the last component of a pathname |
| basename | (1p) | return non-directory portion of a pathname |
| cat | (1) | concatenate files and print on the standard output |
| cat | (1p) | concatenate and print files |
| chcon | (1) | change file SELinux security context |
| chgrp | (1) | change group ownership |
| chgrp | (1p) | change the file group ownership |
| chmod | (1) | change file mode bits |
| chmod | (2) | change permissions of a file |
| chmod | (3p) | change mode of a file relative to directory file descr... |
| chmod | (1p) | change the file modes |
| chown | (1) | change file owner and group |
| chown | (2) | change ownership of a file |
| chown | (3p) | change owner and group of a file relative to directory... |
| chown | (1p) | change the file ownership |
| cksum | (1) | checksum and count the bytes in a file |
| cksum | (1p) | write file checksums and sizes |
| comm | (1) | compare two sorted files line by line |
| comm | (1p) | select or reject lines common to two files |
| cp | (1) | copy files and directories |
| cp | (1p) | copy files |
| csplit | (1) | split a file into sections determined by context lines |
| csplit | (1p) | split files based on context |
| cut | (1) | remove sections from each line of files |
| cut | (1p) | cut out selected fields of each line of a file |
| date | (1) | print or set the system date and time |
| date | (1p) | write the date and time |
| dd | (1) | convert and copy a file |
| dd | (1p) | convert and copy a file |
| df | (1) | report file system disk space usage |
| df | (1p) | report free disk space |
| dir | (1) | list directory contents |
| dircolors | (1) | color setup for ls |
| dirname | (3) | parse pathname components |
| dirname | (1) | strip last component from file name |
| dirname | (3p) | report the parent directory name of a file pathname |
| dirname | (1p) | return the directory portion of a pathname |
| du | (1) | estimate file space usage |
| du | (1p) | estimate file space usage |
| echo | (3x) | curses input options |
| echo | (1) | display a line of text |
| echo | (1p) | write arguments to standard output |
| env | (1) | run a program in a modified environment |
| env | (1p) | set the environment for command invocation |
| expand | (1) | convert tabs to spaces |
| expand | (1p) | convert tabs to spaces |
| expr | (1) | evaluate expressions |
| expr | (1p) | evaluate arguments as an expression |
| factor | (1) | factor numbers |
| false | (1) | do nothing, unsuccessfully |
| false | (1p) | return false value |
| fmt | (1) | simple optimal text formatter |
| fold | (1) | wrap each input line to fit in specified width |
| fold | (1p) | filter for folding lines |
| groups | (1) | print the groups a user is in |
| HEAD | (1) | Simple command line user agent |
| head | (1) | output the first part of files |
| head | (1p) | copy the first part of files |
| hostid | (1) | print the numeric identifier for the current host |
| id | (1) | print real and effective user and group IDs |
| id | (1p) | return user identity |
| install | (1) | copy files and set attributes |
| join | (1) | join lines of two files on a common field |
| join | (1p) | relational database operator |
| link | (1) | call the link function to create a link to a file |
| link | (2) | make a new name for a file |
| link | (3p) | link one file to another file relative to two director... |
| link | (1p) | calllink() function |
| ln | (1) | make links between files |
| ln | (1p) | link files |
| logname | (1) | print user's login name |
| logname | (1p) | return the user's login name |
| ls | (1) | list directory contents |
| ls | (1p) | list directory contents |
| md5sum | (1) | compute and check MD5 message digest |
| mkdir | (1) | make directories |
| mkdir | (2) | create a directory |
| mkdir | (3p) | make a directory relative to directory file descriptor |
| mkdir | (1p) | make directories |
| mkfifo | (3) | make a FIFO special file (a named pipe) |
| mkfifo | (1) | make FIFOs (named pipes) |
| mkfifo | (3p) | make a FIFO special file relative to directory file de... |
| mkfifo | (1p) | make FIFO special files |
| mknod | (1) | make block or character special files |
| mknod | (2) | create a special or ordinary file |
| mknod | (3p) | make directory, special file, or regular file |
| mktemp | (3) | make a unique temporary filename |
| mktemp | (1) | create a temporary file or directory |
| mv | (1) | move (rename) files |
| mv | (1p) | move files |
| nice | (1) | run a program with modified scheduling priority |
| nice | (2) | change process priority |
| nice | (3p) | change the nice value of a process |
| nice | (1p) | invoke a utility with an altered nice value |
| nl | (3x) | curses output options |
| nl | (1) | number lines of files |
| nl | (1p) | line numbering filter |
| nohup | (1) | run a command immune to hangups, with output to a non-tty |
| nohup | (1p) | invoke a utility immune to hangups |
| nproc | (1) | print the number of processing units available |
| numfmt | (1) | Convert numbers from/to human-readable strings |
| od | (1) | dump files in octal and other formats |
| od | (1p) | dump files in various formats |
| paste | (1) | merge lines of files |
| paste | (1p) | merge corresponding or subsequent lines of files |
| pathchk | (1) | check whether file names are valid or portable |
| pathchk | (1p) | check pathnames |
| pinky | (1) | lightweight finger |
| pr | (1) | convert text files for printing |
| pr | (1p) | print files |
| printenv | (1) | print all or part of environment |
| printf | (3) | formatted output conversion |
| printf | (1) | format and print data |
| printf | (3p) | print formatted output |
| printf | (1p) | write formatted output |
| ptx | (1) | produce a permuted index of file contents |
| pwd | (1) | print name of current/working directory |
| pwd | (1p) | return working directory name |
| readlink | (1) | print resolved symbolic links or canonical file names |
| readlink | (2) | read value of a symbolic link |
| readlink | (3p) | read the contents of a symbolic link |
| realpath | (3) | return the canonicalized absolute pathname |
| realpath | (1) | print the resolved path |
| realpath | (3p) | resolve a pathname |
| rm | (1) | remove files or directories |
| rm | (1p) | remove directory entries |
| rmdir | (1) | remove empty directories |
| rmdir | (2) | delete a directory |
| rmdir | (3p) | remove a directory |
| rmdir | (1p) | remove directories |
| runcon | (1) | run command with specified SELinux security context |
| seq | (1) | print a sequence of numbers |
| sha1sum | (1) | compute and check SHA1 message digest |
| sha224sum | (1) | compute and check SHA224 message digest |
| sha256sum | (1) | compute and check SHA256 message digest |
| sha384sum | (1) | compute and check SHA384 message digest |
| sha512sum | (1) | compute and check SHA512 message digest |
| shred | (1) | overwrite a file to hide its contents, and optionally ... |
| shuf | (1) | generate random permutations |
| sleep | (3) | sleep for the specified number of seconds |
| sleep | (1) | delay for a specified amount of time |
| sleep | (3p) | suspend execution for an interval of time |
| sleep | (1p) | suspend execution for an interval |
| sort | (1) | sort lines of text files |
| sort | (1p) | sort, merge, or sequence check text files |
| split | (1) | split a file into pieces |
| split | (1p) | split files into pieces |
| stat | (1) | display file or file system status |
| stat | (2) | get file status |
| stat | (3p) | get file status |
| stdbuf | (1) | Run COMMAND, with modified buffering operations for it... |
| stty | (1) | change and print terminal line settings |
| stty | (2) | unimplemented system calls |
| stty | (1p) | set the options for a terminal |
| sum | (1) | checksum and count the blocks in a file |
| sync | (1) | Synchronize cached writes to persistent storage |
| sync | (2) | commit filesystem caches to disk |
| sync | (3p) | schedule file system updates |
| tac | (1) | concatenate and print files in reverse |
| tail | (1) | output the last part of files |
| tail | (1p) | copy the last part of a file |
| tee | (1) | read from standard input and write to standard output ... |
| tee | (2) | duplicating pipe content |
| tee | (1p) | duplicate standard input |
| test | (1) | check file types and compare values |
| test | (1p) | evaluate expression |
| timeout | (3x) | curses input options |
| timeout | (1) | run a command with a time limit |
| touch | (1) | change file timestamps |
| touch | (1p) | change file access and modification times |
| tr | (1) | translate or delete characters |
| tr | (1p) | translate characters |
| true | (1) | do nothing, successfully |
| true | (1p) | return true value |
| truncate | (1) | shrink or extend the size of a file to the specified size |
| truncate | (2) | truncate a file to a specified length |
| truncate | (3p) | truncate a file to a specified length |
| tsort | (1) | perform topological sort |
| tsort | (1p) | topological sort |
| tty | (4) | controlling terminal |
| tty | (1) | print the file name of the terminal connected to stand... |
| tty | (1p) | return user's terminal name |
| uname | (1) | print system information |
| uname | (2) | get name and information about current kernel |
| uname | (3p) | get the name of the current system |
| uname | (1p) | return system name |
| unexpand | (1) | convert spaces to tabs |
| unexpand | (1p) | convert spaces to tabs |
| uniq | (1) | report or omit repeated lines |
| uniq | (1p) | report or filter out repeated lines in a file |
| unlink | (1) | call the unlink function to remove the specified file |
| unlink | (2) | delete a name and possibly the file it refers to |
| unlink | (3p) | remove a directory entry relative to directory file de... |
| unlink | (1p) | call theunlink() function |
| users | (1) | print the user names of users currently logged in to t... |
| vdir | (1) | list directory contents |
| wc | (1) | print newline, word, and byte counts for each file |
| wc | (1p) | word, line, and byte or character count |
| who | (1) | show who is logged on |
| who | (1p) | display who is on the system |
| whoami | (1) | print effective userid |
| yes | (1) | output a string repeatedly until killed |
| chroot | (1) | run command or interactive shell with special root dir... |
| chroot | (2) | change root directory |
⬆ base64 encode/decode data and print to standard output Written by Simon Josefsson
create a C chars by L lines file of a random ASCII characters:
#!/usr/bin/env bash
ABSPATH="$(dirname $(readlink -f $0))"
L=100
C=80
FILE_PATH="$ABSPATH"/../tests/rnd"$C"x"$L"
[ -e "$FILE_PATH" ] && rm "$FILE_PATH"
for ((n=0;n<"$L";n++)); do
base64 /dev/urandom | head -c "$C" >> "$FILE_PATH"
doneif, for, dirname (1), readlink (1), rm (1), head (1),
⬆ change file mode bits Written by David MacKenzie and Jim Meyering
grant read access (r) to a file to all members of your group (g):
~$ chmod g+r file_name ~$ chmod 0040 file_name
grant read access to a directory to all members your group:
~$ chmod g+rx dir_name ~$ chmod 0050 dir_name
Note: “execute” permission is required in order to read a directory.
grant read permissions to everyone on the system:
~$ chmod ugo+r file_name ~$ chmod 0444 file_name
grant read permissions on a directory to everyone on the system:
~$ chmod ugo+rx dir_name ~$ chmod 0555 dir_name
grant modify or delete permissions to a file which you own for everyone in the group:
~$ chmod ugo+rw file_name ~$ chmod 0666 file_name
Note: In order for modify and delete permissions to be useful, one must be able to modify the directory in which the file is located: chmod ugo+rwx ./
deny read access to a file by everyone except yourself:
~$ chmod go-r file_name
allow everyone in your group to be able to modify the file:
~$ chmod 660 file-name
⬆ compare two sorted files line by line Written by Richard M. Stallman and David MacKenzie
find lines existing only in file_1 against file_2::
~$ comm -23 file_1 file_2 # if both file sorted ~$ comm -23 <(sort -u file_1) <(sort -u file_2) # if both file sorted
⬆ copy files and directories Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering
| OPT | TYPE | ++V | --V | DESC |
|---|---|---|---|---|
--attributes-only | copy just the attributes | |||
--backup[=CONTROL] | make a backup of each existing destination file | |||
--context[=CTX] | like -Z, or if CTX is specified then set the SELinux or SMACK security context to CTX | |||
--copy-contents | copy contents of special files when recursive | |||
--help | display this help and exit | |||
--no-preserve=ATTR_LIST | don’t preserve the specified attributes | |||
--parents | use full source file name under DIRECTORY | |||
--preserve[=ATTR_LIST] | preserve the specified attributes | |||
--reflink[=WHEN] | control clone/CoW copies. See below | |||
--remove-destination | remove each existing destination file before attempting to open it | |||
--sparse=WHEN | control creation of sparse files. See below | |||
--strip-trailing-slashes | remove any trailing slashes from each SOURCE argument | |||
--version | output version information and exit | |||
-H | follow command-line symbolic links in SOURCE | |||
-L, --dereference | always follow symbolic links in SOURCE | |||
-P, --no-dereference | never follow symbolic links in SOURCE | |||
-R, -r, --recursive | copy directories recursively | |||
-S, --suffix=SUFFIX | override the usual backup suffix | |||
-T, --no-target-directory | treat DEST as a normal file | |||
-Z | set SELinux security context of destination file to default type | |||
-a, --archive | same as -dR –preserve=all | |||
-b | like –backup but does not accept an argument | |||
-c | deprecated, same as –preserve=context | |||
-d | same as –no-dereference –preserve=links | |||
-f, --force | if an existing destination file cannot be opened, remove it and try again | |||
-i, --interactive | prompt before overwrite | |||
-l, --link | hard link files instead of copying | |||
-n, --no-clobber | do not overwrite an existing file | |||
-p | same as –preserve=mode,ownership,timestamps | |||
-s, --symbolic-link | make symbolic links instead of copying | |||
-t, --target-directory=DIRECTORY | copy all SOURCE arguments into DIRECTORY | |||
-u, --update | copy newer and missing files only | |||
-v, --verbose | explain what is being done | |||
-x, --one-file-system | stay on this file system |
recursively copy files if destination files are not exist or they are older:
~$ cp --update --recursive --verbose source_path destination_path ~$ cp -urv source_path destination_path
⬆ remove sections from each line of files Written by David M. Ihnat, David MacKenzie, and Jim Meyering
| OPT | TYPE | V++ | V– | DESC |
|---|---|---|---|---|
-b, --bytes=LIST | select only these bytes | |||
-c, --characters=LIST | select only these characters | |||
-d, --delimiter=DELIM | use DELIM instead of TAB for field delimiter | |||
-f, --fields=LIST | select only these fields | |||
-n | with -b: don’t split multibyte characters | |||
--complement | complement the set of selected bytes, characters | |||
-s, --only-delimited | do not print lines not containing delimiters | |||
--output-delimiter=STRING | use STRING as the output delimiter | |||
-z, --zero-terminated | line delimiter is NUL, not newline | |||
--help | display this help and exit | |||
--version | output version information and exit |
slice off lines from left, right, both sides by N chars:
~$ cut -c-N input_f # slice off from left by N ~$ rev input_f | cut -c-N | rev # slice off from rigth by N ~$ cut -c-N input_f | rev | cut -c-N | rev # slice off from both sides by N
⬆ convert and copy a file Written by Paul Rubin, David MacKenzie, and Stuart Kemp.
| OPT | TYPE | V++ | V– | DESC |
|---|---|---|---|---|
bs=BYTES | c w b K MB M GB G | |||
cbs=BYTES | ||||
conv=CONVS | ||||
count=N | ||||
ibs=BYTES | ||||
if=FILE | Input file | |||
iflag=FLAGS | ||||
obs=BYTES | ||||
of=FILE | Output file | |||
oflag=FLAGS | ||||
seek=N | ||||
skip=N | ||||
status=LEVEL | 8.24 |
output your microphone to a remote computer’s speaker:
~$ dd if=/dev/dsp | ssh -c arcfour -C username@host dd of=/dev/dsp
killing features of dd:
~# dd if=/dev/zero of=/dev/sdX # format device ~# dd if=/dev/random of=/dev/port # freeze the system
memory I/O speed test with 34G of data:
~$ time dd if=/dev/zero of=/dev/null bs=1M count=32768
make an image of and device and send it via ssh (or receive it):
~$ dd if=/dev/sda | gzip -1 - | ssh user@local dd of=img.gz # from remote ~$ ssh user@remote "dd if=/dev/sda | gzip -1 -" | dd of=img.zip # from local ~$ ssh user@remote "dd if=/dev/cdrom" | dd of=remote_cdrom.iso
gzip (1) ssh (1)
test NFS, LAN and I/O speed with 268MB image:
#!/usr/bin/bash
NFS_PATH="/mount/point/of/nfs/dir"
echo "Write speed"
time dd if=/dev/urandom of="$NFS_PATH"/test_file bs=16k count=16384
echo "Read speed"
time dd if="$NFS_PATH"/test_file of=/dev/null bs=16kecho time (1)
copy and restore MBR of block device (HDD/SSD):
#!/usr/bin/bash
BLK_CP="/dev/sda"
BLK_RS="/dev/sdb"
MBR="$HOME"/mbr.img
echo "Copy MBR data from $BLK_CP"
dd if="$BLK_CP" of="$MBR" bs=512 count=1
echo "Restore MBR data to $BLK_RS"
dd if="$MBR" of="$BLK_RS"stress test of BLOCK device with random size files to a random sector number:
#!/usr/bin/bash
# WARRNING: IT COULD DAMAGE YOUR DEVICE!
MNT_PATH="/mnt/lv_test"
DEV_NAME="sdc"
DEV_SIZE=$(cat "/sys/block/$DEV_NAME/size")
MBR=512
while true; do
SECTOR="$(shuf -i"$MBR"-"$DEV_SIZE" -n1)"
LENGTH="$(shuf -i"$SECTOR"-"$DEV_SIZE" -n1)"
# write to DEV
dd if=/dev/urandom \
of="$MNT_PATH"/file_name \
seek="$SECTOR" \
count="$LENGTH"b 1>/dev/null
# read from DEV
dd if="$MNT_PATH"/file_name of=/dev/null 1>/dev/null
# clean up and loop
rm "$MNT_PATH"/file_name
doneduplicate several drives concurrently:
#!/usr/bin/env bash
DEV_1="/dev/sda"
DEV_2="/dev/sdb"
DEV_3="/dev/sdc"
DEV_4="/dev/sdd"
# Add as many as needed afte tee command
dd if="$DEV_1" bs=64k \
| tee >(dd of="$DEV_2" bs=64k) \
>(dd of="$DEV_3" bs=64k) \
| dd of="$DEV_4" bs=64kread file1, file2 and append output back to file1:
~$ cat file1 <(dd if=file2 bs="$(wc -c file2)" count=1) >> file2
create a forensic image of all data within device:
~# dd if=/dev/sda1 of=output_file.img conv=notrunc,noerror,sync
without stopping should an error be encountered (such as bad blocks), and replacing any errors found with null bytes.
clone image to drive or USB with showing progress:
~# dd if=guix-system-install-1.0.1.x86_64-linux.iso of=/dev/sdc status=progress; sync
Where /dev/sdc is external drive or USB.
create a 1000M file which contains random data mixture binary and strings:
~$ dd if=/dev/urandom of=file bs=1024k count=1000 status=progress
⬆ estimate file space usage Written by Torbjorn Granlund, David MacKenzie, Paul Eggert, and Jim Meyering.
show size of hiden and normal directorates of the current location:
~$ du -sch .[!.]* * | sort -h
.[!.]* It’s a glob that lists all the files that start with a single .
sort
⬆ evaluate expressions Written by Mike Parker, James Youngman, and Paul Eggert.
expr arg1 operator arg2 [ operator arg3 ... ]
“grep” like examples of using expr: grep patern text matching
#!/usr/bin/env bash
TEST_VAR="Value of Expression is 10"
expr "$TEST_VAR" : '.*' # number of chars
expr "$TEST_VAR" : '\(.*\)' # match all chars and print them
expr "$TEST_VAR" : '[A-Z]*' # number of UPPER case chars at the begining
expr "$TEST_VAR" : '\(.....\)' # truncate to 5 chars⬆ make links between files Written by Mike Parker and David MacKenzie.
repoint symlink to another file:
~$ ln -nfs TARGET_PATH LINK_NAME
⬆ list directory contents. Written by Richard M. Stallman and David MacKenzie.
ls [-aAlbBCdDfFghHiIklLmNopqQrRsStTuvwxXZ1] [FILE/DIRECTORY]
list block devices:
~$ ls -l /dev /dev/mapper | grep '^b'
tree view of all sub directories:
#!/usr/bin/env bash
ls -R \
| grep ":$" \
| sed -e 's/:$//' \
-e 's/[^-][^\/]*\//--/g' \
-e 's/^/ /' \
-e 's/-/|/'sweet memorising examples of using ls:
~$ ls -Rotti ~$ ls -Ham ~$ ls -and ~$ ls -laSh ~$ ls -Qam
List only - dir, files, links, hidden dirs, hidden files hidden links, exec files. Actively using -F key to classify */=>@|:
~$ ls -F | grep '/' | cut -d/ -f1 # directories ~$ ls -p | grep -v / # files ~$ ls -F | grep '[@]' | cut -d@ -f1 # links ~$ ls -d .*/ | cut -d/ -f1 # hiden direcotires ~$ ls -a | grep "^\." # hiden files ~$ ls -Fa | grep "\." | grep "@" | cut -d@ -f1 # hiden links
aliases best practice:
alias ls="ls -1p --color=auto" alias l="ls -lhGgo" alias ll="ls -lh" alias la="ls -lhGgoA" alias lt="ls -lhGgotr" alias lS="ls -lhGgoSr" alias l.="ls -lhGgod .*" alias lhead="ls -lhGgo | head" alias ltail="ls -lhGgo | tail" alias lmore='ls -lhGgo | more'
sort files and directories in current working directory by size:
~$ ls -lS ./ # bigest first ~$ ls -lSr ./ # smallest first ~$ ls -lSh # with human readable file size ~$ ls -lS ./ | grep -v "^d" # just files, ignore directory
⬆ make directories Written by David MacKenzie
| -p | create full path |
create a set of 135199 directories:
~$ mkdir ./test_dir{1..1}{a..z}{A..Z}
create a directory tree with one line:
~$ mkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat/a}
⬆ create a temporary file or directory Written by Jim Meyering and Eric Blake.
| OPT | FROM V.1 | END V. | DESC |
|---|---|---|---|
-d, --directory | create a directory | ||
-u, --dry-run | do not create anything | ||
-q, --quiet | suppress diagnostics | ||
--suffix=SUFF | append SUFF to TEMPLATE | ||
-p DIR, --tmpdir | interpret TEMPLATE relative to DIR | ||
-t |
Create a temporary dirrectory in /tmp:
~$ mktemp -d -t test_dir.XXX
⬆ move (rename) files Written by Mike Parker, David MacKenzie, and Jim Meyering.
replace spaces in file/dir name with underscore “_”:
#!/usr/bin/env bash
SEARCH_PATH="$HOME"/Downloads
# First, replace spaces in dir names,
# Keep trying utill all pathes changed.
while true; do
[[ -z $(find "$SEARCH_PATH" -type d -name "* *") ]] && echo "Clean" && break
find "$SEARCH_PATH" -type d -name "* *" -print \
| while read dir; do
new_dir=$(echo "$dir" | sed 's/ /_/g')
mv "$dir" "$new_dir"
done
done
# Replace spaces in files after that
find "$SEARCH_PATH" -type f -name "* *" -print \
| while read file; do
new_file=$(echo "$file" | sed 's/ /_/g')
mv "$file" "$new_file"
donewhile, read, echo (1), sed (1), mv (1), find (1)
move multiple files or directories to the one target location:
~$ mv -t target_dir /path/to/file-1 /path/to/file-2 /path/to/dir-1
⬆ run a command immune to hangups, with output to a non-tty Written by Jim Meyering.
| 125 | if nohup itself fails, and POSIXLY_CORRECT is not set |
| 126 | if command is found but cannot be invoked |
| 127 | if command cannot be found |
| the exit status of command otherwise |
Run a script and exit from a shell, makes it running on background:
~$ nohup script_to_run.sh &; exit
Run a process as background process:
~$ nohup bash -c "(script_to_run.sh &>/dev/null)" &>/dev/null
It omits bouth stdin and stdout from script and nohup.
⬆ Convert numbers from/to human-readable strings Writen by Assaf Gordon
⬆ merge lines of files Written by David M. Ihnat and David MacKenzie
| OPT | TYPE | V++ | V– | DESC |
|---|---|---|---|---|
-s, --serial | switch | paste one file at a time instead of in parallel | ||
-d, --delimiters=LIST | str | reuse characters from LIST instead of TABs | ||
-z, --zero-terminated | switch | line delimiter is NUL, not newline |
sumirize the size of git directores using Bash
~$ echo $[$(find ~/ -type d -name ".git" -print0 | xargs -0 du -s | cut -f1 | paste -sd+)]
join all lines of the file with a SPACE delimiter:
~$ paste -sd" " input_f
merge a list to 2-n columns (- is a column placeholder):
~$ paste - - < file_name ~$ paste -d":" - - < file_name
list files with specified number of columns:
~$ ls -a | paste -d: - - - - | column -t -s:
⬆ print name of current/working directory Written by Jim Meyering
print relative path to the curent directory:
~$ pwd | rev | cut -d'/' -f1 | rev
⬆ remove files or directories Written by Paul Rubin, David MacKenzie, Richard M. Stallman, and Jim Meyering
remove all but specific file(s):
~$ rm !(save_me_please.log) # just a rest of files ~$ rm !(*.foo|*.bar|*.baz) # wild card
unlink (1), unlink(2), chattr(1), shred(1)
print a list of years +5 (N) from current:
~$ seq "$(date +%Y)" "$(( $(date +%Y) + 5))"
⬆ sort lines of text files Written by Mike Haertel and Paul Eggert
| OPT | TYPE | V++ | V– | DESC |
|---|---|---|---|---|
--batch-size=NMERGE | merge at most NMERGE inputs at once; for more use temp files | |||
--compress-program=PROG | compress temporaries with PROG; decompress them with PROG -d | |||
--debug | annotate the part of the line used to sort, and warn about questionable usage to stderr | |||
--files0-from=F | read input from the files specified by NUL-terminated names in file F; If F is - then read names from standard input | |||
--help | display this help and exit | |||
--parallel=N | change the number of sorts run concurrently to N | |||
--random-source=FILE | get random bytes from FILE | |||
--sort=WORD | sort according to WORD: general-numeric -g, human-numeric -h, month -M, numeric -n, random -R, version -V | |||
--version | output version information and exi | |||
-C, --check=quiet, --check=silent | like -c, but do not report first bad line | |||
-M, --month-sort | compare (unknown) < ‘JAN’ < … < ‘DEC’ | |||
-R, --random-sort | shuffle, but group identical keys. See shuf(1) | |||
-S, --buffer-size=SIZE | use SIZE for main memory buffer | |||
-T, --temporary-directory=DIR | use DIR for temporaries, not $TMPDIR or /tmp; multiple options specify multiple directories | |||
-V, --version-sort | natural sort of (version) numbers within text | |||
-b, --ignore-leading-blanks | ignore leading blanks | |||
-c, --check, --check=diagnose-first | check for sorted input; do not sort | |||
-d, --dictionary-order | consider only blanks and alphanumeric characters | |||
-f, --ignore-case | fold lower case to upper case characters | |||
-g, --general-numeric-sort | compare according to general numerical value | |||
-h, --human-numeric-sort | compare human readable numbers (e.g., 2K 1G) | |||
-i, --ignore-nonprinting | consider only printable characters | |||
-k, --key=KEYDEF | sort via a key; KEYDEF gives location and type | |||
-m, --merge | merge already sorted files; do not sort | |||
-n, --numeric-sort | compare according to string numerical value | |||
-o, --output=FILE | write result to FILE instead of standard output | |||
-r, --reverse | reverse the result of comparisons | |||
-s, --stable | stabilize sort by disabling last-resort comparison | |||
-t, --field-separator=SEP | use SEP instead of non-blank to blank transition | |||
-u, --unique | with -c, check for strict ordering; without -c, output only the first of an equal run | |||
-z, --zero-terminated | line delimiter is NUL, not newline |
list block devices (all, min, max):
~$ egrep -o "[0-9]* [sv]d[a-z]$" /proc/partitions | sort -k2 -u ~$ egrep -o "[0-9]* [sv}d[a-z]$" /proc/partitions | sort | head -n1 ~$ egrep -o "[0-9]* [sv}d[a-z]$" /proc/partitions | sort -r | head -n1
egrep head
sort only by the first letter of each line:
~$ sort -k1.1,1.1 input_f
list files and sort them by specific field (column):
~$ ls -l | sed -n '1!P' | sort -k5 -rn
sort IPv4 incrementally, extract only 10.20.30.0-255 IPs:
~$ zcat /var/log/syslog*gz | grep -oP "10\.20\.30\.[0-9]{1,3}"| sort | uniq | sort -t . -k 3,3n -k 4,4n
zcat uniq
⬆ concatenate and print files in reverse Written by Jay Lepreau and David MacKenzie.
tac [-brs]... [FILE]...
rev (1) tail
⬆ output the last part of file Written by Paul Rubin, David MacKenzie, Ian Lance Taylor, and Jim Meyering.
A tail command appeared in Version 7 AT&T UNIX.
cat (1) tac (1) head (1) sed (1)
⬆ read from standard input and write to standard output and files Written by Mike Parker, Richard M. Stallman, and David MacKenzie.
You can use multiple process substitutions (works with zsh, bash, ksh)
>(COMMAND)
to split pipe in multiple streams:
#!/usr/bin/env bash
echo “Command tee can split a pipe in many!” \
|tee >(rev) \
>(tr " " "_") \
>(wc ) \
>(tr "[:lower:]" "[:upper:]")download a file and validate it on the “fly”:
#!/usr/bin/env bash
# Used a real ISO image of Fedora 24 Workstation
FILE_URL="https://download.fedoraproject.org/pub/fedora/linux/releases/24/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-24-1.2.iso"
FILE_NAME="Fedora-Workstation-Live-x86_64-24-1.2.iso"
CHECKSUM_URL="https://getfedora.org/en_GB/static/checksums/Fedora-Workstation-24-x86_64-CHECKSUM"
CHECKSUM_FILE="FW24.sha245"
wget -O - "$FILE_URL" \
tee >(sha256sum > ./"$CHECKSUM_FILE") > "$FILE_NAME"test from shell script if remote TCP port is open:
~$ timeout $TIMEOUT_SECONDS bash -c "</dev/tcp/${HOST}/${PORT}"; echo $?
~$ timeout $TIMEOUT_SECONDS bash -c 'cat < /dev/null > /dev/tcp/${HOST}/${PORT}'; echo $?
⬆ translate or delete characters Written by Jim Meyering.
rename files in cwd from lower to UPPER case:
#!/usr/bin/env bash
for file in ./*; do
mv "$file" $(echo "$file" | tr '[:lower:]' '[:upper:]')
doneremove new lines from a file:
~$ tr -d '\n' < input_file > input_file_tm && mv input_file_tm input_file
windows files to UNIX format:
~$ tr -d '\15\32' < input_win_file.txt > output_unix_file
replace multiple characters sequence with a single one:
~$ tr -s \| = input_f
| OPTION | TYPE | V++ | V– | DESCRIPTION |
|---|---|---|---|---|
--all-repeated[=METHOD] | like -D, but allow separating groups | |||
--group[=METHOD] | show all items, separating groups with an empty line; | |||
--help | display this help and exit | |||
--version | output version information and exit | |||
-D | print all duplicate lines | |||
-c, --count | prefix lines by the number of occurrences | |||
-d, --repeated | only print duplicate lines, one for each group | |||
-f, --skip-fields=N | avoid comparing the first N fields | |||
-i, --ignore-case | ignore differences in case when comparing | |||
-s, --skip-chars=N | avoid comparing the first N characters | |||
-u, --unique | only print unique lines | |||
-w, --check-chars=N | compare no more than N characters in lines | |||
-z, --zero-terminated | line delimiter is NUL, not newline |
remove all dublicated lines from a input_f:
~$ sort input_f | uniq -u
- Jerry Peek, Shelley Powers, Tim O’Reilly, Mike Loukides. UNIX Power Tools. O’reilly
- Ellen Siever, Stepben Figgins, Robert Love & Arnold Robbins; Linux In a Nutshell, 6th edition; O’reilly 2005;
- Stephen G. Kochan, Patrick Wood; UNIX® Shell Programming, 3thd edition; Sams Publishing, 2003;