ross
Background stuff for people who want a refresher on basic linux knowledge:
Install everything:
sudo apt-get va
Start VM and become root user:
cd Users/Ross/linux/server1
vagrant up
vagrant ssh
sudo -s
List files installed by nginx. -L option specifies package names.
dpkg -L nginx
list all open internet files. I option specifies internet. Note that only root can see all files:
lsof -i
Port forwarding: open vagrantFile in Users/Ross/linux, uncomment port mapping command, type vagrant reload into terminal Allows host machine to access services on guest machine
Good supplemental reading on nginx config files:
How To Set Up Nginx Server Blocks
Check if nginx is listening to port 80 and edit server blocks to determine which static content is being displayed:
vim /etc/nginx/sites-available/default
Copy file first:
cp /etc/nginx/sites-available/default /etc/nginx/sites-available/myConfig
Open myConfig and find root directive in the file, change it from the default file to the html file you created. Remove default-server directives.
Go into sites-available directory and create a symbolic link to your config file, then delete the link to the default one:
ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
rm /etc/nginx/sites-enabled/default
-see kernel and GRUB setup. This file is generated by templates in /etc/grub.d and /etc/default/grub
/boot/grub.cfg
-Reads files from /etc/init in Ubuntu -By default reads from /etc/inittabfind
-Ubuntu alternative is update-rc.d
service - self documenting initctl - similar (see http://upstart.ubuntu.com/cookbook)
add a fake_service that runs on system start up
nano fake_service /etc/init.d/fake_service
chmod 755 /etc/init.d/fake_service
##File System
See all filesystems mounted on boot:
cat /lib/init/fstab
Modify which filesystems are mounted on boot:
nano /etc/fstab
- /bin - executable programs available to single user mode (/usr/bin/ contains more, multi-user executables)
- /dev - device nodes that interact with hardwar/software
- /etc configuration files
- /proc "virtual" fie system. Contains run-time and configuration items. Stored in memory
- /sbin booting and recovery binaries (fsck, init, reboot, update)
- /sys mount point for sysfs pseudo-filesystem, provides sysfs with info it needs to create device nodes
- /usr non-essential files sun-directory, need not be on same partition as root directory.
Link to description of different file types: File Types
##Kernel
Find out which command line a system was booted with:
cat /proc/cmdline
Kernel Config File:
/usr/src/linux-headers-3.13.0-93-generic/scripts/kernel-doc
All parameters reside on pseudofiles in /proc/sys, and boot configuration options are in file /etc/syscntl.conf
###Lab 6.2
Current max pid:
cat /proc/sys/kernel/pid_max
Change max pid:
sysctl -w kernel.pid_max=32000
##Kernel Modules 7.0
List modules: lsmod
Add/Remove modules: modprobe modprobe -r
Display information about a module: modinfo
##udev
Runs as a daemon. Listens for signal from uevent kernel facility. Then "takes appropriate action to create or remove device nodes according to the rules"
udevadmin utility used to manage udev /etc/udev/udev.conf is location of udev config file /etc/udev/rules.d contains rules for device naming
###udev rules
find / -name rules.d
/etc/udev/rules.d
/lib/udev/rules.d #most rules found here
/dev/.udev/rules.d #rules for root found here
###Device Nodes
##Partitions
Display disks and info about them:
blkid
lsblk
Backing up and restoring with dd:
dd if=/dev/sda of=mbrbackup bs=512 count=1
dd if=mbrbackup of=/dev/sda bs=512 count=1
###fdisk
fdisk /dev/sda
m-display menu p-list partition table n-create new partition d-delete partition t-change partition type w-write new partition table q-quit
another way to view partitions:
cat /proc/partitions
manage disk partitions non-interactively from command line with:
parted
###Lab Chapter 9 Creating, Mounting and Partitioning a FS
Create file of 0s:
dd if=/dev/zero of=imagefile bs=1M count=1024
Put a FS on it:
mkfs.ext4 imagefile
Mount it:
mkdir mntpoint
sudo mount -o loop imagefile mntpoint
Unmount:
sudo umount mntpoint
Partition interactively:
sudo fdisk -C 130 imagefile
Partition non-interactively:
check for loop devices
losetup -f
check which loop slots in use
losetup -a
associate file with a loop device
losetup /dev/loop1 imagefile
make a label on loop device (not sure about this, need more research)
parted -s /dev/loop1 mklabel msdos
make three partitions
parted -s /dev/loop1 unit MB mkpart primary ext4 0 256
parted -s /dev/loop1 unit MB mkpart primary ext4 256 512
parted -s /dev/loop1 unit MB mkpart primary ext4 512 1024
check partition table
fdisk -l /dev/loop1
##Encryption and Mounting FS on boot
cryptsetup ###used to encrypt
Encrypt a device using:
sudo cryptsetup luksFormat [device]
modify /etc/fstab
Store information about the file. File names are stored in the directory file. See inode info with:
ls -lai
"Files" exist in three places. Data blocks, which never move, inodes, which contain all the information about the file, and directories, which are basically just tables with links to files.
###chattr flags
-a append-only -i immutable -d no-dump -a no atime update
##Mounting and Unmounting
All filesystems mounted at boot are listed in /etc/fstab. At boot the command
mount -a /etc/fstab
is executed, mounting all filesystems.
#swap
check swaps at
cat /proc/swaps
check current memory usage with
free -o
Check filesystem information with
dumpe2fs [filesystem]
Change filesystem parameters
tune2fs
#LVM
Logical Volume Management - abstraction layer between OS and physical drives. Use cases include creating a volume that spans across physical drives. Advantage is that it is easy to resize and extents (the segments of a LVM) need not come from contiguous sections of a physical drive.
##Setting up a Logical Volume
-
Use VirtualBox to create two new physical disks
-
Format them with mkfs:
sudo mkfs -t ext4 /dev/sdc1 sudo mkfs -t ext4 /dev/sdd1
-
Create two primary partitions with fdisk
-
Now create a volume group:
sudo vgcreate vol1 /dev/sdc1 /dev/sdd1
#RAID - Redundant Array Independent Disks
writing same data to more than one disk
splitting data into more than one disk
extra data stored to allow problem detection and repair, fault tolerance
Manage Linux software RAID with:
mdadm
###RAID Levels
0 - Striping only
1 - Mirroring only
5 - Rotating parity stripe
6 - Like 5, but dual parity, requires 4 disks, but can stand the loss of two. More "important" than 5
10 - Mirrored and striped, 4 disks needed
#Security
##Physical Security Methods
###Set BIOS password - y
Instructions for Ubuntu systems here here:
###File system mounting options
nodev #prevents the creation of device nodes on this file system
nosuid #prevents setuid and setgid from working on filesystem
ro #read-only
##SELinux
Modes -
Enforcing - access denied according to policy
Permissive - access granted but all warnings given about access that would be denied if mode was Enforcing
Disabled - what it sounds like
Can change these settings by modifying:
/etc/selinux/config
Files, when created, inherit the security context of their parent directory. When moved, they often keep the context of their parent directory even in the new directory. This can be changed with the tool
restorecon
##AppArmor
#Processes
###Threads - Process flows. Some processes have multiple flows and are referred to as multi-threaded processes. Linux treats each "thread" as a process.
###Attributes
[] around a process indicates that it is run by the kernel
###Permissions
###Modes
user (limited privilege)
kernel (more privilege and more direct access to hardware)
##ulimit
ulimit -a #lists all current limits
Changes to ulimit effect current shell only. To change for all users, must modify
/etc/security/limits.conf
new process is forked from login shell
wait system call puts parent process to sleep
exec system call puts command in child process' memory space
command executes, child process dies
parent shell awakens
Unless: process is run in background using '&', then the parent does not wait, and is free to execute a new command
ldd [path of program] shows what shared libraries a program needs
ldconfig #runs at runtime and links all the shared libraries, must be run as root
##Most useful ps options:
ps -r #running processes
ps -u #user oriented
#System Monitoring
#Process Monitoring
-Three types of commands: UNIX (-before options), BSD (no - before options, GNU (-- before options)
-PS command uses /proc directory associated with each process to display data
ps auxf displays 'ancestry' of processes
#I/O Monitoring and Tuning
##iostat - monitor system i/o activity
iostat -N #displays device name in different format
iostat -x #extended report
###iostat extented fields:
##iotop - constantly updated table of current i/o usage
Options:
-o only processes actually performing i/o
-P only processes, as opposed to all threads
##ionice - set priority for processes
ionice [-c class] [-n level] [-t] -p PID...
ionice [-c class] [-n level] [-t] command [argument...]
###I/O Scheduling Classes
Idle - Cannot access disk I/O unless no other programs are using it after a defined period
Best Effort - [Default] Programs serviced one by one, based on priority
Real Time - Gets first access. Able to starve other processes. Priority determines amount of time it gets
##bonnie++
Benchmarks system performance
time sudo bonnie++ -n 0 -u 0 -r 100 -f -b -d /mnt
where:
• -n 0 means don’t perform the file creation tests.
• -u 0 means run as root.
• -r 100 means pretend you have 100 MB of RAM.
• -f means skip per character I/O tests.
• -b means do a fsync after every write, which forces flushing to disk rather than just writing to cache.
• -d /mnt just specifies the directory to place the temporary file created; make sure it has enough space, in this case 300 MB, available.
#I/O Scheduling
Find out whether disk is rotational or SSD:
cat /sys/block/[device]/queue/rotational
Check which scheduler is being used:
cat /sys/block/[device]/queue/scheduler
Change scheduler between noop/deadline/cfq:
echo [noop/deadline/cfq] > /sys/block/[device]/queue/scheduler #must be root
#Memory Monitoring Tools
###vmstat
Displays info about system memory usage, I/O, among other things
cat /proc/meminfo #displays detailed info about memory usage
#Package Management
###Types of Packages
Binary - contain executable files and libraries. Architecture dependent. Must be compiled for each type of machine.
Source - used to generate binary packages. One source package can be used for multiple architectures.
##Git
mkdir git-test
cd git-test
git init
echo some junk > somejunkfile
git add somejunkfile
git status
echo another line >> somejunkfile
git diff
git config user.name "someone"
git config user.email "[email protected]"
git commit -m "First Commit" --author="Ross <[email protected]>"
@dpkg
###Queries/Commands
dpkg -l #list all packages installed
dpkg -L [package] #list all files in the package
dpkg -p [package] #list info about the installed package
dpkg -I [.deb file] #show info about the package file
dpkg -c [.deb file] #list files in the package file
dpkg -V [package] #verify package integrity
dpkg -i [.deb file] #install or update package
dpkg -r [package] #remove package but not it's config files
dpkg -P [package] #remove package and it's config files (PURGE)
dpkg -s version currently installed
#apt
###Queries/Commands (apt-cache)
apt-cache search [package/search term] #search repositories for package
apt-cache show [package] #get basic information about package
apt-cache showpkg [package] #get more detailed information about package
apt-cache depends [package] #show package dependencies
###Queries/Commands (apt-get)
Sources of package repositories located in /etc/apt/sources.list
apt-get update #update
apt-get upgrade #apply updates to already installed packages
apt-get dist-upgrade #smart upgrade which removes obsolete packages and installs new dependencies
apt-get autoremove #clear cache and package files already installed
apt-get clean #save space
###/etc/passwd entry for users
vagrant:x:1000:1000::/home/vagrant:/bin/bash
username:userpassword:UID(User ID):GID(Group ID):comment:homedirectory:loginshell
###/etc/shadow entry for users
nobody:*:17022:0:99999:7:::
username:encryptedpassword:dayssince1970pwchanged:daysuntilchange:whenmustchange:7:::
/etc/login.defs #contains control definitions for the login package
##User Management Commands
Adds user, but administrator must then assign a password
sudo useradd [username]
Deletes a user, but leaves their home directory unless -r option is used. All their files will be left on the system regardless
sudo userdel [username]
Change characteristics of user accounts. Use -L to lock an account. Use -U to unlock.
sudo usermod [username]
Change password policies
chage [-m mindays] [-M maxdays] [-d lastday] chage -d 0 [username]
Restricted shell (rbash)
When creating a new user, specify their login shell as rbash to give them restricted access to the system
###SCP Secure Copy
Copy the file "foobar.txt" from a remote host to the local host
scp [email protected]:foobar.txt /some/local/directory
Copy the file "foobar.txt" from the local host to a remote host
scp foobar.txt [email protected]:/some/remote/directory
##SSH Configuration for use without password
##Group management
groupadd, groupdel, groupmod
usermod -G [group] [user]
This command will also remove them from groups not listed.
#File Permissions and Ownership
-rw-rw-r-- 1 vagrant vagrant 50 Oct 27 14:29 abc
- Type of file
- read/write/execute (owner/group/world)
- user name/group name
Permissions tested user>group>world
###Permissions octal shorthand
4 = read permission
2 = write permission
1 = execute permission
Thus, 7 for rwx, 6 rw-, 1 --x
###Access Control Lists
getfacl file|directory
-m : modify
-x : remove
sudo setfacl -m u:vagrant:r /home/sloan/chownfile
#allow to read file in Sloan's home directory without modifying permissions
sudo setfacl -x u:vagrant /home/sloan/chownfile
#remove access to file
#Backup and Recovery Methods
-Compress a directory or file:
tar -czvf name-of-archive.tar.gz /path/to/directory-or-file
-Extract: (-x instead of -c)
tar -xzvf archive.tar.gz
-List contents of a tar archive
tar -tf archive.tar.gz
-Extract a tar residing in one directory to another directory
tar -xC ~/restore -f backup/backup.tar.gz
###rsync
rsync [options] source destination
#Networking
ip [options] object {command | help}
Commands depend on what kind of object is selected*
address - ip address
link - network devices
maaddress - multicast address
monitor - watch for netlink messages
route - routing table entry
rule - rule in the routing policy databse
tunnel - tunnel over IP