Skip to content

Latest commit

 

History

History
7714 lines (5585 loc) · 243 KB

README.md

File metadata and controls

7714 lines (5585 loc) · 243 KB

ℹ️  This repo contains questions and exercises on various technical topics, sometimes related to DevOps and SRE

📊  There are currently 2292 exercises and questions

📚  To learn more about DevOps and SRE, check the resources in devops-resources repository

⚠️  You can use these for preparing for an interview but most of the questions and exercises don't represent an actual interview. Please read FAQ page for more details

👥  Join our DevOps community where we have discussions and share resources on DevOps

📝  You can add more exercises by submitting pull requests :) Read about contribution guidelines here


DevOps
DevOps
cicd
CI/CD
Git
Git
ansible
Ansible
Network
Network
Linux
Linux
programming
Software Development
Python
Python
go
Go
Bash
Shell Scripting
kubernetes
Kubernetes
Prometheus
Prometheus
Cloud
Cloud
aws
AWS
azure
Azure
Google Cloud Platform
Google Cloud Platform
openstack
OpenStack
security
Security
Operating System
Operating System
Monitoring
Monitoring
Elastic
Elastic
Virtualization
Virtualization
DNS
DNS
Misc
Misc
Testing
Testing
Databases
Databases
RegEx
Regex
Design
System Design
Hardware
Hardware
Big Data
Big Data
Certificates
Certificates
Containers
Containers
sql
SQL
OpenShift
OpenShift
Storage
Storage
HR
Soft Skills
Terraform
Terraform
Mongo
Mongo
puppet
Puppet
Distributed
Distributed
you
Questions you can ask
perl
Perl

Network

What do you need in order to communicate?
  • A common language (for the two ends to understand)
  • A way to address who do you want to communicate with
  • A Connection (so the content of of the communication can reach the recipients)
What is TCP/IP?

A set of protocols that define how two or more devices can communicate with each other. To learn more about TCP/IP, read here

What is APIPA?
APIPA is a set of it addresses that devices are allocated when the main DHCP server is not reachable
What ip range does APIPA use?
APIPA uses the ip range: 169.254.0.1 - 169.254.255.254.
What is Ethernet?

Ethernet simply refers to the most common type of Local Area Network (LAN) used today. A LAN—in contrast to a WAN (Wide Area Network), which spans a larger geographical area—is a connected network of computers in a small area, like your office, college campus, or even home.

What is a MAC address? What is it used for?

A MAC address is a unique identification number or code used to identify individual devices on the network.

Packets that are sent on the ethernet are always coming from a MAC address and sent to a MAC address. If a network adapter is receiving a packet, it is comparing the packet’s destination MAC address to the adapter’s own MAC address.

When is this MAC address used?: ff:ff:ff:ff:ff:ff

When a device sends a packet to the broadcast MAC address (FF:FF:FF:FF:FF:FF​), it is delivered to all stations on the local network. Ethernet broadcasts are used to resolve IP addresses to MAC addresses (by ARP) at the datalink layer .

What is an IP address?

An Internet Protocol address (IP address) is a numerical label assigned to each device connected to a computer network that uses the Internet Protocol for communication.An IP address serves two main functions: host or network interface identification and location addressing.

Explain subnet mask and given an example

A Subnet mask is a 32-bit number that masks an IP address, and divides the IP address into network address and host address. Subnet Mask is made by setting network bits to all "1"s and setting host bits to all "0"s. Within a given network, two host addresses are reserved for special purpose, and cannot be assigned to hosts. The "0" address is assigned a network address and "255" is assigned to a broadcast address, and they cannot be assigned to hosts.

For Example

| Address Class | No of Network Bits | No of Host Bits | Subnet mask     | CIDR notation |
| ------------- | ------------------ | --------------- | --------------- | ------------- |
| A             | 8                  | 24              | 255.0.0.0       | /8            |
| A             | 9                  | 23              | 255.128.0.0     | /9            |
| A             | 12                 | 20              | 255.240.0.0     | /12           |
| A             | 14                 | 18              | 255.252.0.0     | /14           |
| B             | 16                 | 16              | 255.255.0.0     | /16           |
| B             | 17                 | 15              | 255.255.128.0   | /17           |
| B             | 20                 | 12              | 255.255.240.0   | /20           |
| B             | 22                 | 10              | 255.255.252.0   | /22           |
| C             | 24                 | 8               | 255.255.255.0   | /24           |
| C             | 25                 | 7               | 255.255.255.128 | /25           |
| C             | 28                 | 4               | 255.255.255.240 | /28           |
| C             | 30                 | 2               | 255.255.255.252 | /30           |

What is a private IP address? In which scenarios/system designs, one should use it?
Private IP addresses are assigned to the hosts in the same network to communicate among one another. As the name "private" suggests, the devices having the priate IP addresses assigned can't be reached by the devices from any external network. For example, if I am living in a hostel and I want my hostelmates to join the game server I have hosted, I will ask them to join via my server's private IP address, since the network is local to the hostel.
What is a public IP address? In which scenarios/system designs, one should use it?
A public IP address is the public facing IP address. In the event that you was hosting a game server that you want your friends to join, you will give your friends your public IP address to allow their computers to identify and locate your network and server in order for the connection to take place. One time that you would not need to use a public facing IP address is in the event that you was playing with friends who was connected to the same network as you, in that case, you would use a private ip address. In order for someone to be able to connect to your server that is located internally, you will have to setup a port forward to tell your router to allow traffic from the public domain into your network and vice versa.
Explain the OSI model. What layers there are? What each layer is responsible for?
  • Application: user end (HTTP is here)
  • Presentation: establishes context between application-layer entities (Encryption is here)
  • Session: establishes, manages and terminates the connections
  • Transport: transfers variable-length data sequences from a source to a destination host (TCP & UDP are here)
  • Network: transfers datagrams from one network to another (IP is here)
  • Data link: provides a link between two directly connected nodes (MAC is here)
  • Physical: the electrical and physical spec the data connection (Bits are here)

You can read more about the OSI model in penguintutor.com

For each of the following determine to which OSI layer it belongs:
  • Error correction
  • Packets routing
  • Cables and electrical signals
  • MAC address
  • IP address
  • Terminate connections
  • 3 way handshake

  • Error correction - Data link
  • Packets routing - Network
  • Cables and electrical signals - Physical
  • MAC address - Data link
  • IP address - Network
  • Terminate connections - Session
  • 3 way handshake - Transport
  • What delivery schemes are you familiar with?

    Unitcast: One to one communication where there is one sender and one receiver.

    Broadcast: Sending a message to everyone in the network. The address ff:ff:ff:ff:ff:ff is used for broadcasting. Two common protocols which use broadcast are ARP and DHCP.

    Multicast: Sending a message to a group of subscribers. It can be one-to-many or many-to-many.

    What is CSMA/CD? Is it used in modern ethernet networks?

    CSMA/CD stands for Carrier Sense Multiple Access / Collision Detection. Its primarily focus it to manage access to shared medium/bus where only one host can transmit at a given point of time.

    CSMA/CD algorithm:

    1. Before sending a frame, it checks whether another host already transmitting a frame.
    2. If no one transmitting, it starts transmitting the frame.
    3. If two hosts transmitted at the same time, we have a collision.
    4. Both hosts stop sending the frame and they send to everyone a 'jam signal' notifying everyone that a collision occurred
    5. They are waiting for a random time before sending again
    6. Once each host waited for a random time, they try to send the frame again and so the
    Describe the following network devices and the difference between them:
    • router
    • switch
    • hub

    How does a router works?

    A router is a physical or virtual appliance that passes information between two or more packet-switched computer networks. A router inspects a given data packet's destination Internet Protocol address (IP address), calculates the best way for it to reach its destination and then forwards it accordingly.

    What is NAT?

    Network Address Translation (NAT) is a process in which one or more local IP address is translated into one or more Global IP address and vice versa in order to provide Internet access to the local hosts.

    What is a proxy? How does it works? What do we need it for?

    A proxy server acts as a gateway between you and the internet. It’s an intermediary server separating end users from the websites they browse.

    If you’re using a proxy server, internet traffic flows through the proxy server on its way to the address you requested. The request then comes back through that same proxy server (there are exceptions to this rule), and then the proxy server forwards the data received from the website to you.

    roxy servers provide varying levels of functionality, security, and privacy depending on your use case, needs, or company policy.

    What is TCP? How does it works? What is the 3 way handshake?

    TCP 3-way handshake or three-way handshake is a process which is used in a TCP/IP network to make a connection between server and client.

    A three-way handshake is primarily used to create a TCP socket connection. It works when:

    • A client node sends a SYN data packet over an IP network to a server on the same or an external network. The objective of this packet is to ask/infer if the server is open for new connections.
    • The target server must have open ports that can accept and initiate new connections. When the server receives the SYN packet from the client node, it responds and returns a confirmation receipt – the ACK packet or SYN/ACK packet.
    • The client node receives the SYN/ACK from the server and responds with an ACK packet.
    What is round-trip delay or round-trip time?

    From wikipedia: "the length of time it takes for a signal to be sent plus the length of time it takes for an acknowledgement of that signal to be received"

    Bonus question: what is the RTT of LAN?

    How does SSL handshake work?
    What is the difference between TCP and UDP?

    TCP establishes a connection between the client and the server to guarantee the order of the packages, on the other hand, UDP does not establish a connection between client and server and doesn't handle package order. This makes UDP more lightweight than TCP and a perfect candidate for services like streaming.

    Penguintutor.com provides a good explanation.

    What TCP/IP protocols are you familiar with?
    Explain "default gateway"

    A default gateway serves as an access point or IP router that a networked computer uses to send information to a computer in another network or the internet.

    What is ARP? How does it works?

    ARP stands for Address Resolution Protocol. When you try to ping an IP address on your local network, say 192.168.1.1, your system has to turn the IP address 192.168.1.1 into a MAC address. This involves using ARP to resolve the address, hence its name.

    Systems keep an ARP look-up table where they store information about what IP addresses are associated with what MAC addresses. When trying to send a packet to an IP address, the system will first consult this table to see if it already knows the MAC address. If there is a value cached, ARP is not used.

    What is TTL? What does it helps to prevent?
    What is DHCP? How does it works?

    It stands for Dynamic Host Configuration Protocol, and allocates IP addresses, subnet masks and gateways to hosts. This is how it works:

    • A host upon entering a network, broadcasts a message in search of a DHCP server (DHCP DISCOVER)
    • An offer message is sent back by the DHCP server as a packet containing lease time, subnet mask, IP addresses, etc (DHCP OFFER)
    • Depending on which offer accepted, the client sends back a reply broadcast letting all DHCP servers know (DHCP REQUEST)
    • Server sends an acknowledgment (DHCP ACK)

    Read more here

    Can you have two DHCP servers in the same network? How it works?
    What is SSL tunneling? How does it works?
    What is a socket? Where can you see the list of sockets in your system?
    What is IPv6? Why should we consider using it if we have IPv4?
    What is VLAN?
    What is MTU?
    What happens if you send a packet that is bigger than the MTU?
    True or False?. Ping is using UDP because it doesn't care about reliable connection
    What is SDN?
    What is ICMP? What is it used for?
    What is NAT? How does it work?

    NAT stands for network address translation. It’s a way to map multiple local private addresses to a public one before transferring the information. Organizations that want multiple devices to employ a single IP address use NAT, as do most home routers. For example, your computer's private IP could be 192.168.1.100, but your router maps the traffic to it's public IP (e.g. 1.1.1.1). Any device on the internet would see the traffic coming from your public IP (1.1.1.1) instead of your private IP (192.168.1.100).

    Which factors affect network performances
    Which port number is used in each of the following protocols?:
    • SSH
    • SMTP
    • HTTP
    • DNS
    • HTTPS
    • FTP
    • SFTP

    • SSH - 22
    • SMTP - 25
    • HTTP - 80
    • DNS - 53
    • HTTPS - 443
    • FTP - 21
    • SFTP - 22

    Network - Data and Control planes

    What "control plane" refers to?

    The control plane is the part of the network that decides how to route and forward packets to a different location.

    What "data plane" refers to?

    The data plane is the part of the network that actually forwards the data/packets.

    What "management plane" refers to?

    Refers to monitoring and management functions.

    To which plane (data, control, ...) is creating routing tables belongs to?

    Control Plane.

    Explain Spanning Tree Protocol (STP)
    What is link aggregation? Why is it used?
    What is Asymmetric Routing? How do deal with it?
    What overlay (tunnel) protocols are you familiar with?
    What is GRE? How does it works?
    What is VXLAN? How does it works?
    What is SNAT?
    Explain OSPF
    What is latency?
    What is bandwidth?
    What is throughput?
    When performing a search query, what is more important, latency or throughput? And how to assure that what managing global infrastructure?

    Latency. To have a good latency, a search query should be forwarded to the closest datacenter.

    When uploading a video, what is more important, latency or throughput? And how to assure that?

    Throughput. To have a good throughput, the upload stream should be routed to an underutilized link.

    What other considerations (except latency and throughput) are there when forwarding requests?
    • Keep caches updated (which means the request could be forwarded not to the closest datacenter)
    Explain Spine & Leaf
    What is Network Congestion? What can cause it?
    What can you tell me about UDP packet format? What about TCP packet format? How is it different?
    What is the exponential backoff algorithm? Where is it used?
    Using Hamming code, what would be the code word for the following data word 100111010001101?

    00110011110100011101

    Give examples of protocols found in the application layer
    • Hypertext Transfer Protocol (HTTP) - used for the webpages on the internet
    • Simple Mail Transfer Protocol (SMTP) - email transmission
    • Telecommunications Network - (TELNET) - terminal emulation to allow client access to telnet server
    • File Transfer Protocol (FTP) - facilitates transfer of files between any two machines
    • Domain Name System (DNS) - domain name translation
    • Dynamic Host Configuration Protocol (DHCP) - allocates IP addresses, subnet masks and gateways to hosts
    • Simple Network Management Protocol (SNMP) - gathers data of devices on the network
    Give examples of protocols found in the network Layer
    • Internet Protocol (IP) - assists in routing packets from one machine to another
    • Internet Control Message Protocol (ICMP) - lets one know what is going such as error messages and debugging information
    What is HSTS?
    HTTP Strict Transport Security is a web server directive that informs user agents and web browsers how to handle its connection through a response header sent at the very beginning and back to the browser. This forces connections over HTTPS encryption, disregarding any script's call to load any resource in that domain over HTTP.

    Read more [here](https://www.globalsign.com/en/blog/what-is-hsts-and-how-do-i-use-it#:~:text=HTTP%20Strict%20Transport%20Security%20(HSTS,and%20back%20to%20the%20browser.)

    Network - Misc

    What is the Internet? Is it the same as the World Wide Web?

    The internet refers to network of networks, transferring huge amounts of data around the globe.
    The World Wide Web is an application running on millions of server, on top of the internet, accessed through what is know as the web browser

    What is the ISP?

    ISP (Internet Service Provider) is the local internet company provider.

    Linux

    Linux Master Application

    A completely free application for testing your knowledge on Linux

    Linux Self Assessment

    What is your experience with Linux?

    Only you know :)

    For example:

    • Administration
    • Troubleshooting & Debugging
    • Storage
    • Networking
    • Development
    • Deployments
    Explain what each of the following commands does and give an example on how to use it:
    • touch
    • ls
    • rm
    • cat
    • cp
    • mkdir
    • pwd
    • cd

    • touch - update file's timestamp. More commonly used for creating files
    • ls - listing files and directories
    • rm - remove files and directories
    • cat - create, view and concatenate files
    • cp - copy files and directories
    • mkdir - create directories
    • pwd - print current working directory (= at what path the user currently located)
    • cd - change directory
    What each of the following commands does?
    • cd /
    • cd ~
    • cd
    • cd ..
    • cd .
    • cd -

    • cd / -> change to the root directory
    • cd ~ -> change to your home directory
    • cd -> change to your home directory
    • cd .. -> change to the directory above your current i.e parent directory
    • cd . -> change to the directory you currently in
    • cd - -> change to the last visited path
    Some of the commands in the previous question can be run with the -r/-R flag. What does it do? Give an example to when you would use it

    The -r (or -R in some commands) flag allows the user to run a certain command recursively. For example, listing all the files under the following tree is possible when done recursively (ls -R):

    /dir1/ dir2/ file1 file2 dir3/ file3

    To list all the files, one can run ls -R /dir1

    Explain each field in the output of `ls -l` command
    It shows a detailed list of files in a long format. From the left:
    • file permissions, number of links, owner name, owner group, file size, timestamp of last modification and directory/file name
    What are hidden files/directories? How to list them?

    These are files directly not displayed after performing a standard ls direct listing. An example of these files are .bashrc which are used to execute some scripts. Some also store configuration about services on your host like .KUBECONFIG. The command used to list them is, ls -a

    What do > and < do in terms of input and output for programs?
    They take in input (<) and output for a given file (>) using stdin and stdout.

    myProgram < input.txt > executionOutput.txt

    Explain what each of the following commands does and give an example on how to use it:
    • sed
    • grep
    • cut
    • awk

    • sed: a stream editor. Can be used for various purposes like replacing a word in a file: sed -i s/salad/burger/g
    • grep: a search tool. Used to search, count or match a text in a file:
      • searching for any line that contains a word in a file: grep 'word' file.md
      • or displaying the total number of times a string appears in a file: grep -c 'This is a string' file.md
    • cut: a tool for cutting out selected portions of each line of a file:
      • syntax: cut OPTION [FILE]
        • cutting first two bytes from a word in a file: cut -b 1-2 file.md, output: wo
    How to rename the name of a file or a directory?

    Using the mv command.

    Specify which command would you use (and how) for each of the following scenarios
    • Remove a directory with files
    • Display the content of a file
    • Provides access to the file /tmp/x for everyone
    • Change working directory to user home directory
    • Replace every occurrence of the word "good" with "great" in the file /tmp/y

    • rm -rf dir
    • cat or less
    • chmod 777 /tmp/x
    • cd ~
    • sed -i s/good/great/g /tmp/y
    How can you check what is the path of a certain command?
    • whereis
    • which
    What is the difference between these two commands? Will it result in the same output?
    echo hello world
    echo "hello world"
    

    The echo command receives two separate arguments in the first execution and in the second execution it gets one argument which is the string "hello world". The output will be the same.

    Explain piping. How do you perform piping?

    Using a pipe in Linux, allows you to send the output of one command to the input of another command. For example: cat /etc/services | wc -l

    Fix the following commands:
    • sed "s/1/2/g' /tmp/myFile
    • find . -iname *.yaml -exec sed -i "s/1/2/g" {} ;

    sed 's/1/2/g' /tmp/myFile  # sed "s/1/2/g" is also fine
    find . -iname "*.yaml" -exec sed -i "s/1/2/g" {} \;
    

    How to check which commands you executed in the past?

    history command or .bash_history file

    Running the command df you get "command not found". What could be wrong and how to fix it?

    Most likely the default/generated $PATH was somehow modified or overridden thus not containing /bin/ where df would normally go. This issue could also happen if bash_profile or any configuration file of your interpreter was wrongly modified, causing erratics behaviours. You would solve this by fixing your $PATH variable:

    As to fix it there are several options:

    1. Manually adding what you need to your $PATH PATH="$PATH":/user/bin:/..etc
    2. You have your weird env variables backed up.
    3. You would look for your distro default $PATH variable, copy paste using method #1

    Note: There are many ways of getting errors like this: if bash_profile or any configuration file of your interpreter was wrongly modified; causing erratics behaviours, permissions issues, bad compiled software (if you compiled it by yourself)... there is no answer that will be true 100% of the time.

    How do you schedule tasks periodically?

    You can use the commands cron and at. With cron, tasks are scheduled using the following format:

    */30 * * * * bash myscript.sh Executes the script every 30 minutes.

    The tasks are stored in a cron file, you can write in it using crontab -e

    Alternatively if you are using a distro with systemd it's recommended to use systemd timers.

    Linux - I/O Redirection

    Explain Linux I/O redirection
    Demonstrate Linux output redirection

    ls > ls_output.txt

    Demonstrate Linux stderr output redirection

    yippiekaiyay 2> ls_output.txt

    Demonstrate Linux stderr to stdout redirection

    yippiekaiyay 1>&2

    What is the result of running the following command? yippiekaiyay 1>&2 die_hard

    An output similar to: yippikaiyay: command not found...
    The file die_hard will not be created

    Linux FHS

    In Linux FHS (Filesystem Hierarchy Standard) what is the /?

    The root of the filesystem. The beginning of the tree.

    What is stored in each of the following paths?
    • /bin, /sbin, /usr/bin and /usr/sbin
    • /etc
    • /home
    • /var
    • /tmp

    • binaries
    • configuration files
    • home directories of the different users
    • files that tend to change and be modified like logs
    • temporary files
    What is special about the /tmp directory when compared to other directories?

    /tmp folder is cleaned automatically, usually upon reboot.

    What kind of information one can find in /proc?
    What makes /proc different from other filesystems?
    True or False? only root can create files in /proc

    False. No one can create file in /proc directly (certain operations can lead to files being created in /proc by the kernel).

    What can be found in /proc/cmdline?

    The command passed to the boot loader to run the kernel

    In which path can you find the system devices (e.g. block storage)?

    Linux - Permissions

    How to change the permissions of a file?

    Using the chmod command.

    What does the following permissions mean?:
    • 777
    • 644
    • 750

    777 - You give the owner, group and other: Execute (1), Write (2) and Read (4); 4+2+1 = 7.
    644 - Owner has Read (4), Write (2), 4+2 = 6; Group and Other have Read (4).
    750 - Owner has x+r+w, Group has Read (4) and Execute (1); 4+1 = 5. Other have no permissions.
    

    What this command does? chmod +x some_file
    It adds execute permissions to all sets i.e user, group and others
    Explain what is setgid and setuid
    • setuid is a linux file permission that permits a user to run a file or program with the permissions of the owner of that file. This is possible by elevation of current user privileges.
    • setgid is a process when executed will run as the group that owns the file.
    What is the purpose of sticky bit?
    Its a bit that only allows the owner or the root user to delete or modify the file.
    What the following commands do?
    • chmod
    • chown
    • chgrp

    • chmod - changes access permissions to files system objects
    • chown - changes the owner of file system files and directories
    • chgrp - changes the group associated with a file system object
    What is sudo? How do you set it up?
    True or False? In order to install packages on the system one must be the root user or use the sudo command

    True

    Explain what are ACLs. For what use cases would you recommend to use them?
    You try to create a file but it fails. Name at least three different reason as to why it could happen
    • No more disk space
    • No more inodes
    • No permissions
    A user accidentally executed the following chmod -x $(which chmod). How to fix it?

    Linux - systemd

    What is systemd?
    Systemd is a daemon (System 'd', d stands for daemon).

    A daemon is a program that runs in the background without direct control of the user, although the user can at any time talk to the daemon.

    systemd has many features such as user processes control/tracking, snapshot support, inhibitor locks..

    If we visualize the unix/linux system in layers, systemd would fall directly after the linux kernel.
    Hardware -> Kernel -> Daemons, System Libraries, Server Display.

    How to start or stop a service?

    To start a service: systemctl start <service name> To stop a service: systemctl stop <service name>

    How to check the status of a service?

    systemctl status <service name>

    On a system which uses systemd, how would you display the logs?

    journalctl

    Describe how to make a certain process/app a service
    Linux - Troubleshooting & Debugging
    Where system logs are located?

    /var/log

    How to follow file's content as it being appended without opening the file every time?

    tail -f <file_name>

    What are you using for troubleshooting and debugging network issues?

    dstat -t is great for identifying network and disk issues. netstat -tnlaup can be used to see which processes are running on which ports. lsof -i -P can be used for the same purpose as netstat. ngrep -d any metafilter for matching regex against payloads of packets. tcpdump for capturing packets wireshark same concept as tcpdump but with GUI (optional).

    What are you using for troubleshooting and debugging disk & file system issues?

    dstat -t is great for identifying network and disk issues. opensnoop can be used to see which files are being opened on the system (in real time).

    What are you using for troubleshooting and debugging process issues?

    strace is great for understanding what your program does. It prints every system call your program executed.

    What are you using for debugging CPU related issues?

    top will show you how much CPU percentage each process consumes perf is a great choice for sampling profiler and in general, figuring out what your CPU cycles are "wasted" on flamegraphs is great for CPU consumption visualization (http://www.brendangregg.com/flamegraphs.html)

    You get a call from someone claiming "my system is SLOW". What do you do?
    • Check with top for anything unusual
    • Run dstat -t to check if it's related to disk or network.
    • Check if it's network related with sar
    • Check I/O stats with iostat
    Explain iostat output
    How to debug binaries?
    What is the difference between CPU load and utilization?
    How you measure time execution of a program?

    Linux - Kernel

    What is a kernel, and what does it do?

    The kernel is part of the operating system and is responsible for tasks like:

    • Allocating memory
    • Schedule processes
    • Control CPU
    How do you find out which Kernel version your system is using?

    uname -a command

    What is a Linux kernel module and how do you load a new module?
    Explain user space vs. kernel space

    The operating system executes the kernel in protected memory to prevent anyone from changing (and risking it crashing). This is what is known as "Kernel space". "User space" is where users executes their commands or applications. It's important to create this separation since we can't rely on user applications to not tamper with the kernel, causing it to crash.

    Applications can access system resources and indirectly the kernel space by making what is called "system calls".

    In what phases of kernel lifecycle, can you change its configuration?
    • Build time (when it's compiled)
    • Boot time (when it starts)
    • Runtime (once it's already running)
    Where can you find kernel's configuration?

    Usually it will reside in /boot/config-<kernel version>.<os release>.<arch>

    Where can you find the file that contains the command passed to the boot loader to run the kernel?

    /proc/cmdline

    How to list kernel's runtime parameters?

    sysctl -a

    Will running sysctl -a as a regular user vs. root, produce different result?

    Yes, you might notice that in most systems, when running systctl -a with root, you'll get more runtime parameters compared to executing the same command with a regular user.

    You would like to enable IPv4 forwarding in the kernel, how would you do it?

    sudo sysctl net.ipv4.ip_forward=1

    To make it persistent (applied after reboot for example): insert net.ipv4.ip_forward = 1 into /etc/sysctl.conf

    Another way to is to run echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward

    How sysctl applies the changes to kernel's runtime parameters the moment you run sysctl command?

    If you strace the sysctl command you can see it does it by changing the file under /proc/sys/...

    In the past it was done with sysctl system call, but it was deprecated at some point.

    How changes to kernel runtime parameters persist? (applied even after reboot to the system for example)

    There is a service called systemd-sysctl that takes the content of /etc/sysctl.conf and applies it. This is how changes persist, even after reboot, when they are written in /etc/sysctl.conf

    Are the changes you make to kernel parameters in a container, affects also the kernel parameters of the host on which the container runs?

    No. Containers have their own /proc filesystem so any change to kernel parameters inside a container, are not affecting the host or other containers running on that host.

    Linux - SSH

    What is SSH? How to check if a Linux server is running SSH?

    Wikipedia Definition: "SSH or Secure Shell is a cryptographic network protocol for operating network services securely over an unsecured network."

    Hostinger.com Definition: "SSH, or Secure Shell, is a remote administration protocol that allows users to control and modify their remote servers over the Internet."

    An SSH server will have SSH daemon running. Depends on the distribution, you should be able to check whether the service is running (e.g. systemctl status sshd).

    Why SSH is considered better than telnet?

    Telnet also allows you to connect to a remote host but as opposed to SSH where the communication is encrypted, in telnet, the data is sent in clear text, so it doesn't considered to be secured because anyone on the network can see what exactly is sent, including passwords.

    What is stored in ~/.ssh/known_hosts?
    You try to ssh to a server and you get "Host key verification failed". What does it mean?

    It means that the key of the remote host was changed and doesn't match the one that stored on the machine (in ~/.ssh/known_hosts).

    What is the difference between SSH and SSL?
    What ssh-keygen is used for?

    ssh-keygen is a tool to generate an authentication key pair for SSH, that consists of a private and a public key. It supports a number of algorithms to generate authentication keys :

    • dsa
    • ecdsa
    • ecdsa-sk
    • ed25519
    • ed25519-sk
    • rsa (default)

    One can also specify number of bits in key. Command below generates an SSH key pair with RSA 4096-bits :

    $ ssh-keygen -t rsa -b 4096
    

    The output looks like this:

    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/user/.ssh/id_rsa): 
    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again: 
    Your identification has been saved in /home/user/.ssh/id_rsa
    Your public key has been saved in /home/user/.ssh/id_rsa.pub
    The key fingerprint is:
    SHA256:f5MOGnhzYfC0ZCHvbSXXiRiNVYETjxpHcXD5xSojx+M user@mac-book-pro
    The key's randomart image is:
    +---[RSA 4096]----+
    |        . ..+***o|
    |         o o++*o+|
    |        . =+.++++|
    |         B.oX+. .|
    |        S *=o+   |
    |       . o oE.   |
    |      . + + +    |
    |       . = + .   |
    |        .   .    |
    +----[SHA256]-----+
    

    One can check how many bits an SSH key has with :

    $ ssh-keygen -l -f /home/user/.ssh/id_rsa
    

    Output should look like this :

    4096 SHA256:f5MOGnhzYfC0ZCHvbSXXiRiNVYETjxpHcXD5xSojx+M user@mac-book-pro (RSA)
    

    It shows the key is RSA 4096-bits.

    -l and -f parameters usage explanation :

    -l          Show the fingerprint of the key file.
    -f filename Filename of the key file.
    

    Learn more : How can I tell how many bits my ssh key is? - Superuser

    What is SSH port forwarding?

    Linux - Globbing, Wildcards

    What is Globbing?
    What are wildcards? Can you give an example of how to use them?
    Explain what will ls [XYZ] match
    Explain what will ls [^XYZ] match
    Explain what will ls [0-5] match
    What each of the following matches
    • ?
    • *

    • The ? matches any single character
    • The * matches zero or more characters
    What do we grep for in each of the following commands?:
    • grep '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}' some_file
    • grep -E "error|failure" some_file
    • grep '[0-9]$' some_file

    1. An IP address
    2. The word "error" or "failure"
    3. Lines which end with a number
    Which line numbers will be printed when running `grep '\baaa\b'` on the following content:

    aaa bbb ccc.aaa aaaaaa


    lines 1 and 3.

    What is the difference single and double quotes?
    What is escaping? What escape character is used for escaping?
    What is an exit code? What exit codes are you familiar with?

    An exit code (or return code) represents the code returned by a child process to its parent process.

    0 is an exit code which represents success while anything higher than 1 represents error. Each number has different meaning, based on how the application was developed.

    I consider this as a good blog post to read more about it: https://shapeshed.com/unix-exit-codes

    Linux Boot Process

    Tell me everything you know about the Linux boot process

    Another way to ask this: what happens from the moment you turned on the server until you get a prompt

    What is GRUB2?
    What is Secure Boot?
    What can you find in /boot?
    Linux Disk & Filesystem
    What's an inode?

    For each file (and directory) in Linux there is an inode, a data structure which stores meta data related to the file like its size, owner, permissions, etc.

    Which of the following is not included in inode:
    • Link count
    • File size
    • File name
    • File timestamp

    File name (it's part of the directory file)

    How to check which disks are currently mounted?

    Run mount

    You run the mount command but you get no output. How would you check what mounts you have on your system?

    cat /proc/mounts

    What is the difference between a soft link and hard link?

    Hard link is the same file, using the same inode. Soft link is a shortcut to another file, using a different inode.

    True or False? You can create an hard link for a directory

    False

    True or False? You can create a soft link between different filesystems

    True

    True or False? Directories always have by minimum 2 links

    True.

    What happens when you delete the original file in case of soft link and hard link?
    Can you check what type of filesystem is used in /home?

    There are many answers for this question. One way is running df -T

    What is a swap partition? What is it used for?
    How to create a
    • new empty file
    • a file with text (without using text editor)
    • a file with given size

    You are trying to create a new file but you get "File system is full". You check with df for free space and you see you used only 20% of the space. What could be the problem?
    How would you check what is the size of a certain directory?

    du -sh

    What is LVM?
    Explain the following in regards to LVM:
    • PV
    • VG
    • LV

    What is NFS? What is it used for?
    What RAID is used for? Can you explain the differences between RAID 0, 1, 5 and 10?
    Describe the process of extending a filesystem disk space
    What is lazy umount?
    What is tmpfs?
    What is stored in each of the following logs?
    • /var/log/messages
    • /var/log/boot.log

    True or False? both /tmp and /var/tmp cleared upon system boot

    False. /tmp is cleared upon system boot while /var/tmp is cleared every a couple of days or not cleared at all (depends on distro).

    Linux Performance Analysis

    How to check what is the current load average?

    One can use uptime or top

    You know how to see the load average, great. but what each part of it means? for example 1.43, 2.34, 2.78

    This article summarizes the load average topic in a great way

    How to check process usage?

    pidstat

    How to check disk I/O?

    iostat -xz 1

    How to check how much free memory a system has? How to check memory consumption by each process?

    You can use the commands top and free

    How to check TCP stats?

    sar -n TCP,ETCP 1

    Linux Processes

    how to list all the processes running in your system?

    ps -ef

    How to run a process in the background and why to do that in the first place?

    You can achieve that by specifying & at the end of the command. As to why, since some commands/processes can take a lot of time to finish execution or run forever, you may want to run them in the background instead of waiting for them to finish before gaining control again in current session.

    How can you find how much memory a specific process consumes?
    mem() { ps -eo rss,pid,euser,args:100 --sort %mem | grep -v grep | grep -i $@ | awk '{printf $1/1024 "MB"; $1=""; print }' } [Source](https://stackoverflow.com/questions/3853655/in-linux-how-to-tell-how-much-memory-processes-are-using)
    What signal is used by default when you run 'kill *process id*'?
    The default signal is SIGTERM (15). This signal kills
    process gracefully which means it allows it to save current
    state configuration.
    
    What signals are you familiar with?

    SIGTERM - default signal for terminating a process SIGHUP - common usage is for reloading configuration SIGKILL - a signal which cannot caught or ignored

    To view all available signals run kill -l

    What kill 0 does?
    What kill -0 does?
    What is a trap?
    Every couple of days, a certain process stops running. How can you look into why it's happening?
    What happens when you press ctrl + c?
    What is a Daemon in Linux?

    A background process. Most of these processes are waiting for requests or set of conditions to be met before actually running anything. Some examples: sshd, crond, rpcbind.

    What are the possible states of a process in Linux?
    Running (R)
    Uninterruptible Sleep (D) - The process is waiting for I/O
    Interruptible Sleep (S)
    Stopped (T)
    Dead (x)
    Zombie (z)
    
    How do you kill a process in D state?
    What is a zombie process?

    A process which has finished to run but has not exited.

    One reason it happens is when a parent process is programmed incorrectly. Every parent process should execute wait() to get the exit code from the child process which finished to run. But when the parent isn't checking for the child exit code, the child process can still exists although it finished to run.

    How to get rid of zombie processes?

    You can't kill a zombie process the regular way with kill -9 for example as it's already dead.

    One way to kill zombie process is by sending SIGCHLD to the parent process telling it to terminate its child processes. This might not work if the parent process wasn't programmed properly. The invocation is kill -s SIGCHLD [parent_pid]

    You can also try closing/terminating the parent process. This will make the zombie process a child of init (1) which does periodic cleanups and will at some point clean up the zombie process.

    How to find all the
    • Processes executed/owned by a certain user
    • Process which are Java processes
    • Zombie Processes

    If you mention at any point ps command with arugments, be familiar with what these arguments does exactly.

    What is the init process?
    It is the first process executed by the kernel during the booting of a system. It is a daemon process which runs till the system is shutdown. That is why, it is the parent of all the processes
    Can you describe how processes are being created?
    How to change the priority of a process? Why would you want to do that?
    Can you explain how network process/connection is established and how it's terminated?>
    What strace does? What about ltrace?
    Find all the files which end with '.yml' and replace the number 1 in 2 in each file

    find /some_dir -iname *.yml -print0 | xargs -0 -r sed -i "s/1/2/g"

    You run ls and you get "/lib/ld-linux-armhf.so.3 no such file or directory". What is the problem?

    The ls executable is built for an incompatible architecture.

    How would you split a 50 lines file into 2 files of 25 lines each?

    You can use the split command this way: split -l 25 some_file

    What is a file descriptor? What file descriptors are you familiar with?
    Kerberos File descriptor, also known as file handler, is a unique number which identifies an open file in the operating system.

    In Linux (and Unix) the first three file descriptors are:

    • 0 - the default data stream for input
    • 1 - the default data stream for output
    • 2 - the default data stream for output related to errors

    This is a great article on the topic: https://www.computerhope.com/jargon/f/file-descriptor.htm

    What is NTP? What is it used for?
    Explain Kernel OOM
    Linux Security
    What is chroot? In what scenarios would you consider using it?
    What is SELiunx?
    What is Kerberos?
    What is nftables?
    What firewalld daemon is responsible for?
    Do you have experience with hardening servers? Can you describe the process?
    Linux - Networking
    How to list all the interfaces?
    ip link show
    

    What is the loopback (lo) interface?

    The loopback interface is a special, virtual network interface that your computer uses to communicate with itself. It is used mainly for diagnostics and troubleshooting, and to connect to servers running on the local machine.

    What the following commands are used for?
    • ip addr
    • ip route
    • ip link
    • ping
    • netstat
    • traceroute

    What is a network namespace? What is it used for?
    How to check if a certain port is being used?

    One of the following would work:

    netstat -tnlp | grep <port_number>
    lsof -i -n -P | grep <port_number>
    

    How can you turn your Linux server into a router?
    What is a virtual IP? In what situation would you use it?
    True or False? The MAC address of an interface is assigned/set by the OS

    False

    Can you have more than one default gateway in a given system?

    Technically, yes.

    What is telnet and why is it a bad idea to use it in production? (or at all)

    Telnet is a type of client-server protocol that can be used to open a command line on a remote computer, typically a server. By default, all the data sent and received via telnet is transmitted in clear/plain text, therefore it should not be used as it does not encrypt any data between the client and the server.

    What is the routing table? How do you view it?
    How can you send an HTTP request from your shell?

    Using nc is one way
    What are packet sniffers? Have you used one in the past? If yes, which packet sniffers have you used and for what purpose?
    It is a network utility that analyses and may inject tasks into the data-stream travelling over the targeted network.
    How to list active connections?
    How to trigger neighbor discovery in IPv6?

    One way would be ping6 ff02::1

    What is network interface bonding and do you know how it's performed in Linux?
    What network bonding modes are there?

    There a couple of modes:

    • balance-rr: round robing bonding
    • active-backup: a fault tolerance mode where only one is active
    • balance-tlb: Adaptive transmit load balancing
    • balance-alb: Adaptive load balancing
    What is a bridge? How it's added in Linux OS?
    Linux - DNS
    How to check what is the hostname of the system?

    cat /etc/hostname

    You can also run hostnamectl or hostname but that might print only a temporary hostname. The one in the file is the permanent one.

    What the file /etc/resolv.conf is used for? What does it include?
    What commands are you using for performing DNS queries (or troubleshoot DNS related issues)?

    You can specify one or more of the following:

    • dig
    • host
    • nslookup
    You run dig codingshell.com and get the following result:
    ANSWER SECTION:
    codingshell.com.	3515	IN	A	185.199.109.153
    

    What is the meaning of the number 3515?


    This is the TTL. When you lookup for an address using a domain/host name, your OS is performing DNS resolution by contacting DNS name servers to get the IP address of the host/domain you are looking for.
    When you get a reply, this reply in cached in your OS for a certain period of time. This is period of time is also known as TTL and this is the meaning of 3515 number - it will be cached for 3515 seconds before removed from the cache and during that period of time, you'll get the value from the cache instead of asking DNS name servers for the address again.

    Linux - Packaging
    Do you have experience with packaging? (as in building packages) Can you explain how does it works?
    How packages installation/removal is performed on the distribution you are using?

    The answer depends on the distribution being used.

    In Fedora/CentOS/RHEL/Rocky it can be done with rpm or dnf commands. In Ubuntu it can be done with the apt command.

    RPM: explain the spec format (what it should and can include)
    How do you list the content of a package without actually installing it?
    How to know to which package a file on the system belongs to? Is it a problem if it doesn't belongs to any package?
    Where repositories are stored? (based on the distribution you are using)
    What is an archive? How do you create one in Linux?
    How to extract the content of an archive?
    Why do we need package managers? Why not simply creating archives and publish them?

    Package managers allow you to manage packages lifecycle as in installing, removing and updating the packages.
    In addition, you can specify in a spec how a certain package will be installed - where to copy the files, which commands to run prior to the installation, post the installation, etc.

    Linux DNF

    What is DNF?

    From the repo:

    "Dandified YUM (DNF) is the next upcoming major version of YUM. It does package management using RPM, libsolv and hawkey libraries."

    Official docs

    How to look for a package that provides the command /usr/bin/git? (the package isn't necessarily installed)

    dnf provides /usr/bin/git

    Linux Applications and Services
    What can you find in /etc/services?
    How to make sure a Service starts automatically after a reboot or crash?

    Depends on the init system.

    Systemd: systemctl enable [service_name] System V: update-rc.d [service_name] and add this line id:5678:respawn:/bin/sh /path/to/app to /etc/inittab Upstart: add Upstart init script at /etc/init/service.conf

    You run ssh 127.0.0.1 but it fails with "connection refused". What could be the problem?
    1. SSH server is not installed
    2. SSH server is not running
    How to print the shared libraries required by a certain program? What is it useful for?
    What is CUPS?
    What types of web servers are you familiar with?

    Nginx, Apache httpd.

    Linux Users and Groups
    What is a "superuser" (or root user)? How is it different from regular users?
    How do you create users? Where user information is stored?

    Command to create users is useradd

    Syntax: useradd [options] Username

    There are 2 configuration files, which stores users information

    1. /etc/passwd - Users information like, username, shell etc is stored in this file

    2. /etc/shadow - Users password is stored in encrypted format

    Which file stores information about groups?

    /etc/groups file stores the group name, group ID, usernames which are in secondary group.

    How do you change/set the password of a user?

    passwd <username> is the command to set/change password of a user.

    Which file stores users passwords? Is it visible for everyone?

    /etc/shadow file holds the passwords of the users in encryted format. NO, it is only visble to the root user

    Do you know how to create a new user without using adduser/useradd command?

    YES, we can create new user by manually adding an entry in the /etc/passwd file.

    For example, if we need to create a user called john.

    Step 1: Add an entry to /etc/passwd file, so user gets created.

    echo "john:x:2001:2001::/home/john:/bin/bash" >> /etc/passwd

    Step 2: Add an entry to /etc/group file, because every user belong to the primary group that has same name as the username.

    echo "john:x:2001:" >> /etc/group

    Step 3: Verify if the user got created

    id john

    What information is stored in /etc/passwd? explain each field

    /etc/passwd is a configuration file, which contains users information. Each entry in this file has, 7 fields,

    username:password:UID:GID:Comment:home directory:shell

    username - The name of the user.

    password - This field is actually a placeholder of the password field. Due to security concerns, this field does not contain the password, just a placeholder (x) to the encrypted password stored in /etc/shadow file.

    UID - User ID of the user.

    GID - Group ID

    Comment - This field is to provide description about the user.

    home directory - Abousulte path of the user's home directory. This directory gets created once the user is added.

    shell - This field contains the absolute path of the shell that will be used by the respective user.

    How to add a new user to the system without providing him the ability to log-in into the system?

    adduser user_name --shell=/bin/false --no-create-home You can also add a user and then edit /etc/passwd.

    How to switch to another user? How to switch to the root user?

    su command. Use su - to switch to root

    What is the UID the root user? What about a regular user?

    UID of root user is 0

    Default values of UID_MIN and UID_MAX in /etc/login.defs UID_MIN is 1000 UID_MAX is 60000

    Actually, we can change this value. But UID < 1000 are reserved for system accounts. Therefore, as per the default configuration, for regular user UID starts from 1000.

    What can you do if you lost/forogt the root password?

    Re-install the OS IS NOT the right answer :)

    What is /etc/skel?

    /etc/skel is a directory, that contains files or directories, so when a new user is created, these files/directories created under /etc/skel will be copied to user's home directory.

    How to see a list of who logged-in to the system?

    Using the last command.

    Explain what each of the following commands does:
    • useradd
    • usermod
    • whoami
    • id

    useradd - Command for creating new users usermod - Modify the users setting whoami - Outputs, the username that we are currently logged in id - Prints the

    You run grep $(whoami) /etc/passwd but the output is empty. What might be a possible reason for that?

    The user you are using isn't defined locally but originates from services like LDAP.
    You can verify with: getent passwd

    Linux Hardware

    Where can you find information on the processor (like number of CPUs)?

    /proc/cpuinfo

    You can also use nproc for number of processors

    How can you print information on the BIOS, motherboard, processor and RAM?

    dmidecoode

    How can you print all the information on connected block devices in your system?

    lsblk

    True or False? In user space, applications don't have full access to hardware resources

    True. Only in kernel space they have full access to hardware resources.

    Linux - Security

    How do you create a private key for a CA (certificate authority)?

    One way is using openssl this way:

    openssl genrsa -aes256 -out ca-private-key.pem 4096

    How do you create a public key for a CA (certificate authority)?

    openssl req -new -x509 -days 730 -key [private key file name] -sha256 -out ca.pem

    If using the private key from the previous question then the command would be:

    openssl req -new -x509 -days 730 -key ca-private-key.pem -sha256 -out ca.pem

    Linux - Namespaces

    What types of namespaces are there in Linux?
    • Process ID namespaces: these namespaces include independent set of process IDs
    • Mount namespaces: Isolation and control of mountpoints
    • Network namespaces: Isolates system networking resources such as routing table, interfaces, ARP table, etc.
    • UTS namespaces: Isolate host and domains
    • IPC namespaces: Isolates interprocess communications
    • User namespaces: Isolate user and group IDs
    • Time namespaces: Isolates time machine
    True or False? In every PID (Process ID) namespace the first process assigned with the process id number 1

    True. Inside the namespace it's PID 1 while to the parent namespace the PID is a different one.

    True or False? In a child PID namespace all processes are aware of parent PID namespace and processes and the parent PID namespace has no visibility of child PID namespace processes

    False. The opposite is true. Parent PID namespace is aware and has visibility of processes in child PID namespace and child PID namespace has no visibility as to what is going on in the parent PID namespace.

    True or False? By default, when creating two separate network namespaces, a ping from one namespace to another will work fine

    False. Network namespace has its own interfaces and routing table. There is no way (without creating a bridge for example) for one network namespace to reach another.

    True or False? With UTS namespaces, processes may appear as if they run on different hosts and domains while running on the same host

    True

    True or False? It's not possible to have a root user with ID 0 in child user namespaces

    False. In every child user namespace, it's possible to have a separate root user with uid of 0.

    What time namespaces are used for?

    In time namespaces processes can use different system time.

    Linux - Virtualization

    What virtualization solutions are available for Linux?
    What is KVM?

    Is an open source virtualization technology used to operate on x86 hardware.

    From the official docs Recommended read:

    What is Libvirt?

    It's an open source collection of software used to manage virtual machines. It can be used with: KVM, Xen, LXC and others. It's also called Libvirt Virtualization API.

    From the official docs Hypervisor supported docs

    Linux - AWK

    What the awk command does? Have you used it? What for?

    From Wikipedia: "AWK is domain-specific language designed for text processing and typically used as a data extraction and reporting tool"

    How to print the 4th column in a file?

    awk '{print $4}' file

    How to print every line that is longer than 79 characters?

    awk 'length($0) > 79' file

    What the lsof command does? Have you used it? What for?
    What is the difference between find and locate?
    How a user process performs a privileged operation, such as reading from the disk?

    Using system calls

    Linux - System Calls

    What is a system call? What system calls are you familiar with?
    How a program executes a system call?
    • A program executes a trap instruction. The instruction jump into the kernel while raising the privileged level to kernel space.
    • Once in kernel space, it can perform any privileged operation
    • Once it's finished, it calls a "return-from-trap" instruction which returns to user space while reducing back the privilege level to user space.
    Explain the fork() system call

    fork() is used for creating a new process. It does so by cloning the calling process but the child process has its own PID and any memory locks, I/O operations and semaphores are not inherited.

    What is the return value of fork()?
    • On success, the PID of the child process in parent and 0 in child process
    • On error, -1 in the parent
    Name one reason for fork() to fail

    Not enough memory to create a new process

    Why do we need the wait() system call?

    wait() is used by a parent process to wait for the child process to finish execution. If wait is not used by a parent process then a child process might become a zombie process.

    How the kernel notifies the parent process about child process termination?

    The kernel notifies the parent by sending the SIGCHLD to the parent.

    How the waitpid() is different from wait()?

    The waitpid() is a non-blocking version of the wait() function.
    It also supports using library routine (e.g. system()) to wait a child process without messing up with other children processes for which the process has not waited.

    True or False? The wait() system call won't return until the child process has run and exited

    True in most cases though there are cases where wait() returns before the child exits.

    Explain the exec() system call

    It transforms the current running program into another program.
    Given the name of an executable and some arguments, it loads the code and static data from the specified executable and overwrites its current code segment and current static code data. After initializing its memory space (like stack and heap) the OS runs the program passing any arguments as the argv of that process.

    True or False? A successful call to exec() never returns

    True
    Since a succesful exec replace the current process, it can't return anything to the process that made the call.

    What system call is used for listing files?
    What system calls are used for creating a new process?

    fork(), exec() and the wait() system call is also included in this workflow.

    What execve() does?

    Executes a program. The program is passed as a filename (or path) and must be a binary executable or a script.

    What is the return value of malloc?
    Explain the pipe() system call. What does it used for?

    Unix pipe implementation

    "Pipes provide a unidirectional interprocess communication channel. A pipe has a read end and a write end. Data written to the write end of a pipe can be read from the read end of the pipe. A pipe is created using pipe(2), which returns two file descriptors, one referring to the read end of the pipe, the other referring to the write end."

    What happens when you execute ls -l?
    • Shell reads the input using getline() which reads the input file stream and stores into a buffer as a string

    • The buffer is broken down into tokens and stored in an array this way: {"ls", "-l", "NULL"}

    • Shell checks if an expansion is required (in case of ls *.c)

    • Once the program in memory, its execution starts. First by calling readdir()

    Notes:

    • getline() originates in GNU C library and used to read lines from input stream and stores those lines in the buffer
    What happens when you execute ls -l *.log?
    What readdir() system call does?
    What exactly the command alias x=y does?
    Why running a new program is done using the fork() and exec() system calls? why a different API wasn't developed where there is one call to run a new program?

    This way provides a lot of flexibility. It allows the shell for example, to run code after the call to fork() but before the call to exec(). Such code can be used to alter the environment of the program it about to run.

    Describe shortly what happens when you execute a command in the shell

    The shell figures out, using the PATH variable, where the executable of the command resides in the filesystem. It then calls fork() to create a new child process for running the command. Once the fork was executed successfully, it calls a variant of exec() to execute the command and finally, waits the command to finish using wait(). When the child completes, the shell returns from wait() and prints out the prompt again.

    Linux Filesystem & Files

    How to create a file of a certain size?

    There are a couple of ways to do that:

    • dd if=/dev/urandom of=new_file.txt bs=2MB count=1
    • truncate -s 2M new_file.txt
    • fallocate -l 2097152 new_file.txt
    What does the following block do?:
    open("/my/file") = 5
    read(5, "file content")
    

    These system calls are reading the file /my/file and 5 is the file descriptor number.

    Describe three different ways to remove a file (or its content)
    What is the difference between a process and a thread?
    What is context switch?

    From wikipedia: a context switch is the process of storing the state of a process or thread, so that it can be restored and resume execution at a later point

    You found there is a server with high CPU load but you didn't find a process with high CPU. How is that possible?
    Linux Advanced - Networking
    When you run ip a you see there is a device called 'lo'. What is it and why do we need it?
    What the traceroute command does? How does it works?

    Another common way to task this questions is "what part of the tcp header does traceroute modify?"

    What is network bonding? What types are you familiar with?
    How to link two separate network namespaces so you can ping an interface on one namespace from the second one?
    What are cgroups?
    Explain Process Descriptor and Task Structure
    What are the differences between threads and processes?
    Explain Kernel Threads
    What happens when socket system call is used?

    This is a good article about the topic: https://ops.tips/blog/how-linux-creates-sockets

    You executed a script and while still running, it got accidentally removed. Is it possible to restore the script while it's still running?

    Linux Memory

    What is the difference between MemFree and MemAvailable in /proc/meminfo?

    MemFree - The amount of unused physical RAM in your system MemAvailable - The amount of available memory for new workloads (without pushing system to use swap) based on MemFree, Active(file), Inactive(file), and SReclaimable.

    What is the difference between paging and swapping?
    Explain what is OOM killer

    Distribution

    What is a Linux distribution?
    What Linux distributions are you familiar with?
    What are the components of a Linux distribution?
    • Kernel
    • Utilities
    • Services
    • Software/Packages Management

    Linux - Sed

    Using sed, extract the date from the following line: 201.7.19.90 - - [05/Jun/1985:13:42:99 +0000] "GET /site HTTP/1.1" 200 32421

    echo $line | sed 's/.*\[//g;s/].*//g;s/:.*//g'

    Linux - Misc

    How to generate a random string?

    One way is to run the following: cat /proc/sys/kernel/random/uuid

    What is a Linux distribution?
    • A collection of packages - kernel, GNU, third party apps, ...
    • Sometimes distributions store some information on the distribution in /etc/*-release file
      • For example for Red Hat distribution it will be /etc/redhat-release and for Amazon it will be /etc/os-release
      • lsb_release is a common command you can use in multiple different distributions
    Name 5 commands which are two letters long

    ls, wc, dd, df, du, ps, ip, cp, cd ...

    What ways are there for creating a new empty file?
    • touch new_file
    • echo "" > new_file
    How `cd -` works? How does it knows the previous location?

    $OLDPWD

    List three ways to print all the files in the current directory
    • ls
    • find .
    • echo *
    How to count the number of lines in a file? What about words?
    You define x=2 in /etc/bashrc and x=6 ~/.bashrc you then login to the system. What would be the value of x?
    What is the difference between man and info?

    A good answer can be found here

    Explain "environment variables". How do you list all environment variables?
    What is a TTY device?
    How to create your own environment variables?

    X=2 for example. But this will persist to new shells. To have it in new shells as well, use export X=2

    What a double dash (--) mean?

    It's used in commands to mark the end of commands options. One common example is when used with git to discard local changes: git checkout -- some_file

    Wildcards are implemented on user or kernel space?
    If I plug a new device into a Linux machine, where on the system, a new device entry/file will be created?

    /dev

    Why there are different sections in man? What is the difference between the sections?
    What is User-mode Linux?
    Under which license Linux is distributed?

    GPL v2

    Operating System

    Operating System Exercises

    Name Topic Objective & Instructions Solution Comments
    Fork 101 Fork Link Link
    Fork 102 Fork Link Link

    Operating System - Self Assessment

    What is an operating system?

    From the book "Operating Systems: Three Easy Pieces":

    "responsible for making it easy to run programs (even allowing you to seemingly run many at the same time), allowing programs to share memory, enabling programs to interact with devices, and other fun stuff like that".

    Operating System - Process

    Can you explain what is a process?

    A process is a running program. A program is one or more instructions and the program (or process) is executed by the operating system.

    If you had to design an API for processes in an operating system, what would this API look like?

    It would support the following:

    • Create - allow to create new processes
    • Delete - allow to remove/destroy processes
    • State - allow to check the state of the process, whether it's running, stopped, waiting, etc.
    • Stop - allow to stop a running process
    How a process is created?
    • The OS is reading program's code and any additional relevant data
    • Program's code is loaded into the memory or more specifically, into the address space of the process.
    • Memory is allocated for program's stack (aka run-time stack). The stack also initialized by the OS with data like argv, argc and parameters to main()
    • Memory is allocated for program's heap which is required for dynamically allocated data like the data structures linked lists and hash tables
    • I/O initialization tasks are performed, like in Unix/Linux based systems where each process has 3 file descriptors (input, output and error)
    • OS is running the program, starting from main()
    True or False? The loading of the program into the memory is done eagerly (all at once)

    False. It was true in the past but today's operating systems perform lazy loading which means only the relevant pieces required for the process to run are loaded first.

    What are different states of a process?
    • Running - it's executing instructions
    • Ready - it's ready to run but for different reasons it's on hold
    • Blocked - it's waiting for some operation to complete. For example I/O disk request
    What are some reasons for a process to become blocked?
    • I/O operations (e.g. Reading from a disk)
    • Waiting for a packet from a network
    What is Inter Process Communication (IPC)?
    What is "time sharing"?

    Even when using a system with one physical CPU, it's possible to allow multiple users to work on it and run programs. This is possible with time sharing where computing resources are shared in a way it seems to the user the system has multiple CPUs but in fact it's simply one CPU shared by applying multiprogramming and multi-tasking.

    What is "space sharing"?

    Somewhat the opposite of time sharing. While in time sharing a resource is used for a while by one entity and then the same resource can be used by another resource, in space sharing the space is shared by multiple entities but in a way where it's not being transferred between them.
    It's used by one entity until this entity decides to get rid of it. Take for example storage. In storage, a file is yours until you decide to delete it.

    What component determines which process runs at a given moment in time?

    CPU scheduler

    Operating System - Memory

    What is "virtual memory" and what purpose it serves?
    What is demand paging?
    What is copy-on-write or shadowing?
    What is a kernel, and what does it do?

    The kernel is part of the operating system and is responsible for tasks like:

    • Allocating memory
    • Schedule processes
    • Control CPU
    True or False? Some pieces of the code in the kernel are loaded into protected areas of the memory so applications can't overwritten them

    True

    What is POSIX?
    Explain what is Semaphore and what its role in operating systems
    What is cache? What is buffer?

    Buffer: Reserved place in RAM which is used to hold data for temporary purposes Cache: Cache is usually used when processes reading and writing to the disk to make the process faster by making similar data used by different programs easily accessible.

    Virtualization

    What is Virtualization?

    Virtualization uses software to create an abstraction layer over computer hardware that allows the hardware elements of a single computer—processors, memory, storage and more - to be divided into multiple virtual computers, commonly called virtual machines (VMs).

    What is a hypervisor?

    Red Hat: "A hypervisor is software that creates and runs virtual machines (VMs). A hypervisor, sometimes called a virtual machine monitor (VMM), isolates the hypervisor operating system and resources from the virtual machines and enables the creation and management of those VMs."

    Read more here

    What types of hypervisors are there?

    Hosted hypervisors and bare-metal hypervisors.

    What are the advantages and disadvantges of bare-metal hypervisor over a hosted hypervisor?

    Due to having its own drivers and a direct access to hardware components, a baremetal hypervisor will often have better performances along with stability and scalability.

    On the other hand, there will probably be some limitation regarding loading (any) drivers so a hosted hypervisor will usually benefit from having a better hardware compatibility.

    What types of virtualization are there?

    Operating system virtualization Network functions virtualization Desktop virtualization

    Is containerization is a type of Virtualization?

    Yes, it's a operating-system-level virtualization, where the kernel is shared and allows to use multiple isolated user-spaces instances.

    How the introduction of virtual machines changed the industry and the way applications were deployed?

    The introduction of virtual machines allowed companies to deploy multiple business applications on the same hardware while each application is separated from each other in secured way, where each is running on its own separate operating system.

    Python - OOP

    Explain inheritance and how to use it in Python
    By definition inheritance is the mechanism where an object acts as a base of another object, retaining all its
    properties.
    
    So if Class B inherits from Class A, every characteristics from class A will be also available in class B.
    Class A would be the 'Base class' and B class would be the 'derived class'.
    
    This comes handy when you have several classes that share the same functionalities.
    
    The basic syntax is:
    
    class Base: pass
    
    class Derived(Base): pass
    
    A more forged example:
    
    class Animal:
        def __init__(self):
            print("and I'm alive!")
    
        def eat(self, food):
            print("ñom ñom ñom", food)
    
    class Human(Animal):
        def __init__(self, name):
            print('My name is ', name)
            super().__init__()
    
        def write_poem(self):
            print('Foo bar bar foo foo bar!')
    
    class Dog(Animal):
        def __init__(self, name):
            print('My name is', name)
            super().__init__()
    
        def bark(self):
            print('woof woof')
    
    
    michael = Human('Michael')
    michael.eat('Spam')
    michael.write_poem()
    
    bruno = Dog('Bruno')
    bruno.eat('bone')
    bruno.bark()
    
    >>> My name is  Michael
    >>> and I'm alive!
    >>> ñom ñom ñom Spam
    >>> Foo bar bar foo foo bar!
    >>> My name is Bruno
    >>> and I'm alive!
    >>> ñom ñom ñom bone
    >>> woof woof
    
    Calling super() calls the Base method, thus, calling super().__init__() we called the Animal __init__.
    
    There is a more advanced python feature called MetaClasses that aid the programmer to directly control class creation.
    

    Explain and demonstrate class attributes & instance attributes

    In the following block of code x is a class attribute while self.y is a instance attribute

    class MyClass(object):
        x = 1
    
        def __init__(self, y):
            self.y = y
    

    Python - Exceptions

    What is an error? What is an exception? What types of exceptions are you familiar with?
    #  Note that you generally don't need to know the compiling process but knowing where everything comes from
    #  and giving complete answers shows that you truly know what you are talking about.
    
    Generally, every compiling process have a two steps.
        - Analysis
        - Code Generation.
    
        Analysis can be broken into:
            1. Lexical analysis   (Tokenizes source code)
            2. Syntactic analysis (Check whether the tokens are legal or not, tldr, if syntax is correct)
    
                   for i in 'foo'
                              ^
                 SyntaxError: invalid syntax
    
            We missed ':'
    
    
            3. Semantic analysis  (Contextual analysis, legal syntax can still trigger errors, did you try to divide by 0,
              hash a mutable object or use an undeclared function?)
    
                     1/0
                    ZeroDivisionError: division by zero
    
        These three analysis steps are the responsible for error handlings.
    
        The second step would be responsible for errors, mostly syntax errors, the most common error.
        The third step would be responsible for Exceptions.
    
        As we have seen, Exceptions are semantic errors, there are many builtin Exceptions:
    
            ImportError
            ValueError
            KeyError
            FileNotFoundError
            IndentationError
            IndexError
            ...
    
        You can also have user defined Exceptions that have to inherit from the `Exception` class, directly or indirectly.
    
        Basic example:
    
        class DividedBy2Error(Exception):
            def __init__(self, message):
                self.message = message
    
    
        def division(dividend,divisor):
            if divisor == 2:
                raise DividedBy2Error('I dont want you to divide by 2!')
            return dividend / divisor
    
        division(100, 2)
    
        >>> __main__.DividedBy2Error: I dont want you to divide by 2!
    

    Explain Exception Handling and how to use it in Python

    Exceptions: Errors detected during execution are called Exceptions.

    Handling Exception: When an error occurs, or exception as we call it, Python will normally stop and generate an error message.
    Exceptions can be handled using try and except statement in python.

    Example: Following example asks the user for input until a valid integer has been entered.
    If user enter a non-integer value it will raise exception and using except it will catch that exception and ask the user to enter valid integer again.

    while True:
        try:
            a = int(input("please enter an integer value: "))
            break
        except ValueError:
            print("Ops! Please enter a valid integer value.")

    For more details about errors and exceptions follow this https://docs.python.org/3/tutorial/errors.html

    What is the result of running the following function?
    def true_or_false():
        try:
            return True
        finally:
            return False
    

    False

    Python Built-in functions

    Explain the following built-in functions (their purpose + use case example):
    • repr
    • any
    • all

    What is the difference between repr function and str?
    What is the __call__ method?

    It is used to emulate callable objects. It allows a class instance to be called as a function.

    • Example code:
    class Foo:
        def __init__(self: object) ->  None:
            pass
        def __call__(self: object) -> None:
            print("Called!")
    
    f = Foo()
    f()
    
    • Result:
    Called!
    

    Do classes has the __call__ method as well? What for?
    What _ is used for in Python?
    1. Translation lookup in i18n
    2. Hold the result of the last executed expression or statement in the interactive interpreter.
    3. As a general purpose "throwaway" variable name. For example: x, y, _ = get_data() (x and y are used but since we don't care about third variable, we "threw it away").
    Explain what is GIL
    Python Global Interpreter Lock (GIL) is a type of process lock which is used by python whenever it deals with processes. Generally, Python only uses only one thread to execute the set of written statements. This means that in python only one thread will be executed at a time
    What is Lambda? How is it used?

    A lambda expression is an 'anonymous' function, the difference from a normal defined function using the keyword `def`` is the syntax and usage.

    The syntax is:

    lambda[parameters]: [expresion]

    Examples:

    • A lambda function add 10 with any argument passed.
    x = lambda a: a + 10
    print(x(10))
    • An addition function
    addition = lambda x, y: x + y
    print(addition(10, 20))
    • Squaring function
    square = lambda x : x ** 2
    print(square(5))

    Generally it is considered a bad practice under PEP 8 to assign a lambda expresion, they are meant to be used as parameters and inside of other defined functions.

    Properties

    Are there private variables in Python? How would you make an attribute of a class, private?
    Explain the following:
    • getter
    • setter
    • deleter

    Explain what is @property
    How do you swap values between two variables?
    x, y = y, x
    

    Explain the following object's magic variables:
    • dict

    Write a function to return the sum of one or more numbers. The user will decide how many numbers to use

    First you ask the user for the amount of numbers that will be use. Use a while loop that runs until amount_of_numbers becomes 0 through subtracting amount_of_numbers by one each loop. In the while loop you want ask the user for a number which will be added a variable each time the loop runs.

    def return_sum():
    	amount_of_numbers = int(input("How many numbers? "))
    	total_sum = 0
    	while amount_of_numbers != 0:
    		num = int(input("Input a number. "))
    		total_sum += num
    		amount_of_numbers -= 1
    	return total_sum
    
    

    Print the average of [2, 5, 6]. It should be rounded to 3 decimal places
    li = [2, 5, 6]
    print("{0:.3f}".format(sum(li)/len(li)))
    

    Python - Lists

    What is a tuple in Python? What is it used for?

    A tuple is a built-in data type in Python. It's used for storing multiple items in a single variable.

    List, like a tuple, is also used for storing multiple items. What is then, the difference between a tuple and a list?

    List, as opposed to a tuple, is a mutable data type. It means we can modify it and at items to it.

    How to add the number 2 to the list x = [1, 2, 3]

    x.append(2)

    How to get the last element of a list?

    some_list[-1]

    How to add the items of [1, 2, 3] to the list [4, 5, 6]?
    x = [4, 5, 6] x.extend([1, 2, 3])

    Don't use append unless you would like the list as one item.

    How to remove the first 3 items from a list?

    my_list[0:3] = []

    How to insert an item to the beginning of a list? What about two items?
    • One item:
    numbers = [1, 2, 3, 4, 5]
    numbers.insert(0, 0)
    print(numbers)
    
    • Multiple items or list:
    numbers_1 = [2, 3, 4, 5]
    numbers_2 = [0, 1]
    numbers_1 = numbers_2 + numbers_1
    print(numbers_1)
    

    How to sort list by the length of items?
    sorted_li = sorted(li, key=len)
    

    Or without creating a new list:

    li.sort(key=len)
    

    Do you know what is the difference between list.sort() and sorted(list)?
    • sorted(list) will return a new list (original list doesn't change)

    • list.sort() will return None but the list is change in-place

    • sorted() works on any iterable (Dictionaries, Strings, ...)

    • list.sort() is faster than sorted(list) in case of Lists

    Convert every string to an integer: [['1', '2', '3'], ['4', '5', '6']]
    nested_li = [['1', '2', '3'], ['4', '5', '6']]
    [[int(x) for x in li] for li in nested_li]
    

    How to merge two sorted lists into one sorted list?
    sorted(li1 + li2)
    

    Another way:

    i, j = 0
    merged_li = []
    
    while i < len(li1) and j < len(li2):
        if li1[i] < li2[j]:
            merged_li.append(li1[i])
            i += 1
        else:
            merged_li.append(li2[j])
            j += 1
    
    merged_li = merged_li + merged_li[i:] + merged_li[j:]
    

    How to check if all the elements in a given lists are unique? so [1, 2, 3] is unique but [1, 1, 2, 3] is not unique because 1 exists twice

    There are many ways of solving this problem:
    # Note: :list and -> bool are just python typings, they are not needed for the correct execution of the algorithm.

    Taking advantage of sets and len:

    def is_unique(l:list) -> bool:
        return len(set(l)) == len(l)
    

    This one is can be seen used in other programming languages.

    def is_unique2(l:list) -> bool:
        seen = []
    
        for i in l:
            if i in seen:
                return False
            seen.append(i)
        return True
    

    Here we just count and make sure every element is just repeated once.

    def is_unique3(l:list) -> bool:
        for i in l:
            if l.count(i) > 1:
                return False
        return True
    

    This one might look more convulated but hey, one liners.

    def is_unique4(l:list) -> bool:
        return all(map(lambda x: l.count(x) < 2, l))
    
    You have the following function
    def my_func(li = []):
        li.append("hmm")
        print(li)
    

    If we call it 3 times, what would be the result each call?


    ['hmm']
    ['hmm', 'hmm']
    ['hmm', 'hmm', 'hmm']
    

    How to iterate over a list?
    for item in some_list:
        print(item)
    

    How to iterate over a list with indexes?
    for i, item in enumerate(some_list):
        print(i)
    

    How to start list iteration from 2nd index?

    Using range like this

    for i in range(1, len(some_list)):
        some_list[i]
    

    Another way is using slicing

    for i in some_list[1:]:
    

    How to iterate over a list in reverse order?

    Method 1

    for i in reversed(li):
        ...
    

    Method 2

    n = len(li) - 1
    while n > 0:
        ...
        n -= 1
    

    Sort a list of lists by the second item of each nested list
    li = [[1, 4], [2, 1], [3, 9], [4, 2], [4, 5]]
    
    sorted(li, key=lambda l: l[1])
    

    or

    li.sort(key=lambda l: l[1)
    

    Combine [1, 2, 3] and ['x', 'y', 'z'] so the result is [(1, 'x'), (2, 'y'), (3, 'z')]
    nums = [1, 2, 3]
    letters = ['x', 'y', 'z']
    
    list(zip(nums, letters))
    

    What is List Comprehension? Is it better than a typical loop? Why? Can you demonstrate how to use it?

    From Docs: "List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.".

    It's better because they're compact, faster and have better readability.

    • For loop:
    number_lists = [[1, 7, 3, 1], [13, 93, 23, 12], [123, 423, 456, 653, 124]]
    odd_numbers = []
    for number_list in number_lists:
        for number in number_list:
            if number % 2 == 0:
                odd_numbers.append(number)
    print(odd_numbers)
    
    • List comprehesion:
    number_lists = [[1, 7, 3, 1], [13, 93, 23, 12], [123, 423, 456, 653, 124]]
    odd_numbers = [number for number_list in number_lists for number in number_list if number % 2 == 0]
    print(odd_numbers)
    

    You have the following list: [{'name': 'Mario', 'food': ['mushrooms', 'goombas']}, {'name': 'Luigi', 'food': ['mushrooms', 'turtles']}] Extract all type of foods. Final output should be: {'mushrooms', 'goombas', 'turtles'}
    brothers_menu =  \
    [{'name': 'Mario', 'food': ['mushrooms', 'goombas']}, {'name': 'Luigi', 'food': ['mushrooms', 'turtles']}]
    
    # "Classic" Way
    def get_food(brothers_menu) -> set:
        temp = []
    
        for brother in brothers_menu:
            for food in brother['food']:
                temp.append(food)
    
        return set(temp)
    
    # One liner way (Using list comprehension)
    set([food for bro in x for food in bro['food']])
    

    Python - Dictionaries

    How to create a dictionary?

    my_dict = dict(x=1, y=2) OR my_dict = {'x': 1, 'y': 2} OR my_dict = dict([('x', 1), ('y', 2)])

    How to remove a key from a dictionary?

    del my_dict['some_key'] you can also use my_dict.pop('some_key') which returns the value of the key.

    How to sort a dictionary by values?
    {k: v for k, v in sorted(x.items(), key=lambda item: item[1])}
    

    How to sort a dictionary by keys?
    dict(sorted(some_dictionary.items()))
    

    How to merge two dictionaries?
    some_dict1.update(some_dict2)
    

    Convert the string "a.b.c" to the dictionary {'a': {'b': {'c': 1}}}
    output = {}
    string = "a.b.c"
    path = string.split('.')
    target = reduce(lambda d, k: d.setdefault(k, {}), path[:-1], output)
    target[path[-1]] = 1
    print(output)
    

    Common Algorithms Implementation
    Can you implement "binary search" in Python?

    Solution

    Python Files

    How to write to a file?
    with open('file.txt', 'w') as file:
        file.write("My insightful comment")
    

    Sum all the integers in a given file
    Print a random line of a given file
    Print every 3rd line of a given file
    Print the number of lines in a given file
    Print the number of of words in a given file
    Can you write a function which will print all the file in a given directory? including sub-directories
    Write a dictionary (variable) to a file
    import json
    
    with open('file.json', 'w') as f:
        f.write(json.dumps(dict_var))
    

    Python OS

    How to print current working directory?
    import os
    
    print(os.getcwd())
    

    Given the path /dir1/dir2/file1 print the file name (file1)
    import os
    
    print(os.path.basename('/dir1/dir2/file1'))
    
    # Another way
    print(os.path.split('/dir1/dir2/file1')[1])
    

    Given the path /dir1/dir2/file1
    1. Print the path without the file name (/dir1/dir2)
    2. Print the name of the directory where the file resides (dir2)

    import os
    
    ## Part 1.
    # os.path.dirname gives path removing the end component
    dirpath = os.path.dirname('/dir1/dir2/file1')
    print(dirpath)
    
    ## Part 2.
    print(os.path.basename(dirpath))
    

    How do you execute shell commands using Python?
    How do you join path components? for example /home and luig will result in /home/luigi
    How do you remove non-empty directory?

    Python Regex

    How do you perform regular expressions related operations in Python? (match patterns, substitute strings, etc.)

    Using the re module

    How to find all the IP addresses in a variable? How to find them in a file?

    Python Strings

    Find the first repeated character in a string

    While you iterate through the characters, store them in a dictionary and check for every character whether it's already in the dictionary.

    def firstRepeatedCharacter(str):
        chars = {}
        for ch in str:
            if ch in chars:
                return ch
            else:
                chars[ch] = 0
    

    How to extract the unique characters from a string? for example given the input "itssssssameeeemarioooooo" the output will be "mrtisaoe"
    x = "itssssssameeeemarioooooo"
    y = ''.join(set(x))
    

    Find all the permutations of a given string
    def permute_string(string):
    
        if len(string) == 1:
            return [string]
    
        permutations = []
        for i in range(len(string)):
            swaps = permute_string(string[:i] + string[(i+1):])
            for swap in swaps:
                permutations.append(string[i] + swap)
    
        return permutations
    
    print(permute_string("abc"))
    

    Short way (but probably not acceptable in interviews):

    from itertools import permutations
    
    [''.join(p) for p in permutations("abc")]
    

    Detailed answer can be found here: http://codingshell.com/python-all-string-permutations

    How to check if a string contains a sub string?
    Find the frequency of each character in string
    Count the number of spaces in a string

    You can use the "count" method like this:

    ImAString.count(" ")

    Given a string, find the N most repeated words
    Given the string (which represents a matrix) "1 2 3\n4 5 6\n7 8 9" create rows and colums variables (should contain integers, not strings)
    What is the result of each of the following?
    >> ', '.join(["One", "Two", "Three"])
    >> " ".join("welladsadgadoneadsadga".split("adsadga")[:2])
    >> "".join(["c", "t", "o", "a", "o", "q", "l"])[0::2]
    

    >>> 'One, Two, Three'
    >>> 'well done'
    >>> 'cool'
    

    If x = "pizza", what would be the result of x[::-1]?

    It will reverse the string, so x would be equal to azzip.

    Reverse each word in a string (while keeping the order)
    What is the output of the following code: "".join(["a", "h", "m", "a", "h", "a", "n", "q", "r", "l", "o", "i", "f", "o", "o"])[2::3]

    mario

    Python Iterators

    What is an iterator?

    Python Misc

    Explain data serialization and how do you perform it with Python
    How do you handle argument parsing in Python?
    What is a generator? Why using generators?
    What would be the output of the following block?
    for i in range(3, 3):
       print(i)
    

    No output :)

    What is yeild? When would you use it?
    Explain the following types of methods and how to use them:
    • Static method
    • Class method
    • instance method

    How to reverse a list?
    How to combine list of strings into one string with spaces between the strings
    You have the following list of nested lists: [['Mario', 90], ['Geralt', 82], ['Gordon', 88]] How to sort the list by the numbers in the nested lists?

    One way is:

    the_list.sort(key=lambda x: x[1])

    Explain the following:
    • zip()
    • map()
    • filter()

    Python - Slicing

    For the following slicing exercises, assume you have the following list: my_list = [8, 2, 1, 10, 5, 4, 3, 9]

    What is the result of `my_list[0:4]`?
    What is the result of `my_list[5:6]`?
    What is the result of `my_list[5:5]`?
    What is the result of `my_list[::-1]`?
    What is the result of `my_list[::3]`?
    What is the result of `my_list[2:]`?
    What is the result of `my_list[:3]`?

    Python Debugging

    How do you debug Python code?

    pdb :D

    How to check how much time it took to execute a certain script or block of code?
    What empty return returns?

    Short answer is: It returns a None object.

    We could go a bit deeper and explain the difference between

    def a ():
        return
    
    >>> None
    

    And

    def a ():
        pass
    
    >>> None
    

    Or we could be asked this as a following question, since they both give the same result.

    We could use the dis module to see what's going on:

      2           0 LOAD_CONST               0 (<code object a at 0x0000029C4D3C2DB0, file "<dis>", line 2>)
                  2 LOAD_CONST               1 ('a')
                  4 MAKE_FUNCTION            0
                  6 STORE_NAME               0 (a)
    
      5           8 LOAD_CONST               2 (<code object b at 0x0000029C4D3C2ED0, file "<dis>", line 5>)
                 10 LOAD_CONST               3 ('b')
                 12 MAKE_FUNCTION            0
                 14 STORE_NAME               1 (b)
                 16 LOAD_CONST               4 (None)
                 18 RETURN_VALUE
    
    Disassembly of <code object a at 0x0000029C4D3C2DB0, file "<dis>", line 2>:
      3           0 LOAD_CONST               0 (None)
                  2 RETURN_VALUE
    
    Disassembly of <code object b at 0x0000029C4D3C2ED0, file "<dis>", line 5>:
      6           0 LOAD_CONST               0 (None)
                  2 RETURN_VALUE
    

    An empty return is exactly the same as return None and functions without any explicit return will always return None regardless of the operations, therefore

    def sum(a, b):
        global c
        c = a + b
    
    >>> None
    

    How to improve the following block of code?
    li = []
    for i in range(1, 10):
        li.append(i)
    

    [i for i in range(1, 10)]
    

    Given the following function
    def is_int(num):
        if isinstance(num, int):
            print('Yes')
        else:
            print('No')
    

    What would be the result of is_int(2) and is_int(False)?


    Python - Linked List

    Can you implement a linked list in Python?

    The reason we need to implement in the first place, it's because a linked list isn't part of Python standard library.
    To implement a linked list, we have to implement two structures: The linked list itself and a node which is used by the linked list.

    Let's start with a node. A node has some value (the data it holds) and a pointer to the next node

    class Node(object):
        def __init__(self, data):
            self.data = data
            self.next = None
    

    Now the linked list. An empty linked list has nothing but an empty head.

    class LinkedList(object):
        def __init__(self):
            self.head = None
    

    Now we can start using the linked list

    ll = Linkedlist()
    ll.head = Node(1)
    ll.head.next = Node(2)
    ll.head.next.next = Node(3)
    

    What we have is:


    | 1 | -> | 2 | -> | 3 |


    Usually, more methods are implemented, like a push_head() method where you insert a node at the beginning of the linked list

    def push_head(self, value):
        new_node = Node(value)
        new_node.next = self.head
        self.head = new_node
    

    Add a method to the Linked List class to traverse (print every node's data) the linked list

    def print_list(self): node = self.head while(node): print(node.data) node = node.next

    Write a method to that will return a boolean based on whether there is a loop in a linked list or not

    Let's use the Floyd's Cycle-Finding algorithm:

    def loop_exists(self):
        one_step_p = self.head
        two_steps_p = self.head
        while(one_step_p and two_steps_p and two_steps_p.next):
            one_step_p = self.head.next
            two_step_p = self.head.next.next
            if (one_step_p == two_steps_p):
                return True 
        return False
    

    Python - Stack

    Implement Stack in Python

    Python Testing

    What is your experience with writing tests in Python?
    What is PEP8? Give an example of 3 style guidelines

    PEP8 is a list of coding conventions and style guidelines for Python

    5 style guidelines:

    1. Limit all lines to a maximum of 79 characters.
    2. Surround top-level function and class definitions with two blank lines.
    3. Use commas when making a tuple of one element
    4. Use spaces (and not tabs) for indentation
    5. Use 4 spaces per indentation level
    

    How to test if an exception was raised?
    What assert does in Python?
    Explain mocks
    How do you measure execution time of small code snippets?
    Why one shouldn't use assert in non-test/production code?

    Flask

    Can you describe what is Django/Flask and how you have used it? Why Flask and not Django? (or vice versa)
    What is a route?
    As every web framework, Flask provides a route functionality that lets you serve a content of a given URL.

    There are multiple ways to map a URL with a function in Python.

    • Decorator: you can use python decorators. In this case we're using app. This app decorator is the instance of the Flask class. And route it's a method of this class.
    @app.route('/')
    def home():
      return 'main website'
    
    • add_url_rule method: This is a method of the Flask class. We can also use it for map the URL with a function.
    def home():
      return 'main website'
    
    app.add_url_rule('/', view_func=home)
    

    What is a blueprint in Flask?
    What is a template?

    zip

    Given x = [1, 2, 3], what is the result of list(zip(x))?
    [(1,), (2,), (3,)]
    

    What is the result of each of the following:
    list(zip(range(5), range(50), range(50)))
    list(zip(range(5), range(50), range(-2)))
    

    [(0, 0, 0), (1, 1, 1), (2, 2, 2), (3, 3, 3), (4, 4, 4)]
    []
    

    Python Descriptors

    Explain Descriptors

    Read about descriptors here

    What would be the result of running a.num2 assuming the following code
    class B:
        def __get__(self, obj, objtype=None):
            reuturn 10
    
    class A:
        num1 = 2
        num2 = Five()
    

    10
    What would be the result of running some_car = Car("Red", 4) assuming the following code
    class Print:
    
        def __get__(self, obj, objtype=None):
            value = obj._color
            print("Color was set to {}".format(valie))
            return value
    
        def __set__(self, obj, value):
            print("The color of the car is {}".format(value))
            obj._color = value
    
    class Car:
    
        color = Print()
    
        def __ini__(self, color, age):
            self.color = color
            self.age = age
    

    An instance of Car class will be created and the following will be printed: "The color of the car is Red"

    Python Misc

    How can you spawn multiple processes with Python?
    Implement simple calculator for two numbers
    def add(num1, num2):
        return num1 + num2
    
    
    def sub(num1, num2):
        return num1 - num2
    
    
    def mul(num1, num2):
        return num1*num2
    
    
    def div(num1, num2):
        return num1 / num2
    
    operators = {
        '+': add,
        '-': sub,
        '*': mul,
        '/': div
    }
    
    if __name__ == '__main__':
        operator = str(input("Operator: "))
        num1 = int(input("1st number: "))
        num2 = int(input("2nd number: "))
        print(operators[operator](num1, num2))
    

    What data types are you familiar with that are not Python built-in types but still provided by modules which are part of the standard library?

    This is a good reference https://docs.python.org/3/library/datatypes.html

    Explain what is a decorator
    In python, everything is an object, even functions themselves. Therefore you could pass functions as arguments for another function eg;
    def wee(word):
        return word
    
    def oh(f):
        return f + "Ohh"
    
    >>> oh(wee("Wee"))
    <<< Wee Ohh
    

    This allows us to control the before execution of any given function and if we added another function as wrapper, (a function receiving another function that receives a function as parameter) we could also control the after execution.

    Sometimes we want to control the before-after execution of many functions and it would get tedious to write

    f = function(function_1()) f = function(function_1(function_2(*args)))

    every time, that's what decorators do, they introduce syntax to write all of this on the go, using the keyword '@'.

    Can you show how to write and use decorators?
    These two decorators (ntimes and timer) are usually used to display decorators functionalities, you can find them in lots of tutorials/reviews. I first saw these examples two years ago in pyData 2017. https://www.youtube.com/watch?v=7lmCu8wz8ro&t=3731s
    Simple decorator:
    
    def deco(f):
        print(f"Hi I am the {f.__name__}() function!")
        return f
    
    @deco
    def hello_world():
        return "Hi, I'm in!"
    
    a = hello_world()
    print(a)
    
    >>> Hi I am the hello_world() function!
        Hi, I'm in!
    

    This is the simplest decorator version, it basically saves us from writting a = deco(hello_world()). But at this point we can only control the before execution, let's take on the after:

    def deco(f):
        def wrapper(*args, **kwargs):
            print("Rick Sanchez!")
            func = f(*args, **kwargs)
            print("I'm in!")
            return func
        return wrapper
    
    @deco
    def f(word):
        print(word)
    
    a = f("************")
    >>> Rick Sanchez!
        ************
        I'm in!
    

    deco receives a function -> f wrapper receives the arguments -> *args, **kwargs

    wrapper returns the function plus the arguments -> f(*args, **kwargs) deco returns wrapper.

    As you can see we conveniently do things before and after the execution of a given function.

    For example, we could write a decorator that calculates the execution time of a function.

    import time
    def deco(f):
        def wrapper(*args, **kwargs):
            before = time.time()
            func = f(*args, **kwargs)
            after = time.time()
            print(after-before)
            return func
        return wrapper
    
    @deco
    def f():
        time.sleep(2)
        print("************")
    
    a = f()
    >>> 2.0008859634399414
    

    Or create a decorator that executes a function n times.

    def n_times(n):
        def wrapper(f):
            def inner(*args, **kwargs):
                for _ in range(n):
                    func = f(*args, **kwargs)
                return func
            return inner
        return wrapper
    
    @n_times(4)
    def f():
        print("************")
    
    a = f()
    
    >>>************
       ************
       ************
       ************
    

    Write a decorator that calculates the execution time of a function
    Write a script which will determine if a given host is accessible on a given port
    Are you familiar with Dataclasses? Can you explain what are they used for?
    You wrote a class to represent a car. How would you compare two cars instances if two cars are equal if they have the same model and color?
    class Car:
        def __init__(self, model, color):
            self.model = model
            self.color = color
    
        def __eq__(self, other):
            if not isinstance(other, Car):
                return NotImplemented
            return self.model == other.model and self.color == other.color
    
    >> a = Car('model_1', 'red')
    >> b = Car('model_2', 'green')
    >> c = Car('model_1', 'red')
    >> a == b
    False
    >> a == c
    True
    

    Explain Context Manager
    Tell me everything you know about concurrency in Python
    Explain the Buffer Protocol
    Do you have experience with web scraping? Can you describe what have you used and for what?
    Can you implement Linux's tail command in Python? Bonus: implement head as well
    You have created a web page where a user can upload a document. But the function which reads the uploaded files, runs for a long time, based on the document size and user has to wait for the read operation to complete before he/she can continue using the web site. How can you overcome this?
    How yield works exactly?

    Monitoring

    Explain monitoring. What is it? What its goal?

    Google: "Monitoring is one of the primary means by which service owners keep track of a system’s health and availability".

    What is wrong with the old approach of watching for a specific value and trigger an email/phone alert while value is exceeded?

    This approach require from a human to always check why the value exceeded and how to handle it while today, it is more effective to notify people only when they need to take an actual action. If the issue doesn't require any human intervention, then the problem can be fixed by some processes running in the relevant environment.

    What types of monitoring outputs are you familiar with and/or used in the past?

    Alerts
    Tickets
    Logging

    What is the difference between infrastructure monitoring and application monitoring? (methods, tools, ...)

    Prometheus

    What is Prometheus? What are some of Prometheus's main features?
    In what scenarios it might be better to NOT use Prometheus?

    From Prometheus documentation: "if you need 100% accuracy, such as for per-request billing".

    Describe Prometheus architecture and components
    Can you compare Prometheus to other solutions like InfluxDB for example?
    What is an Alert?
    Describe the following Prometheus components:
    • Prometheus server
    • Push Gateway
    • Alert Manager

    Prometheus server is responsible for scraping and storing the data
    Push gateway is used for short-lived jobs
    Alert manager is responsible for alerts ;)

    What is an Instance? What is a Job?
    What core metrics types Prometheus supports?
    What is an exporter? What is it used for?
    Which Prometheus best practices are you familiar with?. Name at least three
    How to get total requests in a given period of time?
    What HA in Prometheus means?
    How do you join two metrics?
    How to write a query that returns the value of a label?
    How do you convert cpu_user_seconds to cpu usage in percentage?

    Go

    What are some characteristics of the Go programming language?
    • Strong and static typing - the type of the variables can't be changed over time and they have to be defined at compile time
    • Simplicity
    • Fast compile times
    • Built-in concurrency
    • Garbage collected
    • Platform independent
    • Compile to standalone binary - anything you need to run your app will be compiled into one binary. Very useful for version management in run-time.

    Go also has good community.

    What is the difference between var x int = 2 and x := 2?

    The result is the same, a variable with the value 2.

    With var x int = 2 we are setting the variable type to integer while with x := 2 we are letting Go figure out by itself the type.

    True or False? In Go we can redeclare variables and once declared we must use it.

    False. We can't redeclare variables but yes, we must used declared variables.

    What libraries of Go have you used?

    This should be answered based on your usage but some examples are:

    • fmt - formatted I/O
    What is the problem with the following block of code? How to fix it?
    func main() {
        var x float32 = 13.5
        var y int
        y = x
    }
    

    The following block of code tries to convert the integer 101 to a string but instead we get "e". Why is that? How to fix it?
    package main
    
    import "fmt"
    
    func main() {
        var x int = 101
        var y string
        y = string(x)
        fmt.Println(y)
    }
    

    It looks what unicode value is set at 101 and uses it for converting the integer to a string. If you want to get "101" you should use the package "strconv" and replace y = string(x) with y = strconv.Itoa(x)

    What is wrong with the following code?:
    package main
    
    func main() {
        var x = 2
        var y = 3
        const someConst = x + y
    }
    

    Constants in Go can only be declared using constant expressions. But x, y and their sum is variable.
    const initializer x + y is not a constant

    What will be the output of the following block of code?:
    package main
    
    import "fmt"
    
    const (
    	x = iota
    	y = iota
    )
    const z = iota
    
    func main() {
    	fmt.Printf("%v\n", x)
    	fmt.Printf("%v\n", y)
    	fmt.Printf("%v\n", z)
    }
    

    Go's iota identifier is used in const declarations to simplify definitions of incrementing numbers. Because it can be used in expressions, it provides a generality beyond that of simple enumerations.
    x and y in the first iota group, z in the second.
    Iota page in Go Wiki

    What _ is used for in Go?

    It avoids having to declare all the variables for the returns values. It is called the blank identifier.
    answer in SO

    What will be the output of the following block of code?:
    package main
    
    import "fmt"
    
    const (
    	_ = iota + 3
    	x
    )
    
    func main() {
    	fmt.Printf("%v\n", x)
    }
    

    Since the first iota is declared with the value 3 ( + 3), the next one has the value 4

    What will be the output of the following block of code?:
    package main
    
    import (
    	"fmt"
    	"sync"
    	"time"
    )
    
    func main() {
    	var wg sync.WaitGroup
    
    	wg.Add(1)
    	go func() {
    		time.Sleep(time.Second * 2)
    		fmt.Println("1")
    		wg.Done()
    	}()
    
    	go func() {
    		fmt.Println("2")
    	}()
    
    	wg.Wait()
    	fmt.Println("3")
    }
    

    Output: 2 1 3

    Aritcle about sync/waitgroup

    Golang package sync

    What will be the output of the following block of code?:
    package main
    
    import (
    	"fmt"
    )
    
    func mod1(a []int) {
    	for i := range a {
    		a[i] = 5
    	}
    
    	fmt.Println("1:", a)
    }
    
    func mod2(a []int) {
    	a = append(a, 125) // !
    
    	for i := range a {
    		a[i] = 5
    	}
    
    	fmt.Println("2:", a)
    }
    
    func main() {
    	s1 := []int{1, 2, 3, 4}
    	mod1(s1)
    	fmt.Println("1:", s1)
    
    	s2 := []int{1, 2, 3, 4}
    	mod2(s2)
    	fmt.Println("2:", s2)
    }
    

    Output:
    1 [5 5 5 5]
    1 [5 5 5 5]
    2 [5 5 5 5 5]
    2 [1 2 3 4]

    In mod1 a is link, and when we're using a[i], we're changing s1 value to. But in mod2, append creats new slice, and we're changing only a value, not s2.

    Aritcle about arrays, Blog post about append

    What will be the output of the following block of code?:
    package main
    
    import (
    	"container/heap"
    	"fmt"
    )
    
    // An IntHeap is a min-heap of ints.
    type IntHeap []int
    
    func (h IntHeap) Len() int           { return len(h) }
    func (h IntHeap) Less(i, j int) bool { return h[i] < h[j] }
    func (h IntHeap) Swap(i, j int)      { h[i], h[j] = h[j], h[i] }
    
    func (h *IntHeap) Push(x interface{}) {
    	// Push and Pop use pointer receivers because they modify the slice's length,
    	// not just its contents.
    	*h = append(*h, x.(int))
    }
    
    func (h *IntHeap) Pop() interface{} {
    	old := *h
    	n := len(old)
    	x := old[n-1]
    	*h = old[0 : n-1]
    	return x
    }
    
    func main() {
    	h := &IntHeap{4, 8, 3, 6}
    	heap.Init(h)
    	heap.Push(h, 7)
    
      fmt.Println((*h)[0])
    }
    

    Output: 3

    Golang container/heap package

    Mongo

    What are the advantages of MongoDB? Or in other words, why choosing MongoDB and not other implementation of NoSQL?

    MongoDB advantages are as followings:

    • Schemaless
    • Easy to scale-out
    • No complex joins
    • Structure of a single object is clear

    What is the difference between SQL and NoSQL?

    The main difference is that SQL databases are structured (data is stored in the form of tables with rows and columns - like an excel spreadsheet table) while NoSQL is unstructured, and the data storage can vary depending on how the NoSQL DB is set up, such as key-value pair, document-oriented, etc.

    In what scenarios would you prefer to use NoSQL/Mongo over SQL?
    • Heterogeneous data which changes often
    • Data consistency and integrity is not top priority
    • Best if the database needs to scale rapidly
    What is a document? What is a collection?
    What is an aggregator?
    What is better? Embedded documents or referenced?
    Have you performed data retrieval optimizations in Mongo? If not, can you think about ways to optimize a slow data retrieval?
    Queries
    Explain this query: db.books.find({"name": /abc/})
    Explain this query: db.books.find().sort({x:1})
    What is the difference between find() and find_one()?
    How can you export data from Mongo DB?
    • mongoexport
    • programming languages

    SQL

    SQL Exercises

    Name Topic Objective & Instructions Solution Comments
    Functions vs. Comparisons Query Improvements Exercise Solution

    SQL Self Assessment

    What is SQL?

    SQL (Structured Query Language) is a standard language for relational databases (like MySQL, MariaDB, ...).
    It's used for reading, updating, removing and creating data in a relational database.

    How is SQL Different from NoSQL

    The main difference is that SQL databases are structured (data is stored in the form of tables with rows and columns - like an excel spreadsheet table) while NoSQL is unstructured, and the data storage can vary depending on how the NoSQL DB is set up, such as key-value pair, document-oriented, etc.

    When is it best to use SQL? NoSQL?

    SQL - Best used when data integrity is crucial. SQL is typically implemented with many businesses and areas within the finance field due to it's ACID compliance.

    NoSQL - Great if you need to scale things quickly. NoSQL was designed with web applications in mind, so it works great if you need to quickly spread the same information around to multiple servers

    Additionally, since NoSQL does not adhere to the strict table with columns and rows structure that Relational Databases require, you can store different data types together.

    Practical SQL - Basics

    For these questions, we will be using the Customers and Orders tables shown below:

    Customers

    Customer_ID Customer_Name Items_in_cart Cash_spent_to_Date
    100204 John Smith 0 20.00
    100205 Jane Smith 3 40.00
    100206 Bobby Frank 1 100.20

    ORDERS

    Customer_ID Order_ID Item Price Date_sold
    100206 A123 Rubber Ducky 2.20 2019-09-18
    100206 A123 Bubble Bath 8.00 2019-09-18
    100206 Q987 80-Pack TP 90.00 2019-09-20
    100205 Z001 Cat Food - Tuna Fish 10.00 2019-08-05
    100205 Z001 Cat Food - Chicken 10.00 2019-08-05
    100205 Z001 Cat Food - Beef 10.00 2019-08-05
    100205 Z001 Cat Food - Kitty quesadilla 10.00 2019-08-05
    100204 X202 Coffee 20.00 2019-04-29
    How would I select all fields from this table?

    Select *
    From Customers;

    How many items are in John's cart?

    Select Items_in_cart
    From Customers
    Where Customer_Name = "John Smith";

    What is the sum of all the cash spent across all customers?

    Select SUM(Cash_spent_to_Date) as SUM_CASH
    From Customers;

    How many people have items in their cart?

    Select count(1) as Number_of_People_w_items
    From Customers
    where Items_in_cart > 0;

    How would you join the customer table to the order table?

    You would join them on the unique key. In this case, the unique key is Customer_ID in both the Customers table and Orders table

    How would you show which customer ordered which items?

    Select c.Customer_Name, o.Item
    From Customers c
    Left Join Orders o
    On c.Customer_ID = o.Customer_ID;

    Using a with statement, how would you show who ordered cat food, and the total amount of money spent?

    with cat_food as (
    Select Customer_ID, SUM(Price) as TOTAL_PRICE
    From Orders
    Where Item like "%Cat Food%"
    Group by Customer_ID
    )
    Select Customer_name, TOTAL_PRICE
    From Customers c
    Inner JOIN cat_food f
    ON c.Customer_ID = f.Customer_ID
    where c.Customer_ID in (Select Customer_ID from cat_food);

    Although this was a simple statement, the "with" clause really shines when a complex query needs to be run on a table before joining to another. With statements are nice, because you create a pseudo temp when running your query, instead of creating a whole new table.

    The Sum of all the purchases of cat food weren't readily available, so we used a with statement to create the pseudo table to retrieve the sum of the prices spent by each customer, then join the table normally.

    Which of the following queries would you use?
    SELECT count(*)                             SELECT count(*)
    FROM shawarma_purchases                     FROM shawarma_purchases
    WHERE                               vs.     WHERE
      YEAR(purchased_at) == '2017'              purchased_at >= '2017-01-01' AND
                                                purchased_at <= '2017-31-12'
    

    SELECT count(*)
    FROM shawarma_purchases
    WHERE
      purchased_at >= '2017-01-01' AND
      purchased_at <= '2017-31-12'
    

    When you use a function (YEAR(purchased_at)) it has to scan the whole database as opposed to using indexes and basically the column as it is, in its natural state.

    Azure

    What is Azure Portal?

    Microsoft Docs: "The Azure portal is a web-based, unified console that provides an alternative to command-line tools. With the Azure portal, you can manage your Azure subscription by using a graphical user interface."

    What is Azure Marketplace?

    Microsoft Docs: "Azure marketplace helps connect users with Microsoft partners, independent software vendors, and startups that are offering their solutions and services, which are optimized to run on Azure."

    Explain availability sets and availability zones

    An availability set is a logical grouping of VMs that allows Azure to understand how your application is built to provide redundancy and availability. It is recommended that two or more VMs are created within an availability set to provide for a highly available application and to meet the 99.95% Azure SLA.

    What is Azure Policy?
    What is the Azure Resource Manager? Can you describe the format for ARM templates?
    Explain Azure managed disks

    Azure - Compute

    What Azure compute services are you familiar with?
    • Azure Virtual Machines
    • Azure Batch
    • Azure Service Fabric
    • Azure Container Instances
    • Azure Virtual Machine Scale Set?s
    What "Azure Virtual Machines" service is used for?

    Windows or Linux virtual machines

    What "Azure Virtual Machine Scale Sets" service is used for?

    Scaling Linux or Windows virtual machines used in Azure

    What "Azure Functions" service is used for?

    Azure Functions is the serverless compute service of Azure.

    What "Azure Container Instances" service is used for?

    Running containerized applications (without the need to provision virtual machines).

    What "Azure Batch" service is used for?

    Running parallel and high-performance computing applications

    What "Azure Service Fabric" service is used for?
    What "Azure Kubernetes" service is used for?

    Azure - Network

    What Azure network services are you familiar with?
    What's an Azure region?
    What is the N-tier architecture?

    Azure Storage

    What Azure storage services are you familiar with?
    What storage options Azure supports?

    Azure Security

    What is the Azure Security Center? What are some of its features?

    It's a monitoring service that provides threat protection across all of the services in Azure. More specifically, it:

    • Provides security recommendations based on your usage
    • Monitors security settings and continuously all the services
    • Analyzes and identifies potential inbound attacks
    • Detects and blocks malware using machine learning
    What is Azure Active Directory?

    Azure AD is a cloud-based identity service. You can use it as a standalone service or integrate it with existing Active Directory service you already running.

    What is Azure Advanced Threat Protection?
    What components are part of Azure ATP?
    Where logs are stored in Azure Monitor?
    Explain Azure Site Recovery
    Explain what the advisor does
    Explain VNet peering
    Which protocols are available for configuring health probe
    Explain Azure Active
    What is a subscription? What types of subscriptions are there?
    Explain what is a blob storage service

    GCP

    Explain GCP's architecture
    What are the main components and services of GCP?
    What GCP management tools are you familiar with?
    Tell me what do you know about GCP networking
    Explain Cloud Functions
    What is Cloud Datastore?
    What network tags are used for?
    What are flow logs? Where are they enabled?
    How do you list buckets?
    What Compute metadata key allows you to run code at startup?

    startap-script

    What the following commands does? `gcloud deployment-manager deployments create`
    What is Cloud Code?
    It is a set of tools to help developers write, run and debug GCP kubernetes based applications. It provides built-in support for rapid iteration, debugging and running applications in development and production K8s environments.

    Google Kubernetes Engine (GKE)

    What is GKE
    • It is the managed kubernetes service on GCP for deploying, managing and scaling containerised applications using Google infrastructure.

    Anthos

    What is Anthos
    It is a managed application platform for organisations like enterprises that require quick modernisation and certain levels of consistency for their legacy applications in a hybrid or multicloud world. From this explanation the core ideas can be drawn from these statements;
    • Managed -> the customer does not need to worry about the underlying software intergrations, they just enable the API.
    • application platform -> It consists of open source tools like K8s, Knative, Istio and Tekton
    • Enterprises -> these are usually organisations with complex needs
    • Consistency -> to have the same policies declaratively initiated to be run anywhere securely e.g on-prem, GCP or other-clouds (AWS or Azure)

    fun fact: Anthos is flower in greek, they grow in the ground (earth) but need rain from the clouds to flourish.

    List the technical components that make up Anthos
    • Infrastructure management - Google Kubernetes Engine (GKE)
    • Cluster management - GKE, Ingress for Anthos
    • Service management - Anthos Service Mesh
    • Policy enforcement - Anthos Config Management, Anthos Enterprise Data Protection, Policy Controller
    • Application deployment - CI/CD tools like Cloud Build, GitLab
    • Application development - Cloud Code
    What is the primary computing environment for Anthos to easily manage workload deployment?
    • Google Kubernetes Engine (GKE)
    How does Anthos handle the control plane and node components for GKE?

    On GCP the kubernetes api-server is the only control plane component exposed to customers whilst compute engine manages instances in the project.

    Which load balancing options are available?
    • Networking load balancing for L4 and HTTP(S) Load Balancing for L7 which are both managed services that do not require additional configuration.
    • Ingress for Anthos which allows the ability to deploy a load balancer that serves an application across multiple clusters on GKE
    Can you deploy Anthos on AWS?
    • Yes, Anthos on AWS is now GA. For more read here
    List and explain the enterprise security capabilities provided by Anthos
    • Control plane security - GCP manages and maintains the K8s control plane out of the box. The user can secure the api-server by using master authorized networks and private clusters. These allow the user to disable access on the public IP address by assigning a private IP address to the master.
    • Node security - By default workloads are provisioned on Compute engine instances that use Google's Container Optimised OS. This operating system implements a locked-down firewall, limited user accounts with root disabled and a read-only filesystem. There is a further option to enable GKE Sandbox for stronger isolation in multi-tenant deployment scenarios.
    • Network security - Within a created cluster VPC, Anthos GKE leverages a powerful software-defined network that enables simple Pod-to-Pod communications. Network policies allow locking down ingress and egress connections in a given namespace. Filtering can also be implemented to incoming load-balanced traffic for services that require external access, by supplying whitelisted CIDR IP ranges.
    • Workload security - Running workloads run with limited privileges, default Docker AppArmor security policies are applied to all Kubernetes Pods. Workload identity for Anthos GKE aligns with the open source kubernetes service accounts with GCP service account permissions.
    • Audit logging - Adminstrators are given a way to retain, query, process and alert on events of the deployed environments.
    How can workloads deployed on Anthos GKE on-prem clusters securely connect to Google Cloud services?
    • Google Cloud Virtual Private Network (Cloud VPN) - this is for secure networking
    • Google Cloud Key Management Service (Cloud KMS) - for key management
    What is Island Mode configuration with regards to networking in Anthos GKE deployed on-prem?
    • This is when pods can directly talk to each other within a cluster, but cannot be reached from outside the cluster thus forming an "island" within the network that is not connected to the external network.
    Explain Anthos Config Management

    It is a core component of the Anthos stack which provides platform, service and security operators with a single, unified approach to multi-cluster management that spans both on-premises and cloud environments. It closely follows K8s best practices, favoring declarative approaches over imperative operations, and actively monitors cluster state and applies the desired state as defined in Git. It includes three key components as follows:

    1. An importer that reads from a central Git repository
    2. A component that synchronises stored configuration data into K8s objects
    3. A component that monitors drift between desired and actual cluster configurations with a capability of reconciliation when need rises.
    How does Anthos Config Management help?

    It follows common modern software development practices which makes cluster configuration, management and policy changes auditable, revertable, and versionable easily enforcing IT governance and unifying resource management in an organisation.

    What is Anthos Service Mesh?
    • It is a suite of tools that assist in monitoring and managing deployed services on Anthos of all shapes and sizes whether running in cloud, hybrid or multi-cloud environments. It leverages the APIs and core components from Istio, a highly configurable and open-source service mesh platform.
    Describe the two main components of Anthos Service Mesh
    1. Data plane - it consists of a set of distributed proxies that mediate all inbound and outbound network traffic between individual services which are configured using a centralised control plane and an open API
    2. Control plane - is a fully managed offering outside of Anthos GKE clusters to simplify management overhead and ensure highest possible availability.
    What are the components of the managed control plane of Anthos Service Mesh?
    1. Traffic Director - it is GCP's fully managed service mesh traffic control plane, responsible for translating Istio API objects into configuration information for the distributed proxies, as well as directing service mesh ingress and egress traffic
    2. Managed CA - is a centralised certificate authority responsible for providing SSL certificates to each of the distributed proxies, authentication information and distributing secrets
    3. Operations tooling - formerly stackdriver, provides a managed ingestion point for observability and telemetry, specifically monitoring, tracing and logging data generated by each of the proxies. This powers the observability dashboard for operators to visually inspect their services and service dependencies assisting in the implementation of SRE best practices for monitoring SLIs and establishing SLOs.
    How does Anthos Service Mesh help?
    Tool and technology integration that makes up Anthos service mesh delivers signficant operational benefits to Anthos environments, with minimal additional overhead such as follows:
    • Uniform observability - the data plane reports service to service communication back to the control plane generating a service dependency graph. Traffic inspection by the proxy inserts headers to facilitate distributed tracing, capturing and reporting service logs together with service-level metrics (i.e latency, errors, availability).
    • Operational agility - fine-grained controls for managing the flow of inter-mesh (north-south) and intra-mesh (east-west) traffic are provided.
    • Policy-driven security - policies can be enforced consistently across diverse protocols and runtimes as service communications are secured by default.
    List possible use cases of traffic controls that can be implemented within Anthos Service Mesh
    • Traffic splitting across differing service versions for canary or A/B testing
    • Circuit breaking to prevent cascading failures
    • Fault injection to help build resilient and fault-tolerant deployments
    • HTTP header-based traffic steering between individual services or versions
    What is Cloud Run for Anthos?

    It is part of the Anthos stack that brings a serverless container experience to Anthos, offering a high-level platform experience on top of K8s clusters. It is built with Knative, an open-source operator for K8s that brings serverless application serving and eventing capabilities.

    How does Cloud Run for Anthos simplify operations?

    Platform teams in organisations that wish to offer developers additional tools to test, deploy and run applications can use Knative to enhance this experience on Anthos as Cloud Run. Below are some of the benefits;

    • Easy migration from K8s deployments - Without Cloud Run, platform engineers have to configure deployment, service, and HorizontalPodAutoscalers(HPA) objects to a loadbalancer and autoscaling. If application is already serving traffic it becomes hard to change configurations or roll back efficiently. Using Cloud Run all this is managed thus the Knative service manifest describes the application to be autoscaled and loadbalanced
    • Autoscaling - a sudden traffic spike may cause application containers in K8s to crash due to overload thus an efficient automated autoscaling is executed to serve the high volume of traffic
    • Networking - it has built-in load balancing capabilities and policies for traffic splitting between multiple versions of an application.
    • Releases and rollouts - supports the notion of the Knatibe API's revisions which describe new versions or different configurations of your application and canary deployments by splitting traffic.
    • Monitoring - observing and recording metrics such as latency, error rate and requests per second.
    List and explain three high-level out of the box autoscaling primitives offered by Cloud Run for Anthos that do not exist in K8s natively
    • Rapid, request-based autoscaling - default autoscalers monitor request metrics which allows Cloud Run for Anthos to handle spiky traffic patterns smoothly
    • Concurrency controls - limits such as max in-flight requests per container are enforced to ensure the container does not become overloaded and crash. More containers are added to handle the spiky traffic, buffering the requests.
    • Scale to zero - if an application is inactive for a while Cloud Run scales it down to zero to reduce its footprint. Alternatively one can turn off scale-to-zero to prevent cold starts.
    List some Cloud Run for Anthos use cases

    As it does not support stateful applications or sticky sessions, it is suitable for running stateless applications such as:

    • Machine learning model predictions e.g Tensorflow serving containers
    • API gateways, API middleware, web front ends and Microservices
    • Event handlers, ETL

    OpenStack

    What components/projects of OpenStack are you familiar with?
    Can you tell me what each of the following services/projects is responsible for?:
    • Nova
    • Neutron
    • Cinder
    • Glance
    • Keystone

    • Nova - Manage virtual instances
    • Cinder - Block Storage
    • Keystone - Authentication service across the cloud
    Identify the service/project used for each of the following:
    • Copy or snapshot instances
    • GUI for viewing and modifying resources
    • Block Storage
    • Manage virtual instances

    • Glance - Images Service. Also used for copying or snapshot instances
    • Horizon - GUI for viewing and modifying resources
    • Cinder - Block Storage
    • Nova - Manage virtual instances
    What is a tenant/project?
    Determine true or false:
    • OpenStack is free to use
    • The service responsible for networking is Glance
    • The purpose of tenant/project is to share resources between different projects and users of OpenStack

    Describe in detail how you bring up an instance with a floating IP
    You get a call from a customer saying: "I can ping my instance but can't connect (ssh) it". What might be the problem?
    What types of networks OpenStack supports?
    How do you debug OpenStack storage issues? (tools, logs, ...)
    How do you debug OpenStack compute issues? (tools, logs, ...)

    OpenStack Deployment & TripleO

    Have you deployed OpenStack in the past? If yes, can you describe how you did it?
    Are you familiar with TripleO? How is it different from Devstack or Packstack?

    You can read about TripleO right here

    OpenStack Compute

    Can you describe Nova in detail?
    • Used to provision and manage virtual instances
    • It supports Multi-Tenancy in different levels - logging, end-user control, auditing, etc.
    • Highly scalable
    • Authentication can be done using internal system or LDAP
    • Supports multiple types of block storage
    • Tries to be hardware and hypervisor agnostice
    What do you know about Nova architecture and components?
    • nova-api - the server which serves metadata and compute APIs
    • the different Nova components communicate by using a queue (Rabbitmq usually) and a database
    • a request for creating an instance is inspected by nova-scheduler which determines where the instance will be created and running
    • nova-compute is the component responsible for communicating with the hypervisor for creating the instance and manage its lifecycle

    OpenStack Networking (Neutron)

    Explain Neutron in detail
    • One of the core component of OpenStack and a standalone project
    • Neutron focused on delivering networking as a service
    • With Neutron, users can set up networks in the cloud and configure and manage a variety of network services
    • Neutron interacts with:
      • Keystone - authorize API calls
      • Nova - nova communicates with neutron to plug NICs into a network
      • Horizon - supports networking entities in the dashboard and also provides topology view which includes networking details
    Explain each of the following components:
    • neutron-dhcp-agent
    • neutron-l3-agent
    • neutron-metering-agent
    • neutron-*-agtent
    • neutron-server

    • neutron-l3-agent - L3/NAT forwarding (provides external network access for VMs for example)
    • neutron-dhcp-agent - DHCP services
    • neutron-metering-agent - L3 traffic metering
    • neutron-*-agtent - manages local vSwitch configuration on each compute (based on chosen plugin)
    • neutron-server - exposes networking API and passes requests to other plugins if required
    Explain these network types:
    • Management Network
    • Guest Network
    • API Network
    • External Network

    • Management Network - used for internal communication between OpenStack components. Any IP address in this network is accessible only within the datacetner
    • Guest Network - used for communication between instances/VMs
    • API Network - used for services API communication. Any IP address in this network is publicly accessible
    • External Network - used for public communication. Any IP address in this network is accessible by anyone on the internet
    In which order should you remove the following entities:
    • Network
    • Port
    • Router
    • Subnet

    • Port
    • Subnet
    • Router
    • Network

    There are many reasons for that. One for example: you can't remove router if there are active ports assigned to it.

    What is a provider network?
    What components and services exist for L2 and L3?
    What is the ML2 plug-in? Explain its architecture
    What is the L2 agent? How does it works and what is it responsible for?
    What is the L3 agent? How does it works and what is it responsible for?
    Explain what the Metadata agent is responsible for
    What networking entities Neutron supports?
    How do you debug OpenStack networking issues? (tools, logs, ...)

    OpenStack - Glance

    Explain Glance in detail
    • Glance is the OpenStack image service
    • It handles requests related to instances disks and images
    • Glance also used for creating snapshots for quick instances backups
    • Users can use Glance to create new images or upload existing ones
    Describe Glance architecture
    • glance-api - responsible for handling image API calls such as retrieval and storage. It consists of two APIs: 1. registry-api - responsible for internal requests 2. user API - can be accessed publicly
    • glance-registry - responsible for handling image metadata requests (e.g. size, type, etc). This component is private which means it's not available publicly
    • metadata definition service - API for custom metadata
    • database - for storing images metadata
    • image repository - for storing images. This can be a filesystem, swift object storage, HTTP, etc.

    OpenStack - Swift

    Explain Swift in detail
    • Swift is Object Store service and is an highly available, distributed and consistent store designed for storing a lot of data
    • Swift is distributing data across multiple servers while writing it to multiple disks
    • One can choose to add additional servers to scale the cluster. All while swift maintaining integrity of the information and data replications.
    Can users store by default an object of 100GB in size?

    Not by default. Object Storage API limits the maximum to 5GB per object but it can be adjusted.

    Explain the following in regards to Swift:
    • Container
    • Account
    • Object

    • Container - Defines a namespace for objects.
    • Account - Defines a namespace for containers
    • Object - Data content (e.g. image, document, ...)
    True or False? there can be two objects with the same name in the same container but not in two different containers

    False. Two objects can have the same name if they are in different containers.

    OpenStack - Cinder

    Explain Cinder in detail
    • Cinder is OpenStack Block Storage service
    • It basically provides used with storage resources they can consume with other services such as Nova
    • One of the most used implementations of storage supported by Cinder is LVM
    • From user perspective this is transparent which means the user doesn't know where, behind the scenes, the storage is located or what type of storage is used
    Describe Cinder's components
    • cinder-api - receives API requests
    • cinder-volume - manages attached block devices
    • cinder-scheduler - responsible for storing volumes

    OpenStack - Keystone

    Can you describe the following concepts in regards to Keystone?
    • Role
    • Tenant/Project
    • Service
    • Endpoint
    • Token

    • Role - A list of rights and privileges determining what a user or a project can perform
    • Tenant/Project - Logical representation of a group of resources isolated from other groups of resources. It can be an account, organization, ...
    • Service - An endpoint which the user can use for accessing different resources
    • Endpoint - a network address which can be used to access a certain OpenStack service
    • Token - Used for access resources while describing which resources can be accessed by using a scope
    What are the properties of a service? In other words, how a service is identified?

    Using:

    • Name
    • ID number
    • Type
    • Description
    Explain the following: - PublicURL - InternalURL - AdminURL
    • PublicURL - Publicly accessible through public internet
    • InternalURL - Used for communication between services
    • AdminURL - Used for administrative management
    What is a service catalog?

    A list of services and their endpoints

    OpenStack Advanced - Services

    Describe each of the following services
    • Swift
    • Sahara
    • Ironic
    • Trove
    • Aodh
    • Ceilometer

    • Swift - highly available, distributed, eventually consistent object/blob store
    • Sahara - Manage Hadoop Clusters
    • Ironic - Bare Metal Provisioning
    • Trove - Database as a service that runs on OpenStack
    • Aodh - Alarms Service
    • Ceilometer - Track and monitor usage
    Identify the service/project used for each of the following:
    • Database as a service which runs on OpenStack
    • Bare Metal Provisioning
    • Track and monitor usage
    • Alarms Service
    • Manage Hadoop Clusters
    • highly available, distributed, eventually consistent object/blob store

    • Database as a service which runs on OpenStack - Trove
    • Bare Metal Provisioning - Ironic
    • Track and monitor usage - Ceilometer
    • Alarms Service - Aodh
    • Manage Hadoop Clusters
    • Manage Hadoop Clusters - Sahara
    • highly available, distributed, eventually consistent object/blob store - Swift

    OpenStack Advanced - Keystone

    Can you describe Keystone service in detail?
    • You can't have OpenStack deployed without Keystone
    • It Provides identity, policy and token services
      • The authentication provided is for both users and services
      • The authorization supported is token-based and user-based.
    • There is a policy defined based on RBAC stored in a JSON file and each line in that file defines the level of access to apply
    Describe Keystone architecture
    • There is a service API and admin API through which Keystone gets requests
    • Keystone has four backends:
      • Token Backend - Temporary Tokens for users and services
      • Policy Backend - Rules management and authorization
      • Identity Backend - users and groups (either standalone DB, LDAP, ...)
      • Catalog Backend - Endpoints
    • It has pluggable environment where you can integrate with:
      • LDAP
      • KVS (Key Value Store)
      • SQL
      • PAM
      • Memcached
    Describe the Keystone authentication process
    • Keystone gets a call/request and checks whether it's from an authorized user, using username, password and authURL
    • Once confirmed, Keystone provides a token.
    • A token contains a list of user's projects so there is no to authenticate every time and a token can submitted instead

    OpenStack Advanced - Compute (Nova)

    What each of the following does?:
    • nova-api
    • nova-compuate
    • nova-conductor
    • nova-cert
    • nova-consoleauth
    • nova-scheduler

    • nova-api - responsible for managing requests/calls
    • nova-compute - responsible for managing instance lifecycle
    • nova-conductor - Mediates between nova-compute and the database so nova-compute doesn't access it directly
    What types of Nova proxies are you familiar with?
    • Nova-novncproxy - Access through VNC connections
    • Nova-spicehtml5proxy - Access through SPICE
    • Nova-xvpvncproxy - Access through a VNC connection

    OpenStack Advanced - Networking (Neutron)

    Explain BGP dynamic routing
    What is the role of network namespaces in OpenStack?

    OpenStack Advanced - Horizon

    Can you describe Horizon in detail?
    • Django-based project focusing on providing an OpenStack dashboard and the ability to create additional customized dashboards
    • You can use it to access the different OpenStack services resources - instances, images, networks, ...
      • By accessing the dashboard, users can use it to list, create, remove and modify the different resources
    • It's also highly customizable and you can modify or add to it based on your needs
    What can you tell about Horizon architecture?
    • API is backward compatible
    • There are three type of dashboards: user, system and settings
    • It provides core support for all OpenStack core projects such as Neutron, Nova, etc. (out of the box, no need to install extra packages or plugins)
    • Anyone can extend the dashboards and add new components
    • Horizon provides templates and core classes from which one can build its own dashboard

    Puppet

    What is Puppet? How does it works?
    Explain Puppet architecture
    Can you compare Puppet to other configuration management tools? Why did you chose to use Puppet?
    Explain the following:
    • Module
    • Manifest
    • Node

    Explain Facter
    What is MCollective?
    Do you have experience with writing modules? Which module have you created and for what?
    Explain what is Hiera

    Elastic

    What is the Elastic Stack?

    The Elastic Stack consists of:

    • Elasticsearch
    • Kibana
    • Logstash
    • Beats
    • Elastic Hadoop
    • APM Server

    Elasticserach, Logstash and Kibana are also known as the ELK stack.

    Explain what is Elasticsearch

    From the official docs:

    "Elasticsearch is a distributed document store. Instead of storing information as rows of columnar data, Elasticsearch stores complex data structures that have been serialized as JSON documents"

    What is Logstash?

    From the blog:

    "Logstash is a powerful, flexible pipeline that collects, enriches and transports data. It works as an extract, transform & load (ETL) tool for collecting log messages."

    Explain what beats are

    Beats are lightweight data shippers. These data shippers installed on the client where the data resides. Examples of beats: Filebeat, Metricbeat, Auditbeat. There are much more.

    What is Kibana?

    From the official docs:

    "Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. You use Kibana to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualize your data in a variety of charts, tables, and maps."

    Describe what happens from the moment an app logged some information until it's displayed to the user in a dashboard when the Elastic stack is used

    The process may vary based on the chosen architecture and the processing you may want to apply to the logs. One possible workflow is:

    1. The data logged by the application is picked by filebeat and sent to logstash
    2. Logstash process the log based on the defined filters. Once done, the output is sent to Elasticsearch
    3. Elasticsearch stores the document it got and the document is indexed for quick future access
    4. The user creates visualizations in Kibana which based on the indexed data
    5. The user creates a dashboard which composed out of the visualization created in the previous step
    Elasticsearch
    What is a data node?

    This is where data is stored and also where different processing takes place (e.g. when you search for a data).

    What is a master node?

    Par of a master node responsibilites:

    • Track the status of all the nodes in the cluster
    • Verify replicas are working and the data is available from every data node.
    • No hot nodes (no data node that works much harder than other nodes)

    While there can be multiple master nodes in reality only of them is the elected master node.

    What is an ingest node?

    A node which responsible for parsing the data. In case you don't use logstash then this node can recieve data from beats and parse it, similarly to how it can be parsed in Logstash.

    What is Coordinating node?

    A Coordinating node responsible for routing requests out and in to the cluser (data nodes).

    How data is stored in elasticsearch?
    • Data is stored in an index
    • The index is spread across the cluster using shards
    What is an Index?

    Index in Elastic is in most cases compared to a whole database from the SQL/NoSQL world.
    You can choose to have one index to hold all the data of your app or have multiple indices where each index holds different type of your app (e.g. index for each service your app is running).

    The official docs also offer a great explanation (in general, it's really good documentation, as every project should have):

    "An index can be thought of as an optimized collection of documents and each document is a collection of fields, which are the key-value pairs that contain your data"

    Explain Shards

    An index is split into shards and documents are hashed to a particular shard. Each shard may be on a different node in a cluster and each one of the shards is a self contained index.
    This allows Elasticsearch to scale to an entire cluster of servers.

    What is an Inverted Index?

    From the official docs:

    "An inverted index lists every unique word that appears in any document and identifies all of the documents each word occurs in."

    What is a Document?

    Continuing with the comparison to SQL/NoSQL a Document in Elastic is a row in table in the case of SQL or a document in a collection in the case of NoSQL. As in NoSQL a Document is a JSON object which holds data on a unit in your app. What is this unit depends on the your app. If your app related to book then each document describes a book. If you are app is about shirts then each document is a shirt.

    You check the health of your elasticsearch cluster and it's red. What does it mean? What can cause the status to be yellow instead of green?

    Red means some data is unavailable. Yellow can be caused by running single node cluster instead of multi-node.

    True or False? Elasticsearch indexes all data in every field and each indexed field has the same data structure for unified and quick query ability

    False. From the official docs:

    "Each indexed field has a dedicated, optimized data structure. For example, text fields are stored in inverted indices, and numeric and geo fields are stored in BKD trees."

    What reserved fields a document has?
    • _index
    • _id
    • _type
    Explain Mapping
    What are the advantages of defining your own mapping? (or: when would you use your own mapping?)
    • You can optimize fields for partial matching
    • You can define custom formats of known fields (e.g. date)
    • You can perform language-specific analysis
    Explain Replicas

    In a network/cloud environment where failures can be expected any time, it is very useful and highly recommended to have a failover mechanism in case a shard/node somehow goes offline or disappears for whatever reason. To this end, Elasticsearch allows you to make one or more copies of your index’s shards into what are called replica shards, or replicas for short.

    Can you explain Term Frequency & Document Frequency?

    Term Frequency is how often a term appears in a given document and Document Frequency is how often a term appears in all documents. They both are used for determining the relevance of a term by calculating Term Frequency / Document Frequency.

    You check "Current Phase" under "Index lifecycle management" and you see it's set to "hot". What does it mean?

    "The index is actively being written to". More about the phases here

    What this command does? curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'{ "name": "John Doe" }'

    It creates customer index if it doesn't exists and adds a new document with the field name which is set to "John Dow". Also, if it's the first document it will get the ID 1.

    What will happen if you run the previous command twice? What about running it 100 times?
    1. If name value was different then it would update "name" to the new value
    2. In any case, it bumps version field by one
    What is the Bulk API? What would you use it for?

    Bulk API is used when you need to index multiple documents. For high number of documents it would be significantly faster to use rather than individual requests since there are less network roundtrips.

    Query DSL
    Explain Elasticsearch query syntax (Booleans, Fields, Ranges)
    Explain what is Relevance Score
    Explain Query Context and Filter Context

    From the official docs:

    "In the query context, a query clause answers the question “How well does this document match this query clause?” Besides deciding whether or not the document matches, the query clause also calculates a relevance score in the _score meta-field."

    "In a filter context, a query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No — no scores are calculated. Filter context is mostly used for filtering structured data"

    Describe how would an architecture of production environment with large amounts of data would be different from a small-scale environment

    There are several possible answers for this question. One of them is as follows:

    A small-scale architecture of elastic will consist of the elastic stack as it is. This means we will have beats, logstash, elastcsearch and kibana.
    A production environment with large amounts of data can include some kind of buffering component (e.g. Reddis or RabbitMQ) and also security component such as Nginx.

    Logstash
    What are Logstash plugins? What plugins types are there?
    • Input Plugins - how to collect data from different sources
    • Filter Plugins - processing data
    • Output Plugins - push data to different outputs/services/platforms
    What is grok?

    A logstash plugin which modifies information in one format and immerse it in another.

    How grok works?
    What grok patterns are you familiar with?
    What is `_grokparsefailure?`
    How do you test or debug grok patterns?
    What are Logstash Codecs? What codecs are there?
    Kibana
    What can you find under "Discover" in Kibana?

    The raw data as it is stored in the index. You can search and filter it.

    You see in Kibana, after clicking on Discover, "561 hits". What does it mean?

    Total number of documents matching the search results. If not query used then simply the total number of documents.

    What can you find under "Visualize"?

    "Visualize" is where you can create visual representations for your data (pie charts, graphs, ...)

    What visualization types are supported/included in Kibana?
    What visualization type would you use for statistical outliers
    Describe in detail how do you create a dashboard in Kibana

    Filebeat

    What is Filebeat?
    If one is using ELK, is it a must to also use filebeat? In what scenarios it's useful to use filebeat?
    What is a harvester?

    Read here

    True or False? a single harvester harvest multiple files, according to the limits set in filebeat.yml

    False. One harvester harvests one file.

    What are filebeat modules?

    Elastic Stack

    How do you secure an Elastic Stack?

    You can generate certificates with the provided elastic utils and change configuration to enable security using certificates model.

    Distributed

    Explain Distributed Computing (or Distributed System)

    According to Martin Kleppmann:

    "Many processes running on many machines...only message-passing via an unreliable network with variable delays, and the system may suffer from partial failures, unreliable clocks, and process pauses."

    Another definition: "Systems that are physically separated, but logically connected"

    What can cause a system to fail?
    • Network
    • CPU
    • Memory
    • Disk
    Do you know what is "CAP theorem"? (aka as Brewer's theorem)

    According to the CAP theorem, it's not possible for a distributed data store to provide more than two of the following at the same time:

    • Availability: Every request receives a response (it doesn't has to be the most recent data)
    • Consistency: Every request receives a response with the latest/most recent data
    • Partition tolerance: Even if some the data is lost/dropped, the system keeps running
    What are the problems with the following design? How to improve it?

    1. The transition can take time. In other words, noticeable downtime. 2. Standby server is a waste of resources - if first application server is running then the standby does nothing
    What are the problems with the following design? How to improve it?

    Issues: If load balancer dies , we lose the ability to communicate with the application.

    Ways to improve:

    • Add another load balancer
    • Use DNS A record for both load balancers
    • Use message queue
    What is "Shared-Nothing" architecture?

    It's an architecture in which data is and retrieved from a single, non-shared, source usually exclusively connected to one node as opposed to architectures where the request can get to one of many nodes and the data will be retrieved from one shared location (storage, memory, ...).

    Explain the Sidecar Pattern (Or sidecar proxy)

    Misc

    Name Topic Objective & Instructions Solution Comments
    Highly Available "Hello World" Exercise Solution
    What happens when you type in a URL in an address bar in a browser?
    1. The browser searches for the record of the domain name IP address in the DNS in the following order:
    • Browser cache
    • Operating system cache
    • The DNS server configured on the user's system (can be ISP DNS, public DNS, ...)
    1. If it couldn't find a DNS record locally, a full DNS resolution is started.
    2. It connects to the server using the TCP protocol
    3. The browser sends an HTTP request to the server
    4. The server sends an HTTP response back to the browser
    5. The browser renders the response (e.g. HTML)
    6. The browser then sends subsequent requests as needed to the server to get the embedded links, javascript, images in the HTML and then steps 3 to 5 are repeated.

    TODO: add more details!

    API

    Explain what is an API

    I like this definition from blog.christianposta.com:

    "An explicitly and purposefully defined interface designed to be invoked over a network that enables software developers to get programmatic access to data and functionality within an organization in a controlled and comfortable way."

    What is an API specification?

    From swagger.io:

    "An API specification provides a broad understanding of how an API behaves and how the API links with other APIs. It explains how the API functions and the results to expect when using the API"

    True or False? API Definition is the same as API Specification

    False. From swagger.io:

    "An API definition is similar to an API specification in that it provides an understanding of how an API is organized and how the API functions. But the API definition is aimed at machine consumption instead of human consumption of APIs."

    What is a Payload in API?
    What is Automation? How it's related or different from Orchestration?

    Automation is the act of automating tasks to reduce human intervention or interaction in regards to IT technology and systems.
    While automation focuses on a task level, Orchestration is the process of automating processes and/or workflows which consists of multiple tasks that usually across multiple systems.

    Tell me about interesting bugs you've found and also fixed
    What is a Debuggger and how it works?
    What services an application might have?
    • Authorization
    • Logging
    • Authentication
    • Ordering
    • Front-end
    • Back-end ...
    What is Metadata?

    Data about data. Basically, it describes the type of information that an underlying data will hold.

    You can use one of the following formats: JSON, YAML, XML. Which one would you use? Why?

    I can't answer this for you :)

    What's KPI?
    What's OKR?
    What's DSL (Domain Specific Language)?

    Domain Specific Language (DSLs) are used to create a customised language that represents the domain such that domain experts can easily interpret it.

    What's the difference between KPI and OKR?

    YAML

    What is YAML?

    Data serialization language used by many technologies today like Kubernetes, Ansible, etc.

    True or False? Any valid JSON file is also a valid YAML file

    True. Because YAML is superset of JSON.

    What is the format of the following data?
    {
        applications: [
            {
                name: "my_app",
                language: "python",
                version: 20.17
            }
        ]
    }
    

    JSON
    What is the format of the following data?
    applications:
      - app: "my_app"
        language: "python"
        version: 20.17
    

    YAML
    How to write a multi-line string with YAML? What use cases is it good for?
    someMultiLineString: |
      look mama
      I can write a multi-line string
      I love YAML
    

    It's good for use cases like writing a shell script where each line of the script is a different command.

    What is the difference between someMultiLineString: | to someMultiLineString: >?

    using > will make the multi-line string to fold into a single line

    someMultiLineString: >
      This is actually
      a single line
      do not let appearances fool you
    

    What are placeholders in YAML?

    They allow you reference values instead of directly writing them and it is used like this:

    username: {{ my.user_name }}
    

    How can you define multiple YAML components in one file?

    Using this: --- For Examples:

    document_number: 1
    ---
    document_number: 2
    

    Firmware

    Explain what is a firmware

    Wikipedia: "In computing, firmware is a specific class of computer software that provides the low-level control for a device's specific hardware. Firmware, such as the BIOS of a personal computer, may contain basic functions of a device, and may provide hardware abstraction services to higher-level software such as operating systems."

    Customers and Service Providers

    What is SLO (service-level objective)?
    What is SLA (service-level agreement)?

    Jira

    Explain/Demonstrate the following types in Jira:
    • Epic
    • Story
    • Task

    What is a project in Jira?

    Kafka

    What is Kafka?
    In Kafka, how to automatically balance brokers leadership of partitions in a cluster?
    • Enable auto leader election and reduce the imbalance percentage ratio
    • Manually rebalance by using kafkat
    • Configure group.initial.rebalance.delay.ms to 3000
    • All of the above

    Cassandra

    When running a cassandra cluster, how often do you need to run nodetool repair in order to keep the cluster consistent?
    • Within the columnFamily GC-grace Once a week
    • Less than the compacted partition minimum bytes
    • Depended on the compaction strategy

    HTTP

    What is HTTP?
    Describe HTTP request lifecycle
    • Resolve host by request to DNS resolver
    • Client SYN
    • Server SYN+ACK
    • Client SYN
    • HTTP request
    • HTTP response
    True or False? HTTP is stateful

    False. It doesn't maintain state for incoming request.

    How HTTP request looks like?

    It consists of:

    • Request line - request type
    • Headers - content info like length, enconding, etc.
    • Body (not always included)
    What HTTP method types are there?
    • GET
    • POST
    • HEAD
    • PUT
    • DELETE
    • CONNECT
    • OPTIONS
    • TRACE
    What HTTP response codes are there?
    • 1xx - informational
    • 2xx - Success
    • 3xx - Redirect
    • 4xx - Error, client fault
    • 5xx - Error, server fault
    What is HTTPS?
    Explain HTTP Cookies

    HTTP is stateless. To share state, we can use Cookies.

    TODO: explain what is actually a Cookie

    What is HTTP Pipelining?
    You get "504 Gateway Timeout" error from an HTTP server. What does it mean?

    The server didn't receive a response from another server it communicates with in a timely manner.

    What is a proxy?
    What is a reverse proxy?
    What is CDN?
    When you publish a project, you usually publish it with a license. What types of licenses are you familiar with and which one do you prefer to use?
    Explain what is "X-Forwarded-For"

    Wikipedia: "The X-Forwarded-For (XFF) HTTP header field is a common method for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer."

    Load Balancers

    What is a load balancer?

    A load balancer accepts (or denies) incoming network traffic from a client, and based on some criteria (application related, network, etc.) it distributes those communications out to servers (at least one).

    Why to used a load balancer?
    • Scalability - using a load balancer, you can possibly add more servers in the backend to handle more requests/traffic from the clients, as opposed to using one server.
    • Redundancy - if one server in the backend dies, the load balancer will keep forwarding the traffic/requests to the second server so users won't even notice one of the servers in the backend is down.
    What load balancer techniques/algorithms are you familiar with?
    • Round Robin
    • Weighted Round Robin
    • Least Connection
    • Weighted Least Connection
    • Resource Based
    • Fixed Weighting
    • Weighted Response Time
    • Source IP Hash
    • URL Hash
    What are the drawbacks of round robin algorithm in load balancing?
    • A simple round robin algorithm knows nothing about the load and the spec of each server it forwards the requests to. It is possible, that multiple heavy workloads requests will get to the same server while other servers will got only lightweight requests which will result in one server doing most of the work, maybe even crashing at some point because it unable to handle all the heavy workloads requests by its own.
    • Each request from the client creates a whole new session. This might be a problem for certain scenarios where you would like to perform multiple operations where the server has to know about the result of operation so basically, being sort of aware of the history it has with the client. In round robin, first request might hit server X, while second request might hit server Y and ask to continue processing the data that was processed on server X already.
    What is an Application Load Balancer?
    In which scenarios would you use ALB?
    At what layers a load balancer can operate?

    L4 and L7

    Can you perform load balancing without using a dedicated load balancer instance?

    Yes, you can use DNS for performing load balancing.

    What is DNS load balancing? What its advantages? When would you use it?

    Load Balancers - Sticky Sessions

    What are sticky sessions? What are their pros and cons?

    Recommended read:

    Cons:

    • Can cause uneven load on instance (since requests routed to the same instances) Pros:
    • Ensures in-proc sessions are not lost when a new request is created
    Name one use case for using sticky sessions

    You would like to make sure the user doesn't lose the current session data.

    What sticky sessions use for enabling the "stickiness"?

    Cookies. There are application based cookies and duration based cookies.

    Explain application-based cookies
    • Generated by the application and/or the load balancer
    • Usually allows to include custom data
    Explain duration-based cookies
    • Generated by the load balancer
    • Session is not sticky anymore once the duration elapsed

    Load Balancers - Load Balancing Algorithms

    Explain each of the following load balancing techniques
    • Round Robin
    • Weighted Round Robin
    • Least Connection
    • Weighted Least Connection
    • Resource Based
    • Fixed Weighting
    • Weighted Response Time
    • Source IP Hash
    • URL Hash

    Explain use case for connection draining?
    To ensure that a Classic Load Balancer stops sending requests to instances that are de-registering or unhealthy, while keeping the existing connections open, use connection draining. This enables the load balancer to complete in-flight requests made to instances that are de-registering or unhealthy.

    The maximum timeout value can be set between 1 and 3,600 seconds on both GCP and AWS.

    Licenses

    Are you familiar with "Creative Commons"? What do you know about it?
    Explain the differences between copyleft and permissive licenses

    In Copyleft, any derivative work must use the same licensing while in permissive licensing there are no such condition. GPL-3 is an example of copyleft license while BSD is an example of permissive license.

    Random

    How a search engine works?
    How auto completion works?
    What is faster than RAM?

    CPU cache. Source

    What is a memory leak?
    What is your favorite protocol?

    SSH HTTP DHCP DNS ...

    What is Cache API?
    What is the C10K problem? Is it relevant today?

    https://idiallo.com/blog/c10k-2016

    Storage

    What types of storage formats are there?
    • File
    • Block
    • Object
    What types of storage devices are there?
    Explain IOPS
    Explain storage throughput
    What is a filesystem?
    Explain Dark Data

    Questions you CAN ask

    A list of questions you as a candidate can ask the interviewer during or after the interview. These are only a suggestion, use them carefully. Not every interviewer will be able to answer these (or happy to) which should be perhaps a red flag warning for your regarding working in such place but that's really up to you.

    What do you like about working here?
    How does the company promote personal growth?
    What is the current level of technical debt you are dealing with?

    Be careful when asking this question - all companies, regardless of size, have some level of tech debt. Phrase the question in the light that all companies have the deal with this, but you want to see the current pain points they are dealing with

    This is a great way to figure how managers deal with unplanned work, and how good they are at setting expectations with projects.

    Why I should NOT join you? (or 'what you don't like about working here?')
    What was your favorite project you've worked on?

    This can give you insights in some of the cool projects a company is working on, and if you would enjoy working on projects like these. This is also a good way to see if the managers are allowing employees to learn and grow with projects outside of the normal work you'd do.

    If you could change one thing about your day to day, what would it be?

    Similar to the tech debt question, this helps you identify any pain points with the company. Additionally, it can be a great way to show how you'd be an asset to the team.

    For Example, if they mention they have problem X, and you've solved that in the past, you can show how you'd be able to mitigate that problem.

    Let's say that we agree and you hire me to this position, after X months, what do you expect that I have achieved?

    Not only this will tell you what is expected from you, it will also provide big hint on the type of work you are going to do in the first months of your job.

    Testing

    Explain white-box testing
    Explain black-box testing
    What are unit tests?
    What types of tests would you run to test a web application?
    Explain test harness?
    What is A/B testing?
    What is network simulation and how do you perform it?
    What types of performances tests are you familiar with?
    Explain the following types of tests:
    • Load Testing
    • Stress Testing
    • Capacity Testing
    • Volume Testing
    • Endurance Testing

    Databases

    Name Topic Objective & Instructions Solution Comments
    Message Board Tables Relational DB Tables Exercise Solution
    What is a relational database?
    • Data Storage: system to store data in tables
    • SQL: programming language to manage relational databases
    • Data Definition Language: a standard syntax to create, alter and delete tables
    What does it mean when a database is ACID compliant?

    ACID stands for Atomicity, Consistency, Isolation, Durability. In order to be ACID compliant, the database must meet each of the four criteria

    Atomicity - When a change occurs to the database, it should either succeed or fail as a whole.

    For example, if you were to update a table, the update should completely execute. If it only partially executes, the update is considered failed as a whole, and will not go through - the DB will revert back to it's original state before the update occurred. It should also be mentioned that Atomicity ensures that each transaction is completed as it's own stand alone "unit" - if any part fails, the whole statement fails.

    Consistency - any change made to the database should bring it from one valid state into the next.

    For example, if you make a change to the DB, it shouldn't corrupt it. Consistency is upheld by checks and constraints that are pre-defined in the DB. For example, if you tried to change a value from a string to an int when the column should be of datatype string, a consistent DB would not allow this transaction to go through, and the action would not be executed

    Isolation - this ensures that a database will never be seen "mid-update" - as multiple transactions are running at the same time, it should still leave the DB in the same state as if the transactions were being run sequentially.

    For example, let's say that 20 other people were making changes to the database at the same time. At the time you executed your query, 15 of the 20 changes had gone through, but 5 were still in progress. You should only see the 15 changes that had completed - you wouldn't see the database mid-update as the change goes through.

    Durability - Once a change is committed, it will remain committed regardless of what happens (power failure, system crash, etc.). This means that all completed transactions must be recorded in non-volatile memory.

    Note that SQL is by nature ACID compliant. Certain NoSQL DB's can be ACID compliant depending on how they operate, but as a general rule of thumb, NoSQL DB's are not considered ACID compliant

    What is sharding?

    Sharding is a horizontal partitioning.

    Are you able to explain what is it good for?

    You find out your database became a bottleneck and users experience issues accessing data. How can you deal with such situation?

    Not much information provided as to why it became a bottleneck and what is current architecture, so one general approach could be
    to reduce the load on your database by moving frequently-accessed data to in-memory structure.

    What is a connection pool?

    Connection Pool is a cache of database connections and the reason it's used is to avoid an overhead of establishing a connection for every query done to a database.

    What is a connection leak?

    A connection leak is a situation where database connection isn't closed after being created and is no longer needed.

    What is Table Lock?
    Your database performs slowly than usual. More specifically, your queries are taking a lot of time. What would you do?
    • Query for running queries and cancel the irrelevant queries
    • Check for connection leaks (query for running connections and include their IP)
    • Check for table locks and kill irrelevant locking sessions
    What is a Data Warehouse?

    "A data warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of organisation's decision-making process"

    Explain what is a time-series database
    What is OLTP (Online transaction processing)?
    What is OLAP (Online Analytical Processing)?
    What is an index in a database?

    A database index is a data structure that improves the speed of operations in a table. Indexes can be created using one or more columns, providing the basis for both rapid random lookups and efficient ordering of access to records.

    What data types are there in relational databases?
    Explain Normalization

    Data that is used multiple times in a database should be stored once and referenced with a foreign key.
    This has the clear benefit of ease of maintenance where you need to change a value only in a single place to change it everywhere.

    Explain Primary Key and Foreign Key

    Primary Key: each row in every table should a unique identifier that represents the row.
    Foreign Key: a reference to another table's primary key. This allows you to join table together to retrieve all the information you need without duplicating data.

    What types of data tables have you used?
    • Primary data table: main data you care about
    • Details table: includes a foreign key and has one to many relationship
    • Lookup values table: can be one table per lookup or a table containing all the lookups and has one to many relationship
    • Multi reference table
    What is ORM? What benefits it provides in regards to relational databases usage?

    Wikipedia: "is a programming technique for converting data between incompatible type systems using object-oriented programming languages"

    In regards to the relational databases:

    • Database as code
    • Database abstraction
    • Encapsulates SQL complexity
    • Enables code review process
    • Enables usage as a native OOP structure
    What is DDL?

    Wikipedia: "In the context of SQL, data definition or data description language (DDL) is a syntax for creating and modifying database objects such as tables, indices, and users."

    Regex

    Given a text file, perform the following exercises

    Extract

    Extract all the numbers
    Extract the first word of each line

    Bonus: extract the last word of each line

    Extract all the IP addresses
    Extract dates in the format of yyyy-mm-dd or yyyy-dd-mm
    Extract email addresses

    Replace

    Replace tabs with four spaces
    Replace 'red' with 'green'

    System Design

    Explain what is a "Single point of failure"?
    Explain "3-Tier Architecture" (including pros and cons)
    What are the drawbacks of monolithic architecture?
    • Not suitable for frequent code changes and the ability to deploy new features
    • Not designed for today's infrastructure (like public clouds)
    • Scaling a team to work monolithic architecture is more challenging
    What are the advantages of microoservices architecture over a monolithic architecture?
    • Each of the services individually fail without escalating into an application-wide outage.
    • Each service can be developed and maintained by a separate team and this team can choose its own tools and coding language
    What's a service mesh?

    This article provides a great explanation.

    Explain "Loose Coupling"
    What is a message queue? When is it used?

    Scalability

    Explain Scalability

    The ability easily grow in size and capacity based on demand and usage.

    Explain Elasticity

    The ability to grow but also to reduce based on what is required

    Explain Disaster Recovery
    Explain Fault Tolerance and High Availability

    Fault Tolerance - The ability to self-heal and return to normal capacity. Also the ability to withstand a failure and remain functional.

    High Availability - Being able to access a resource (in some use cases, using different platforms)

    What is the difference between high availability and Disaster Recovery?

    wintellect.com: "High availability, simply put, is eliminating single points of failure and disaster recovery is the process of getting a system back to an operational state when a system is rendered inoperative. In essence, disaster recovery picks up when high availability fails, so HA first."

    Explain Vertical Scaling

    Vertical Scaling is the process of adding resources to increase power of existing servers. For example, adding more CPUs, adding more RAM, etc.

    What are the disadvantages of Vertical Scaling?

    With vertical scaling alone, the component still remains a single point of failure. In addition, it has hardware limit where if you don't have more resources, you might not be able to scale vertically.

    Which type of cloud services usually support vertical scaling?

    Databases, cache. It's common mostly for non-distributed systems.

    Explain Horizontal Scaling

    Horizontal Scaling is the process of adding more resources that will be able handle requests as one unit

    What is the disadvantage of Horizontal Scaling? What is often required in order to perform Horizontal Scaling?

    A load balancer. You can add more resources, but if you would like them to be part of the process, you have to serve them the requests/responses. Also, data inconsistency is a concern with horizontal scaling.

    Explain in which use cases will you use vertical scaling and in which use cases you will use horizontal scaling
    Explain Resiliency and what ways are there to make a system more resilient
    Explain "Consistent Hashing"
    How would you update each of the services in the following drawing without having app (foo.com) downtime?

    What is the problem with the following architecture and how would you fix it?

    The load on the producers or consumers may be high which will then cause them to hang or crash.
    Instead of working in "push mode", the consumers can pull tasks only when they are ready to handle them. It can be fixed by using a streaming platform like Kafka, Kinesis, etc. This platform will make sure to handle the high load/traffic and pass tasks/messages to consumers only when the ready to get them.

    Users report that there is huge spike in process time when adding little bit more data to process as an input. What might be the problem?

    How would you scale the architecture from the previous question to hundreds of users?

    Cache

    What is "cache"? In which cases would you use it?
    What is "distributed cache"?
    What is a "cache replacement policy"?

    Take a look here

    Which cache replacement policies are you familiar with?

    You can find a list here

    Explain the following cache policies:
    • FIFO
    • LIFO
    • LRU

    Read about it here

    Why not writing everything to cache instead of a database/datastore?

    Migrations

    How you prepare for a migration? (or plan a migration)

    You can mention:

    roll-back & roll-forward cut over dress rehearsals DNS redirection

    Explain "Branch by Abstraction" technique

    Design a system

    Can you design a video streaming website?
    Can you design a photo upload website?
    How would you build a URL shortener?

    More System Design Questions

    Additional exercises can be found in system-design-notebook repository.

    Hardware

    What is a CPU?

    A central processing unit (CPU) performs basic arithmetic, logic, controlling, and input/output (I/O) operations specified by the instructions in the program. This contrasts with external components such as main memory and I/O circuitry, and specialized processors such as graphics processing units (GPUs).

    What is RAM?

    RAM (Random Access Memory) is the hardware in a computing device where the operating system (OS), application programs and data in current use are kept so they can be quickly reached by the device's processor. RAM is the main memory in a computer. It is much faster to read from and write to than other kinds of storage, such as a hard disk drive (HDD), solid-state drive (SSD) or optical drive.

    What is an embedded system?

    An embedded system is a computer system - a combination of a computer processor, computer memory, and input/output peripheral devices—that has a dedicated function within a larger mechanical or electronic system. It is embedded as part of a complete device often including electrical or electronic hardware and mechanical parts.

    Can you give an example of an embedded system?

    Raspberry Pi

    What types of storage are there?

    Big Data

    Explain what is exactly Big Data

    As defined by Doug Laney:

    • Volume: Extremely large volumes of data
    • Velocity: Real time, batch, streams of data
    • Variety: Various forms of data, structured, semi-structured and unstructured
    • Veracity or Variability: Inconsistent, sometimes inaccurate, varying data
    What is DataOps? How is it related to DevOps?

    DataOps seeks to reduce the end-to-end cycle time of data analytics, from the origin of ideas to the literal creation of charts, graphs and models that create value. DataOps combines Agile development, DevOps and statistical process controls and applies them to data analytics.

    What is Data Architecture?

    An answer from talend.com:

    "Data architecture is the process of standardizing how organizations collect, store, transform, distribute, and use data. The goal is to deliver relevant data to people who need it, when they need it, and help them make sense of it."

    Explain the different formats of data
    • Structured - data that has defined format and length (e.g. numbers, words)
    • Semi-structured - Doesn't conform to a specific format but is self-describing (e.g. XML, SWIFT)
    • Unstructured - does not follow a specific format (e.g. images, test messages)
    What is a Data Warehouse?

    Wikipedia's explanation on Data Warehouse Amazon's explanation on Data Warehouse

    What is Data Lake?

    Data Lake - Wikipedia

    Can you explain the difference between a data lake and a data warehouse?
    What is "Data Versioning"? What models of "Data Versioning" are there?
    What is ETL?

    Apache Hadoop

    Explain what is Hadoop

    Apache Hadoop - Wikipedia

    Explain Hadoop YARN

    Responsible for managing the compute resources in clusters and scheduling users' applications

    Explain Hadoop MapReduce

    A programming model for large-scale data processing

    Explain Hadoop Distributed File Systems (HDFS)
    • Distributed file system providing high aggregate bandwidth across the cluster.
    • For a user it looks like a regular file system structure but behind the scenes it's distributed across multiple machines in a cluster
    • Typical file size is TB and it can scale and supports millions of files
    • It's fault tolerant which means it provides automatic recovery from faults
    • It's best suited for running long batch operations rather than live analysis
    What do you know about HDFS architecture?

    HDFS Architecture

    • Master-slave architecture
    • Namenode - master, Datanodes - slaves
    • Files split into blocks
    • Blocks stored on datanodes
    • Namenode controls all metadata

    Ceph

    Explain what is Ceph
    True or False? Ceph favor consistency and correctness over performances
    True
    Which services or types of storage Ceph supports?
    • Object (RGW)
    • Block (RBD)
    • File (CephFS)
    What is RADOS?
    • Reliable Autonomic Distributed Object Storage
    • Provides low-level data object storage service
    • Strong Consistency
    • Simplifies design and implementation of higher layers (block, file, object)
    Describe RADOS software components
    • Monitor
      • Central authority for authentication, data placement, policy
      • Coordination point for all other cluster components
      • Protect critical cluster state with Paxos
    • Manager
      • Aggregates real-time metrics (throughput, disk usage, etc.)
      • Host for pluggable management functions
      • 1 active, 1+ standby per cluster
    • OSD (Object Storage Daemon)
      • Stores data on an HDD or SSD
      • Services client IO requests
    What is the workflow of retrieving data from Ceph?
    What is the workflow of retrieving data from Ceph?
    What are "Placement Groups"?
    Describe in the detail the following: Objects -> Pool -> Placement Groups -> OSDs
    What is OMAP?
    What is a metadata server? How it works?

    Packer

    What is Packer? What is it used for?

    In general, Packer automates machine images creation. It allows you to focus on configuration prior to deployment while making the images. This allows you start the instances much faster in most cases.

    Packer follows a "configuration->deployment" model or "deployment->configuration"?

    A configuration->deployment which has some advantages like:

    1. Deployment Speed - you configure once prior to deployment instead of configuring every time you deploy. This allows you to start instances/services much quicker.
    2. More immutable infrastructure - with configuration->deployment it's not likely to have very different deployments since most of the configuration is done prior to the deployment. Issues like dependencies errors are handled/discovered prior to deployment in this model.

    Certificates

    If you are looking for a way to prepare for a certain exam this is the section for you. Here you'll find a list of certificates, each references to a separate file with focused questions that will help you to prepare to the exam. Good luck :)

    AWS

    Azure

    • AZ-900 (Latest update: 2021)

    Kubernetes

    Other DevOps Projects

    Credits

    Thanks to all of our amazing contributors who make it easy for everyone to learn new things :)

    Logos credits can be found here

    License

    License: CC BY-NC-ND 3.0