diff --git a/404.html b/404.html new file mode 100644 index 0000000..48d5d6c --- /dev/null +++ b/404.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

404: Page not found

\ No newline at end of file diff --git a/CNAME b/CNAME new file mode 100644 index 0000000..b9fdefb --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +aaqa.dev \ No newline at end of file diff --git a/contact/index.html b/contact/index.html new file mode 100644 index 0000000..5ef6757 --- /dev/null +++ b/contact/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Contact

Email: aaqaishtyaq at gmail.com


My other identities:

Note: I hate phone calls. Also, a polite request to please follow the NoHello rule in case you're reaching out for technical help.

\ No newline at end of file diff --git a/favicon.ico b/favicon.ico new file mode 100644 index 0000000..f2785f0 Binary files /dev/null and b/favicon.ico differ diff --git a/fonts/iosevka-bold.woff2 b/fonts/iosevka-bold.woff2 new file mode 100644 index 0000000..7081b43 Binary files /dev/null and b/fonts/iosevka-bold.woff2 differ diff --git a/fonts/iosevka-bolditalic.woff2 b/fonts/iosevka-bolditalic.woff2 new file mode 100644 index 0000000..993f5de Binary files /dev/null and b/fonts/iosevka-bolditalic.woff2 differ diff --git a/fonts/iosevka-italic.woff2 b/fonts/iosevka-italic.woff2 new file mode 100644 index 0000000..7a531ca Binary files /dev/null and b/fonts/iosevka-italic.woff2 differ diff --git a/fonts/iosevka-regular.woff2 b/fonts/iosevka-regular.woff2 new file mode 100644 index 0000000..2817816 Binary files /dev/null and b/fonts/iosevka-regular.woff2 differ diff --git a/highlight_themes/tomorrow-dark.tmTheme b/highlight_themes/tomorrow-dark.tmTheme new file mode 100644 index 0000000..3931aa8 --- /dev/null +++ b/highlight_themes/tomorrow-dark.tmTheme @@ -0,0 +1,282 @@ + + + + + comment + http://aaqa.dev + name + tomorrow-dark + settings + + + settings + + background + #000000 + caret + #9F9F9F + foreground + #DEDEDE + invisibles + #343434 + lineHighlight + #2A2A2A + selection + #424242 + + + + name + Comment + scope + comment, string.quoted.double.block.python + settings + + foreground + #999999 + + + + name + Foreground + scope + keyword.operator.class, constant.other, source.php.embedded.line + settings + + fontStyle + + foreground + #EEEEEE + + + + name + Variable, String Link, Regular Expression, Tag Name + scope + variable, support.other.variable, string.other.link, string.regexp, entity.name.tag, entity.other.attribute-name, meta.tag, declaration.tag + settings + + foreground + #D54E53 + + + + name + Number, Constant, Function Argument, Tag Attribute, Embedded + scope + constant.numeric, constant.language, support.constant, constant.character, variable.parameter, punctuation.section.embedded, keyword.other.unit + settings + + fontStyle + + foreground + #E78C45 + + + + name + Class, Support + scope + entity.name.class, entity.name.type.class, support.type, support.class + settings + + fontStyle + + foreground + #E7C547 + + + + name + String, Symbols, Inherited Class, Markup Heading + scope + string, constant.other.symbol, entity.other.inherited-class, markup.heading + settings + + fontStyle + + foreground + #B9CA4A + + + + name + Operator, Misc + scope + keyword.operator, constant.other.color + settings + + foreground + #70C0B1 + + + + name + Function, Special Method, Block Level + scope + entity.name.function, meta.function-call, support.function, keyword.other.special-method, meta.block-level + settings + + fontStyle + + foreground + #4271AE + + + + name + Keyword, Storage + scope + keyword, storage, storage.type, entity.name.tag.css + settings + + fontStyle + + foreground + #C397D8 + + + + name + Invalid + scope + invalid + settings + + background + #DF5F5F + fontStyle + + foreground + #CED2CF + + + + name + Separator + scope + meta.separator + settings + + background + #82A3BF + foreground + #CED2CF + + + + name + Deprecated + scope + invalid.deprecated + settings + + background + #B798BF + fontStyle + + foreground + #CED2CF + + + + name + Diff foreground + scope + markup.inserted.diff, markup.deleted.diff, meta.diff.header.to-file, meta.diff.header.from-file + settings + + foreground + #FFFFFF + + + + name + Diff insertion + scope + markup.inserted.diff, meta.diff.header.to-file + settings + + foreground + #718c00 + + + + name + Diff deletion + scope + markup.deleted.diff, meta.diff.header.from-file + settings + + foreground + #c82829 + + + + name + Diff header + scope + meta.diff.header.from-file, meta.diff.header.to-file + settings + + foreground + #FFFFFF + background + #4271ae + + + + name + Diff range + scope + meta.diff.range + settings + + fontStyle + italic + foreground + #3e999f + + + + name + diff.deleted + scope + markup.deleted + settings + + foreground + #F92672 + + + + name + diff.inserted + scope + markup.inserted + settings + + foreground + #A6E22E + + + + name + diff.changed + scope + markup.changed + settings + + foreground + #967EFB + + + + uuid + 33D8C715-AD3A-455B-8DF2-56F708909FFE + colorSpaceName + sRGB + semanticClass + tomorrow-dark + + \ No newline at end of file diff --git a/highlight_themes/tomorrow-light.tmTheme b/highlight_themes/tomorrow-light.tmTheme new file mode 100644 index 0000000..c876263 --- /dev/null +++ b/highlight_themes/tomorrow-light.tmTheme @@ -0,0 +1,249 @@ + + + + + comment + http://aaqa.dev + name + tomorrow-light + settings + + + settings + + background + #FFFFFF + caret + #AEAFAD + foreground + #1D1F21 + invisibles + #D1D1D1 + lineHighlight + #EFEFEF + selection + #D6D6D6 + + + + name + Comment + scope + comment, string.quoted.double.block.python + settings + + foreground + #999999 + + + + name + Foreground + scope + keyword.operator.class, constant.other, source.php.embedded.line + settings + + fontStyle + + foreground + #666969 + + + + name + Variable, String Link, Regular Expression, Tag Name + scope + variable, support.other.variable, string.other.link, string.regexp, entity.name.tag, entity.other.attribute-name, meta.tag, declaration.tag + settings + + foreground + #CC6666 + + + + name + Number, Constant, Function Argument, Tag Attribute, Embedded + scope + constant.numeric, constant.language, support.constant, constant.character, variable.parameter, punctuation.section.embedded, keyword.other.unit + settings + + fontStyle + + foreground + #DE935F + + + + name + Class, Support + scope + entity.name.class, entity.name.type.class, support.type, support.class + settings + + fontStyle + + foreground + #E6C547 + + + + name + String, Symbols, Inherited Class, Markup Heading + scope + string, constant.other.symbol, entity.other.inherited-class, markup.heading + settings + + fontStyle + + foreground + #B5BD68 + + + + name + Operator, Misc + scope + keyword.operator, constant.other.color + settings + + foreground + #70C0BA + + + + name + Function, Special Method, Block Level + scope + entity.name.function, meta.function-call, support.function, keyword.other.special-method, meta.block-level + settings + + fontStyle + + foreground + #4271AE + + + + name + Keyword, Storage + scope + keyword, storage, storage.type + settings + + fontStyle + + foreground + #B294BB + + + + name + Invalid + scope + invalid + settings + + background + #C82829 + fontStyle + + foreground + #FFFFFF + + + + name + Separator + scope + meta.separator + settings + + background + #4271AE + foreground + #FFFFFF + + + + name + Deprecated + scope + invalid.deprecated + settings + + background + #B77EE0 + fontStyle + + foreground + #FFFFFF + + + + name + Diff foreground + scope + markup.inserted.diff, markup.deleted.diff, meta.diff.header.to-file, meta.diff.header.from-file + settings + + foreground + #FFFFFF + + + + name + Diff insertion + scope + markup.inserted.diff, meta.diff.header.to-file + settings + + background + #718c00 + + + + name + Diff deletion + scope + markup.deleted.diff, meta.diff.header.from-file + settings + + background + #c82829 + + + + name + Diff header + scope + meta.diff.header.from-file, meta.diff.header.to-file + settings + + foreground + #FFFFFF + background + #4271ae + + + + name + Diff range + scope + meta.diff.range + settings + + fontStyle + italic + foreground + #3e999f + + + + uuid + 82CCD69C-F1B1-4529-B39E-780F91F07604 + colorSpaceName + sRGB + semanticClass + tomorrow-dark + + \ No newline at end of file diff --git a/hl-dark.css b/hl-dark.css new file mode 100644 index 0000000..63d8abb --- /dev/null +++ b/hl-dark.css @@ -0,0 +1,74 @@ +/* + * theme "tomorrow-dark" generated by syntect + */ + +.z-code { + color: #dedede; + background-color: #000000; +} + +.z-comment, .z-string.z-quoted.z-double.z-block.z-python { + color: #999999; +} +.z-keyword.z-operator.z-class, .z-constant.z-other, .z-source.z-php.z-embedded.z-line { + color: #eeeeee; +} +.z-variable, .z-support.z-other.z-variable, .z-string.z-other.z-link, .z-string.z-regexp, .z-entity.z-name.z-tag, .z-entity.z-other.z-attribute-name, .z-meta.z-tag, .z-declaration.z-tag { + color: #d54e53; +} +.z-constant.z-numeric, .z-constant.z-language, .z-support.z-constant, .z-constant.z-character, .z-variable.z-parameter, .z-punctuation.z-section.z-embedded, .z-keyword.z-other.z-unit { + color: #e78c45; +} +.z-entity.z-name.z-class, .z-entity.z-name.z-type.z-class, .z-support.z-type, .z-support.z-class { + color: #e7c547; +} +.z-string, .z-constant.z-other.z-symbol, .z-entity.z-other.z-inherited-class, .z-markup.z-heading { + color: #b9ca4a; +} +.z-keyword.z-operator, .z-constant.z-other.z-color { + color: #70c0b1; +} +.z-entity.z-name.z-function, .z-meta.z-function-call, .z-support.z-function, .z-keyword.z-other.z-special-method, .z-meta.z-block-level { + color: #4271ae; +} +.z-keyword, .z-storage, .z-storage.z-type, .z-entity.z-name.z-tag.z-css { + color: #c397d8; +} +.z-invalid { + color: #ced2cf; + background-color: #df5f5f; +} +.z-meta.z-separator { + color: #ced2cf; + background-color: #82a3bf; +} +.z-invalid.z-deprecated { + color: #ced2cf; + background-color: #b798bf; +} +.z-markup.z-inserted.z-diff, .z-markup.z-deleted.z-diff, .z-meta.z-diff.z-header.z-to-file, .z-meta.z-diff.z-header.z-from-file { + color: #ffffff; +} +.z-markup.z-inserted.z-diff, .z-meta.z-diff.z-header.z-to-file { + color: #718c00; +} +.z-markup.z-deleted.z-diff, .z-meta.z-diff.z-header.z-from-file { + color: #c82829; +} +.z-meta.z-diff.z-header.z-from-file, .z-meta.z-diff.z-header.z-to-file { + color: #ffffff; + background-color: #4271ae; +} +.z-meta.z-diff.z-range { + color: #3e999f; +font-style: italic; +} +.z-markup.z-deleted { + color: #f92672; +} +.z-markup.z-inserted { + color: #a6e22e; +} +.z-markup.z-changed { + color: #967efb; +} diff --git a/hl-light.css b/hl-light.css new file mode 100644 index 0000000..4b9e378 --- /dev/null +++ b/hl-light.css @@ -0,0 +1,65 @@ +/* + * theme "tomorrow-light" generated by syntect + */ + +.z-code { + color: #1d1f21; + background-color: #ffffff; +} + +.z-comment, .z-string.z-quoted.z-double.z-block.z-python { + color: #999999; +} +.z-keyword.z-operator.z-class, .z-constant.z-other, .z-source.z-php.z-embedded.z-line { + color: #666969; +} +.z-variable, .z-support.z-other.z-variable, .z-string.z-other.z-link, .z-string.z-regexp, .z-entity.z-name.z-tag, .z-entity.z-other.z-attribute-name, .z-meta.z-tag, .z-declaration.z-tag { + color: #cc6666; +} +.z-constant.z-numeric, .z-constant.z-language, .z-support.z-constant, .z-constant.z-character, .z-variable.z-parameter, .z-punctuation.z-section.z-embedded, .z-keyword.z-other.z-unit { + color: #de935f; +} +.z-entity.z-name.z-class, .z-entity.z-name.z-type.z-class, .z-support.z-type, .z-support.z-class { + color: #e6c547; +} +.z-string, .z-constant.z-other.z-symbol, .z-entity.z-other.z-inherited-class, .z-markup.z-heading { + color: #b5bd68; +} +.z-keyword.z-operator, .z-constant.z-other.z-color { + color: #70c0ba; +} +.z-entity.z-name.z-function, .z-meta.z-function-call, .z-support.z-function, .z-keyword.z-other.z-special-method, .z-meta.z-block-level { + color: #4271ae; +} +.z-keyword, .z-storage, .z-storage.z-type { + color: #b294bb; +} +.z-invalid { + color: #ffffff; + background-color: #c82829; +} +.z-meta.z-separator { + color: #ffffff; + background-color: #4271ae; +} +.z-invalid.z-deprecated { + color: #ffffff; + background-color: #b77ee0; +} +.z-markup.z-inserted.z-diff, .z-markup.z-deleted.z-diff, .z-meta.z-diff.z-header.z-to-file, .z-meta.z-diff.z-header.z-from-file { + color: #ffffff; +} +.z-markup.z-inserted.z-diff, .z-meta.z-diff.z-header.z-to-file { + background-color: #718c00; +} +.z-markup.z-deleted.z-diff, .z-meta.z-diff.z-header.z-from-file { + background-color: #c82829; +} +.z-meta.z-diff.z-header.z-from-file, .z-meta.z-diff.z-header.z-to-file { + color: #ffffff; + background-color: #4271ae; +} +.z-meta.z-diff.z-range { + color: #3e999f; +font-style: italic; +} diff --git a/icons/github.svg b/icons/github.svg new file mode 100644 index 0000000..4a415af --- /dev/null +++ b/icons/github.svg @@ -0,0 +1 @@ + diff --git a/icons/twitter.svg b/icons/twitter.svg new file mode 100644 index 0000000..37e49f5 --- /dev/null +++ b/icons/twitter.svg @@ -0,0 +1 @@ + diff --git a/images/aaqa.png b/images/aaqa.png new file mode 100644 index 0000000..973de26 Binary files /dev/null and b/images/aaqa.png differ diff --git a/images/reading/1984.jpg b/images/reading/1984.jpg new file mode 100644 index 0000000..ecac445 Binary files /dev/null and b/images/reading/1984.jpg differ diff --git a/images/reading/animal_farm.jpg b/images/reading/animal_farm.jpg new file mode 100644 index 0000000..c0fb995 Binary files /dev/null and b/images/reading/animal_farm.jpg differ diff --git a/images/reading/the_idiot.jpg b/images/reading/the_idiot.jpg new file mode 100644 index 0000000..0bae208 Binary files /dev/null and b/images/reading/the_idiot.jpg differ diff --git a/images/travel/hp24/1.png b/images/travel/hp24/1.png new file mode 100644 index 0000000..8cb8aac Binary files /dev/null and b/images/travel/hp24/1.png differ diff --git a/images/travel/hp24/2.png b/images/travel/hp24/2.png new file mode 100644 index 0000000..8fa466f Binary files /dev/null and b/images/travel/hp24/2.png differ diff --git a/images/travel/hp24/3.png b/images/travel/hp24/3.png new file mode 100644 index 0000000..0d78c9d Binary files /dev/null and b/images/travel/hp24/3.png differ diff --git a/images/travel/hp24/4.png b/images/travel/hp24/4.png new file mode 100644 index 0000000..05d842f Binary files /dev/null and b/images/travel/hp24/4.png differ diff --git a/images/travel/hp24/5.png b/images/travel/hp24/5.png new file mode 100644 index 0000000..c1c5bf7 Binary files /dev/null and b/images/travel/hp24/5.png differ diff --git a/index.html b/index.html new file mode 100644 index 0000000..6cdd7bd --- /dev/null +++ b/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

My name is Aaqa Ishtyaq, I'm a Polyglot Software Engineer, with an interest in Distributed Systems and Observability.

I've been writing code professionally for 5 years now, and as a hobby, since I was 12. I'm a massive cyberpunk nerd. I like to read books, take photos, and hike in the mountains near my place.

I'm a Backend Engineer working remotely from India on the infrastructure engineering team at Deel. Previously, I was a Senior Software Engineer at HackerRank, focusing on their core remote code execution system.

The theme of my blog is software engineering, but I tend to derive broader life lessons from my retrospectives.

Here are some featured posts and pages:

Skills

Languages

  • Go
  • Rust
  • Ruby
  • Typescript
  • Python
  • Bash
  • HTML5
  • CSS3
  • Sass
  • SQL
  • Markdown

Tools

  • Vim / VS Code
  • Tmux
  • Z Shell
  • Firecracker
  • Buildkit
  • Containerd
  • Docker
  • Virtualbox
  • Vagrant
  • Ansible
  • Gulp

Frameworks

  • OpenTelemetry
  • Ruby on Rails
  • Scrapy
  • Django
  • Flask
  • Node.js
  • React.js
  • Mocha / Chai

Others

  • Message Brokers
  • Kubernetes
  • Nomad
  • Basic UNIX Admin
  • PostgreSQL
  • DyanamoDB
  • gRPC / REST
  • AWS / GCP / OCI
\ No newline at end of file diff --git a/notes/algorithms-in-python-bubble-sort/index.html b/notes/algorithms-in-python-bubble-sort/index.html new file mode 100644 index 0000000..e7b37d8 --- /dev/null +++ b/notes/algorithms-in-python-bubble-sort/index.html @@ -0,0 +1,71 @@ +Aaqa Ishtyaq

Algorithms in Python: Bubble Sort

3206 words, 16 minutes to read

Some theory

Bubble sort is another commonly known sorting algorithm. The idea here is to scan a list of items (say integers) sequentially (from left to right) and compare consecutive pairs of elements starting at index 0.

Example:


+my_numbers = [92,11,45,2234,0,7,65]
+# 92 is index 0 and the consecutive pairs are
+# (92,11), (11,45), (45,2234) and so on ...
+

At first we compare elements (list[0],list[1]) then (list[1],list[2]) then (list[2],list[3]) and so on until the end of the list is reached.

When comparing we check if element i is greater than element i + 1, if they are we just swap the two elements and move on to the next pair. If they are not this means that the pair is already sorted, so we also move on to the next pair.

Example:

my_numbers = [92,11,45,2234,0,7,65]
+
+# Let's compare my_numbers[0] and my_numbers[1]
+if my_numbers[0] > my_numbers[1]:
+    swap(my_numbers[0], my_numbers[1])
+
+print(my_numbers) # [11, 92, 45, 2234, 0, 7, 65]
+

This process has to be repeated for however many items are on the list. So if the list holds 9 items, it means we need to loop through it 9 times at most. But what if our original list is partially sorted? We might not need 9 passes through the list.

One way for us to know that the list is fully sorted is if we have made no swaps during our pass. For that, we need a variable to keep track of how many swaps were made during a pass.

Example:

my_numbers = [92,11,45,2234,0,7,65]
+
+# Elements (0,1) are compared and swapped. List is now 11,92,45,2234,0,7,65
+# Elements (1,2) are compared and swapped. List now 11,45,92,2234,0,7,65
+# Elements (2,3) are compared and not swapped. List remains the same.
+# Elements (3,4) are compared and swapped. List is now 11,45,92,0,2234,0,7,65
+# Elements (4,5) are compared and swapped. List is now 11,45,92,0,7,2234,65
+# Elements (5,6) are compared and swapped. List is now 11,45,92,0,7,65,2234
+
+# This represents one unique pass through the list.
+

Notice how after each pass the highest value number is pushed at len(list) - 1.

Some code

Let's look at how to implement Bubble Sort using Python:

def bubble_sort(some_list):
+
+    is_sorted = False
+
+    while not is_sorted:
+
+        is_sorted = True
+
+        for i in range(0, len(some_list) - 1):
+
+            if some_list[i] > some_list[i + 1]:
+
+                some_list[i], some_list[i+1] = some_list[i+1], some_list[i]
+                is_sorted = False
+

This works right and it will sort any list you throw at it. However we can slightly optimise it: We know that, after each pass the highest value element is guaranteed to be sorted and placed at len(some_list) - 1. Because of this, for each subsequent pass, we can stop comparing the last sorted item. instead of comparing pairs that we know are already sorted. This is what it looks like:

def bubble_sort(some_list):
+
+    is_sorted = False
+    last_sorted_item = len(some_list) - 1
+
+    while not is_sorted:
+
+        is_sorted = True
+
+        for i in range(0, last_sorted_item):
+
+            if some_list[i] > some_list[i + 1]:
+
+                some_list[i], some_list[i+1] = some_list[i+1], some_list[i]
+                is_sorted = False
+
+        last_sorted_item -= 1
+

After each pass through the loop, we know the right side of the list is sorted so we decrement the value of last_sorted_item. What this means is that the 1st pass will loop from 0 to len(some_list) -1, the second time, it will be from 0 to len(some_list) - 2 and so on ...

Time complexity

The rate of growth of this algorithm is quadratic. Expressed as O(n^2) in "big-oh" notation.

def bubble_sort(some_list):
+
+    is_sorted = False                       # time here is constant
+    last_sorted_item = len(some_list) - 1
+
+    while not is_sorted: # We go through this first loop n times
+
+        is_sorted = True
+
+        for i in range(0, last_sorted_item): # we go through this loop n-1 times
+
+            if some_list[i] > some_list[i + 1]:
+
+                # execution here is constant
+                some_list[i], some_list[i+1] = some_list[i+1], some_list[i]
+                is_sorted = False
+
+        last_sorted_item -= 1 # constant time
+

It's O(n^2) because, for each pass through the loop n times, we loop n times through the consecutive pairs. It's not a very efficient algorithm when used on large samples of data. It should only be used if you have a specific case on a small data set.

Next in the series is QuickSort, another interesting and more efficient sorting algorithm. As always, if you have questions, comments or if you spotted a typo or a mistake, please feel free to let me know on Twitter, I'm @aaqaishtyaq and always happy to help!

\ No newline at end of file diff --git a/notes/algorithms-in-python-quick-sort/index.html b/notes/algorithms-in-python-quick-sort/index.html new file mode 100644 index 0000000..a68a023 --- /dev/null +++ b/notes/algorithms-in-python-quick-sort/index.html @@ -0,0 +1,34 @@ +Aaqa Ishtyaq

Algorithms in Python: Quick Sort

2225 words, 11 minutes to read

Theory

Quicksort is a "divide and conquer" type of algorithm. The good thing about it is that the worst case can almost always be avoided by using what is called a randomized version of quicksort (more on that later).

The idea of Quicksort is to take an unsorted list and select an element (on that list) called a "pivot". Then the list is rearranged such that all elements greater (in value) than the pivot are placed to its right, and all elements lesser (in value) are placed to its left.

This process is called partitioning. At this stage in the execution of the algorithm, the order of the elements doesn't matter so long as the lesser/bigger values are placed on the correct side of the pivot.

Partitioning will produce two sublists with the pivot as a separator ( this is because the pivot will be at its natural place after the first pass aka sorted). The problem then becomes sorting these two sublists.

Note: Partitioning does not require creating copies of the list, we work on it directly as long as we keep track of the start and end indices of each sublist.

To sort the two sublists, we can apply the same logic as above (choosing a pivot, and sorting the two resulting sublists) because QuickSort is a recursive algorithm.

When a sublist only contains a single element, it's already sorted so we can stop the recursion at this point, it's our exit condition.

Note on choosing a pivot

Some people use the last item of the list, and some people use the median of the first, last, and medium elements but the most common way is to choose a random pivot to ensure n log n execution.

Some Code

def swap_values(lst, val1, val2):
+    lst[val1], lst[val2] = lst[val2], lst[val1]
+
+def quicksort(array, start, end):
+
+    if start < end:
+
+        partition_index = partition(array, start, end) #
+        quicksort(array, start, partition_index - 1)
+        quicksort(array, partition_index + 1, end)
+
+def partition(array, start, end):
+
+    pivot = end
+    partition_index = start
+
+    for i in range(start, end):
+
+        if array[i] < array[pivot]:
+            print("{} is less than {}".format(array[i], array[pivot]))
+            swap_values(array, partition_index, i)
+            partition_index += 1
+
+    array[pivot], array[partition_index] = array[partition_index], array[pivot]
+    return partition_index
+

A randomized version of Quicksort would look similar to what's above except that we must randomize the selection of our pivot.

import random
+# ...
+def partition(array, start, end):
+    if start < end:
+        pivot = random.randint(start, end)
+        array[end], array[pivot] = array[pivot], array[end]
+        partition_index(array, start, end)
+        # ...
+

Here, we set the pivot to a random integer in the range between start and end. Then, we swap the value at that index with the value at array[end]. If you run the code successively, you'll notice that the pivot is different every time. It's a nice optimization that can save some time.

Time Complexity

It's one of the most efficient sorting algorithm. In fact, most sorting functions that come packaged in many language's standard libraries use an implementation of QuickSort.

The order of growth for QuickSort in the worst case is quadratic O(n^2). The average case, however, which is the most common scenario, has a complexity of O(n log n).

QuickSort works best when used on large sets of data because of its recursive nature.

\ No newline at end of file diff --git a/notes/basic-file-operations-in-python/index.html b/notes/basic-file-operations-in-python/index.html new file mode 100644 index 0000000..e1848a2 --- /dev/null +++ b/notes/basic-file-operations-in-python/index.html @@ -0,0 +1,43 @@ +Aaqa Ishtyaq

Basic file operations in Python

2144 words, 11 minutes to read

In this short (spoiler: it's actually quite lengthy) post, I will be going through a list of very useful and handy methods in the os module (which is part of the Python standard library) for handling files and directories.

1. Create a directory

This one is pretty straightforward. If you're comfortable with the linux shell, you know that mkdir is the command to use to create directories. Unsurprisingly, Python uses the same naming convention.

Example:

import os
+
+os.mkdir("my_awesome_directory")
+

The method takes a string its argument and will create the directory under the file's parent folder. (For instance, if the path to the file calling os.mkdir() is /home/username/Documents/app.py, the "my_awesome_directory" will be created under /home/username/Documents)

2. Get a file's parent directory.

This is useful if you want to get the path to a file/folder that you know is under the working file's parent directory. If this was confusing, here's an example.

In the Linux shell (bash or otherwise), you can issue the command pwd (which I believe stands for "print working directory") to quickly print your current location within a given session.

In Python you would achieve this like so:

import os
+
+CURRENT_DIR = os.getcwd()
+

Notice I used all caps for the variable name. This is because it's usually a constant. This variable isn't meant to be changed. You can now use this variable to locate any file or folder within that directory. Keep reading and I'll show you how.

3. Concatenate paths

This is something you'll find yourself doing a lot. Especially on large projects that require configuration files and other such things. If you have experience working with Django for example, they have a settings.py file littered with calls to the os module. There are many benefits to this approach. Perhaps the most obvious being is that if you ever decide to move your project to another location, you don't want to keep modifying the path every time. Remember, programming is all about being lazy.

So this is how you would do it:

import os
+
+# Assuming this file is located at /home/username/myproject/app.py
+# and that you want to operate on a a file called config.cfg within the same
+# directory :
+
+CURRENT_DIR = os.getcwd() # evaluates to /home/username/myproject
+MY_TEXT_FILE = os.path.join(CURRENT_DIR, "config.cfg")
+
+print(MY_TEXT_FILE) # /home/username/myproject/config.cfg
+

An important note: os.path.join() merely concatenates the two paths together. It doesn't check whether the path is valid. So be careful when using this method. Also notice how the method call is to os.path.join() and not os.join()

4. Check that a path exists

The other day I was working on a small web scraper for a side project of mine. After the it was done fetching data, my script would save the results into a pickle file (don't worry if you don't know what it is) that would be read by my program, saving me the trouble of sitting there waiting to fetch the same info over and over again each time I run the script.

The solution was to tell my script to check whether a specific file (let's call it results.pkl) exists at a given path. If it does, the program continues and if not, the program executes the crawler function.

This is clever because now I only have to fetch the data and if the file gets deleted I know I can rely on the program to go and crawl the sites as expected.

And now for the example:

import os
+
+CURRENT_DIR = os.getcwd()
+RESULTS_FILE = os.path.join(CURRENT_DIR, "results.pkl")
+
+def crawl_data():
+    # scrapes a bunch of websites and saves the result in a file called
+    # results.pkl under the current directory
+    pass
+
+if not os.path.exists(RESULTS_FILE):
+    crawl_data()
+
+else:
+    ## the file exists so we can open it and work with its content
+

The same thing can be done to check that the path exists AND that it's a directory:

CURRENT_DIR = os.getcwd()
+MY_DIRECTORY = os.path.join(CURRENT_DIR, "my_directory")
+
+if os.path.exists(MY_DIRECTORY) and os.path.isdir(MY_DIRECTORY):
+    ## do something with the files inside the folder
+else:
+    os.mkdir(MY_DIRECTORY)
+

5. List files within a given directory

Very useful when you want to read several files that are under the same directory. It can be done in two ways: conventional and pythonic. I'll show you both.

for file in os.listdir("/path/to/dir"):
+    # do something with the filename (open it, copy it,  move it, rename it...)
+
+

or

filenames = [file for file in os.listdir("/path/to/dir")]
+

Guess which way is more pythonic!

Conclusion

These have been the most useful file/directory functions for me in Python. I really love the fact that the method names sound natural and are (for the most part) similar to linux commands. What are your favourite file operation methods? Any tip or trick you want to share with me? Something I've missed? Ping on twitter! I'm @aaqaishtyaq.

If you found this article useful, please share it with your nerd friends/coworkers and spread the word!

\ No newline at end of file diff --git a/notes/build-deploy-react-app-with-nginx/index.html b/notes/build-deploy-react-app-with-nginx/index.html new file mode 100644 index 0000000..5c158cf --- /dev/null +++ b/notes/build-deploy-react-app-with-nginx/index.html @@ -0,0 +1,55 @@ +Aaqa Ishtyaq

Deploy a React app with sass using Nginx

2655 words, 13 minutes to read

A couple of days ago (at the time of writing), I started my newest side project. It's a portfolio showcasing my (very very very) amateur photography. It's written in React.js with Sass and I have to say it was extremely enjoyable to work on. Unsurprisingly though, I ran into some issues while deploying to production, which after a lot of head banging against every possible flat surface I could find, I managed to sort. So this post will be about how to make React.js work with sass in production and how to serve the project using Nginx as a front end web server.

We'll be using the official starter kit / CLI tool provided by the facebook team called create-react-app.

You can install it by running the following command:

npm install -g create-react-app
+

Adding Sass to a React project

Including sass in a React app can be done in two ways: You can either eject the project and manually modify the webpack config files or you can follow the procedure in the official docs. I've personally chosen to go with the first approach. You can just follow the steps in this tutorial to get up and running.

Note: If you went with the second approach, you can just skip the following and directly jump to the next section

Once you've done that, open the config/webpack.config.prod.js file, locate the rules section under module.exports and add the following snippet to it.

{
+    test: /\.sass$/,
+    use: ExtractTextPlugin.extract({
+        fallback: require.resolve("style-loader"),
+        use: [require.resolve("css-loader"), require.resolve("sass-loader")]
+    }),
+    include: paths.appSrc,
+}
+

This config will be executed when you call npm run build. If you don't do this, you'll end up with an empty css file.

Important note

Before building the project for the first time, we must first unregister the service worker. Doing this will ensure that the static assets aren't cached by the client's browsers. It's important because if you skip this step, any subsequent change / build you'll deploy won't necessarily be reflected right away client side. (I'm not knowledgable enough on service workers to provide details on this behaviour, but it's something to note. If you really need service workers in your project, you might want to explore other solutions to avoid asset caching).

Your index.js file should now look something like this:

import { unregister } from './registerServiceWorker'
+import App from './App';
+ReactDOM.render(<App />, document.getElementById('root'));
+unregister()
+

Building the project

This is the part where we actually build our project. By building I mean compiling all React files and their related dependencies, transpiling them into vanilla Javascript, compiling sass files into css, minifying them etc ... To do this, we can simply run npm run build and voila ! You should have a brand new build folder in your project.

Creating our deploy script

Your app is now compiled and ready to be served. Now is a good time to start thinking about deployment strategies. In this basic scenario (and quite frankly, most scenraios), you really just want to upload the build folder to a remote server and have it accessible as a static site.

You can use ftp to transfer the files and it would be perfectly acceptable, but it's not the most flexible solution. The alternative is to use a CLI utility called rsync (which is available on mac and linux, not sure about windows). With rsync, you can synchronise files and folders within the same computer or across machines you have ssh access to.

Here's the command we would run to synchronise the build folder to a server on the internet:

# Asssuming we're inside the project folder
+rsync -avP build/ username@remote_ip:/destination/path
+

Let's break down this command:

-a means archive, which is a shortcut for multiple switches. It recursively syncs all files and subfolders within build to the destnation path, keeping the modification dates, permissions and other metadata unchanged.

-v means verbose. It just outputs the steps to the screen so you can see what happens in real time.

-P stands for progress. This is particularly useful in this case because you rely on a network connection to sync the files. Using this option will display a progress bar for each file in the queue.

But you don't want to keep doing all of that every time we want to push now do you ?

Thankfully, you can use create a bash script to automate this process a litte bit. Here's how mine looks like:

#!/bin/sh
+
+echo "[log] - Merging branch to master"
+git checkout master && git merge develop && git push origin master
+echo "[log] - Merge completed"
+
+echo "[log] - Compiling project to build folder ..."
+npm run build
+echo "[log] - Build process done"
+
+echo "[log] - Deploying files to server"
+rsync -avP build/ user@host:/destination/path
+echo "[log] - Deployment completed"
+
+echo "[log] - Switching to develop"
+git checkout develop
+echo "[log] - Done!"
+

Again, let's walk through that script section by section:

  1. I checkout to master, merge develop and push. This ensures that my master branch is always up to date with the latest version of my working codebase.
  2. I execute npm run build which, as previously explained, will create the build directory with our compiled, ready to be deployed files.
  3. I use rsync to copy over the contents of the build folder to the destination path in the remote machine I administer. (notice the trailing slash after build/, this tells rsync to copy the contents of the folder and not the folder itself).
  4. I switch the current working branch back to develop so that I can start developing without accidentally altering the state of master.

Obviously this is very basic and in a more complex project, you'd have to run unit tests and do other things your project requires.

Finally, you need to give executable permissions to the file by running:

sudo chmod +x deploy.sh
+

Now all you have to do when you want to deploy your project to production is run

./deploy.sh
+

Serving our site with Nginx

Note: this assumes your server is running ubuntu or any other debian based distro

Ok, so if you've followed the steps correctly, you should have your project files uploaded to your remote server. Now we need to use Nginx to make the site accessible to the internet.

First, create a new config file inside /etc/nginx/sites-available.

cd /etc/nginx/sites-available
+touch mywebsiteconfig # File extension is not required in this case
+

Next step is to edit the file using either vim or nano (sudo privileges may be required).

server {
+
+    listen 80;
+    server_name mywebsite.com www.mywebsite.com;
+
+    location / {
+        root /path/to/your/project;
+        index index.html index.htm;
+
+        default_type "text/html";
+    }
+
+    access_log /var/log/nginx/mywebsite_access.log;
+    error_log /var/log/nginx/mywebsite_errors.log;
+}
+

If you're not familiar with Nginx, let me explain what you just copied.

  1. We created a server block to hold our configuration and keep it separate from other configs we may add in the future (like https for example).
  2. We declare the listen directive which tells nginx to listen on port 80.
  3. We set the server_name to our domain name. This tells nginx to apply the config settings to any incoming request from any of listed urls.
  4. Finally, we specify the paths to both the access and error logs. It's optional but highly recommended, so that you know exactly where to look when errors happen. This will save you a tonne of time when troubleshooting issues in the future.

Close the file and exit nano or vim.

Nginx keeps its configuration files in two separate directories: /etc/nginx/sites-available and /etc/nginx/sites-enabled. It will serve any website whose configuration file is in the latter folder.

All you have to do now is create a symlink (think of it like a shortcut to an app in a desktop GUI) to your config, and store it in sites-enabled. That way, if you ever decide to shut down the site, you'll simply need to delete the symlink and you're good.

Before creating the symlink, it's good measure to check if the configuration file has any errors in it. To check for errors just run the following command:

sudo nginx -t
+

It will scan all of your config files and check for errors (and return them to you if there are any).

Now you can create the symlink by executing:

sudo ln -s /etc/nginx/sites-available/mywebsite /etc/nginx/sites-enabled
+

Note: If you decide to use service workers with your project and you're still running into caching issues, you should know that Nginx can also be used as a static assets server. I haven't looked into this scenario yet. I might test that approach in the future and detail the process in another post.

Lastly, reload nginx and your website should now be accessible via its URL (provided you correctly setup the DNS settings with your domain name registrar)

sudo service nginx reload
+

If you read this article all the way through, thanks for sticking with me ! You can send me questions, remarks, or comments on twitter, I'm @aaqaishtyaq on twitter.

\ No newline at end of file diff --git a/notes/deploy-django-application-git/index.html b/notes/deploy-django-application-git/index.html new file mode 100644 index 0000000..e867a51 --- /dev/null +++ b/notes/deploy-django-application-git/index.html @@ -0,0 +1,78 @@ +Aaqa Ishtyaq

Deploy your Django application with git

2595 words, 13 minutes to read

I'm going to make a bold statement: Django replaced Ruby on Rails in the hearts of many developers. With this increase in popularity, we've seen tons or articles, videos and websites dedicated to setting un Django and creating apps using the framework.

Unfortunately, when it comes to deployment, many of these resources only mention heroku or pythonanywhere. While these are excellent solutions for quickly shipping your MVP or prototype, it lacks a bit in terms of flexibility if you want to create your custom deployment pipeline.

tl-dr: If you manage your own server infrastructure, we're going to setup a git deployment workflow with django

What you'll need

  • Working knowledge of ssh
  • Working knowledge of git
  • Working knowledge of the bash shell
  • Basic linux command line skills (sorry hipsters)
  • (Very) Basic knowledge of vi/vim
  • Patience

The typical workflow usually looks like this:

  • You have your development environment, either on your local machine or a remote server
  • A git server (on GitHub, BitBucket, GitLab ...) that you and your team push your work to
  • A production server (aka your live app).

Usually when you commit and push work you do something like:

git push origin <branch_name>
+

origin being the name of the remote server your code is being pushed to. What took me a while to realise is that you can have many remotes for your repo that point to different servers.

The idea here is to add a new remote to our repo. It will point to our production server such that when we run git push live master, our code will be copied over to there.

On the production server

To achieve this, we have some setup work to do on our live server. So go ahead and connect to it via ssh.

ssh user@host_or_ip_address
+
+# If your server's ssh service listens on a port other than 22, you'll need
+# to add the -p switch
+
+ssh -p PORT user@host_or_ip_address
+

Once we're in, we need to create a new directory for our application. This is where our deployed code will be copied to.

mkdir -p /home/user/sites/myawesomedjangoproject
+
+# Some people prefer to use /var/www/<project_name>, it's really up to you. Just
+# make sure remember the path to your project
+

Now head over to /var and create another directory called repos

cd /var
+sudo mkdir repos
+# Depending on your setup, you might need sudo priveleges
+

Inside that directory, we need to create a folder named after our project (or domain name) and append it with .git (not necessary but it's good practice)

sudo mkdir myawsomedjangoproject.com.git
+

Inside this folder we'll create what is called a bare repository. To do this just run:

git init --bare
+

If you run ls inside that folder you'll see a bunch of files and directories (the same ones found inside the .git folder in normal repos). One of these directories is called hooks.

Inside that folder, we'll need create a file called post-receive.

# Assuming you are inside /var/repos/yourproject.git
+sudo touch hooks/post-receive
+

Now open it up with vi/vim

sudo vim hooks/post-receive
+

Hit i to switch to insert mode, and add the following to the file:

#!/bin/bash
+DEPLOYDIR=/home/username/sites/myawesomedjangoproject # or whatever path you chose
+GIT_WORK_TREE="$DEPLOYDIR" git checkout -f
+

Please note that the first shebang line is important, it instructs git to use bash instead of the default shell. Otherwise it won't activate our (soon to be created) virtual environment

exit vim by hitting :wq (which in vim lingo means write and quit)

What we've done here is set two variables. DEPLOYDIR is an alias for our project path on the server, and GIT_WORK_TREE which is a special variable that tells git to copy the code it receives inside of our DEPLOYDIR. This ensures that we're always running the latest version of our code.

As you've probably noticed, this post-receive file looks very much like a shell script. That's because it is (as explained above). It's executed every time you push code to the repo.

The last thing we need to is make the script executable, so as soon as you're back in the shell run:

sudo chmod +x hooks/post-receive
+

You can now exit the server and go back to your local machine.

On our local dev environment

Now that we've created our remote repository, we need to add it to our project (I like to call mine live).

It takes one simple command:

git remote add live root@ip_address:/var/repos/myawesomedjangoproject.git
+
+# And if your server's ssh service listens on a different port :
+
+git remote add live ssh://root@ip_address:PORT/var/repos/myawesomedjangoproject.git
+
+

To make sure it was added, you can print the list of available remotes by running:

git remote -v # v for verbose
+
+

and that's it ! You can now make changes locally, commit and deploy them live (or staging if it's a staging server) and see your changes instantly.

You can obviously still push to github/lab or bitbucket with git push origin <branch> like you normally would.

Bonus

As I mentioned in the first part, the post-receive hook is a shell script. Which means you can use it to perform all kinds of tasks against your code, like running front-end builds, installing dependencies, etc ...

Here's an example for a basic Django App:

#!/bin/bash
+
+DEPLOYDIR=/home/username/site/myawesomedjangoproject
+
+echo "[log] - Starting code update "
+GIT_WORK_TREE="$DEPLOYDIR" git checkout -f
+echo "[log] - Finished code update "
+
+if [[ -d "$DEPLOYDIR/ENV_projectname"]]; then
+    echo "[log] - Cleaning virtualenv"
+    cd "$DEPLOYDIR"; rm -rf ENV_projectname cd -
+    echo "[log] - Finished creating virtualenv"
+fi
+
+echo "[log] - Creating virtualenv"
+cd "$DEPLOYDIR"; virtualenv -p python3 ENV_projectname; cd -
+echo "[log] - Finished creating virtualenv"
+
+echo "[log] - Activating virtualEnv"
+cd "$DEPLOYDIR"; source ENV_projectname/bin/activate; cd -
+echo "[log] - Finished activating virtualenv"
+
+echo "[log] - Pulling down pip dependencies"
+cd "$DEPLOYDIR"; pip install -r requirements.txt; cd -
+echo "[log] - Finished pulling down pip dependencies"
+
+echo "[log] - Staring DB migration"
+cd "$DEPLOYDIR"; python manage.py makemigrations; python manage.py migrate; cd -
+echo "[log] - Finished DB migration "
+
+echo "[log] - Pulling Node Dependencies"
+cd "$DEPLOYDIR"; sudo npm install; cd -
+echo "[log] - Finished Pulling Node Dependencies"
+
+echo "[log] - Building the Front end"
+cd "$DEPLOYDIR"; sudo gulp build; cd -
+echo "[log] - Finished building the Front end"
+
+echo "[log] - Collecting static assets"
+cd "$DEPLOYDIR"; python manage.py collectstatic --clear --no-input; cd -
+echo "[log] - Finished collecting static assets"
+
+echo "[log] - Restarting App"
+sudo service myawesomedjangoapp restart;
+echo "[log] - Finished collecting static assets"
+
+

I run my Django Apps as systemd services, if you don't you can just call python manage.py runserver. If you want to know how to setup Django the way I do just follow this very comprehensive tutorial over on Digital Ocean

Conclusion

I am fully aware that there are more sophisticated methods of deployment through Docker, Travis (For continious integration) etc. But if you have a small app that you want to ship and you already have an infrastructure, I've found this method to be more than suitable.

Please report any missing info, mistake, error, typo. I'm on twitter if you wanna chat.

\ No newline at end of file diff --git a/notes/dont-be-scared-switch-to-vim/index.html b/notes/dont-be-scared-switch-to-vim/index.html new file mode 100644 index 0000000..8867e64 --- /dev/null +++ b/notes/dont-be-scared-switch-to-vim/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Don't be scared. Switch to vim.

608 words, 3 minutes to read

I'm currently sitting at the most boring meetup I've probably ever attended in Delhi. It's about chatbots. I don't care about chatbots, I care about free stickers and pizza. So I'll take this opportunity to open up about a subject that's dear to my heart: vim.

I used to believe vim was exclusive to this superior race of developers who gulp coffee like it's water and seem to only read HN and nothing else. (Hi, if you're coming from HN). Architecture and Software design comes naturally to them, they never run into bugs and they can recognize the most obscure of algorithms at a glance (Shout out to Shashank, one of my mentors).

Shanky is a good, productive developer. Shany uses vim. I want to be like Shanky. I want to use vim.

There are a million reasons why you should jump ship and join the cult. In the next paragraphs, I will detail some of these reasons.

It's not (that) hard

There's a learning curve to vim. But it's worth the trouble. And if you're on Linux or MacOS, there's a built-in tool called vimtutor (just fire it up from a terminal, I am not sure about Windows though) and a wide variety of online tools to learn vim. Namely openvim, vim adventures, and vim genius.

Personally, The way I learned was by using it on small, fun side projects of mine during the weekends, mostly to become familiar with the new mental model. And just like everything in life, shit takes time, and practice makes perfect. So keep at it and you'll eventually come to your "aha" moment. As you get more and more comfortable using vim, it will become harder and harder to go back to a regular editor / IDE.

It's Fast and Customisable

Because it runs on the terminal, you'll never have to wait 20 seconds to get on with your work. (Atom anyone ?)

And if you like pretty things, there's a large selection of colorschemes for you to choose from. On top of that, there's a plugin for just about anything you might need. And if there isn't, you can program your own.

Ubiquity

Not really, but I wanted to place a complicated word to sound smart. Seriously though, it's everywhere. On Mac OS, Windows and of course Linux/Unix. If you work on remote servers you can quickly edit files on the fly without having to use nano. (Don't use nano)

Say for example a coworker/friend is running into a bug, you come to help and they're using an IDE you're not familiar with, well you can just access the files from their terminal and start debugging right away.

Or if you're like me, and you spill water on your Macbook keyboard and it becomes toast, you can spin up a VPS on Digital Ocean or AWS, and pick up where you left off (almost) right away.

Bonus: Some of my favorite plugins

My color scheme of choice (at the time of writing) is afterglow.

And here's a list of my favorite plugins:

  • Nerdtree (A tree explorer much like the sidebar in traditional IDEs)
  • Airline (A sleek, customizable status bar)
  • Surround (Helpful tool that helps with "surrounding" words with brackets etc)
  • CtrlP (A fuzzy finder for vim)
  • UtilSnips (Snippet utility for many languages)
  • Vim Markdown (Markdown syntax highlighting)
  • Goyo (Allows for distraction-free editing)

I'll end this article with a quote from a Chamillionaire:

They see you vimmin', they hatin'. Patroling they tryna catch me coding dirty

\ No newline at end of file diff --git a/notes/firecracker-dev-machine-setup/index.html b/notes/firecracker-dev-machine-setup/index.html new file mode 100644 index 0000000..7eb6abb --- /dev/null +++ b/notes/firecracker-dev-machine-setup/index.html @@ -0,0 +1,29 @@ +Aaqa Ishtyaq

Firecracker Dev Machine Setup (GCP)

1107 words, 6 minutes to read

VM on Google Compute Engine (GCE) supports nested virtualisation, unlike AWS, and allows to run KVM. We can create a VM on GCE to run and test Firecracker microVM.

Here are the steps

  1. Setup GCP Project and Zone

    FC_PROJECT=your_name-firecracker
    +FC_REGION=us-east1
    +FC_ZONE=us-east1-b
    +
    gcloud config set project ${FC_PROJECT}
    +gcloud config set compute/region ${FC_REGION}
    +gcloud config set compute/zone ${FC_ZONE}
    +
  2. Create a VM image (Machine Image) able to run KVM

    FC_VDISK=disk-ub22
    +FC_IMAGE=ub22-nested-kvm
    +
    +gcloud compute disks create ${FC_VDISK} \
    +  --image-project ubuntu-os-cloud \
    +  --image-family ubuntu-2204-lts
    +
    +gcloud compute images create ${FC_IMAGE} \
    +  --source-disk ${FC_VDISK} \
    +  --source-disk-zone ${FC_ZONE} \
    +  --licenses "https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"
    +
  3. Create the VM

    FC_VM=firecracker-vm
    +
    +gcloud compute instances create ${FC_VM} \
    +  --zone ${FC_ZONE} \
    +  --image ${FC_IMAGE}
    +
  4. Connect to the VM via SSH.

    gcloud compute ssh ${FC_VM}
    +
  5. Verify that VMX is enabled, enable KVM

    $ grep -cw vmx /proc/cpuinfo
    +1
    +$ sudo setfacl -m u:${USER}:rw /dev/kvm
    +$ [ -r /dev/kvm ] && [ -w /dev/kvm ] && echo "OK" || echo "FAIL"
    +OK
    +
\ No newline at end of file diff --git a/notes/git-exclude-files/index.html b/notes/git-exclude-files/index.html new file mode 100644 index 0000000..04685c2 --- /dev/null +++ b/notes/git-exclude-files/index.html @@ -0,0 +1,3 @@ +Aaqa Ishtyaq

Git exclude files from working copy

260 words, 1 minutes to read

How to ignore new files

Globally

Add the path(s) to your file(s) which you would like to ignore to your .gitignore file (and commit them). These file entries will also apply to others checking out the repository.

Locally

Add the path(s) to your file(s) which you would like to ignore to your .git/info/exclude file. These file entries will only apply to your local working copy.

How to ignore changed files (temporarily)

In order to ignore changed files to being listed as modified, you can use the following git command:

git update-index --assume-unchanged <file-1> <file-2> <file-3>
+

To revert that ignorance use the following command:

git update-index --no-assume-unchanged <file-1> <file-2> <file-3>
+

Note: If you find errors, typos or would like to add new tips, feel free to reach out to me on twitter. I'm @aaqaishtyaq. Thank you for reading ! And if you find this useful, share it with your friends and coworkers !

\ No newline at end of file diff --git a/notes/hugo-open-markdown-new-tab/index.html b/notes/hugo-open-markdown-new-tab/index.html new file mode 100644 index 0000000..d85f57f --- /dev/null +++ b/notes/hugo-open-markdown-new-tab/index.html @@ -0,0 +1,15 @@ +Aaqa Ishtyaq

Open markdown links in a new tab

1132 words, 6 minutes to read

I created a static website linkbin, Where I could save links in a markdown file that can be viewed later. It is serving its purpose and because it uses hugo, I don't have to think about VPS cost.

Recently, I ran into an issue where my website serves links but when you click on it, It will open the link in the same tab. Ugggh! I don't want to close my website just to open a link. Ideally, it should open the link in a new tab.

How can we do it? It would be easy Eh?

The generic way

Since markdown is eventually rendered as HTML, The most naive way I found was to write HTML in markdown

<a href="http://example.com/" target="_blank">example</a>
+

No way, I am going to rewrite all my links into HTML. It adds cost later on as well. I will have to add HTML tags to write the link instead of this syntax:

[link]example.com
+

The specialized way

This could vary depending upon the markdown parser you are using. If you're using Jekyll, like this website, then you can use the following syntax to open links in a new tab.

[link]url{:target="_blank"}
+

But, I am not using Jekyll for linkbin. It uses a fairly newer version of Hugo. Hugo ,v0.62.0 or later, uses Goldmark parser and supports Markdown render hooks. Markdown Render Hooks offer you several ways to extend the default markdown behavior, e.g. resizing uploaded images, opening links in new tabs, or creating mermaid diagrams from code. You can do this by creating templates in the layouts/_default/_markup directory with base names render-link or render-image or render-codeblock. Your directory layout may look like this:

layouts
+└── _default
+    └── _markup
+        ├── render-image.html
+        ├── render-image.rss.xml
+        ├── render-link.html
+        ├── render-codeblock.html
+        └── render-codeblock-bash.html
+

Add the following HTML template file (or render hook) at layouts/_default/_markup/render-link.html:


+<a href="{{ .Destination | safeURL }}"{{ with .Title}} title="{{ . }}"{{ end }}{{ if strings.HasPrefix .Destination "http" }} target="_blank" rel="noopener"{{ end }}>{{ .Text | safeHTML }}</a>
+

You'll find that the links now open in a new tab! For internal blog links (which you would want to open in the same tab), you can use the relative link of the post, e.g. for an other-post.md file within the posts directory, you could use

[Other post]/posts/other-post/
+

If I’ve missed something or made a horrible mistake if you have any questions regarding this article then feel free to ping me on Twitter. I’m @aaqaishtyaq.

\ No newline at end of file diff --git a/notes/index.html b/notes/index.html new file mode 100644 index 0000000..667252b --- /dev/null +++ b/notes/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Notes

\ No newline at end of file diff --git a/notes/linux-container-networking/index.html b/notes/linux-container-networking/index.html new file mode 100644 index 0000000..fcf0200 --- /dev/null +++ b/notes/linux-container-networking/index.html @@ -0,0 +1,73 @@ +Aaqa Ishtyaq

Linux Container Networking from Scratch

2839 words, 14 minutes to read

In this article, we will be looking into setting up networking on a linux box from scratch.

We will be creating a fresh new VM using Lima on macOS. You can create a new VM using VirtualBox, Vagrant, or even create a VM on OCI for free.

Isolation based on Network Namespace

The common thing that constitutes a network stack includes the:

  • Network Devices
  • Routing Rules
  • Netfilter Hooks

We can create a non-comprehensive inspect-net-stack script.

#!/usr/bin/env bash
+
+echo "> Network devices"
+ip link
+
+echo -e "\n> Route table"
+ip route
+
+echo -e "\n> Iptables rules"
+iptables --list-rules
+

After the execution of the inspect script on my machine produces the following input:

% sudo ./inspect-net-stack
+> Network devices
+1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
+    link/ether 52:55:55:a3:ad:92 brd ff:ff:ff:ff:ff:ff
+    altname enp0s1
+
+> Route table
+default via 192.168.5.3 dev eth0 proto dhcp src 192.168.5.15 metric 100
+192.168.5.0/24 dev eth0 proto kernel scope link src 192.168.5.15 metric 100
+
+> Iptables rules
+-P INPUT ACCEPT
+-P FORWARD ACCEPT
+-P OUTPUT ACCEPT
+

We want to make sure that each of the containers we are going to create soon will get a separate network stack.

Linux namespaces used for container isolation are called network namespaces. We won't be creating the fully isolated container, rather restrict the scope to only the network stack. One of the ways to create a network stack is by using the ip tool, part of iproute2

% sudo ip netns add netns0
+% ip netns
+netns0
+

We have a new network namespace but to start using the network namespace, We can use a command called nsenter. It enters one or more of the specified namespaces and then executes the given program:

# This is create a new bash process under netns0 namespace
+% sudo nsenter --net=/var/run/netns/netns0 bash
+
+
+% sudo ./inspect-net-stack
+> Network devices
+1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+
+> Route table
+
+> Iptables rules
+-P INPUT ACCEPT
+-P FORWARD ACCEPT
+-P OUTPUT ACCEPT
+

As you can see from the above output, the bash process running inside the netns0 namespace sees a different network stack. There are no routing rules and no custom iptables chain. Only one loopback device (lo).

                        Network Namespace
+┌─────────────────────────────────────────────────────────────────────┐
+
+                                          Root Namespace             │
+
+ lo0                    ┌────────────────────────────────────────┐   │
+ ┌─┐                    │                                        │   │
+ │ │                    │    lo0             Container Namespace │   │
+ │ │                    │    ┌─┐              netns0             │   │
+ │ │    ┌────────────┐  │    │ │                                 │   │
+ │ │    │            │  │    │ │  ┌───────────┐                  │   │
+ │ │    │ routes     │  │    │ │  │           │                  │   │
+ │ │    │            │  │    │ │  │routes     │                  │   │
+ │ │    └────────────┘  │    │ │  └───────────┘                  │   │
+ └─┘                    │    └─┘                                 │   │
+                        │                                        │   │
+ ┌─┐    ┌────────────┐  │         ┌───────────┐                  │   │
+ │ │    │            │  │         │           │                  │   │
+ │ │    │ iptables   │  │         │iptables   │                  │   │
+ │ │    │            │  │         └───────────┘                  │   │
+ │ │    └────────────┘  │                                        │   │
+ │ │                    │                                        │   │
+ │ │                    │                                        │   │
+ └─┘                    │                                        │   │
+ eth0                   └────────────────────────────────────────┘   │
+
+
+└─────────────────────────────────────────────────────────────────────┘
+

Beware: The nsenter command from above started a nested bash session in the netns0 network namespace. Don't forget to exit from it.


In the upcoming posts, I will discuss connecting the host with this namespace. Interconnecting various network switches, Just like patching the physical switch with a LAN cable.

\ No newline at end of file diff --git a/notes/mind-blowing-git-tips/index.html b/notes/mind-blowing-git-tips/index.html new file mode 100644 index 0000000..33851ac --- /dev/null +++ b/notes/mind-blowing-git-tips/index.html @@ -0,0 +1,7 @@ +Aaqa Ishtyaq

Mind-blowing git tips for beginners

954 words, 5 minutes to read

As developers, we all (hopefully) use git. It's not extremely hard nor time-consuming to get started with it and you will surely thank your future self for taking the time to learn it. Ok so that was for the cringy intro.

Now let's get down to business: what I really want to share in this post is a list of tricks I've learned during the past 2 1/2 years of using git. Some of it might seem trivial to seasoned developers but if you're just getting started, stick with me because this might just blow your mind.

The difference between git fetch and pull

git fetch only updates the tracking remote branches. If you actually want to update the local repo, you need to merge the local branch with the remote tracking branch using git merge.

remote branches are prefixed with the name of the remote and a slash: origin/branchname

git pull on the other hand, will execute both commands for you so when you issue a git pull in a branch it will fetch that branch from the remote repo and merge it with yours.

To list all remote trackin branches, git branch -r is your friend.

Merge conflicts

At some point you'll inevitably have to deal with merge conflicts. Essentially what this means is that git noticed a file was modified on both branches and it does not know which version is correct. It leaves it up to you to decide which one you want to push.

if you open the file you will see a HEAD part prefixed with <<<< signs and a second part containing the code on the branch you're trying to merge. Both sections are separated with an equal sign.

The top HEAD section contains the version of the file as it is on your current branch. The other shows you what the code looks like on the branch you're trying to merge from.

To resolve the conflict just delete the part you don't want (Including all the equal signs etc), save the file and commit it again.

Note that as a safety measure it's always good practice to do a git pull to see if you don't have any remaining conflicts. If not you can just push your code and you're all set!

As for the tips:

git merge --abort will clean up the current working directory and go back to the last version before the merge (it pretty much nullfies the merge).

git merge --squash is an interesting one as it will bring the changes you made in the feature branch and create a new commit on your current branch without mixing the two histories.

Dealing with files

Now this one's tricky !

To remove a file from both the working directory and version control, you can use git rm <filename> but if you already deleted a file in the cli like I (and almost everyone) usually do then just run git rm <path to the file> and it will stage it as deleted so you can commit.

But what if you ramrafed (rm -rf) a bunch of files in your working directory? You don't really want to manually stage every file for deletion, do you? (DO YOU ?) So what do you do in that case? Well, you just run git add -u (u for updating the working tree) and all of your deleted files will be staged so you can commit and push.

"What if a man accidentally tracked a file and man doesn't want it deleted from the index ?" I hear you ask with an exquisite South London accent.

Well, my G, just run:

git rm --cached <name of the file>
+

and you're done. Efficience ting.

Something else that happens quite often is moving/renaming files (which is essentially the same thing on Linux systems, as the path/address of the file changed).

Here's an example: imagine you have a file called index.html that you want to rename to home.html. you can use git mv which takes a source and a destination:

git mv index.html home.html
+

if you want to move and rename just run

git mv index.html path/to/home.html
+

However if you moved / renamed the file manually on the command line, running git status will tell you that index.html was deleted and home.html is untracked.

To fix this you will have to run two commands

git rm index.html # aka the file you moved/renamed
+
git add home.html # aka the file with the new name x path
+

Running git status again will mark the file as renamed.

Note that this also works if you don't rename the file. For example, if you just want to move index.html to src/index.html. The same command will apply (git mv index.html src/index.html).

The other way of achieving this is:

git add -A
+

which will pick up the changes and automatically stage them for commit.

So there you go, I hope you learned something useful in this article. If you know other mind-blowing tips and tricks for git, @ me on twitter.

\ No newline at end of file diff --git a/notes/mind-blowing-python-tips/index.html b/notes/mind-blowing-python-tips/index.html new file mode 100644 index 0000000..aef6ce1 --- /dev/null +++ b/notes/mind-blowing-python-tips/index.html @@ -0,0 +1,81 @@ +Aaqa Ishtyaq

Mind-blowing Python tips

4477 words, 22 minutes to read

0 - Loop over a range of numbers

Use range instead of xrange. In python3, the former creates an iterator that produces the values one at a time making it much more efficient and fast.


+nums = [0,2, 34, 55, 32]
+for i in range(nums):
+    print i
+
+

1 - Looping backwards

.reversed use Just


+names = ["Case", "Molly", "Armitage", "Maelcum"]
+for name in reversed(names):
+    print name
+
+

2 - Looping over a list and its indices

To keep track of the index of each item in a collection, enumerate is your buddy.

names = ["Case", "Molly", "Armitage", "Maelcum"]
+for index, name in enumerate(names):
+    print index, name
+
+

3 - Looping over two lists simultaneously

Yeah you could use zip, but izip is faster, so use that instead.


+from itertools import izip
+
+names = ["Case", "Molly", "Armitage", "Maelcum"]
+ages = [23, 27, 41, 24]
+for name, age in izip(names, ages):
+    print name, age
+
+

4 - Looping over a sorted list

You can sort out the list first and then loop through it, or you could use sorted.


+names = ["Case", "Molly", "Armitage", "Maelcum"]
+for name in sorted(names):
+    print name
+
+

And BAM, you're ... sorted.

5 - Call a function until a sentinel value is returned

To do that, use iter().

Bad example:

Loop over a file containing a list of names until the loop returns an empty string, in which case we break out of it.


+names = []
+while True:
+    name = file.read(32)
+    if name = "":
+        break
+    names.append(name)
+

Beautiful example:

In this case, we call a function (f.read) until it returns the sentinel value passed as a second argument to iter. That way we avoid having to make the unnecessary if check.

for name in iter( partial(f.read(32)), ""):
+    print name
+

6 - Looping over a dictionary

The normal way to do it:

molly = { "name": "Molly Millions", "Age": 27, "Occupation": "Professional Killer"}
+
+for key in molly:
+    print key
+

If you wish to mutate the data, prefer dict.keys().


+molly = { "name": "Molly Millions", "Age": 27, "Occupation": "Professional Killer"}
+
+for key in molly.keys():
+    # do the mutation
+
+

7 - Looping over a dict keys AND values

Don't do this:


+molly = { "name": "Molly Millions", "Age": 27, "Occupation": "Professional Killer"}
+
+for key in molly:
+    print molly[key]
+
+

It's slow because we have to rehash the dictionary and do a lookup everytime.

Instead choose iteritems():


+molly = { "name": "Molly Millions", "Age": 27, "Occupation": "Professional Killer"}
+
+for key, value in molly.iteritems():
+    print key, value
+
+

8 - Create a dict out of two lists

Just instantiate a new dict with two zipped lists. Real magic.


+from itertools import izip
+
+names = ["Case", "Molly", "Armitage", "Maelcum"]
+ages = [23, 27, 41, 24]
+
+characters = dict(izip(names, ages))
+
+

9 - Use named tuples for returning multiple values

Like in the case of an API response in Flask.


+from collections import namedtuple
+
+Response = namedtuple('APIResponse', ['status_code', 'body', 'headers'])
+
+@app.route('/users/1'):
+
+    try:
+        user = db.getuserbyid(1)
+    except:
+        return Response(404, user.notfound(), {'content-type': 'application/json'}
+    else:
+        return Response(200, user.json(), {'content-type': 'application/json'}
+
+

Other

  • Always clarify function calls by using keyword arguments

If you learned something from this article, share it with your co-workers and fellow hackers. If you notice any typo, error etc let me know on twitter.

\ No newline at end of file diff --git a/notes/recipe-of-great-rest-api/index.html b/notes/recipe-of-great-rest-api/index.html new file mode 100644 index 0000000..bbf8910 --- /dev/null +++ b/notes/recipe-of-great-rest-api/index.html @@ -0,0 +1,65 @@ +Aaqa Ishtyaq

The Recipe of a great RESTful API

2830 words, 14 minutes to read

I personally love writing APIs, it's my favourite thing about backend development. As developers we interact with APIs all the time, to process payments with stripe, for geolocation with google maps, to retrieve tweets etc ...

So what exactly makes a RESTful API great ? What can you do to ensure that it's easy to use for other developers ? Let's dive right in.

Responses

In your responses, you should separate the metadata from the the body and put them in their own objects.

// Example: GET /api/user/2
+
+{
+
+    "meta": {
+        "type": "success",
+        "code": 200,
+        "request_id": "someid"
+        // etc ...
+    },
+
+    "body": {
+
+        "id": 2,
+        "name": "Aaqa",
+        "location": "Delhi",
+        "job": "Software Developer"
+    }
+
+}
+

When creating a new resource, always return a 201 created instead of a 200 OK. It's a common mistake developers make. Another best practice is to return a url to the newly created resource in the location header of your response. You can also choose to send a serialised version of the resource in the response body. (I personally like to do that).


+// Example: POST /api/users
+
+// Response Headers
+
+// Content-Type: application/json
+// ...
+// Location: https://yoursite.com/api/users/2
+
+{
+    "meta": {
+        "type": "success",
+        "code": 201,
+        "request_id": "some_id"
+    },
+
+    "body": {
+
+        "message": "User successfully added",
+        "user": {
+
+            "id": 3,
+            "name": "Harry_Potter",
+            "location": "London",
+            "job": "Auror",
+            "href": "https://yoursite.com/api/users/3"
+        }
+    }
+}
+

Errors

Properly dealing with errors is crucial to provide the best experience possible to your users. Just like with normal responses, always include the status code in the meta part.

You should also provide a nice descriptive message detailing the cause of the error as well as a potential solution to fix it.

Doing this will help the consumers of your API to gracefully handle these errors on the client side. They will thank you for it.

// Example GET /api/secretendpoint
+
+{
+
+    "meta": {
+        "type": "error",
+        "status": 401,
+        "request_id": "weoifwe"
+    },
+    "body": {
+       "message": "Unauthorized. Please login or create an account"
+    }
+}
+

Status codes

This part is subject to many passionate debates, apparently it's a very touchy topic.

Below I will list the most common status codes, what they mean and how I use them.

  • 200 - OK - Request went well, errthang's fine.
  • 201 - Resource created - The resource was successfully saved to the database.
  • 204 - No Content - There used to be a resource at this endpoint but not anymore (Used for DELETE requests)
  • 400 - Bad request - The request was badly formatted. (ex: Invalid JSON)
  • 401 - Unauthorized - Authentication failed due to invalid credentials.
  • 403 - Forbidden - Authentication passed but the user does not have permission to access the resource
  • 404 - Not Found - Well, you know what it is
  • 405 - Method Not Allowed - The HTTP verb used to interact with the resource is not allowed. (eg: User tried to POST to a GET only endpoint)
  • 409 - Conflict - The data passed in the payload conflicts with the resource in the server. (I return this status code in cases where a user creates/updates a resource with a unique field constraint attached to it such as a username or email)
  • 422 - Unprocessable Entity - Again, another controversial one. I return a 422 when the user sends an incomplete payload thereby violating the NOT NULL constraint attached to those fields.
  • 429 - Too many requests - You made too many requests.
  • 50x - Server Related Errors - Something went wrong server side.

Caching

Caching is great and is a must when deploying an API. It helps a huge amount with performace by reducing server load as you are not required to perform the same action multiple times. You could choose to do it manually or use a reverse proxy like nginx or even something like varnish to handle that for you.

Caching will return a 304 Not Modified if the resource hasn't changed. The great thing about this is that it does not affect your rate limiting, so there's literally no excuse not to implement it.

Rate Limiting

For security and performance reasons you want limit the amount of requests made against your API. You must return the maximum and remaining number of request allowed for a given user in a particular time window (which you are free to determine).

CORS

You want to enable CORS to allow requests from browsers using AJAX. This can be done by setting the following headers:

Access-Control-Allow-Origin: * # this means allow all domains to interact with the API
+Access-Control-Allow-Headers: # All the headers you want to accept should be listed here
+

A few tips

  • Return urls to resources and total count when GETting collections
  • Map your endpoints to collections and resources
  • Always use plural nouns (Ex: /api/users not /api/user)
  • Actions on resources must be described with HTTP verbs
  • for partial updates of resources use PATCH instead of PUT
  • for versioning, use dates instead of numbers in the URL (foursquare does exactly that)

If you have any comment, suggestions or remarks of any sort, you can ping me on twitter @aaqaishtyaq !

\ No newline at end of file diff --git a/notes/scrapy-splash-setup/index.html b/notes/scrapy-splash-setup/index.html new file mode 100644 index 0000000..f86d170 --- /dev/null +++ b/notes/scrapy-splash-setup/index.html @@ -0,0 +1,19 @@ +Aaqa Ishtyaq

Setting up Scrapy Splash Plugin

1067 words, 5 minutes to read

Scrapy is good for scraping static web pages using python but when it comes to dynamic web pages scrapy can't do wonders, and there comes Selenium but as good as selenium is, it just got beaten by Scrapy in terms or speed.

Web nowdays is all about Dynamic JS based pages and AJAX. So for this very scenario the guys over scrapy-plugins created scrapy-splash. Scrapy-Splash is a plugin that connects Scrapy with Splash (Lightweight, scriptable browser as a service with an HTTP API). In a nutshell what splash do is it traps the response recieved from the server and renders it. Then it return a render.html which is static and can be easily scraped.

0 - Setting up the machine

A. Before we begin you need to install Docker first, You can follow the official instruction as per your Operating System.

B. After installing docker navigate to your project folder, activate virtualenv and install scrapy-splsh plugin

pip3 install scrapy-splash
+

C. Pull the Splash Docker Image and run it

docker pull scrapinghub/splash
+docker run -p 8050:8050 scrapinghub/splash
+

1 - Configuration

A. Add the Splash server address to settings.py of your Scrapy project like this:

SPLASH_URL = 'http://localhost:8050'
+

If you are running docker on your local machine then you can simply use http://localhost:<port> , but if you are running it on a remote machine you need to specify it's I.P. Address like this http://192.168.59.103:<port>

B. Enable the Splash middleware by adding it to DOWNLOADER_MIDDLEWARES in your settings.py file and changing HttpCompressionMiddleware priority:

DOWNLOADER_MIDDLEWARES = {
+    'scrapy_splash.SplashCookiesMiddleware': 723,
+    'scrapy_splash.SplashMiddleware': 725,
+    'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
+}
+

C. Enable SplashDeduplicateArgsMiddleware by adding it to SPIDER_MIDDLEWARES in your settings.py:

SPIDER_MIDDLEWARES = {
+    'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
+}
+

D. Set a custom DUPEFILTER_CLASS:

DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter'
+

2 - Scraping with Splash

Before you use scrapy-splash you need to import it in your spider. You can do that by adding this line:

from scrapy_splash import SplashRequest
+

from now on insted of using scrapy.Request you can simply use SplashRequest to get response from Splash insted of directly from ther server.

Bonus - Using Scrapy-Splash in Shell

It's all well and good but actual spider buiding does not happens in vim or sublime, it takes place in shell.

So how to use Splash in the shell?

Good Question.

Insted of invoking shell with:

scrapy shell
+>>> fetch(http://domain.com/page-with-javascript.html)
+

or with this:

scrapy shell http://domain.com/page-with-javascript.html
+

You invoke shell with this:

scrapy shell 'http://localhost:8050/render.html?url=http://domain.com/page-with-javascript.html&timeout=10&wait=0.5'
+

Let me explain

  • localhost:port is where your splash service is running
  • url is url you want to crawl
  • render.html is one of the possible http api endpoints, returns redered html page in this case
  • timeout time in seconds for timeout
  • wait time in seconds to wait for javascript to execute before reading/saving the html.

If I’ve missed something, made a horrible mistake of if you have any questions regarding this article then feel free to ping me on Twitter. I’m @aaqaishtyaq.

\ No newline at end of file diff --git a/notes/the-ultimate-postgresql-cheatsheet/index.html b/notes/the-ultimate-postgresql-cheatsheet/index.html new file mode 100644 index 0000000..a099854 --- /dev/null +++ b/notes/the-ultimate-postgresql-cheatsheet/index.html @@ -0,0 +1,261 @@ +Aaqa Ishtyaq

The ultimate PostgreSQL cheatsheet

4200 words, 21 minutes to read

So I had been working with Django, Flask and Express.js for a while now, and my database of choice for every single project has of course always been Postgres. What makes these frameworks great (regardless of language) is the ability to use an ORM (Object Relational Mapper) that sits between your code and the DB. It does all the heavy lifting for you and takes care of executing SQL queries on your behalf.

This is great but I couldn't stop thinking "what if I have to manually debug something directly into postgres ?" This is when reality slapped me in the face, I barely knew the SQL Language. So I thought It'd be fun to create a cheatsheet that I (and you) could keep as a reference for these times when you absolutely need to set this column to UNIQUE and you don't know how.

Enjoy ! (and yeah I went all in with the pokemon references)

Create a User (or Role)

/* create a user without privileges*/
+CREATE ROLE aaqa;
+
+/* create a user with privileges*/
+CREATE ROLE aaqa LOGIN CREATEDB CREATEROLE REPLICATION;
+
+/* Add privileges to existing user*/
+ALTER ROLE aaqa WITH LOGIN CREATEROLE CREATEDB REPLICATION;
+

Make a user superuser (bump their privileges)

ALTER ROLE aaqa WITH superuser;
+

Rename an existing user

ALTER ROLE psyduck RENAME TO brock;
+

Create a DB

CREATE DATABASE pokemons;
+

Rename a DB

ALTER DATABASE pokemons RENAME TO charizard;
+

Create a table in a DB

CREATE TABLE trainers(
+
+    id INT PRIMARY KEY NOT NULL,
+    pokemon_type CHAR(50),
+    name CHAR(50) NOT NULL UNIQUE,
+    gender CHAR(50) NOT NULL UNIQUE
+);
+

Delete a DB

DROP DATABASE pokemons;
+

Delete a user

/* assuming there's a pikachu role on the system */
+DROP ROLE pikachu;
+

Change DB ownership

ALTER DATABASE pokemons OWNER TO aaqa;
+

Rename a table

ALTER TABLE trainers RENAME TO gym_trainers;
+

Change column type

ALTER TABLE gym_trainers ALTER COLUMN pokemon_type TYPE TEXT; /* it makes zero
+sense to want to change this column type to TEXT but YOLO */
+

Rename a column

ALTER TABLE gym_trainers ALTER COLUMN name RENAME TO trainer_name;
+

Add a column to a table

ALTER TABLE gym_trainers ADD COLUMN bio TEXT;
+
+/*
+ALTER TABLE <table_name>
+ADD COLUMN <column_name> <data_type> <constraints if any>
+*/
+

Add a column with a UNIQUE constraint

ALTER TABLE gym_trainers ADD COLUMN age INT UNIQUE; /* cause why not */
+

Add a column with a NOT NULL constraint

ALTER TABLE gym_trainers ADD COLUMN main_pokemon CHAR(60) NOT NULL;
+

Remove NOT NULL CONSTRAINT from a column

ALTER TABLE gym_trainers ALTER COLUMN main_pokemon DROP NOT NULL;
+

Add a column with a NOT NULL constraint and a DEFAULT value

ALTER TABLE gym_trainers ADD COLUMN city CHAR(80) NOT NULL DEFAULT 'Indigo
+Plateau';
+

Add a column with a CHECK constraint


+CREATE TABLE trainers(
+
+    id INT PRIMARY KEY,
+    trainer_name CHAR(50) NOT NULL,
+
+    /*
+        method 1
+        Add the check yolo style
+    */
+    age INT NOT NULL CHECK (age > 18)
+
+    /*
+        method 2
+        Add a named constraint for better error handling
+    */
+    age INT CONSTRAINT legal_age CHECK (age > 18) NOT NULL
+
+    /*
+        method 3
+        Add the constraint at the end
+        for more clarity
+    */
+    age INT NOT NULL,
+    CONSTRAINT legal_age CHECK (age > 18) NOT NULL
+);
+

Add a CONSTRAINT to an existing column

ALTER TABLE trainers ADD CONSTRAINT unique_name UNIQUE (trainer_name);
+

Remove a named CONSTRAINT from a table

ALTER TABLE trainers DROP CONSTRAINT unique_name;
+

Insert a row into a table

INSERT INTO trainers VALUES (1, 23, 'brock');
+
+/* or */
+
+INSERT INTO trainers (age, trainer_name)
+VALUES (1, 23, 'brock');
+
+/*
+    Note that in the second case we don't have pass
+    the id. Postgres will automatically generate and autoincrement
+    it for us. To omit the id column we must use named inserts otherwise an
+    error is raised.
+*/
+

Insert multiple rows into a table


+INSERT INTO trainers (age, trainer_name) VALUES
+(19, 'misty'),
+(22, 'chen'),
+/* ... */
+/* ... */
+

Clear a table (without deleting it)

TRUNCATE trainers;
+

Set the primary key type to a serial (An auto incrementing integer)


+/* Considering this table structure */
+
+CREATE TABLE pokemon_list (
+
+    id INT PRIMARY KEY,
+    /* ... */
+    /* ... */
+    /* ... */
+);
+
+
+/*
+    1.
+    Create a sequence for the auto generating prinary key
+    It follows the tablename_columnname_seq
+*/
+CREATE SEQUENCE pokemon_list_id_seq;
+
+/* 2. Set the id column to not null */
+ALTER TABLE pokemon_list ALTER COLUMN id SET NOT NULL;
+
+/* 3. Set the default value to the next value in the sequence*/
+ALTER TABLE pokemon_list
+ALTER COLUMN pokemon_list
+SET DEFAULT nextval('pokemon_list_id_seq');
+
+/* 4. Link the sequence to the correct table and column */
+ALTER SEQUENCE pokemon_list_id_seq OWNED BY pokemon_list.id;
+

Import data from a file

/*
+    Must use absolute path and the user must have appropriate permissions
+    Defaults to importing using TAB as the default parameter.
+    We'll use a csv file as an example
+*/
+
+COPY pokemon_list FROM '/path/to/yourfile.csv' DELIMITER ',';
+
+/*
+    This only works if a pk is specified for each row
+    The (my) prefered way to do it is the following
+*/
+
+COPY pokemon_list (name, level, type) FROM '/path/to/yourfile.csv' DELIMITER ',';
+
+/* It's much more flexible because you control what data you actually import*/
+

Export a table to a file

To be able to export a table to a file, we need to ensure that postgres has write permissions to the file.

sudo chmod 777 /path/to/directory # This is just an example, edit this as needed
+

We can now safely copy the table to the file.

COPY pokemon_list TO '/path/to/file.csv' DELIMITER ',';
+

Select columns by using aliases

SELECT name AS pokemon_name, type AS pokemon_type
+FROM pokemon_list;
+

Select elements based on a criteria

SELECT *
+FROM pokemon_list
+WHERE id > 3;
+

Select elements based on string comparison

SELECT *
+FROM pokemon_list
+WHERE type LIKE '%water%';
+

Select all results and order them by id in reverse

SELECT *
+FROM pokemon_list
+ORDER BY id DESC;
+

Select all results and order them by a column name

SELECT *
+FROM pokemon_list
+ORDER BY level;
+
+/* If the column you're ordering by is not of type INT then the ordering will be
+done alphabetically */
+

Select DISTINCT column from table

SELECT DISTINCT type AS pokemon_type
+FROM pokemon_list;
+

Limit the results from a SELECT query

SELECT *
+FROM pokemon_list
+LIMIT 3;
+

Select the last 3 items

SELECT *
+FROM pokemon_list
+ORDER BY id DESC
+LIMIT 3;
+

Create two tables with a foreign key relationship

CREATE TABLE pokemon_types(
+
+    id SERIAL PRIMARY KEY,
+    type_name CHAR(120) NOT NULL
+)
+
+CREATE TABLE pokemon_list(
+
+    id serial PRIMARY KEY,
+    pokemon_name CHAR(120) NOT NULL,
+    pokemon_level INT NOT NULL,
+    pokemon_type INT REFERENCES pokemon_types(id) NOT NULL,
+    CONSTRAINT pokemon_level_not_zero CHECK (pokemon_level > 0)
+
+);
+

Perform Joins based on a criteria

SELECT name, level, pokemon_types.name AS type
+FROM pokemon_list
+JOIN pokemon_types
+ON pokemon_type_id  = pokemon_types.id
+WHERE pokemon_type_id = 1;
+
+/* Will return the name, level and type name for all water pokemons */
+

Perform joins

SELECT name, level, pokemon_types.name AS type
+FROM pokemon_list
+JOIN pokemon_types
+ON pokemon_type_id  = pokemon_types.id;
+

Create a VIEW based on a JOIN

CREATE VIEW pokemonswithtypes AS
+SELECT name, level pokemon_types.name AS type
+FROM pokemon_list
+JOIN pokemon_types
+ON pokemon_type_id = pokemon_types.id;
+
+/* To see the data */
+
+SELECT * FROM pokemonswithtypes;
+

Update the VIEW (Change the query)

SELECT name, pokemon_types.name AS type
+FROM pokemon_list
+JOIN pokemon_types
+ON pokemon_type = pokemon_types.id
+WHERE pokemon_type = 1;
+
+/* only show the name and type for water pokemons */
+

Delete the VIEW

DROP VIEW pokemonswithtypes;
+

Use aggregate functions (MIN, MAX, SUM, COUNT, AVG)

/* MAX */
+SELECT MAX(pokemon_level)
+FROM pokemon_list;
+
+/* MIN */
+SELECT MIN(pokemon_level)
+FROM pokemon_list;
+
+/* AVG */
+SELECT AVG(pokemon_level)
+FROM pokemon_list;
+
+/* ROUND */
+SELECT ROUND(AVG(pokemon_level))
+FROM pokemon_list;
+
+/* COUNT */
+SELECT COUNT(*)
+FROM pokemon_list;
+
+/* SUM */
+SELECT SUM(pokemon_level)
+FROM pokemon_list;
+

Use boolean aggregate functions

/* Add a column is_legendary of type boolean to table pokemon_list */
+ALTER TABLE pokemon_list ADD COLUMN is_legendary BOOL NOT NULL DEFAULT TRUE;
+
+/*  BOOL_AND
+    returns a result if **ALL** records have that column set to true
+*/
+
+SELECT BOOL_AND(is_legendary) FROM pokemon_list;
+
+
+/*
+    BOOL_OR
+    returns a result if one or more records have that column set to true
+*/
+
+SELECT BOOL_OR(is_legendary) FROM pokemon_list;
+

Update a table and change all column values

UPDATE pokemon_list
+SET is_legendary = FALSE;
+

Update a table and change value based on a criteria

UPDATE pokemon_list
+SET is_legendary = TRUE
+WHERE id = 2;
+

Delete row with specific id

DELETE FROM pokemon_list WHERE id = 4
+

Delete rows withing a range of ids

DELETE FROM pokemon_list
+WHERE id BETWEEN 1 AND 4;
+

Delete all rows

DELETE FROM pokemon_list;
+

Note: The difference between DELETE and DROP or TRUNCATE is that the former can be undone (rolled back) the latter can't

Alter a table to drop a CONSTRAINT if it exist

ALTER TABLE pokemon_types
+DROP CONSTRAINT IF EXISTS unique_type_name;
+

Comments

COMMENT ON TABLE pokemon_types is 'pokemon with types'
+
+/* To display the comment, in psql simply run \dt+. It will return a description
+column containing that comment. It's useful when working on a legacy database
+for example*/
+
+/* Please note that comments aren't exclusive to tables, they can be executed on
+schemas and multiple other objects.*/
+

Note: If you find errors, typos or would like to add new tips, feel free to reach out to me on twitter. I'm @aaqaishtyaq. Thank you for reading ! And if you find this useful, share it with your friends and coworkers !

\ No newline at end of file diff --git a/notes/the-ultimate-setup-for-remote-development/index.html b/notes/the-ultimate-setup-for-remote-development/index.html new file mode 100644 index 0000000..9d273e4 --- /dev/null +++ b/notes/the-ultimate-setup-for-remote-development/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

The ultimate setup for remote development

843 words, 4 minutes to read

I'm a programmer. Like most programmers, I use an Apple laptop. They're just the best on the market and the best purchase you can make as a developer. No fanboyism here, just stating facts. They're fast, well built, and durable.

I rely on this laptop for all my personal / client work. I run vagrant boxes, docker containers in it and all the usual stuff.

As a result, I've developed this irrational fear that it would get stolen or fall off my bedroom window (things like that happen, trust me). Interestingly enough, it coincides with my growing interest for remote work.

So I started to look for ways to create a development server that would allow me to remain productive even if I lose / break my laptop, or when I'm on the road.

Why

Ok, so you've read all this and you're thinking "How is it going to benefit me ?".

The main selling points to creating a remote development environment are the following:

  • It's cheap
  • You learn linux
  • You become OS agnostic (I said no fanboyism)
  • You can work from almost anywhere

What you need

Laptop

Because all of your work is now done on a remote machine you don't really have to care about what computer you're using. It can be a super expensive (notice I didn't say overpriced) Apple laptop or any of its really good windows competitors (the DELL xps 13/15 comes to mind) or even a super cheap, 35$ raspberry Pi. Some people even use chromebooks ! They stick ubuntu on them and use them as their primary machines (SSH is a bit tricky to setup on chrome OS) but hey! Official Linux support is coming for you ChromeOS folks.

Dev Tools

Git

This one's obvious. While git is a life saver and a great tool for collaboration in large teams (distributed or not), you can also use it as a backup system for your code when you're a solo developer.

Vim

I've used sublime text but felt bad for continuously extending the trial because I couldn't pay for it. Then I switched back to VSCode and got frustrated by how slow it runs after using Sublime. Then i tried vim, and I never looked back. It's the lightest, fastest and overall best text editor out there by a huge margin (in my opinion).

Vim is highly customizable and lets you save your settings inside a .vimrc file, which makes it version control friendly. It also makes your vim environment 100% portable as it comes standard in most linux server distros.

Tmux

The best companion to vim. Tmux is a terminal multiplexer. With Tmux you can essentially access multiple terminal sessions inside a single window. This gives you the ability to work on multiple projects at a time. You can even save sessions, attach and reattach to them. It's painless. Just like vim, it's extremely customizable. All of your settings can be stored inside of a .tmux.conf file.

A VPS

To start developing on a remote server, you'll need ... well, a remote server. If you don't know what a VPS is, it stands for Virtual Private Server. It's essentially a virtual machine that you pay for monthly, which runs a server distribution of linux (Ubuntu, CentOS ...), or Unix (FreeBSD). You can connect to it via SSH (it has a public ip address) and start playing around. You have complete control over the server, you can configure it however way you want.

There's a large range of VPS providers on the market, the most notable ones are Linode, Digital Ocean, and Amazon EC2.

Dotfiles

As I previously mentioned above, you can save your settings for vim and tmux in what are called dotfiles (files that start with a period). However dotfiles are not exclusive to vim and tmux, you can for example save your shell configuration in a .zshrc or .bashrc config file, or your git settings in a .gitconfig file.

Doing this is extremely powerful because you can store these configuration files on github and always pull the latest version when you launch a new development server.

You can check out my own dotfiles on github to help you get started.

(Optional) Ansible

Manually setting up servers is fun for the first couple of times, then it just becomes repetitive. And what do you do with repetitive tasks ? You automate them.

Ansible is a provisioning tool written in python that will help you do just that. You give it the ip address (or addresses) of the server you want to configure and it will execute all the tasks you tell it to.

With Ansible you can:

  • run shell commands
  • install packages
  • create directories and files
  • add users and groups
  • clone git repos
  • use templates and pass variables to them
  • and much more

If you don't like Ansible, there are other server provisioning tools like puppet, chef and salt that will work just as good.

Caveats

  • it takes time to learn linux and networking (SSH etc)
  • vim has a learning curve to it
  • you become reliant on an internet connection
\ No newline at end of file diff --git a/processed_images/1.02c4f95e638d8bba.png b/processed_images/1.02c4f95e638d8bba.png new file mode 100644 index 0000000..5716962 Binary files /dev/null and b/processed_images/1.02c4f95e638d8bba.png differ diff --git a/processed_images/1984.13ea14afa9ae362f.jpg b/processed_images/1984.13ea14afa9ae362f.jpg new file mode 100644 index 0000000..161a6d4 Binary files /dev/null and b/processed_images/1984.13ea14afa9ae362f.jpg differ diff --git a/processed_images/2.9d3fa136d4a81628.png b/processed_images/2.9d3fa136d4a81628.png new file mode 100644 index 0000000..e3ecb32 Binary files /dev/null and b/processed_images/2.9d3fa136d4a81628.png differ diff --git a/processed_images/3.a4b549d8212cc4ec.png b/processed_images/3.a4b549d8212cc4ec.png new file mode 100644 index 0000000..1843349 Binary files /dev/null and b/processed_images/3.a4b549d8212cc4ec.png differ diff --git a/processed_images/4.3d09062762f469f3.png b/processed_images/4.3d09062762f469f3.png new file mode 100644 index 0000000..82963dd Binary files /dev/null and b/processed_images/4.3d09062762f469f3.png differ diff --git a/processed_images/5.bf8c45ae45919fef.png b/processed_images/5.bf8c45ae45919fef.png new file mode 100644 index 0000000..b4b4a74 Binary files /dev/null and b/processed_images/5.bf8c45ae45919fef.png differ diff --git a/processed_images/aaqa.8647585b0e7821a3.webp b/processed_images/aaqa.8647585b0e7821a3.webp new file mode 100644 index 0000000..faba331 Binary files /dev/null and b/processed_images/aaqa.8647585b0e7821a3.webp differ diff --git a/processed_images/animal_farm.9d3a4a7459522a23.jpg b/processed_images/animal_farm.9d3a4a7459522a23.jpg new file mode 100644 index 0000000..5110a20 Binary files /dev/null and b/processed_images/animal_farm.9d3a4a7459522a23.jpg differ diff --git a/processed_images/the_idiot.eeb7e4d02729b6de.jpg b/processed_images/the_idiot.eeb7e4d02729b6de.jpg new file mode 100644 index 0000000..668cdc9 Binary files /dev/null and b/processed_images/the_idiot.eeb7e4d02729b6de.jpg differ diff --git a/reading/1984-george-orwell/index.html b/reading/1984-george-orwell/index.html new file mode 100644 index 0000000..6950cfe --- /dev/null +++ b/reading/1984-george-orwell/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

1984 by George Orwell: A Journey into Dystopia

456 words, 2 minutes to read

Image generated using Adobe Firefly AI
Image generated using Adobe Firefly AI

As I sit down to pen my thoughts on George Orwell's seminal work, "1984," I find myself grappling with a myriad of emotions and contemplations. This dystopian masterpiece has left an indelible mark on my psyche, provoking deep introspection and stirring a range of reactions.

From the outset, Orwell thrusts readers into the bleak and oppressive world of Oceania, a totalitarian regime where every aspect of life is tightly controlled by the ominous figure of Big Brother. As I followed the protagonist, Winston Smith, through his harrowing journey of rebellion and defiance against the Party, I couldn't help but feel a sense of unease and resonance with the themes explored.

One of the most striking aspects of "1984" is its chillingly prophetic depiction of surveillance, propaganda, and the manipulation of truth. Orwell's portrayal of a society where individual freedom is subjugated to the will of the Party and where historical facts are distorted to suit its narrative feels hauntingly relevant in today's world of mass surveillance and information warfare.

As I delved deeper into the novel, I found myself captivated by Orwell's exploration of power dynamics and the fragility of truth. The Party's relentless pursuit of control and its perversion of language to stifle dissent served as a stark reminder of the dangers of unchecked authoritarianism and the erosion of fundamental freedoms.

Yet, amidst the darkness and despair, there are moments of profound humanity in "1984." Winston's quest for love and connection, his defiance in the face of oppression, and his yearning for truth and freedom resonated deeply with me. Despite the overwhelming odds stacked against him, Winston's unwavering spirit serves as a beacon of hope in a world consumed by fear and oppression.

Orwell's writing is both evocative and incisive, drawing readers into the bleak and desolate landscape of Oceania with its vivid imagery and haunting prose. His critique of totalitarianism and his exploration of the human condition linger long after the final page has been turned, prompting reflection on the nature of power, truth, and freedom.

In conclusion, "1984" stands as a timeless masterpiece that continues to captivate and provoke readers with its relevance and insight into the complexities of the human condition. It serves as a poignant reminder of the dangers of authoritarianism and the imperative to safeguard individual liberties and uphold the truth. For me, reading "1984" has been a transformative experience, one that has deepened my understanding of the world and reinforced my commitment to defending freedom and truth in all its forms


Should you have any feedback, recommendations, or thoughts of any nature, feel free to reach out to me on Twitter at @aaqaishtyaq!

\ No newline at end of file diff --git a/reading/animal-farm-george-orwell/index.html b/reading/animal-farm-george-orwell/index.html new file mode 100644 index 0000000..dfc92ee --- /dev/null +++ b/reading/animal-farm-george-orwell/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Animal Farm by George Orwell: A Journey into Allegory

429 words, 2 minutes to read

Image generated using Adobe Firefly AI
Image generated using Adobe Firefly AI

After completing 1984, a friend suggested I read this book. As I ponder my recent exploration of George Orwell's "Animal Farm," I am engrossed by the depth of its messages and the profound wisdom nestled in its narrative. Crafted as an allegory, this timeless tale provides a poignant critique of authority, corruption, and the complexities of governance.

From the outset, Orwell draws readers into the world of Manor Farm, where the animals rise against their human oppressors under the rallying cry of "All animals are equal." Led by the pigs, notably the clever and manipulative Napoleon and the idealistic Snowball, the animals establish their society based on the principles of Animalism. At first, their revolution seems to promise a utopian existence, free from the tyranny of human exploitation.

However, as the story unfolds, we witness the gradual erosion of the animals' ideals as the pigs consolidate their power and become indistinguishable from the humans they once opposed. Through cunning manipulation and propaganda, Napoleon and his cohorts subvert the principles of Animalism, betraying the very creatures they claim to represent.

Orwell masterfully uses animal characters to represent different aspects of human nature and political ideologies. The pigs, with their intelligence and ability to organize, symbolize the ruling elite, while the other animals represent various segments of society, from the loyal but naive working class to the cynical skeptics who question authority.

One of the most chilling aspects of "Animal Farm" is its portrayal of how power can corrupt even the noblest of intentions. As Napoleon consolidates his control, he employs fear, manipulation, and violence to maintain his grip on power, ultimately betraying the revolutionary ideals upon which Animal Farm was founded.

Through its allegorical narrative, "Animal Farm" serves as a stark warning about the dangers of totalitarianism and the perversion of revolutionary ideals. Orwell's keen observations about human nature and politics resonate just as strongly today as they did when the novel was first published.

As I close the book, I am left with a profound sense of unease and reflection. "Animal Farm" is not just a story about animals; it is a powerful allegory that holds a mirror to our society, prompting us to question the nature of power, authority, and the pursuit of utopia. It is a timeless masterpiece that reminds us of the importance of vigilance in safeguarding our freedoms and ideals.


If you have any comments, suggestions or remarks of any sort, you can ping me on Twitter @aaqaishtyaq!

\ No newline at end of file diff --git a/reading/index.html b/reading/index.html new file mode 100644 index 0000000..aa35dc9 --- /dev/null +++ b/reading/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Reading

\ No newline at end of file diff --git a/reading/the-idiot-fyodor-dostoevsky/index.html b/reading/the-idiot-fyodor-dostoevsky/index.html new file mode 100644 index 0000000..72154d2 --- /dev/null +++ b/reading/the-idiot-fyodor-dostoevsky/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

The Idiot by Fyodor Dostoevsky: Navigating the Complexities of Humanity

442 words, 2 minutes to read

Image generated using Adobe Firefly AI
Image generated using Adobe Firefly AI

As I take a moment to contemplate my recent voyage through the intricacies of Fyodor Dostoevsky's masterpiece, "The Idiot", I find myself immersed in a whirlwind of emotions and reflections. This novel transcends mere storytelling; it delves deep into the essence of the human experience, brimming with layers of complexity and profound insights that linger long after the final chapter has been read.

At the core of "The Idiot" lies Prince Myshkin, a character who eludes simplistic characterization. He embodies an extraordinary blend of compassion and empathy, yet is burdened by a naïveté and innocence that sharply contrast with the cynicism of the world around him. Myshkin's journey captivated me, drawing me towards his pure-heartedness and unyielding faith in the inherent goodness of humanity.

Dostoevsky intricately intertwines the lives of a diverse cast of characters around Myshkin, each grappling with their own inner demons and ethical quandaries. From the enigmatic Nastasya Filippovna to the tormented Rogozhin, each figure serves as a mirror reflecting the intricate tapestry of human nature, capable of both immense kindness and profound cruelty.

A central theme of the novel revolves around the interplay between intellect and emotion, reason and passion. Myshkin, with his steadfast belief in the transformative power of love and forgiveness, starkly contrasts with the calculating and cynical ethos of St. Petersburg society. Through his interactions with other characters, we are compelled to confront our perceptions of what it truly means to be human.

Moreover, "The Idiot" serves as a profound exploration of identity and authenticity. Myshkin grapples with questions of selfhood and belonging amidst the turbulent waters of social hierarchy and moral ambiguity. His inability to conform to societal norms renders him both a subject of ridicule and a beacon of hope in a world overshadowed by greed and ambition. As I concluded "The Idiot," I found myself deeply moved by Dostoevsky's penetrating insight into the human psyche. This is not a novel that provides easy solutions or neatly tied endings. Instead, it challenges us to confront the complexities of our humanity and to reconcile the inherent contradictions that define us.

Ultimately, "The Idiot" stands as a testament to the enduring potency of compassion and empathy in a world often lacking in both. It serves as a poignant reminder that amidst our imperfections and shortcomings, goodness can still be discovered in the most unexpected of places. For this profound journey, I am immensely grateful.


If you read this article all the way through, thanks for sticking with me! You can send me questions, remarks, or comments on Twitter, I'm @aaqaishtyaq.

\ No newline at end of file diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..4f50037 --- /dev/null +++ b/robots.txt @@ -0,0 +1,4 @@ +User-agent: * +Disallow: +Allow: / +Sitemap: https://aaqa.dev/sitemap.xml diff --git a/rss.xml b/rss.xml new file mode 100644 index 0000000..188e2ff --- /dev/null +++ b/rss.xml @@ -0,0 +1,2057 @@ + + + + Aaqa Ishtyaq + https://aaqa.dev/ + Aaqa Ishtyaq, Ramblings on programming, tech and life. + Zola + en + + Tue, 19 Mar 2024 00:00:00 +0000 + + Move untracked git files to another directory + Tue, 19 Mar 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/tils/move-git-untracked-files-to-temp-dir/ + https://aaqa.dev/tils/move-git-untracked-files-to-temp-dir/ + <p>I had a lot of files in a git repository, which I didn't want to add in <code>git</code>, So to remove all of them +to a temporary directory I used a collection of Unix commands.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> status<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>porcelain</span></span> <span class="z-keyword z-operator z-logical z-pipe z-shell">|</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">awk</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>{print $2}<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span></span> <span class="z-keyword z-operator z-logical z-pipe z-shell">|</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">xargs</span></span><span class="z-meta z-function-call z-arguments z-shell"> mv<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> ../scratch</span> +</span></code></pre> + + + + Animal Farm by George Orwell: A Journey into Allegory + Thu, 07 Mar 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/reading/animal-farm-george-orwell/ + https://aaqa.dev/reading/animal-farm-george-orwell/ + <p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;animal_farm.9d3a4a7459522a23.jpg" alt="Image generated using Adobe Firefly AI"> + <figcaption>Image generated using Adobe Firefly AI</figcaption> +</figure> + </p> +<p>After completing <a href="/reading/1984-george-orwell">1984</a>, a friend suggested I read this book. As I ponder my recent exploration of George Orwell's &quot;Animal Farm,&quot; I am engrossed by the depth of its messages and the profound wisdom nestled in its narrative. Crafted as an allegory, this timeless tale provides a poignant critique of authority, corruption, and the complexities of governance.</p> +<p>From the outset, Orwell draws readers into the world of <em>Manor Farm</em>, where the animals rise against their human oppressors under the rallying cry of <strong>&quot;All animals are equal.&quot;</strong> Led by the pigs, notably the clever and manipulative Napoleon and the idealistic Snowball, the animals establish their society based on the principles of Animalism. At first, their revolution seems to promise a utopian existence, free from the tyranny of human exploitation.</p> +<p>However, as the story unfolds, we witness the gradual erosion of the animals' ideals as the pigs consolidate their power and become indistinguishable from the humans they once opposed. Through cunning manipulation and propaganda, Napoleon and his cohorts subvert the principles of Animalism, betraying the very creatures they claim to represent.</p> +<p>Orwell masterfully uses animal characters to represent different aspects of human nature and political ideologies. The pigs, with their intelligence and ability to organize, symbolize the ruling elite, while the other animals represent various segments of society, from the loyal but naive working class to the cynical skeptics who question authority.</p> +<p>One of the most chilling aspects of &quot;Animal Farm&quot; is its portrayal of how power can corrupt even the noblest of intentions. As Napoleon consolidates his control, he employs fear, manipulation, and violence to maintain his grip on power, ultimately betraying the revolutionary ideals upon which Animal Farm was founded.</p> +<p>Through its allegorical narrative, &quot;Animal Farm&quot; serves as a stark warning about the dangers of totalitarianism and the perversion of revolutionary ideals. Orwell's keen observations about human nature and politics resonate just as strongly today as they did when the novel was first published.</p> +<p>As I close the book, I am left with a profound sense of unease and reflection. &quot;Animal Farm&quot; is not just a story about animals; it is a powerful allegory that holds a mirror to our society, prompting us to question the nature of power, authority, and the pursuit of utopia. It is a timeless masterpiece that reminds us of the importance of vigilance in safeguarding our freedoms and ideals.</p> +<hr /> +<p><em>If you have any comments, suggestions or remarks of any sort, you can ping me on Twitter <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.twitter.com/aaqaishtyaq">@aaqaishtyaq</a>!</em></p> + + + + Pulga, Kalga - Himachal Pradesh 2024 + Tue, 05 Mar 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/travel/fairy-forest-pulga-kalga/ + https://aaqa.dev/travel/fairy-forest-pulga-kalga/ + <p>In the heart of Himachal's Parvati Valley lie its hidden gems: Pulga and Kalga. Picture-perfect villages embraced by towering mountains, where every corner tells a story of nature's magnificence. Here, snow-capped peaks stand tall, kissing the sky, while icy waterfalls dance down the slopes, painting the landscape with their pure, untamed beauty. Despite the biting cold, every moment feels like a warm embrace from Mother Nature herself. It's not just a trip; it's a journey of awe, wonder, and connection with the raw essence of the mountains.</p> +<p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;1.02c4f95e638d8bba.png" alt=""> + <figcaption></figcaption> +</figure> + </p> +<p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;2.9d3fa136d4a81628.png" alt=""> + <figcaption></figcaption> +</figure> + </p> +<p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;3.a4b549d8212cc4ec.png" alt=""> + <figcaption></figcaption> +</figure> + </p> +<p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;4.3d09062762f469f3.png" alt=""> + <figcaption></figcaption> +</figure> + </p> +<p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;5.bf8c45ae45919fef.png" alt=""> + <figcaption></figcaption> +</figure> + </p> + + + + 1984 by George Orwell: A Journey into Dystopia + Sat, 03 Feb 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/reading/1984-george-orwell/ + https://aaqa.dev/reading/1984-george-orwell/ + <p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;1984.13ea14afa9ae362f.jpg" alt="Image generated using Adobe Firefly AI"> + <figcaption>Image generated using Adobe Firefly AI</figcaption> +</figure> + </p> +<p>As I sit down to pen my thoughts on George Orwell's seminal work, &quot;1984,&quot; I find myself grappling with a myriad of emotions and contemplations. This dystopian masterpiece has left an indelible mark on my psyche, provoking deep introspection and stirring a range of reactions.</p> +<p>From the outset, Orwell thrusts readers into the bleak and oppressive world of Oceania, a totalitarian regime where every aspect of life is tightly controlled by the ominous figure of Big Brother. As I followed the protagonist, Winston Smith, through his harrowing journey of rebellion and defiance against the Party, I couldn't help but feel a sense of unease and resonance with the themes explored.</p> +<p>One of the most striking aspects of &quot;1984&quot; is its chillingly prophetic depiction of surveillance, propaganda, and the manipulation of truth. Orwell's portrayal of a society where individual freedom is subjugated to the will of the Party and where historical facts are distorted to suit its narrative feels hauntingly relevant in today's world of mass surveillance and information warfare.</p> +<p>As I delved deeper into the novel, I found myself captivated by Orwell's exploration of power dynamics and the fragility of truth. The Party's relentless pursuit of control and its perversion of language to stifle dissent served as a stark reminder of the dangers of unchecked authoritarianism and the erosion of fundamental freedoms.</p> +<p>Yet, amidst the darkness and despair, there are moments of profound humanity in &quot;1984.&quot; Winston's quest for love and connection, his defiance in the face of oppression, and his yearning for truth and freedom resonated deeply with me. Despite the overwhelming odds stacked against him, Winston's unwavering spirit serves as a beacon of hope in a world consumed by fear and oppression.</p> +<p>Orwell's writing is both evocative and incisive, drawing readers into the bleak and desolate landscape of Oceania with its vivid imagery and haunting prose. His critique of totalitarianism and his exploration of the human condition linger long after the final page has been turned, prompting reflection on the nature of power, truth, and freedom.</p> +<p>In conclusion, &quot;1984&quot; stands as a timeless masterpiece that continues to captivate and provoke readers with its relevance and insight into the complexities of the human condition. It serves as a poignant reminder of the dangers of authoritarianism and the imperative to safeguard individual liberties and uphold the truth. For me, reading &quot;1984&quot; has been a transformative experience, one that has deepened my understanding of the world and reinforced my commitment to defending freedom and truth in all its forms</p> +<hr /> +<p><em>Should you have any feedback, recommendations, or thoughts of any nature, feel free to reach out to me on Twitter at <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.twitter.com/aaqaishtyaq">@aaqaishtyaq</a>!</em></p> + + + + The Idiot by Fyodor Dostoevsky: Navigating the Complexities of Humanity + Mon, 01 Jan 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/reading/the-idiot-fyodor-dostoevsky/ + https://aaqa.dev/reading/the-idiot-fyodor-dostoevsky/ + <p> +<figure> + <img src="https:&#x2F;&#x2F;aaqa.dev&#x2F;processed_images&#x2F;the_idiot.eeb7e4d02729b6de.jpg" alt="Image generated using Adobe Firefly AI"> + <figcaption>Image generated using Adobe Firefly AI</figcaption> +</figure> + </p> +<p>As I take a moment to contemplate my recent voyage through the intricacies of Fyodor Dostoevsky's masterpiece, &quot;The Idiot&quot;, I find myself immersed in a whirlwind of emotions and reflections. This novel transcends mere storytelling; it delves deep into the essence of the human experience, brimming with layers of complexity and profound insights that linger long after the final chapter has been read.</p> +<p>At the core of &quot;The Idiot&quot; lies Prince Myshkin, a character who eludes simplistic characterization. He embodies an extraordinary blend of compassion and empathy, yet is burdened by a naïveté and innocence that sharply contrast with the cynicism of the world around him. Myshkin's journey captivated me, drawing me towards his pure-heartedness and unyielding faith in the inherent goodness of humanity.</p> +<p>Dostoevsky intricately intertwines the lives of a diverse cast of characters around Myshkin, each grappling with their own inner demons and ethical quandaries. From the enigmatic Nastasya Filippovna to the tormented Rogozhin, each figure serves as a mirror reflecting the intricate tapestry of human nature, capable of both immense kindness and profound cruelty.</p> +<p>A central theme of the novel revolves around the interplay between intellect and emotion, reason and passion. Myshkin, with his steadfast belief in the transformative power of love and forgiveness, starkly contrasts with the calculating and cynical ethos of <em>St. Petersburg</em> society. Through his interactions with other characters, we are compelled to confront our perceptions of what it truly means to be human.</p> +<p>Moreover, &quot;The Idiot&quot; serves as a profound exploration of identity and authenticity. Myshkin grapples with questions of selfhood and belonging amidst the turbulent waters of social hierarchy and moral ambiguity. His inability to conform to societal norms renders him both a subject of ridicule and a beacon of hope in a world overshadowed by greed and ambition. +As I concluded &quot;The Idiot,&quot; I found myself deeply moved by Dostoevsky's penetrating insight into the human psyche. This is not a novel that provides easy solutions or neatly tied endings. Instead, it challenges us to confront the complexities of our humanity and to reconcile the inherent contradictions that define us.</p> +<p>Ultimately, &quot;The Idiot&quot; stands as a testament to the enduring potency of compassion and empathy in a world often lacking in both. It serves as a poignant reminder that amidst our imperfections and shortcomings, goodness can still be discovered in the most unexpected of places. For this profound journey, I am immensely grateful.</p> +<hr /> +<p><em>If you read this article all the way through, thanks for sticking with me! You</em> +<em>can send me questions, remarks, or comments on Twitter, I'm <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.twitter.com/aaqaishtyaq">@aaqaishtyaq</a>.</em></p> + + + + Firecracker Dev Machine Setup (GCP) + Thu, 26 Jan 2023 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/firecracker-dev-machine-setup/ + https://aaqa.dev/notes/firecracker-dev-machine-setup/ + <p>VM on Google Compute Engine (GCE) supports nested virtualisation, unlike AWS, and allows to run KVM. +We can create a VM on GCE to run and test Firecracker microVM.</p> +<h3 id="here-are-the-steps">Here are the steps</h3> +<ol> +<li> +<p>Setup GCP Project and Zone</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_PROJECT</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">your_name-firecracker</span> +</span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_REGION</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">us-east1</span> +</span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_ZONE</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">us-east1-b</span> +</span></code></pre> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> config set project <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_PROJECT</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> config set compute/region <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_REGION</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> config set compute/zone <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_ZONE</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span></span> +</span></code></pre> +</li> +<li> +<p>Create a VM image (Machine Image) able to run KVM</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_VDISK</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">disk-ub22</span> +</span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_IMAGE</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">ub22-nested-kvm</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> compute disks create <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_VDISK</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>image-project</span> ubuntu-os-cloud <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>image-family</span> ubuntu-2204-lts</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> compute images create <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_IMAGE</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>source-disk</span> <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_VDISK</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>source-disk-zone</span> <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_ZONE</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>licenses</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span></code></pre> +</li> +<li> +<p>Create the VM</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">FC_VM</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">firecracker-vm</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> compute instances create <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_VM</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>zone</span> <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_ZONE</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span> <span class="z-punctuation z-separator z-continuation z-line z-shell">\ +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>image</span> <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_IMAGE</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span></span> +</span></code></pre> +</li> +<li> +<p>Connect to the VM via SSH.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">gcloud</span></span><span class="z-meta z-function-call z-arguments z-shell"> compute ssh <span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">FC_VM</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span></span> +</span></code></pre> +</li> +<li> +<p>Verify that VMX is enabled, enable KVM</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">$</span></span><span class="z-meta z-function-call z-arguments z-shell"> grep<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>cw</span> vmx /proc/cpuinfo</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">1</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">$</span></span><span class="z-meta z-function-call z-arguments z-shell"> sudo setfacl<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>m</span> u:<span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-punctuation z-section z-expansion z-parameter z-begin z-shell">{</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-variable z-other z-readwrite z-shell">USER</span></span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-section z-expansion z-parameter z-end z-shell">}</span></span>:rw /dev/kvm</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">$</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-keyword z-control z-regexp z-set z-begin z-shell">[</span> <span class="z-keyword z-operator z-word z-shell">-</span>r /dev/kvm <span class="z-keyword z-control z-regexp z-set z-end z-shell">]</span></span> <span class="z-keyword z-operator z-logical z-and z-shell">&amp;&amp;</span> <span class="z-support z-function z-test z-begin z-shell">[</span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>w</span> /dev/kvm <span class="z-support z-function z-test z-end z-shell">]</span></span> <span class="z-keyword z-operator z-logical z-and z-shell">&amp;&amp;</span> <span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>OK<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> <span class="z-keyword z-operator z-logical z-or z-shell">||</span> <span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>FAIL<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">OK</span></span> +</span></code></pre> +</li> +</ol> + + + + Linux Container Networking from Scratch + Thu, 26 Jan 2023 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/linux-container-networking/ + https://aaqa.dev/notes/linux-container-networking/ + <p>In this article, we will be looking into setting up networking on a linux box from scratch.</p> +<p>We will be creating a fresh new VM using Lima on macOS. +You can create a new VM using VirtualBox, Vagrant, or even create a VM on OCI for free.</p> +<h2 id="isolation-based-on-network-namespace">Isolation based on Network Namespace</h2> +<p>The common thing that constitutes a network stack includes the:</p> +<ul> +<li>Network Devices</li> +<li>Routing Rules</li> +<li>Netfilter Hooks</li> +</ul> +<p>We can create a non-comprehensive <code>inspect-net-stack</code> script.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell">!/usr/bin/env bash</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>&gt; Network devices<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">ip</span></span><span class="z-meta z-function-call z-arguments z-shell"> link</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>e</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>\n&gt; Route table<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">ip</span></span><span class="z-meta z-function-call z-arguments z-shell"> route</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>e</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>\n&gt; Iptables rules<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">iptables</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>list-rules</span></span> +</span></code></pre> +<p>After the execution of the inspect script on my machine produces the following input:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell"><span class="z-meta z-group z-expansion z-job z-shell"><span class="z-punctuation z-definition z-variable z-job z-shell">%</span></span></span></span><span class="z-meta z-function-call z-arguments z-shell"> sudo ./inspect-net-stack</span> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Network <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">devices</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">1:</span></span><span class="z-meta z-function-call z-arguments z-shell"> lo: <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>LOOPBACK,UP,LOWER_UP<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">link/loopback</span></span><span class="z-meta z-function-call z-arguments z-shell"> 00:00:00:00:00:00 brd 00:00:00:00:00:00</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">2:</span></span><span class="z-meta z-function-call z-arguments z-shell"> eth0: <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>BROADCAST,MULTICAST,UP,LOWER_UP<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">link/ether</span></span><span class="z-meta z-function-call z-arguments z-shell"> 52:55:55:a3:ad:92 brd ff:ff:ff:ff:ff:ff</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">altname</span></span><span class="z-meta z-function-call z-arguments z-shell"> enp0s1</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Route <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">table</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">default</span></span><span class="z-meta z-function-call z-arguments z-shell"> via 192.168.5.3 dev eth0 proto dhcp src 192.168.5.15 metric 100</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">192.168.5.0/24</span></span><span class="z-meta z-function-call z-arguments z-shell"> dev eth0 proto kernel scope link src 192.168.5.15 metric 100</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Iptables <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">rules</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> INPUT ACCEPT</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> FORWARD ACCEPT</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> OUTPUT ACCEPT</span> +</span></code></pre> +<p>We want to make sure that each of the containers we are going to create soon will get a separate network stack.</p> +<p>Linux namespaces used for container isolation are called <a rel="noopener nofollow noreferrer" target="_blank" href="https://man7.org/linux/man-pages/man8/ip-netns.8.html">network namespaces</a>. We won't be creating the fully isolated container, rather restrict the scope to only the network stack. One of the ways to create a network stack is by using the <code>ip</code> tool, part of <a rel="noopener nofollow noreferrer" target="_blank" href="https://en.wikipedia.org/wiki/Iproute2"><code>iproute2</code></a></p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell"><span class="z-meta z-group z-expansion z-job z-shell"><span class="z-punctuation z-definition z-variable z-job z-shell">%</span></span></span></span><span class="z-meta z-function-call z-arguments z-shell"> sudo ip netns add netns0</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell"><span class="z-meta z-group z-expansion z-job z-shell"><span class="z-punctuation z-definition z-variable z-job z-shell">%</span></span></span></span><span class="z-meta z-function-call z-arguments z-shell"> ip netns</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">netns0</span></span> +</span></code></pre> +<p>We have a new network namespace but to start using the network namespace, We can use a command called <code>nsenter</code>. It enters one or more of the specified namespaces and then executes the given program:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> This is create a new bash process under netns0 namespace</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell"><span class="z-meta z-group z-expansion z-job z-shell"><span class="z-punctuation z-definition z-variable z-job z-shell">%</span></span></span></span><span class="z-meta z-function-call z-arguments z-shell"> sudo nsenter<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>net</span><span class="z-keyword z-operator z-assignment z-option z-shell">=</span>/var/run/netns/netns0 bash</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell"><span class="z-meta z-group z-expansion z-job z-shell"><span class="z-punctuation z-definition z-variable z-job z-shell">%</span></span></span></span><span class="z-meta z-function-call z-arguments z-shell"> sudo ./inspect-net-stack</span> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Network <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">devices</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">1:</span></span><span class="z-meta z-function-call z-arguments z-shell"> lo: <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>LOOPBACK<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">link/loopback</span></span><span class="z-meta z-function-call z-arguments z-shell"> 00:00:00:00:00:00 brd 00:00:00:00:00:00</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Route <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">table</span></span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Iptables <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">rules</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> INPUT ACCEPT</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> FORWARD ACCEPT</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">-P</span></span><span class="z-meta z-function-call z-arguments z-shell"> OUTPUT ACCEPT</span> +</span></code></pre> +<p>As you can see from the above output, the <code>bash</code> process running inside the <code>netns0</code> namespace sees a different network stack. There are no routing rules and no custom iptables chain. Only one loopback device (<code>lo</code>).</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">Network</span></span><span class="z-meta z-function-call z-arguments z-shell"> Namespace</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">┌─────────────────────────────────────────────────────────────────────┐</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> Root Namespace │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> lo0 ┌────────────────────────────────────────┐ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> ┌─┐ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ lo0 Container Namespace │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ ┌─┐ netns0 │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ ┌────────────┐ │ │ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │ │ │ ┌───────────┐ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ routes │ │ │ │ │ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │ │ │ │routes │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ └────────────┘ │ │ │ └───────────┘ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> └─┘ │ └─┘ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> ┌─┐ ┌────────────┐ │ ┌───────────┐ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │ │ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ iptables │ │ │iptables │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │ └───────────┘ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ └────────────┘ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │ │ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> └─┘ │ │ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> eth0 └────────────────────────────────────────┘ │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">│</span></span><span class="z-meta z-function-call z-arguments z-shell"> │</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">└─────────────────────────────────────────────────────────────────────┘</span></span> +</span></code></pre> +<p><em>Beware</em>: The <code>nsenter</code> command from above started a nested bash session in the netns0 network namespace. Don't forget to exit from it.</p> +<hr /> +<p>In the upcoming posts, I will discuss connecting the host with this namespace. Interconnecting various network switches, Just like patching the physical switch with a LAN cable.</p> + + + + Git exclude files from working copy + Fri, 29 Jul 2022 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/git-exclude-files/ + https://aaqa.dev/notes/git-exclude-files/ + <h2 id="how-to-ignore-new-files">How to ignore new files</h2> +<h3 id="globally">Globally</h3> +<p>Add the path(s) to your file(s) which you would like to ignore to your <code>.gitignore</code> file (and commit them). These file entries will also apply to others checking out the repository.</p> +<h3 id="locally">Locally</h3> +<p>Add the path(s) to your file(s) which you would like to ignore to your <code>.git/info/exclude</code> file. These file entries will only apply to your local working copy.</p> +<h2 id="how-to-ignore-changed-files-temporarily">How to ignore changed files (temporarily)</h2> +<p>In order to ignore changed files to being listed as modified, you can use the following git command:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> update-index<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>assume-unchanged</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-1<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-2<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-3<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> +</span></span></code></pre> +<p>To revert that ignorance use the following command:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> update-index<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>no-assume-unchanged</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-1<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-2<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>file-3<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> +</span></span></code></pre> +<p><em>Note: If you find errors, typos or would like to add new tips, feel free to +reach out to me on twitter. I'm <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a>. Thank +you for reading ! And if you find this useful, share it with your friends and +coworkers !</em></p> + + + + Open markdown links in a new tab + Fri, 05 Jun 2020 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/hugo-open-markdown-new-tab/ + https://aaqa.dev/notes/hugo-open-markdown-new-tab/ + <p>I created a static website <a rel="noopener nofollow noreferrer" target="_blank" href="https://linkbin.aaqa.dev/">linkbin</a>, Where I could save links in a markdown file that can be viewed later. +It is serving its purpose and because it uses <a rel="noopener nofollow noreferrer" target="_blank" href="https://gohugo.io/">hugo</a>, I don't have to think about VPS cost.</p> +<p>Recently, I ran into an issue where my website serves links but when you click on it, It will open the link in the same tab. Ugggh! I don't want to close my website just to open a link. Ideally, it should open the link in a new tab.</p> +<p>How can we do it? It would be easy Eh?</p> +<h2 id="the-generic-way">The generic way</h2> +<p>Since markdown is eventually rendered as HTML, The most naive way I found was to write HTML in markdown</p> +<pre data-lang="html" class="language-html z-code"><code class="language-html" data-lang="html"><span class="z-text z-html z-basic"><span class="z-meta z-tag z-inline z-a z-html"><span class="z-punctuation z-definition z-tag z-begin z-html">&lt;</span><span class="z-entity z-name z-tag z-inline z-a z-html">a</span> <span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">href</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>http://example.com/<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span> <span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">target</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>_blank<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span><span class="z-punctuation z-definition z-tag z-end z-html">&gt;</span></span>example<span class="z-meta z-tag z-inline z-a z-html"><span class="z-punctuation z-definition z-tag z-begin z-html">&lt;/</span><span class="z-entity z-name z-tag z-inline z-a z-html">a</span><span class="z-punctuation z-definition z-tag z-end z-html">&gt;</span></span> +</span></code></pre> +<p>No way, I am going to rewrite all my links into HTML. It adds cost later on as well. I will have to add HTML tags to write the link instead of this syntax:</p> +<pre data-lang="markdown" class="language-markdown z-code"><code class="language-markdown" data-lang="markdown"><span class="z-text z-html z-markdown"><span class="z-meta z-paragraph z-markdown"><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-begin z-markdown">[</span></span><span class="z-meta z-link z-inline z-description z-markdown">link</span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-end z-markdown">]</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-metadata z-begin z-markdown">(</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-markup z-underline z-link z-markdown">example.com</span><span class="z-punctuation z-definition z-metadata z-end z-markdown">)</span></span> +</span></span></code></pre> +<h2 id="the-specialized-way">The specialized way</h2> +<p>This could vary depending upon the markdown parser you are using. +If you're using Jekyll, like this website, then you can use the following syntax to open links in a new tab.</p> +<pre data-lang="markdown" class="language-markdown z-code"><code class="language-markdown" data-lang="markdown"><span class="z-text z-html z-markdown"><span class="z-meta z-paragraph z-markdown"><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-begin z-markdown">[</span></span><span class="z-meta z-link z-inline z-description z-markdown">link</span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-end z-markdown">]</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-metadata z-begin z-markdown">(</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-markup z-underline z-link z-markdown">url</span><span class="z-punctuation z-definition z-metadata z-end z-markdown">)</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-attributes z-begin z-markdown">{</span><span class="z-meta z-attribute-with-value z-markdown"><span class="z-entity z-other z-attribute-name z-markdown">:target</span><span class="z-punctuation z-separator z-key-value z-markdown">=</span><span class="z-string z-quoted z-double z-markdown"><span class="z-punctuation z-definition z-string z-begin z-markdown">&quot;</span>_blank<span class="z-punctuation z-definition z-string z-end z-markdown">&quot;</span></span></span><span class="z-punctuation z-definition z-attributes z-end z-markdown">}</span></span> +</span></span></code></pre> +<p>But, I am not using Jekyll for <a rel="noopener nofollow noreferrer" target="_blank" href="https://linkbin.aaqa.dev">linkbin</a>. It uses a fairly newer version of <a href="httpsP://gohugo.io">Hugo</a>. Hugo ,<code>v0.62.0 or later</code>, uses <a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/yuin/goldmark">Goldmark</a> parser and supports <a rel="noopener nofollow noreferrer" target="_blank" href="https://gohugo.io/templates/render-hooks/">Markdown render hooks</a>. Markdown Render Hooks offer you several ways to extend the default markdown behavior, e.g. resizing uploaded images, opening links in new tabs, or creating mermaid diagrams from code. You can do this by creating templates in the <code>layouts/_default/_markup</code> directory with base names <code>render-link</code> or <code>render-image</code> or <code>render-codeblock</code>. Your directory layout may look like this:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">layouts</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">└──</span></span><span class="z-meta z-function-call z-arguments z-shell"> _default</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">└──</span></span><span class="z-meta z-function-call z-arguments z-shell"> _markup</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">├──</span></span><span class="z-meta z-function-call z-arguments z-shell"> render-image.html</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">├──</span></span><span class="z-meta z-function-call z-arguments z-shell"> render-image.rss.xml</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">├──</span></span><span class="z-meta z-function-call z-arguments z-shell"> render-link.html</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">├──</span></span><span class="z-meta z-function-call z-arguments z-shell"> render-codeblock.html</span> +</span><span class="z-source z-shell z-bash"> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">└──</span></span><span class="z-meta z-function-call z-arguments z-shell"> render-codeblock-bash.html</span> +</span></code></pre> +<h3 id="render-hook-to-open-link-in-a-new-tab">Render hook to open link in a new tab</h3> +<p>Add the following HTML template file (or render hook) at <code>layouts/_default/_markup/render-link.html</code>:</p> +<pre data-lang="html" class="language-html z-code"><code class="language-html" data-lang="html"><span class="z-text z-html z-basic"> +</span><span class="z-text z-html z-basic"><span class="z-meta z-tag z-inline z-a z-html"><span class="z-punctuation z-definition z-tag z-begin z-html">&lt;</span><span class="z-entity z-name z-tag z-inline z-a z-html">a</span> <span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">href</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>{{ .Destination | safeURL }}<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">{{</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">with</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">.Title}}</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">title</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>{{ . }}<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">{{</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">end</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">}}{{</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">if</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">strings.HasPrefix</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">.Destination</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html"><span class="z-invalid z-illegal z-attribute-name z-html">&quot;</span>http<span class="z-invalid z-illegal z-attribute-name z-html">&quot;</span></span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">}}</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">target</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>_blank<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span> <span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">rel</span><span class="z-punctuation z-separator z-key-value z-html">=</span><span class="z-string z-quoted z-double z-html"><span class="z-punctuation z-definition z-string z-begin z-html">&quot;</span>noopener<span class="z-punctuation z-definition z-string z-end z-html">&quot;</span></span></span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">{{</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">end</span> </span><span class="z-meta z-attribute-with-value z-html"><span class="z-entity z-other z-attribute-name z-html">}}</span></span><span class="z-punctuation z-definition z-tag z-end z-html">&gt;</span></span>{{ .Text | safeHTML }}<span class="z-meta z-tag z-inline z-a z-html"><span class="z-punctuation z-definition z-tag z-begin z-html">&lt;/</span><span class="z-entity z-name z-tag z-inline z-a z-html">a</span><span class="z-punctuation z-definition z-tag z-end z-html">&gt;</span></span> +</span></code></pre> +<p>You'll find that the links now open in a new tab! +For internal blog links (which you would want to open in the same tab), you can use the relative link of the post, e.g. for an <code>other-post.md</code> file within the <code>posts</code> directory, you could use</p> +<pre data-lang="markdown" class="language-markdown z-code"><code class="language-markdown" data-lang="markdown"><span class="z-text z-html z-markdown"><span class="z-meta z-paragraph z-markdown"><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-begin z-markdown">[</span></span><span class="z-meta z-link z-inline z-description z-markdown">Other post</span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-link z-end z-markdown">]</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-punctuation z-definition z-metadata z-begin z-markdown">(</span></span><span class="z-meta z-link z-inline z-markdown"><span class="z-markup z-underline z-link z-markdown">/posts/other-post/</span><span class="z-punctuation z-definition z-metadata z-end z-markdown">)</span></span> +</span></span></code></pre> +<hr /> +<p>If I’ve missed something or made a horrible mistake if you have any questions regarding this article then feel free to ping me on Twitter. I’m +<a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a>.</p> + + + + The Recipe of a great RESTful API + Sun, 30 Dec 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/recipe-of-great-rest-api/ + https://aaqa.dev/notes/recipe-of-great-rest-api/ + <p>I personally love writing APIs, it's my favourite thing about backend +development. As developers we interact with APIs all the time, to +process payments with stripe, for geolocation with google maps, to retrieve +tweets etc ...</p> +<p>So what exactly makes a RESTful API great ? What can you do to +ensure that it's easy to use for other developers ? Let's dive right in.</p> +<h2 id="responses">Responses</h2> +<p>In your responses, you should separate the metadata from the the body and put them in their own objects.</p> +<pre data-lang="javascript" class="language-javascript z-code"><code class="language-javascript" data-lang="javascript"><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Example: GET /api/user/2</span> +</span><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>meta<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>type<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>success<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>code<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-constant z-numeric z-decimal z-ts">200</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>request_id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>someid<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-punctuation z-whitespace z-comment z-leading z-ts"> </span><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> etc ...</span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>body<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-constant z-numeric z-decimal z-ts">2</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>name<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Aaqa<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>location<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Delhi<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>job<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Software Developer<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></code></pre> +<p>When creating a new resource, always return a <code>201 created</code> instead of a <code>200 OK</code>. It's a common mistake developers make. +Another best practice is to return a url to the newly created resource in the location header of your response. +You can also choose to send a serialised version of the resource in the response body. (I personally like to do that).</p> +<pre data-lang="javascript" class="language-javascript z-code"><code class="language-javascript" data-lang="javascript"><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Example: POST /api/users</span> +</span><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Response Headers</span> +</span><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Content-Type: application/json</span> +</span><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> ...</span> +</span><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Location: https://yoursite.com/api/users/2</span> +</span><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>meta<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>type<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>success<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>code<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-constant z-numeric z-decimal z-ts">201</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>request_id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>some_id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>body<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>message<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>User successfully added<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>user<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-meta z-objectliteral z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-constant z-numeric z-decimal z-ts">3</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>name<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Harry_Potter<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>location<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>London<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>job<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Auror<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>href<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>https://yoursite.com/api/users/3<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></code></pre> +<h3 id="errors">Errors</h3> +<p>Properly dealing with errors is crucial to provide the best experience possible +to your users. Just like with normal responses, always include the status code in the meta part.</p> +<p>You should also provide a nice descriptive message detailing the cause of the +error as well as a potential solution to fix it.</p> +<p>Doing this will help the consumers of your API to gracefully handle these errors on the client side. They will thank you for it.</p> +<pre data-lang="javascript" class="language-javascript z-code"><code class="language-javascript" data-lang="javascript"><span class="z-source z-ts"><span class="z-comment z-line z-double-slash z-ts"><span class="z-punctuation z-definition z-comment z-ts">//</span></span><span class="z-comment z-line z-double-slash z-ts"> Example GET /api/secretendpoint</span> +</span><span class="z-source z-ts"> +</span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>meta<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>type<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>error<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>status<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-constant z-numeric z-decimal z-ts">401</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>request_id<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>weoifwe<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>body<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span>:<span class="z-meta z-objectliteral z-ts"> <span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>message<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span></span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>Unauthorized. Please login or create an account<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></code></pre> +<h2 id="status-codes">Status codes</h2> +<p>This part is subject to many passionate debates, apparently it's a very touchy +topic.</p> +<p>Below I will list the most common status codes, what they mean and how I use +them.</p> +<ul> +<li>200 - OK - Request went well, errthang's fine.</li> +<li>201 - Resource created - The resource was successfully saved to the database.</li> +<li>204 - No Content - There used to be a resource at this endpoint but not +anymore (Used for <code>DELETE</code> requests)</li> +<li>400 - Bad request - The request was badly formatted. (ex: Invalid JSON)</li> +<li>401 - Unauthorized - Authentication failed due to invalid credentials.</li> +<li>403 - Forbidden - Authentication passed but the user does not have permission to access the resource</li> +<li>404 - Not Found - Well, <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.youtube.com/watch?v=yX2vWL2b2Rc">you know what it is</a></li> +<li>405 - Method Not Allowed - The HTTP verb used to interact with the resource is not allowed. (eg: User tried to POST to a GET only endpoint)</li> +<li>409 - Conflict - The data passed in the payload conflicts with the resource in +the server. (I return this status code in cases where a user creates/updates a +resource with a unique field constraint attached to it such as a username or +email)</li> +<li>422 - Unprocessable Entity - Again, another controversial one. I return a 422 +when the user sends an incomplete payload thereby violating the NOT NULL constraint attached to those fields.</li> +<li>429 - Too many requests - You made too many requests.</li> +<li>50x - Server Related Errors - Something went wrong server side.</li> +</ul> +<h3 id="caching">Caching</h3> +<p>Caching is great and is a must when deploying an API. It helps a huge +amount with performace by reducing server load as you are not required to +perform the same action multiple times. You could choose to do it manually or use a reverse proxy like nginx or even something like varnish to handle that for you.</p> +<p>Caching will return a <code>304 Not Modified</code> if the resource hasn't changed. The great thing about this is that it does not affect your rate limiting, so there's +literally no excuse not to implement it.</p> +<h3 id="rate-limiting">Rate Limiting</h3> +<p>For security and performance reasons you want limit the amount of requests made +against your API. +You must return the maximum and remaining number of request allowed for a given user in a particular time window (which you are free to determine).</p> +<h3 id="cors">CORS</h3> +<p>You want to enable CORS to allow requests from browsers using AJAX. This can +be done by setting the following headers:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">Access-Control-Allow-Origin:</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-keyword z-operator z-regexp z-quantifier z-shell">*</span> <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> this means allow all domains to interact with the API</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">Access-Control-Allow-Headers: <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> All the headers you want to accept should be listed here</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></span></code></pre> +<h3 id="a-few-tips">A few tips</h3> +<ul> +<li>Return urls to resources and total count when GETting collections</li> +<li>Map your endpoints to collections and resources</li> +<li>Always use plural nouns (Ex: /api/users not /api/user)</li> +<li>Actions on resources must be described with HTTP verbs</li> +<li>for partial updates of resources use PATCH instead of PUT</li> +<li>for versioning, use dates instead of numbers in the URL (foursquare does exactly that)</li> +</ul> +<hr /> +<p>If you have any comment, suggestions or remarks of any sort, you can ping +me on twitter <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.twitter.com/aaqaishtyaq">@aaqaishtyaq</a> !</p> + + + + Setting up Scrapy Splash Plugin + Sat, 01 Sep 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/scrapy-splash-setup/ + https://aaqa.dev/notes/scrapy-splash-setup/ + <p>Scrapy is good for scraping static web pages using python but when it comes to dynamic web pages scrapy can't do wonders, and there comes <code>Selenium</code> but as good as selenium is, it just got beaten by Scrapy in terms or speed.</p> +<p>Web nowdays is all about Dynamic JS based pages and AJAX. So for this very scenario the guys over <a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/scrapy-plugins">scrapy-plugins</a> created <code>scrapy-splash</code>. +Scrapy-Splash is a plugin that connects Scrapy with Splash (Lightweight, scriptable browser as a service with an HTTP API). +In a nutshell what splash do is it traps the response recieved from the server and renders it. Then it return a <code>render.html</code> which is static and can be easily scraped.</p> +<h2 id="0-setting-up-the-machine">0 - Setting up the machine</h2> +<p>A. Before we begin you need to install <code>Docker</code> first, You can follow the <a rel="noopener nofollow noreferrer" target="_blank" href="https://docs.docker.com/install/">official instruction</a> as per your Operating System.</p> +<p>B. After installing docker navigate to your project folder, activate <code>virtualenv</code> and install scrapy-splsh plugin</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">pip3</span></span><span class="z-meta z-function-call z-arguments z-shell"> install scrapy-splash</span> +</span></code></pre> +<p>C. Pull the Splash Docker Image and run it</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> pull scrapinghub/splash</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">docker</span></span><span class="z-meta z-function-call z-arguments z-shell"> run<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> 8050:8050 scrapinghub/splash</span> +</span></code></pre> +<h2 id="1-configuration">1 - Configuration</h2> +<p>A. Add the Splash server address to <code>settings.py</code> of your Scrapy project like this:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">SPLASH_URL</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">http://localhost:8050<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span> +</span></code></pre> +<p>If you are running docker on your local machine then you can simply use <code>http://localhost:&lt;port&gt;</code> , but if you are running it on a remote machine you need to specify it's I.P. Address like this <code>http://192.168.59.103:&lt;port&gt;</code></p> +<p>B. Enable the Splash middleware by adding it to <code>DOWNLOADER_MIDDLEWARES</code> in your <code>settings.py</code> file and changing HttpCompressionMiddleware priority:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">DOWNLOADER_MIDDLEWARES</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping-or-set z-python"><span class="z-punctuation z-section z-mapping-or-set z-begin z-python">{</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping-or-set z-python"></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">scrapy_splash.SplashCookiesMiddleware<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">723</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">scrapy_splash.SplashMiddleware<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">725</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">810</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span></code></pre> +<p>C. Enable SplashDeduplicateArgsMiddleware by adding it to SPIDER_MIDDLEWARES in your settings.py:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">SPIDER_MIDDLEWARES</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping-or-set z-python"><span class="z-punctuation z-section z-mapping-or-set z-begin z-python">{</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping-or-set z-python"></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">scrapy_splash.SplashDeduplicateArgsMiddleware<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">100</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> +</span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span></code></pre> +<p>D. Set a custom DUPEFILTER_CLASS:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">DUPEFILTER_CLASS</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">scrapy_splash.SplashAwareDupeFilter<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span> +</span></code></pre> +<h2 id="2-scraping-with-splash">2 - Scraping with Splash</h2> +<p>Before you use <code>scrapy-splash</code> you need to import it in your spider. +You can do that by adding this line:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">scrapy_splash</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">SplashRequest</span></span> +</span></code></pre> +<p>from now on insted of using <code>scrapy.Request</code> you can simply use <code>SplashRequest</code> to get response from <code>Splash</code> insted of directly from ther server.</p> +<h2 id="bonus-using-scrapy-splash-in-shell">Bonus - Using Scrapy-Splash in Shell</h2> +<p>It's all well and good but actual spider buiding does not happens in <code>vim</code> or <code>sublime</code>, it takes place in <code>shell</code>.</p> +<p><strong>So how to use Splash in the shell?</strong></p> +<p>Good Question.</p> +<p>Insted of invoking shell with:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">scrapy</span></span><span class="z-meta z-function-call z-arguments z-shell"> shell</span> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;&gt;</span><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> fetch<span class="z-punctuation z-definition z-compound z-begin z-shell">(</span><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">http://domain.com/page-with-javascript.html</span></span><span class="z-punctuation z-definition z-compound z-end z-shell">)</span> +</span></code></pre> +<p>or with this:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">scrapy</span></span><span class="z-meta z-function-call z-arguments z-shell"> shell http://domain.com/page-with-javascript.html</span> +</span></code></pre> +<p><strong>You invoke shell with this</strong>:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">scrapy</span></span><span class="z-meta z-function-call z-arguments z-shell"> shell <span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>http://localhost:8050/render.html?url=http://domain.com/page-with-javascript.html&amp;timeout=10&amp;wait=0.5<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span></span> +</span></code></pre> +<p>Let me explain</p> +<ul> +<li><code>localhost:port</code> is where your splash service is running</li> +<li><code>url</code> is url you want to crawl</li> +<li><code>render.html</code> is one of the possible http api endpoints, returns redered html page in this case</li> +<li><code>timeout</code> time in seconds for timeout</li> +<li><code>wait</code> time in seconds to wait for javascript to execute before reading/saving the html.</li> +</ul> +<p>If I’ve missed something, made a horrible mistake of if you have any questions regarding this article then feel free to ping me on Twitter. I’m +<a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a>.</p> + + + + Deploy a React app with sass using Nginx + Sat, 02 Jun 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/build-deploy-react-app-with-nginx/ + https://aaqa.dev/notes/build-deploy-react-app-with-nginx/ + <p>A couple of days ago (at the time of writing), I started my newest side +project. It's a portfolio showcasing my (very very very) amateur +photography. It's written in React.js with Sass and I have to say it was +extremely enjoyable to work on. Unsurprisingly though, I ran into some issues while +deploying to production, which after a lot of head banging against every +possible flat surface I could find, I managed to sort. So this post will be about +how to make React.js work with sass in production and how to serve the project +using Nginx as a front end web server.</p> +<p>We'll be using the official starter kit / CLI tool provided by the facebook +team called <code>create-react-app</code>.</p> +<p>You can install it by running the following command:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">npm</span></span><span class="z-meta z-function-call z-arguments z-shell"> install<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>g</span> create-react-app</span> +</span></code></pre> +<h2 id="adding-sass-to-a-react-project">Adding Sass to a React project</h2> +<p>Including sass in a React app can be done in two ways: You can either eject the +project and manually modify the webpack config files or you can follow the +procedure in the <a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/facebookincubator/create-react-app/#getting-started">official docs</a>. I've personally chosen to go with the first approach. +You can just follow the steps in this <a rel="noopener nofollow noreferrer" target="_blank" href="https://medium.com/@Connorelsea/using-sass-with-create-react-app-7125d6913760">tutorial</a> to get up and running.</p> +<p><em>Note: If you went with the second approach, you can just skip the following and +directly jump to the next section</em></p> +<p>Once you've done that, open the config/webpack.config.prod.js file, locate the +<code>rules</code> section under <code>module.exports</code> and add the following snippet to it.</p> +<pre data-lang="javascript" class="language-javascript z-code"><code class="language-javascript" data-lang="javascript"><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-entity z-name z-label z-ts">test</span><span class="z-punctuation z-separator z-label z-ts">:</span><span class="z-string z-regexp z-ts"> <span class="z-punctuation z-definition z-string z-begin z-ts">/</span><span class="z-constant z-character z-escape z-backslash z-regexp">\.</span>sass<span class="z-keyword z-control z-anchor z-regexp">$</span><span class="z-punctuation z-definition z-string z-end z-ts">/</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-entity z-name z-label z-ts">use</span><span class="z-punctuation z-separator z-label z-ts">:</span> <span class="z-meta z-function-call z-ts"><span class="z-variable z-other z-object z-ts">ExtractTextPlugin</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-entity z-name z-function z-ts">extract</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-meta z-objectliteral z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts">fallback</span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span> <span class="z-meta z-function-call z-ts"><span class="z-variable z-other z-object z-ts">require</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-support z-function z-promise z-ts">resolve</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>style-loader<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span><span class="z-meta z-brace z-round z-ts">)</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"> <span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts">use</span></span><span class="z-meta z-object z-member z-ts"><span class="z-meta z-object-literal z-key z-ts"><span class="z-punctuation z-separator z-key-value z-ts">:</span></span><span class="z-meta z-array z-literal z-ts"> <span class="z-meta z-brace z-square z-ts">[</span><span class="z-meta z-function-call z-ts"><span class="z-variable z-other z-object z-ts">require</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-support z-function z-promise z-ts">resolve</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>css-loader<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span><span class="z-meta z-brace z-round z-ts">)</span><span class="z-punctuation z-separator z-comma z-ts">,</span> <span class="z-meta z-function-call z-ts"><span class="z-variable z-other z-object z-ts">require</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-support z-function z-promise z-ts">resolve</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-string z-quoted z-double z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&quot;</span>sass-loader<span class="z-punctuation z-definition z-string z-end z-ts">&quot;</span></span><span class="z-meta z-brace z-round z-ts">)</span><span class="z-meta z-brace z-square z-ts">]</span></span> +</span></span></span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-meta z-objectliteral z-ts"><span class="z-meta z-object z-member z-ts"> </span><span class="z-punctuation z-definition z-block z-ts">}</span></span><span class="z-meta z-brace z-round z-ts">)</span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"> <span class="z-entity z-name z-label z-ts">include</span><span class="z-punctuation z-separator z-label z-ts">:</span> <span class="z-variable z-other z-object z-ts">paths</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-variable z-other z-property z-ts">appSrc</span><span class="z-punctuation z-separator z-comma z-ts">,</span> +</span></span><span class="z-source z-ts"><span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">}</span></span> +</span></code></pre> +<p>This config will be executed when you call <code>npm run build</code>. If you don't do +this, you'll end up with an empty css file.</p> +<h2 id="important-note">Important note</h2> +<p>Before building the project for the first time, we must first unregister the +service worker. Doing this will ensure that the static assets aren't cached by +the client's browsers. It's important because if you skip this step, any +subsequent change / build you'll deploy won't necessarily be reflected right away +client side. (I'm not knowledgable enough on service workers to provide details +on this behaviour, but it's something to note. If you really need service +workers in your project, you might want to explore other solutions to avoid +asset caching).</p> +<p>Your index.js file should now look something like this:</p> +<pre data-lang="javascript" class="language-javascript z-code"><code class="language-javascript" data-lang="javascript"><span class="z-source z-ts"><span class="z-meta z-import z-ts"><span class="z-keyword z-control z-import z-ts">import</span> <span class="z-meta z-block z-ts"><span class="z-punctuation z-definition z-block z-ts">{</span> <span class="z-variable z-other z-readwrite z-alias z-ts">unregister</span> <span class="z-punctuation z-definition z-block z-ts">}</span></span> <span class="z-keyword z-control z-from z-ts">from</span> <span class="z-string z-quoted z-single z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&#39;</span>./registerServiceWorker<span class="z-punctuation z-definition z-string z-end z-ts">&#39;</span></span></span> +</span><span class="z-source z-ts"><span class="z-meta z-import z-ts"><span class="z-keyword z-control z-import z-ts">import</span> <span class="z-variable z-other z-readwrite z-alias z-ts">App</span> <span class="z-keyword z-control z-from z-ts">from</span> <span class="z-string z-quoted z-single z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&#39;</span>./App<span class="z-punctuation z-definition z-string z-end z-ts">&#39;</span></span></span><span class="z-punctuation z-terminator z-statement z-ts">;</span> +</span><span class="z-source z-ts"><span class="z-meta z-function-call z-ts"><span class="z-variable z-other z-object z-ts">ReactDOM</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-entity z-name z-function z-ts">render</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-cast z-expr z-ts"><span class="z-meta z-brace z-angle z-ts">&lt;</span><span class="z-entity z-name z-type z-ts">App</span> /<span class="z-meta z-brace z-angle z-ts">&gt;</span></span><span class="z-punctuation z-separator z-comma z-ts">,</span> <span class="z-meta z-function-call z-ts"><span class="z-support z-variable z-dom z-ts">document</span><span class="z-punctuation z-accessor z-ts">.</span><span class="z-support z-function z-dom z-ts">getElementById</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-string z-quoted z-single z-ts"><span class="z-punctuation z-definition z-string z-begin z-ts">&#39;</span>root<span class="z-punctuation z-definition z-string z-end z-ts">&#39;</span></span><span class="z-meta z-brace z-round z-ts">)</span><span class="z-meta z-brace z-round z-ts">)</span><span class="z-punctuation z-terminator z-statement z-ts">;</span> +</span><span class="z-source z-ts"><span class="z-meta z-function-call z-ts"><span class="z-entity z-name z-function z-ts">unregister</span></span><span class="z-meta z-brace z-round z-ts">(</span><span class="z-meta z-brace z-round z-ts">)</span> +</span></code></pre> +<h2 id="building-the-project">Building the project</h2> +<p>This is the part where we actually build our project. By building I mean +compiling all React files and their related dependencies, transpiling them into +vanilla Javascript, compiling sass files into css, minifying them etc ... +To do this, we can simply run <code>npm run build</code> and voila ! You should have a +brand new <code>build</code> folder in your project.</p> +<h2 id="creating-our-deploy-script">Creating our deploy script</h2> +<p>Your app is now compiled and ready to be served. Now is a good time to start +thinking about deployment strategies. In this basic scenario +(and quite frankly, most scenraios), you really just want to upload the build +folder to a remote server and have it accessible as a static site.</p> +<p>You can use ftp to transfer the files and it would be perfectly acceptable, but +it's not the most flexible solution. The alternative is to use a CLI utility called +<code>rsync</code> (which is available on mac and linux, not sure about windows). With +rsync, you can synchronise files and folders within the same computer or across +machines you have ssh access to.</p> +<p>Here's the command we would run to synchronise the build folder to a server +on the internet:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> Asssuming we&#39;re inside the project folder</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">rsync</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>avP</span> build/ username@remote_ip:/destination/path</span> +</span></code></pre> +<p>Let's break down this command:</p> +<p><code>-a</code> means archive, which is a shortcut for multiple switches. It +recursively syncs all files and subfolders within <code>build</code> to the +destnation path, keeping the modification dates, permissions and other metadata +unchanged.</p> +<p><code>-v</code> means verbose. It just outputs the steps to the screen so you can see +what happens in real time.</p> +<p><code>-P</code> stands for progress. This is particularly useful in this case because you +rely on a network connection to sync the files. Using this option will +display a progress bar for each file in the queue.</p> +<p>But you don't want to keep doing all of that every time we want to push now do +you ?</p> +<p>Thankfully, you can use create a bash script to automate this process a litte +bit. Here's how mine looks like:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell">!/bin/sh</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Merging branch to master<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> checkout master</span> <span class="z-keyword z-operator z-logical z-and z-shell">&amp;&amp;</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> merge develop</span> <span class="z-keyword z-operator z-logical z-and z-shell">&amp;&amp;</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> push origin master</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Merge completed<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Compiling project to build folder ...<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">npm</span></span><span class="z-meta z-function-call z-arguments z-shell"> run build</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Build process done<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Deploying files to server<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">rsync</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>avP</span> build/ user@host:/destination/path</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Deployment completed<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Switching to develop<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> checkout develop</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Done!<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span></code></pre> +<p>Again, let's walk through that script section by section:</p> +<ol> +<li>I checkout to master, merge develop and push. This ensures that my master +branch is always up to date with the latest version of my working codebase.</li> +<li>I execute <code>npm run build</code> which, as previously explained, will create the +build directory with our compiled, ready to be deployed files.</li> +<li>I use rsync to copy over the contents of the build folder to the destination +path in the remote machine I administer. (notice the trailing slash after +build/, this tells rsync to copy the contents of the folder and not the +folder itself).</li> +<li>I switch the current working branch back to develop so that I can start +developing without accidentally altering the state of <code>master</code>.</li> +</ol> +<p>Obviously this is very basic and in a more complex project, you'd have to run +unit tests and do other things your project requires.</p> +<p>Finally, you need to give executable permissions to the file by running:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> chmod +x deploy.sh</span> +</span></code></pre> +<p>Now all you have to do when you want to deploy your project to production is run</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">./deploy.sh</span></span> +</span></code></pre> +<h2 id="serving-our-site-with-nginx">Serving our site with Nginx</h2> +<p><em>Note: this assumes your server is running ubuntu or any other debian based +distro</em></p> +<p>Ok, so if you've followed the steps correctly, you should have your project +files uploaded to your remote server. Now we need to use Nginx to make the site +accessible to the internet.</p> +<p>First, create a new config file inside <code>/etc/nginx/sites-available</code>.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-cd z-shell">cd</span></span><span class="z-meta z-function-call z-arguments z-shell"> /etc/nginx/sites-available</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">touch</span></span><span class="z-meta z-function-call z-arguments z-shell"> mywebsiteconfig <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> File extension is not required in this case</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></code></pre> +<p>Next step is to edit the file using either vim or nano (sudo privileges may be +required).</p> +<pre data-lang="conf" class="language-conf z-code"><code class="language-conf" data-lang="conf"><span class="z-source z-genconfig"><span class="z-storage z-type z-genconfig">server </span>{ +</span><span class="z-source z-genconfig"> +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">listen</span> </span><span class="z-constant z-numeric z-genconfig">80</span>; +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">server_name</span> </span>mywebsite<span class="z-keyword z-operator z-genconfig">.</span>com www<span class="z-keyword z-operator z-genconfig">.</span>mywebsite<span class="z-keyword z-operator z-genconfig">.</span>com; +</span><span class="z-source z-genconfig"> +</span><span class="z-source z-genconfig"><span class="z-keyword z-other z-genconfig"> location</span> <span class="z-keyword z-operator z-genconfig">/</span> { +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">root</span> </span><span class="z-keyword z-operator z-genconfig">/</span>path<span class="z-keyword z-operator z-genconfig">/</span>to<span class="z-keyword z-operator z-genconfig">/</span>your<span class="z-keyword z-operator z-genconfig">/</span>project; +</span><span class="z-source z-genconfig"><span class="z-keyword z-other z-genconfig"> index</span> index<span class="z-keyword z-operator z-genconfig">.</span>html index<span class="z-keyword z-operator z-genconfig">.</span>htm; +</span><span class="z-source z-genconfig"> +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">default_type</span> </span><span class="z-string z-quoted z-double z-genconfig">&quot;text/html&quot;</span>; +</span><span class="z-source z-genconfig"> } +</span><span class="z-source z-genconfig"> +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">access_log</span> </span><span class="z-keyword z-operator z-genconfig">/</span>var<span class="z-keyword z-operator z-genconfig">/</span>log<span class="z-keyword z-operator z-genconfig">/</span>nginx<span class="z-keyword z-operator z-genconfig">/</span>mywebsite_access<span class="z-keyword z-operator z-genconfig">.</span>log; +</span><span class="z-source z-genconfig"><span class="z-meta z-param z-genconfig"> <span class="z-variable z-parameter z-genconfig">error_log</span> </span><span class="z-keyword z-operator z-genconfig">/</span>var<span class="z-keyword z-operator z-genconfig">/</span>log<span class="z-keyword z-operator z-genconfig">/</span>nginx<span class="z-keyword z-operator z-genconfig">/</span>mywebsite_errors<span class="z-keyword z-operator z-genconfig">.</span>log; +</span><span class="z-source z-genconfig">} +</span></code></pre> +<p>If you're not familiar with Nginx, let me explain what you just copied.</p> +<ol> +<li>We created a server block to hold our configuration and keep it separate from +other configs we may add in the future (like https for example).</li> +<li>We declare the listen directive which tells nginx to listen on port 80.</li> +<li>We set the server_name to our domain name. This tells nginx to apply the +config settings to any incoming request from any of listed urls.</li> +<li>Finally, we specify the paths to both the access and error logs. It's +optional but highly recommended, so that you know exactly where to look when +errors happen. This will save you a tonne of time when troubleshooting issues +in the future.</li> +</ol> +<p>Close the file and exit nano or vim.</p> +<p>Nginx keeps its configuration files in two separate directories: +<code>/etc/nginx/sites-available</code> and <code>/etc/nginx/sites-enabled</code>. It will serve +any website whose configuration file is in the latter folder.</p> +<p>All you have to do now is create a symlink (think of it like a shortcut to an app +in a desktop GUI) to your config, and store it in <code>sites-enabled</code>. +That way, if you ever decide to shut down the site, you'll simply need to +delete the symlink and you're good.</p> +<p>Before creating the symlink, it's good measure to check if the configuration +file has any errors in it. To check for errors just run the following command:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> nginx<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span></span> +</span></code></pre> +<p>It will scan all of your config files and check for errors (and return them to +you if there are any).</p> +<p>Now you can create the symlink by executing:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> ln<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>s</span> /etc/nginx/sites-available/mywebsite /etc/nginx/sites-enabled</span> +</span></code></pre> +<p><em>Note: If you decide to use service workers with your project and you're still +running into caching issues, you should know that Nginx can also be used as a +static assets server. I haven't looked into this scenario yet. I might test that +approach in the future and detail the process in another post.</em></p> +<p>Lastly, reload nginx and your website should now be accessible via its URL +(provided you correctly setup the DNS settings with your domain name registrar)</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> service nginx reload</span> +</span></code></pre> +<p>If you read this article all the way through, thanks for sticking with me ! You +can send me questions, remarks, or comments on twitter, I'm <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a> on +twitter.</p> + + + + The ultimate PostgreSQL cheatsheet + Wed, 09 May 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/the-ultimate-postgresql-cheatsheet/ + https://aaqa.dev/notes/the-ultimate-postgresql-cheatsheet/ + <p>So I had been working with Django, Flask and Express.js for a while now, and my +database of choice for every single project has of course always been Postgres. +What makes these frameworks great (regardless of language) is the ability to use +an ORM (Object Relational Mapper) that sits between your code and the DB. +It does all the heavy lifting for you and takes care of executing SQL queries on +your behalf.</p> +<p>This is great but I couldn't stop thinking &quot;what if I have to +manually debug something directly into postgres ?&quot; This is when reality slapped +me in the face, I barely knew the SQL Language. So I thought It'd be fun to +create a cheatsheet that I (and you) could keep as a reference for these times +when you absolutely need to set this column to UNIQUE and you don't know how.</p> +<p>Enjoy ! (and yeah I went all in with the pokemon references)</p> +<h3 id="create-a-user-or-role">Create a User (or Role)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> create a user without privileges*/</span> +</span><span class="z-source z-sql">CREATE ROLE aaqa; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> create a user with privileges*/</span> +</span><span class="z-source z-sql">CREATE ROLE aaqa LOGIN CREATEDB CREATEROLE REPLICATION; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> Add privileges to existing user*/</span> +</span><span class="z-source z-sql">ALTER ROLE aaqa <span class="z-keyword z-other z-DML z-sql">WITH</span> LOGIN CREATEROLE CREATEDB REPLICATION; +</span></code></pre> +<h3 id="make-a-user-superuser-bump-their-privileges">Make a user superuser (bump their privileges)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql">ALTER ROLE aaqa <span class="z-keyword z-other z-DML z-sql">WITH</span> superuser; +</span></code></pre> +<h3 id="rename-an-existing-user">Rename an existing user</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql">ALTER ROLE psyduck RENAME TO brock; +</span></code></pre> +<h3 id="create-a-db">Create a DB</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">DATABASE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">pokemons</span></span>; +</span></code></pre> +<h3 id="rename-a-db">Rename a DB</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">DATABASE</span> </span>pokemons RENAME TO charizard; +</span></code></pre> +<h3 id="create-a-table-in-a-db">Create a table in a DB</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">TABLE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">trainers</span></span>( +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> id <span class="z-storage z-type z-sql">INT</span> <span class="z-storage z-modifier z-sql">PRIMARY KEY</span> <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> pokemon_type <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">50</span>), +</span><span class="z-source z-sql"> name <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">50</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> UNIQUE, +</span><span class="z-source z-sql"> gender <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">50</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> UNIQUE +</span><span class="z-source z-sql">); +</span></code></pre> +<h3 id="delete-a-db">Delete a DB</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-drop z-sql"><span class="z-keyword z-other z-create z-sql">DROP</span> <span class="z-keyword z-other z-sql">DATABASE</span></span> pokemons; +</span></code></pre> +<h3 id="delete-a-user">Delete a user</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> assuming there&#39;s a pikachu role on the system */</span> +</span><span class="z-source z-sql">DROP ROLE pikachu; +</span></code></pre> +<h3 id="change-db-ownership">Change DB ownership</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">DATABASE</span> </span>pokemons OWNER TO aaqa; +</span></code></pre> +<h3 id="rename-a-table">Rename a table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>trainers RENAME TO gym_trainers; +</span></code></pre> +<h3 id="change-column-type">Change column type</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ALTER COLUMN pokemon_type TYPE <span class="z-storage z-type z-sql">TEXT</span>; <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> it makes zero +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">sense to want to change this column type to TEXT but YOLO */</span> +</span></code></pre> +<h3 id="rename-a-column">Rename a column</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ALTER COLUMN name RENAME TO trainer_name; +</span></code></pre> +<h3 id="add-a-column-to-a-table">Add a column to a table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ADD COLUMN bio <span class="z-storage z-type z-sql">TEXT</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">ALTER TABLE &lt;table_name&gt; +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">ADD COLUMN &lt;column_name&gt; &lt;data_type&gt; &lt;constraints if any&gt; +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span></code></pre> +<h3 id="add-a-column-with-a-unique-constraint">Add a column with a UNIQUE constraint</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ADD COLUMN age <span class="z-storage z-type z-sql">INT</span> UNIQUE; <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> cause why not */</span> +</span></code></pre> +<h3 id="add-a-column-with-a-not-null-constraint">Add a column with a NOT NULL constraint</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ADD COLUMN main_pokemon <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">60</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>; +</span></code></pre> +<h3 id="remove-not-null-constraint-from-a-column">Remove NOT NULL CONSTRAINT from a column</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ALTER COLUMN main_pokemon DROP <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>; +</span></code></pre> +<h3 id="add-a-column-with-a-not-null-constraint-and-a-default-value">Add a column with a NOT NULL constraint and a DEFAULT value</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>gym_trainers ADD COLUMN city <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">80</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> <span class="z-storage z-modifier z-sql">DEFAULT</span> <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>Indigo +</span></span><span class="z-source z-sql"><span class="z-string z-quoted z-single z-sql">Plateau<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>; +</span></code></pre> +<h3 id="add-a-column-with-a-check-constraint">Add a column with a CHECK constraint</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">TABLE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">trainers</span></span>( +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> id <span class="z-storage z-type z-sql">INT</span> <span class="z-storage z-modifier z-sql">PRIMARY KEY</span>, +</span><span class="z-source z-sql"> trainer_name <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">50</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> method 1 +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Add the check yolo style +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> */</span> +</span><span class="z-source z-sql"> age <span class="z-storage z-type z-sql">INT</span> <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> <span class="z-storage z-modifier z-sql">CHECK</span> (age <span class="z-keyword z-operator z-comparison z-sql">&gt;</span> <span class="z-constant z-numeric z-sql">18</span>) +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> method 2 +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Add a named constraint for better error handling +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> */</span> +</span><span class="z-source z-sql"> age <span class="z-storage z-type z-sql">INT</span> <span class="z-storage z-modifier z-sql">CONSTRAINT</span> legal_age <span class="z-storage z-modifier z-sql">CHECK</span> (age <span class="z-keyword z-operator z-comparison z-sql">&gt;</span> <span class="z-constant z-numeric z-sql">18</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> method 3 +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Add the constraint at the end +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> for more clarity +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> */</span> +</span><span class="z-source z-sql"> age <span class="z-storage z-type z-sql">INT</span> <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> <span class="z-storage z-modifier z-sql">CONSTRAINT</span> legal_age <span class="z-storage z-modifier z-sql">CHECK</span> (age <span class="z-keyword z-operator z-comparison z-sql">&gt;</span> <span class="z-constant z-numeric z-sql">18</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> +</span><span class="z-source z-sql">); +</span></code></pre> +<h3 id="add-a-constraint-to-an-existing-column">Add a CONSTRAINT to an existing column</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>trainers ADD <span class="z-storage z-modifier z-sql">CONSTRAINT</span> unique_name UNIQUE (trainer_name); +</span></code></pre> +<h3 id="remove-a-named-constraint-from-a-table">Remove a named CONSTRAINT from a table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>trainers DROP <span class="z-storage z-modifier z-sql">CONSTRAINT</span> unique_name; +</span></code></pre> +<h3 id="insert-a-row-into-a-table">Insert a row into a table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">INSERT INTO</span> trainers <span class="z-keyword z-other z-DML z-II z-sql">VALUES</span> (<span class="z-constant z-numeric z-sql">1</span>, <span class="z-constant z-numeric z-sql">23</span>, <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>brock<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>); +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> or */</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">INSERT INTO</span> trainers (age, trainer_name) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-II z-sql">VALUES</span> (<span class="z-constant z-numeric z-sql">1</span>, <span class="z-constant z-numeric z-sql">23</span>, <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>brock<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>); +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Note that in the second case we don&#39;t have pass +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> the id. Postgres will automatically generate and autoincrement +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> it for us. To omit the id column we must use named inserts otherwise an +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> error is raised. +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span></code></pre> +<h3 id="insert-multiple-rows-into-a-table">Insert multiple rows into a table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">INSERT INTO</span> trainers (age, trainer_name) <span class="z-keyword z-other z-DML z-II z-sql">VALUES</span> +</span><span class="z-source z-sql">(<span class="z-constant z-numeric z-sql">19</span>, <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>misty<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>), +</span><span class="z-source z-sql">(<span class="z-constant z-numeric z-sql">22</span>, <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>chen<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>), +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ... */</span> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ... */</span> +</span></code></pre> +<h3 id="clear-a-table-without-deleting-it">Clear a table (without deleting it)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">TRUNCATE</span> trainers; +</span></code></pre> +<h3 id="set-the-primary-key-type-to-a-serial-an-auto-incrementing-integer">Set the primary key type to a serial (An auto incrementing integer)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> Considering this table structure */</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">TABLE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">pokemon_list</span></span> ( +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> id <span class="z-storage z-type z-sql">INT</span> <span class="z-storage z-modifier z-sql">PRIMARY KEY</span>, +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ... */</span> +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ... */</span> +</span><span class="z-source z-sql"> <span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ... */</span> +</span><span class="z-source z-sql">); +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> 1. +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Create a sequence for the auto generating prinary key +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> It follows the tablename_columnname_seq +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">SEQUENCE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">pokemon_list_id_seq</span></span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> 2. Set the id column to not null */</span> +</span><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>pokemon_list ALTER COLUMN id <span class="z-keyword z-other z-DML z-sql">SET</span> <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> 3. Set the default value to the next value in the sequence*/</span> +</span><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>pokemon_list +</span><span class="z-source z-sql">ALTER COLUMN pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SET</span> <span class="z-storage z-modifier z-sql">DEFAULT</span> nextval(<span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>pokemon_list_id_seq<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>); +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> 4. Link the sequence to the correct table and column */</span> +</span><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">SEQUENCE</span> </span>pokemon_list_id_seq OWNED BY <span class="z-constant z-other z-database-name z-sql">pokemon_list</span>.<span class="z-constant z-other z-table-name z-sql">id</span>; +</span></code></pre> +<h3 id="import-data-from-a-file">Import data from a file</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Must use absolute path and the user must have appropriate permissions +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> Defaults to importing using TAB as the default parameter. +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> We&#39;ll use a csv file as an example +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql">COPY pokemon_list <span class="z-keyword z-other z-DML z-sql">FROM</span> <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>/path/to/yourfile.csv<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span> DELIMITER <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>,<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> This only works if a pk is specified for each row +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> The (my) prefered way to do it is the following +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql">COPY pokemon_list (name, level, type) <span class="z-keyword z-other z-DML z-sql">FROM</span> <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>/path/to/yourfile.csv<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span> DELIMITER <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>,<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> It&#39;s much more flexible because you control what data you actually import*/</span> +</span></code></pre> +<h3 id="export-a-table-to-a-file">Export a table to a file</h3> +<p>To be able to export a table to a file, we need to ensure that postgres has +write permissions to the file.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> chmod 777 /path/to/directory <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> This is just an example, edit this as needed</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></code></pre> +<p>We can now safely copy the table to the file.</p> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql">COPY pokemon_list TO <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>/path/to/file.csv<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span> DELIMITER <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>,<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>; +</span></code></pre> +<h3 id="select-columns-by-using-aliases">Select columns by using aliases</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> name <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> pokemon_name, type <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> pokemon_type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span></code></pre> +<h3 id="select-elements-based-on-a-criteria">Select elements based on a criteria</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> id <span class="z-keyword z-operator z-comparison z-sql">&gt;</span> <span class="z-constant z-numeric z-sql">3</span>; +</span></code></pre> +<h3 id="select-elements-based-on-string-comparison">Select elements based on string comparison</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> type <span class="z-keyword z-operator z-logical z-sql">LIKE</span> <span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>%water%<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span>; +</span></code></pre> +<h3 id="select-all-results-and-order-them-by-id-in-reverse">Select all results and order them by id in reverse</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">ORDER BY</span> id <span class="z-keyword z-other z-order z-sql">DESC</span>; +</span></code></pre> +<h3 id="select-all-results-and-order-them-by-a-column-name">Select all results and order them by a column name</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">ORDER BY</span> level; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> If the column you&#39;re ordering by is not of type INT then the ordering will be +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">done alphabetically */</span> +</span></code></pre> +<h3 id="select-distinct-column-from-table">Select DISTINCT column from table</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT DISTINCT</span> type <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> pokemon_type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span></code></pre> +<h3 id="limit-the-results-from-a-select-query">Limit the results from a SELECT query</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">LIMIT</span> <span class="z-constant z-numeric z-sql">3</span>; +</span></code></pre> +<h3 id="select-the-last-3-items">Select the last 3 items</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">ORDER BY</span> id <span class="z-keyword z-other z-order z-sql">DESC</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">LIMIT</span> <span class="z-constant z-numeric z-sql">3</span>; +</span></code></pre> +<h3 id="create-two-tables-with-a-foreign-key-relationship">Create two tables with a foreign key relationship</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">TABLE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">pokemon_types</span></span>( +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> id <span class="z-storage z-type z-sql">SERIAL</span> <span class="z-storage z-modifier z-sql">PRIMARY KEY</span>, +</span><span class="z-source z-sql"> type_name <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">120</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> +</span><span class="z-source z-sql">) +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">TABLE</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">pokemon_list</span></span>( +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> id <span class="z-storage z-type z-sql">serial</span> <span class="z-storage z-modifier z-sql">PRIMARY KEY</span>, +</span><span class="z-source z-sql"> pokemon_name <span class="z-storage z-type z-sql">CHAR</span>(<span class="z-constant z-numeric z-sql">120</span>) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> pokemon_level <span class="z-storage z-type z-sql">INT</span> <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> pokemon_type <span class="z-storage z-type z-sql">INT</span> <span class="z-storage z-modifier z-sql">REFERENCES</span> pokemon_types(id) <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span>, +</span><span class="z-source z-sql"> <span class="z-storage z-modifier z-sql">CONSTRAINT</span> pokemon_level_not_zero <span class="z-storage z-modifier z-sql">CHECK</span> (pokemon_level <span class="z-keyword z-operator z-comparison z-sql">&gt;</span> <span class="z-constant z-numeric z-sql">0</span>) +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql">); +</span></code></pre> +<h3 id="perform-joins-based-on-a-criteria">Perform Joins based on a criteria</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> name, level, <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">name</span> <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">JOIN</span> pokemon_types +</span><span class="z-source z-sql">ON pokemon_type_id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">id</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> pokemon_type_id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-numeric z-sql">1</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> Will return the name, level and type name for all water pokemons */</span> +</span></code></pre> +<h3 id="perform-joins">Perform joins</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> name, level, <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">name</span> <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">JOIN</span> pokemon_types +</span><span class="z-source z-sql">ON pokemon_type_id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">id</span>; +</span></code></pre> +<h3 id="create-a-view-based-on-a-join">Create a VIEW based on a JOIN</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-create z-sql"><span class="z-keyword z-other z-create z-sql">CREATE</span> <span class="z-keyword z-other z-sql">VIEW</span> </span><span class="z-meta z-toc-list z-full-identifier z-sql">pokemonswithtypes </span><span class="z-meta z-toc-list z-full-identifier z-sql"><span class="z-entity z-name z-function z-sql">AS</span></span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> name, level <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">name</span> <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">JOIN</span> pokemon_types +</span><span class="z-source z-sql">ON pokemon_type_id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">id</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> To see the data */</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-variable z-language z-star z-sql">*</span> <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemonswithtypes; +</span></code></pre> +<h3 id="update-the-view-change-the-query">Update the VIEW (Change the query)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> name, <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">name</span> <span class="z-keyword z-operator z-assignment z-alias z-sql">AS</span> type +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">JOIN</span> pokemon_types +</span><span class="z-source z-sql">ON pokemon_type <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-other z-database-name z-sql">pokemon_types</span>.<span class="z-constant z-other z-table-name z-sql">id</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> pokemon_type <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-numeric z-sql">1</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> only show the name and type for water pokemons */</span> +</span></code></pre> +<h3 id="delete-the-view">Delete the VIEW</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-drop z-sql"><span class="z-keyword z-other z-create z-sql">DROP</span> <span class="z-keyword z-other z-sql">VIEW</span></span> pokemonswithtypes; +</span></code></pre> +<h3 id="use-aggregate-functions-min-max-sum-count-avg">Use aggregate functions (MIN, MAX, SUM, COUNT, AVG)</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> MAX */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-support z-function z-aggregate z-sql">MAX</span>(pokemon_level) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> MIN */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-support z-function z-aggregate z-sql">MIN</span>(pokemon_level) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> AVG */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-support z-function z-aggregate z-sql">AVG</span>(pokemon_level) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> ROUND */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> ROUND(<span class="z-support z-function z-aggregate z-sql">AVG</span>(pokemon_level)) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> COUNT */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-support z-function z-aggregate z-sql">COUNT</span>(<span class="z-variable z-language z-star z-sql">*</span>) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> SUM */</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> <span class="z-support z-function z-aggregate z-sql">SUM</span>(pokemon_level) +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span></code></pre> +<h3 id="use-boolean-aggregate-functions">Use boolean aggregate functions</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> Add a column is_legendary of type boolean to table pokemon_list */</span> +</span><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>pokemon_list ADD COLUMN is_legendary BOOL <span class="z-keyword z-operator z-logical z-sql">NOT</span> <span class="z-constant z-language z-sql">NULL</span> <span class="z-storage z-modifier z-sql">DEFAULT</span> <span class="z-constant z-boolean z-sql">TRUE</span>; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> BOOL_AND +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> returns a result if **ALL** records have that column set to true +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> BOOL_AND(is_legendary) <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> BOOL_OR +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c"> returns a result if one or more records have that column set to true +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">*/</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SELECT</span> BOOL_OR(is_legendary) <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span></code></pre> +<h3 id="update-a-table-and-change-all-column-values">Update a table and change all column values</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">UPDATE</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SET</span> is_legendary <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-boolean z-sql">FALSE</span>; +</span></code></pre> +<h3 id="update-a-table-and-change-value-based-on-a-criteria">Update a table and change value based on a criteria</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">UPDATE</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">SET</span> is_legendary <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-boolean z-sql">TRUE</span> +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-numeric z-sql">2</span>; +</span></code></pre> +<h3 id="delete-row-with-specific-id">Delete row with specific id</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">DELETE</span> <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list <span class="z-keyword z-other z-DML z-sql">WHERE</span> id <span class="z-keyword z-operator z-comparison z-sql">=</span> <span class="z-constant z-numeric z-sql">4</span> +</span></code></pre> +<h3 id="delete-rows-withing-a-range-of-ids">Delete rows withing a range of ids</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">DELETE</span> <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list +</span><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">WHERE</span> id <span class="z-keyword z-operator z-logical z-sql">BETWEEN</span> <span class="z-constant z-numeric z-sql">1</span> <span class="z-keyword z-operator z-logical z-sql">AND</span> <span class="z-constant z-numeric z-sql">4</span>; +</span></code></pre> +<h3 id="delete-all-rows">Delete all rows</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-DML z-sql">DELETE</span> <span class="z-keyword z-other z-DML z-sql">FROM</span> pokemon_list; +</span></code></pre> +<p><em>Note: The difference between DELETE and DROP or TRUNCATE is that the former can +be undone (rolled back) the latter can't</em></p> +<h3 id="alter-a-table-to-drop-a-constraint-if-it-exist">Alter a table to drop a CONSTRAINT if it exist</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-meta z-alter z-sql"><span class="z-keyword z-other z-create z-sql">ALTER</span> <span class="z-keyword z-other z-table z-sql">TABLE</span> </span>pokemon_types +</span><span class="z-source z-sql">DROP <span class="z-storage z-modifier z-sql">CONSTRAINT</span> IF <span class="z-keyword z-operator z-logical z-sql">EXISTS</span> unique_type_name; +</span></code></pre> +<h3 id="comments">Comments</h3> +<pre data-lang="sql" class="language-sql z-code"><code class="language-sql" data-lang="sql"><span class="z-source z-sql"><span class="z-keyword z-other z-object-comments z-sql">COMMENT ON TABLE pokemon_types is </span><span class="z-string z-quoted z-single z-sql"><span class="z-punctuation z-definition z-string z-begin z-sql">&#39;</span>pokemon with types<span class="z-punctuation z-definition z-string z-end z-sql">&#39;</span></span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> To display the comment, in psql simply run \dt+. It will return a description +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">column containing that comment. It&#39;s useful when working on a legacy database +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">for example*/</span> +</span><span class="z-source z-sql"> +</span><span class="z-source z-sql"><span class="z-comment z-block z-c"><span class="z-punctuation z-definition z-comment z-sql">/*</span> Please note that comments aren&#39;t exclusive to tables, they can be executed on +</span></span><span class="z-source z-sql"><span class="z-comment z-block z-c">schemas and multiple other objects.*/</span> +</span></code></pre> +<p><em>Note: If you find errors, typos or would like to add new tips, feel free to +reach out to me on twitter. I'm <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a>. Thank +you for reading ! And if you find this useful, share it with your friends and +coworkers !</em></p> + + + + The ultimate setup for remote development + Tue, 08 May 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/the-ultimate-setup-for-remote-development/ + https://aaqa.dev/notes/the-ultimate-setup-for-remote-development/ + <p>I'm a programmer. Like most programmers, I use an Apple laptop. They're just the +best on the market and the best purchase you can make as a developer. No +fanboyism here, just stating facts. They're fast, well built, and durable.</p> +<p>I rely on this laptop for all my personal / client work. I run vagrant boxes, docker containers in it and all the usual stuff.</p> +<p>As a result, I've developed this irrational fear that it would get stolen or +fall off my bedroom window (things like that happen, trust me). Interestingly +enough, it coincides with my growing interest for remote work.</p> +<p>So I started to look for ways to create a development server that would allow me +to remain productive even if I lose / break my laptop, or when I'm on the road.</p> +<h2 id="why">Why</h2> +<p>Ok, so you've read all this and you're thinking &quot;How is it going to benefit me ?&quot;.</p> +<p>The main selling points to creating a remote development environment are the +following:</p> +<ul> +<li>It's cheap</li> +<li>You learn linux</li> +<li>You become OS agnostic (I said no fanboyism)</li> +<li>You can work from almost anywhere</li> +</ul> +<h2 id="what-you-need">What you need</h2> +<h3 id="laptop">Laptop</h3> +<p>Because all of your work is now done on a remote machine you don't really have +to care about what computer you're using. It can be a super expensive (notice I +didn't say overpriced) Apple laptop or any of its really good windows competitors (the DELL xps 13/15 comes to mind) or even a super cheap, 35$ raspberry Pi. Some people even use chromebooks ! They stick ubuntu on them and use them as their primary machines (SSH is a bit tricky to setup on chrome OS) but hey! Official <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.xda-developers.com/chromebooks-linux-app-support/">Linux support</a> is coming for you ChromeOS folks.</p> +<h2 id="dev-tools">Dev Tools</h2> +<h3 id="git">Git</h3> +<p>This one's obvious. While git is a life saver and a great tool for collaboration in large teams (distributed or not), you can also use it as a backup system for your code when you're a solo developer.</p> +<h3 id="vim">Vim</h3> +<p>I've used sublime text but felt bad for continuously extending the trial because I +couldn't pay for it. Then I switched back to VSCode and got frustrated by how slow it +runs after using Sublime. Then i tried vim, and I never +looked back. It's the lightest, fastest and overall best text editor out there by a huge +margin (in my opinion).</p> +<p>Vim is highly customizable and lets you save your settings inside a +<code>.vimrc</code> file, which makes it version control friendly. It also makes your vim +environment 100% portable as it comes standard in most linux server distros.</p> +<h3 id="tmux">Tmux</h3> +<p>The best companion to vim. Tmux is a terminal multiplexer. With Tmux you can +essentially access multiple terminal sessions inside a single window. This gives +you the ability to work on multiple projects at a time. You can even save +sessions, attach and reattach to them. It's painless. +Just like vim, it's extremely customizable. All of your settings can be stored +inside of a <code>.tmux.conf</code> file.</p> +<h3 id="a-vps">A VPS</h3> +<p>To start developing on a remote server, you'll need ... well, a remote +server. If you don't know what a VPS is, it stands for <strong>Virtual Private Server</strong>. +It's essentially a virtual machine that you pay for monthly, which runs a server +distribution of linux (Ubuntu, CentOS ...), or Unix (FreeBSD). You can connect to it via SSH (it has a public ip address) and start playing around. You have complete control over the server, you can configure it however way you want.</p> +<p>There's a large range of VPS providers on the market, the most notable +ones are <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.linode.com/">Linode</a>, <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.digitalocean.com">Digital Ocean</a>, and <a rel="noopener nofollow noreferrer" target="_blank" href="https://aws.amazon.com/ec2/">Amazon EC2</a>.</p> +<h3 id="dotfiles">Dotfiles</h3> +<p>As I previously mentioned above, you can save your settings for vim and tmux in +what are called <strong>dotfiles</strong> (files that start with a period). However dotfiles +are not exclusive to vim and tmux, you can for example save your shell configuration in a <strong>.zshrc</strong> or <strong>.bashrc</strong> config file, or your git settings in a <strong>.gitconfig</strong> file.</p> +<p>Doing this is extremely powerful because you can store these configuration files +on github and always pull the latest version when you launch a new development server.</p> +<p>You can check out my own <a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/aaqaishtyaq/dotfiles">dotfiles</a> on github to help you get started.</p> +<h3 id="optional-ansible">(Optional) Ansible</h3> +<p>Manually setting up servers is fun for the first couple of times, then it just +becomes repetitive. And what do you do with repetitive tasks ? You automate +them.</p> +<p>Ansible is a provisioning tool written in python that will help you do just that. +You give it the ip address (or addresses) of the server you want to configure and it will execute all the tasks you tell it to.</p> +<p>With Ansible you can:</p> +<ul> +<li>run shell commands</li> +<li>install packages</li> +<li>create directories and files</li> +<li>add users and groups</li> +<li>clone git repos</li> +<li>use templates and pass variables to them</li> +<li>and much more</li> +</ul> +<p>If you don't like Ansible, there are other server provisioning tools like +<a rel="noopener nofollow noreferrer" target="_blank" href="https://puppet.com/">puppet</a>, <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.chef.io/">chef</a> and <a rel="noopener nofollow noreferrer" target="_blank" href="https://saltstack.com/">salt</a> that will work just as good.</p> +<!--If you're interested, you can have a look at the [ansible playbook I've made](https://github.com/aaqaishtyaq/night-city) to provision my development servers.--> +<h2 id="caveats">Caveats</h2> +<ul> +<li>it takes time to learn linux and networking (SSH etc)</li> +<li>vim has a learning curve to it</li> +<li>you become reliant on an internet connection</li> +</ul> + + + + Deploy your Django application with git + Mon, 07 May 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/deploy-django-application-git/ + https://aaqa.dev/notes/deploy-django-application-git/ + <p>I'm going to make a bold statement: Django replaced Ruby on Rails in the hearts +of many developers. With this increase in popularity, we've seen tons or +articles, videos and websites dedicated to setting un Django and creating apps +using the framework.</p> +<p>Unfortunately, when it comes to deployment, many of these resources only mention +heroku or pythonanywhere. While these are excellent solutions for quickly +shipping your MVP or prototype, it lacks a bit in terms of flexibility if you +want to create your custom deployment pipeline.</p> +<p><strong>tl-dr: If you manage your own server infrastructure, we're going to setup a git +deployment workflow with django</strong></p> +<h2 id="what-you-ll-need">What you'll need</h2> +<ul> +<li>Working knowledge of ssh</li> +<li>Working knowledge of git</li> +<li>Working knowledge of the bash shell</li> +<li>Basic linux command line skills (sorry hipsters)</li> +<li>(Very) Basic knowledge of vi/vim</li> +<li>Patience</li> +</ul> +<p>The typical workflow usually looks like this:</p> +<ul> +<li>You have your development environment, either on your local machine or a remote server</li> +<li>A git server (on GitHub, BitBucket, GitLab ...) that you and your team push your work to</li> +<li>A production server (aka your live app).</li> +</ul> +<p>Usually when you commit and push work you do something like:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> push origin <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>branch_name<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> +</span></span></code></pre> +<p><code>origin</code> being the name of the remote server your code is being pushed to. +What took me a while to realise is that you can have many remotes for your repo +that point to different servers.</p> +<p>The idea here is to add a new remote to our repo. It will point to our +production server such that when we run <code>git push live master</code>, our code will be +copied over to there.</p> +<h2 id="on-the-production-server">On the production server</h2> +<p>To achieve this, we have some setup work to do on our live server. So go ahead +and connect to it via ssh.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">ssh</span></span><span class="z-meta z-function-call z-arguments z-shell"> user@host_or_ip_address</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> If your server&#39;s ssh service listens on a port other than 22, you&#39;ll need</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> to add the -p switch</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">ssh</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> PORT user@host_or_ip_address</span> +</span></code></pre> +<p>Once we're in, we need to create a new directory for our application. This is +where our deployed code will be copied to.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">mkdir</span></span><span class="z-meta z-function-call z-arguments z-shell"><span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>p</span> /home/user/sites/myawesomedjangoproject</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> Some people prefer to use /var/www/&lt;project_name&gt;, it&#39;s really up to you. Just</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> make sure remember the path to your project</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></code></pre> +<p>Now head over to <code>/var</code> and create another directory called <code>repos</code></p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-cd z-shell">cd</span></span><span class="z-meta z-function-call z-arguments z-shell"> /var</span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> mkdir repos</span> +</span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> Depending on your setup, you might need sudo priveleges</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></code></pre> +<p>Inside that directory, we need to create a folder named after our project +(or domain name) and append it with <code>.git</code> (not necessary but it's good practice)</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> mkdir myawsomedjangoproject.com.git</span> +</span></code></pre> +<p>Inside this folder we'll create what is called a <code>bare</code> repository. To do this +just run:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> init<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>bare</span></span> +</span></code></pre> +<p>If you run <code>ls</code> inside that folder you'll see a bunch of files and directories +(the same ones found inside the <code>.git</code> folder in normal repos). One of these +directories is called <code>hooks</code>.</p> +<p>Inside that folder, we'll need create a file called post-receive.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> Assuming you are inside /var/repos/yourproject.git</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> touch hooks/post-receive</span> +</span></code></pre> +<p>Now open it up with vi/vim</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> vim hooks/post-receive</span> +</span></code></pre> +<p>Hit <code>i</code> to switch to insert mode, and add the following to the file:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell">!/bin/bash</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">DEPLOYDIR</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">/home/username/sites/myawesomedjangoproject</span> <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> or whatever path you chose</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">GIT_WORK_TREE</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> checkout<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span></span> +</span></code></pre> +<p><strong>Please note that the first shebang line is important, it instructs git to use +bash instead of the default shell. Otherwise it won't activate our (soon to be +created) virtual environment</strong></p> +<p>exit vim by hitting <code>:wq</code> (which in vim lingo means write and quit)</p> +<p>What we've done here is set two variables. <code>DEPLOYDIR</code> is an alias for our +project path on the server, and <code>GIT_WORK_TREE</code> which is a special variable that +tells git to copy the code it receives inside of our <code>DEPLOYDIR</code>. This ensures +that we're always running the latest version of our code.</p> +<p>As you've probably noticed, this post-receive file looks very much like a shell +script. That's because it is (as explained above). It's executed every time you +push code to the repo.</p> +<p>The last thing we need to is make the script executable, so as soon as you're +back in the shell run:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">sudo</span></span><span class="z-meta z-function-call z-arguments z-shell"> chmod +x hooks/post-receive</span> +</span></code></pre> +<p>You can now exit the server and go back to your local machine.</p> +<h2 id="on-our-local-dev-environment">On our local dev environment</h2> +<p>Now that we've created our remote repository, we need to add it to our +project (I like to call mine <code>live</code>).</p> +<p>It takes one simple command:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> remote add live root@ip_address:/var/repos/myawesomedjangoproject.git</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> And if your server&#39;s ssh service listens on a different port :</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> remote add live ssh://root@ip_address:PORT/var/repos/myawesomedjangoproject.git</span> +</span><span class="z-source z-shell z-bash"> +</span></code></pre> +<p>To make sure it was added, you can print the list of available remotes by running:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> remote<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>v <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> v for verbose</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></span><span class="z-source z-shell z-bash"> +</span></code></pre> +<p>and that's it ! You can now make changes locally, commit and deploy +them live (or staging if it's a staging server) and see your changes instantly.</p> +<p>You can obviously still push to github/lab or bitbucket with +<code>git push origin &lt;branch&gt;</code> +like you normally would.</p> +<h2 id="bonus">Bonus</h2> +<p>As I mentioned in the first part, the post-receive hook is a shell script. Which +means you can use it to perform all kinds of tasks against your code, like +running front-end builds, installing dependencies, etc ...</p> +<p>Here's an example for a basic Django App:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell">!/bin/bash</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">DEPLOYDIR</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell">/home/username/site/myawesomedjangoproject</span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Starting code update <span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"><span class="z-variable z-other z-readwrite z-assignment z-shell">GIT_WORK_TREE</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> checkout<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>f</span></span> +</span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-echo z-shell">echo</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished code update <span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span> +</span><span class="z-source z-shell z-bash"> +</span><span class="z-source z-shell z-bash"><span class="z-keyword z-control z-conditional z-if z-shell">if</span> <span class="z-meta z-function-call z-arguments z-shell"><span class="z-support z-function z-double-brace z-begin z-shell">[[</span> <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>d</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span>/ENV_projectname<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>]]; then +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Cleaning virtualenv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; rm -rf ENV_projectname cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished creating virtualenv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">fi +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Creating virtualenv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; virtualenv <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>p</span> python3 ENV_projectname; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished creating virtualenv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Activating virtualEnv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; source ENV_projectname/bin/activate; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished activating virtualenv<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Pulling down pip dependencies<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; pip install <span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell">-</span>r</span> requirements.txt; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished pulling down pip dependencies<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Staring DB migration<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; python manage.py makemigrations; python manage.py migrate; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished DB migration <span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Pulling Node Dependencies<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; sudo npm install; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished Pulling Node Dependencies<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Building the Front end<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; sudo gulp build; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished building the Front end<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Collecting static assets<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">cd <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span><span class="z-meta z-group z-expansion z-parameter z-shell"><span class="z-punctuation z-definition z-variable z-shell">$</span><span class="z-variable z-other z-readwrite z-shell">DEPLOYDIR</span></span><span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>; python manage.py collectstatic --clear --no-input; cd - +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished collecting static assets<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Restarting App<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">sudo service myawesomedjangoapp restart; +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell">echo <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>[log] - Finished collecting static assets<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span> +</span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"> +</span></span></code></pre> +<p><em>I run my Django Apps as systemd services, if you don't you can just call python +manage.py runserver. If you want to know how to setup Django the way I do just +follow this very comprehensive tutorial over on <a rel="noopener nofollow noreferrer" target="_blank" href="https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-14-04">Digital Ocean</a></em></p> +<h2 id="conclusion">Conclusion</h2> +<p>I am fully aware that there are more sophisticated methods of deployment through +Docker, Travis (For continious integration) etc. But if you have a small app that +you want to ship and you already have an infrastructure, I've found this method +to be more than suitable.</p> +<p>Please report any missing info, mistake, error, typo. I'm on <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq"> twitter </a> if you +wanna chat.</p> + + + + Basic file operations in Python + Wed, 28 Mar 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/basic-file-operations-in-python/ + https://aaqa.dev/notes/basic-file-operations-in-python/ + <p>In this short (spoiler: it's actually quite lengthy) post, I will be going +through a list of very useful and handy methods in the <code>os</code> module +(which is part of the Python standard library) for handling files and +directories.</p> +<h2 id="1-create-a-directory">1. Create a directory</h2> +<p>This one is pretty straightforward. If you're comfortable with the linux shell, +you know that <code>mkdir</code> is the command to use to create directories. +Unsurprisingly, Python uses the same naming convention.</p> +<p>Example:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">mkdir</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">my_awesome_directory<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span></code></pre> +<p>The method takes a string its argument and will create the directory under the +file's parent folder. (For instance, if the path to the file calling os.mkdir() +is /home/username/Documents/app.py, the &quot;my_awesome_directory&quot; will be created +under /home/username/Documents)</p> +<h2 id="2-get-a-file-s-parent-directory">2. Get a file's parent directory.</h2> +<p>This is useful if you want to get the path to a file/folder that you know is +under the working file's parent directory. If this was confusing, here's an +example.</p> +<p>In the Linux shell (bash or otherwise), you can issue the command <code>pwd</code> +(which I believe stands for &quot;print working directory&quot;) to quickly print your +current location within a given session.</p> +<p>In Python you would achieve this like so:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getcwd</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span></code></pre> +<p>Notice I used all caps for the variable name. This is because it's usually a +constant. This variable isn't meant to be changed. +You can now use this variable to locate any file or folder within that +directory. Keep reading and I'll show you how.</p> +<h2 id="3-concatenate-paths">3. Concatenate paths</h2> +<p>This is something you'll find yourself doing a lot. Especially on large projects +that require configuration files and other such things. If you have experience +working with Django for example, they have a <code>settings.py</code> file littered with +calls to the os module. There are many benefits to this approach. Perhaps the +most obvious being is that if you ever decide to move your project to another +location, you don't want to keep modifying the path every time. Remember, +programming is all about being lazy.</p> +<p>So this is how you would do it:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Assuming this file is located at /home/username/myproject/app.py +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> and that you want to operate on a a file called config.cfg within the same +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> directory : +</span></span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getcwd</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> evaluates to /home/username/myproject +</span></span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_TEXT_FILE</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">join</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">config.cfg<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_TEXT_FILE</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> /home/username/myproject/config.cfg +</span></span></code></pre> +<p><em>An important note: os.path.join() merely concatenates the two paths together. +It doesn't check whether the path is valid. So be careful when using this +method. Also notice how the method call is to os.path.join() and not os.join()</em></p> +<h2 id="4-check-that-a-path-exists">4. Check that a path exists</h2> +<p>The other day I was working on a small web scraper for a side project of mine. +After the it was done fetching data, my script would save the results into a +pickle file (don't worry if you don't know what it is) that would be read by my +program, saving me the trouble of sitting there waiting to fetch the same info +over and over again each time I run the script.</p> +<p>The solution was to tell my script to check whether a specific file (let's call +it results.pkl) exists at a given path. If it does, the program continues and +if not, the program executes the crawler function.</p> +<p>This is clever because now I only have to fetch the data and if the file gets +deleted I know I can rely on the program to go and crawl the sites as expected.</p> +<p>And now for the example:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getcwd</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">RESULTS_FILE</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">join</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">results.pkl<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">crawl_data</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> scrapes a bunch of websites and saves the result in a file called +</span></span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> results.pkl under the current directory +</span></span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-pass z-python">pass</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-keyword z-operator z-logical z-python">not</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">exists</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">RESULTS_FILE</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">crawl_data</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-else z-python"><span class="z-keyword z-control z-conditional z-else z-python">else</span><span class="z-punctuation z-section z-block z-conditional z-else z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span># the file exists so we can open it and work with its content +</span></span></code></pre> +<p>The same thing can be done to check that the path exists AND that it's a +directory:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getcwd</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_DIRECTORY</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">join</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CURRENT_DIR</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">my_directory<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">exists</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_DIRECTORY</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-keyword z-operator z-logical z-python">and</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">isdir</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_DIRECTORY</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span># do something with the files inside the folder +</span></span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-else z-python"><span class="z-keyword z-control z-conditional z-else z-python">else</span><span class="z-punctuation z-section z-block z-conditional z-else z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">mkdir</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">MY_DIRECTORY</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span></code></pre> +<h2 id="5-list-files-within-a-given-directory">5. List files within a given directory</h2> +<p>Very useful when you want to read several files that are under the same +directory. +It can be done in two ways: conventional and pythonic. I'll show you both.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">file</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">listdir</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">/path/to/dir<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> do something with the filename (open it, copy it, move it, rename it...) +</span></span><span class="z-source z-python"> +</span></code></pre> +<p>or</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">filenames</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">file</span></span> <span class="z-meta z-expression z-generator z-python"><span class="z-keyword z-control z-loop z-for z-generator z-python">for</span> <span class="z-meta z-generic-name z-python">file</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">listdir</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">/path/to/dir<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span></code></pre> +<p>Guess which way is more pythonic!</p> +<h2 id="conclusion">Conclusion</h2> +<p>These have been the most useful file/directory functions for me in Python. I +really love the fact that the method names sound natural and are (for the most +part) similar to linux commands. What are your favourite file operation methods? Any tip or trick you want to share with me? Something I've missed? Ping on +twitter! I'm <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@aaqaishtyaq</a>.</p> +<p>If you found this article useful, please share it with your nerd friends/coworkers and spread the word!</p> + + + + Mind-blowing Python tips + Thu, 15 Feb 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/mind-blowing-python-tips/ + https://aaqa.dev/notes/mind-blowing-python-tips/ + <h2 id="0-loop-over-a-range-of-numbers">0 - Loop over a range of numbers</h2> +<p>Use <code>range</code> instead of <code>xrange</code>. +In python3, the former creates an iterator that produces the values one at +a time making it much more efficient and fast.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">nums</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">2</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">34</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">55</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">32</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">i</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">range</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">nums</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="1-looping-backwards">1 - Looping backwards</h2> +<p>.reversed use Just</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Case<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Armitage<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Maelcum<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">name</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">reversed</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="2-looping-over-a-list-and-its-indices">2 - Looping over a list and its indices</h2> +<p>To keep track of the index of each item in a collection, enumerate is your buddy.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Case<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Armitage<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Maelcum<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">index</span><span class="z-punctuation z-separator z-target-list z-python">,</span> <span class="z-meta z-generic-name z-python">name</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">enumerate</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">index</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="3-looping-over-two-lists-simultaneously">3 - Looping over two lists simultaneously</h2> +<p>Yeah you could use zip, but izip is faster, so use that instead.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">itertools</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">izip</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Case<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Armitage<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Maelcum<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ages</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">23</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">27</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">41</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">24</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">name</span><span class="z-punctuation z-separator z-target-list z-python">,</span> <span class="z-meta z-generic-name z-python">age</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">izip</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ages</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">age</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="4-looping-over-a-sorted-list">4 - Looping over a sorted list</h2> +<p>You can sort out the list first and then loop through it, or you could use +sorted.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Case<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Armitage<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Maelcum<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">name</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">sorted</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<p>And BAM, you're ... sorted.</p> +<h2 id="5-call-a-function-until-a-sentinel-value-is-returned">5 - Call a function until a sentinel value is returned</h2> +<p>To do that, use iter().</p> +<p>Bad example:</p> +<p>Loop over a file containing a list of names +until the loop returns an empty string, +in which case we break out of it.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-empty z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-constant z-language z-python">True</span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">file</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">read</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">32</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> = <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-break z-python">break</span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">append</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span></code></pre> +<p>Beautiful example:</p> +<p>In this case, we call a function (f.read) until it returns the sentinel value +passed as a second argument to iter. +That way we avoid having to make the unnecessary if check.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">name</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">iter</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">partial</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">f</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">read</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">32</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">name</span></span> +</span></code></pre> +<h2 id="6-looping-over-a-dictionary">6 - Looping over a dictionary</h2> +<p>The normal way to do it:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly Millions<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Age<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">27</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Occupation<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Professional Killer<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">key</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">key</span></span> +</span></code></pre> +<p>If you wish to mutate the data, prefer <code>dict.keys()</code>.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly Millions<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Age<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">27</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Occupation<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Professional Killer<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">key</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">keys</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> do the mutation +</span></span><span class="z-source z-python"> +</span></code></pre> +<h2 id="7-looping-over-a-dict-keys-and-values">7 - Looping over a dict keys AND values</h2> +<p>Don't do this:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly Millions<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Age<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">27</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Occupation<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Professional Killer<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">key</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">key</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<p>It's slow because we have to rehash the dictionary and do a lookup everytime.</p> +<p>Instead choose <code>iteritems()</code>:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly Millions<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Age<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">27</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Occupation<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Professional Killer<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">key</span><span class="z-punctuation z-separator z-target-list z-python">,</span> <span class="z-meta z-generic-name z-python">value</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">molly</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">iteritems</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-other z-print z-python">print</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">key</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">value</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="8-create-a-dict-out-of-two-lists">8 - Create a dict out of two lists</h2> +<p>Just instantiate a new dict with two zipped lists. Real magic.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">itertools</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">izip</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Case<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Molly<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Armitage<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Maelcum<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ages</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">23</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">27</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">41</span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">24</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">characters</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">dict</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">izip</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">names</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ages</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span></code></pre> +<h2 id="9-use-named-tuples-for-returning-multiple-values">9 - Use named tuples for returning multiple values</h2> +<p>Like in the case of an API response in Flask.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">collections</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">namedtuple</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">namedtuple</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">APIResponse<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">status_code<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">body<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">headers<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-annotation z-python"><span class="z-punctuation z-definition z-annotation z-python">@</span><span class="z-meta z-annotation z-function z-python"></span></span><span class="z-meta z-annotation z-function z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">app</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-annotation z-function z-python"><span class="z-meta z-generic-name z-python">route</span></span></span></span><span class="z-meta z-annotation z-function z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span></span><span class="z-meta z-annotation z-arguments z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">/users/1<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-annotation z-function z-python"><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-annotation z-variable z-python">:</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-exception z-try z-python"><span class="z-keyword z-control z-exception z-try z-python">try</span><span class="z-punctuation z-section z-block z-exception z-try z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">user</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">db</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getuserbyid</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-exception z-catch z-python"><span class="z-keyword z-control z-exception z-catch z-python">except</span><span class="z-punctuation z-section z-block z-exception z-catch z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Response</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">404</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">user</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">notfound</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span></span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">content-type<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">application/json<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-keyword z-control z-conditional z-else z-python">else</span>: +</span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-invalid z-illegal z-name z-python">return</span></span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Response</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">200</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">user</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">json</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span></span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">content-type<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&#39;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-single z-python">application/json<span class="z-punctuation z-definition z-string z-end z-python">&#39;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> +</span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> +</span></span></code></pre> +<h2 id="other">Other</h2> +<ul> +<li>Always clarify function calls by using keyword arguments</li> +</ul> +<p>If you learned something from this article, share it with your co-workers and +fellow hackers. If you notice any typo, error etc let me know on +<a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">twitter</a>.</p> + + + + Mind-blowing git tips for beginners + Mon, 05 Feb 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/mind-blowing-git-tips/ + https://aaqa.dev/notes/mind-blowing-git-tips/ + <p>As developers, we all (hopefully) use git. It's not extremely hard nor time-consuming to get started with it and you will surely thank your future self for taking the time to learn it. Ok so that was for the cringy intro.</p> +<p>Now let's get down to business: what I really want to share in this post is a list of tricks I've learned during the past 2 1/2 years of using git. Some of it might seem trivial to seasoned developers but if you're just getting started, stick with me because this might just blow your mind.</p> +<h2 id="the-difference-between-git-fetch-and-pull">The difference between git fetch and pull</h2> +<p><code>git fetch</code> only updates the tracking remote branches. If you actually want to +update the local repo, you need to merge the local branch with the remote +tracking branch using <code>git merge</code>.</p> +<p><strong>remote branches are prefixed with the name of the remote and a slash: +origin/branchname</strong></p> +<p>git pull on the other hand, will execute both commands for you so when you issue +a <code>git pull</code> in a branch it will fetch that branch from the remote repo <strong>and</strong> +merge it with yours.</p> +<p>To list all remote trackin branches, <code>git branch -r</code> is your friend.</p> +<h2 id="merge-conflicts">Merge conflicts</h2> +<p>At some point you'll inevitably have to deal with merge conflicts. Essentially what this means is that git noticed a file was modified on both branches and it does not know which version is correct. It leaves it up to you to decide which one you want to push.</p> +<p>if you open the file you will see a <code>HEAD</code> part prefixed with <code>&lt;&lt;&lt;&lt;</code> signs and a +second part containing the code on the branch you're trying to merge. Both +sections are separated with an equal sign.</p> +<p>The top HEAD section contains the version of the file as it is on your current +branch. The other shows you what the code looks like on the branch you're trying to merge from.</p> +<p>To resolve the conflict just delete the part you don't want (Including all the equal signs etc), save the file and commit it again.</p> +<p>Note that as a safety measure it's always good practice to do a <code>git pull</code> to +see if you don't have any remaining conflicts. If not you can just push your code and you're all set!</p> +<p>As for the tips:</p> +<p><code>git merge --abort</code> will clean up the current working directory and go back to +the last version before the merge (it pretty much nullfies the merge).</p> +<p><code>git merge --squash</code> is an interesting one as it will bring the changes you made in the feature branch and create a new commit on your current branch without +mixing the two histories.</p> +<h2 id="dealing-with-files">Dealing with files</h2> +<p>Now this one's tricky !</p> +<p>To remove a file from <strong>both</strong> the working directory and version control, you +can use <code>git rm &lt;filename&gt;</code> but if you already deleted a file in the cli like I (and almost everyone) usually do then just run <code>git rm &lt;path to the file&gt;</code> and it will stage it as deleted so you can commit.</p> +<p>But what if you ramrafed (<code>rm -rf</code>) a bunch of files in your working directory? +You don't really want to manually stage every file for deletion, do you? (DO YOU ?) So what do you do in that case? Well, you just run <code>git add -u</code> (u for updating the working tree) and all of your deleted files will be staged so you can commit and push.</p> +<p>&quot;What if a man accidentally tracked a file and man doesn't want it deleted +from the index ?&quot; I hear you ask with an exquisite South London accent.</p> +<p>Well, my G, just run:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> rm<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>cached</span> <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>name of the file<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> +</span></span></code></pre> +<p>and you're done. Efficience ting.</p> +<p>Something else that happens quite often is moving/renaming files (which is +essentially the same thing on Linux systems, as the path/address of the file +changed).</p> +<p>Here's an example: imagine you have a file called index.html that you +want to rename to home.html. you can use <code>git mv</code> which takes a source and a +destination:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> mv index.html home.html</span> +</span></code></pre> +<p>if you want to move <strong>and</strong> rename just run</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> mv index.html path/to/home.html</span> +</span></code></pre> +<p>However if you moved / renamed the file manually on the command line, running +<code>git status</code> will tell you that index.html was deleted and home.html is untracked.</p> +<p>To fix this you will have to run two commands</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> rm index.html <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> aka the file you moved/renamed</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></code></pre> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> add home.html <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> aka the file with the new name x path</span><span class="z-comment z-line z-number-sign z-shell"> +</span></span></span></code></pre> +<p>Running git status again will mark the file as renamed.</p> +<p>Note that this also works if you don't rename the file. For example, if you just +want to move index.html to src/index.html. The same command will apply +(<code>git mv index.html src/index.html</code>).</p> +<p>The other way of achieving this is:</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> add<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>A</span></span> +</span></code></pre> +<p>which will pick up the changes and automatically stage them for commit.</p> +<p>So there you go, I hope you learned something useful in this article. If you know +other mind-blowing tips and tricks for git, <a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/aaqaishtyaq">@ me on twitter</a>.</p> + + + + Don't be scared. Switch to vim. + Sat, 03 Feb 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/dont-be-scared-switch-to-vim/ + https://aaqa.dev/notes/dont-be-scared-switch-to-vim/ + <p>I'm currently sitting at the most boring meetup I've probably ever attended in +Delhi. It's about chatbots. I don't care about chatbots, I care about free +stickers and pizza. So I'll take this opportunity to open up about a subject +that's dear to my heart: vim.</p> +<p>I used to believe vim was exclusive to this superior race of developers who +gulp coffee like it's water and seem to only read HN and nothing else. (Hi, if +you're coming from HN). Architecture and Software design comes naturally to them, +they never run into bugs and they can recognize the most obscure of algorithms +at a glance (Shout out to Shashank, one of my mentors).</p> +<p>Shanky is a good, productive developer. Shany uses vim. I want to be like Shanky. I want +to use vim.</p> +<p>There are a million reasons why you should jump ship and join the cult. In the +next paragraphs, I will detail some of these reasons.</p> +<h2 id="it-s-not-that-hard">It's not (that) hard</h2> +<p>There's a learning curve to vim. But it's worth the +trouble. And if you're on Linux or MacOS, there's a built-in tool called +<code>vimtutor</code> (just fire it up from a terminal, I am not sure about Windows though) and +a wide variety of online tools to learn vim. Namely <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.openvim.com/">openvim</a>, +<a rel="noopener nofollow noreferrer" target="_blank" href="https://vim-adventures.com/">vim adventures</a>, and <a rel="noopener nofollow noreferrer" target="_blank" href="http://www.vimgenius.com/">vim genius</a>.</p> +<p>Personally, The way I learned was by using it on small, fun side projects of +mine during the weekends, mostly to become familiar with the new mental model. +And just like everything in life, shit takes time, and practice makes perfect. +So keep at it and you'll eventually come to your &quot;aha&quot; moment. +As you get more and more comfortable using vim, it will become harder and harder +to go back to a regular editor / IDE.</p> +<h2 id="it-s-fast-and-customisable">It's Fast and Customisable</h2> +<p>Because it runs on the terminal, you'll never have to wait 20 seconds to get +on with your work. (Atom anyone ?)</p> +<p>And if you like pretty things, there's a <a rel="noopener nofollow noreferrer" target="_blank" href="http://vimcolors.com/">large selection of colorschemes</a> +for you to choose from. On top of that, there's a plugin for just about anything +you might need. And if there isn't, you can program your own.</p> +<h2 id="ubiquity">Ubiquity</h2> +<p>Not really, but I wanted to place a complicated word to sound smart. +Seriously though, it's everywhere. On Mac OS, Windows and of course Linux/Unix. If +you work on remote servers you can quickly edit files on the fly without having +to use nano. (Don't use nano)</p> +<p>Say for example a coworker/friend is running into a bug, you come to help and +they're using an IDE you're not familiar with, well you can just access the files +from their terminal and start debugging right away.</p> +<p>Or if you're like me, and you spill water on your Macbook keyboard and it becomes +toast, you can spin up a VPS on Digital Ocean or AWS, and pick up where you +left off (almost) right away.</p> +<h2 id="bonus-some-of-my-favorite-plugins">Bonus: Some of my favorite plugins</h2> +<p>My color scheme of choice (at the time of writing) is <a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/danilo-augusto/vim-afterglow">afterglow</a>.</p> +<p>And here's a list of my favorite plugins:</p> +<ul> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/scrooloose/nerdtree"> Nerdtree </a> (A tree explorer much like the sidebar in traditional IDEs)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/vim-airline/vim-airline"> Airline </a> (A sleek, customizable status bar)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/tpope/vim-surround"> Surround </a> (Helpful tool that helps with &quot;surrounding&quot; words with brackets etc)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/kien/ctrlp.vim"> CtrlP </a> (A fuzzy finder for vim)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/SirVer/ultisnips"> UtilSnips </a> (Snippet utility for many languages)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/plasticboy/vim-markdown"> Vim Markdown</a> (Markdown syntax highlighting)</li> +<li><a rel="noopener nofollow noreferrer" target="_blank" href="https://github.com/junegunn/goyo.vim"> Goyo </a> (Allows for distraction-free editing)</li> +</ul> +<p>I'll end this article with a quote from a Chamillionaire:</p> +<blockquote> +<p>They see you vimmin', they hatin'. Patroling they tryna catch me coding dirty</p> +</blockquote> + + + + Algorithms in Python: Bubble Sort + Mon, 08 Jan 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/algorithms-in-python-bubble-sort/ + https://aaqa.dev/notes/algorithms-in-python-bubble-sort/ + <h2 id="some-theory">Some theory</h2> +<p>Bubble sort is another commonly known sorting algorithm. The idea here is to +scan a list of items (say integers) sequentially (from left to right) and +compare consecutive pairs of elements starting at index 0.</p> +<p>Example:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">92</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">11</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">45</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">2234</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">7</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">65</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> 92 is index 0 and the consecutive pairs are +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> (92,11), (11,45), (45,2234) and so on ... +</span></span></code></pre> +<p>At first we compare elements (list[0],list[1]) then (list[1],list[2]) then +(list[2],list[3]) and so on until the end of the list is reached.</p> +<p>When comparing we check if element i is greater than element i + 1, if they are +we just swap the two elements and move on to the next pair. If they are not this +means that the pair is already sorted, so we also move on to the next pair.</p> +<p>Example:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">92</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">11</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">45</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">2234</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">7</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">65</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Let&#39;s compare my_numbers[0] and my_numbers[1] +</span></span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">0</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-comparison z-python">&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">swap</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">0</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> [11, 92, 45, 2234, 0, 7, 65] +</span></span></code></pre> +<p>This process has to be repeated for however many items are on the list. So if +the list holds 9 items, it means we need to loop through it 9 times at most. +But what if our original list is partially sorted? We might not need 9 passes +through the list.</p> +<p>One way for us to know that the list is fully sorted is if we have made no +swaps during our pass. For that, we need a variable to keep track of how many +swaps were made during a pass.</p> +<p>Example:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">my_numbers</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-constant z-numeric z-integer z-decimal z-python">92</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">11</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">45</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">2234</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">7</span><span class="z-punctuation z-separator z-sequence z-python">,</span><span class="z-constant z-numeric z-integer z-decimal z-python">65</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (0,1) are compared and swapped. List is now 11,92,45,2234,0,7,65 +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (1,2) are compared and swapped. List now 11,45,92,2234,0,7,65 +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (2,3) are compared and not swapped. List remains the same. +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (3,4) are compared and swapped. List is now 11,45,92,0,2234,0,7,65 +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (4,5) are compared and swapped. List is now 11,45,92,0,7,2234,65 +</span></span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Elements (5,6) are compared and swapped. List is now 11,45,92,0,7,65,2234 +</span></span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> This represents one unique pass through the list. +</span></span></code></pre> +<p>Notice how after each pass the highest value number is pushed at len(list) - 1.</p> +<h2 id="some-code">Some code</h2> +<p>Let's look at how to implement Bubble Sort using Python:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">bubble_sort</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">some_list</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-keyword z-operator z-logical z-python">not</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">True</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">i</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">range</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">len</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-keyword z-operator z-arithmetic z-python">-</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-comparison z-python">&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> +</span></code></pre> +<p>This works right and it will sort any list you throw at it. However we can +slightly optimise it: We know that, after each pass the highest value element is +guaranteed to be sorted and placed at len(some_list) - 1. Because of this, for +each subsequent pass, we can stop comparing the last sorted item. instead of +comparing pairs that we know are already sorted. +This is what it looks like:</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">bubble_sort</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">some_list</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">len</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-keyword z-operator z-arithmetic z-python">-</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-keyword z-operator z-logical z-python">not</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">True</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">i</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">range</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-comparison z-python">&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span> <span class="z-keyword z-operator z-assignment z-augmented z-python">-=</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span> +</span></code></pre> +<p>After each pass through the loop, we know the right side of the list is sorted +so we decrement the value of last_sorted_item. What this means is that the 1st +pass will loop from 0 to len(some_list) -1, the second time, it will be from 0 +to len(some_list) - 2 and so on ...</p> +<h2 id="time-complexity">Time complexity</h2> +<p>The rate of growth of this algorithm is quadratic. Expressed as O(n^2) in +&quot;big-oh&quot; notation.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">bubble_sort</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">some_list</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> time here is constant +</span></span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">len</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-keyword z-operator z-arithmetic z-python">-</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-keyword z-operator z-logical z-python">not</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> We go through this first loop n times +</span></span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">True</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">i</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">range</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-numeric z-integer z-decimal z-python">0</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> we go through this loop n-1 times +</span></span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-comparison z-python">&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> execution here is constant +</span></span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-keyword z-operator z-arithmetic z-python">+</span><span class="z-constant z-numeric z-integer z-decimal z-python">1</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">some_list</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">is_sorted</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">False</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">last_sorted_item</span></span> <span class="z-keyword z-operator z-assignment z-augmented z-python">-=</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> constant time +</span></span></code></pre> +<p>It's O(n^2) because, for each pass through the loop n times, we loop n times +through the consecutive pairs. It's not a very efficient algorithm when used on large samples of data. It should only be used if you have a +specific case on a small data set.</p> +<p>Next in the series is QuickSort, another interesting and more efficient sorting +algorithm. As always, if you have questions, comments or if you spotted a typo +or a mistake, please feel free to let me know on Twitter, I'm +<a rel="noopener nofollow noreferrer" target="_blank" href="https://twitter.com/zabanaa">@aaqaishtyaq</a> and always happy to help!</p> + + + + Algorithms in Python: Quick Sort + Mon, 08 Jan 2018 00:00:00 +0000 + Unknown + https://aaqa.dev/notes/algorithms-in-python-quick-sort/ + https://aaqa.dev/notes/algorithms-in-python-quick-sort/ + <h2 id="theory">Theory</h2> +<p>Quicksort is a &quot;divide and conquer&quot; type of algorithm. The good thing about +it is that the worst case can almost always be avoided by using what is called a +randomized version of quicksort (more on that later).</p> +<p>The idea of Quicksort is to take an unsorted list and select an element (on that +list) called a &quot;pivot&quot;. Then the list is rearranged such that all elements greater +(in value) than the pivot are placed to its right, and all elements lesser (in +value) are placed to its left.</p> +<p>This process is called partitioning. At this stage in the execution of the +algorithm, the order of the elements doesn't matter so long as the +lesser/bigger values are placed on the correct side of the pivot.</p> +<p>Partitioning will produce two sublists with the pivot as a separator ( +this is because the pivot will be at its natural place after the first pass aka +sorted). The problem then becomes sorting these two sublists.</p> +<p><em>Note: Partitioning does not require creating copies of the +list, we work on it directly as long as we keep track of the start and end +indices of each sublist.</em></p> +<p>To sort the two sublists, we can apply the same logic as above (choosing a +pivot, and sorting the two resulting sublists) because QuickSort is a recursive +algorithm.</p> +<p>When a sublist only contains a single element, it's already sorted so we can +stop the recursion at this point, it's our exit condition.</p> +<p><strong>Note on choosing a pivot</strong></p> +<p>Some people use the last item of the list, and some people use the median of the +first, last, and medium elements but the most common way is to choose a random +pivot to ensure <code>n log n</code> execution.</p> +<h2 id="some-code">Some Code</h2> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">swap_values</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">lst</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">val1</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">val2</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">lst</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">val1</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">lst</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">val2</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">lst</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">val2</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">lst</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">val1</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">quicksort</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">array</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">start</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">end</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span> <span class="z-keyword z-operator z-comparison z-python">&lt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">partition</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> +</span></span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">quicksort</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> <span class="z-keyword z-operator z-arithmetic z-python">-</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">quicksort</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">partition</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">array</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">start</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">end</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">i</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">range</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-comparison z-python">&lt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-constant z-other z-placeholder z-python"><span class="z-punctuation z-definition z-placeholder z-begin z-python">{</span><span class="z-punctuation z-definition z-placeholder z-end z-python">}</span></span> is less than <span class="z-constant z-other z-placeholder z-python"><span class="z-punctuation z-definition z-placeholder z-begin z-python">{</span><span class="z-punctuation z-definition z-placeholder z-end z-python">}</span></span><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">format</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">swap_values</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">i</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> <span class="z-keyword z-operator z-assignment z-augmented z-python">+=</span> <span class="z-constant z-numeric z-integer z-decimal z-python">1</span> +</span><span class="z-source z-python"> +</span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span> +</span></code></pre> +<p>A randomized version of Quicksort would look similar to what's above except that +we must randomize the selection of our pivot.</p> +<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">random</span></span></span> +</span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> ... +</span></span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">partition</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">array</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">start</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">end</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span> <span class="z-keyword z-operator z-comparison z-python">&lt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">random</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">randint</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">pivot</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span>, <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> +</span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">partition_index</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">array</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">start</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">end</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> +</span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> ... +</span></span></code></pre> +<p>Here, we set the pivot to a random integer in the range between <code>start</code> +and <code>end</code>. Then, we swap the value at that index with the value at array[end]. +If you run the code successively, you'll notice that the pivot is +different every time. It's a nice optimization that can save some time.</p> +<h2 id="time-complexity">Time Complexity</h2> +<p>It's one of the most efficient sorting algorithm. In fact, most sorting +functions that come packaged in many language's standard libraries use an +implementation of QuickSort.</p> +<p>The order of growth for QuickSort in the worst case is quadratic O(n^2). The +average case, however, which is the most common scenario, has a complexity of +O(n log n).</p> +<p>QuickSort works best when used on large sets of data because of its recursive nature.</p> + + + + diff --git a/sitemap.xml b/sitemap.xml new file mode 100644 index 0000000..fe55966 --- /dev/null +++ b/sitemap.xml @@ -0,0 +1,108 @@ + + + + https://aaqa.dev/ + + + https://aaqa.dev/contact/ + + + https://aaqa.dev/notes/ + + + https://aaqa.dev/notes/algorithms-in-python-bubble-sort/ + 2018-01-08 + + + https://aaqa.dev/notes/algorithms-in-python-quick-sort/ + 2018-01-08 + + + https://aaqa.dev/notes/basic-file-operations-in-python/ + 2018-03-28 + + + https://aaqa.dev/notes/build-deploy-react-app-with-nginx/ + 2018-06-02 + + + https://aaqa.dev/notes/deploy-django-application-git/ + 2018-05-07 + + + https://aaqa.dev/notes/dont-be-scared-switch-to-vim/ + 2018-02-03 + + + https://aaqa.dev/notes/firecracker-dev-machine-setup/ + 2023-01-26 + + + https://aaqa.dev/notes/git-exclude-files/ + 2022-07-29 + + + https://aaqa.dev/notes/hugo-open-markdown-new-tab/ + 2020-06-05 + + + https://aaqa.dev/notes/linux-container-networking/ + 2023-01-26 + + + https://aaqa.dev/notes/mind-blowing-git-tips/ + 2018-02-05 + + + https://aaqa.dev/notes/mind-blowing-python-tips/ + 2018-02-15 + + + https://aaqa.dev/notes/recipe-of-great-rest-api/ + 2018-12-30 + + + https://aaqa.dev/notes/scrapy-splash-setup/ + 2018-09-01 + + + https://aaqa.dev/notes/the-ultimate-postgresql-cheatsheet/ + 2018-05-09 + + + https://aaqa.dev/notes/the-ultimate-setup-for-remote-development/ + 2018-05-08 + + + https://aaqa.dev/reading/ + + + https://aaqa.dev/reading/1984-george-orwell/ + 2024-02-03 + + + https://aaqa.dev/reading/animal-farm-george-orwell/ + 2024-03-07 + + + https://aaqa.dev/reading/the-idiot-fyodor-dostoevsky/ + 2024-01-01 + + + https://aaqa.dev/tils/ + + + https://aaqa.dev/tils/move-git-untracked-files-to-temp-dir/ + 2024-03-19 + + + https://aaqa.dev/travel/ + + + https://aaqa.dev/travel/fairy-forest-pulga-kalga/ + 2024-03-05 + + + https://aaqa.dev/uses/ + + diff --git a/skills.json b/skills.json new file mode 100644 index 0000000..a2869d8 --- /dev/null +++ b/skills.json @@ -0,0 +1,48 @@ +{ + "languages": [ + "Go", + "Rust", + "Ruby", + "Typescript", + "Python", + "Bash", + "HTML5", + "CSS3", + "Sass", + "SQL", + "Markdown" + ], + "tools": [ + "Vim / VS Code", + "Tmux", + "Z Shell", + "Firecracker", + "Buildkit", + "Containerd", + "Docker", + "Virtualbox", + "Vagrant", + "Ansible", + "Gulp" + ], + "frameworks": [ + "OpenTelemetry", + "Ruby on Rails", + "Scrapy", + "Django", + "Flask", + "Node.js", + "React.js", + "Mocha / Chai" + ], + "other": [ + "Message Brokers", + "Kubernetes", + "Nomad", + "Basic UNIX Admin", + "PostgreSQL", + "DyanamoDB", + "gRPC / REST ", + "AWS / GCP / OCI" + ] +} diff --git a/theme.css b/theme.css new file mode 100644 index 0000000..abfe704 --- /dev/null +++ b/theme.css @@ -0,0 +1 @@ +@import url("hl-dark.css") (prefers-color-scheme: dark);@import url("hl-light.css") (prefers-color-scheme: light);:root{--bg_h: #ffffff;--bg: #ffffff;--bg_s: #ffffff;--bg1: #ffffff;--bg2: #ffffff;--bg3: #ffffff;--bg4: #ffffff;--fg: #1d1f21;--fg1: #1d1f21;--fg2: #1d1f21;--fg3: #1d1f21;--fg4: #1d1f21;--red: #cc6666;--green: #b5bd68;--yellow: #e6c547;--blue: #81a2be;--purple: #b294bb;--aqua: #70c0ba;--orange: #DE935F;--gray: #373b41;--red-dim: #ff3334;--green-dim: #9ec400;--yellow-dim: #f0c674;--blue-dim: #81a2be;--purple-dim: #b77ee0;--aqua-dim: #54ced6;--orange-dim: #E78C45;--gray-dim: #282a2e;--bgcolor: #ffffff;--fontcolor: #000000;--linkcolor: #b3004d;--hovercolor: #ffa8ce;--precolor: #1d1f21;--prebgcolor: #ffffff;--white: #ffffff;--visitedfg: #1d1f21;--visitedbord: #1d1f21;--visitedhov: #ffffff;--visitedhovbg: #1d1f21;--title: #1d1f21;--selection-fg: #ffb5d2}@media (prefers-color-scheme: dark){:root{--bg_h: #000000;--bg: #000000;--bg_s: #000000;--bg1: #000000;--bg2: #000000;--bg3: #000000;--bg4: #000000;--fg: #eaeaea;--fg1: #eaeaea;--fg2: #eaeaea;--fg3: #eaeaea;--fg4: #eaeaea;--red: #d54e53;--green: #b9ca4a;--yellow: #e6c547;--blue: #7aa6da;--purple: #c397d8;--aqua: #70c0ba;--gray: #424242;--orange: #fe8019;--red-dim: #ff3334;--green-dim: #9ec400;--yellow-dim: #e7c547;--blue-dim: #7aa6da;--purple-dim: #b77ee0;--aqua-dim: #54ced6;--gray-dim: #2a2a2a;--orange-dim: #E78C45;--bgcolor: #1d2021;--fontcolor: #eaeaea;--linkcolor: #ff5d9b;--hovercolor: #ffa8ce;--precolor: #eaeaea;--prebgcolor: #383838;--green: #b9ca4a;--yellow: #e6c547;--white: #eaeaea;--visitedfg: #c1b7a9;--visitedbord: #a89984;--visitedhov: #fdf4c1;--visitedhovbg: #282828;--title: #eaeaea;--selection-fg: #3d0428}}@font-face{font-family:"Iosevka";src:url("/fonts/iosevka-regular.woff2") format("woff2");font-display:swap}@font-face{font-family:"Iosevka";src:url("/fonts/iosevka-italic.woff2") format("woff2");font-style:italic;font-display:swap}@font-face{font-family:"Iosevka";src:url("/fonts/iosevka-bold.woff2") format("woff2");font-weight:bold;font-display:swap}@font-face{font-family:"Iosevka";src:url("/fonts/iosevka-bolditalic.woff2") format("woff2");font-weight:bold;font-style:italic;font-display:swap}body{font:100% "Iosevka",sans-serif;background:var(--bg_h);color:var(--fontcolor)}body{display:grid;grid-template-columns:auto;grid-template-areas:"header" "main" "footer"}header{padding:8px 8px 0 8px;grid-area:header;display:flex;justify-content:space-between;align-items:center}main{padding:8px;grid-area:main}p{text-align:justify}footer{color:var(--fg);padding:0 8px 8px 8px;grid-area:footer}.footer{display:flex;justify-content:space-between;align-items:center}nav{list-style-type:none}.nav{display:inline-block;padding:0em .5em 0em 0em}ul{list-style:none inside none;list-style-type:"- ";padding:0}li::marker{color:var(--aqua)}ul ul{padding-left:1em}ul ul ul{padding-left:2em}a{font-size:none;color:var(--linkcolor);text-decoration:none;border-color:var(--linkcolor);border-bottom:.0625em solid}.pgp{border-bottom:.0625em dotted}a:hover{background-color:var(--hovercolor);color:var(--white)}h1{font-size:24px;margin:5px 0;font-weight:bold;color:var(--blue)}h2{font-size:22px;font-weight:bold;color:var(--blue)}h3{font-size:20px;font-weight:bold;color:var(--blue)}h4{font-size:18px;font-weight:bold;color:var(--blue)}h5{font-size:16px;font-weight:bold;color:var(--blue)}h6{font-size:14px;font-weight:bold;color:var(--blue)}h1 a,h2 a,h3 a,h4 a,h5 a,h6 a{color:var(--yellow-dim);border-bottom-color:var(--yellow-dim)}pre{padding:1em;margin:1em;overflow-x:auto;color:var(--precolor)}code{font:16px "Iosevka",monospace}p>code{font-weight:bold;color:var(--aqua)}.logo{font-size:26px;font-weight:bold}.date{color:var(--fg4);font-size:.875em}.reading-time{color:var(--fg4);font-size:.875em}.page-meta{display:flex;justify-content:space-between;align-items:center}.border{color:--fg1;margin:8px 0 8px 0;border-top:2px dashed}.notfound{text-align:center}.icon{display:inline-block;vertical-align:middle;width:16px;height:16px}.content{margin:0 auto;width:860px}.skills .container{display:flex;flex-wrap:wrap}.skills .container+mobile{flex-direction:column;flex-wrap:nowrap}.skills .skill-group{margin-bottom:1em;flex-basis:50%}.skills .skill-group+mobile{margin-bottom:1em}.skills .skill-group+mobile:last-child{margin-bottom:0}.home-list{list-style-type:""}@media (max-width: 1200px){.content{margin-left:1em;margin-right:1em;width:auto;max-width:1200px;min-width:200px}}figcaption{padding-top:.5em;text-align:center}li{padding-bottom:1em}::selection{background:var(--selection-fg)}a::selection{background:var(--selection-fg)}.center{width:50%}a:link{color:var(--linkcolor)}a:visited{color:var(--linkcolor)}a:active{color:var(--linkcolor)}figure{margin:auto}img{display:block;max-width:100%;height:auto;margin:auto}.image-block{filter:grayscale(0);transition:all .4s}.image-block:hover{filter:grayscale(95%)}.head-text{padding:0em 3em 0em 0em}.head-column{display:flex;flex-flow:row;align-items:stretch}@media (max-width: 1200px){.head-column{display:flex;align-items:stretch;flex-direction:column;flex-wrap:wrap;flex-flow:column}header{padding:8px 8px 0 8px;grid-area:header;display:flex;justify-content:center;flex-direction:column}.head-text{padding:0em}} \ No newline at end of file diff --git a/tils/index.html b/tils/index.html new file mode 100644 index 0000000..46cec70 --- /dev/null +++ b/tils/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Today I Learned

A collection of bite-sized notes on various topics that I’ve learned over time.

\ No newline at end of file diff --git a/tils/move-git-untracked-files-to-temp-dir/index.html b/tils/move-git-untracked-files-to-temp-dir/index.html new file mode 100644 index 0000000..b4ffdf0 --- /dev/null +++ b/tils/move-git-untracked-files-to-temp-dir/index.html @@ -0,0 +1,2 @@ +Aaqa Ishtyaq

Move untracked git files to another directory

123 words, 1 minutes to read

I had a lot of files in a git repository, which I didn't want to add in git, So to remove all of them to a temporary directory I used a collection of Unix commands.

git status --porcelain | awk '{print $2}' | xargs mv -t ../scratch
+
\ No newline at end of file diff --git a/tils/rss.xml b/tils/rss.xml new file mode 100644 index 0000000..68d4075 --- /dev/null +++ b/tils/rss.xml @@ -0,0 +1,24 @@ + + + + Aaqa Ishtyaq - Today I Learned + https://aaqa.dev/tils/ + Aaqa Ishtyaq, Ramblings on programming, tech and life. + Zola + en + + Tue, 19 Mar 2024 00:00:00 +0000 + + Move untracked git files to another directory + Tue, 19 Mar 2024 00:00:00 +0000 + Unknown + https://aaqa.dev/tils/move-git-untracked-files-to-temp-dir/ + https://aaqa.dev/tils/move-git-untracked-files-to-temp-dir/ + <p>I had a lot of files in a git repository, which I didn't want to add in <code>git</code>, So to remove all of them +to a temporary directory I used a collection of Unix commands.</p> +<pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> status<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>porcelain</span></span> <span class="z-keyword z-operator z-logical z-pipe z-shell">|</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">awk</span></span><span class="z-meta z-function-call z-arguments z-shell"> <span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>{print $2}<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span></span> <span class="z-keyword z-operator z-logical z-pipe z-shell">|</span> <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">xargs</span></span><span class="z-meta z-function-call z-arguments z-shell"> mv<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>t</span> ../scratch</span> +</span></code></pre> + + + + diff --git a/travel/fairy-forest-pulga-kalga/index.html b/travel/fairy-forest-pulga-kalga/index.html new file mode 100644 index 0000000..fd6d851 --- /dev/null +++ b/travel/fairy-forest-pulga-kalga/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Pulga, Kalga - Himachal Pradesh 2024

122 words, 1 minutes to read

In the heart of Himachal's Parvati Valley lie its hidden gems: Pulga and Kalga. Picture-perfect villages embraced by towering mountains, where every corner tells a story of nature's magnificence. Here, snow-capped peaks stand tall, kissing the sky, while icy waterfalls dance down the slopes, painting the landscape with their pure, untamed beauty. Despite the biting cold, every moment feels like a warm embrace from Mother Nature herself. It's not just a trip; it's a journey of awe, wonder, and connection with the raw essence of the mountains.

\ No newline at end of file diff --git a/travel/index.html b/travel/index.html new file mode 100644 index 0000000..352350e --- /dev/null +++ b/travel/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Travel

\ No newline at end of file diff --git a/uses/index.html b/uses/index.html new file mode 100644 index 0000000..97afbb0 --- /dev/null +++ b/uses/index.html @@ -0,0 +1 @@ +Aaqa Ishtyaq

Uses

Hardware

  • Apple Macbook Pro 16 (2023): Work issued Macbook Pro. M3 Max Pro paired with 36GB RAM.
  • Apple Macbook Pro 13 (2018): Personal MBP with 8GB RAM. A comfy size for travel and Netflix.
  • Keychron K2 Keyboard: Nice mechanical keyboard with Mac layout and Cherry MX Brown switches.
  • Apple iPhone 15 Pro: Switched to 15 Pro, Wow I missed USB-C after using 11 for around 4 years.
  • Apple Airpods Pro: Best for listening to music while working, relaxing and whatever the mood is.
  • Bose QC 35II: Use it for work meetings. All day comfort with excellent noise cancellation.
  • Dell 24" 1080p Monitor: I opted for 24" instead of 27" with 1080p. Does the job with decent sharpness.
  • Raspberry Pi 4 8GB: Runs personal git, nextcloud and bunch of self-hosted services.

Software

  • nix: Most of my machines runs nix in some capacity. macOS machines uses nix package manager.
  • nixOS: Raspberry Pi and Oracle VM uses nixOS. I love the declarative nature to setup the machine.
  • tmux: This is the first program that I run in the terminal. Paired with Alacritty for my pty needs.
  • zsh: Primary shell of choice. Like the customisation options with it alongside the POSIX compatibility.
  • Alacritty: Terminal emulator of choice. Using it since 2019 and not had a single issue.
  • Code Font: IBM Plex Mono for Editor and Term.

The dotfiles for all software mentioned, and more. This is my nix based dotfiles (home_ops).

\ No newline at end of file