-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
905 lines (866 loc) · 34.8 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<title>Hacker Tools II</title>
<link rel="stylesheet" href="css/reveal.css">
<link rel="stylesheet" href="css/theme/black.css">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="lib/css/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'css/print/pdf.css' : 'css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<style>
#logo-img {
background: white;
border: 10px solid #fff;
border-radius: 10px;
}
.reveal .slides .font75 {
font-size: 75%;
}
.reveal .slides .font45 {
font-size: 45%;
}
.reveal .slides .underline {
text-decoration: underline;
}
.reveal .slides .mt0 {
margin-top: 0;
}
.reveal .slides .mb0 {
margin-bottom: 0;
}
</style>
</head>
<body>
<div class="reveal">
<div class="slides">
<section class="center">
<h1>Hacker Tools II</h1>
<img id="logo-img" src="nushackers_logo.svg">
</section>
<section>
<h2>Brief overview</h2>
<ul>
<li>Browser and web tips and tricks</li>
<li>Security and privacy</li>
<li>Remoting</li>
<li>System and application introspection and debugging</li>
<li>Network debugging</li>
</ul>
<p class="font75">This workshop is meant as an exposure to commonly-used tools; there is no intended link between everything mentioned today, although some stories will be thrown in here and there to provide some rationale and context.</p>
</section>
<section class="center">
<h1>Browser and web</h1>
</section>
<section>
<h2>Keyboard shortcuts</h2>
<ul>
<li><kbd>C-t</kbd>: new tab</li>
<li><kbd>C-T</kbd>: reopen closed tab</li>
<li><kbd>C-l</kbd>/<kbd>M-d</kbd>/<kbd>F6</kbd>: jump to address bar</li>
<li><kbd>C-f</kbd>: search in page</li>
<li><kbd>C-w</kbd>/<kbd>C-F4</kbd>/<kbd>Middle-click</kbd>: close tab</li>
<li><kbd>F12</kbd>: developer tools</li>
</ul>
<p>More: <a href="https://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly">Firefox</a>; <a href="https://support.google.com/chrome/answer/157179?hl=en">Chrome</a></p>
</section>
<section>
<h2>Google search operators</h2>
<ul>
<li><code>"X"</code>: must appear and must appear exactly</li>
<li><code>site:xyz.com</code>: results from xyz.com only</li>
<li><code>filetype:xyz</code>: xyz filetype only</li>
<li><code>define:xyz</code>: define xyz</li>
</ul>
<p>More: <a href="https://support.google.com/websearch/answer/2466433">Google docs</a>; <a href="https://ahrefs.com/blog/google-advanced-search-operators/">External docs</a></p>
</section>
<section>
<section>
<h2>Firefox bookmark keywords</h2>
<p>No real equivalent exists in Chrome/ium.</p>
<img src="img/ff0.png">
<img src="img/ff1.png">
</section>
</section>
<section>
<h2>Browser extensions</h2>
<ul>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/">uBlock Origin</a><a href="https://chrome.google.com/webstore/detail/ublock-origin/cjpalhdlnbpafiamejdnhcphjbkeiagm">[C]</a>: adblock</li>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/umatrix/">uMatrix</a><a href="https://chrome.google.com/webstore/detail/umatrix/ogfcmafjalglgifnmanfmnieipoejdcf?hl=en">[C]</a>: adblock/matrix filtering</li>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/styl-us/">Stylus</a><a href="https://chrome.google.com/webstore/detail/stylus/clngdbkpkpeebahjckkjfobafhncgmne">[C]</a>: <a href="https://userstyles.org/">user styles</a> for websites</li>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/violentmonkey/">Violentmonkey</a><a href="https://chrome.google.com/webstore/detail/violentmonkey/jinjaccalgkegednnccohejagnlnfdag?hl=en">[C]</a>/<a href="https://addons.mozilla.org/en-US/firefox/addon/tampermonkey/">Tampermonkey</a><a href="https://chrome.google.com/webstore/detail/tampermonkey/dhdgffkkebhmkfjojejmpbldmpobfkfo">[C]</a>: user scripts from <a href="https://openuserjs.org/">OpenUserJS</a>, <a href="https://greasyfork.org/en/scripts">GreasyFork</a></li>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/tree-style-tab/">Tree Style Tabs</a> (FF only)</li>
<li><a href="https://addons.mozilla.org/en-US/firefox/addon/google-search-link-fix/">Google link search fix</a> (FF only)</li>
</ul>
</section>
<section>
<h2>Misc. useful web APIs</h2>
<ul>
<li><a href="https://api.telegram.org/">Telegram bot API</a>: e.g. as simple push notifications</li>
<li><a href="https://ifttt.com/">IFTTT</a>: service that glues many services together</li>
</ul>
</section>
<section class="center">
<h1>Security and privacy</h1>
</section>
<section>
<h2>Password managers</h2>
<ul>
<li><a href="https://keepass.info/">KeePass</a>: open-source, offline, .NET</li>
<li><a href="https://keepassxc.org/">KeePassXC</a>: open-source, C++ port of above (works better for Linux/Mac)</li>
<li><a href="https://www.passwordstore.org/">pass</a>: open-source, CLI, GPG-based</li>
<li><a href="https://bitwarden.com/">Bitwarden</a>: open-source, self-hostable cloud based <span>(also see <a href="https://github.com/dani-garcia/bitwarden_rs">bitwarden_rs</a>)</span></li>
<li><a href="https://1password.com/">1Password</a>: commercial 👎, cloud-based</li>
<li><a href="https://www.lastpass.com">LastPass</a>: commercial 👎, cloud-based</li>
</ul>
</section>
<section>
<h2>Full-disk encryption</h2>
<p>Protect your data at rest</p>
<ul>
<li>Windows: BitLocker (only available on Pro)</li>
<li>Linux: LUKS/dm-crypt</li>
<li>macOS: FileVault</li>
<li>Cross-platform: <a href="https://www.veracrypt.fr/en/Home.html">VeraCrypt</a></li>
</ul>
</section>
<section>
<h2>File-based encryption</h2>
<p>Protect individual files</p>
<ul>
<li>Linux: eCryptFS, <a href="https://www.cryfs.org/tutorial">CryFS</a></li>
<li>Cross-platform: <a href="https://www.veracrypt.fr/en/Home.html">VeraCrypt</a></li>
<li>7-Zip</li>
</ul>
</section>
<section>
<h2>VPN services (paid)</h2>
<p>Protect your online privacy</p>
<ul>
<li><a href="https://www.tunnelbear.com/">Tunnelbear</a> (based in Canada)</li>
<li><a href="https://www.privateinternetaccess.com/">Private Internet Access</a> (based in USA)</li>
<li><a href="https://nordvpn.com/">NordVPN</a> (based in Panama)</li>
<li><a href="https://protonvpn.com/">ProtonVPN</a> (based in Switzerland)</li>
</ul>
<p><a href="https://thatoneprivacysite.net/vpn-comparison-chart/">Comparison table</a></p>
</section>
<section>
<h2>VPN software</h2>
<p>Run your own VPN</p>
<ul>
<li><a href="https://openvpn.net/community-downloads/">OpenVPN</a></li>
<li><a href="https://www.wireguard.com/">WireGuard</a></li>
</ul>
<p>Setting up a server is out of scope. Here are some guides: <a href="https://wiki.archlinux.org/index.php/OpenVPN">OpenVPN</a>, <a href="https://wiki.archlinux.org/index.php/WireGuard">WireGuard</a></p>
<p>WireGuard is a very modern (but very new) VPN. If you are setting up a new VPN, and it meets your needs, use it!</p>
</section>
<section>
<h2>Messengers</h2>
<ul>
<li><a href="https://signal.org/">Signal</a>: Trusted by security community</li>
<li><a href="https://about.riot.im/">Riot.im/Matrix</a>: Also trusted; federated</li>
<li><a href="https://www.whatsapp.com/">WhatsApp</a>: Supposedly uses Signal protocol</li>
<li><a href="https://telegram.org/">Telegram</a>: Questionable encryption</li>
</ul>
<p>There are many other messengers but most offer no privacy at all.</p>
</section>
<section class="center">
<p>See <a href="https://privacytools.io">privacytools.io</a> for more privacy-focused alternatives to common software.</p>
<p>You could consider reducing your dependency on Google, for example, by moving to a different email provider.</p>
</section>
<section class="center">
<h1>Remoting</h1>
<p>SSH and Windows Remote Desktop</p>
</section>
<section>
<h2>SSH on Windows</h2>
<p>Did you know? Microsoft ported OpenSSH to Windows.</p>
<p>It is included by default from Windows 10 1803 and up. Try it! If not, <a href="https://superuser.com/a/1348488">see here</a>.</p>
<img src="img/win-ssh.png" style="max-height: 20rem;">
</section>
<section>
<section>
<h2>SSH keys</h2>
<p>A more convenient and secure alternative to passwords.</p>
<p>Many servers nowadays are configured to reject allow password authentication by SSH.</p>
<p>If you use Git on the command line with GitHub/GitLab/etc., you already have an SSH key!</p>
</section>
<section>
<h2>SSH key generation</h2>
<ol>
<li>
<pre>ssh-keygen -t ed25519</pre>
(or <code>-t rsa -b 4096</code> if you need RSA)
</li>
<li>Default location is OK</li>
<li>Enter a passphrase</li>
</ol>
<p>Your private key is in <code>.ssh/id_ed25519</code> <strong>(KEEP THIS SAFE)</strong>, and public key <code>.ssh/id_ed25519.pub</code></p>
</section>
<section>
<h2>SSH key generation</h2>
<ul>
<li>On Windows >=10 1803, you can just use ssh-keygen.</li>
<li>Alternatively, use <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY/PuTTYgen</a>.</li>
<img src="img/puttygen0.png" style="max-height: 25rem;">
<img src="img/puttygen1.png" style="max-height: 25rem;">
</ul>
</section>
</section>
<section>
<h2>SSH key usage</h2>
<p>Just <code>ssh-copy-id username@server</code>.</p>
<p>Alternatively, manually (on the remote): (if you use PuTTY/etc.)</p>
<pre>mkdir .ssh
echo "<insert id_ed25519.pub here>" >> .ssh/authorized_keys
chmod 700 .ssh
chmod 600 .ssh/authorized_keys</pre>
<p class="font75">(insert your public key into <code>.ssh/authorized_keys</code> on the remote and set the permissions correctly)</p>
<p>(For SoC students: try this with Sunfire!)</p>
</section>
<section>
<section>
<h2>SSH key agents</h2>
<p>A daemon that runs in the background and keeps your keys unlocked for use, so you don't have to keep keying in passwords.</p>
<p>Try:</p>
<pre>eval $(ssh-agent)
ssh-add
ssh ...</pre>
<p>Typically we <a href="https://wiki.archlinux.org/index.php/SSH_keys#SSH_agents">configure our <code>profile</code>/<code>bashrc</code>/etc.</a> to automatically start an agent if needed and include the environment variables in each shell.</p>
</section>
<section>
<h2>SSH key agents</h2>
<p>On Windows with PuTTY, use <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">Pageant</a></p>
<img src="img/pageant0.png" style="max-height: 25rem;">
</section>
</section>
<section>
<section class="font75">
<h2>SSH port forwarding</h2>
<p>E.g. if you need to connect to a local service on the remote, or if you need the remote to connect to a service on your computer.</p>
<ul>
<li><code>ssh -L 12345:localhost:56789</code>: 12345 locally -> localhost:56789 from the remote</li>
<li><code>ssh -R 12345:localhost:56789</code>: 12345 on remote -> localhost:56789 from local machine</li>
</ul>
<p>You can connect to other hosts, of course.</p>
<ul>
<li><code>ssh -L 12345:google.com:443</code>: you can open https://localhost:12345, and it will forward to Google but from the remote</li>
</ul>
<p>This is how SSH can be used as a proxy of sorts.</p>
</section>
<section>
<h2>SSH SOCKS proxy</h2>
<p>SSH can act as a SOCKS proxy and let any application forward its connections.</p>
<ul>
<li><code>ssh -D 12345</code>: open a SOCKS proxy locally, forward to remote</li>
<li><code>ssh -R 12345</code>: open a SOCKS proxy on the remote, forward to local</li>
</ul>
<p>You could even use this with your browser, if you needed to e.g. quickly bypass a firewall.</p>
</section>
<section>
<h2>Windows with PuTTY</h2>
<img src="img/putty-tunnel0.png">
</section>
</section>
<section>
<h2>SSH X forwarding</h2>
<p>Allow applications on the remote to access your local X server</p>
<p>This lets you interact with graphical programs on the remote.</p>
<p><code>ssh -X</code></p>
<p>(For SoC students: try this with Sunfire!)</p>
<p>(Windows: <a href="http://x.cygwin.com/">Cygwin</a> with PuTTY; out of scope)</p>
</section>
<section>
<section>
<h2>SSH configuration</h2>
<p>Set settings for certain hosts so you can avoid specifying username/other settings every time</p>
<p>Insert to <code>.ssh/config</code>: (for SoC students, this example is for Sunfire)</p>
<pre>Host xcn?? xcn??? sunfire sunfire0
HostName %h.comp.nus.edu.sg
User soc_username_here</pre>
<p>Then you can just <code>ssh sunfire</code> in future!</p>
<p>Other common host options: <code>LocalForward RemoteForward ForwardX11 ProxyJump</code></p>
</section>
<section>
<h2>SSH configuration</h2>
<p class="font75"><code>ProxyJump</code> causes "<code>ssh(1)</code> to connect to the target host by first making a <code>ssh(1)</code> connection to the specified <code>ProxyJump</code> host and then establishing a TCP forwarding to the ultimate target from there."</p>
<p>In <code>.ssh/config</code>: <span>(for SoC students: try this with the compute cluster)</span></p>
<pre>Host xcn?? xcn???
ProxyJump [email protected]
User soc_username_here</pre>
<p>Then <code>ssh xcnc7</code>, etc.</p>
</section>
</section>
<section>
<h2>SSHFS</h2>
<p>Mount a remote directory that you can access via SSH</p>
<pre>mkdir directory_name
sshfs [email protected]: directory_name</pre>
<p>Now your Sunfire home directory is accessible at <code>directory_name/</code></p>
<p>To umount:</p>
<pre>fusermount -u directory_name</pre>
</section>
<section>
<h2><a href="https://mosh.org/">Mosh</a></h2>
<p>A shell (like SSH) that works with intermittent internet connections (like on your phone).</p>
<p>Needs to be installed on both client and server (but root not required!)</p>
<p>Just replace <code>ssh</code> by <code>mosh</code>. <a href="https://mosh.org/#usage">More details</a></p>
</section>
<section>
<section>
<h2>Windows RDP</h2>
<p>Remotely access a Windows computer.</p>
<p>RDP client included with all Windows versions.</p>
<p class="font75">Officially only Pro and above editions can be connected to.</p>
<p class="font75">But see <a href="https://github.com/stascorp/rdpwrap">rdpwrap</a>.</p>
</section>
<section class="center">
<h2>Linux RDP server</h2>
<p><a href="http://www.xrdp.org/">Xrdp</a> <a href="https://wiki.archlinux.org/index.php/Xrdp">(Arch wiki setup)</a></p>
<h2>Linux RDP client</h2>
<p><a href="http://www.freerdp.com/">FreeRDP</a></p>
</section>
</section>
<section class="center">
<h1>System and application introspection and debugging</h1>
</section>
<section>
<section>
<h2>GDB</h2>
<p>Command line debugger: <code>gdb myprogram</code></p>
<p class="mb0">Common commands:</p>
<ul class="font75">
<li><code>r</code>: run program</li>
<li><code>start</code>: run and break at entrypoint</li>
<li><code>c</code>: continue program</li>
<li><code>bt</code>: show backtrace/stacktrace</li>
<li><code>b <u>line</u></code>: set breakpoint at <u>line</u></li>
<li><code>d <u>n</u></code>: delete breakpoint <u>n</u></li>
<li><code>p <u>expr</u></code>: print the value of <u>expr</u></li>
<li><code>l</code>: dump source around current line</li>
<li><code>disas</code>: dump machine code around current instruction</li>
<li><code>help <u>cmd</u></code>: show help for <u>cmd</u></li>
</ul>
</section>
<section>
<h2>GDB</h2>
<p class="mb0">Stepping commands:</p>
<ul class="font75">
<li><code>s</code>: step one line</li>
<li><code>si</code>: step one instruction</li>
<li><code>n</code>: step one line, but don't follow calls</li>
<li><code>ni</code>: step one instruction, but don't follow calls</li>
<li><code>finish</code>: run until this function returns</li>
<li><code>c</code>: continue a paused program</li>
<li><code>reverse-<u>cmd</u></code>: the above stepping commands, but in reverse; only after <code>record</code></li>
<li><code>record</code>: start recording execution</li>
</ul>
</section>
<section>
<p>Example program</p>
<pre><code>#include <stdio.h>
int access(int *array, int index) {
return array[index];
}
int main() {
int arr[] = { 0, 1, 2 };
printf("%d\n", access(0, 1));
}</code></pre>
<p>Compile: <code>gcc -g -o test test.c</code></p>
<p>Run: ./test</p>
<p><code>Segmentation fault</code> ☹️</p>
</section>
<section>
<p>Debug it</p>
<pre class="font45">$ gdb test
GNU gdb (GDB) 8.2.1
(gdb) r
Starting program: test
Program received signal SIGSEGV, Segmentation fault.
0x0000000100001168 in access (array=0x0, index=1) at test.c:4
4 return array[index];
(gdb) bt
#0 0x0000000100001168 in access (array=0x0, index=1) at test.c:4
#1 0x00000001000011a7 in main () at test.c:9
(gdb) b 4
Breakpoint 1 at 0x100001154: file test.c, line 4.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/angelsl/test
Breakpoint 1, access (array=0x0, index=1) at test.c:4
4 return array[index];
(gdb) p array
$1 = (int *) 0x0
(gdb) p index
$2 = 1</pre>
</section>
<section>
<p>Debug it in reverse!</p>
<pre class="font45">$ gdb test
GNU gdb (GDB) 8.2.1
(gdb) start
...
(gdb) record
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x0000000100001168 in access (array=0x0, index=1) at test.c:4
4 return array[index];
(gdb) rs
4 return array[index];
(gdb) rs
main () at test.c:9
9 printf("%d\n", access(0, 1));
(gdb) rs
8 int arr[] = { 0, 1, 2 };
(gdb) rs
No more reverse-execution history.
main () at test.c:7
7 int main() {
(gdb)</pre>
</section>
</section>
<section>
<section>
<h2>strace</h2>
<p>Look at the syscalls a program is making</p>
<p class="mb0">Example program:</p>
<pre><code>#include <stdio.h>
int main() {
printf("Hello world!\n");
}</code></pre>
<p>Compile: <code>gcc -g -o hello hello.c</code></p>
</section>
<section>
<h2>strace</h2>
<pre><strong>$ strace ./hello</strong>
execve("./hello", ["./hello"], 0x76f6e85d7ab0 /* 47 vars */) = 0
...
write(1, "Hello world!\n", 13) = 13
exit_group(0) = ?
+++ exited with 0 +++</pre>
<p><code>strace -f</code> to follow child processes.</p>
<p>Syscalls are how programs interact with the operating system. Out of scope for today.</p>
<p>You can <code>man 2 <u>syscall</u></code> to read the manpage for the syscall.</p>
</section>
</section>
<section>
<h2>ltrace</h2>
<p>Look at library calls a program is making</p>
<p>Using the same example program:</p>
<pre><strong>$ ltrace ./hello</strong>
puts("Hello world!") = 13
+++ exited (status 0) +++</pre>
</section>
<section>
<h2><a href="https://docs.microsoft.com/en-us/sysinternals/downloads/procmon">ProcMon</a> (Windows)</h2>
<p>Windows alternative for ltrace/strace.</p>
<img src="img/procmon0.png">
</section>
<section>
<h2>time</h2>
<p>Time a program's execution</p>
<pre>$ time sleep 5
real 0m5.004s
user 0m0.003s
sys 0m0.001s
$ time ./hello
Hello world!
real 0m0.003s
user 0m0.000s
sys 0m0.003s</pre>
<p>Real: real time; user: CPU time in userspace; sys: CPU time in kernel</p>
</section>
<section>
<h2>perf</h2>
<p class="font75">Measure statistics about a program, and many other things</p>
<pre><strong>$ perf stat ./hello</strong>
Hello world!
Performance counter stats for './hello':
1.24 msec task-clock # 0.581 CPUs utilized
0 context-switches # 0.000 K/sec
0 cpu-migrations # 0.000 K/sec
52 page-faults # 0.042 M/sec
982,828 cycles # 0.793 GHz
841,761 instructions # 0.86 insn per cycle
196,065 branches # 158.177 M/sec
7,276 branch-misses # 3.71% of all branches
0.002133030 seconds time elapsed
0.002084000 seconds user
0.000000000 seconds sys</pre>
<a href="http://www.brendangregg.com/perf.html">More usages</a>
</section>
<section>
<section>
<h2>procfs</h2>
<p>A virtual filesystem providing system and process information. Many tools we will mention take their information from procfs.</p>
<p class="mb0">Some files in <code>/proc</code>:</p>
<ul>
<li><code>cpuinfo</code>: lists CPU information</li>
<li><code>cmdline</code>: kernel command line</li>
<li><code>config.gz</code>: gzipped kernel config</li>
</ul>
<p>A lot of kernel information is in /sys instead.</p>
</section>
<section>
<h2>procfs</h2>
<p class="mb0">Some files in <code>/proc/<u>pid</u></code>:</p>
<p class="mt0 mb0 font75">(<u>pid</u> can be self, for current process)</p>
<ul class="font75">
<li><code>cmdline</code>: the command line provided to the process</li>
<li><code>cwd</code>: the process's current working directory</li>
<li><code>exe</code>: the executable being run</li>
<li><code>environ</code>: environment variables seen by the process</li>
<li><code>status</code>: process status</li>
<li><code>fd/<u>fd</u></code>: open file descriptors</li>
</ul>
<p>Typically we use tools like <code>top</code>, <code>ps</code> when we want to view process information.</p>
</section>
</section>
<section>
<section class="center">
<h2>Logs</h2>
<p class="mb0">What's in logs?</p>
<ul>
<li>Kernel errors</li>
<li>Network connection details</li>
<li>Application errors</li>
<li>...</li>
</ul>
<p>Many times, the first thing one does when something on the system doesn't work is look at logs.</p>
</section>
<section>
<h2>dmesg</h2>
<p>View the kernel log: <code>dmesg</code></p>
<h2>journalctl</h2>
<p>View the system log (for systemd systems): <code>journalctl</code></p>
<p class="mb0 font75">Common flags:</p>
<ul class="font75">
<li><code>-b</code>: current boot only</li>
<li><code>-k</code>: kernel messages only (same as dmesg)</li>
<li><code>-f</code>: follow/tail the log</li>
<li><code>-e</code>: jump to end</li>
</ul>
</section>
<section>
<h2>/var/log</h2>1
<p>Log files of some applications: Xorg, CUPS, LightDM, ...</p>
<p>Also where systemd journal is stored (<code>/var/log/journal</code>)</p>
<h2>tail</h2>
<p>To tail a log file (or any file): <code>tail -f <u>file</u></code></p>
</section>
<section>
<h2>Windows Event Viewer</h2>
<img src="img/eventviewer0.png">
</section>
</section>
<section>
<h2>(h)top</h2>
<p class="font75">Two different tools, both to view processes on the system; choose your poison</p>
<p class="mb0">Common <code>top</code> keys</p>
<ul>
<li>h: help</li>
<li>V: tree mode</li>
<li>f: add/remove columns</li>
<li>z: toggle colours</li>
<li>c: toggle cmdline/task name</li>
<li></>: change sort field</li>
</ul>
</section>
<section>
<section>
<h2>lsof/fuser</h2>
<p class="mb0">List open files: <code>lsof</code></p>
<ul>
<li><code>lsof <u>file</u></code>: list processes using <u>file</u></li>
<li><code>lsof -p <u>pid</u></code>: list files used by process <u>pid</u></li>
</ul>
<p class="mb0">List processes using <u>file</u>: <code>fuser <u>file</u></code></p>
<ul>
<li><code>fuser -k <u>file</u></code>: Kill processes using <u>file</u></li>
</ul>
<p>Both tools are similar; <code>lsof</code> is more powerful, but <code>fuser</code> is slightly more common.</p>
</section>
<section>
<h2>Windows: <a href="https://processhacker.sourceforge.io/">Process Hacker</a></h2>
<img src="img/ph0.png">
<p>Screenshots are from the website.</p>
</section>
<section>
<img src="img/ph1.png">
<img src="img/ph2.png">
</section>
</section>
<section>
<section class="font75">
<h2>lsblk/df</h2>
<p>View block devices (disk partitions, etc.)</p>
<pre><strong>$ lsblk</strong>
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 465.8G 0 disk
├─sda1 8:1 0 256M 0 part /boot/efi
├─sda2 8:2 0 464.5G 0 part
│ └─root 254:0 0 464.5G 0 crypt /
└─sda3 8:3 0 1G 0 part</pre>
<p>View disk free space</p>
<pre><strong>$ df -h</strong>
Filesystem Size Used Avail Use% Mounted on
dev 5.9G 0 5.9G 0% /dev
run 5.9G 1.1M 5.9G 1% /run
/dev/mapper/root 457G 58G 377G 14% /
tmpfs 5.9G 82M 5.8G 2% /dev/shm
tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup
tmpfs 5.9G 26M 5.8G 1% /tmp
/dev/sda1 253M 24M 229M 10% /boot/efi
tmpfs 1.2G 28K 1.2G 1% /run/user/1000</pre>
<p><code>-h</code>: human-readable sizes</p>
</section>
<section>
<h2>Windows: Disk Management</h2>
<p>Also <a href="https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/diskpart"><code>diskpart</code></a>; out of scope</p>
<img src="img/diskmgmt.png">
</section>
</section>
<section>
<section>
<h2>du</h2>
<p>View disk usage by folders</p>
<pre><strong>Dropbox/ $ du -hd1</strong>
250M ./Screenshots
66M ./Backup
2.5G ./Notes
8.0M ./Work
20K ./.dropbox.cache
2.8G .</pre>
<ul>
<li><code>-h</code>: human-readable sizes</li>
<li><code>-d <u>n</u></code>: summarise to <u>n</u> levels</li>
<li><code>-s</code>: show only a total for each argument</li>
</ul>
</section>
<section>
<h2>Windows: <a href="https://windirstat.net/">WinDirStat</a></h2>
<img src="img/windirstat.png">
</section>
</section>
<section>
<section>
<h2>dstat</h2>
<p>View system resource statistics</p>
<pre><strong>$ dstat</strong>
You did not select any stats, using -cdngy by default.
--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
9 5 85 1 0| 28k 43k| 0 0 | 0 0 | 738 3086
4 3 90 2 0| 0 0 | 336k 3190B| 0 0 |1979 9753
5 3 91 0 0| 0 0 | 238B 178B| 0 0 |1919 9188
6 5 87 2 0| 0 0 | 571B 356B| 0 0 |1916 9077
5 3 92 0 0| 0 108k|1465B 0 | 0 0 |1863 8952
4 3 91 2 0| 0 0 | 238B 178B| 0 0 |1716 9049</pre>
<p>Windows: Use Process Hacker</p>
</section>
</section>
<section>
<section>
<h2>ss</h2>
<p class="mb0">View network connections: <code>ss</code></p>
<ul>
<li><code>-t/-u</code>: TCP/UDP only</li>
<li><code>-4/-6</code>: IPv4/IPv6 only</li>
<li><code>-l</code>: listening ports only</li>
<li><code>-a</code>: all states (listening and non-listening)</li>
<li><code>-p</code>: view process information</li>
</ul>
</section>
<section>
<h2>Windows: <a href="https://processhacker.sourceforge.io/">Process Hacker</a></h2>
<img src="img/ph3.png">
<p>Screenshots are from the website.</p>
</section>
</section>
<section class="center">
<h1>Network debugging</h1>
</section>
<section class="font75">
<h2>Traceroute</h2>
<p>See the route a packet takes from you to a remote host</p>
<pre><strong># traceroute -Tn 1.0.0.1</strong>
traceroute to 1.0.0.1 (1.0.0.1), 30 hops max, 60 byte packets
1 172.24.212.1 0.509 ms 0.493 ms 0.516 ms
2 172.18.20.145 0.498 ms 0.495 ms 0.612 ms
3 172.18.20.5 0.475 ms 0.586 ms 0.592 ms
4 172.18.20.202 0.581 ms 0.692 ms 0.691 ms
5 172.18.20.98 0.552 ms 0.647 ms 0.631 ms
6 172.18.20.102 0.615 ms 0.598 ms 0.729 ms
7 * * *
...
11 * 1.0.0.1 3.384 ms *</pre>
<p class="mb0">Common flags</p>
<ul>
<li><code>-n</code>: don't resolve addresses</li>
<li><code>-4/-6</code>: use IPv4/IPv6</li>
<li><code>-I/-T/-U</code>: use ICMP/TCP/UDP</li>
</ul>
<p>Some flags may require privileges. <code>tracepath</code> is an alternative that does not require root.</p>
</section>
<section class="font75">
<h2>Ping</h2>
<p>Send packets to a host and see if it responds</p>
<pre><strong>$ ping -A 1.0.0.1</strong>
PING 1.0.0.1 (1.0.0.1) 56(84) bytes of data.
64 bytes from 1.0.0.1: icmp_seq=2 ttl=54 time=2.100 ms
...
64 bytes from 1.0.0.1: icmp_seq=43 ttl=54 time=3.25 ms
^C
--- 1.0.0.1 ping statistics ---
43 packets transmitted, 29 received, 32.5581% packet loss, time 241ms
rtt min/avg/max/mdev = 2.740/3.476/10.983/1.466 ms, ipg/ewma 219.812/3.802 ms</pre>
<p class="font45">NUS network was having problems ☹️</p>
<p class="mb0">Common flags</p>
<ul>
<li><code>-A</code>: adaptive (send next packet on receiving response)</li>
<li><code>-q</code>: quiet</li>
<li><code>-f</code>: flood (don't wait for response); needs root</li>
<li><code>-n</code>: don't resolve addresses</li>
<li><code>-4/-6</code>: use IPv4/IPv6</li>
</ul>
</section>
<section>
<section class="font75">
<h2>nslookup</h2>
<p>Perform DNS queries</p>
<pre><strong>$ nslookup google.com</strong>
Server: 1.0.0.1
Address: 1.0.0.1#53
Non-authoritative answer:
Name: google.com
Address: 74.125.24.100
Name: google.com
Address: 2404:6800:4003:c02::8b
<strong>$ nslookup -query=MX google.com</strong>
Server: 1.0.0.1
Address: 1.0.0.1#53
Non-authoritative answer:
google.com mail exchanger = 40 alt3.aspmx.l.google.com.</pre>
</section>
<section>
<h2>nslookup</h2>
<p>Use a different nameserver</p>
<pre><strong>$ nslookup google.com 8.8.8.8</strong>
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: google.com
Address: 74.125.24.138
Name: google.com
Address: 2404:6800:4003:c01::65</pre>
</section>
<section>
<h2>dig</h2>
<pre><strong>$ dig @8.8.8.8 google.com MX</strong>
;; ANSWER SECTION:
google.com. 323 IN MX 10 aspmx.l.google.com.
;; Query time: 5 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Mar 30 02:28:35 +08 2019
;; MSG SIZE rcvd: 147</pre>
<p>What's the difference? <code>nslookup</code> uses the OS's resolver libraries; <code>nslookup</code> uses its own. They are otherwise used for the same purpose.</p>
</section>
</section>
<section>
<section>
<h2>whois</h2>
<p>Look up information on an IP or domain</p>
<pre class="font45"><strong>$ whois nus.edu.sg</strong>
----------------------------------------------------------------------
SGNIC WHOIS Server
----------------------------------------------------------------------
The following data is provided for information purposes only.
Registrar: SINGNET PTE LTD
Domain Name: NUS.EDU.SG
Creation Date: 03-Sep-1996 00:00:00
Modified Date: 28-Aug-2018 09:31:22
Expiration Date: 03-Sep-2019 00:00:00
Domain Status: OK
Domain Status: VerifiedID@SG-Not Required
Registrant:
Name: NATIONAL UNIVERSITY OF SINGAPORE (SGNIC-ORGNA66731)</pre>
</section>
<section>
<h2>whois</h2>
<pre class="font45"><strong>$ whois 4.2.2.1</strong>
NetRange: 4.0.0.0 - 4.255.255.255
CIDR: 4.0.0.0/8
NetName: LVLT-ORG-4-8
NetHandle: NET-4-0-0-0-1
Parent: ()
NetType: Direct Allocation
OriginAS:
Organization: Level 3 Parent, LLC (LPL-141)
RegDate: 1992-12-01
Updated: 2018-02-20
Ref: https://rdap.arin.net/registry/ip/4.0.0.0</pre>
</section>
</section>
<section class="center">
<h1>Linux containers</h1>
<p>This is extra, and very brief, material (not going through live)</p>
</section>
<section>
<h2>Why containers?</h2>
<p>If you need another distribution's environment (e.g. you run Arch but need Debian), or if you need to test something in a clean environment, or just need some isolation, etc</p>
<h2>Why not a VM?</h2>
<p>A container is much lighter in terms of memory/storage/CPU resources, and will run faster, compared to a VM.</p>
</section>
<section>
<h2>Plain chroot</h2>
<p>The most basic "container".</p>
<p class="mb0">Basic steps:</p>
<ol>
<li>Bootstrap the chroot: <a href="https://wiki.debian.org/Debootstrap">debootstrap</a>, <a href="https://wiki.archlinux.org/index.php/Install_from_existing_Linux#Creating_a_chroot">pacstrap</a>, etc</li>
<li>Chroot into it</li>
<li>Done!</li>
</ol>
</section>
<section>
<h2>LXC/LXD</h2>
<p>Tools to build and run containers.</p>
<p><a href="https://linuxcontainers.org/lxd/getting-started-cli/">Introduction guide</a></p>
<h2>Docker</h2>
<p>Powerful platform to build and run containers.</p>
<p>A topic for an entire workshop.</p>
<p><a href="https://docs.docker.com/get-started/">Introduction guide</a></p>
</section>
<section class="center">
<h1>Thank you!</h1>
</section>
</div>
</div>
<script src="lib/js/head.min.js"></script>
<script src="js/reveal.js"></script>
<script>
// More info about config & dependencies:
// - https://github.com/hakimel/reveal.js#configuration
// - https://github.com/hakimel/reveal.js#dependencies
Reveal.initialize({
dependencies: [
{ src: 'plugin/notes/notes.js', async: true },
{ src: 'plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } }
],
slideNumber: true,
fragmentInURL: true,
transitionSpeed: 'fast',
center: false
});
</script>
</body>
</html>