Skip to content

Commit 09621c4

Browse files
mhjensenmhjensen
committed
added exercises week 47
1 parent 7413b1f commit 09621c4

File tree

3 files changed

+276
-65
lines changed

3 files changed

+276
-65
lines changed

‎doc/LectureNotes/exercisesweek47.ipynb

+153
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "73c0b63f",
6+
"metadata": {
7+
"editable": true
8+
},
9+
"source": [
10+
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
11+
"doconce format html exercisesweek47.do.txt -->\n",
12+
"<!-- dom:TITLE: Exercise week 47 -->"
13+
]
14+
},
15+
{
16+
"cell_type": "markdown",
17+
"id": "8cb10d9d",
18+
"metadata": {
19+
"editable": true
20+
},
21+
"source": [
22+
"# Exercise week 47\n",
23+
"**November 18-22, 2024**\n",
24+
"\n",
25+
"Date: **Deadline is Friday November 22 at midnight**"
26+
]
27+
},
28+
{
29+
"cell_type": "markdown",
30+
"id": "2b90add8",
31+
"metadata": {
32+
"editable": true
33+
},
34+
"source": [
35+
"# Overarching aims of the exercises this week\n",
36+
"\n",
37+
"The exercise set this week is meant as a summary of many of the\n",
38+
"central elements in various machine learning algorithms, with a slight\n",
39+
"bias towards deep learning methods and their training. You don't need to answer all questions.\n",
40+
"\n",
41+
"The last weekly exercise (week 48) is a general course survey."
42+
]
43+
},
44+
{
45+
"cell_type": "markdown",
46+
"id": "b4f3ae78",
47+
"metadata": {
48+
"editable": true
49+
},
50+
"source": [
51+
"## Exercise 1: Linear and logistic regression methods\n",
52+
"\n",
53+
"1. What is the main difference between ordinary least squares and Ridge regression?\n",
54+
"\n",
55+
"2. Which kind of data set would you use logistic regression for?\n",
56+
"\n",
57+
"3. In linear regression you assume that your output is described by a continuous non-stochastic function $f(x)$. Which is the equivalent function in logistic regression?\n",
58+
"\n",
59+
"4. Can you find an analytic solution to a logistic regression type of problem?\n",
60+
"\n",
61+
"5. What kind of cost function would you use in logistic regression?"
62+
]
63+
},
64+
{
65+
"cell_type": "markdown",
66+
"id": "755cfd27",
67+
"metadata": {
68+
"editable": true
69+
},
70+
"source": [
71+
"## Exercise 2: Deep learning\n",
72+
"\n",
73+
"1. What is an activation function and discuss the use of an activation function? Explain three different types of activation functions?\n",
74+
"\n",
75+
"2. Describe the architecture of a typical feed forward Neural Network (NN). \n",
76+
"\n",
77+
"3. You are using a deep neural network for a prediction task. After training your model, you notice that it is strongly overfitting the training set and that the performance on the test isn’t good. What can you do to reduce overfitting?\n",
78+
"\n",
79+
"4. How would you know if your model is suffering from the problem of exploding Gradients?\n",
80+
"\n",
81+
"5. Can you name and explain a few hyperparameters used for training a neural network?\n",
82+
"\n",
83+
"6. Describe the architecture of a typical Convolutional Neural Network (CNN)\n",
84+
"\n",
85+
"7. What is the vanishing gradient problem in Neural Networks and how to fix it?\n",
86+
"\n",
87+
"8. When it comes to training an artificial neural network, what could the reason be for why the cost/loss doesn't decrease in a few epochs?\n",
88+
"\n",
89+
"9. How does L1/L2 regularization affect a neural network?\n",
90+
"\n",
91+
"10. What is(are) the advantage(s) of deep learning over traditional methods like linear regression or logistic regression?"
92+
]
93+
},
94+
{
95+
"cell_type": "markdown",
96+
"id": "85175b87",
97+
"metadata": {
98+
"editable": true
99+
},
100+
"source": [
101+
"## Exercise 3: Decision trees and ensemble methods\n",
102+
"\n",
103+
"1. Mention some pros and cons when using decision trees\n",
104+
"\n",
105+
"2. How do we grow a tree? And which are the main parameters? \n",
106+
"\n",
107+
"3. Mention some of the benefits with using ensemble methods (like bagging, random forests and boosting methods)?\n",
108+
"\n",
109+
"4. Why would you prefer a random forest instead of using Bagging to grow a forest?\n",
110+
"\n",
111+
"5. What is the basic philosophy behind boosting methods?"
112+
]
113+
},
114+
{
115+
"cell_type": "markdown",
116+
"id": "fbfdfe68",
117+
"metadata": {
118+
"editable": true
119+
},
120+
"source": [
121+
"## Exercise 4: Optimization part\n",
122+
"\n",
123+
"1. Which is the basic mathematical root-finding method behind essentially all gradient descent approaches(stochastic and non-stochastic)? \n",
124+
"\n",
125+
"2. And why don't we use it? Or stated differently, why do we introduce the learning rate as a parameter?\n",
126+
"\n",
127+
"3. What might happen if you set the momentum hyperparameter too close to 1 (e.g., 0.9999) when using an optimizer for the learning rate?\n",
128+
"\n",
129+
"4. Why should we use stochastic gradient descent instead of plain gradient descent?\n",
130+
"\n",
131+
"5. Which parameters would you need to tune when use a stochastic gradient descent approach?"
132+
]
133+
},
134+
{
135+
"cell_type": "markdown",
136+
"id": "92fc1b0c",
137+
"metadata": {
138+
"editable": true
139+
},
140+
"source": [
141+
"## Exercise 5: Analysis of results\n",
142+
"1. How do you assess overfitting and underfitting?\n",
143+
"\n",
144+
"2. Why do we divide the data in test and train and/or eventually validation sets?\n",
145+
"\n",
146+
"3. Why would you use resampling methods in the data analysis? Mention some widely popular resampling methods."
147+
]
148+
}
149+
],
150+
"metadata": {},
151+
"nbformat": 4,
152+
"nbformat_minor": 5
153+
}

‎doc/src/week47/exercisesweek47.do.txt

+39-65
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,58 @@
11
TITLE: Exercise week 47
2-
AUTHOR: November 20-24, 2023
3-
DATE: Deadline is Sunday November 26 at midnight
2+
AUTHOR: November 18-22, 2024
3+
DATE: Deadline is Friday November 22 at midnight
44

55

66
======= Overarching aims of the exercises this week =======
77

8-
The exercise this week is a simple course survey and feedback. This
9-
is important for us in order to improve our teaching material, the
10-
active learning format and anything else related to a succesful
11-
mastering of central machine learning methods and their applications.
8+
The exercise set this week is meant as a summary of many of the
9+
central elements in various machine learning algorithms, with a slight
10+
bias towards deep learning methods and their training. You don't need to answer all questions.
1211

12+
The last weekly exercise (week 48) is a general course survey.
1313

14-
=== Why did you choose this course? ===
1514

16-
=== What was your programming knowledge before you started? ===
17-
And do you feel this course added to your programming competences and skills?
15+
===== Exercise 1: Linear and logistic regression methods =====
1816

19-
=== How do you judge your own level of knowledge on machine learning before and after this course? ===
17+
o What is the main difference between ordinary least squares and Ridge regression?
18+
o Which kind of data set would you use logistic regression for?
19+
o In linear regression you assume that your output is described by a continuous non-stochastic function $f(x)$. Which is the equivalent function in logistic regression?
20+
o Can you find an analytic solution to a logistic regression type of problem?
21+
o What kind of cost function would you use in logistic regression?
2022

21-
Here you can discuss your level of skill/knowledge at start of course
22-
and at the end of the course and how these matched the level of
23-
skill/knowledge needed to complete the projects.
2423

25-
=== Did the projects and the teaching material allow you to deepen your insights about Machine Learning? ===
24+
===== Exercise 2: Deep learning =====
2625

27-
Feel free to comment here.
26+
o What is an activation function and discuss the use of an activation function? Explain three different types of activation functions?
27+
o Describe the architecture of a typical feed forward Neural Network (NN).
28+
o You are using a deep neural network for a prediction task. After training your model, you notice that it is strongly overfitting the training set and that the performance on the test isn’t good. What can you do to reduce overfitting?
29+
o How would you know if your model is suffering from the problem of exploding Gradients?
30+
o Can you name and explain a few hyperparameters used for training a neural network?
31+
o Describe the architecture of a typical Convolutional Neural Network (CNN)
32+
o What is the vanishing gradient problem in Neural Networks and how to fix it?
33+
o When it comes to training an artificial neural network, what could the reason be for why the cost/loss doesn't decrease in a few epochs?
34+
o How does L1/L2 regularization affect a neural network?
35+
o What is(are) the advantage(s) of deep learning over traditional methods like linear regression or logistic regression?
2836

37+
===== Exercise 3: Decision trees and ensemble methods =====
2938

30-
=== Project based teaching and active learning ===
39+
o Mention some pros and cons when using decision trees
40+
o How do we grow a tree? And which are the main parameters?
41+
o Mention some of the benefits with using ensemble methods (like bagging, random forests and boosting methods)?
42+
o Why would you prefer a random forest instead of using Bagging to grow a forest?
43+
o What is the basic philosophy behind boosting methods?
3144

32-
This is a project based course and we as teachers would like to keep
33-
it as it is since we see very clearly that people who attend this
34-
course have a very good learning outcome. Project based courses are
35-
however demanding (and expensive seen from the university admin) when
36-
it comes to proper feedback and evaluations. Feel free to discuss
37-
whether you found a project-based and active learning approach
38-
useful. Feel also free to comment upon things we can improve upon or
39-
alternative ways to assess whether the learning outcomes have been
40-
achieved. Would you for example prefer a standard 4 hours written exam
41-
be something you would prefer? Or other alternatives to projects? We
42-
would very much value your thoughts here since projects are an
43-
essential part of this course.
45+
===== Exercise 4: Optimization part =====
4446

45-
=== Usefulness of the weekly exercises ===
47+
o Which is the basic mathematical root-finding method behind essentially all gradient descent approaches(stochastic and non-stochastic)?
48+
o And why don't we use it? Or stated differently, why do we introduce the learning rate as a parameter?
49+
o What might happen if you set the momentum hyperparameter too close to 1 (e.g., 0.9999) when using an optimizer for the learning rate?
50+
o Why should we use stochastic gradient descent instead of plain gradient descent?
51+
o Which parameters would you need to tune when use a stochastic gradient descent approach?
4652

47-
Did the weekly exercises help in getting started with the projects?
48-
How relevant where they for solving the projects? Feel free to
49-
elaborate
50-
51-
=== Active learning/lab sessions and lectures ===
52-
53-
Was there a good link between lectures and active learning sessions?
54-
Would you prefer an active learning environment only with no lectures
55-
or would you prefer a more lecture based format with lab sessions only
56-
(that is no discussion at the beginning of the lab sessions)? Feel
57-
free to comment.
58-
59-
60-
61-
=== How would you improve this course? ===
62-
63-
Are there topics which are missing, topics which could have been
64-
omitted and/or discussed in more depth? Feel free to add your comments
65-
here such as how to improve to teaching material and more.
66-
67-
68-
69-
===== Then some basic questions =====
70-
71-
=== Which is your preferred information chanel, Canvas, Discord, mail or other? ===
72-
73-
=== Was the weekly update with plans etc useful? ===
74-
75-
=== Was it easy to access the course material? ===
76-
77-
=== Which resources and tools did you use? Jupyter-notebooks, GitHub, the various textbooks we have recommended, etc etc ===
78-
79-
=== If you did not attend the lectures or the active learning/lab sessions, which resources did you use? ===
80-
81-
82-
=== Any other topics, impressions, ideas etc you would like to share with us? ===
8353

8454

55+
===== Exercise 5: Analysis of results =====
56+
o How do you assess overfitting and underfitting?
57+
o Why do we divide the data in test and train and/or eventually validation sets?
58+
o Why would you use resampling methods in the data analysis? Mention some widely popular resampling methods.

‎doc/src/week48/exercisesweek48.do.txt

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
TITLE: Exercise week 47
2+
AUTHOR: November 20-24, 2023
3+
DATE: Deadline is Sunday November 26 at midnight
4+
5+
6+
======= Overarching aims of the exercises this week =======
7+
8+
The exercise this week is a simple course survey and feedback. This
9+
is important for us in order to improve our teaching material, the
10+
active learning format and anything else related to a succesful
11+
mastering of central machine learning methods and their applications.
12+
13+
14+
=== Why did you choose this course? ===
15+
16+
=== What was your programming knowledge before you started? ===
17+
And do you feel this course added to your programming competences and skills?
18+
19+
=== How do you judge your own level of knowledge on machine learning before and after this course? ===
20+
21+
Here you can discuss your level of skill/knowledge at start of course
22+
and at the end of the course and how these matched the level of
23+
skill/knowledge needed to complete the projects.
24+
25+
=== Did the projects and the teaching material allow you to deepen your insights about Machine Learning? ===
26+
27+
Feel free to comment here.
28+
29+
30+
=== Project based teaching and active learning ===
31+
32+
This is a project based course and we as teachers would like to keep
33+
it as it is since we see very clearly that people who attend this
34+
course have a very good learning outcome. Project based courses are
35+
however demanding (and expensive seen from the university admin) when
36+
it comes to proper feedback and evaluations. Feel free to discuss
37+
whether you found a project-based and active learning approach
38+
useful. Feel also free to comment upon things we can improve upon or
39+
alternative ways to assess whether the learning outcomes have been
40+
achieved. Would you for example prefer a standard 4 hours written exam
41+
be something you would prefer? Or other alternatives to projects? We
42+
would very much value your thoughts here since projects are an
43+
essential part of this course.
44+
45+
=== Usefulness of the weekly exercises ===
46+
47+
Did the weekly exercises help in getting started with the projects?
48+
How relevant where they for solving the projects? Feel free to
49+
elaborate
50+
51+
=== Active learning/lab sessions and lectures ===
52+
53+
Was there a good link between lectures and active learning sessions?
54+
Would you prefer an active learning environment only with no lectures
55+
or would you prefer a more lecture based format with lab sessions only
56+
(that is no discussion at the beginning of the lab sessions)? Feel
57+
free to comment.
58+
59+
60+
61+
=== How would you improve this course? ===
62+
63+
Are there topics which are missing, topics which could have been
64+
omitted and/or discussed in more depth? Feel free to add your comments
65+
here such as how to improve to teaching material and more.
66+
67+
68+
69+
===== Then some basic questions =====
70+
71+
=== Which is your preferred information chanel, Canvas, Discord, mail or other? ===
72+
73+
=== Was the weekly update with plans etc useful? ===
74+
75+
=== Was it easy to access the course material? ===
76+
77+
=== Which resources and tools did you use? Jupyter-notebooks, GitHub, the various textbooks we have recommended, etc etc ===
78+
79+
=== If you did not attend the lectures or the active learning/lab sessions, which resources did you use? ===
80+
81+
82+
=== Any other topics, impressions, ideas etc you would like to share with us? ===
83+
84+

0 commit comments

Comments
 (0)