CAS about artificial intelligence made at BFH in 2020 and 2021.
- Ressources
- 2020-10-20 1.Einführung in AI Grundtechniken: Gradient Descent,partielle Ableitungen, Matrix Algebra, AI Frameworks
- 2020-10-26 2.Tensorflow und PyTorch Frameworks
- 2020-11-03 3.Fundamentale Neuronale Netze: MLP und Autoencoder
- 2020-11-10 4.Anomaly Detection mit Autoencoder
- 2020-11-17 5.Variational Autoencoder
- 2020-11-24 6.CNN
- 2020-12-01 7.CNN - Transfer Learning
- 2020-12-08 8.GAN
- 2020-12-15 9.RNN
- 2021-01-05 10.RL Grundlagen
- 2021-01-12 11.RL Cross Entropy
- 2021-01-19 12.RL Value iteration
- 2021-01-26 13. Tabular Q-Learning
- 2021-02-02 14. Deep Q Learning
- 2021-02-09 15. Reinforce
- 2021-02-16 16. A2C
- 2021-02-23 17. Tween Delayed Deep Deterministic Policy Gradient - TD3
- 2021-03-02 18. Evolution strategy
- 2021-03-09 19. RL mit Trend Following Strategie
- 2021-03-16 20. Q-Learning Lab
- 2021-03-23 21. Klausur Vorbereitung I
- 2021-03-30 22. Klausur Vorbereitung II - reinforcement learning
Table of contents generated with markdown-toc
Machine learning data repository
Linear Regression in Python with Cost function and Gradient descent
Deep Learning (CAS machine intelligence, 2019)
- Machine Learning, Tom Mitchell, McGraw Hill, 1997 http://www.cs.cmu.edu/~tom/mlbook.html
- Deeplearning, Ian Goodfellow https://www.deeplearningbook.org/contents/mlp.html
- Stanford Cheat sheets about AI https://stanford.edu/~shervine/teaching/
- Deep reinforcement learning, Miguel Morales: https://drive.google.com/file/d/1VWO0Ji-iK5Z7iqe8XwgOivZLC_SCXIL8/view?usp=sharing
Reinforcement learning glossary
- Sample: A sample is a single row of data (=an instance, an observation, an input vector, or a feature vector.).
- Batch size: The batch size defines the number of samples to work through before updating the internal model parameters.
- Epoch: The number of epochs defines the number times that the learning algorithm will work through the entire training dataset.
- Gradient descent: Gradient descent is an optimization algorithm used to find the values of parameters (coefficients) of a function (f) that minimizes a cost function (cost).
- Deep learning algorithms
Update python packages
pip list --outdated
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U
2020-10-20 1.Einführung in AI Grundtechniken: Gradient Descent,partielle Ableitungen, Matrix Algebra, AI Frameworks
Y: A label is the thing we're predicting
X: A feature is an input variable (can be a list)
x: An example is a particular instance of data (a vector of values for a feature)
A labeled example includes both feature(s) and the label.
An unlabeled example contains features but not the label.
Once we've trained our model with labeled examples, we use that model to predict the label on unlabeled examples.
General AI function: Y = wX + b
| Doc | |
|---|---|
Compute a gradient descent for a complex function and determine iteratively m and b: Colab
Tensorflow quickstart for beginners
Predict fuel efficiency with a Tensorfol regressition. Dataset: MPG
Pytorch quickstart for beginners
| Doc | |
|---|---|
- Change values N, D_in, H, D
- Add a new layer/activation function with H hidden size
Pytorch gradients - trainer notebook / exercice notebook
Pytorch linear regressions - trainer notebook / exercice notebook
Pytorch NN - trainer notebook / exercice notebook
Pytorch datasets management - trainer notebook / exercice notebook
Tensorflow quickstart for experts - trainer notebook / exercice notebook
Tensorflow autoencoder - trainer notebook / exercice notebook
Migrate Iris exercice to Tensorflow: https://colab.research.google.com/drive/1gPMNk24EuvBKun5oCfV_mGrCtUA2rmoy?usp=sharing
Tensorflow Stacked MLP autoencode - trainer notebook / exercice notebook
Original autoencoder for ECG validation
A Gentle Introduction to Anomaly Detection with Autoencoders
Goal of the exercice is to use another data set: Anomaly detection for credit card with autoencoder
Variational autoencode - trainer notebook / exercice notebook / video
Exercice: use electrocardiogram data with previous variational autoencoder
Divergence de Kullback-Leibler / video
6 Different Ways of Implementing VAE with TensorFlow 2 and TensorFlow Probability
-
Basierend auf dem vorliegenden Notebook, VAE mit den ECG Daten trainieren.
-
Training von VAE
-
Anomaly Detection mit VAE und Vergleich von drei Metrics - accuracy, precision und recall von einem vanilla Autoencoder aus Aufgabe 4
Robust Variational Autoencoder trainer notebook / my homework notebook
Convolution is a serie of scalar product.
Notebook for understanding convolutions
Softmax: compute probability of vector element
CNN homework with preprocessing -> finally cancelled
CNN Advanced homework #2 - With CIFAR 10
Conv Variational Autoencoder homework - With CIFAR 10 -> finally cancelled
Robust Conv Variational Autoencoder homework - with MNIST - Trainer notebook / Homework #3
VGG16 – Convolutional Network for Classification and Detection
07 TF2.0 Transfer Learning - Special - Trainer notebook / My notebook
07 TF2.0 Transfer Learning with Data Augmentation - Classic - Trainer notebook / My notebook
07 TF2.0 Transfer Learning - Special CIFAR / Trainer notebook
Generative adversarial network
A Friendly Introduction to Generative Adversarial Networks (GANs) - Video
A guide to convolution arithmetic for deep learning
Cheatsheet recurrent neural networks
RNN for calligraphy / Source code
RNN with pytorch - Trainer notebook
LSTM with pytorch - Trainer notebook
Stock return - Trainer notebook
Stock prediction by Boris Banushev
Reinforcement Learning algorithms — an intuitive overview
We will work with Anaconda and PyCharm
A new python environment has been installed with Anaconda: casaai2020
Install gym and pygame:
source activate casaai2020
pip install gym
pip install pygame
Take the trainer notebook and make it working on pycharm -> export to .py. In order to work with PyCharm, several components had to be installed with pip: opencv, opencv-python, torchvision, cmake and atari-py
Tip: load tensorboard by starting it in the PyCharm console:
tensorboard --logdir=runs
GYM environments - Trainer notebook / My notebook
CrossEntropy example - Cart Pole agent and mountain car agent
Tip: python environment library version:
source activate casaai2020
pip list
Policy Gradient with gym-MiniGrid
Moodle description / mini-grid code
Comments about value iteration example:
- Observations: the possible agent positions in the 4x4 grids. 16 possibilities.
- Actions: the possible actions made by the agent: up, down, left and right
- Rewards: the possible rewards, depending of the currrent state, the next state and the action: 16 * 16 * 4
- Transitions: the possible paths, depending of the currrent state, the next state and the action:: 16 * 16 * 4
- Values: the Q-values, depending og the state and the action: 16 * 4
PyTorch uncertainty estimation - trainer notebook / my notebook / video
Tabular Q-Learning source code
Avocado exercise - trainer notebook / my notebook
Advanced Forecasting with LSTM and verification of results using Multi-Step Forecasting Devoir - trainer notebook / my notebook
Exercise - my notebook with price only
Exercise - my notebook with cryptocurrencies
Exercise - my second notebook with cryptocurrencies
Reinforce example with lunar lander
An Intuitive Explanation of Policy Gradient
Claude Shannon entropy computation
Reinforcement learning cheat sheet
Sequence modeling with attenion - trainer notebook with ECG data - my notebook
Dynamic Content Personalization Using LinUCB - trainer notebook
Reinforcement Learning:An Introduction Stanford
Colab cheetah / other implementation
Use cases of T3D:
- machine control
- trading forecaster
- sensor control/management (example of insulin: measure = state / inject insulin = action)
- Select a box environment: https://github.com/openai/gym/wiki/Table-of-environments
- Make a training with 1000 steps like on https://colab.research.google.com/drive/1XcF9Lekdl3VQMK1IbnDQp4ZKFY2HBK0r?usp=sharing
Blog about evolution strategies
Colab Evolution Strategies Supervised / my notebook
Colab Evolution Strategies Half Cheetah
Regression with evolution strategies - my notebook
Regression with evolution strategies and PyTorch - solution
Trend Following Strategie (SMA) - trainer notebook
Q-Learning Algo Trader - trainer notebook
Trend Following Strategie (SMA) - my notebook
Trend Following Strategies revisited , with env and agent - my notebook
Q-Learning trader - my notebook
Deep reinforcement learning book
Grokking Deep Reinforcement Learning
2 exercises: 1 deep learning / 1 reinforcement learning
Introduction to gradients and automatic differentiation - my notebook
Basic training loops - multidimensional
- Markov property is the probability of the next state, given the current state and current action, will be the same as if you give the entire history of interactions (states and actions)
- | means "gegeben"
- Transition function is defined as the probability fo transitioning to state s' at time step t given action a was selected on state s in the previous time step t-1. Given these are the probabilities, we expect the sum of the probabilities across all possible next states to sum to 1. Thta's true for all states s in the set of states s, and all actions a in the set of actions available in state s.
- Reward function can be defined as a function that takes in a state-action pair. Andm it's the expectation of reward at time step t, given the state-action pair in the previous time step. But, it can also be defined as a function that takes a full transition tuple s, a, s'. And it's also defined as the expectation, but now given that transition tuple. the reward at time step t comes from a set of all rewards R, whihc is a subset of all real numbers.
- MDP: S, A, T, R, Stheta, gamma, horizon
- POMDP: S, A, T, R, Stheta, gamma, horizon, observation, epsilon