Skip to content

Simple Neural Network using Non-linear Least Squares Algorithms

Notifications You must be signed in to change notification settings

SecureSheII/Simple-Neural-Network

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 

Repository files navigation

Simple-Neural-Network

Train a simple neural network using non-linear least squares algorithms (Levenberg-Marquardt), and test them to approximate certain non-linear functions.

Simple Neural Network

A neural network is widely used parametric model to approximate non-linear functions. These networks are parameterized by a set of weights. Neural networks are often referred to as “universal approximators", which means that they can approximate a large class of non-linear functions by suitably tuning the weights. Here I programmed a simple neural network to approximate different types of non-linear functions using “non-linear" least squares algorithms.

A neural network is a model of the form
image
image

I used sigmoid as our psi function.

The non-linear function to approximate is
image
and
image

Training

I used Levenberg-Marquardt algorithm to determine the weighhts for approximating different non-linear functions.

image

The loss function is defined as
image
where r is simply the difference between our prediction fw, and the ground-truth values y.

About

Simple Neural Network using Non-linear Least Squares Algorithms

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published