A neural network implementation. More...

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <learning/learn_debug.h>
#include <learning/string_utils.h>
#include <learning/List.h>
#include <learning/real.h>

Include dependency graph for ANN.h:

Go to the source code of this file.

Classes
struct	Connection_
	A linear connection between two neural elements. More...

struct	RBFConnection_
	An RBF connection between two neural elements. More...

struct	Layer_
	A collection of connections from one layer to another, plus management functions and data. More...

struct	ANN_
	ANN management structure. More...

Typedefs
typedef struct Connection_	Connection
	A linear connection between two neural elements. More...

typedef struct RBFConnection_	RBFConnection
	An RBF connection between two neural elements. More...

typedef struct Layer_	Layer
	A collection of connections from one layer to another, plus management functions and data. More...

typedef struct ANN_	ANN
	ANN management structure. More...

Functions
ANN *	NewANN (int n_inputs, int n_outputs)
	Create a new ANN. More...

int	DeleteANN (ANN *ann)
	Delete a neural network. More...

ANN *	LoadANN (char *filename)
	Load an ANN from a filename. More...

ANN *	LoadANN (FILE *f)
	Load the ANN from a C file handle. More...

int	SaveANN (ANN ann, char filename)
	Save the ANN to a filename. More...

int	SaveANN (ANN ann, FILE f)
	Save the ANN to a C file handle. More...

int	ANN_AddHiddenLayer (ANN *ann, int n_nodes)
	Add a hidden layer with n_nodes. More...

int	ANN_AddRBFHiddenLayer (ANN *ann, int n_nodes)
	Add an RBF layer with n_nodes. More...

int	ANN_Init (ANN *ann)
	Initialise neural network. More...

void	ANN_SetOutputsToTanH (ANN *ann)
	Set outputs to hyperbolic tangent. More...

void	ANN_SetOutputsToLinear (ANN *ann)
	Set outputs to linear. More...

void	ANN_SetLearningRate (ANN *ann, real a)
	Set the learning rate to a. More...

void	ANN_SetLambda (ANN *ann, real lambda)
	Set lambda, eligibility decay. More...

void	ANN_SetZeta (ANN *ann, real lambda)
	Set zeta, parameter variance smoothing. More...

void	ANN_Reset (ANN *ann)
	Resets the eligbility traces and batch updates. More...

real	ANN_Input (ANN ann, real x)
	Give an input vector to the neural network. More...

real	ANN_StochasticInput (ANN ann, real x)
	Stochastically generate an output, depending on parameter distributions. More...

real	ANN_Train (ANN ann, real x, real *t)
	Perform mean square error training, where the aim is to minimise the cost function \(\sum_i \|f(x_i)-t_i\|^2\), where \(x_i\) is input data, \(f(\cdot)\) is the mapping performed by the neural network, \(t_i\) is the desired output and \(i\) denotes the example index. More...

real	ANN_Delta_Train (ANN ann, real delta, real TD=0.0)
	Minimise a custom cost function. More...

void	ANN_SetBatchMode (ANN *ann, bool batch)
	Set batch updates. More...

void	ANN_BatchAdapt (ANN *ann)
	Adapt the parameters after a series of patterns has been seen. More...

real	ANN_Test (ANN ann, real x, real *t)
	Given an input and test pattern, return the MSE between the network's output and the test pattern. More...

real *	ANN_GetOutput (ANN *ann)
	Get the output for the current input. More...

real	ANN_GetError (ANN *ann)
	Get the error for the current input/output pair. More...

real *	ANN_GetErrorVector (ANN *ann)
	Return the error vector for pattern. More...

Layer *	ANN_AddLayer (ANN ann, int n_inputs, int n_outputs, real x)
	Low-level code to add a weighted sum layer. More...

Layer *	ANN_AddRBFLayer (ANN ann, int n_inputs, int n_outputs, real x)
	Low-level code to add an RBF layer. More...

void	ANN_FreeLayer (void *l)
	Free this layer - low level. More...

void	ANN_FreeLayer (Layer *l)
	Free this layer - low level. More...

void	ANN_CalculateLayerOutputs (Layer *current_layer, bool stochastic=false)
	Calculate layer outputs. More...

real	ANN_Backpropagate (LISTITEM p, real d, bool use_eligibility=false, real TD=0.0)
	d are the derivatives at the outputs. More...

void	ANN_RBFCalculateLayerOutputs (Layer *current_layer, bool stochastic=false)
	Calculate layer outputs. More...

real	ANN_RBFBackpropagate (LISTITEM p, real d, bool use_eligibility=false, real TD=0.0)
	Backpropagation for an RBF layer. More...

void	ANN_LayerBatchAdapt (Layer *l)
	Perform batch adaptation. More...

real	Exp (real x)
	Exponential hook. More...

real	Exp_d (real x)
	Exponential derivative hook. More...

real	htan (real x)
	Hyperbolic tangent hook. More...

real	htan_d (real x)
	Hyperbolic tangent derivative hook. More...

real	dtan (real x)
	Discrete htan hook. More...

real	dtan_d (real x)
	Discrete htan derivative hook. More...

real	linear (real x)
	linear hook More...

real	linear_d (real x)
	linear derivative hook More...

real	ANN_LayerShowWeights (Layer *l)
	Dump the weights of a particular layer on stdout. More...

real	ANN_ShowWeights (ANN *ann)
	Dump the weights on stdout. More...

void	ANN_ShowOutputs (ANN *ann)
	Dump outputs to stdout. More...

real	ANN_ShowInputs (ANN *ann)
	Dump inputs to all layers on stdout. More...

real	ANN_LayerShowInputs (Layer *l)
	Dump inputs to a particular layer on stdout. More...

Detailed Description

A neural network implementation.

A neural network is a parametric function composed of a number of 'layers'. Each layer can be expressed as a function \(g(y) =g(\sum_i w_i f_i (x))\), where the \(w\) are a set of weights and \(f(\cdot)\) is a set of basis functions. The basis functions can be fixed or they can be another layer. The neural network can be adapted to minimise some cost criterion \(C\) (defined on some data) via gradient descent. The gradient of the cost with respect to the data is \(\partial C/\partial x\). By expanding this with the chain rule, we have: \(\partial C/\partial x = \partial g/\partial y \partial y/\partial w \partial w/\partial x\).

Definition in file ANN.h.

Typedef Documentation

◆ ANN

typedef struct ANN_ ANN

ANN management structure.

◆ Connection

typedef struct Connection_ Connection

A linear connection between two neural elements.

It is used to express the sum weights in \(y = \sum_i w_i f_i (x)\). This type of connection currently also supports eligibility traces gradient descent updates, batch updates and variance estimates.

◆ Layer

typedef struct Layer_ Layer

A collection of connections from one layer to another, plus management functions and data.

◆ RBFConnection

typedef struct RBFConnection_ RBFConnection

An RBF connection between two neural elements.

It is used to express: \(y = \sum_i \big((m_i-f_i(x)) w_i\big)^2\). This type of connection has no extra features. When an RBF connection is created through the standard high-level function AddRBFHiddenLayer, then the layer output is \(g(y) = e^y\).

Function Documentation

◆ ANN_AddHiddenLayer()

int ANN_AddHiddenLayer	(	ANN *	ann,
		int	n_nodes
	)

Add a hidden layer with n_nodes.

Definition at line 111 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_AddLayer()

Layer* ANN_AddLayer	(	ANN *	ann,
		int	n_inputs,
		int	n_outputs,
		real *	x
	)

Low-level code to add a weighted sum layer.

Definition at line 152 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_AddRBFHiddenLayer()

int ANN_AddRBFHiddenLayer	(	ANN *	ann,
		int	n_nodes
	)

Add an RBF layer with n_nodes.

Definition at line 131 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_AddRBFLayer()

Layer* ANN_AddRBFLayer	(	ANN *	ann,
		int	n_inputs,
		int	n_outputs,
		real *	x
	)

Low-level code to add an RBF layer.

Definition at line 240 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_Backpropagate()

real ANN_Backpropagate	(	LISTITEM *	p,
		real *	d,
		bool	use_eligibility = `false`,
		real	TD = `0.0`
	)

d are the derivatives at the outputs.

Definition at line 610 of file ANN.cpp.

◆ ANN_BatchAdapt()

void ANN_BatchAdapt ( ANN * ann )

Adapt the parameters after a series of patterns has been seen.

Definition at line 923 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_CalculateLayerOutputs()

void ANN_CalculateLayerOutputs	(	Layer *	current_layer,
		bool	stochastic = `false`
	)

Calculate layer outputs.

Definition at line 449 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_Delta_Train()

real ANN_Delta_Train	(	ANN *	ann,
		real *	delta,
		real	TD
	)

Minimise a custom cost function.

The argument delta is a the derivative of the cost function with respect to the neural network outputs. In this case you must call ANN_Input() yourself. As an example, to use MSE cost with ANN_Delta_Train(), you should call ANN_Input() with your data, call ANN_GetOutput() to take a look at the outputs and then call ANN_DeltaTrain() with a real vector delta equal to the target vector minus the ANN output vector. The argument TD is useful if you want to separate the calculation of the derivative of the output with respect to the parameters from the derivative the cost function with respect to the output. This is done in reinforcement learning with eligibility traces for example, where the cost function depends upon previous outputs.

Definition at line 584 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_FreeLayer() [1/2]

void ANN_FreeLayer ( void * l )

Free this layer - low level.

Definition at line 313 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_FreeLayer() [2/2]

void ANN_FreeLayer ( Layer * l )

Free this layer - low level.

Definition at line 322 of file ANN.cpp.

◆ ANN_GetError()

real ANN_GetError ( ANN * ann )

Get the error for the current input/output pair.

Definition at line 833 of file ANN.cpp.

◆ ANN_GetErrorVector()

real* ANN_GetErrorVector ( ANN * ann )

Return the error vector for pattern.

Definition at line 847 of file ANN.cpp.

◆ ANN_GetOutput()

real* ANN_GetOutput ( ANN * ann )

Get the output for the current input.

Definition at line 824 of file ANN.cpp.

◆ ANN_Init()

int ANN_Init ( ANN * ann )

Initialise neural network.

Call this function after you have added all the layers. It adds an extra output layer.

Definition at line 346 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_Input()

real ANN_Input	(	ANN *	ann,
		real *	x
	)

Give an input vector to the neural network.

Calculate a new output given the input. If the vector length is incorrect, you will have trouble.

Definition at line 406 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_LayerBatchAdapt()

void ANN_LayerBatchAdapt ( Layer * l )

Perform batch adaptation.

Definition at line 773 of file ANN.cpp.

◆ ANN_LayerShowInputs()

real ANN_LayerShowInputs ( Layer * l )

Dump inputs to a particular layer on stdout.

Definition at line 996 of file ANN.cpp.

◆ ANN_LayerShowWeights()

real ANN_LayerShowWeights ( Layer * l )

Dump the weights of a particular layer on stdout.

Definition at line 958 of file ANN.cpp.

◆ ANN_RBFBackpropagate()

real ANN_RBFBackpropagate	(	LISTITEM *	p,
		real *	d,
		bool	use_eligibility,
		real	TD
	)

Backpropagation for an RBF layer.

Definition at line 727 of file ANN.cpp.

◆ ANN_RBFCalculateLayerOutputs()

void ANN_RBFCalculateLayerOutputs	(	Layer *	current_layer,
		bool	stochastic = `false`
	)

Calculate layer outputs.

Definition at line 503 of file ANN.cpp.

◆ ANN_Reset()

void ANN_Reset ( ANN * ann )

Resets the eligbility traces and batch updates.

Definition at line 379 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetBatchMode()

void ANN_SetBatchMode	(	ANN *	ann,
		bool	batch
	)

Set batch updates.

Definition at line 906 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetLambda()

void ANN_SetLambda	(	ANN *	ann,
		real	lambda
	)

Set lambda, eligibility decay.

Definition at line 872 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetLearningRate()

void ANN_SetLearningRate	(	ANN *	ann,
		real	a
	)

Set the learning rate to a.

Definition at line 856 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetOutputsToLinear()

void ANN_SetOutputsToLinear ( ANN * ann )

Set outputs to linear.

Definition at line 1033 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetOutputsToTanH()

void ANN_SetOutputsToTanH ( ANN * ann )

Set outputs to hyperbolic tangent.

Definition at line 1050 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_SetZeta()

void ANN_SetZeta	(	ANN *	ann,
		real	zeta
	)

Set zeta, parameter variance smoothing.

Useful for ANN_StochasticInput()

Definition at line 890 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_ShowInputs()

real ANN_ShowInputs ( ANN * ann )

Dump inputs to all layers on stdout.

Definition at line 979 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_ShowOutputs()

void ANN_ShowOutputs ( ANN * ann )

Dump outputs to stdout.

Definition at line 1017 of file ANN.cpp.

◆ ANN_ShowWeights()

real ANN_ShowWeights ( ANN * ann )

Dump the weights on stdout.

Definition at line 941 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_StochasticInput()

real ANN_StochasticInput	(	ANN *	ann,
		real *	x
	)

Stochastically generate an output, depending on parameter distributions.

This is an option for people that understand what they are doing.

Definition at line 429 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_Test()

real ANN_Test	(	ANN *	ann,
		real *	x,
		real *	t
	)

Given an input and test pattern, return the MSE between the network's output and the test pattern.

Definition at line 802 of file ANN.cpp.

Here is the call graph for this function:

◆ ANN_Train()

real ANN_Train	(	ANN *	ann,
		real *	x,
		real *	t
	)

Perform mean square error training, where the aim is to minimise the cost function \(\sum_i |f(x_i)-t_i|^2\), where \(x_i\) is input data, \(f(\cdot)\) is the mapping performed by the neural network, \(t_i\) is the desired output and \(i\) denotes the example index.

Under mild assumptions, this is equivalent to minimising \(E\{|f(X)-T|^2\}\), the expected value of the squared error.

Definition at line 544 of file ANN.cpp.

Here is the call graph for this function:

◆ DeleteANN()

int DeleteANN ( ANN * ann )

Delete a neural network.

Definition at line 77 of file ANN.cpp.

Here is the call graph for this function:

◆ dtan()

real dtan ( real x )

Discrete htan hook.

Definition at line 1105 of file ANN.cpp.

◆ dtan_d()

real dtan_d ( real x )

Discrete htan derivative hook.

Definition at line 1119 of file ANN.cpp.

◆ Exp()

real Exp ( real x )

Exponential hook.

Definition at line 1068 of file ANN.cpp.

◆ Exp_d()

real Exp_d ( real x )

Exponential derivative hook.

Definition at line 1077 of file ANN.cpp.

◆ htan()

real htan ( real x )

Hyperbolic tangent hook.

Definition at line 1086 of file ANN.cpp.

◆ htan_d()

real htan_d ( real x )

Hyperbolic tangent derivative hook.

Definition at line 1095 of file ANN.cpp.

◆ linear()

real linear ( real x )

linear hook

Definition at line 1133 of file ANN.cpp.

◆ linear_d()

real linear_d ( real x )

linear derivative hook

Definition at line 1142 of file ANN.cpp.

◆ LoadANN() [1/2]

ANN* LoadANN ( char * filename )

Load an ANN from a filename.

Definition at line 1171 of file ANN.cpp.

Here is the call graph for this function:

◆ LoadANN() [2/2]

ANN* LoadANN ( FILE * f )

Load the ANN from a C file handle.

Definition at line 1194 of file ANN.cpp.

Here is the call graph for this function:

◆ NewANN()

ANN* NewANN	(	int	n_inputs,
		int	n_outputs
	)

Create a new ANN.

Definition at line 25 of file ANN.cpp.

Here is the call graph for this function:

◆ SaveANN() [1/2]

int SaveANN	(	ANN *	ann,
		char *	filename
	)

Save the ANN to a filename.

Definition at line 1182 of file ANN.cpp.

Here is the call graph for this function:

◆ SaveANN() [2/2]

int SaveANN	(	ANN *	ann,
		FILE *	f
	)

Save the ANN to a C file handle.

Definition at line 1249 of file ANN.cpp.

Here is the call graph for this function:

Classes

Typedefs

Functions

Detailed Description

Typedef Documentation

◆ ANN

◆ Connection

◆ Layer

◆ RBFConnection

Function Documentation

◆ ANN_AddHiddenLayer()

◆ ANN_AddLayer()

◆ ANN_AddRBFHiddenLayer()

◆ ANN_AddRBFLayer()

◆ ANN_Backpropagate()

◆ ANN_BatchAdapt()

◆ ANN_CalculateLayerOutputs()

◆ ANN_Delta_Train()

◆ ANN_FreeLayer() [1/2]

◆ ANN_FreeLayer() [2/2]

◆ ANN_GetError()

◆ ANN_GetErrorVector()

◆ ANN_GetOutput()

◆ ANN_Init()

◆ ANN_Input()

◆ ANN_LayerBatchAdapt()

◆ ANN_LayerShowInputs()

◆ ANN_LayerShowWeights()

◆ ANN_RBFBackpropagate()

◆ ANN_RBFCalculateLayerOutputs()

◆ ANN_Reset()

◆ ANN_SetBatchMode()

◆ ANN_SetLambda()

◆ ANN_SetLearningRate()

◆ ANN_SetOutputsToLinear()

◆ ANN_SetOutputsToTanH()

◆ ANN_SetZeta()

◆ ANN_ShowInputs()

◆ ANN_ShowOutputs()

◆ ANN_ShowWeights()

◆ ANN_StochasticInput()

◆ ANN_Test()

◆ ANN_Train()

◆ DeleteANN()

◆ dtan()

◆ dtan_d()

◆ Exp()

◆ Exp_d()

◆ htan()

◆ htan_d()

◆ linear()

◆ linear_d()

◆ LoadANN() [1/2]

◆ LoadANN() [2/2]

◆ NewANN()

◆ SaveANN() [1/2]

◆ SaveANN() [2/2]