PDA

View Full Version : recurrent neural network



cdenis
11-30-2007, 08:56 AM
I am using the C# library and in the documentation (white papers) you talk about recurrent neural network, but they are not available in the list.

I would like to know if it is possible to do a recurrent neural network, and if so, I would like to know if we need to program Forecast and GetForecastGradient functions on our own or if we could use some other way to facilitate this part of the programming.

Thanks for your help,

cdenis



public class RecurrentNetwork : Imsl.DataMining.Neural.Network
{
public RecurrentNetwork()
{
}
private int _numberOfInputs;
public override int NumberOfInputs
{
get { return _numberOfInputs; }

}


private int _numberOfOutputs;
public override int NumberOfOutputs
{
get { return _numberOfOutputs; }

}

private InputLayer _inputLayer;

public override InputLayer InputLayer
{
get { return _inputLayer; }

}

private OutputLayer _outputLayer;
public override OutputLayer OutputLayer
{
get { return _outputLayer; }

}

private HiddenLayer[] _hiddenLayers;
public HiddenLayer[] HiddenLayers
{
get { return _hiddenLayers; }
set { _hiddenLayers = value; }
}


private Perceptron[] _perceptrons;

public override Perceptron[] Perceptrons
{
get { return _perceptrons; }
}

private Link[] _links;
public override Link[] Links
{
get { return _links; }
}

private int _numberOfLinks;
public override int NumberOfLinks
{
get
{ return _numberOfLinks; }
}

private double[] _weights;
public override double[] Weights
{
get
{
return _weights;
}
set
{
_weights = value;
}
}

private int _numberOfWeights;
public override int NumberOfWeights
{
get { return _numberOfWeights; }
}

public override HiddenLayer CreateHiddenLayer()
{
return _hiddenLayers[0];
}


public override double[] Forecast(double[] x)
{
double[] dblArray = null;

return dblArray;
}

public override double[,] GetForecastGradient(double[] x)
{
double[,] dblArray = null;

return dblArray;
}
}

afernandez
11-30-2007, 01:47 PM
Hi,

To your question, it is possible to represent a fairly arbitrary neural network topology employing methods in the FeedforwardNetwork class such as Link and LinkAll (manual, pp. 1004-1008). Thus, it is quite possible to represent a Recurrent Artificial Neural Network using the C# Numerical Libraries. Just make the appropriate links.

However, we do not have any training methods specifically for these kinds of networks; therefore, you may have to write your own. Of course, you can bootstrap your development by using one of our feedforward artificial neural network training methods as a starting point, since so many recurrent training methods iterate around the traditional training methods.

Now, if your interest is modelling time series, bear in mind that while you can use a recurrent neural network, you can also use a Feedforward Neural Network with lagged inputs. We provide methods to help "massage" your data into the proper format for that approach, as well as examples - cf. TimeSeries Class and TimeSeriesFilter Class, pp. 1124-1148 of the manual.

cdenis
12-10-2007, 06:32 PM
Thanks for the info.

It's a bit sad that it is not yet available. Hopefully, it will be available in future versions.

juanchoc
02-21-2008, 07:22 PM
Hello i was going to ask the same question cdenis asked.
Im not quite sure i understand how "...it is quite possible to represent a Recurrent Artificial Neural Network using the C# Numerical Libraries. Just make the appropriate links.", if when we try linking a node in a higher layer to a lower one, we get a exception saying "Only forward links are allowed, but the link is from layer 2 to layer 1.".



Dim hiddenLayers As HiddenLayer() = network.HiddenLayers
Dim layer1Node As Node() = hiddenLayers(0).Nodes
Dim layer2Node As Node() = hiddenLayers(1).Nodes
network.Link(layer2Node(0), layer1Node(0))


Is there another way that im not aware of? I looked over and over the help files but couldnt find any reference to this.

By the way, recurrent networks work much better and are much more dynamic than time delay feed forward networks. For example, using delays you are limited to N number of delays per input (where N is the delay you selected), that ends up making training a lot slower and overfitting chances grow. When using recurrent architectures, first you have a lot of different approachs to implement it (fully recurrent or partial recurrent, output error recurrency, output recurrency, hidden layers recurrency, just to name a few), and each approach has diferent benefits (and by far you can exceed the results of a FFTD network.

Thanks,
JCC