Can anyone help me with these q's/doubts on Neural Networks

[sanjay]

Hi all,

I am just a beginner on neural networks, and so reading books (mainly thanks to elektrodians) on it, I just had some doubts and questions which I was wondering if any of you can help me with ?

1. When designing Neural Networks, as a designer, how do you choose that how many hidden layers you require ? (I know the books say, the least the better, but still just curious, how does one go about deciding the amount of hidden layers required)

2.Regarding activation functions, the same question follows, how do you select, which is the best activation function that would suit your application ?, are there any guidlines which help us to decide like which one to select. In genreal so far, I have seen authors go for mostly SIGMOID function in their books. Is there particular reason for this function ?

3. Regarding back-propagation, in the beginning when patterns are fed
a) How do you go about selecting the value of the weight ? (Is it just like use random values, or is there any particular way)
b) A question on desired output, if for example, one is designing a system where one doesn't actually know wht to expect, in this case, how do you go about setting a desired output ? (Am I right, because at the end of day, the NN will try to get results as much close as possible to this desired output afterall)
c) It is said that in delta rule, the drawback is that not the entire system get's is value changed, what I mean is we do know that the weight value of the units from hidden layer to output layer do get changed, but what alters the values of weights change from input units to the hidden layer units ?

Any suggestions, thoughts would be appreciated.
Regards

 

[brmadhukar]

Re: Can anyone help me with these q's/doubts on Neural Netwo

Hi,

0) Read some amount of adaptive signal processing.

1) No. It is not known really how many layers would be sufficient. The lesser the better fo implementation.

2) Sigmoid because, if the statistic is too low or too high the decision is simple. Near the threshold any decision becomes difficult and sigmaoid helps in minimising the errors, rather than step.

3) I donot remember.

hope this helps,
brmadhukar

 

[anto4]

Re: Can anyone help me with these q's/doubts on Neural Netwo

first get a book like neural network for signal processing,, all of the questions you as are dependent on which type of neural network you want, therefore there is no shortcut,, you are just going to have to read up on neural networks,, all of your three questions are delt with differently (with regards to radial basis networks,, mlps, etc etc,,

 

[as_518]

Re: Can anyone help me with these q's/doubts on Neural Netwo

ther is a poof that 3 layer neural network could approximate any function to any required degree. the number of nodes in the hidden layer is the key point if it is larger than enough the neural network will be overtrained (i.e generalization is very bad) while if itis smaller than enough the neural network will be under learning i.e error is large.

 

[sanjay]

Hi as_518,

is it possible if you can send me that proof document. Would like to go through that document. Regarding overfitting and underlying, I agree, as the project that I am working on right now, is giving me that problem as my training works nicely but during simulation encountering problems, so busy playing around with values.

 

[as_518]

Re: Can anyone help me with these q's/doubts on Neural Netwo

you can find it in the second edition of

"neural networks" by simon haykin p208-209

 

[rs_408]

hi
I need:

Neural Networks: A Comprehensive Foundation (2nd Edition) -- by Simon Haykin

book. anybody know how can i find it

 

[leekk8]

Re: Can anyone help me with these q's/doubts on Neural Netwo

sanjay,

The architecture of the backpropagation is dependable. Different application may need different architecture for the best performance. You can try it with different architecture and then get the best architecture.

I've implemented fingerprint recognition system using BP network.