Multilevel Perceptron Neural Network

Description of Work:

For years people have tried to make computers that learn things the way that people do, yet today, such a "multilevel perceptron neural network" is still an unconventional type of computing machine. It is particularly good at solving problems for which we have incomplete information, for example, handwriting or voice recognition or making robots walk in a "natural-looking" way on two legs (which is far from solved). More specifically, handwriting recognition, how does one tell an "a" from a "b"? Conventional algorithms to distinguish characters would have to be told exactly what to look for in those characters, which is a very difficult thing to do. Neural networks, on the other hand, can be TRAINED by showing it examples of "a"s and "b"s, and telling it what each character is supposed to be; the programmer does not have to tell the neural network exactly what to look for in each character. However to train a neural network to be able to work correctly at a high rate, it must be shown each training example many, many times over. In fact, using the algorithms that are known in the literature, each example can be shown millions of times and the neural network may still give incorrect answers! In a case like this the training process is said to "not converge,” which typically can occur 30% of the time or more.

Outcomes/Impacts:

With assistance from the current NSF-Idaho EPSCoR project, Dr. Kantabutra’s research group has discovered an algorithm that appears to converge 100% of the time, at least on the hard problems that have been tested thus far. To the best of their knowledge, there has never been an algorithm that converges with this level of reliability (with the exception of algorithms that build the network as the training proceeds). Rediscoveries happen, but thus far their results appear to be new.

Principal Citation:

Vitit Kantabutra, Batsukh Tsendjav, and Elena Zheleva, "Glide Algorithm with Tunneling: a fast, reliably convergent algorithm for neural network training," ANNIE'2003: Artificial Neural Networks in Engineering. Accepted.