Preskoči na glavno vsebino

Border Pairs Method—constructive MLP learning classification algorithm



Border Pairs Method (BPM) is a new constructive method for supervised learning of multilayer perceptron (MLP), which calculates, values of weights and biases directly from the geometry of learning patterns.

To determine BPM’s capabilities, we compared it with three other supervised machine learning methods: Backpropagation , SVM  and Decision Trees. The comparison were made on six databases: XOR, Triangle, Iris, Pen-Based Recognition of Handwritten Digits, Online Pen-Based Recognition of Handwritten Digits and synthetically generated noisy data. Border Pairs Method found near minimal MLP architecture in all described cases. For classification of the Iris Setosa only two border pairs (only four patterns out of 150) were enough for learning the whole data set correctly. In the classification of ‘Pen-Based Recognition of Handwritten Digits’ dataset only 200 learning patterns were used for learning. The BPM correctly identified more than 95% from 3498 handwritten digits, which did not participate in the learning proces. We are particularly satisfied with the results of online learning. The adaptability of the BPM to new incoming patterns is so high, that it easily follows and sometimes even exceeds the offline version of the algorithm. Even with a noisy data, some very good results were obtained. RMSE error of the BPM was significantly lower than that of Backpropagation, especially for the noise lower than 10%.
The initial results of the research are  good starting point for further research work. Some possible directions  are:
·      Noise reduction using border pairs.
·      Algorithm improvement - better integration of border pairs.


Link to the scientific paper
Link to the Book

Bojan PLOJ, PhD



Oznake:

Komentarji

Objavite komentar

Priljubljene objave iz tega spletnega dnevnika

Artificial intelligence

          Artificial intelligence (AI) is a relatively young branch of science that stirs the imagination of many. Even movie directors from hollywood are not exceptions. Development in AI area is very fast and there is no indication that this will change soon. I still remember my first contact with learning devices. This happend at the end of the last millennium when I realized neural networks (NN). They have immediately attracted my attention, because such devices were not known till then.           NN are made along the lines of mammalian brain. During the learning NN extract the essence from the data. After the learning we can ask NN questions. It gives us the right answers even to questions that during learning did not participate.  NN learns autonomously and therefore may exceed the teacher's (poeple's) knowledge. Here are some important achievements of artificial intelligence: A couple of years ago the co...
7 komentarjev
Preberite več

Bipropagation demonstration in MatLAB

Here is given an example of the "bipropagation" algorithm for learning of NN. It is written in MatLAB language (R2015a) and is as similar as possible  to  "Deep learning" example "AutoencoderDigitsExample.m" which is included in MatLAB's Neural Network Toolbox. So you can easily compare both algorithms. I believe that my algorithm have few advantages over autoencoder. Please tell me what do you think about it. Please cite me in your works. Thanks a lot. Download demo ====================================================== %% Training a Deep Neural Network for Digit Classification % This example shows how to use the Neural Network Toolbox(TM) to train a % deep neural network to classify images of digits and is very similar to % "AutoencoderDigitsExample.m" which is included in % Neuronal Network ToolBox from MatLAB. This example is made for comparison % of both algorithms. % % Neural networks with multiple hidden layers can be usefu...
Objavite komentar
Preberite več