Preskoči na glavno vsebino

Border Pairs Method—constructive MLP learning classification algorithm



Border Pairs Method (BPM) is a new constructive method for supervised learning of multilayer perceptron (MLP), which calculates, values of weights and biases directly from the geometry of learning patterns.



To determine BPM’s capabilities, we compared it with three other supervised machine learning methods: Backpropagation , SVM  and Decision Trees. The comparison were made on six databases: XOR, Triangle, Iris, Pen-Based Recognition of Handwritten Digits, Online Pen-Based Recognition of Handwritten Digits and synthetically generated noisy data. Border Pairs Method found near minimal MLP architecture in all described cases. For classification of the Iris Setosa only two border pairs (only four patterns out of 150) were enough for learning the whole data set correctly. In the classification of ‘Pen-Based Recognition of Handwritten Digits’ dataset only 200 learning patterns were used for learning. The BPM correctly identified more than 95% from 3498 handwritten digits, which did not participate in the learning proces. We are particularly satisfied with the results of online learning. The adaptability of the BPM to new incoming patterns is so high, that it easily follows and sometimes even exceeds the offline version of the algorithm. Even with a noisy data, some very good results were obtained. RMSE error of the BPM was significantly lower than that of Backpropagation, especially for the noise lower than 10%.
The initial results of the research are  good starting point for further research work. Some possible directions  are:
·      Noise reduction using border pairs.
·      Algorithm improvement - better integration of border pairs.


Link to the scientific paper
Link to the Book

Bojan PLOJ, PhD



Komentarji

Priljubljene objave iz tega spletnega dnevnika

Bipropagation demo in TensorFlow

Bipropagation is a new Deep Learning algorithm. It is much faster and much more reliable than Backpropagation. Here is the demo from the  ResearchGate and GitHub. Inner layers of the Neural Network have not hidden anymore. Learning is done layer by layer with much fewer iterations. Please cite me in your work.


Click the G+button if you like this demo. Any comments are desirable.

A new Deep Learning Algorithm: One-Step Method

We are living in the AI era where progres is faster and faster each and every single day. Here is another one discovery in this field: One Step Method, a new machine learning algorithm which can do many things, amongst other can replace digital circuits with neurons, can find the even better construction of neural network than Border Pairs Method. More you can find in the 3rd chapter of our book: Machine Learning: Advances in Research and Applications from Nova Science Publishers.




This new algorithm is also suitable for Deep Learning in combination with other methods like convolutional learning, bipropagation, border pairs method, autoencoder and others.

Po poteh nekega algoritma

Ko sem med raziskovanjem za potrebe podiplomskega študija dobil idejo za nov algoritem strojnega učenja, me je prevzel notranji nemir. Zaslutil sem, da sem na sledi pomembnega odkritja in v hipu sem začutil kako se mi po žilah pretaka adrenalin. Pravijo, da je raziskovalna strast lahko večja  celo od tiste hazarderske,  ki je menda zakrivila številne zgodbe iz črne kronike. No, na vso srečo pa raziskovalna strast ni povezana s tako nizkotnimi pobudami kot hazarderska. Ideji algoritma je nato sledil njegov razvoj, ki je trajal več kot leto in je bil prežet s številnimi vzponi in padci. Navidezne težavice so pogosto preraščale v težave, a na srečo se je vedno našla rešitev za njih. V meni sta se tako prepletala dvom in radost, dokler eksperimenti niso potrdili vseh mojih pričakovanj. Takrat so me preplavili prijetni občutki vznesenosti, ki bi jih lahko primerjali z nekakšno zaljubljenostjo. Ko si vznesen si stvarnost slikaš lepšo, kot je v resnici in tako sem naivno pričakoval, da bo s…