online read us now
Paper details
Number 4 - December 2018
Volume 28 - 2018
Comparison of prototype selection algorithms used in construction of neural networks learned by SVD
Norbert Jankowski
Abstract
Radial basis function networks (RBFNs) or extreme learning machines (ELMs) can be seen as linear combinations of
kernel functions (hidden neurons). Kernels can be constructed in random processes like in ELMs, or the positions of
kernels can be initialized by a random subset of training vectors, or kernels can be constructed in a (sub-)learning process
(sometimes by k-means, for example). We found that kernels constructed using prototype selection algorithms provide very
accurate and stable solutions. What is more, prototype selection algorithms automatically choose not only the placement of
prototypes, but also their number. Thanks to this advantage, it is no longer necessary to estimate the number of kernels with
time-consuming multiple train-test procedures. The best results of learning can be obtained by pseudo-inverse learning
with a singular value decomposition (SVD) algorithm. The article presents a comparison of several prototype selection
algorithms co-working with singular value decomposition-based learning. The presented comparison clearly shows that the
combination of prototype selection and SVD learning of a neural network is significantly better than a random selection
of kernels for the RBFN or the ELM, the support vector machine or the kNN. Moreover, the presented learning scheme
requires no parameters except for the width of the Gaussian kernel.
Keywords
radial basis function network, extreme learning machines, kernel methods, prototypes, prototype selection, machine learning, k nearest neighbours