GPUMLib

                 About GPUMLib

GPUMLib is an open source Graphic Processing Unit Machine Learning Library. This library aims to provide machine learning researchers and practitioners with a high performance library by taking advantage of the GPU enormous computational power. The library is developed in C++ and CUDA.

Version 0.2.1 includes the following machine learning algorithms and components (complete and working):

• Back-Propagation (BP)
• Multiple Back-Propagation (MBP)
• Non-Negative Matrix Factorization (NMF)
• Semi-Supervised Non-Negative Matrix Factorization (SSNMF)
• Radial Basis Function Networks (RBF)
• Autonomous Training System (ATS) for creating BP and MBP networks
• Neural Selective Input Model (NSIM) for BP and MPB (NSIM allows neural networks to handle missing values directly)
• Restricted Boltzmann Machines (RBMs)
• Deep Belief Networks (DBNs)
• Support Vector Machines (SVMs)

Next version will include:

• Graphical interface (using Qt)

Download

GPUMLib Homepage: http://gpumlib.sourceforge.net/
GPUMLib Download: http://sourceforge.net/projects/gpumlib

Getting Started

Together with the code there is a folder examples, which allows you to use the algorithms. For Windows you can download the executables directly.

Getting Help and Reporting Problems

To get help with GPUMLib or to report a problem please send an email to [e-mail] (Please include GPUMLib keyword in the subject).

References

The following publications describe work integrated in GPUMLib:

• General
  • Noel Lopes and Bernardete Ribeiro. GPUMLib: An efficient open-source GPU machine learning library. International Journal of Computer Information Systems and Industrial Management Applications 3:355-362, 2011.
  • Noel Lopes, Bernardete Ribeiro, and Ricardo Quintas. GPUMLib: A new library to combine machine learning algorithms with graphics processing units. In IEEE 10th International Conference on Hybrid Intelligent Systems (HIS 2010) , pages 229-232, August 2010.
• Back-Propagation, Multiple Back-Propagation and Autonomous Training System
  • Noel Lopes and Bernardete Ribeiro. An evaluation of multiple feed-forward networks on GPUs. International Journal of Neural Systems (IJNS) , 21(1):31-47, 2011.
  • Noel Lopes and Bernardete Ribeiro. Stochastic GPU-based multithread implementation of multiple back-propagation. In Second International Conference on Agents and Artificial Intelligence (ICAART 2010) , pages 271-276, 2010.
  • Noel Lopes and Bernardete Ribeiro. GPU implementation of the multiple back-propagation algorithm. In 10th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2009), LNCS 5788 , pages 449-456. Springer, 2009.
  • Noel Lopes and Bernardete Ribeiro. An efficient gradient-based learning algorithm applied to neural networks with selective actuation neurons. Neural, Parallel and Scientific Computations , 11:253-272, 2003.
  • Noel Lopes and Bernardete Ribeiro. Hybrid learning in a multi-neural network architecture. In INNS-IEEE International Joint Conference on Neural Networks (IJCNN 2001) , volume 4, pages 2788-2793, 2001.
• Neural Selective Input Model (NSIM)
  • Noel Lopes and Bernardete Ribeiro. Handling missing values via a neural selective input model. Neural Network World , 22(4):357-370, 2012.
  • Noel Lopes and Bernardete Ribeiro. A robust learning model for dealing with missing values in many-core architectures. In 10th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA 2011), Part II, LNCS 6594 , pages 108-117. Springer Berlin Heidelberg, 2011.
  • Noel Lopes and Bernardete Ribeiro. A strategy for dealing with missing values by using selective activation neurons in a multi-topology framework. In IEEE World Congress on Computational Intelligence (WCCI 2010) , 2010.
• Deep Belief Networks and Restricted Boltzmann Machines
  • Noel Lopes and Bernardete Ribeiro. Towards adaptive learning with improved convergence of deep belief networks on graphics processing units. Pattern Recognition , 47(1):114-127, 2014.
  • Noel Lopes, Bernardete Ribeiro, and Jo?o Gon?alves. Restricted Boltzmann machines and deep belief networks on multi-core processors. In The 2012 International Joint Conference on Neural Networks (IJCNN) , 2012.
  • Noel Lopes and Bernardete Ribeiro. Improving convergence of restricted Boltzmann machines via a learning adaptive step size. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, LNCS 7441 , pages 511-518. Springer Berlin / Heidelberg, 2012.
• Semi-Supervised Non-Negative Matrix Factorization and Non-Negative Matrix Factorization
  • Noel Lopes and Bernardete Ribeiro. A fast optimized semi-supervised non-negative matrix factorization algorithm. In IEEE International Joint Conference on Neural Networks (IJCNN 2011) , pages 2495-2500, 2011.
  • Noel Lopes and Bernardete Ribeiro. Non-negative matrix factorization implementation using graphic processing units. In 11th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2010), LNCS 6283 , pages 275-283. Springer, 2010.
• Resource Allocating Networks and Radial Basis Functions
  • Bernardete Ribeiro, Ricardo Quintas, and Noel Lopes. Evaluation of a resource allocating network with long term memory using GPU. In 10th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA 2011), Part II, LNCS 6594 , pages 41-47. Springer, 2011.
• Support Vector Machines
• Jo?o Gon?alves, Noel Lopes, and Bernardete Ribeiro. Multi-threaded support vector machines for pattern recognition. In International Conference on Neural Information Processing (ICONIP 2012), Part II, LNCS 7664 , pages 616-623. Springer Berlin / Heidelberg, 2012.

Other related articles:

• Bernardete Ribeiro and Noel Lopes. Extreme learning classifier with deep concepts. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, LNCS 8258 pages 182-189. Springer Berlin / Heidelberg, 2013.
• Shafaatunnur Hasan, Siti~Mariyam Shamsuddin, and Noel Lopes. Soft computing methods for big data problems. In 2013 Symposium on GPU Computing and Applications, 2013.
• Noel Lopes and Bernardete Ribeiro. Towards a hybrid NMF-based neural approach for face recognition on GPUs. International Journal of Data Mining, Modelling and Management (IJDMMM) , 4(2):138-155, 2012.
• Bernardete Ribeiro and Noel Lopes. Deep belief networks for financial prediction. In International Conference on Neural Information Processing (ICONIP 2011), Part III, LNCS 7064 , pages 766-773. Springer, 2011.
• Bernardete Ribeiro, Noel Lopes, and Catarina Silva. High-performance bankruptcy prediction model using graphics processing units. In IEEE World Congress on Computational Intelligence (WCCI 2010) , 2010.
• Noel Lopes and Bernardete Ribeiro. A hybrid face recognition approach using GPUMLib. In 15th Iberoamerican Congress on Pattern Recognition (CIARP 2010), LNCS 6419 , pages 96-103. Springer, 2010.
• Noel Lopes and Bernardete Ribeiro. MBPGPU: A supervised pattern classifier for graphical processing units. In 15th edition of the Portuguese Conference on Pattern Recognition (RECPAD 2009) , 2009.
• Noel Lopes and Bernardete Ribeiro. Fast pattern classification of ventricular arrhythmias using graphics processing units. In 14th Iberoamerican Congress on Pattern Recognition (CIARP 2009), LNCS 5856 , pages 603-610. Springer, 2009.

Citing GPUMLib

If you make use of GPUMLib in your work and want to cite GPUMLib (thank you), we would prefer that you would cite the appropriate papers above, since they form the core of GPUMLib.

Release Notes

Release: 0.2.1:

• Added NMF and SSNMF application (Windows only).
• Added option for excluding modules that depend on CULA (RBF and RAN networks) and compiling GPUMLib without CULA.
• Added option for supporting compilation with CULA dense R17 (free edition)
• Support for CUDA 5.5

Release: 0.2.0:

• Added Support Vector Machines algorithm.
• Corrected bug in BP, MBP, RBM and DBN algorithms (training those networks with more than 65535 samples would fail).
• Support for CUDA 5.0
• Removed dependencies from CUtil

Release: 0.1.9:

• Complete version of RBM and DBN algorithms.

Release: 0.1.8:

• Support for handling missing values directly in BP and MBP networks (using a Neural Selective Input Model - NSIM).
• Prelimininary version of RBM and DBN algorithms.
• Corrected a bug that may cause some reduction functions to fail.

Release: 0.1.7:

• Corrected bugs that may cause an incorrect number of blocks being called.
• Added a class for random numbers generation.
• Added a class for reduction (min, max, average, ...).

Release: 0.1.6:

• Corrected some problems that prevented compilation under linux
• C++ 11 compilation is now off by default (CUDA 4.0 requires gcc 4.4 which does not fully support it).
• Added classes CudaArray and CudaMatrix to the Memory Access Framework.

Release: 0.1.5:

• Changed the Memory Access Framework
• Added an Autonomous Training System (examples)

Release: 0.1.4:

• Changed the Non-Negative Matrix Factorization classess
• Added Radial Basis Function network classes
• Added makefiles

Release: 0.1.3:

• Added Non-Negative Matrix Factorization classes

Release: 0.1.2:

• Host (CPU) and device (GPU) memory access framework:
  • Changed Matrix and Arrays classes.
  • HostArray and DeviceArray classes now support row-major and column-major ordering.
  • DeviceArray class now provides matrix multiplication (requires CUBLAS library).

Release: 0.1.1 implements the following:

• Common framework
  • CudaStream
• Back-Propagation and Multiple Back-Propagation kernels
  • FireLayer
  • FireOutputLayer
  • CalculateLocalGradient
  • CorrectLayerWeights
  • CalculateRMS
  • RobustLearning
• Back-Propagation class
• Multiple Back-Propagation class
• Host (CPU) and device (GPU) memory access framework
  • DeviceAccessibleVariable
  • DeviceArray
  • HostArray
  • HostMatrix
  • DeviceMatrix

Developers

Noel Lopes^1,2
Bernardete Ribeiro¹ (Advisor)
Ricardo Quintas¹
Jo?o Gon?alves<sup>1

1</sup>CISUC - Center for Informatics and Systems of University of Coimbra, Portugal
²UDI/IPG - Research Unit, Polytechnic Institute of Guarda, Portugal

If you are interested in join the development teem, please send an email to [e-mail]