Random neural network

The random neural network (RNN) is a mathematical representation of an interconnected network of neurons or cells which exchange spiking signals. It was invented by Erol Gelenbe and is linked to the G-network model of queueing networks as well as to Gene Regulatory Network models. Each cell state is represented by an integer whose value rises when the cell receives an excitatory spike and drops when it receives an inhibitory spike. The spikes can originate outside the network itself, or they can come from other cells in the networks. Cells whose internal excitatory state has a positive value are allowed to send out spikes of either kind to other cells in the network according to specific cell-dependent spiking rates. The model has a mathematical solution in steady-state which provides the joint probability distribution of the network in terms of the individual probabilities that each cell is excited and able to send out spikes. Computing this solution is based on solving a set of non-linear algebraic equations whose parameters are related to the spiking rates of individual cells and their connectivity to other cells, as well as the arrival rates of spikes from outside the network. The RNN is a recurrent model, i.e. a neural network that is allowed to have complex feedback loops.

A highly energy-efficient implementation of random neural networks was demonstrated by Krishna Palem et al. using the Probabilistic CMOS or PCMOS technology and was shown to be c. 226–300 times more efficient in terms of Energy-Performance-Product.^[1]

RNNs are also related to artificial neural networks, which (like the random neural network) have gradient-based learning algorithms. The learning algorithm for an n-node random neural network that includes feedback loops (it is also a recurrent neural network) is of computational complexity O(n^3) (the number of computations is proportional to the cube of n, the number of neurons). The random neural network can also be used with other learning algorithms such as reinforcement learning can also be used. The RNN has been shown to be a universal approximator for bounded and continuous functions.

See also

• Linear-nonlinear-Poisson cascade model

References and sources

References

Sources

• E. Gelenbe, Random neural networks with negative and positive signals and product form solution, Neural Computation, vol. 1, no. 4, pp. 502–511, 1989.
• E. Gelenbe, Stability of the random neural network model, Neural Computation, vol. 2, no. 2, pp. 239–247, 1990.
• E. Gelenbe, A. Stafylopatis, and A. Likas, Associative memory operation of the random network model, in Proc. Int. Conf. Artificial Neural Networks, Helsinki, pp. 307–312, 1991.
• E. Gelenbe, F. Batty, Minimum cost graph covering with the random neural network, Computer Science and Operations Research, O. Balci (ed.), New York, Pergamon, pp. 139–147, 1992.
• E. Gelenbe, Learning in the recurrent random neural network, Neural Computation, vol. 5, no. 1, pp. 154–164, 1993.
• E. Gelenbe, V. Koubi, F. Pekergin, Dynamical random neural network approach to the traveling salesman problem, Proc. IEEE Symp. Syst., Man, Cybern., pp. 630–635, 1993.
• E. Gelenbe, C. Cramer, M. Sungur, P. Gelenbe "Traffic and video quality in adaptive neural compression", Multimedia Systems, 4, 357–369, 1996.
• C. Cramer, E. Gelenbe, H. Bakircioglu Low bit rate video compression with neural networks and temporal sub-sampling, Proceedings of the IEEE, Vol. 84, No. 10, pp. 1529–1543, October 1996.
• E. Gelenbe, T. Feng, K.R.R. Krishnan Neural network methods for volumetric magnetic resonance imaging of the human brain, Proceedings of the IEEE, Vol. 84, No. 10, pp. 1488–1496, October 1996.
• E. Gelenbe, A. Ghanwani, V. Srinivasan, "Improved neural heuristics for multicast routing", IEEE J. Selected Areas in Communications, 15, (2), 147–155, 1997.
• E. Gelenbe, Z. H. Mao, and Y. D. Li, "Function approximation with the random neural network", IEEE Trans. Neural Networks, 10, (1), January 1999.
• E. Gelenbe, J.M. Fourneau '"Random neural networks with multiple classes of signals", Neural Computation, 11, 721–731, 1999.
• Ugur Halici "Reinforcement learning with internal expectation for the random neural network", European Journal of Operational Research 126 (2): 288–307, 2000.
• Aristidis Likas, Andreas Stafylopatis "Training the random neural network using quasi-Newton methods", European Journal of Operational Research 126 (2): 331–339, 2000.
• Samir Mohamed, Gerardo Rubino, Martín Varela "Performance evaluation of real-time speech through a packet network: a random neural networks-based approach", Perform. Eval. 57 (2): 141–161, 2004.
• E. Gelenbe, Z.-H. Mao and Y-D. Li "Function approximation by random neural networks with a bounded number of layers", 'Differential Equations and Dynamical Systems', 12 (1&2), 143–170, Jan. April 2004.
• Gerardo Rubino, Pierre Tirilly, Martín Varela "Evaluating Users' Satisfaction in Packet Networks Using Random Neural Networks", ICANN (1) 2006: 303–312, 2006.
• Gülay Öke and Georgios Loukas. A denial of service detector based on maximum likelihood detection and the random neural network. Computer Journal, 50(6):717–727, November 2007.
• S. Timotheou. Nonnegative least squares learning for the random neural network. In Proceedings of the 18th International Conference on Artificial Neural Networks, Prague, Czech Republic, pages 195–204, 2008.
• S. Timotheou. A novel weight initialization method for the random neural network. In Fifth International Symposium on Neural Networks (ISNN), Beijing, China, 2008.
• Stelios Timotheou "The Random Neural Network: A Survey", Comput. J. 53 (3): 251–267, 2010.
• Pedro Casas, Sandrine Vaton "On the use of random neural networks for traffic matrix estimation in large-scale IP networks", IWCMC 2010: 326–330, 2010.
• Sebastián Basterrech, Samir Mohamed, Gerardo Rubino, Mostafa Soliman "Levenberg-Marquardt Training Algorithms for Random Neural Networks", Comput. J. 54 (1): 125–135, 2011.
• Michael Georgiopoulos, Cong Li and Taskin Kocak "Learning in the feed-forward random neural network: A critical review", Performance Evaluation, 68 (4): 361–384, 2011.