About the Authors Preface Acknowledgments List of Important Symbols and Operators List of Important Abbreviations PARTI Fundamental Neurocomputing Concepts and Selected Neural Network Architectures and Learning Rules 1 Introduction to Neurocomputing 1.1 What Is Neurocomputing? 1.2 Historical Notes 1.3 Neurocomputing and Neuroscience 1.4 Classification of Neural Networks 1.5 Guide to the Book References 2 Fundamental Neurocomputing Concepts 2.1 Introduction 2.2 Basic Models of Artificial Neurons 2.3 Basic Activation Functions 2.4 Hopfield Model of the Artificial Neuron 2.5 Adaline and Madaline 2.6 Simple Perceptron 2.7 Feedforward Multilayer Perceptron 2.8 Overview of Basic Learning Rules for a Single Neuron 2.9 Data Preprocessing Problems References Mapping Networks 3.1 Introduction 3.2 Associative Memory Networks 3.3 Backpropagation Learning Algorithms 3.4 Accelerated Learning Backpropagation Algorithms 3.5 Counterpropagation 3.6 Radial Basis Function Neural Networks Problems References 4 Self-Organizing Networks 4.1 Introduction 4.2 Kohonen Self-Organizing Map 4.3 Learning Vector Quantization 4.4 Adaptive Resonance Theory (ART) Neural Networks Problems References 5 Recurrent Networks and Temporal Feedforward Networks 5.1 Introduction 5.2 Overview of Recurrent Neural Networks 5.3 Hopfield Associative Memory 5.4 Simulated Annealing 5.5 Boltzmann Machine 5.6 Overview of Temporal Feedforward Networks 5.7 Simple Recurrent Network 5.8 Time-Delay Neural Networks 5.9 Distributed Time-Lagged Feedforward Neural Networks Problems References PART II Applications of Neurocomputing 6 Neural Networks for Optimization Problems 6.1 Introduction 6.2 Neural Networks for Linear Programming Problems 6.3 Neural Networks for Quadratic Programming Problems 6.4 Neural Networks for Nonlinear Continuous Constrained Optimization Problems Problems References Solving Matrix Algebra Problems with Neural Networks 7.1 Introduction 7.2 Inverse and Pseudoinverse of a Matrix 7.3 LU Decomposition 7.4 QR Factorization 7.5 Schur Decomposition 7.6 Spectral Factorization - Eigenvalue Decomposition (EVD) (Symmetric Eigenvalue Problem) 7.7 Neural Network Approach for the Symmetric Eigenvalue Problem 7.8 Singular Value Decomposition 7.9 A Neurocomputing Approach for Solving the Algebraic Lyapunov Equation 7.10 A Neurocomputing Approach for Solving the Algebraic Riccati Equation Problems References 8 Solution of Linear Algebraic Equations Using Neural Networks 8.1 Introduction 8.2 Systems of Simultaneous Linear Algebraic Equations 8.3 Least-Squares Solution of Systems of Linear Equations 8.4 A Least-Squares Neurocomputing Approach for Solving Systems of Linear Equations 8.5 Conjugate Gradient Learning Rule for Solving Systems of Linear Equations 8.6 A Generalized Robust Approach for Solving Systems of Linear Equations Corrupted with Noise 8.7 Regularization Methods for Ill-Posed Problems with Ill-Determined Numerical Rank 8.8 Matrix Splittings for Iterative Discrete-Time Methods for Solving Linear Equations 8.9 Total Least-Squares problem 8.10 An L-Norm (Minimax) Neural Network for Solving Linear Equations 8.11 An L1-Norm (Least-Absolute-Deviations) Neural Network for Solving Linear Equations Problems References 9 Statistical Methods Using Neural Networks 9.1 Introduction 9.2 Principal-Component Analysis 9.3 Learning Algorithms for Neural Network Adaptive Estimation of Principal Components 9.4 Principal-Component Regression 9.5 Partial Least-Squares Regression 9.6 A Neural Network Approach for Partial Least-Squares Regression 9.7 Robust PLSR: A Neural Network Approach Problems References 10 Identification, Control, and Estimation Using Neural Networks 10.1 Introduction 10.2 Linear System Representation 10.3 Autoregressive Moving Average Models 10.4 Identification of Linear Systems with ARMA Models 10.5 Parametric System Identification of Linear Systems Using PLSNET 10.6 Nonlinear System Representation 10.7 Identification and Control of Nonlinear Dynamical Systems 10.8 Independent-Component Analysis: Blind Separation of Unknown Source Signals 10.9 Spectrum Estimation of Sinusoids in Additive Noise 10.10 Other Case Studies Problems References App A Mathematical Foundation for Neurocomputing A.1 Introduction A.2 Linear Algebra A.3 Principles of Multivariable Analysis A.4 Lyapunov's Direct Method A.5 Unconstrained Optimization Methods A.6 Constrained Nonlinear Programming A.7 Random Variables and Stochastic Processes A.8 Fuzzy Set Theory A.9 Selected Trigonometric Identities References Name Index Subject Index