International Journal of Modern Research in Engineering & Management (IJMREM) ||Volume|| 1||Issue|| 5 ||Pages|| 01-06 ||May 2018|| ISSN: 2581-4540
CMAC Neural Networks 1,
Amira Elsir Tayfour Ahmed, 2, Omer Elsir Tayfour Ahmed 1,
2,
Information System Department, King Khalid University, Saudi Arabia Engineering & Networks Department, King Khalid University, Saudi Arabia
-----------------------------------------------------ABSTRACT---------------------------------------------------The Cerebellar Model Articulation Controller (CMAC) is an influential cerebrum propelled processing model in numerous pertinent fields. There are different researches done using CMAC in many applications using its characteristics in easy implementation and good results for example: facial expression recognition, pattern recognition etc. In this paper we have presented some methods of using CMAC and presents their results.
KEYWORDS: Artificial Neural Networks (ANN), Cerebella Model Articulation (CMAC). -----------------------------------------------------------------------------------------------------------------------------------------Date of Submission: Date, 03 May 2017 Date of Accepted: 08 May 2018 ------------------------------------------------------------------------------------------------------------------------------------------
I. INTRODUCTION An artificial neural network (ANN), also called a simulated neural network (SNN) or commonly just neural network (NN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist. There are various types of neural networks one of them is the Cerebellar Model Articulation Controller (CMAC). The CMAC model has been intensively examined and numerous variations of the model such as KCMAC, MCMAC, and LCMAC, have been proposed. The CMAC was first proposed as a function modeler for robotic controllers by James Albus in 1975[1] , but has been extensively used in reinforcement learning and also as for automated classification in the machine learning community. CMAC computes a function , where is the number of input dimensions. The input space is divided up into hyper-rectangles, each of which is associated with a memory cell. The contents of the memory cells are the weights, which are adjusted during training. Usually, more than one quantization of input space is used, so that any point in input space is associated with a number of hyperrectangles, and therefore with a number of memory cells. The output of a CMAC is the algebraic sum of the weights in all the memory cells activated by the input point. The basic operation of a two-input two-output CMAC network is illustrated in fig. (1a). It has three layers, labeled L1, L2, L3 in the figure. The inputs are the values detecting” neurons
for each input
output zero (figure 3.1b). For any input (
and
. Layer 1 contains an array of “feature
. Each of these outputs one for inputs in a limited range, otherwise they a fixed number of neurons (
) in each layer 1 array will be activated
= 5 in the example). The layer 1 neurons effectively quantize the inputs. Layer 2 contains
association
neurons which are connected to one neuron from each layer 1 input array ( ; ). Each layer 2 neuron outputs 1.0 when all its inputs are nonzero, otherwise it outputs zero—these neurons compute the logical AND of their inputs. They are arranged so exactly Layer 3 contains the
are activated by any input (5 in the example).
output neurons, each of which computes a weighted sum of all layer 2 outputs, i.e.: (1)
www.ijmrem.com
IJMREM
Page 1
CMAC Neural Networks.
Figure 1: (a) An example two-input two-output CMAC, in neural network form ( = 2, = 5, = 72, = 2). (b) Responses of the feature detecting neurons for input 1 The parameters
are the weights which parameterize the CMAC mapping (
connects
to output i).
There are weights for every layer 2 association neuron, which makes weights in total. Only a fraction of all the possible association neurons is used. They are distributed in a pattern which conserves weight parameters without degrading the local generalization properties too much. Each layer 2 neuron has a receptive field that is × units in size, i.e. this is the size of the input space region that activates the neuron. The CMAC has the following properties: Limited input space, Piecewise constant, Local generalization, Training sparsity, Training interference, and Multidimensional inflexibility. The advantages when using CMASC is the mapping and training operations are extremely fast, the time taken is proportional to the number of association units, the algorithms are easy to implement and Local generalization prevents over-training in one area of the input space from degrading the mapping in another (unless there are too few physical weights). While the disadvantages are: Many more weight parameters are needed than for, say, the multi-layer perceptron, the generalization is not global, so useful interpolation will only occur if there are enough training points—points further apart than the local generalization distance will not be correctly interpolated, the input-to-output mapping is discontinuous, without analytical derivatives, although this can be remedied with higher order CMACs [2], and the selection of CMAC parameters to prevent excessive hash collision can be a large design problem.
II. METHODS USING CMAC NEURAL NETWORK: Method 1: Facial Expression Recognition using Gabor wavelets and Neural Networks : Amira Tayfour [3] presented method for identifying emotional classification using a combination of texture oriented method with dimensional reduction and use for training three (ANN) which are BPN, SLN Cerebellar Model Articulation Controller (CMAC) for recognizing facial emotions to suit the varieties in the facial emotions and consequently end up being better for untrained facial expressions. Gabor wavelet is used in different angles to extract possible textures of the facial expression. The higher dimensions of the extracted texture features are further reduced by using Fisher’s linear discriminant function for increasing the accuracy of the proposed method. Different facial exprssions considered are angry, disgust, happy, sad, surprise and fear are used. Amira Tayfour [3] found that the combined CMAC provides highest emotion expression identification when compared to emotion expression identification accuracy of FLD/SLN/BPA. Also, the output of CMAC depends upon the number of nodes used in the hidden layer. The results for method in different 6 exaprssions using JAFEE databses are shown in fig. (2)
www.ijmrem.com
IJMREM
Page 2
CMAC Neural Networks.
Figure (2) The Accuracy Percentage for the 6 Experssins uning CMAC Method 2: Hazardous Odor Recognition by CMAC Based Neural Networks: İhsan Ömür Buck, and Bekir Karlık [4], The electronic nose developed in this research consists of a sensor array in which each sensor gives a different electrical response for a particular target vapor introduced into the sensing chamber. Pattern recognition techniques based on the principal component analysis and the CMAC neural network model have been developed for learning different chemical odor vapors. This study has shown the attainability of an electronic nose and the CMAC neural network to distinguish and recognize a portion of the hazardous odors. Hundred percent achievement rate of classification was expert with the outline of CMAC ANN design for hazardous odor recognition system. The other normal MLP design is additionally ready to sum up with high recognition accuracy. However, the training time of MLP is longer than CMAC. The CMAC algorithm is demonstrated as follows: Step 1: Design configuration of the CMAC odor recognition system. Step 2: Normalize, load and input the training data, through quantization, memory addressing, and the weights of the summation of excited memory addresses to produce the output nodes. Step 3: Calculate the difference between actual output and desired output to find the weights, which minimize the error as based on the LMS rule Step 4: Training is done! Save the memory weights. Step 5: Normalize, load and input the testing data, through quantization, memory addressing, and the weights of the summation of excited memory addresses to produce the output nodes. (If the input signals are the same as the training patterns, they will excite the same memory addresses.) Step 6: Output the testing result. “Table (1)” Described the results when using four different gases is: Table (1) Results of using four different gases Types of Gas
Recognition rates for Validation (%)
Recognition rate for Test (%)
CO
97
85
Acetone
98
99
Ammonia
99
100
Lighter
98.5
99
www.ijmrem.com
IJMREM
Page 3
CMAC Neural Networks. Method 3: Applying CMAC-Based On-Line Learning to Intrusion Detection :Innovative work of IDSs has been continuous since the mid 1980's and the difficulties looked by planners increment as the focused-on frameworks since more various and complex. The outcomes exhibit the potential for a capable new investigation part of an entire IDS that would be fit for distinguishing priori and from the a priori denial of service attack patterns. Based on the results of the tests that were conducted on James Cannady [5] approach there were several significant advances in the detection of network attacks: •
On-line learning of attack patterns - The approach has shown the capability to quickly learn new attack pattern without the total retraining required in other neural network approaches. This is a critical advantage that could enable the IDS to constantly enhance its explanatory capacity without the prerequisite for outside updates.
•
Extremely accurate in identifying priori attack patterns – The utilization of the dynamic learning factor brought about a normal error of 0.12%, contrasted and a normal error of 15% in existing IDSs. Since other data security segments depend on the exact recognition of computer attacks the capability to precisely recognize network events could incredibly upgrade the overall security of computer systems.
•
Rapid learning of data – The CMAC could precisely recognize the data vectors after just a single training iteration. This is a huge change over other neural system approaches that may require a large number of preparing training iterations to precisely learn patterns of data.
•
Immediate identification of a priori attacks - The approach has exhibited the capability to successfully recognize potential attacks during initial presentation prior to receiving feedback from the protected host. While the error in the reaction was higher than during subsequent presentations of the pattern after feedback had been received, the average error rate of 15.2% is consistent with normal results from existing IDSs. Moreover, the capability of this way to deal with use speculation to give some sign of attacks is favorable position over expert’s system approaches that require a correct match to coded patterns to give an alarm.
•
Adaptive learning algorithm– The utilization of an adaptive learning factor, based on the current state of the protected host, gives the capability to quickly learn new attacks, in this way altogether diminishing learning time in periods when rapid attacks identification is required. Method 3: Applying CMAC-Based On-Line Learning to Intrusion Detection : Innovative work of IDSs has been continuous since the mid 1980's and the difficulties looked by planners increment as the focused-on frameworks since more various and complex. The outcomes exhibit the potential for a capable new investigation part of an entire IDS that would be fit for distinguishing priori and from the a priori denial of service attack patterns. Based on the results of the tests that were conducted on James Cannady [5] approach there were several significant advances in the detection of network attacks:
•
On-line learning of attack patterns - The approach has shown the capability to quickly learn new attack pattern without the total retraining required in other neural network approaches. This is a critical advantage that could enable the IDS to constantly enhance its explanatory capacity without the prerequisite for outside updates.
•
Extremely accurate in identifying priori attack patterns – The utilization of the dynamic learning factor brought about a normal error of 0.12%, contrasted and a normal error of 15% in existing IDSs. Since other data security segments depend on the exact recognition of computer attacks the capability to precisely recognize network events could incredibly upgrade the overall security of computer systems.
•
Rapid learning of data – The CMAC could precisely recognize the data vectors after just a single training iteration. This is a huge change over other neural system approaches that may require a large number of preparing training iterations to precisely learn patterns of data.
•
Immediate identification of a priori attacks - The approach has exhibited the capability to successfully recognize potential attacks during initial presentation prior to receiving feedback from the protected host. While the error in the reaction was higher than during subsequent presentations of the pattern after feedback had been received, the average error rate of 15.2% is consistent with normal results from existing IDSs. Moreover, the capability of this way to deal with use speculation to give some sign of attacks is favorable position over expert’s system approaches that require a correct match to coded patterns to give an alarm. www.ijmrem.com
IJMREM
Page 4
CMAC Neural Networks. •
Adaptive learning algorithm – The utilization of an adaptive learning factor, based on the current state of the protected host, gives the capability to quickly learn new attacks, in this way altogether diminishing learning time in periods when rapid attacks identification is required. The results of the a priori attacks resulted in an average error of the CMAC output of 0.4% (fig. 3)
Figure 3: The Desired Response via Events Method 4: Handwritten Chinese Character Recognition Technology based on CMAC Neural Network Yan Shen, Lina Liu, Guoqiang Li [6] used 2000 Chinese characters as a template for training. The result is found that the handwritten character recognition accuracy rate of writing neat, appropriate proportion and straight strokes was up to 89.76%, however, the free writing recognition effect is not very good, to be corrected manually. With the improvement of the CMAC neural network theory and the depth study of the character feature extraction technology, CMAC neural network will be better used in the field of handwritten character recognition and has a very good application prospects. The figure below describes the flow chart of the proposed system. Fig. 4 described the flow chart for the proposed system.
Figure1: Flowchart of character recognition based on neural network 1000 and 2000 Chinese characters are used as a template for training in the study. It is found that the handwritten character recognition accuracy rate of writing neat, appropriate proportion and straight strokes was up to 95.52 % in using 1000 characters and 89.76% in using 2000 charcaters, “Table” (2) describes the recognition rates of these characters using the BP and CMAC ANN.
www.ijmrem.com
IJMREM
Page 5
CMAC Neural Networks. Table (2) Recognition Rates using BP & CMAC Test Text
Recognition rates
Recognition rates
(1000 Character) (%)
(12000 Character) (%)
Text text1 (BP)
83.68
68.64
Text text2 (CMAC)
95.52
89.76
III. CONCLUSION Four methods were presented as an example in this paper to declare the advantages of using CMAC ANN. Every method has proved and showed the good and accurate results were found when using the CMAC neural network. For Future works for more method using different neural networks comparing with CMAC will be presented.
REFERENCES 1. 2. 3. 4. 5. 6.
J. S. Albus, A theory of cerebellar function, Math. Biosci. 10 pp 25_61, (1971). Albus J.S., Mechanisms of Planning and Problem Solving in the Brain, Mathematical Biosciences, Vol.45, pp.247-293, 1979. Ahmed, Amira Elsir Tayfour, Facial Expression Recognition using Gabor wavelets and Neural Networks, University of Sudan for Science and Tchnologies, Phd thesis, 2016. İhsan Ömür Bucak, and Bekir Karlık , Hazardous Odor Recognition by CMAC Based Neural Networks Sensors 9, 7308-7319; doi:10.3390/s90907308, 2009. James Cannady, Applying CMAC-Based On-Line Learning to Intrusion Detection, Ph.D. Candidate, Nova Southeastern University Fort Lauderdale, FL 33314 Yan Shen, Lina Liu, Guoqiang Li, Handwritten Chinese Character Recognition Technology based on CMAC Neural Network, Atlantis Press, 2013.
www.ijmrem.com
IJMREM
Page 6