A Novel Approach to Network Intrusion Detection using Spiking Neural Networks by ISERP ISERP

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

A Novel Approach to Network Intrusion Detection using Spiking Neural Networks. Disha Sharma*

Veenu Mangat

Student M.E. (I.T.) 2nd yr. University Institute of Engineering and Technology Panjab University

Assistant Professor (I.T.) University Institute of Engineering and Technology Panjab University

Chandigarh, India

dishasharma210@gmail.com

veenumangat@yahoo.com

IJ A

Abstract - Although the nervous system and nervous cells have been studied since ancient times, almost all the important advances on the knowledge base took place along the past century, when neuroscience evolved and developed more complex and detailed neuron models. However, engineering applications were â&#x20AC;&#x201C; and still are â&#x20AC;&#x201C; practically limited to the basic neuron models and its variations. In traditional artificial neural networks, the neuron behavior is described only in terms of firing rate, while most real neurons, commonly known as spiking neurons, transmit information by pulses, also called action potentials or spikes. From these considerations a major question raises immediately: if we were able to build powerful applications in all fields of engineering using these simple models, what would it be possible to do with the more complex models? This question is the fundamental motivation of the present work. Given the importance of more realistic neuron models, our main objective is to present a general and comprehensive overview of spiking neurons, ranging from biological neuron features to examples of practical applications. The aim of the present work is therefore to highlight what we believe will be one of the main components of the future computing machines: the spiking neuron. Besides the review itself, we present also a novel approach to the spiking neuron network architecture used in intrusion detection.

effectively explored in the molecular level. The research in molecular biology contributed to the knowledge about ion channels and receptors, two important elements in neural signaling, making it possible to describe the first molecular structure of an ionic channel [KSJ00]. This enhanced capacity of neuronal modeling came to shed some light into questions like: How do the nervous cells communicate among them? How is this communication modified by experience? How different interconnection patterns originate different perceptions and motor actions?

Disha Sharma dishasharma210@gmail.com +91-7837448470

Keywords- Spiking Neural Networks, Intrusion detection, data clustering, time-varying inputs, Spike coding.

BACKGROUND

Although the nervous system and nervous cells have been studied since ancient times, almost all the important advances on the knowledge base took place were founded on five experimental disciplines: anatomy, embryology, physiology, pharmacology, and psychology [KSJ00]. Then, beginning in the middle of the past century, engineering was added as a sixth discipline and, reciprocally, neuroscience was also adopted by engineering, ensuing all the development of computational intelligence and making this subject a rather interdisciplinary one.

From engineering point of view, it is clear that answers for all these questions will only be possible with a deeper comprehension of the biological neuron and how they can do fast and reliable computation [MB98]. Although much progress has been achieved in the last two decades, there are still a few fundamental questions, like how do real neurons transmit information or how to use the spike timing efficiently to process information [Nat96, GK02b]. The present knowledge of the nervous system has reached an enormous level of detail, making it impossible even to summarize. Therefore, we limited ourselves to mention only the main components with a brief description of its basic properties. We concisely present some of the most used theoretical neuron models, from the simple ones, like the integrate-and-fire, to the more complex ones, like the compartmental models. In real nervous systems there is a wide range of neuron types, each one assigned to do a specific function, making it virtually impossible to create a model that meets all the requirements. In this work we intend to make a re view on spiking neurons, presenting a comprehensive scenario of the foundations and its application to intrusion detection using clustering methods.

In the last ten years all the main questions formerly addressed by neuroscientists only to cellular biology began to be

ISSN: 2230-7818

Page 36

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

INTRODUCTION

Decades passed by since the introduction of Artificial neural networks (ANN), the fairly old technique is still on the path of improvement coming up with new generations of neuron models. McCulloch-Pitts gave the first generation of ANN based on a simple threshold neuron model: if the sum of its weighted incoming signals rises above a threshold value, a neuron sends a binary „high‟ signal. Although these neurons can only give digital output, they have successfully been implemented in ANN like multi layer perceptrons and Hopfield nets. The next generation used continuous activation function like sigmoid and hyperbolic tangent, making them suitable for analog in- and output, e.g. feed-forward and recurrent neural networks. Neuron models of both generations do not employ individual pulses, but their output typically lies between 0 and 1. These signals can be seen as normalized firing rates of the neuron within a certain period of time called rate coding, where, High rate of coding ↔ high output signal.

Figure.2. shows a spiking neuron with multiple synapse inputs [2]. The inputs comprises of a sequence of unit amplitude digital spikes having a very short duration. A sequence of inputs entering a synapse is called a spike train. The interspike times and their sequence order form the input and the response function is based on Leaky Integrator Fire Neuron (LIFN) model. One of the most exciting characteristics of spiking neural networks, with the potential to create a step-change in our knowledge of neural computation, is that they are embedded in time (Maass 2001). Spike latencies, axonal conduction delays, refractory periods, neuron resonance and network oscillations all give rise to an intrinsic ability to process timevarying data in a more natural and computationally powerful way than is available to 2nd generation models. A spiking neuron can be differentiated by traditional artificial neuron as [3]: 1) Both combinational as well as inter-spike information can be processed by a spiking neuron. 2) A feedback connection is not required by a spiking neuron for sequential input learning. 3) The only requirement is local timing reference.

II.

Fig.1. Artificial Neuron model

Networks of the earlier generations have proven effective at modeling some cognitive processes and have been successful in many engineering applications. However the fidelity of these models with regards to neurophysiologic data is minimal and this has several drawbacks.

IJ A

The third generation [1] raises the level of biological realism by using individual spikes, incorporating spatial-temporal information in communication and computation like real neurons. So instead of using rate coding these neurons use pulse coding mechanism where neurons receive and do send out individual pulses, allowing multiplexing of frequency and amplitude of sound.

Fig.2. Spiking Neuron with spiking train inputs

ISSN: 2230-7818



Neurophysiologic knowledge cannot be integrated easily into the models and as such cannot be tested for applicability to or effect upon neural computation.



Real neurons exhibit a very broad range of behaviors (tonic (continuous) and phasic (once-off) spiking, bursting, spike latency, spike frequency adaptation, resonance, threshold variability, input accommodation and bistability (Izhikevich 2004)). It's unlikely that these behaviors have no computational significance.



There are specific interesting processes occurring at the spike level (such as Spike Timing Dependent Plasticity (Bi and Poo 1998)) that cannot be modeled without spikes.



The dynamics of spiking networks are much richer, allowing for example o

oscillations in network activity which could implement multiple concurrent processing streams (Izhikevich 1999), figure/ground segmentation and binding (Csibra, Davis et al. 2000; Engel, Fries et al. 2001), short term memory (Jensen, Idiart et al. 1996; Jensen, Gelfand et al. 2002) etc

Page 37

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

much increased (perhaps by orders of magnitude) memory capacity (Izhikevich 2005). Transmission delays are very significant for computation particularly because they are random or Gaussian for real neurons - this causes the formation of polychronous (as against synchronous) spiking neuron groups which could possibly store many more population-encoded memories than there are synaptic weights. This idea is still to be fully researched and analyzed. III.

SPIKING NEURON MODELS

A spiking neuron model accounts for the impact of impinging action potentials – spikes – on the targeted neuron in terms of the internal state of the neuron, as well as how this state relates to the spikes the neuron fires. We divide spiking neuron models into four main classes as follows:

The SRM model describes the state of a neuron by a single variable, the membrane potential. It expresses the membrane potential Vi at time t as an integral over the past, including a model of refractoriness. The SRM is phenomenological model of neuron, based on the occurrence of spike emissions. Before any input spike has arrived at the postsynaptic neuron i, the variable Vi (t) has a value 0. The firing of a presynaptic neuron j at time tj(f) evokes a postsynaptic potential in the neuron i modeled by the response. Each incoming spike will perturb the value of Vi and if, after the summation of the inputs, the membrane potential Vi reaches the threshold then an output spike is generated. After the neuron has fired the membrane potential returns to a low value which is described by the refractory period function. B. Conductance based models This is quite complex class of neuron models, based on the simulation of the intricate behavior of ionic channels. As these channels open and close, their conductance changes accordingly, yielding a set of differential equations describing the process. The variations among the models in this class are mostly due to the choice of channels used and the parameters of the resulting differential equations.

A. Threshold Fire Models These models are based on the temporal summation of all contribution to the membrane potential u(t) received from all presynaptic neurons. If this contribution exceeds a threshold θ, then the postsynaptic neuron will fire. Two main models could be considered for this category: Integrate and Fire (I & F) model and the Spike Response Model (SRM).

2) Spike Response Model (SRM)

IJ A

1) Integrate-and-fire Model – I & F The simplest (I&F) model was originally developed when dominant thinking stated that neuron function can be wellenough approximated by simply integrating input and then firing at a given threshold. Properties like spike frequency adaptation, bursting, resonance, latency and variable thresholds were incorporated into models as needs arose and thinking changed. However unfortunately no single I&F model displays all these characteristics. The basic model is also called Leaky Integrate and Fire because the membrane is assumed to be leaky due to ion channels, such that after a PostSynaptic Potential (PSP) the membrane potential approaches again a reset potential urest. Threshold

V in

Reset

Fig. 3.Integrate and Fire model

ISSN: 2230-7818

1) Hodgkin Huxley model Hodgkin and Huxley modeled the electro-chemical information transmission of natural neurons with electrical circuits consisting of capacitors and resistors. Based on the experiments three types of ionic currents are found namely sodium (Na), potassium (K), and a leak current. The first two types are controlled by specific voltage dependent ion channels. The third type takes care of other channels. The cell membrane is a good insulator and acts as a capacitor. Besides the capacitor, there are three resistances, one for each ion channel considered in the model. The conservation of electric charge on a piece of membrane implies that the applied current I(t) may be split in a capacitive current IC which charges the capacitor C and further components Ik which pass through the ion channels. Thus

Output

I(t) = IC(t) +

Ik(t)

where the sum runs over all ion channels. From the definition of a capacity C = Q/u where Q is a charge and u the voltage across the capacitor, we find the charging current IC = C du/dt. Hence

Ik(t) + I(t) .

Page 38

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

is described only in terms of spike rate. A good example of rate model is the Perceptrons.

C. Compartmental modes Compartmental models better capture the complexity of a neuron, taking into account the spatial structure of the dendritic tree and also model the synaptic transmission at a greater level of detail. With this approach, it is possible to consider other ion currents, beyond sodium and potassium currents incorporated in Hodgkin-Huxley model.

Fig.4. Schematic diagram of Hodgkin-Huxley model

Fig.6. Configuration of a Perceptron

IV.

SPIKE CODING

Spiking neurons encode information through their average spike rate over some time window called a rate code. Sensory cells such as in the cochlear and the retina use a rate code (Izhikevich 2005), however response time in the visual cortex is known to be too fast to continue processing with this coding regime (Thorpe, Fize et al. 1996; Thorpe, Delorme et al. 2001) – each neuron in the visual processing hierarchy only has time to fire one or occasionally two spikes prior to recognition, so clearly the visual cortex cannot be using a rate code, but is instead somehow utilizing the presence and/or the timing of spikes called a temporal code.

IJ A

Fig.5. A generic equivalent circuit of a neural compartment

The basic idea is to divide the components into smaller uniform components or compartments. Each compartment is then modeled with equations describing the equivalent electrical circuit. The use of appropriate differential equations for each compartment enables the simulation of their behavior as well as their interactions with other compartments. The notion of an equivalent electrical circuit for a small piece of cellular membrane is the basis for all compartmental models. D. Rate models The rate models, also known as sigmoidal units, are the most traditional and widely used models for the analysis of learning and memory in ANNs. The first choice to be made before the construction of a neuronal model is the level of abstraction and complexity of the model. The class of rate models represents the highest abstraction level, as they neglect the pulse structure of the neuronal systems and the neural activity

ISSN: 2230-7818

There are a number of different coding strategies possible using spike times, shown below in increasing order of information encoding capacity (Thorpe, Delorme et al. 2001).  Count coding: counts the total number of spikes of a neuron population in a given time – similar to rate coding except it entails one spike from each of many neurons instead of many spikes from one neuron; however the information capacity of rate coding is the same (very small).  Binary coding: encodes a binary number, each digit represented by the presence (1) or absence (0) of a spike.  Rank order coding: the order of firing of the neurons encodes the information.  Delay coding: type of temporal coding, to determine time-slice length between minimum and maximum activation levels.

Page 39

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

Binary coding

Rank order coding

Delay coding

Description

counts the total number of spikes

a binary number

the order the neurons fire

the order and delays both encode information

Information capacity (bits)

log2(n+1)

log2(n!)

n.log2(T/p)

Each of these coding strategies has very different information capacities summarized in the table. In the table, n is the number of neurons under consideration, T is the time window over which each neuron can fire either 0 or 1 spikes, and p is the precision, which is the minimum inter-spike interval which can be discerned by postsynaptic neurons. V.

SPIKING NEURAL NETWORKS APPLIED TO INTRUSION DETECTION

An engineering application [15] employs the coincidence detection property of spiking neurons and a Hebbian based delay shifting rule. The application considered is that of control chart pattern recognition. Control chart patterns indicate the state of a process being monitored and can be utilized to detect abnormal behavior of the process. Also spiking neurons can act as a coincidence detector for incoming pulses and can detect coincidence of the input signals with ease unlike classical neural networks where this is computationally expensive to realize [16]. The network architecture shown [15] is a simple two – layered fully connected feed forward network. The input layer has neurons equal to number of input parameters. The output layer is constructed with coincidence detecting spiking neurons. Each feed-forward connection is assigned a random weight and a delay value. Connection delays are adapted through a Hebbian-based rule which enables the inputs from a class of inputs to coincide at some group of neurons.

Any attempt to compromise the integrity, confidentiality or availability of a resource is called an intrusion. Researchers have developed intrusion detection system for various environments depending upon the security concerns of different networks. The function of Intrusion Detection System [4] is to gather and analyze information from various areas within a computer or a network to determine all possible security breaches.

temporal coding were proposed in [8, 9, 10, 11, 12, 13, and 14] and their efficiency was found comparable with popular sigmoidal neural networks.

Spike Coding

Count coding

IJ A

Intrusion detection systems can be of two types: signature based and anomaly based. Signature detection systems are based on pattern matching i.e. they try to match the scenarios with already recorded signatures from the database while anomaly detection techniques compare the behavior of data creating a baseline profile of the normal system, any deviation from the normal data is considered to be an anomaly. Both the approaches have their own pros and cons. In Signature detection the known attacks can be detected reliably with low false positive rates but the major drawback is that such systems require a timely and continuously updated database of signatures for all possible attacks against a network. On the other hand, anomaly detection has two major advantages over signature detection. First, ability to detect unknown attacks as well as “zero day” attacks. Second, every network has its own customized profiles of normal activity, which makes it difficult for the attacker to know with confidence what activities can be carried out without getting detected. Hopfield [5] introduced the idea of using the timing action potentials to represent the values for computation within a network. Maass [6] showed that a network of spiking neurons can simulate arbitrary feed-forward sigmoidal networks and can approximate any continuous functions. Spiking neural networks [7] which convey information by individual spike times are more computationally expressive than networks with sigmoidal activation. Many learning algorithms for SNNs with

ISSN: 2230-7818

Fig.7. Pattern Detection by a Spiking Neural Network

VI.

CLUSTERING SPIKING NEURAL NETWORKS

Clustering relies on a single output neuron firing earlier than the other output neurons for data points from a single cluster. The optimal activation of such an output neuron is achieved when the spikes of input arrive at the output neuron simultaneously.

Page 40

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042

VII. DISCUSSION

REFERENCES

[1] Jilles Vreeken, “Spiking Neural Networks, an introduction”, Adaptive Intelligence Laboratory, Institute of Information and computing science, Utrecht University.

The figures show how outlined encoding allows for increased capacity, e.g., by encoding variables with more neurons, many different clusters can be separated. In Fig.8 input has two separately encoded variables and found that a network with 24 input neurons was easily capable of correctly classifying 17 evenly distributed clusters, demonstrating a significant increase in the clustering capacity [17]. After presenting 750 randomly chosen data points, all 1275 cluster points were correctly classified, fig.9 shows correct clustering of less regular input.

Fig.8. Some 17 clusters in 2-D space, represented by two 1-D input variables, each variable encoded by 12 neurons (five broadly tuned, seven sharply tuned).

With the knowledge we are currently obtaining of the fundamental importance of spike timings and oscillations to neural processing, 2nd generation ANNs can no longer provide a viable basis for neural modeling. Spiking Neural Networks present many new challenges but also afford many new opportunities for breaking entirely new ground in artificial intelligence research. Apart from various advantages posed by SNNs there still remain few bottlenecks: biologically realistic spiking models have required intensive computations for even small amounts of simulated time, making simulations of large networks or long time periods impractical in most situations. One of the advantages may also be a disadvantage in that the complex behavior needs to be understood and effectively managed. Also much less is known about networks of spiking neurons than the more established ANN paradigms, and many well-accepted methodologies need to be adapted or possibly replaced.

[2] W.Gerstner and W.Kistler, “Spiking neuron models”, Cambridge University press, 2002. [3] T.Ichishita and R.H. Fujii, “Performance evaluation of a Temporal Sequence Learning Spiking Neural Network”, 7th International Conference on Computer and Information technology, pp. 616 – 620, 2007 IEEE.

IJ A

[4] Animesh Patcha, Jung-Min Park, “An overview of anomaly detection techniques: existing solution and latest technological trends”, Computer Networks 51(2007), pp.3448-3470, February 2007.

Fig.9. Classification of ten irregularly spaced clusters. For reference, the different classes as visually extractable were all correctly clustered, as indicated by the symbol/graylevel coding.

Likewise clustering could be used for detecting abnormal behavior of data points indicating it to be an anomaly. The data having similar characteristics fall in the same cluster following the principle of inter-cluster similarity and intracluster dissimilarity. The data points that do not fall into any cluster are considered as anomalous candidates.

ISSN: 2230-7818

[5] J.J.Hopfield, “Pattern recognition computation using action potential timing for stimulus representation”, Nature, 376, pp. 33 – 36, 1999. [6] W.Maass, “Fast sigmoidal networks via spiking neurons”, Neural computation, 9(2), pp.279 – 304, 1999. [7] W.Maass, “Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons”, in advances in Neural Information Processing Systems, volume 9, MIT press, Cambridge, pp. 211 – 217, 1997. [8] B.Ruf and M.Schmitt “Self organization of spiking neurons using action potential timing”, IEEE Transactions on Neural Networks, 9(3), pp. 319 – 332, 1998. [9] S.M.Bohte, J.N.Kok and H.La Poutre, “Error back propagation on temporally encoded networks of spiking neurons”, Neurocomputing, 48, pp. 17 - 37, 2002. [10] B.Ruf and M.Schmitt, “Learning temporally encoded patterns in networks of spiking neurons”, Neural Processing Letters, 5(1), pp. 9 – 18, 1997. [11] T.Natschlager and B.Ruf, “Spatial and temporal pattern analysis via spiking neurons”, Network: Computational Neural Systems, 9(3), pp.319 – 332, 1998.

Page 41

Disha Sharma et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 2, Issue No. 1, 036 - 042 [12] S.M.Bohte, H.La Poutre and J.N.Kok, “Unsupervised clustering with spiking neurons by sparse temporal coding and multi layer RBF networks”, IEEE Transactions on Neural Networks, 13(2), pp.426 – 435, 2002. [13] X.Tao and H.E.Michel, “Data clustering via spiking neural networks through spike timing dependent plasticity”, IC-AI, pp.168 – 173, 2004. [14] D.T.Pham, M.S.Packianather, E.Y.A.Charles, “A novel self organized learning model with temporal coding for spiking neural networks”, Intelligent Production machines and systems, pp.307 – 312, 2006. [15] D.T.Pham, M.S.Packianather, E.Y.A.Charles, “A self organizing spiking neural network trained using delay adaptation”, MEC, Cardiff University, UK, pp. 3441 – 3446, 2007. [16] W.Maass, “Computing with spiking Neurons”, in Pulsed Neural Networks, The MIT Press, Cambridge, pp. 55 – 85, 2001.

IJ A

[17] Sander M. Bohte, Han La Poutré, and Joost N. Kok, “Unsupervised Clustering With Spiking Neurons By Sparse Temporal Coding and Multilayer RBF Networks”, IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 13, NO. 2, pp. 426-435, 2002.

ISSN: 2230-7818

Page 42