A Survey of functional verification techniques

IJSRD - International Journal for Scientific Research & Development| Vol. 3, Issue 10, 2015 | ISSN (online): 2321-0613

A Survey of Functional Verification Techniques Aartika Bansal1 Nagendra Sah2 Anantharaj T.V.3 1 P.G Student 2Assistant Professor 1,2 PEC University of Technology, Chandigarh, India 3ASIC Verification Department SanDisk, Bangalore, India Abstract— In this paper, we present a survey of various techniques used in functional verification of industry hardware designs. Although the use of formal verification techniques has been increasing over time, there is still a need for an immediate practical solution resulting in an increased interest in hybrid verification techniques. Hybrid techniques combine formal and informal (traditional simulation based) techniques to take the advantage of both the worlds. A typical hybrid technique aims to address the verification bottleneck by enhancing the state space coverage.

Key words: Functional verification, simulation, formal methodology, hybrid techniques I. INTRODUCTION Functional verification of a circuit is the process to ensure that the design implementation conforms to the original circuit specifications. The traditional and the most common method of verification has been through simulation. But recently, there is an increased interest in formal verification methods such as model checking, equivalence checking, language containment and theorem proving. Formal verification is a viable option to improve design quality. As the size and complexity of System on Chip (SoC) design has been ever increasing, verification engineers face the challenge in developing efficient verification strategies and ensure that the designs are completely bug-free. Early detection of errors is desired since it saves a lot of time, effort and manufacturing costs. A general theme being practiced is to apply multiple verification techniques which complement one another, resulting in the overall efficiency of verification tool. Such integration must be carried out strategically and precisely.

Simulation remains the most-practiced technique for most of the real life verification issues in industry practice. Its scalability and easy applicability to almost any design makes it suitable for most of the verification tasks.

But over time, its effectiveness to find corner case errors is significantly decreasing. When used as a stand-alone technique, it can detect simple bugs only. On the contrary, formal techniques can, in principle, find all the bugs in a design model but their applicability in practice is limited.

The skilled manual guidance needed for theorem proving and the state explosion in model checking are the problems confronted in these techniques. So there is a need of an immediate practical solution which gives rise to a keen interest in the integration of formal and informal techniques, referred to as hybrid methodology.

II. FORMAL VERIFICATION Formal verification methods comprise the techniques that logically prove the correctness of a hardware design. No test vectors are required in formal verification as opposed to the

traditional simulation based methods. There are three subcategories in these methods: a) Theorem proving b) Model checking c) Equivalence checking A. Theorem proving methods: These methods demand the representation of both the system and its properties in the form of formulas in some mathematical (formal) logic [1]. The logic defines a set of axioms and inference rules on the basis of which the tool finds the proof of a property. Thus, theorem proving is the process of finding the proofs from the axioms of the system.

Some well-known theorem proving systems are: BoyerMoore (first order logic) [2], Higher-order logic (HOL), PVS [3]. But these systems are hard to use since the designers need expertise in this area to be capable of using these systems.

B. Model Checking: In model checking, a finite state model (FSM) for the system is built and then the tool checks whether the desired property holds. The proofs are attempted by exhaustion which means for a given system, the tool generates all the possible states the system can be in and then tests each state individually [1, 2, 4]. K. Fishler et al. have explored the verification of VHDL designs with COSPAN which is a model checker [5]. K. McMillan details the Symbolic Model Checking to solve the problem of state space explosion faced in model checking. Cadence Design Systems [7] and Lucent Technologies [8] have developed Formal Check Model Checker for model checking verification. The major drawback of model checking is the state space explosion.

These tools can handle a limited size circuit only. Moreover, model checkers can skip an error if the designer did not specify a property.

C. Equivalence checking: This technique is also based on the FSM. This method checks the equivalence between a reference model and the design to be verified, both expressed as FSMs. The two are said to be equivalent if they produce the same output for every possible input stimulus. Design Verifyer developed by Chrysalic Symbolic Design [9], Affirma Equivalence Checker developed by Cadence Design System [10], Formality developed by Synopsys [11] are some of the examples of equivalence checkers. This method can be used to verify the equivalence between gate level description and the RTL code. But to verify RTL code, we cannot use this method as there is no reference model for comparison. State space explosion is a problem in this approach too.

III. INFORMAL VERIFICATION Since formal verification techniques cannot handle large designs due to state space explosion, simulation based

All rights reserved by www.ijsrd.com

512

A Survey of Functional Verification Techniques (IJSRD/Vol. 3/Issue 10/2015/106)

verification remains the major technique to perform hardware design verification. A traditional methodology for the VLSI design and verification is shown in figure 1. As shown, once the circuit specifications are ready and it has been partitioned into blocks, the next step is to write RTL code for each block and then combine these blocks into whole chip.

Fig. 2: Concurrent Verification Methodology IV. HYBRID TECHNIQUES These techniques combine at least two methods which complement one anotherâ€&#x;s strengths and weaknesses effectively. Bartley et al. showed that with the increasing complexity and size of the circuit, the most effective way of functional verification is to combine the strengths of different techniques [14]. We classify the hybrid functional verification methods as follows: a) Combination of formal and informal techniques, b) Combination of two formal techniques, c) Combination of two informal techniques, d) Combination of multiple verification techniques.

Fig. 1: Traditional VLSI Design Methodology The verification engineers then develop the test benches for the simulation of the device under test (DUT). The actual circuit response is compared against the expected response for the pre-determined stimuli. If the match occurs, the test passes otherwise it fails. It takes a huge amount of time and resources to write test benches to verify the circuit at RTL level.

Instead of waiting to complete the RTL description of the circuit and then start verification process, we must adopt a concurrent verification methodology as shown in figure 2. In this methodology, the verification team starts writing test bench as soon as the system specification is complete and the designers start writing the RTL code.

Initially they run simulation on the developed behavioral models of the blocks. Once the RTL code is ready, the model is replaced by the RTL code. This saves a lot of time.

The Vera Verification System developed by Synopsys [12] and the Spec-Man: Spec-based functional verification by Versity Design [13] are the simulation based tools that use better test bench construction languages.

Major drawbacks of informal techniques comprise long simulation runs, a lot of effort and time needed to write the test bench and difficulty in finding the stimuli for the corner cases.

A. Combination of formal and informal techniques: Since the informal techniques are inherently incomplete in finding all the design errors, the combination of one with formal technique yields a technique that is incomplete.

1) Control circuit exploration: The exploration of control circuits can address the problem of increasing state space coverage. Horowitz et al. detailed a generalized technique that is viable for a large set of circuit types [15]. This technique automatically generates the test vectors that make the microprocessor exercise all control logic transitions in simulation. It does not target only certain test cases but delves into many improbable conditions in corner cases. Hence, it maximizes the probability of catching bugs through simulation.

Moundanos et al. explored coverage-directed test generation in which user provided stimuli help to discover the interesting control space behavior of large designs.

Geist et al. have proposed to construct an abstract model with key features of the original design [17]. These models help to alleviate the state explosion problem by the imposition of a less constrained upper bound.

2) Directed Functional Test Generation: Typically, verification teams in industries have a set of conditions to meet before the design is taped out. These conditions are constrained by a combination of coverage metrics, corner cases and design complexity. The verification engineers are to detect the design errors in the way of reaching these conditions. Pseudo Random Test

All rights reserved by www.ijsrd.com

513

A Survey of Functional Verification Techniques (IJSRD/Vol. 3/Issue 10/2015/106)

Generation methods are ineffective to cover specific corner cases. So, only simulation is not enough to meet the needs of directed verification. Geist et al. presented a study of a formal verification using coverage models to specify functional tests [17]. This hybrid technique has, in fact, resulted in improved functional verification. Ganai et al. proposed a rarity based metric for state prioritization that enabled an efficient directed functional search of a large state space [18].

Symbolic simulation is an effective formal verification technique but fails for large and complex real designs. Yuan et al. showed that the combination of formal verification and simulation to achieve better coverage in larger circuits can find bugs far more quickly as compared to using only formal verification [19].

3) Automatic Test Pattern Generation (ATPG) and formal techniques: Traditionally, techniques using ATPG localize the bug search instead of searching the entire state space at once.

Since formal techniques can address the incompleteness of ATPG, the two can be combined effectively.

Jones and Privitera proposed the technique of automatic generation of test vectors for functional verification which gives advantage of random as well as directed testing [20].

They showed that formal specifications can be given as inputs to a test generator. The main algorithms combine ATPG and BDDs (Binary Decision Diagrams) to generate the directed test vectors.

Boppana et al. proposed the usage of sequential ATPG for model checking [21]. They studied the efficiency of sequential ATPG for state space exploration. Huan and Cheng proposed the combination of word-level sequential ATPG techniques with modular constraint solving techniques to verify safety properties [22]. They transformed the problem into a counter example generation problem. Sequential ATPG loses its effectiveness for large complex circuits. Vedula et al. solved this problem by using a well-known technique called program slicing. They reduced the module under test, hence accelerating sequential ATPG efficiency in solving the Bounded Model Checking (BMC) problem.

4) Constraint-based verification: In a constraint-based random verification methodology, the user provides constraints that model the appropriate interaction of the DUT with its environment. Using these constraints, random stimuli are generated. The generated stimuli can mimic the actual environment. Yuan et al. invented such a set of constraint-based verification techniques [24]. Wu and Huang proposed to implement constraints with variable ordering so as to put the required efforts on the cared patterns [25]. However, they found that it may lead to degradation of pattern generation speed and distribution. To overcome this challenge, they provided a technique to preanalyze the solution space by splitting the ranges of variables. Yehia showed a standard and effective way to avoid redundant stimuli by constructing a link between the collected coverage and constrained random stimuli at runtime [26]. The beauty of the constraint-based verification lies in the fact that it works at various abstraction levels of a design viz. module, block and unit level.

B. Combination of formal techniques: Researchers face the challenge of effectively combining theorem proving and model checking. Industrial-scale efforts encounter limits on model-checking capacity even today. Some of these limits can be alleviated by using theorem proving to compose verification results.

1) Trajectory evaluation and theorem proving: Joyce and Seger proposed the combination of trajectory evaluation and theorem proving [27]. They used the former as a decision procedure for the HOL proof system. Later, Seger et al. showed that combination of STE with lightweight theorem proving in HOL can be useful in industrial scale verification[28].

2) Composition of model checker runs: Camelleri used a theorem prover in combination with a model checker to verify a cache protocol [29]. He used separate tools to verify various properties but did not combine the results mechanically. Jang et al. used a technique of model checking referred to as computation tree logic (CTL) to verify different properties of a microcontroller [30]. Then, he achieved the proof of the top-level specification through a composition of model checking runs.

C. Combination of informal techniques: Kuehlmann et al. detailed an algorithm that uses probabilistic techniques [31]. The algorithm assigns each state a probability based on its chance to lead to a target state. On the basis of assigned probabilities, a set of ranks is assigned to the design states. This algorithm uses the ranking system to find a path from the initial states to the target states and can act as complement to the existing hybrid techniques for state space research. But it may lead to dead-end states. Yuan et al. proposed the use of input biasing to cover corner cases instead of using an explicit test bench [32]. Researchers have also explored the learning-based techniques. Fine and Ziv reported the use of computer learning and Bayesian networks [33]. Shimizu and Dill describe the coverage-directed informal methods that collect coverage information and derive the stimuli through the use of formal descriptions [34]. The description lists the interface properties of a bus protocol. But the listed properties are localized in time. Tasiran et al. proposed a tag coverage metric which considers a code segment to be covered only when it has executed and the effect has been recorded [35]. The metric is used to guide a semiformal test generation algorithm which is based on the analysis of the circuit modeled as a Markov chain in the steady state.

D. Combination of multiple techniques: Ho et al. developed the very first successful tool viz.

Ketchum which combined several verification techniques into a single hybrid technique [36]. The tool used the capabilities like test pattern generation and non-reachability analysis in order to get improvement over the traditional simulation techniques. Ketchum combines simulation with symbolic simulation, SAT-based BMC, symbolic fixpoint computation and many more. Non-reachability analysis helps to focus on coverage issues earlier in the design cycle. Ho et al. experimentally found that Ketchum provides a tenfold capacity increase compared to previous results.

All rights reserved by www.ijsrd.com

514

A Survey of Functional Verification Techniques (IJSRD/Vol. 3/Issue 10/2015/106)

V. CONCLUSION In this paper, we have presented a number of verification techniques for industrial hardware designs. A single verification technique is not enough to fulfill all the needs of today‟s industrial design verification environment.

Hence, further improvements are sought in this area. Future directions may include efforts to increase the tool capacity and accuracy to develop new verification techniques which would be better optimized to solve the overall verification challenges. The hybrid tools may be taken a step further to include features like circuit pre-analysis, dynamic interaction between verification methods and manual guidance to handle real industrial and complex decision issues.

REFERENCES [1] E. M. Clarke, J. M. Wing, “Formal Methods: State of the Art and Future Directions,” ACM Computing Surveys, Vol. 28, No. 4, pp. 627-643, December 1996.

[2] R.S. Boyer and J.S. Moore, A Computational Logic, Academic Press, New York, 1979.

[3] S. Owre, J. Rushby, and N. Shankar, “PVS: A prototype verification system,” in Eleventh International Conference on Automated Deduction (CADRE), Vol. 607 of lecture Notes in Artificial Intelligence, D. Kapur Ed., Springer-Verlag, pp. 748752, 1992.

[4] E. Cerny, X. Song, “Formal Verification Methods”: A short course, University of Montreal, Montreal, Canada, 1997.

[5] K. Fishler, and R. Kurshan, “Verifying VHDL designs with COSPAN,” in Formal Hardware Verification: Method and Systems in Comparison, LNCS 1287, Ed. Kropf, Springer-Verlag, pp. 206-247, 1997.

[6] K. McMillan, Symbolic Model Checking: An Approach to the state Explosion Problem, Kluwer Academic Publishers, 1993.

[7] Cadence Design Systems, Inc., “FormalCheck model checker,” http://www.cadence.com [8] Lucent Technologies, Inc. “FormalCheck-Model checker,” http://www.belllabs.com/org/blda/product. formal.htm1.

[9] Chrysalis Symbolic Design, Inc., “Design Verifyer,” http://www.chrysalis.com [10] Cadence Design Systems, Inc., “Affirma Equivalence Checker,” http://www.cadence.com [11] Synopsys, Inc., “Formality,” http://www.synopsys.com [12] Synopsys, Inc., “The Vera Verification System,” http://www.synopsys.com [13] Versity Design, Inc., “Spec-man: spec-based functional verification” http://www.versity.com [14] M.G. Bartley, D. Galpin, and T. Blackmore, „„A Comparison of Three Verification Techniques: Directed Testing, Pseudo-Random Testing and Property Checking,‟‟ Proc. 39th Design Automation Conf. (DAC 02), ACM Press, 2002, pp. 819-823.

[15] M.A. Horowitz et al., „„Architecture Validation for Processors,‟‟ Proc. 22nd Ann. Int‟l Symp. Computer Architecture (ISCA 95), IEEE CS Press, 1995, pp.

404-413.

[16] D. Moundanos, J.A. Abraham, and Y.V. Hoskote, „„Abstraction Techniques for Validation Coverage Analysis and Test Generation,‟‟ IEEE Trans.

Computers, vol. 47, no. 1, Jan. 1998, pp. 2-14.

[17] D. Geist et al., „„Coverage-Directed Test Generation Using Symbolic Techniques,‟‟ Proc. 1st Int‟l Conf.

Formal Methods in Computer-Aided Design, LNCS 1166, Springer-Verlag, 1996, pp. 143-158.

[18] M. Ganai et al., „„SIVA: A System for CoverageDirected State Space Search,‟‟ J. Electronic Testing: Theory and Applications, vol. 17, no. 1, Feb. 2001, pp.

11-27.

[19] J. Yuan et al., „„On Combining Formal and Informal Verification,‟‟ Proc. 9th Int‟l Conf. Computer-Aided Verification, LNCS 1254, Springer-Verlag, 1997, pp.

376-387.

[20] K.D. Jones and J.P. Privitera, „„The Automatic Generation of Functional Test Vectors for Rambus Designs,‟‟ Proc. 33rd Design Automation Conf. (DAC 96), ACM Press, 1996, pp. 415-420.

[21] V. Boppana et al., „„Model Checking Based on Sequential ATPG,‟‟ Proc. 11th Int‟l Conf. Computer Aided Verification, LNCS 1633, Springer, 1999, pp.

418-430.

[22] C.-Y. Huan and K.-T. Cheng, „„Using Word-Level ATPG and Modular Arithmetic Constraint-Solving Techniques for Assertion Property Checking,‟‟ IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 20, no. 3, Mar. 2001, pp. 381-391.

[23] V.M. Vedula, W.J. Townsend, and J.A. Abraham, „„Program Slicing for ATPG-Based Property Checking,‟‟ Proc. 17th Int‟l Conf. VLSI Design (VLSID 04), IEEE CS Press, 2004, pp. 591-596.

[24] J. Yuan, C. Pixley, and A. Aziz, Constraint-Based Verification, Springer, 2006.

[25] Bo-Han Wu, Chung-Yang Huang, “A Robust General Constrained Random Pattern Generator for Constraints with Variable Ordering”, IEEE/ACM International Conference on Computer-Aided Design (ICCAD) 2012, November 5-8, 2012, San Jose, California, USA. [26] A. Yehia, “Faster Coverage Closure: Runtime Guidance of Constrained Random Stimuli by Collected Coverage” IEEE Design and Verification Conference (DVC) 2013.

[27] J.J. Joyce and C.H. Seger, „„Linking BDD-Based Symbolic Evaluation to Interactive Theorem Proving,‟‟ Proc. 30th Design Automation Conf. (DAC 93), ACM Press, 1993, pp. 469-474.

[28] C.-J.H. Seger et al., „„An Industrially Effective Environment for Formal Hardware Verification,‟‟ IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 24, no. 9, Sept. 2005, pp.

1381-1405.

[29] A. Camilleri, „„A Hybrid Approach to Verifying Liveness in a Symmetric Multi-Processor,‟‟ Proc. 10th Int‟l Conf. Theorem Proving in Higher-Order Logics (TPHOLs 97), LNCS 1275, Springer, 1997, pp. 33-48.

[30] J.-Y. Jang et al., „„Formal Verification of FIRE: A Case Study,‟‟ Proc. 34th Design Automation Conf.

(DAC 97), ACM Press, 1997, pp. 173-177.

[31] A. Kuehlmann, K.L. McMillan, and R.K. Brayton, „„Probabilistic State Space Search,‟‟ Proc. Int‟l Conf.

All rights reserved by www.ijsrd.com

515

A Survey of Functional Verification Techniques (IJSRD/Vol. 3/Issue 10/2015/106)

Computer-Aided Design (ICCAD 99), IEEE CS Press, 1999, pp. 574-579.

[32] J. Yuan et al., „„Modeling Design Constraints and Biasing in Simulation Using BDDs,‟‟ Proc. Int‟l Conf.

Computer- Aided Design (ICCAD 99), IEEE CS Press, 1999, pp. 584-590.

[33] S. Fine and A. Ziv, „„Coverage Directed Test Generation for Functional Verification Using Bayesian Networks,‟‟ Proc. 40th Design Automation Conf.

(DAC 03), ACM Press, 2003, pp. 286-291.

[34] K. Shimizu and D.L. Dill, „„Deriving a Simulation Input Generator and a Coverage Metric from a Formal Specification,‟‟ Proc. 39th Design Automation Conf.

(DAC 02), ACM Press, 2002, pp. 801-806.

[35] S. Tasiran et al., „„A Functional Validation Technique: Biased-Random Simulation Guided by ObservabilityBased Coverage,‟‟ Proc. IEEE Int‟l Conf. Computer Design (ICCD 01), IEEE CS Press, 2001, pp. 82-88.

[36] P.-H. Ho et al., „„Smart Simulation Using Collaborative Formal and Simulation Engines,‟‟ Proc.

Int‟l Conf. Computer-Aided Design (ICCAD 00), IEEE CS Press, 2000, pp. 120-126.

All rights reserved by www.ijsrd.com

516