GRD Journals- Global Research and Development Journal for Engineering | Volume 6 | Issue 1 | December 2020 ISSN- 2455-5703
Formal Bug Hunting: An Immensely Powerful Merger of Simulation and Formal Verification Methodologies Joydeep Bhattacharyya Department of Electronics & Telecommunication Intel Corporation, USA
Abstract Traditional simulation-based hardware verification methods suffer from missing completeness, whereas formal verification tools are computationally expensive, resulting in missing verification convergence, a must-have criteria for sign-off. In the last few years, many techniques were developed which incorporated learnings from both the simulation and formal methodologies. This paper discusses several such “bug-hunting” strategies, along with their possible implementation details and challenges. Keywords- Chip Design, Functional Verification, Formal Verification, Semiconductor Design
I. INTRODUCTION There are some historical precedence’s for the growing popularity of formal verification. Many people suggest the incidence of finding the (in) famous Pentium Floating point unit bug in an Intel processor, leading to the recall of faulty processors and Intel having to bear a loss of close to $500 million. People also have suggested that various other incidents, such as the Ariane 5 explosion and overexposure of radiation in the Panama Cancer Institute, could have been avoided with formal verification. In the last couple of decades, as semiconductor System-on-Chip (SoC) designs grew bigger, so did their verification complexities. The risk of an unveiled design issue is very high now, potentially in a few million dollars and several weeks of delay in product launch timeframe. Simulation based verification approaches still carry on most of the quality checking tasks, while formal verification has been successfully used to augment many of the shortcomings of the traditional test benches. However, the current industry standard tools have limitations in handling very large designs. This often results in a compromise in the formal test bench coverage, either by removing portions of logic from verification, or by closing verification with “bounded-proofs” for design assertions. To overcome these challenges, design verification engineers have come up with various “semi-formal” techniques. It is worth mentioning that most of these techniques have been built upon learnings from standard simulation based verification practices. With both the simulation and semi-formal methodologies in place, the chance of a serious bug escaping the pre-tape out checks has been significantly lowered, as found by many independent insights.
II. FORMAL VERIFICATION PROCESS Traditionally, formal tools were used only in equivalence checking between various models (architectural, RTL, GDS). However, the recent innovations in technologies and methodology, the benefits of formal functional verification apply in many more areas. A deep understanding of the Design-Under-Test (DUT) and the presence of good quality specification documents (architectural and micro-architectural) reveal many areas of logic in most of the IPs who are great candidates for formal verification. While proposing the formal verification strategies, strong focus is kept on the Return-On-Investment (ROI) compared to a standard simulation test bench methodology, since significant effort and ramp-up is involved in a formal test bench bring-up. Often, the teams end up a creating two separate verification goals, one for the simulation and another for the formal test bench. Technically speaking, there are three primary objectives for use of formal tools: (1) proving what other standard methods are unable to, e.g. proving certain design properties are always true or there are exceptions, (2) speeding up proof of certain properties faster than simulation (such as if a “FIFO-full” condition can happen), and (3) provide a sense of completeness to the verification suite, by establishing 100% coverage.
All rights reserved by www.grdjournals.com
1
Formal Bug Hunting: An Immensely Powerful Merger of Simulation and Formal Verification Methodologies (GRDJE/ Volume 6 / Issue 1 / 001)
Fig. 1: Bugs Found over time in a Formal Test bench
Figure (1) shows the typical temporal performance of a formal test bench. There is always an initial setup time needed to write effective assertions and assumptions (and in some cases, some constraining on the design input space), only after which real bugs start to show themselves. It is, however, noteworthy that in almost all cases, the rate of finding bugs in this initial phase is much higher than that in a traditional simulation test bench. Most of the properties which can be proved as “full-proof” (i.e., without any bound) are proved here. A general guideline is to run the tool for an hour to check which properties are yet to be fully proven, and then use a “bug-hunting” technique for them. Once the formal test bench attains a certain level of maturity, it is highly recommended to include all the assertions already developed (and the future ones) in the standard simulation environment. The author, from his own experiences, also suggests running nightly regression jobs on the formal properties, to ensure no ongoing design changes break the already passing SVAs.
III. BUG HUNTING TECHNIQUES Bug hunting techniques are more of an art than a standard algorithmic guideline and is highly dependent on one’s experiencebased instincts upon where bugs could be hidden. However, all approaches generally fall into one of these categories. 1) Y-Swarm: The classic formal verification aims at “full-proof”s, i.e., exhaustively verifying design assertions. But since this is computationally very expensive, the search can be configured to skip or partially analyze state space areas. This enables the tool to quickly go into deeper states, and enable massive parallel search productivity gain. Verification engineers can utilize hundreds of parallel jobs in a single session. 2) Z-Swarm: Similar to Y-swarm, but this uses key states to guide formal analysis. At first, interesting states of the design are determined and then the tool is directed to go through the sequence of these states as intermediate jumps. The key states should be close to each other (so that movement is easy from one state to the next one), and diverse (so that the same scenario can be hit in multiple ways). Examples of interesting states could be (a) a FIFO full and a write being stalled or (b) all the request ports of an arbiter are simultaneously asserted. Again, like Y-swarm, Z-swarm can operate many jobs in parallel, significantly reducing time to find a bug. 3) Guide Pointing: This method uses a tunneling mechanism through interesting states using different engines/strategies. Guide pointing is typically used to verify assertions after hitting a certain scenario. In all of these scenarios, the design under test is always initialized to a known state (usually an interesting configuration from the testing perspective). Figure 2 below captures the various bug hunting techniques and their relationships to the fully formal or fully simulation based verification.
Fig. 2: Various Bug Hunting Techniques
All rights reserved by www.grdjournals.com
2
Formal Bug Hunting: An Immensely Powerful Merger of Simulation and Formal Verification Methodologies (GRDJE/ Volume 6 / Issue 1 / 001)
IV. CASE STUDY As a real-life case study, the author would like to share the experiences of applying bug-hunting techniques in verifying a complex interface handling logic in an ASIC. The design under test (DUT) contained multiple interconnected state machines with sufficiently long latencies, and hence was difficult to verify through traditional simulation methods. A formal test bench was built and some of the easier properties were fully verified. The test bench then had several assertions which were not converging even after 48 hours of runtime. A careful mixture of Z-swarm and Guide pointing strategies were then used which started finding bugs in less than an hour of runtime. Multiple interesting coverage’s were written and analyzed for interesting design corner cases and later used to determine the “safe bound” for all the bounded assertions. The silicon was bug free, further showing as a testament of the robustness in the verification methodology.
V. CONCLUSION Formal bug hunting process is essentially a white-box technique, involving deep knowledge about the design micro-architecture. Custom assertions (SVAs) written to target DUT’s internal components and interfaces, along with IP level end-to-end checkers are used with carefully instrumented design initializations and guided walks along design corner cases yield many hard-to-reach bugs, in realistic convergence time. However, based on the author’s experiences and several other independent studies conducted, the recommended approach should be to use a combination of bug hunting formal along with simulation based verification (often at a slightly higher, such as a cluster, level) methods to achieve complete verification. Bug-hunting helps by providing laser sharp focus to zoom into concerned functional area and catch bugs early, but it may still miss design issues because the whole design is never tested under formal test bench. Simulation and emulation are the primary vehicles to unveil chip/system level bugs, especially given the ever-increasing complexities of the present and future generation of semiconductor chips.
REFERENCES [1] [2] [3] [4] [5] [6]
Joydeep Bhattacharyya, Latest trends in Hardware Verification, ICSES Transactions on Computer Hardware and Electrical Engineering (ITCHEE), June, 2018. Dominik Stoffel, Formal Verification of Systems-on-Chip - Industrial Experiences and Scientific Perspectives, 20th International Workshop on Database and Expert Systems Application Ashok B. Mehta, SystemVerilog Assertions and Functional Coverage: Guide to Language, Methodology and Applications. 2nd ed., pp. 1-8, Asa Ben-Tzur, Deep Bug Hunting, Formal Verification Conference, Reading, UK, June 2016 Intel official website, http://intel.com ,2020. JasperGold Formal Verification Platform,https://www.cadence.com/en_US/home/tools/system-design-and-verification/formal-and-static-verification/jaspergold-verification-platform.html
All rights reserved by www.grdjournals.com
3