Combining model checking and symbolic execution for software testing

Section 4 describes the case study, where these technologies are applied to a planetary rover controller. Symbolic execution and model checking for testing youtube. Then, it was found that applying bddbased exact symbolic model checking for test case generation. Combining symbolic execution and model checking for data. Combines symbolic execution, testing, model checking and theorem proving recent extensions. We believe that this could be changed if the developers could use the tools in the same way they already use testing tools.

The korat approach acm sigsoft impact paper award 2012. Selecta formal system for testing and debugging programs by symbolic execution. Symbolic execution is a software testing technique that is useful to aid the generation of test data and in proving the program quality. Combining symbolic execution with model checking to verify. Combining unitlevel symbolic execution and systemlevel concrete. Combining model checking and symbolic execution for.

Model checking has grown in scalability and new applications but attempts to combine. The execution requires a selection of paths that are exercised by a set of data values. Blackbox checking 39 is intended for acceptance tests where one. Over the years, we have developed a tool, symbolic pathfinder spf, that aims to leverage the power of systematic analysis techniques, such as model checking and symbolic execution, for thorough testing of complex software. Symbolic execution tree of function foobar given in figure 1. Request pdf on may 1, 2015, ting su and others published combining symbolic execution and model checking for data flow testing find, read and cite all the research you need on researchgate. Model checking exhaustively analyzes all program executions in a systematic way, but it su. Abstract state matching is used to avoid generation of. Dart 19 is the first concolic testing tool that combines dynamic test generation with random testing and model checking techniques with the goal of systematically executing all or as many as possible feasible paths of a program, while checking.

Siegel university of delaware anastasia mironova university of utah and george s. Symbolic execution, searchbased software engineering acm reference format. Existing automated techniques, like model checking and symbolic execution, are highly effective cadar 2008, holzmann 2008, but their adoption in industrial generalpurpose software testing has been limited. It discusses some key technical challenges, solutions and milestones, but is not an exhaustive survey of this research area.

Pdf combining symbolic execution with model checking to. Combining model checking and testing microsoft research. Combining unitlevel symbolic execution and systemlevel. Data flow testing dft focuses on the flow of data through a program. In computer science, model checking or property checking is a method for checking whether a finitestate model of a system meets a given specification a. In particular, we have extended the java pathfinder model checking tool jpf 3 with a symbolic execution capability 4,2 to enable test. Combining symbolic execution and model checking for data flow testing ting su zhoulai fuy geguang puz jifeng he zhendong suy shanghai key laboratory of trustworthy computing, east china normal university, shanghai, china ydepartment of computer science, university of california, davis, usa email.

Clarke university of massachusetts we present a method to verify the correctness of parallel programs that perform complex numerical. Acm sigsoft 20th international symposium on the foundations of software engineering fse, 1 page, research triangle park, nc, november 2012. Barr, mark harman, phil mcminn, muzammil shahbaz and shin yoo abstract testing involves examining the behaviour of a system in order to discover potential faults. We have developed symbolic java pathfinder, a symbolic execution framework that implements a nonstandard bytecode interpreter on top of the java pathfinder model checking tool. Combining model checking and symbolic execution for software. The aim of this chapter is to present an overview of this second approach to software model checking. Over the last two decades, significant progress has been made on how to broaden the scope of model checking from finitestate abstractions to actual software implementations. Dynamic software model checking marktoberdorf2015 iccut. Introduction symbolic execution has gathered a lot of attention in recent years as an effective technique for generating highcoverage test suites and for. Modern software systems, which often are concurrent and manipulate complex data structures must be extremely reliable. In the software development life cycle sdlc, testing is an important step to reveal and fix the vulnerabilities and flaws in the software. Combining symbolic execution and model checking for data flow.

Khannur, software testing techniques and applications. A survey of symbolic execution techniques acm computing. Automated testing using symbolic model checking and. The main idea behind symbolic execution 40 is to use sym. Dart was first implemented at bell labs for testing c. Machine learning for input fuzzing ase2017 a general framework for dynamic stub injection icse2017 between testing and verification. A brief discussion of the relationship between symbolic execution and program. Without access to source code, binary executables of such applications are employed for testing. Dart 19 is the first concolic testing tool that combines dynamic test generation with random testing and model checking techniques with the goal of systematically executing all or as many as possible feasible paths of a program, while checking each execution for various types of errors. Symbolic execution is used to reason about a program pathbypath which is an advantage over reasoning about a program inputbyinput as other testing paradigms use e. However, cegar can tell the feasibility of test objectives by doing reachability checking but its performance is lim.

Testing commercial offtheshelf applications for security has never been easy, and this is exacerbated when their source code is not accessible. Symbolic execution is a widely used technique for different software analysis purposes such as generating test cases, automatically checking programs against annotated properties, and detecting. Combining static analysis and model checking for software. Symbolic execution and model checking for testing request pdf. Google tech talks november, 16 2007 this talk describes techniques that use model checking and symbolic execution for test input generation. Software testing, symbolic execution, and model checking c. Modeling languages programming languages model checking systematic testing verisoft.

Typically, one has hardware or software systems in mind, whereas the specification contains safety requirements such as. Using model checking with symbolic execution to verify. Mixed symbolic representations for model checking software programs. Combine monte carlo simulations and symbolic execution for system level testing future hybrid approaches. Jpf is a model checkerwhich operateson principlessimilar to the spin model checker 7, i.

Combining unitlevel symbolic execution and systemlevel concrete execution for. Java pathfinder jpf model checker has been applied to the veri. Combining model checking and testing patrice godefroid. Code model checking is a rapidly advancing research topic. This paper tackles this challenge by introducing a hybrid dft framework.

Static analysis is scalable and exhaustive, but it may give many warnings that. To this end, egt in termixes concrete and symbolic execution by dynamically checking before every operation if the values involved are all concrete. In principle, dse dynamically explores programpathstoidentifytestinputsforfeasibletest objectives, but fails to cover infeasible ones and wastes testing time on them. The paper describes an application of the technology to a nasa rover controller. We aim to use the power of exhaustive techniques, such as model checking and symbolic execution, to enable thorough testing of complex software. One way to do this consists of adapting model checking into a form of systematic testing that is applicable to. In proceedings of the 31st acm sigplan conference on programming language design and implementation pldi10. A methodology is advocated that automatically generates properties specific to each input rather than formulating properties uniformly true for all inputs.

In fact, the same techniques can be applied for white box testing. Proceedings of ieeeacm 37th international conference on software engineering, florence, 2015. Ranged symbolic execution uses two test inputs to define a. Automated testing, test case generation, model checking, symbolic execution, runtime. Symbolic execution achieves high test coverage in a setting where the source code is completely available. The key idea is to use model checking, together with symbolic execution, to establish the equivalence of the two programs. Combining symbolic execution and model checking to verify. Combining symbolic execution and model checking for data flow testing. It has gained attention since its introduction in the 1970s 1,2 and is used in testing, invariant detection, model checking, and proving software correctness 3,4,5,6.

Over the years, we have developed a tool, symbolic. Techniques for checking complex software range from model checking and static analysis to testing. Combining symbolic execution and searchbased testing for programs with. Apr 01, 2008 read combining symbolic execution with model checking to verify parallel numerical programs, acm transactions on software engineering and methodology tosem on deepdyve, the largest online rental service for scholarly research with thousands of academic publications available at your fingertips. In this approach the path condition from symbolic execution of the. Using model checking to find serious file system errors. Software program verifi cationformal methods, model checking, validation. Proceedings of the symposium on operating system design and implementation. Various approaches to model checking software 6 hypothesis model checking is an algorithmic approach to analysis of finitestate systems model checking has been originally developed for analysis of hardware designs and communication protocols model checking algorithms and tools have to be tuned to be applicable to analysis of software. Symbolic model checking is superior in this case as the one capable of handling large state spaces, while in explicitstate model checking the number of states in the model grows exponentially with the test case length required to achieve coverage. Parallel symbolic execution for automated realworld. May 24, 2015 combining symbolic execution and model checking for data flow testing abstract. Combining symbolic execution and model checking for data flow testing abstract.

However, if few inputs take the same path through the program, there is little savings over testing each of the inputs separately. Symbolic execution for software testing in practice imperial. Combining symbolic execution with model checking to verify parallel numerical programs stephen f. Veri cation of java programs using symbolic execution and invariant generation. In this approach the path condition from symbolic execution of the sequential program is used to constrain the search through the parallel program. Symbolic execution generated 150 test cases in 30 seconds covered all. Combining model checking and testing joint work with koushik sen chapter 19 of the handbook of model checking, pages 6649, springer, 2018. Some are based on symbolic execution 4, andor constraint resolution. Generalized symbolic execution for model checking and testing sarfraz khurshid1, corina s. Combining symbolic execution and model checking to verify mpi programs. A survey of symbolic execution techniques roberto baldoni, emilio coppa, daniele cono delia, camil demetrescu, and irene finocchi, sapienza university of rome many security and software testing applications require checking whether certain properties of a.

We provide a twofold generalization of traditional symbolic execution based approaches. This paper presents a short introduction to automatic codedriven test generation using symbolic execution. Symbolic execution university of maryland, college park. Symbolic execution is a method to analyze software systems. A symbolic execution framework often uses also some elements exploration, search of symbolic model checking to be usable for testing etc. This chapter discusses advances in software model checking and focuses on techniques that use the software as its model and embedded exceptions or assertions as the properties to be verified. Citeseerx combining symbolic execution and model checking. Our research on concolic testing 1,6,4 shows that we can combine random testing and symbolic. Symbolic execution and software testing corina pasareanu. Functional and model based testing sample the input space according to specifications and models, structural testing techniques are. Request pdf symbolic execution and model checking for testing. Combining symbolic execution and model checking for data flow testing ting su, zhoulai fu, geguang pu, jifeng he, zhendong su 37th ieeeacm international conference on software engineering icse 2015 acceptance rate. Combining test case generation and runtime verification abstract. This is typically associated with hardware or software systems, where the specification contains liveness requirements such as avoidance of livelock as well as safety requirements such as avoidance of states representing.

Corina pasareanu, peter mehlitz, david bushnell, karen gundyburlet, michael lowry, suzette person, mark pape, combining unitlevel symbolic execution and systemlevel concrete execution for testing nasa software, proc. The egt approach 9, implemented and extended by the exe 10 and klee 8 tools, works by making a distinction between the concrete and symbolic state of a program. The key idea is to use model checking, together with symbolic execution, to establish the equivalence of the tw o programs. Some are very specific to model checking and some are modular and used in a standalone symbolic execution framework, as it was defined by the inventors of symbolic execution. Each execution state, labeled with an upper case letter, shows the statement to be executed, the symbolic store.

Despite its higher faultdetection ability over other structural testing techniques, practical dft remains a significant challenge. Mpisv exploits symbolic execution to automatically generate pathlevel models, and performs model checking on the models w. Deep reinforcement fuzzing joint work with konstantin bottinger and rishabh singh proceedings of dls2018 1st deep learning and security workshop, san francisco, may 2018. We describe the main ideas and techniques used to sys. Given an input for a system, the challenge of distinguishing the corresponding desired, correct behaviour from potentially. In computer science, model checking, or property checking, is, for a given finitestate model of a system, exhaustively and automatically checking whether this model meets a given specification a. Combining symbolic execution and searchbased testing for programs with complex heap inputs pietro braione.

Generalized symbolic execution for model checking and testing. Combining closedloop test generation and execution by means. Recently, novel approaches to combining model checking and testing have been proposed, which involve learning strategies 38. Combining symbolic execution and searchbased testing for. Patrice godefroid abstract model checking and testing have a lot in common. Section 2 outlines our technology for test case generation. Generalized symbolic execution for model checking and testing sarfraz khurshid 1, corina pasareanu 2, and willem visser 2 1mit laboratory for computer science, cambridge, ma 029 khurshidolcs. We present a novel framework based on symbolic execution, for automated checking of such systems. Section 3 describes the runtime analysis techniques. Robust software engineering software model checking. Model checking, testing and verification working together. Combining symbolic execution and model checking for data flow testing shanghai jiaotong university, shanghai, china, 4 may, 2015 invited by prof. Generalized symbolic execution for model checking and.