Automated test of evolving software
Abstract
Computers and the software they run are pervasive, yet released software is often unreliable, which has many consequences. Loss of time and earnings can be caused by application software (such as word processors) behaving incorrectly or crashing. Serious disruption can occur as in the l4th August 2003 blackouts in North East USA and Canadal, or serious injury or death can be caused as in the Therac-25 overdose incidents. One way to improve the quality of software is to test it thoroughly. However, software testing is time consuming, the resources, capabilities and skills needed to carry it out are often not available and the time required is often curtailed because of pressures to meet delivery deadlines3. Automation should allow more thorough testing in the time available and improve the quality of delivered software, but there are some problems with automation that this research addresses. Firstly, it is difficult to determine ifthe system under test (SUT) has passed or failed a test. This is known as the oracle problem4 and is often ignored in software testing research. Secondly, many software development organisations use an iterative and incremental process, known as evolutionary development, to write software. Following release, software continues evolving as customers demand new features and improvements to existing ones5. This evolution means that automated test suites must be maintained throughout the life ofthe software. A contribution of this research is a methodology that addresses automatic generation of the test cases, execution of the test cases and evaluation of the outcomes from running each test. "Predecessor" software is used to solve the oracle problem. This is software that already exists, such as a previous version of evolving software, or software from a different vendor that solves the same, or similar, problems. However, the resulting oracle is assumed not be perfect, so rules are defined in an interface, which are used by the evaluator in the test evaluation stage to handle the expected differences. The interface also specifies functional inputs and outputs to the SUT. An algorithm has been developed that creates a Markov Chain Transition Matrix (MCTM) model of the SUT from the interface. Tests are then generated automatically by making a random walk of the MCTM. This means that instead of maintaining a large suite of tests, or a large model of the SUT, only the interface needs to be maintained. 1) NERC Steering Group (2004). Technical Analysis ofthe August 14,2003, Blackout: What Happened, Why, and What Did We Learn? July 13th 2004. Available from: ftp:/ /www.nerc.com/pub/sys/all_ updl/docslblackoutINERC ]inatBlackout_Report _ 07_13_ 04.pdf 2) Leveson N. G., Turner C. S. (1993) An investigation of the Therac-25 accidents. IEEE Computer, Vo126, No 7, Pages 18-41. 3) LogicaCMG (2005) Testing Times for Board Rooms. Available from http://www.logicacmg.com/pdf/trackeditestingTimesBoardRooms.pdf 4) Bertolino, A. (2003) Software Testing Research and Practice, ASM 2003, Lecture Notes in Computer Science, Vol 2589, Pages 1-21. 5) Sommerville, 1. (2004) Software Engineering, 7th Edition. Addison Wesley. ISBN 0-321-21026-3.Citation
Shaw, H.A. (2005) 'Automated test of evolving software' PhD thesis. University of Luton.Publisher
University of BedfordshireType
Thesis or dissertationLanguage
enDescription
A thesis submitted to the University of Luton, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyCollections
The following license files are associated with this item:
Related items
Showing items related by title, author, creator and subject.
-
Identifying benefits and challenges in the application of agile methodologies in software developmentSoftware has generally been developed using the ‘Waterfall’ methodology which is now believed to be cumbersome and slow to react to change. This belief has spawned new ‘Agile’ methodologies that aim to deliver small pieces of working software on a frequent basis. Many claims have been made about Agile, but little academic research has been carried out to justify them. This study identifies the significance of benefits and challenges practitioners have found when adapting to Agile methodologies. Further, the benefits and challenges are linked to the Agile features in use and assesses what contributes to the level of user satisfaction. A survey instrument was employed to collect data from as many practitioners as possible followed by post survey interviews. Claims made for Agile by service providers are compared to the experiences of those canvassed in the survey. Results reveal that the most significant benefit is closer collaboration with other members of the development team. Significant challenges were the estimation of the time and effort Agile projects take to complete and that Agile only works with competent, motivated people. The insights gained in this study will be circulated for the benefit of future research on software development methodologies and Agile practice.
-
An approach to locating delayed activities in software processesActivity is now playing a vital role in software processes. To ensure the high-level efficiency of software processes, a key point is to locate those activities that own bigger resource occupation probabilities with respect to average execution time, called delayed activities, and then improve them. To this end, we firstly propose an approach to locating delayed activities in software processes. Furthermore, we present a case study, which exhibits the high-level efficiency of the approach, to concretely illustrate this new solution. Some beneficial analysis and reasonable modification are developed in the end.
-
A scalable and license free 5G Internet of radio light architecture for services in homes businessesIn this paper we present a 5G Internet Radio-Light (IoRL) architecture for homes that can be readily deployed because it utilizes unlicensed visible light and millimeter wave part of the spectrum, which does not require Mobile Network Operator (MNO) permission to deploy and which is used to provide inhabitants of houses with accurate location, interaction, access to Internet and Cloud based services such as high resolution video on a Tablet PC. The paper describes the home use cases and the IoRL architecture.
