Automated test of evolving software
Abstract
Computers and the software they run are pervasive, yet released software is often unreliable, which has many consequences. Loss of time and earnings can be caused by application software (such as word processors) behaving incorrectly or crashing. Serious disruption can occur as in the l4th August 2003 blackouts in North East USA and Canadal, or serious injury or death can be caused as in the Therac-25 overdose incidents. One way to improve the quality of software is to test it thoroughly. However, software testing is time consuming, the resources, capabilities and skills needed to carry it out are often not available and the time required is often curtailed because of pressures to meet delivery deadlines3. Automation should allow more thorough testing in the time available and improve the quality of delivered software, but there are some problems with automation that this research addresses. Firstly, it is difficult to determine ifthe system under test (SUT) has passed or failed a test. This is known as the oracle problem4 and is often ignored in software testing research. Secondly, many software development organisations use an iterative and incremental process, known as evolutionary development, to write software. Following release, software continues evolving as customers demand new features and improvements to existing ones5. This evolution means that automated test suites must be maintained throughout the life ofthe software. A contribution of this research is a methodology that addresses automatic generation of the test cases, execution of the test cases and evaluation of the outcomes from running each test. "Predecessor" software is used to solve the oracle problem. This is software that already exists, such as a previous version of evolving software, or software from a different vendor that solves the same, or similar, problems. However, the resulting oracle is assumed not be perfect, so rules are defined in an interface, which are used by the evaluator in the test evaluation stage to handle the expected differences. The interface also specifies functional inputs and outputs to the SUT. An algorithm has been developed that creates a Markov Chain Transition Matrix (MCTM) model of the SUT from the interface. Tests are then generated automatically by making a random walk of the MCTM. This means that instead of maintaining a large suite of tests, or a large model of the SUT, only the interface needs to be maintained. 1) NERC Steering Group (2004). Technical Analysis ofthe August 14,2003, Blackout: What Happened, Why, and What Did We Learn? July 13th 2004. Available from: ftp:/ /www.nerc.com/pub/sys/all_ updl/docslblackoutINERC ]inatBlackout_Report _ 07_13_ 04.pdf 2) Leveson N. G., Turner C. S. (1993) An investigation of the Therac-25 accidents. IEEE Computer, Vo126, No 7, Pages 18-41. 3) LogicaCMG (2005) Testing Times for Board Rooms. Available from http://www.logicacmg.com/pdf/trackeditestingTimesBoardRooms.pdf 4) Bertolino, A. (2003) Software Testing Research and Practice, ASM 2003, Lecture Notes in Computer Science, Vol 2589, Pages 1-21. 5) Sommerville, 1. (2004) Software Engineering, 7th Edition. Addison Wesley. ISBN 0-321-21026-3.Citation
Shaw, H.A. (2005) 'Automated test of evolving software' PhD thesis. University of Luton.Publisher
University of BedfordshireType
Thesis or dissertationLanguage
enDescription
A thesis submitted to the University of Luton, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyCollections
The following license files are associated with this item:
Related items
Showing items related by title, author, creator and subject.
-
A software definable MIMO testbed: architecture and functionalityFollowing the intensive theoretical studies of recently emerged MIMO technology, a variety of performance measures become important to investigate the challenges and trade-offs at various levels throughout MIMO system design process. This paper presents a review of the MIMO testbed recently set up at King’s College London. The architecture that distinguishes the testbed as a flexible and reconfigurable system is first preseneted. This includes both the hardware and software aspects, and is followed by a discussion of implementation methods and evaluation of system research capabilities.
-
Picture theory: algorithms and softwareThis thesis is concerned with developing and implementing algorithms based upon the geometry of pictures. Spherical pictures have been used in many areas of combinatorial group theory, and particularly, they have shown to be a useful method when studying the second homotopy module, 1T2, of a presentation ([3],[4],[7],[12],[41] and [64]). Computational programs that implement picture theoretical and design algorithms could advance the areas in which picture theory can be used, due to the much faster time taken to derive results than that of manual calculations. A variety of algorithms are presented. A data structure has been devised to represent spherical pictures. A method is given that verifies that a given data structure represents a picture, or set of pictures, over a group presentation. This method includes a new planarity testing algorithm, which can be performed on any graph. A computational algorithm has been implemented that determines if a given presentation defines a group extension. This work is based upon the algorithm of Baik et al. [1] which has been developed using the theory of pictures. A 3-presentation for a group G is given by < P, s >, where P is a presentation for G and s is a set of generators for 1T2. The set s can be described in a number of ways. An algorithm is given that produces a generating set of spherical pictures for 1T2 when s is given in the form of identity sequences. Conversely, if s is given in terms of spherical pictures, then the corresponding identity sequences that describe 1T2 can be determined. The above algorithms are contained in the Spherical PIcture Editor (SPICE). SPICE is a software package that enables a user to manually draw pictures over group presentations and, for these pictures, call the algorithms described above. It also contains a library of generating pictures for the non abelian groups of order at most 30. Furthermore, a method has been implemented that automatically draws a spherical picture from a corresponding identity sequence. Again, this new graph drawing technique can be performed on any arbitrary graph.
