![\"\"](\"https:\/\/www.bridge-global.com\/blog\/\/wp-content\/uploads\/2015\/12\/Untitled4.png\" "\\"Untitled\\"")
<\/a><\/p>\n\u00a0<\/span>1. Requirements \u2013 seen as inherently informal for Meyers so the acceptance test must engage with the user in agreeing the requirements have been met.<\/span><\/p>\n2. Objectives are more tightly and clearly specified and quantitative issues of costs, sizes, speeds together with trade offs have been established. The system test should be directed to see that objectives have been met.<\/span><\/p>\n3. The external specification is a black-box specification of the functionality of the system and thus the function test is directed to checking the implementation of the specified function.<\/span><\/p>\n4. System design provides a structure that can be tested by Integration testing (inherently dependent of the structure of the system).<\/span><\/p>\n5. Module interface specifications are the basis for rigorous unit test (both black-box and white-box).<\/span><\/p>\nTest Objectives<\/span><\/strong><\/p>\n\u2022 Given there are many different types of test there is the possibility of considerable redundancy across tests and the potential to waste effort.<\/span><\/p>\n\u2022 One way of managing this is to set a test objective for each level of test.<\/span><\/p>\n\u2022 Ideally an objective should be dealt with at the lowest level testing.<\/span><\/p>\n\u2022 For example, we might take the following objectives:<\/span><\/p>\n\u2013 Module test aims to find discrepancies between a modules behavior and its interface specification.<\/span><\/p>\n\u2013 Function test aims to demonstrate that the program does not conform to its external specification.<\/span><\/p>\n\u2013 System test aims to show the product is inconsistent with the original objectives (e.g. that some quantitative aspect of the system is not delivered by the product).<\/span><\/p>\nAnalysis of the V-Model<\/strong><\/span><\/p>\n\u2022 The V-model is often criticized because it provides a very highly constrained model of the interaction of test and development and of the sequencing of testing activities.<\/span><\/p>\n\u2022 However, the model is useful in identifying different kinds of test that need orchestrating in any development process.<\/span><\/p>\n\u2022 More iterative or agile processes will have all these types of test in one iteration that delivers new functionality.<\/span><\/p>\n\u2022 For some of the software life cycles it may be that there is no independent notion of e.g. requirements and so that particular type of testing will be de-emphasized in that approach.<\/span><\/p>\n\u2022 Increasingly, as products become longer-lived, there is the issue of how to use test to evolve fielded products and how to use field data to improve product quality (e.g. beta testing uses a range of test sites to test systems).<\/span><\/p>\nTranslating the V-Model to Other Processes<\/span><\/strong><\/p>\nIf we consider an XP approach to development and consider the V-model classification of testing we can see that:<\/span><\/p>\n\u2022 Module, Integration, and Function test are carried out against module specifications and external specifications of the system.<\/span><\/p>\n\u2022 System and acceptance testing is moved more to the user who may be the embedded customer along with the team. The use of users as beta-testers helps identify real requirements issues as soon as significant functionality is available.<\/span><\/p>\n\u2022 Regression testing is linked to refactoring as a means of ensuring no change in the delivered behavior.<\/span><\/p>\n\u2022 It may be that the system has no well-documented requirements and so it is impossible to carry out a system test. It may also be that requirements are acquired iteratively as the system evolves.<\/span><\/p>\nModule\/Unit\/Integration Testing<\/span><\/strong><\/p>\n\u2022 It is important that this activity is appropriately targeted to avoid duplication of effort from other high-level test activity.<\/span><\/p>\n\u2022 The focus here should be on verifying that the behavior of the components satisfies the interface specifications.<\/span><\/p>\n\u2022 In XP it may be that the collection of tests serve as a specification for the system.<\/span><\/p>\nFunctional Testing<\/span><\/strong><\/p>\n\u2022 Functional testing is aimed at discovering discrepancies between the behavior of the system and its external specification.<\/span><\/p>\n\u2022 An external specification is a precise description of the required behavior form a users point of view.<\/span><\/p>\n\u2022 Typically systems are seen as delivering some key functionality and functional testing is directed to uncovering failures to provide the functions of the system.<\/span><\/p>\n\u2022 Functional testing techniques are black-box and use the approaches we have discussed earlier in the course e.g. category partition, common error lists, boundary-value testing.<\/span><\/p>\nSystem Testing<\/span><\/strong><\/p>\n\u2022 System testing is dependent on their being a quantified set of objectives for the system these may be derived iteratively but we need to capture them to allow effective systems testing.<\/span><\/p>\n\u2022 Focus of system testing is to find discrepancies in the translation of the objectives into the external specifications.<\/span><\/p>\n\u2022 So the system test should be related to the objectives of the system these should be captured in user-oriented term in the user documentations so we use the user document to help design he system tests.<\/span><\/p>\n\u2022 The problem in creating system tests is that there is no exhaustive list of the kinds of test that might be necessary to test the objectives of a system.<\/span><\/p>\n\u2022 Meyers provides a list of 15 different kinds of test that might be involved in testing the objectives of a system but this is not exhaustive it may be necessary to add some other testing approaches depending on the objectives<\/span><\/p>\nSystem Testing<\/span><\/strong><\/p>\n\u2022 Facility Testing: this is checking that user accessible facilities (or functions) are implemented. This is an all-or-nothing test either the system attempts to provide a facility or not this might take the form of a visual inspection of the system.<\/span><\/p>\n\u2022 Volume Testing: objectives will usually have requirements on the volume of objects that a system can handle. E.g. a simulator might have a specification that it is capable of handling up to 100,000 logic gates or a database system might have a lower bound on the number of records handled.<\/span><\/p>\n\u2022 Stress Testing: systems involving real-time constraints will almost always have objective in terms of how rapidly they can deal with events (e.g. 10,000 transactions per hour, the VISA payments system supports more than 5000 card transactions per second with very high reliability). In addition many system have multiple inputs and stress testing might investigate how well the system handles many input parameters changing simultaneously.<\/span><\/p>\nSystem Testing<\/span><\/strong><\/p>\n\u2022 Usability Testing: often carried out with real usability testers who can discover serious usability flaws even in well-implemented systems. Typical issues include:<\/span><\/p>\n1. Appropriateness of interface to the user group<\/span><\/p>\n2. Meaningful interactions, warnings etc..<\/span><\/p>\n3. Use of consistent styles, metaphors etc..<\/span><\/p>\n4. Is redundancy\/diversity of interfaces used where accuracy is essential?<\/span><\/p>\n5. Is the interface too configurable?<\/span><\/p>\n6. Is it clear that the system is always live?<\/span><\/p>\n7. Is it easy to get the system to do what the user wants?<\/span><\/p>\n\u2022 Performance Testing: many systems will have throughput objectives. These may be express statistically in terms of rates and likelihood of failure. The system should be instrumented to allow these objectives to by subjected test when the system is in the field.<\/span><\/p>\nSystem Testing<\/span><\/strong><\/p>\n\u2022 Security Testing: The external specification will usually have been derived from the objectives by developing a security model that constrains the flow of information from one area of the system to another. System testing should identify any flaws in moving from objectives to a behavioral specification. In addition the objectives may include a list of common threats that we should demonstrate are dealt with in the external specification.<\/span><\/p>\n\u2022 Storage Testing: systems will claim to run happily in a given amount of storage we should check this.<\/span><\/p>\n\u2022 Configuration Testing: many systems are highly configurable (e.g. SAP and Oracle). A good example of this is a multi-platform system where considerable configuration effort might be required to get a system to run on a particular platform<\/span><\/p>\nSystem Testing<\/span><\/strong><\/p>\n\u2022 Compatibility\/Compliance Testing: Objectives will often include compliance statements that the system is compliant with a standard this style of testing aims to cover a significant subset of the range of different configurations.<\/span><\/p>\n\u2022 Installability Testing: Is the system easy to install correctly? Reliability Testing: is the mean time to failure long enough<\/span><\/p>\n\u2022 Recovery Testing: assessing the mean time to repair a fault is often a critical objective of a system<\/span><\/p>\n\u2022 Serviceability Testing: how easy is it to upgrade the system?<\/span><\/p>\n\u2022 Documentation Testing: how well does the documentation match the actual system behavior<\/span><\/p>\n\u2022 Procedure Testing: many systems involve complex interactions between computers and human procedures are the procedures and the computer system compatible?<\/span><\/p>\nSystem Testing Process\/Management<\/span><\/strong><\/p>\n\u2022 System testing should not be performed by the developers ideally it should be an independent organization.<\/span><\/p>\n\u2022 Seemingly independent objectives have dependencies and the planning for system test has to take account of these dependencies (in order to get better coverage).<\/span><\/p>\nAcceptance Testing<\/span><\/strong><\/p>\n\u2022 Carried out by the customer on their premises.<\/span><\/p>\n\u2022 Testing is carried out independent of the development organization.<\/span><\/p>\n\u2022 The acceptance test marks the transition from development before use to development in use.<\/span><\/p>\nInstallation Testing<\/span><\/strong><\/p>\n\u2022 Does the proposed software run on the proposed user\u2019s configuration?<\/span><\/p>\n\u2022 Many vendors now offer to integrate systems offsite and bring them to the user\u2019s site.<\/span><\/p>\n\u2022 Issues like versions of software, library versions, hardware, networking all impact on the run ability of a system.<\/span><\/p>\n\u2022 Installation testing should aim to characterize the hardware \/software combination.<\/span><\/p>\nTest Planning and Control<\/span><\/strong><\/p>\n\u2022 Even for modest-sized systems, there is a considerable management load in ensuring the tests are well-managed.<\/span><\/p>\n\u2022 Plans should be realistic and should allow for the discovery and repair of the most serious classes of error. A good test plan should:<\/span><\/p>\na) Have clear objectives for each phase of testing.<\/span><\/p>\nb) Have clear completion criteria for each stage of test.<\/span><\/p>\nc) Have a project plan with clear timings on activities and the dependency between activities.<\/span><\/p>\nd) Testing responsibilities should be clearly allocated.<\/span><\/p>\ne) Large projects should systematically manage resources.<\/span><\/p>\nf) Appropriate hardware may need to be ordered\/configured\/secured.<\/span><\/p>\ng) Good tracking of bugs is essential as is the tracking of all testing activity.<\/span><\/p>\nh) Good debugging procedures.<\/span><\/p>\ni) The capacity to regression test quickly after a fault has been repaired.<\/span><\/p>\nTest Completion Criteria<\/span><\/strong><\/p>\n\u2022 Bad criteria include:<\/span><\/p>\n\u00a0 \u00a0 \u2013 Stop when we run out of time.<\/span><\/p>\n\u00a0 \u00a0 \u2013 Stop when test cases fail to find errors.<\/span><\/p>\n\u2022 One possible approach is to target a particular residual defect density.<\/span><\/p>\n\u2022 We have estimates of the effectiveness of bug finding for different parts of the system and different approaches to bug finding.<\/span><\/p>\n\u2022 Providing a target RDD (Residual Defect Density) is a good way of structuring the completion of particular stage in the system test.<\/span><\/p>\n<\/p>\n<\/p>\n
<\/p>\n<\/p>\n
<\/p>\n<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
We need to develop a broader range of tasks that present significant problems, require sustained reasoning, and require component information to be integrated to form a conclusion. High-order testing checks that the software meets customer requirements and that the software, …<\/p>\n","protected":false},"author":73,"featured_media":8122,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[110],"tags":[104,105,131,132],"class_list":["post-8067","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-quality-assurance","tag-higher-order-testing-2","tag-validation-testing-2","tag-system-testing","tag-acceptance-testing"],"featured_image_src":"https:\/\/www.bridge-global.com\/blog\/wp-content\/uploads\/2015\/12\/test11.png","author_info":{"display_name":"Saji Xavier","author_link":"https:\/\/www.bridge-global.com\/blog\/author\/saji\/"},"_links":{"self":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/8067","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/users\/73"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/comments?post=8067"}],"version-history":[{"count":3,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/8067\/revisions"}],"predecessor-version":[{"id":8086,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/8067\/revisions\/8086"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/media\/8122"}],"wp:attachment":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/media?parent=8067"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/categories?post=8067"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/tags?post=8067"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}