Software Testing-

1. What is Software Testing ?
   
   Testing involves operation of a system or application under controlled conditions and evaluating the results (eg, 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. It is oriented to 'detection'. 
   
   
2. What is black box, white box, gray box testing?
   
   Black-box and white-box are test design methods. Black-box test design treats the system as a “black-box”, so it doesn’t explicitly use knowledge of the internal structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box. White-box test design allows one to peek inside the “box”, and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box and clear-box.

   While black-box and white-box are terms that are still in popular use, many people prefer the terms 'behavioral' and 'structural'. Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but it's still discouraged. In practice, it hasn't proven useful to use a single test design method. One has to use a mixture of different methods so that they aren't hindered by the limitations of a particular one. Some call this 'gray-box' or 'translucent-box' test design, but others wish we'd stop talking about boxes altogether.
   
   

3. What are unit and integration testing?
   
   Unit - The smallest compliable component. A unit typically is the work of one programmer (At least in principle). As defined, it does not include any called sub-components (for procedural languages) or communicating components in general.
   Unit Testing: in unit testing called components (or communicating components) are replaced with stubs, simulators, or trusted components. Calling components are replaced with drivers or trusted super-components. The unit is tested in isolation.
   
   Integration - Two components (actually one or more) are said to be integrated when:
   a. They have been compiled, linked, and loaded together.
   b. They have successfully passed the integration tests at the interface between them.
   
   Thus, components A and B are integrated to create a new, larger, component (A,B).So, In Integration testing, We basically check the output of A is correct or not, the way with which the output will go to B means the communication way between A & B is correct or not and the input received by B is correct or not.
   
   
4. What's the difference between load and stress testing ?
   
   One of the most common, but unfortunate misuse of terminology is treating “load testing” and “stress testing” as synonymous. The consequence of this ignorant semantic abuse is usually that the system is neither properly “load tested” nor subjected to a meaningful stress test.

   Stress testing is subjecting a system to an unreasonable load while denying it the resources (e.g., RAM, disc, mips, interrupts, etc.) needed to process that load. The idea is to stress a system to the breaking point in order to find bugs that will make that break potentially harmful. The system is not expected to process the overload without adequate resources, but to behave (e.g., fail) in a decent manner (e.g., not corrupting or losing data). Bugs and failure modes discovered under stress testing may or may not be repaired depending on the application, the failure mode, consequences, etc. The load (incoming transaction stream) in stress testing is often deliberately distorted so as to force the system into resource depletion.

   Load testing is subjecting a system to a statistically representative (usually) load. The two main reasons for using such loads is in support of software reliability testing and in performance testing. The term 'load testing' by itself is too vague and imprecise to warrant use. For example, do you mean representative load,' 'overload,' 'high load,' etc. In performance testing, load is varied from a minimum (zero) to the maximum level the system can sustain without running out of resources or having, transactions >suffer (application-specific) excessive delay.

   A third use of the term is as a test whose objective is to determine the maximum sustainable load the system can handle. In this usage, 'load testing' is merely testing at the highest transaction arrival rate in performance testing.
   
   

5. What's the difference between QA and testing? 
   
   QA is more a preventive thing, ensuring quality in the company and therefore the product rather than just testing the product for software bugs.

   TESTING means 'quality control'
   QUALITY CONTROL measures the quality of a product 
   QUALITY ASSURANCE measures the quality of processes used to create a quality product.
   
   

6. What is Software Quality Assurance? 
   
   Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.
   
   
7. Why does Software have bugs? 
   
   
   • Miscommunication or no communication - as to specifics of what an application should or shouldn't do (the application's requirements). 
     
   • Software complexity - the complexity of current software applications can be difficult to comprehend for anyone without experience in modern-day software development. Windows-type interfaces, client-server and distributed applications, data communications, enormous relational databases, and sheer size of applications have all contributed to the exponential growth in software/system complexity. And the use of object-oriented techniques can complicate instead of simplify a project unless it is well-engineered. 
     
   • Programming errors - programmers, like anyone else, can make mistakes. 
     
   • changing requirements - the customer may not understand the effects of changes, or may understand and request them anyway - redesign, rescheduling of engineers, effects on other projects, work already completed that may have to be redone or thrown out, hardware requirements that may be affected, etc. If there are many minor changes or any major changes, known and unknown dependencies among parts of the project are likely to interact and cause problems, and the complexity of keeping track of changes may result in errors. Enthusiasm of engineering staff may be affected. In some fast-changing business environments, continuously modified requirements may be a fact of life. In this case, management must understand the resulting risks, and QA and test engineers must adapt and plan for continuous extensive testing to keep the inevitable bugs from running out of control.
     
   • time pressures - scheduling of software projects is difficult at best, often requiring a lot of guesswork. When deadlines loom and the crunch comes, mistakes will be made. 
     
   • poorly documented code - it's tough to maintain and modify code that is badly written or poorly documented; the result is bugs. In many organizations management provides no incentive for programmers to document their code or write clear, understandable code. In fact, it's usually the opposite: they get points mostly for quickly turning out code, and there's job security if nobody else can understand it ('if it was hard to write, it should be hard to read'). 
     
   • software development tools - visual tools, class libraries, compilers, scripting tools, etc. often introduce their own bugs or are poorly documented, resulting in added bugs.
   
   
8. What is verification & validation? 
   
   Verification typically involves reviews and meetings to evaluate documents, plans, code, requirements, and specifications. This can be done with checklists, issues lists, walkthroughs, and inspection meetings. Validation typically involves actual testing and takes place after verifications are completed. The term 'IV & V' refers to Independent Verification and Validation.
   
   
9. What is a 'walkthrough'? 
   
   A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or no preparation is usually required.
   
   
10. What's an 'inspection'? 
    
    An inspection is more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requiremenor a test plan, and the purpose is to find problems and see what's missing, not to fix anything.The result of the inspection meeting should be a written report.
    
    
11. What kinds of testing are there? 
    
    
    • Black box testing - not based on any knowledge of internal design or code. Tests are based on requirements and functionality. 
      
    • White box testing - based on knowledge of the internal logic of an application's code. Tests are based on coverage of code statements, branches, paths, conditions. 
      
    • Unit testing - the most 'micro' scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code. 
      
    • Integration testing - testing of combined parts of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems. 
      
    • Functional testing - black-box type testing geared to functional requirements of an application; this type of testing should be done by testers. This doesn't mean that the programmers shouldn't check that their code works before releasing it (which of course applies to any stage of testing.) 
      
    • System testing - black-box type testing that is based on overall requirements specifications; covers all combined parts of a system. 
      
    • Sanity testing - typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or destroying databases, the software may not be in a 'sane' enough condition to warrant further testing in its current state. 
      
    • Regression testing - re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing. 
      
    • Load testing - testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system's response time degrades or fails. 
      
    • Stress testing - term often used interchangeably with 'load' and 'performance' testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc. 
      
    • Install/uninstall testing - testing of full, partial, or upgrade install/uninstall processes. 
      
    • Security testing - testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques. 
      
    • Compatability testing - testing how well software performs in a particular hardware/software/operating system/network/etc. environment. 
      
    • Ad-hoc testing - similar to exploratory testing, but often taken to mean that the testers have significant understanding of the software before testing it. 
      
    • Alpha testing - testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers. 
      
    • Beta testing - testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers. 
      
    
    
12. What are 5 common problems in the software development process? 
    
    
    • Poor requirements - if requirements are unclear, incomplete, too general, or not testable, there will be problems.
      
    • Unrealistic schedule - if too much work is crammed in too little time, problems are inevitable. 
      
    • Inadequate testing - no one will know whether or not the program is any good until the customer complains or systems crash. 
      
    • Featuritis - requests to pile on new features after development is underway; extremely common. 
      
    • Miscommunication - if developers don't know what's needed or customer's have erroneous expectations, problems are guaranteed. 
      
    
    
13. What are 5 common solutions to software development problems? 
    
    
    • Solid requirements - clear, complete, detailed, cohesive, attainable, testable requirements that are agreed to by all players. Use prototypes to help nail down requirements. 
      
    • Realistic schedules - allow adequate time for planning, design, testing, bug fixing, re-testing, changes, and documentation; personnel should be able to complete the project without burning out. 
      
    • Adequate testing - start testing early on, re-test after fixes or changes, plan for adequate time for testing and bug-fixing. 
      
    • Stick to initial requirements as much as possible - be prepared to defend against changes and additions once development has begun, and be prepared to explain consequences. If changes are necessary, they should be adequately reflected in related schedule changes. If possible, use rapid prototyping during the design phase so that customers can see what to expect. This will provide them a higher comfort level with their requirements decisions and minimize changes later on. 
      
    • Proper communication - require walkthroughs and inspections when appropriate; make extensive use of group communication tools - e-mail, groupware, networked bug-tracking tools and change management tools, intranet capabilities, etc.; insure that documentation is available and up-to-date - preferably electronic, not paper; promote teamwork and cooperation; use prototypes early on so that customers' expectations are clarified. 
      
    
    
14. What is software 'quality'? 
    
    Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free. 
    
    
15. What is SEI & CMM ? 
    
    
    • SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes. 
      
      
    • CMM = 'Capability Maturity Model', developed by the SEI. It's a model of 5 levels of organizational 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMM ratings by undergoing assessments by qualified auditors. 
      
      Level 1 (Initial)- characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place;successes may not be repeatable.
      Level 2 (Repeatable) - software project tracking, requirements management,realistic planning, and configuration management processes are in place; successful practices can be repeated.
      Level 3 (Defined) - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
      Level 4 (Managed) - metrics are used to track productivity, processes,and products. Project performance is predictable,and quality is consistently high.
      Level 5 (Optimizing) - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.
      
      
16. What is the 'software life cycle'? 
    
    The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects.
    
    
17. Will automated testing tools make testing easier? 
    
    Possibly. For small projects, the time needed to learn and implement them may not be worth it. For larger projects, or on-going long-term projects they can be valuable.
    A common type of automated tool is the 'record/playback' type. For example, a tester could click through all combinations of menu choices, dialog box choices, buttons, etc. in an application GUI and have them 'recorded' and the results logged by a tool. The 'recording' is typically in the form of text based on a scripting language that is interpretable by the testing tool. If new buttons are added, or some underlying code in the application is changed, etc. the application can then be retested by just 'playing back' the 'recorded' actions, and comparing the logging results to check effects of the changes. The problem with such tools is that if there are continual changes to the system being tested, the 'recordings' may have to be changed so much that it becomes very time-consuming to continuously update the scripts. Additionally, interpretation of results (screens, data, logs, etc.) can be a difficult task. 
    
    
18. What makes a good test engineer? 
    
    A good test engineer has a 'test to break' attitude, an ability to take the point of view of the customer, a strong desire for quality, and an attention to detail. Tact and diplomacy are useful in maintaining a cooperative relationship with developers, and an ability to communicate with both technical (developers) and non-technical (customers, management) people is useful. Previous software development experience can be helpful as it provides a deeper understanding of the software development process, gives the tester an appreciation for the developers' point of view, and reduce the learning curve in automated test tool programming. Judgment skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited.
    
    
19. What's a 'test plan'? 
    
    A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:
    • Title 
      
    • Identification of software including version/release numbers 
      
    • Revision history of document including authors, dates, approvals 
      
    • Table of Contents 
      
    • Purpose of document, intended audience 
      
    • Objective of testing effort 
      
    • Software product overview 
      
    • Relevant related document list, such as requirements, design documents, other test plans, etc. 
      
    • Relevant standards or legal requirements 
      
    • Traceability requirements 
      
    • Relevant naming conventions and identifier conventions 
      
    • Overall software project organization and personnel/contact-info/responsibilties 
      
    • Test organization and personnel/contact-info/responsibilities 
      
    • Assumptions and dependencies 
      
    • Project risk analysis 
      
    • Testing priorities and focus 
      
    • Scope and limitations of testing 
      
    • Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable 
      
    • Outline of data input equivalence classes, boundary value analysis, error classes 
      
    • Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems 
      
    • Test environment validity analysis - differences between the test and production systems and their impact on test validity. 
      
    • Test environment setup and configuration issues 
      
    • Software migration processes 
      
    • Software CM processes 
      
    • Test data setup requirements 
      
    • Database setup requirements 
      
    • Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
    • Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs 
      
    • Test automation - justification and overview 
      
    • Test tools to be used, including versions, patches, etc. 
      
    • Test script/test code maintenance processes and version control 
      
    • Problem tracking and resolution - tools and processes 
      
    • Project test metrics to be used 
      
    • Reporting requirements and testing deliverables 
      
    • Software entrance and exit criteria 
      
    • Initial sanity testing period and criteria 
      
    • Test suspension and restart criteria 
      
    • Personnel allocation 
      
    • Personnel pre-training needs 
      
    • Test site/location 
      
    • Outside test organizations to be utilized and their purpose, responsibilities, deliverables, contact persons, and coordination issues 
      
    • Relevant proprietary, classified, security, and licensing issues. 
      
    • Open issues 
      
    • Appendix - glossary, acronyms, etc.
    
    
20. What's a 'test case'? 
    
    A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results. 
    Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible. 
    
    
21. What should be done after a bug is found? 
    
    The bug needs to be communicated and assigned to developers that can fix it. After the problem is resolved, fixes should be re-tested, and determinations made regarding requirements for regression testing to check that fixes didn't create problems elsewhere. If a problem-tracking system is in place, it should encapsulate these processes. A variety of commercial problem-tracking/management software tools are available. 
    
    
22. What is 'configuration management'? 
    
    Configuration management covers the processes used to control, coordinate, and track: code, requirements, documentation, problems, change requests, designs, tools/compilers/libraries/patches, changes made to them, and who makes the changes.
    
    
23. How can it be known when to stop testing? 
    
    This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are:
    • Deadlines (release deadlines, testing deadlines, etc.) 
      
    • Test cases completed with certain percentage passed 
      
    • Test budget depleted 
      
    • Coverage of code/functionality/requirements reaches a specified point 
      
    • Bug rate falls below a certain level 
      
    • Beta or alpha testing period ends