What is End To End Testing ?

End-to-end testing is a methodology used to test whether the flow of an application is performing as designed from start to finish. The purpose of carrying out end-to-end tests is to identify system dependencies and to ensure that the right information is passed between various system components and systems.

Why End to End Testing?

Many subsystems are tested in isolation. This is fine for many scenarios that are local to that subsystem, but there are some scenarios that cross the boundary of several subsystems. This is good because a lot of testing can be done in isolation but at some point a complete end to end test needs to be carried out on the whole system.

Objective of E2E Testing

• The objective of E2E is to look at the state of the system as well as its behavior. A realization needs to be made that the user interface is not what we are primarily interested in as it simply provides a representation of the underlying data and displays some of its behavior. In E2E we should be more interested in the underlying data and scenarios across all subsystems.
• Clearly fully automated E2E testing can be very time consuming and to get the most value a 80:20 perspective is useful, Looking at the scenarios of a typical system only about 20% of the functionality is used on a regular basis. So the initial objective should be to automatically test the 20% first and gradually adding the lesser used functionality.

Requirements:

• Given that we need to look at the data and the system behavior and not just the user interface a different set of skills are needed to automate E2E as specialized tools will almost certainly be required to drive the system as well as possible instrumentation of a subsystem to extract the data necessary.
• In addition to development skills analytical skills are necessary to determine the 20% of functionality that is really worth testing.
• Finally as with all projects, good project management is mandatory. Thus an agile approach is most successful, that is, short value driven iterations delivering useful tests as quickly as possible to get feedback as early as possible on the effectiveness and potential cost of automating E2E.

System Testing vs. End-to-End Testing

• System Testing and “End-to-End” Testing can be very similar but typically the difference is that with System Testing you are exercising the functionality of your application with how it interfaces with other components (applications) in your environment. An example of this is with data feeds, your app may send a data file/feed to another application in your system. In Functional testing testers will either use a “stub” for the interface, or they will just capture the data file/feed and verify its content but not actually send it to the other application.  But in System Testing you want to verify that the app will successfully connect and pass the data file to the other system, and that the other system can open it up and process the data.
• End-to-End Testing is when you take a process from the very beginning all the way to the end – like accepting an order from a customer and then processing that order all the way through fulfillment, accounting and shipping. This is basically exercising an entire “workflow”. Although System Testing is similar, in System Testing you do not have to complete the entire workflow but may exercise individual interfaces one at a time.  However by the end of System Testing most test environments will exercise (or should exercise) an entire end-to-end test.
• System Testing and End-to-End Testing is very important in your testing efforts and where lots of defects appear because a lot of testers do not take processes all the way through to other systems. The biggest challenge with this type of testing is that you really need a complete system in your QA environment.

Phases in End to End Testing

End to End Testing consists of the following phases:

1. Analyze end to end requirements
2. Create Test Plan
3. Create Test Cases
4. Testing Environment Readiness
5. Execute Test Cases
6. Review Test Results
7. Prepare Test Report

 

Analyze the end to end testing Requirements :

This phase consists of analyzing requirements from High level Design Documents and requirement documents, and arriving at the prioritized test objectives, The entry level of this phase is to make sure the requirements and High level design documents are available and the task of this phase is to study and review the system design documents, understand, identify functional interfaces. At the end this phase testing team should be completely clear about the requirements.
Creation of Test plan:

• This phase consists of creation of Integration Test Plan. The plan covers items such as Test Criteria, Scope, Environment, Estimations, Risks, Defect Tracking and Build Management.
• With the clear knowledge of the system and application functionality and requirements, test data creation and test plan needs to be created. Once test plan is created the plan must be reviewed with customer and get the test plan signed – off.

Creation of test cases:
With the Test plan signed off in the last phase, this phase consist of preparation of test Cases based on the requirements, functionality as per the test plan.
Test environment readiness:
This phase consists of preparation of Test Cases based on the requirements, functionality mentioned in the test plan. With the test plan signed off, Identify and create a test plan in the repository to upload all the test cases which was created in the earlier phase. Setup the access to applications for required users. Verification of environment, Access rights applications and Test data should be completed in this phase. Execution of environment shakedown test cases
Execution of test cases:
Once the environment is ready for execution, this phase consists of test cases, evaluation of testing and entering the identified defects into defect tracker
Review of Test results:
This phase consists of review the test results; once the execution is complete analyze the results and the defect logs.
Preparation of Test report:
This consists of preparation of Test report. This report contains the summary information such as defect data, test evaluation summary

Advantages of End To End Testing

• Involves complete application testing which mimics the real world, hence any defect/issues related to interfaces can be resolved during this stage itself.
• Fault localization easier
• Few or no drivers needed
• Different order of testing/implementation possible
• Major design flaws can be found first
• Flaws in the interfaces can be found out easily

Problems and Issues

• Unfortunately a lot of testing of complex systems is carried out manually because the level of effort to automate the E2E test is considered to be too expensive. This is primarily due to the fact that testers take a user interface centric approach to testing and do not have the necessary development skills to produce the tools needed to automate testing. Often they rely on tools such as QTP / RFT / Win Runner to drive the user interface.
• Manual testing is unfortunate because as we all know systems are not static, they evolve. Sometimes this evolution is minor and does not impact other systems, or so we think. At other times the impact is greater and the E2E tests need to be rerun. At this point we take the hit re-testing system with the inevitable delay to delivery. But what if a test fails? Do we restart and rerun all our tests after the fix is made. In most cases this is unlikely. The failed test is simply rerun and the hope is there are no side effects.
• This haphazard way of testing may be fine when the risk to life or loss of money is low, but it becomes increasingly difficult to justify when the risks are high. A better approach is to automate end to end testing so quality is guaranteed.
• Given that more business are trying to become agile in order to respond quickly to changing business needs the need for automated E2E becomes stronger, for without it testing cycles can significantly delay the introduction of new functionality. I’d go as far as saying you won’t be truly agile until you’ve automated E2E testing.

Example

End to end testing : involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Consider a telecom service provider, An end to end testing in this domain would start right from provisioning of orders in the upstream system i.e., raising a request for a service from the service provider portal till to verifying the downstream systems NMS, Billing system, reporting systems.
Once the provisioning order is triggered from upstream system, verify that the order an customer ref/order no is generated. Now login to order management system and verify that order has flown to this system.
Login to downstream systems into NMS ( network Management system),billing systems and verify that the order details are updated in all these downstream systems correctly.
Refer to the HLD or requirements doc to get know  changes to test in all these systems.