Automated Data Migration Testing for Luxaviation
Luxaviation is one of the largest private aviation operators with a fleet of 260 aircraft and over 1,700 employees worldwide.
In the scope of an Integration programme in the aviation application of all Luxaviation entities, the project objective is to migrate a sixth Luxaviation entity, Luxaviation Belgium, to ” FlightWare “.
More than 100,000 of data in the following categories had to be migrated: accounts, account parameters, contacts, pax and pax credentials, flight logs, ground logs, and positioning.
Data Analysis & Data Quality checks
Data Migration Testing
API-based Test Automation
UI-based Test Automation
- On top of having data files as inputs with a big volume of data, the data quality was very poor and not following conventions defined for the different types of data
- Data collection was happening outside the project iterations. Input files were received late and/or with very poor quality of data, adding extra work during the sprints to the development team
- Limited number of people in the Agile team
- Challenging and inflexible target release dates
- Dependence on a third-party development team who had to develop new features in the target system to be able to migrate all the data
target system’s database schema doesn’t follow rules applied at system’s UI level (validation rules between UI fields)
- Migrate all Abelag data in Flightware correctly according to Flightware specifications
- Choice of the tools to test the migration .
- Analyse the features of FlightWare
- Transform the data received according to the rules in FlightWare specifications.
- Test whole quite the lines of datum of the file of entrance .
- Find the working methodology to make a good migration
- Ensure the overall quality of the data in FlightWare
To assure the quality of this data migration, the tester worked very closely with the business analyst and the developer to support the project.
During the project, the tester was :
- contributing to the data quality check activities with the Business Analyst (use of Talend Data Preparation tool and manual checks)
- responsible for the data correctness, data completeness, data consistency in the target system
To detect issues as early as possible, the following approach was applied :
- Data quality testing: Test the data completeness and consistency after extraction from the customer legacy system and before its encoding, transformation and loading in the new system
- Component testing: Test the output of encoding and transformation jobs (unit tests were performed by developers on the jobs themselves) generated by the data migration tool (Talend)
- Data loading testing: Test the loading of data through the Salesforce’s database API
- Acceptance testing: Test the migrated data from the system’s UI to validate customer’s use cases
To ensure a very high coverage of data migration tests in a limited timeframe, API- and UI-based automated tests were used. Since it was the 6th migration to this target system, a lot of automated scripts were reused from previous migration projects.
Technologies & methodologies used
In comparison with the previous migration projects,
- Developments and testing were finished in time (even 2 weeks before due date).
- 100% of API and UI-based tests were automated.
- 1 testing FTE dedicated to the project vs 1.8 FTE.
- More than half of the defects have been detected in the early stages of each iteration (during the data quality testing). Critical and major defects have also been raised at this stage.
- Only minor but very isolated defects were detected after loading of the data in the target system.
- Move to production was made with high confidence and went without unexpected issues.