QA Audit for EMEA bank
Head of QA, QA Automation, QA Analyst
.Net, Oracle, REST API
Stages of Auditing a Product Quality System
SDLC processes built without a dedicated testing team are inside a large banking project that has been running for five years and consists of numerous microservices and integrations with other systems.
Using our methodology, we first prepared the evaluation criteria and information necessary to get started.
Success Metrics
- COBIT/ITIL/RUP/ISO/CMMI/PMBOK maturity level parameters and points to improve to reach target level
- Measurement of development team performance as a current state with points to improve: some text
- Code quality controlsome text
- Defects per 1000 lines of code.
- Technical debt.
- Unit test coverage.
- ROI for Autotests - time for development autotest / time for manual testing
- Time to fix defects
- Time to market (feature to production)
- Average time for regression testing
- Code quality controlsome text
Recommendations for establishing a QA process in a project without dedicated testers and using .NET and Oracle technologies:
1. Establishing a Testing Structure
Initially, it is important to create a structure for testing, even if it is not the current developers' primary responsibility.
Recommendations:
- Appoint responsible individuals in the development team to coordinate the testing process.
- Develop a basic test plan that includes key components:some text
- Functional testing.
- Integration testing.
- Performance testing (if critical).
- Both manual and automated testing.
- Implement testing practices at every stage of development (starting with unit tests).
2. Test Automation
In a .NET project, it's essential to use automation tools to reduce the workload on developers.
Recommendations:
- Unit testing: Set up unit testing to test individual modules of the system.
- Integration testing: This is especially important for microservices and interactions with databases (Oracle).
- Contract testing: For microservices, ensure they interact correctly via APIs.
Tools for .NET:
- NUnit or xUnit: writing unit tests in .NET.
- Moq: To create mocks and stubs for testing individual modules.
- SpecFlow: A tool for writing tests using BDD (Behavior Driven Development), useful for integration testing.
- Selenium: For automating user interface testing (if the project includes a web application).
- RestSharp or Postman: For testing microservice APIs. Postman can also be used for API test automation.
3. Tools for Database Testing
Since Oracle is used in the project, attention should be paid to database testing, query validation, and optimization.
Recommendations:
- Test the performance of queries to Oracle, especially if the system needs to handle large volumes of data.
- Check data integrity during integration testing.
Tools for Oracle:
- SQL Developer: A tool for testing SQL queries, analyzing query performance, and testing stored procedures.
- TSQLt: A framework for testing databases that can be adapted for testing SQL queries and procedures in Oracle.
- Oracle SQL Performance Analyzer: To test query performance and assess how code changes impact database performance.
4. CI/CD and Testing at All Stages
Introducing automatic testing within CI/CD will significantly improve quality control and speed up the release process.
Recommendations:
- Set up a CI/CD process with automatic test runs at every stage of development. For example, run unit tests with every commit, and run integration tests at the build stage.
- Use code reviews as part of the QA process. Integrate code checks for compliance with standards through static code analyzers.
Tools:
- Jenkins or Azure DevOps: Support .NET and Oracle and can help automate test runs.
- SonarQube: A tool for static code analysis, which can integrate with CI/CD to control code quality and test coverage.
5. Test Case Documentation and Management
Even without a dedicated testing team, it's important to organize a process for managing test cases so that every team member can contribute to testing.
Recommendations:
- Establish a system for test case management and documentation of test scenarios. This will help track which features have been tested and which need additional attention.
- Introduce a practice of regression testing to check key functionalities after each change.
Tools:
- TestRail or Zephyr: Tools for managing test cases and test documentation. They can also integrate with Jira or other project management systems.
- Azure Test Plans (in Azure DevOps): Supports creating test cases, maintaining test documentation, and integration with CI/CD processes.
6. Manual Testing and Basic QA Processes
At the start of the project, when automation is not yet developed, it is important to establish basic manual testing processes.
Recommendations:
- Implement a practice of manual verification of new functionality until it is automated.
- Defect tracking: Ensure that every identified defect is documented in a bug tracking system (e.g., Jira) and that tests are created based on it.
Tools:
- Jira or Azure DevOps: For managing tasks and bugs.
- Exploratory Testing: Conduct exploratory testing to identify hidden defects and bugs.
7. Monitoring and Production Support
After the product goes live, it's essential to maintain the system through monitoring and performance testing.
Recommendations:
- Set up a system for monitoring performance and identifying issues in production (e.g., microservice failures, and database crashes).
- Implement a practice of regular stress testing to ensure the system can handle loads.
Tools:
- Grafana + Prometheus: For monitoring the health of microservices and the database.
- JMeter or k6: For load testing to ensure the system can handle large volumes of requests.
Summary of Implementation Plan:
- Establish a basic testing structure:some text
- Assign responsible individuals and develop a test plan.
- Set up unit testing and microservice testing:some text
- Introduce NUnit/xUnit, Moq, and automated API testing via RestSharp or Postman.
- Integrate tests into CI/CD:some text
- Configure Jenkins or Azure DevOps for automated test runs.
- Document and manage test cases:some text
- Use TestRail or Zephyr for test case management.
- Monitoring and control in production:some text
- Install Grafana for monitoring and use JMeter for load testing.
Other Case Studies
With the Right Software, Great Things Can Happen
Facet Search
Voice Interface for Warehouse
Warehouse Management APP
POS Check Constructor
Temperature Monitoring System
HR Helper Bot
Flexible work formats
For convenience we offer several standard models of work with our clients.
Time and material
Model works best when you don’t have a clear scope and want to be deeply involved in the development process.
Choose it if:
- you want to follow agile methodologies
- you need to be flexible due to quickly changing requirements and taste hypothesis time to time.
- you don’t have strict deadlines
All of that doesn’t mean you go in blind. Just like in the fixed-price model, you start your cooperation with planning, but only for the upcoming week or two. So instead of determining and fixing requirements for the whole project, you start fast and can further adjust the scope and priorities.
Fixed-capacity
This model focuses on ensuring the efficiency and velocity of the team. To achieve this, the supplier must ensure that different skill sets of members are assigned to a development squad to effectively deliver the project.
Choose it if:
- you want to follow agile methodologies
- you would like to have a fixed budget for a certain period, but don't have a clear idea on the defined scope of work or specifications
- you understand further support steps and define continued predictable costs
We need to work with the client to clarify the scope of work and prioritize the backlogs before the next iteration starts. As the project progresses, it allows the client to mold the project along the way to take advantage of newly released features.
Fixed-price
Model works best when you don’t have a clear scope and want to be deeply involved in the development process.
Choose it if:
- you want to follow agile methodologies
- you need to be flexible due to quickly changing requirements and taste hypothesis time to time.
- you don’t have strict deadlines
All of that doesn’t mean you go in blind. Just like in the fixed-price model, you start your cooperation with planning, but only for the upcoming week or two. So instead of determining and fixing requirements for the whole project, you start fast and can further adjust the scope and priorities.
The discount is applied annually for a period of 6 months.
We provide discount system for long-term cooperation customers. If the threshold is reached for the sum of all orders from one customer, a discount applies to all new subsequent projects.
