Infrastructure for Running Long Tests
It has become apparent that we would benefit from having testing infrastructure in place for long-running OpenQuake engine jobs. We think that if large jobs are continuously run against the latest oq-engine code we can better test for performance and scalability, as well as catch job configuration corner case issues.
Adding more CI-like infrastructure is one way to solve this problem.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Medium
- Drafter:
- Lars Butler
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Discussion
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
-
0.7.0
- Started by
- Completed by
Related branches
Related bugs
| Bug #977810: Set up long-running test environment for oq-engine | Fix Released |
| Bug #977811: New VPS for long-running test infrastructure | Fix Released |
| Bug #978861: Document existing Jenkins setup | Fix Released |
Sprints
Whiteboard
Use Jenkins (or similar) to execute long-running QA tests on the open-quake engine and related projects.
Some defects are identified only with larger sets then can be conveniently executed in a short time on a development workstation. In order to identify such problems we should execute long running tests periodically/
It seems likely that a dedicated server will be required in order to prevent the standard Jenkins builds from blocking or failing due to lack of computing resources.
The defect identified 2012-03-22 installing version 0.6.0 on the cluster suggests that a performing tests periodically in a distributed environment would be beneficial in detecting some kinds of problem.
The suggested approach would be to turn up another Linode, in the London datacenter where the existing CI server is located. The 2nd CI configuration would have a full OpenQuake install, but would only act as a worker with the Primary CI server acting as the Control Node. This will emulate our cluster environment.
Total effort to create this solution would be in the range of 3 days effort.
