Improve Horizon testing
Summary
=======
Improve Horizon testing using Selenium and implement cross-project job to test
Horizon changes over plugins.
Motivation
==========
We need better test coverage both for Horizon itself and plugins. We should be
sure that submitted patches won't break anything in Horizon or in any plugin.
Description
===========
We don't have Selenium-based tests now. the selenium-headless job always reports
success. Integration tests are totally broken and we even don't run them on
gates. We need to fix selenium-headless job and integration tests too.
It would be great to have new gate job per each plugin described at [1] per any
Horizon code change to be sure that we don't break anything. The same job with
plugin-specific selenium or integration tests should be executed against each
Horizon plugin's change request.
Action Items
============
* Get a rid of Nose test runner: it's in a maintainance mode for the laset two
years and doesn't work as expected [2]
* To not break any plugin we need to remove
``TEST_RUNNER = 'django_
be sure that it works without nose
* Use django.test.tag to split tests into the following groups: unit, selenium,
integration. We use environment variables for it now. It leads to having a
lot of skipped tests per each test run. It makes troubleshooting more complicated.
* Add new gate jobs described in the 'Testing' section of this blueprint.
* Update `Contributing Guide` if needed
Testing
======
* Selenium job should be fixed
* Basic functional tests framework should be implemented
* Add jobs for each supported Horizon plugin to Horizon gates
* Add Horizon integration job to plugins gates
NOTE: All new jobs should be enabled in non-voting mode by default. It's up to
each team to make this job voting once it becomes stable.
Doc Impact
=========
Current `Contributing Guide` [3] looks corret but we need to check it
Links
=====
[1] https:/
[2] https:/
[3] https:/
[4] https:/
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Ivan Kolodyazhny
- Direction:
- Needs approval
- Assignee:
- Ivan Kolodyazhny
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
Some related patches:
https:/
https:/
https:/
Gerrit topic: https:/
Gerrit topic: https:/
Gerrit topic: https:/
Addressed by: https:/
Integration test fixes
Addressed by: https:/
integration test fixes
Addressed by: https:/
integration tests - added chrome browser support
Addressed by: https:/
WIP. Integration tests job