Rally & Tempest integration (base)

Registered by Alex Baretto

Use tempest (https://github.com/openstack/tempest) inside rally to organize base for verification functionality & for benchmarks scenario based on tempest.

###########
Tempest base
###########

We should implement couple of features inside rally:

new directory:
/verification - that will contains all stuff related to tempest

I think that best way is to have Verifier class that will contains all required method to work with tempest.

For start I think it will be enough to have:

outside of Verifier class
1) dictionally that contains {set_name: list_of_regexp_that_descirbes_set}

inside Verifier class:
1) run_set(set_name) # runs set of tests
2) run_tempest_test() # runs tempest tests
3) run # run all tempest tests

##############
Verification stuff
##############

1) Add CLI command that installs tempest
2) Add CLI command that makes git checkout <#hash_of_commit>, and precreated values like grizzly-1 grizzly-2 ..., havana-1, havana-2, ... , master
3) Add CLI command rally verify start (with 2 optional parameters --set <set_name> , --times <number>, --last (run what you run last time) and --last_failed (run last failed)

That's all

Blueprint information

Status:
Complete
Approver:
Boris Pavlovic
Priority:
High
Drafter:
Boris Pavlovic
Direction:
Approved
Assignee:
Maksym Iarmak
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Boris Pavlovic
Completed by
Boris Pavlovic

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/tempest-verification,n,z

Addressed by: https://review.openstack.org/70131
    Rally & Tempest integration (base) - part 1

Addressed by: https://review.openstack.org/74076
    Rally & Tempest integration (base) - part 2

Gerrit topic: https://review.openstack.org/#q,topic:add_tempest_v3_uri,n,z

Addressed by: https://review.openstack.org/79664
    Make tempest integration works without "sudo"

Addressed by: https://review.openstack.org/81863
    Refactor Tempest installation directories structure

Addressed by: https://review.openstack.org/82446
    Use ConfigParser to generate tempest config

Addressed by: https://review.openstack.org/83031
    Use testr directly in tempest verifier

Addressed by: https://review.openstack.org/83771
    Adds result saver results of `rally verify`

Addressed by: https://review.openstack.org/85584
    Adds 2 new commands to `rally verify`

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.