Migrate the negative tests to use auto-gen framework

Registered by David Kranz

The framework for generating negative tests from json schemas now exists. This blueprint is to track progress migrating existing negative tests to use that framework. Since the framework is json-only, the existing xml negative tests will remain. Here are the steps to migrate a negative test file:

1. Copy the file xxx_negative.py to xxx_negative_xml.py and remove everything related to JSON
2. Create a new schema in tempest/etc/schemas/.. based on the service type and api being tested
3. Replace xxx_negative.py with a class based on the auto-gen framework.

Note that there are some negative tests that cannot be expressed in the auto-gen framework. In that case, such tests should be preserved in the file being migrated.

Given the state of flux with nova api v2/v3 it would probably be better to focus on the other services. We may find a way to get schemas for nova without having to write all of them in tempest.

Blueprint information

Status:
Complete
Approver:
Matthew Treinish
Priority:
Medium
Drafter:
David Kranz
Direction:
Approved
Assignee:
Marc Koderer
Definition:
Obsolete
Series goal:
Accepted for juno
Implementation:
Implemented
Milestone target:
milestone icon juno-rc
Started by
Matthew Treinish
Completed by
Ken'ichi Ohmichi

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/autogen-negative-tests,n,z

Addressed by: https://review.openstack.org/73982
    Add multiple negative test generator support

Addressed by: https://review.openstack.org/77823
    Add support for negative tests with admin client

Addressed by: https://review.openstack.org/83063
    Deduplicate negative test calls

Addressed by: https://review.openstack.org/95658
    Add qa spec for bp/autogen-negative-tests

Addressed by: https://review.openstack.org/115258
    Change default size for strings to 1

Addressed by: https://review.openstack.org/113899
    Add functools.wraps on all function decorators

Addressed by: https://review.openstack.org/115273
    Reduce complexity during import phase

Gerrit topic: https://review.openstack.org/#q,topic:autogen,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bug/1375142,n,z

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.