tempest

Tempest CLI Unit Test Coverage

Registered by Ghanshyam Mann

Tempest has many CLI - http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd

But There is no or very minimal unit test coverage for CLI and many time we broke them. One good example of that while migrating Tempest run to stestr [1] and we broke the Tempest run in many way because of no unit tests [2].

We need full coverage of each CLI, it's arg and behaviour in both positive and negative way. Exiting test case can be referred [3] and all new test cases will go under same location.

This specless BP is to track that work.

[1] https://review.openstack.org/#/c/504345/
[2]
  - https://review.openstack.org/#/c/547073/
  - https://review.openstack.org/#/c/547106/
  - https://review.openstack.org/#/c/547276/
  - https://review.openstack.org/#/c/546905/

[3] https://github.com/openstack/tempest/tree/master/tempest/tests/cmd

Blueprint information

Status:
Not started
Approver:
Ghanshyam Mann
Priority:
Medium
Drafter:
Ghanshyam Mann
Direction:
Approved
Assignee:
Soniya Murlidhar Vyas
Definition:
Approved
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/tempest-cli-unit-test-coverage,n,z

Addressed by: https://review.openstack.org/587748
    [WIP] Adding missing test cases in " test_run.py"

Gerrit topic: https://review.openstack.org/#q,topic:dupRemove,n,z

Addressed by: https://review.openstack.org/589842
    Fixing test "test_verify_glance_version_no_v2_with_v1_1"

Gerrit topic: https://review.openstack.org/#q,topic:configtest,n,z

Addressed by: https://review.openstack.org/586950
    [WIP] Adding unit test of setting config path for "tempest run"

Addressed by: https://review.openstack.org/597381
    Adding missing test cases of "test_verify_tempest_config"

Addressed by: https://review.openstack.org/601973
    Adding missing test cases of tempest workspace

Gerrit topic: https://review.openstack.org/#q,topic:bug/1791007,n,z

Addressed by: https://review.openstack.org/603058
    Removing duplicate variable "parsed_args.config_file"

Gerrit topic: https://review.openstack.org/#q,topic:606880,n,z

Addressed by: https://review.openstack.org/606909
    Adding missing test case of account generator resources

Gerrit topic: https://review.openstack.org/#q,topic:bug/1766354,n,z

Addressed by: https://review.openstack.org/577730
    Fixing ignored value of --config parameter in tempest account-generator

Addressed by: https://review.openstack.org/613219
    Handling invalid path of workspace register and move.

Gerrit topic: https://review.openstack.org/#q,topic:608878,n,z

Addressed by: https://review.openstack.org/608878
    Test coverage of tempest account-generator for config and account file

Addressed by: https://review.openstack.org/620920
    Trivial Fix: Checking config file actually exist or not

Gerrit topic: https://review.openstack.org/#q,topic:bug/1808473,n,z

Addressed by: https://review.openstack.org/625162
    Checking config file actually exist or not in tempest run

Addressed by: https://review.openstack.org/630024
    Adding coverage of account-generator for tag and concurrency parameters

Gerrit topic: https://review.openstack.org/#q,topic:bug/1811349,n,z

Addressed by: https://review.openstack.org/630128
    Correcting concurrency parameter for negative value

Addressed by: https://review.openstack.org/633666
    Add whitelist based tests for tempest run command.

Addressed by: https://review.openstack.org/633694
    Add black-list based tests for tempest run command

Addressed by: https://review.openstack.org/633700
    Adding missing tests for regex and black-regex in tempest run

Addressed by: https://review.openstack.org/639622
    Add concurrency based tests for tempest run

Addressed by: https://review.openstack.org/639956
    Add unit tests for tempest run for generic parameters

Gerrit topic: https://review.opendev.org/#/q/topic:bug/1811349

Addressed by: https://review.opendev.org/630128
    Correcting concurrency parameter for negative value

Gerrit topic: https://review.opendev.org/#/q/topic:bp/tempest-cli-unit-test-coverage

Addressed by: https://review.opendev.org/608878
    Test coverage of tempest account-generator for config and account file

Gerrit topic: https://review.opendev.org/#/q/topic:608878

Addressed by: https://review.opendev.org/620920
    Checking config file actually exist or not in account generator

(gmann:2020-05-13)need to add test fo tempest/lib/cmd/check_uuid.py also so that we can verify the changes.

Gerrit topic: https://review.opendev.org/#/q/topic:fix/account_generator

Addressed by: https://review.opendev.org/733127
    Fix the argument check for account-generator

Addressed by: https://review.opendev.org/731231
    Add check_service_availability test

Gerrit topic: https://review.opendev.org/#/q/topic:fix/import_line_decorators

Addressed by: https://review.opendev.org/740723
    Create tests for check-uuid CLI

Gerrit topic: https://review.opendev.org/#/q/topic:feature/check_uuid_cli_tests

Addressed by: https://review.opendev.org/744048
    Add import line unit tests for check_uuid command

Addressed by: https://review.opendev.org/755072
    Add more unittests to tempest-cli

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.
Edit subscription

Subscribers

No subscribers.