Count resources to check quota in API for cells

Registered by melanie witt

For cellsv2, quota tables are moving to the API database as data global to
a deployment. Currently, for instance delete, quota reservations are made in
the API and then committed in compute. This is a disconnect which couples
compute cells with the API cell. In cellsv2, we endeavor to decouple compute
cells from the API cell as much as possible -- ideally, cells should not
need to have the API database connection in their configuration.

We propose a new approach of counting consumed resources and checking the
count against the quota limits in the API instead of the current reserve/commit
model where a reservation record is created, quota usage records are created
and marked as "in_use" when they are committed, and the reservarion record
deleted.

Blueprint information

Status:
Complete
Approver:
Matt Riedemann
Priority:
Essential
Drafter:
melanie witt
Direction:
Approved
Assignee:
melanie witt
Definition:
Approved
Series goal:
Accepted for pike
Implementation:
Implemented
Milestone target:
milestone icon pike-3
Started by
Matt Riedemann
Completed by
Matt Riedemann

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/cells-move-quota-commits-to-api,n,z

Addressed by: https://review.openstack.org/388262
    Proposal for moving quota commits to API for cells

Gerrit topic: https://review.openstack.org/#q,topic:bp/cells-count-resources-to-check-quota-in-api,n,z

Addressed by: https://review.openstack.org/399750
    Amend proposal to count resources to check quota in API for cells

Gerrit topic: https://review.openstack.org/#q,topic:cells-quota,n,z

Addressed by: https://review.openstack.org/416521
    WIP Count resources to check quota for cells

Addressed by: https://review.openstack.org/410945
    Make Quotas object favor the API database

Addressed by: https://review.openstack.org/410946
    Add online migration to move quotas to API database

I've deferred this to Pike. Let's re-propose the spec for Pike and then continue the work there. -- mriedem 20170203

Addressed by: https://review.openstack.org/431749
    Proposal to count resources to check quota in API for cells

Addressed by: https://review.openstack.org/446239
    Add Quotas.check_deltas() and set the stage for magic to happen

Addressed by: https://review.openstack.org/446240
    Count server group quotas

Addressed by: https://review.openstack.org/446241
    Count tenant_networks quotas

Addressed by: https://review.openstack.org/446242
    Count used_limits quotas

Addressed by: https://review.openstack.org/446243
    Remove useless quota_usage_refresh from nova-manage

Addressed by: https://review.openstack.org/446244
    Add get_count_by_vm_state() to Instance object

Addressed by: https://review.openstack.org/446245
    Add SecurityGroup.get_counts()

Addressed by: https://review.openstack.org/446246
    Add FixedIP.get_count_by_project()

Addressed by: https://review.openstack.org/446247
    Add FloatingIP.get_count_by_project()

Addressed by: https://review.openstack.org/457857
    Add InstanceGroup.get_counts()

Addressed by: https://review.openstack.org/457858
    Add InstanceGroup._remove_members_in_db

Addressed by: https://review.openstack.org/457859
    Count server group members to check quota

Addressed by: https://review.openstack.org/457860
    Count security groups to check quota

Addressed by: https://review.openstack.org/457861
    Count fixed ips for checking quota

Addressed by: https://review.openstack.org/457862
    Count floating ips to check quota

Addressed by: https://review.openstack.org/471729
    Clarify 'cells-count-resources-to-check-quota-in-api'

Addressed by: https://review.openstack.org/471973
    Default to 0 when merging values in limit check

Addressed by: https://review.openstack.org/475957
    Add scatter_gather_all_cells() utility

Addressed by: https://review.openstack.org/477699
    Make key_pairs use check_deltas for quota

Addressed by: https://review.openstack.org/477700
    Make security_group_rules use check_deltas for quota

Addressed by: https://review.openstack.org/481739
    Limit maximum pending resizes for preventing over quota

Gerrit topic: https://review.openstack.org/#q,topic:bug/1681989,n,z

Addressed by: https://review.openstack.org/483133
    Stop counting hw_video:ram_max_mb against quota

Addressed by: https://review.openstack.org/483917
    Default reservations=None in Cells v1 and conductor APIs

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.