zaqar

Partition Messages Collection by Project

Registered by Kurt Griffiths

In order to keep hot queue indexes in RAM as well as removing project ID from queries and reducing document size, create a separate messages collection per project.

Advise operators to increase nssize to max number of project IDs they expect (probably just max it out to 2047 MB, to provide ~3 million project names)

TBD: Cons of doing this, GC for inactive projects?

Blueprint information

Status:
Not started
Approver:
None
Priority:
Low
Drafter:
None
Direction:
Approved
Assignee:
Kurt Griffiths
Definition:
Approved
Series goal:
None
Implementation:
Deferred
Milestone target:
None

Related branches

Sprints

Whiteboard

Let's do this now to avoid data migration later

Gerrit topic: https://review.openstack.org/#q,topic:bp/partition-messages-collection,n,z

Addressed by: https://review.openstack.org/45966
    perf: Partition messages collection by project

Addressed by: https://review.openstack.org/44340
    fix: Requests get slower when queues have a lot of messages

Performance wasn't great in testing this patch - putting on the backburner.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.