Consul based matchmaker driver

Registered by Harish Kumar

To have consul (https://consul.io/intro/) based matchmaker driver which avoid single point of failure as redis drive (matchmaker_redis) and static nature of ring driver (matchmaker_ring).

Problem statement
=================
Here are the problems with current matchmaker drivers.

1. matchmaker_ring
  This driver support loading the hosts per topic from static json file. The problem with this driver is the static nature, say for example, if you have two nova scheduler hosts say host1 and host2, and even if host1 is down, this driver cause the request direct to that node. So there is no failure protection in this driver.

2. matchmaker_redis
  This driver use redis to load the hash table for the hosts. This support dynamic host/topic registrations, host expiration, and hooks for consuming applications to acknowledge or neg-acknowledge topic.host service availability. The problem with this driver is that the redis is going to be single point of failure - even redis cluster is not stable to survive network partitions and other failure scenarios.

Solution Details
================
The new driver which use reliable, distributed, consul service discovery infrastructure to use for matchmaking.

This driver may run in two modes:

1. Consul to manage the host availability
  There would be users who use consul service discovery/healthcheck infrastructure which could setup to add the services in consul service catalog with the topic names like scheduler for nova scheduler, conductor for nova conductor, compute for nova compute etc with appropriate healthchecks. In this case consul can manage the host/service availability by automatically register/unregister the hosts from consul service catalog.

  In this mode, the driver would be derived from MatchMakerBase, so it will not handle register/unregister/host expire on itself, it just depend consul on those operations. In this mode, the driver expect the consul service in the name of topics. It may make sense to use this mode if there is any operational toolsets around consul which may affect how openstack services/hosts to be used.

2. Driver to manage register/unregister hosts on consul distributed KV store
  In this case the driver would be handling dynamic host/topic registrations, host expiration, and hooks for consuming applications to acknowledge or neg-acknowledge topic.host service availability. It would update consul KV store on the operations to host/topic registration, unregistration, host expiration etc. In this mode, there is no special assumption or requirements from consul side, the openstack would be handling all the operations itself.

About consul
============
Consul is a reliable, distributed service discovery toolset which have built-in distributed key-value store, healthchecking, and with multi-datacenter capability. consul have one rest based api and standard dns based interfaces for service discovery - all active services and its server IP addresses will be exposed using these interfaces.

Consul has integrated the service discovery with built-in healthchecking mechanism, so that only healthy service instances would be exposed to the infrastructure - if the healthcheck fail for a service instance, consul will take down that instance from its service catalog automatically.

Please see https://consul.io/intro/ for introduction and basic concepts.

Blueprint information

Status:
Started
Approver:
None
Priority:
Undefined
Drafter:
Harish Kumar
Direction:
Needs approval
Assignee:
Harish Kumar
Definition:
New
Series goal:
None
Implementation:
Slow progress
Milestone target:
None
Started by
Harish Kumar

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.