Service graph: next generation of service chaining

For the next version of service chaining, we have the following goals:
1. Allow any arbitrary topology of services. This means that instead of a service chain, it will really be a service graph. For this, the following will be required
 - A service instance could have more than one left and/or right interfaces
 - There should be a way to specify how those ports are connected
2. Ability to specify whether routes are exchanged bidirectionally, only in one direction or not at all. Currently, this is based on service mode being in-network-nat or not and is not available for transparent services at all.
3. Allow the customer to independently spawn a VM and use it as a service instance. This allows us to not having to support every nova related feature for service instances.
4. Decouple network connectivity from ACL. Currently, both are derived from network policies. In some cases, it is desirable to have these two separate.

For these, below are the items discussed:

Changes to service template:
1. Add a new field to determine L2 or L3 service
2. Interface list should be able to have any number of left and right interfaces, each with their own label (e.g. left1, left2, right1, right2, etc).
3. Add a new field to determine route-exchange. This field could have values: bidirectional, left-to-right, right-to-left, none.

Changes to service instance:
1. Add a new field called ‘reusable’ which allows the service instances to be part of multiple service graphs. Currently, only transparent services are reusable, but with new scheme, any service could be reusable.

New object: ServiceGraph
We will define a new object called ServiceGraph. This object will define the topology of the services using a series of PortClass objects (defined later).

New object PortClass:
We will also define another new object called PortClass. A PortClass object simply represents all ports of a service instance of the same type (have the same label in the template) and belong to the same service graph. It will have these properties/links:
 - Parent: service instance to which it belongs
 - ServiceGraph ref: reference to service graph to which it belongs
 - PortClass refs: references to other PortClass objects to which it is connected
 - VirtualNetwork refs: references to virtual networks to which it is connected
 - Other port specific properties:
  - interface static routes
  - shared ip
  - allowed address pair
  - health-check
This object allows us to define properties on a single port object even for a scaled out service instance and through links to other PortClass objects, it helps us build the service graph. Connections to virtual network usually represent an end-point in the graph at which the networks are attached. When a network VN1 is attached to a PortClass object representing left interfaces, schema transformer will find all networks attached to PortClasses representing right interfaces of the same service graphs such that all service instances connecting the two network allow right to left route exchange. Similarly, for networks attached to right PortClasses, we will find all networks connected to left PortClasses such that all service instances in the path allow left to right route exchange

New object RoutingPolicy:
We will define a new object called RoutingPolicy, which will define how routes are imported from one routing instance to another. Exact format of routing policy rules is TBD.
In the first phase, we will attach it to the service instance objects and specify whether it applies to left interface or right interface. Schema transformer will then also link the corresponding service routing instances with the appropriate routing policies. Later, we will allow these routing policies to be attached to PortClass objects. This will allow intermediate routing instances to have their own routing policy.

PortTuple object:
We will define a new object called PortTuple. Each PortTuple will represent all ports belonging to the same ‘VM’ of a service instance. For a service instance with scale out of 3, we will have 3 PortTuple objects. Users will be allowed to create VMI objects, link them to PortTuples and add the PortTuples to a service instance. However, for traditional service instances also, service monitor will create PortTuples so that schema transformer always deals with them instead of VMs.

Sharing L3 service instances:
Currently, only transparent services can be shared/reused. However, we already have a concept of vlan sub-interfaces. We could share L3 service instances using these vlan sub-interfaces. Details are still to be worked out.

New keyword ‘via’ in network policy action list:
We will define a new field in action list of network policy rules. The value will be FQ name of a service graph. This will be used only to create ACLs and will not be used for routing. This rule is optional for the user and is only required if they want to use the service graph only for certain type of traffic (e.g. only for port 80). E.g.
VN1 <> VN2, port 80, via ServiceGraph1

Intra-network service chain:
To support an intra-network service chain of this type: VN1 <> VN1 apply service [S1, S2, S3], we will create the service with one interface each. Each service instance will have one PortClass object (lets say PC1, PC2, PC3). PC1 will be connected to VN1 and PC2. PC2 will be connected to PC1 and PC3. PC3 will only be connected to PC2. Schema transformer will create one routing instance for each of these (say, RI1, RI2, RI3). Each routing instance will have routes with nexthop as the address of the VMI in that instance. We will setup an ACL in VN1 to send all traffic to RI1. For traffic coming out of S1, we will setup a VrfAssignRule to send it to RI2. For traffic coming out of S2,we will setup VrfAssignRule to send it to RI3 and for traffic coming out of SI3, we will set VrfAssignRule to send it to the primary RI so that it could be delivered to the final destination.