Cisco ACI Logical Constructs (Tenant)

 The Cisco Application-Centric Infrastructure (ACI) is designed to scale from smaller commercial environments, which may use a single tenant, to large cloud providers with many tenants. A single enterprise can leverage tenants to enforce administrative and operational separation between internal businesses.

Overview of Tenant and its basic element

• A tenant is a logical container for application policies that enable an administrative to exercise domain-based access control. A tenant represents a unit of isolation from a policy perspective, such as a customer in a service provider setting, and organization or domain in an enterprise setting, or just a convenient grouping of policies.
• A VRF (Context) is a unique Layer 3 forwarding domain. All endpoints within the Layer 3 domain must have unique IP addresses. The terms network, context, private network, and VRF are synonymous. Just as a router can have multiple VRFs configured, a tenant can have multiple networks that are associated with it.
• A bridge domain is a logical grouping of endpoints that appear to be on the same LAN. A bridge group can contain one or more subnets. One or more bridge domains are associated with a network. Subnets may overlap across tenants.
• A subnet corresponds to an IP subnet and is configured as a default gateway address for that subnet.
• Endpoint groups (EPGs) are collections of similar endpoints representing an application tier or a set of services. They provide a logical grouping for objects that require a similar policy.
• An application profile is the combination of EPGs and the policies that define their interactions. The interaction policies are called contracts.

Tenant

A tenant (fvTenant) is a container for policies that enable an administrator to exercise domain-based access control. The system provides the following four kinds of tenants:

• User tenants are defined by the administrator according to the needs of users. They contain policies that govern the operation of resources such as applications, databases, web servers, network-attached storage, virtual machines, and so on.
• The common tenant is provided by the system but can be configured by the fabric administrator. It is usually used as a shared services tenant. Objects created inside the common tenant are available to all other tenants, such as firewalls, load balancers, Layer 4 to Layer 7 services, intrusion detection appliances, and so on.
• The infrastructure tenant is provided by the system but can be configured by the fabric administrator. It contains policies that govern the operation of infrastructure resources such as the fabric VXLAN overlay. It also enables a fabric provider to selectively deploy resources to one or more user tenants. The infra tenant is used to expand the infrastructure. Multi-Pod and Multi-Site fabrics use the infra tenant to connect the pods or sites to each other.
• The management tenant is provided by the system but can be configured by the fabric administrator. Most of the management configuration is performed in the management tenant. It contains policies that govern the operation of fabric management functions used for in-band and out-of-band configuration of fabric nodes. The management tenant contains a private out-of-bound address space for the APIC/fabric internal communications that is outside the fabric data path that provides access through the management port of the switches. The management tenant enables discovery and automation of communications with virtual machine controllers.

A tenant represents a unit of isolation from a policy perspective, but it does not represent a private network. Tenants can be isolated from one another or can share resources. The primary elements that the tenant contains are filters, contracts, outside networks, bridge domains, Virtual Routing and Forwarding (VRF) instances, and application profiles that contain endpoint groups (EPGs). Entities in the tenant inherit its policies. Tenants are logical containers for application policies. The fabric can contain multiple tenants. You must configure a tenant before you can deploy any Layer 4 to Layer 7 services. The ACI fabric supports IPv4, IPv6, and dual-stack configurations for tenant networking.

VRFs (Context)

• A Virtual Routing and Forwarding (VRF) object (fvCtx) or context are a tenant network. 
• A tenant can have multiple VRFs. A VRF is a unique Layer 3 forwarding and application policy domain. 
• A VRF defines a Layer 3 address domain. One or more bridge domains are associated with a VRF. 
• All of the endpoints within the Layer 3 domain must have unique IP addresses because it is possible to forward packets directly between these devices if the policy allows it.

Application Profiles

• Application Network Profiles (ANP) - are a group of EPGs and the Contracts (policies) that define the communication between them.
• An application profile (fvAp) defines the policies, services, and relationships between endpoint groups (EPGs). Application profiles contain one or more EPGs.

Bridge Domains and Subnets

A bridge domain (fvBD) represents a Layer 2 forwarding construct within the fabric. A BD must be linked to a VRF (also known as a context or private network). Except for a Layer 2 VLAN, it must have at least one subnet (fvSubnet) associated with it.

• The bridge domain is the layer two domain in ACI. Everything in a BD is layer two adjacent. A bridge domain is a member of a VRF, even if there is no IP configuration on the bridge domain.
• The configuration of a BD has a lot of impact on the way the network behaves, and the way endpoints are learned.
• Bridge domains can span multiple switches. A bridge domain can contain multiple subnets, but a subnet is contained within a single bridge domain.
• A bridge domain can have zero or more subnets, but it needs to have at least one subnet if it is to perform routing for the hosts residing in the BD. 

L2 Forwarding Behavior in the Bridge Domain

• ACI fabric is host routed but what if there is no layer 3 header?
• If Destination MAC is known lookup location, route to egress leaf (vtep)
• L2 Unknown Unicast is either flooded within the BD or sent to Spine proxy.
• L2 Unknown Multicast is set to flood (default)
• ARP defaults to L3 unicast lookup but can be set to flood (legacy mode)

Subnets

• Each subnet is a child of one bridge domain
• A BD may have more than one subnet
• Subnets must have unique IP addresses with their context
• Subnets can span multiple EPGs

There are several types of Subnets:

• Private to VRF: This option indicates that this subnet is contained within the Cisco ACI fabric and is not advertised to external routers by the border leaf switch. This option has been removed in the latest releases as it is the opposite of Advertised Externally.
• Advertised Externally: This option indicates that this subnet should be advertised to an external router by a border leaf switch (through an L3Out connection). A subnet that is configured to be advertised externally is also referred to as a public subnet.
• Shared Between VRF Instances: These subnets will be shared between VRFs inside of the fabric. It is used to indicate that this subnet should be leaked to one or more VRFs. The shared subnet attribute is applicable to both public and private subnets.

The subnet can be shared with and exported to multiple VRFs in the same tenant or across tenants as part of a shared service. You can combine the Shared subnet option with either Private or Public subnets. However, Private and Public subnets can’t be combines.

An example of a shared service is a routed connection to an EPG present in another VRF in a different tenant. This enables traffic to pass in both directions across VRFs. An EPG that provides a shared service must have its subnet configured under that EPG (not under a BD), and its scope must be set to advertised externally, and shared between VRFs.

Subnets are most often seen as part of a BD. Which is a logical way to look at them. It is however also possible to configure subnets directly at the EPG level. This is required if you want to advertise the subnet between VRFs. Keep in mind that when configuring a subnet at the BD level you can use the same subnet in all EPGs that are member of that BD. However, if you configure the subnet at the EPG level only that specific EPG can use the subnet, regardless of the number of EPGs that are part of the same BD.

The differences between a subnet under the bridge domain and a subnet under the EPG are as follows:

• Subnet under the bridge domain: If you do not plan any route leaking among VRF instances and tenants, the subnets should be placed only under the bridge domain. If Cisco ACI provides the default gateway function, the IP address of the SVI providing the default gateway function should be entered under the bridge domain.
• Subnet under the EPG: If you plan to make servers on a given EPG accessible from other tenants (such as in the case of shared services), you must configure the provider-side subnet also at the EPG level. This is because a contract will then also place a route for this subnet in the respective VRF instances that consume this EPG. The subnets configured on the EPGs under the same VRF must be nonoverlapping. The subnet defined under the EPG should have the No Default SVI Gateway option selected.

Endpoint Groups (EPGs)

EPG stands for End Point Group. A group of endpoints. The definition of endpoint itself can be broad. It can be anything that communicates on the network.

An EPG is always member of a Bridge Domain. An EPG can’t be member of multiple Bridge Domains, but multiple EPGs can be member of the same BD. Communication between EPGs is blocked by default. To allow communication you need either configure the EPG to be a member of a preferred group, which enables free communication between all EPGs that are member of the same preferred group. You can also configure the VRF to be ‘unenforced’, which causes the whole VRF to allow traffic between EPGs that are member of the VRF. These are valid options, but if you want to maintain control over which traffic is exchanged between EPGs you will use contracts.

Categories of EPGs

• Application endpoint group (fvAEPg)
• Layer 2 external outside network instance endpoint group (l2extInstP)
• Layer 3 external outside network instance endpoint group (l3extInstP)
• Management endpoint groups for out-of-band (mgmtOoB) or in-band ( mgmtInB) access.

Two Types of EPGs:

• Regular (Base EPGs)
• Microsegment/Attribute-based (uSeg EPGs)

An EPG can be statically configured by an administrator in the APIC, or dynamically configured by an automated system such as vCenter or OpenStack.

Applying Policy to End-Points

• Endpoint attaches to fabric
• APIC detects Endpoint, derives source EPG
• APIC pushes required policy to leaf switch

Policy pushed to Leaf nodes based on Resolution Immediacy

• On Demand – Policies only pushed to leaf node upon pNIC attachment and vNIC association with port-group (EPG)
• Immediate – policies pushed to leaf node upon Hypervisor pNIC attachment, LLDP or OpFlex resolves Hypervisor to Leaf node attachment.

Policy programming in Leaf node hardware based on Deploy Immediacy

• On Demand – Policies programmed into Policy CAM only when reachability is learnt through data path.
• Immediate – Policies programmed in Policy CAM once received by APIC as defined by Resolution Immediacy Policy.

Contracts, Filter, Subject

Contracts are groups of Subjects which define communication between source and destination EPGs.
Subjects are a combination of a filter, an action, and optional label.

In addition to EPGs, contracts (vzBrCP) are key objects in the policy model. A contract is the object which is used to enable communication between two EPG’s. Whereas by default EPGs can’t communicate with each other, till the contracts enables certain types of traffic.
A contract needs to be provided by an EPG and consumed by another EPG to allow traffic. Sometimes these terms might be confusing, so I’ll try my best to explain them.

• When an EPG provides a contract that means that the services specified in the contract (using subjects and filters) are provided by the EPG. Say for example you have a Webserver in an EPG. We’ll call this EPG “WEB”. This server provides two services, HTTP and HTTPS. This means that we will define a contract that specifies these services. The “WEB” EPG then provides this contract.
• Another endpoint, located in another EPG needs to load a webpage from the webserver. Before it can do that, it needs to consume the contract which is provided by the “WEB” EPG. As soon as it does so it can communicate with the webserver on the ports specified in the contract.

To allow a contract to work it must contain a subject and filters. These define the type of traffic that is allowed (or disallowed when using Taboo contracts).

In shorts,

• Contracts à Group of Subjects. Define Scope (Global, Tenant, AP)
• Subjects à Group of Filters. Unidirectional/Bi-directional, QoS and Service Graph Insertion Point
• Filters à Lowest Level ACL Entries

Contracts are used for more than ‘just’ allowing traffic between EPGs. They are also used to integrate L4-L7 services into the fabric and to limit the scope of route leaking between Contexts.

Contracts govern the following types of endpoint group communications:

• Between ACI fabric application EPGs (fvAEPg), both intra-tenant and inter-tenant
• Between ACI fabric application EPGs and Layer 2 external outside network instance EPGs (l2extInstP)
• Between ACI fabric application EPGs and Layer 3 external outside network instance EPGs (l3extInstP)
• Between ACI fabric out-of-band (mgmtOoB) or in-band (mgmtInB) management EPGs

A filter is a set of rules that define the traffic on which to match. A filter can contain one or more entries, but it is recommended to keep them as simple as possible. (Keep them focussed on a protocol). For example, DNS. DNS uses port 53 on both TCP and UDP. It would be great to create a filter that contains both TCP and UDP port 53.

Filter entries can be configured to be stateful. This means that the fabric will monitor for the 3-way handshake in TCP sessions to establish correct filtering and session setup.

Subjects are another layer of abstraction available on ACI to build complex contracts. Subjects allow you to apply the filters and to choose whether you apply them in both directions or not, configure DSCP and much more.

Taboo contracts can be used to deny specific traffic that is otherwise allowed by contracts. The traffic to be dropped matches a pattern (such as, any EPG, a specific EPG, or traffic matching a filter). Taboo rules are unidirectional, denying any matching traffic coming toward an EPG that provides the contract.

Tenant network configurations

In a traditional network infrastructure, the configuration steps consist of the following:

1. Define several VLANs at the access and aggregation layers.
2. Configure access ports to assign server ports to VLANs.
3. Define a VRF instance at the aggregation-layer switches.
4. Define an SVI for each VLAN and map these to a VRF instance.
5. Define Hot Standby Router Protocol (HSRP) parameters for each SVI.
6. Create and apply Access Control Lists (ACLs) to control traffic between server VLANs and from server VLANs to the core.

A similar configuration in Cisco ACI requires the following steps:

1. Create a tenant and a VRF instance.
2. Define one or more bridge domains, configured either for traditional flooding or for using the optimized configuration available in Cisco ACI.
3. Create EPGs for each server security zone and map them to ports and VLANs.
4. Configure the default gateway (known as a subnet in Cisco ACI) as part of the bridge domain or the EPG.
5. Create contracts.
6. Configure the relationship between EPGs and contracts.

Summary

• Tenant is the top-level container for endpoint policies.
• Private Network is the equivalent to a Virtual Route Forwarder (VRF)
• Bridge Domain is a Layer2 boundary which can SPAN multiple Leaf switches.
• Application Profiles are groupings of End Point Groups which can be collectively health monitored together.
• End Point Groups are logical groupings of endpoints which will share policy.
• Contracts define the communication policies between EPGs.