Cisco SDWAN Architecture Components and its Terminology

The Cisco SD-WAN solution is comprised of separate orchestration, management, control, and data planes.

• The orchestration plane assists in the automatic onboarding of the SD-WAN routers into the SD-WAN overlay.
• The management plane is responsible for central configuration and monitoring.
• The control plane builds and maintains the network topology and makes decisions on where traffic flows.
• The data plane is responsible for forwarding packets based on decisions from the control plane.

Cisco vBond

Cisco vBond is the Orchestration Plane of the SD-WAN system. Its job is to orchestrate the process of onboarding new unconfigured devices to the SD-WAN fabric. It is responsible for the authentication and whitelisting of vEdge routers and control/management information distribution.

• In other words, it tells our vEdges where and how to connect to our organizations vManage and vSmart controllers, while also advising our vSmart controllers as new vEdges join the SD-WAN fabric.
• It also serves the role of informing our vEdges if they are behind a NAT device which facilitates IPsec NAT traversal and allows Authentication Header security to be applied to our data plane tunnels (more on that in upcoming posts).
• vBond is the first point of contact and thus our first point of authentication for all SD-WAN components as they boot up and join the SD-WAN fabric.
• On-Premises deployments can be hosted on either ESXi or KVM hypervisors.
• The service can also be run as an agent service on one of your vEdge hardware appliances (although this is strongly discouraged). Each vBond requires a dedicated public IP address.
• Requires public IP Address (could sit behind 1:1 NAT)
• Highly resilient

Cisco vManage
Cisco vMange is the Management Plane of the SD-WAN system. It runs the user interface of the system and is the dashboard network administrators interact with daily. It is responsible for collecting network telemetry data, run analytics, and alert on events in the SD-WAN fabric. It is also the tool that admins use to create device templates, push configurations, and perform overlay traffic engineering.

Cisco vManage can be deployed on-prem, in the public cloud, or in the Cisco cloud-hosted environment. It is significantly resource-intensive, and most customers go with the cloud options.

• It provides a single pane of glass for Day 0, Day 1, and Day 2 operations.
• It is also the location where you will build your device configurations (Device Templates) and overlay traffic engineering policies.
• If you are familiar with the Meraki dashboard you can very much think of vManage in the same light.
• This is also the programmatic interface into the system supporting REST API.
• On-Premises deployments can be hosted on either ESXi or KVM hypervisors, with even the smallest footprint requiring a minimum of 16 vCPUs, 32GB of dedicated RAM and 500GB of storage. Now you can see why the cloud hosted option is so appealing.
• A single vManage instance can support up to 2,000 devices and can be deployed as part of a cluster containing 6 instances.
• Multitenant with web scale
• Its help in Software upgrades
• Highly resilient

vSmart Controller

Cisco vSmart is the Control Plane of the SD-WAN system. vSmart controllers are the brain of the overlay fabric. They advertise routing, policies, and security. They are positioned as hub routers in the control plane topology and all vEdge routers peer with all vSmart controllers. For experienced network engineers, vSmart controllers are like BGP Route-reflectors or DMVPN NHRP routers. However, it is important to understand these appliances are not part of the Data Plane and do not participate in packet forwarding.

• It also orchestrates the secure data plane connectivity between the WAN Edge routers by reflecting crypto key information originating from WAN Edge routers, allowing for a very scalable, IKE-less architecture.
• It facilitates fabric discovery
• Dissimilates control plane information between vEdges.
• Distributes data plane and app-aware routing policies to the vEdge routers
• Implements control plane policies, such as service chaining, multi-topology, and multi-hop
• Dramatically reduces control plane complexity
• Highly resilient

Cisco vEdge

Cisco vEdge devices represent the Data Plane of the SD-WAN system. They sit at the WAN edge and establish the network fabric and join the SD-WAN overlay. If you look at the architecture shown in figure, everything southbound of the vEdge routers is typically traditional networking - offices, data centers, and branches. Everything northbound of the vEdge routers is the SD-WAN system itself. vEdge routers exchange routing information with the vSmart controllers over the Overlay Management Protocol (OMP).

The WAN Edge routers could be Viptela platforms or Cisco IOS-XE devices. They can be virtual or physical appliances. vEdges are auto configured by the system. Back in the Viptela days, this process was called Zero-Touch Provisioning (ZTP) and nowadays with the Cisco devices, it is called Cisco Plug-and-Play (PnP). Both terms mean the same and are interchangeable.  

• vEdges are responsible for the data plane of the SD-WAN fabric as they bring up IPsec or GRE tunnels between your sites.
• As mentioned above vEdges form control plane connections with vSmart controllers, and not between each other.
• vEdge hardware comes in many form factors. You have the 100, 1000, 2000 and 5000 models.
• The main difference being greater interface choices, higher supported throughput and data plane tunnels as the model number increases.
• With the Cisco integration you can now utilize Cisco’s ASR1K, ISR4K and ISR1K router platforms along with the ENCS to perform this SD-WAN role.
• Implements data plane and application aware routing policies
• Exports performance statistics
• Leverages traditional routing protocols like OSPF, BGP, and VRRP
• Physical or Virtual form factor (100Mb, 1 Gb, 10 Gb)

The WAN Edge routers form a permanent Datagram Transport Layer Security (DTLS) or Transport Layer Security (TLS) control connection to the vSmart controllers and connect to both vSmart controllers over each transport. 
The routers also form a permanent DTLS or TLS control connection to the vManage server, but over just one of the transports. 
The WAN Edge routers securely communicate to other WAN Edge routers using IPsec tunnels over each transport. 
The Bidirectional Forwarding Detection (BFD) protocol is enabled by default and runs over each of these tunnels, detecting loss, latency, jitter, and path failures.

Terminology

Site ID

• A site ID is a unique identifier of a site in the SD-WAN overlay network with a numeric value 1 through 4294967295 (2^32-1) and it identifies the source location of an advertised prefix. 
• This ID must be configured on every WAN Edge device, including the controllers, and must be the same for all WAN Edge devices that reside at the same site. 
• A site could be a data center, a branch office, a campus, or something similar. 
• By default, IPsec tunnels are not formed between WAN Edge routers within the same site which share the same site-id.

System IP

• A System IP is a persistent, system-level IPv4 address that uniquely identifies the device independently of any interface addresses. 
• It acts much like a router ID, so it doesn't need to be advertised or known by the underlay. 
• It is assigned to the system interface that resides in VPN 0 and is never advertised. 
• A best practice, however, is to assign this system IP address to a loopback interface and advertise it in any service VPN. 
• It can then be used as a source IP address for SNMP and logging, making it easier to correlate network events with vManage information.

Organization Name

• Organization Name is a name that is assigned to the SD-WAN overlay. 
• It is case-sensitive and must match the organization name configured on all the SD-WAN devices in the overlay. 
• It is used to define the Organization Unit (OU) field to match in the Certificate Authentication process when an SD-WAN device is brought into the overlay network.

Private IP Address

• On WAN Edge routers, the private IP address is the IP address assigned to the interface of the SD-WAN device. 
• This is the pre-NAT address, and despite the name, can be a public address (publicly routable) or a private address (RFC 1918).

Public IP Address

• The Post-NAT address detected by the vBond orchestrator. 
• This address can be either a public address (Publicly routable) or a private address (RFC 1918). 
• In the absence of NAT, the private and public IP address of the SD-WAN device are the same.

TLOC

• A TLOC, or Transport Location, is the attachment point where a WAN Edge router connects to the WAN transport network. 
• A TLOC is uniquely identified and represented by a three-tuple, consisting of system IP address, link color, and encapsulation (Generic Routing Encapsulation [GRE] or IPsec).

Color

• The color attribute applies to WAN Edge routers or vManage and vSmart controllers and helps to identify an individual TLOC; different TLOCs are assigned different color labels. 
• The SD-WAN topology uses a public color called biz-internet for the Internet transport TLOC and a private color called mpls for the other transport TLOC. 
• You cannot use the same color twice on a single WAN Edge router.

Overlay Management Protocol (OMP)

• The OMP routing protocol, which has a structure like BGP, manages the SD-WAN overlay network. 
• The protocol runs between vSmart controllers and between vSmart controllers and WAN Edge routers where control plane information, such as route prefixes, next-hop routes, crypto keys, and policy information, is exchanged over a secure DTLS or TLS connection. 
• The vSmart controller acts like a BGP route reflector; it receives routes from WAN Edge routers, processes and applies any policy to them, and then advertises the routes to other WAN Edge routers in the overlay network.

Virtual private networks (VPNs)

• In the SD-WAN overlay, virtual private networks (VPNs) provide segmentation, much like Virtual Routing and Forwarding instances (VRFs) that many are already familiar with. 
• Each VPN is isolated from one another, and each have their own forwarding table. 
• An interface or subinterface is explicitly configured under a single VPN and cannot be part of more than one VPN. 
• Labels are used in OMP route attributes and in the packet encapsulation, which identifies the VPN a packet belongs to.
• The VPN number is a four-byte integer with a value from 0 to 65535, but several VPNs are reserved for internal use, so the maximum VPN that can or should be configured is 65527. 
• There are two main VPNs present by default in the WAN Edge devices and controllers, VPN 0 and VPN 512. 
• Note that VPN 0 and 512 are the only VPNs that can be configured on vManage and vSmart controllers. For the vBond orchestrator, although more VPNs can be configured, only VPN 0 and 512 are functional and the only ones that should be used.
• VPN 0 is the transport VPN. 

It contains the interfaces that connect to the WAN transports. 

Secure DTLS/TLS connections to the controllers are initiated from this VPN. 

Static or default routes or a dynamic routing protocol needs to be configured inside this VPN to get appropriate next-hop information so the control plane can be established, and IPsec tunnel traffic can reach remote sites.

• VPN 512 is the management VPN. 

It carries the out-of-band management traffic to and from the Cisco SD-WAN devices. 

This VPN is ignored by OMP and not carried across the overlay network.

• In addition to the default VPNs that are already defined, one or more service-side VPNs need to be created that contain interfaces that connect to the local-site network and carry user data traffic. 
• It is recommended to select service VPNs in the range of 1-511, but higher values can be chosen if they do not overlap with default and reserved VPNs. 
• Service VPNs can be enabled for features such as OSPF or BGP, Virtual Router Redundancy Protocol (VRRP), QoS, traffic shaping, or policing. 
• User traffic can be directed over the IPsec tunnels to other sites by redistributing OMP routes received from the vSmart controllers at the site into the service-side VPN routing protocol. 
• In turn, routes from the local site can be advertised to other sites by advertising the service VPN routes into the OMP routing protocol, which is sent to the vSmart controllers and redistributed to the other WAN Edge routers in the network.

The following figure demonstrates VPNs on a WAN Edge router. The interfaces, Int0 and Int2, are part of the transport VPN; Int1 and Int3 are part of the service VPN, which is attached to the local network at the site; and the mgmt0 port is part of VPN 512.
Note that any interface could also be a subinterface. In that case, the main (or parent) physical interface that the subinterface belongs to must be configured in VPN 0. The subinterface MTU also must be 4 bytes lower than the physical interface due to the 802.1Q tag. To fulfil this requirement, set the main interface MTU to 1504 and leave the subinterface MTU at default (1500).

Note: The above illustrates how VPNs are represented directly on the vEdge router and through the vManage configuration. When configurations get pushed from vManage to the IOS XE SD-WAN routers, they are automatically converted into a format accepted by the IOS XE SD-WAN software parser. Some differences include:

• VRF terminology is used instead of the VPN keyword
• The global table is used to represent VPN 0
• VRF Mgmt-intf is enabled by default on the management interface and is used to represent VPN 512