Cisco SDWAN Management Plane

In traditional networking, configurations are typically applied on a device-per-device basis using CLI. This leads to a lot of boilerplate code and management inefficiencies. Cisco SD-WAN has been designed to overcome this by implementing a centralized management-plane that administers all devices. 

The solution uses policies to manipulate the overlay fabric in a centralized fashion and templates to eliminate the boilerplate configurations and reuse code.

Cisco SD-WAN Policies

• Policies are an essential part of the Cisco SD-WAN solution and are used to influence the packet flow across the overlay fabric. 
• They are created on vManage through the Policy Wizard GUI and when applied, are pushed via NETCONF transactions either to vSmart controllers (centralized policies) or directly to vEdges (localized policies).
• A Cisco SD-WAN policy is the sum of at least one list, that identifies interesting values, one policy definition, that defines actions, and at least one application, that defines where this policy will be applied.

• It is important to understand that policies are configured on vSmart or vEdge.
• vManage is only a graphical user interface used to create and store policies, but once a policy is activated through the vManage GUI, it is configured with a NETCONF transaction either on vSmart or vEdge. 
• Therefore, activating a policy via vManage is equal to manipulating the configuration of vSmart.
• vSmart does not store policies, it only loads the currently active policy in its running-configuration. 
• All policy versions and revisions are stored on vManage. 
• Therefore, vManage is responsible for rollbacks, version control, and making sure that policy changes are persistent across multiple vSmart controllers.
• While all policies are defined using the vManage Policy Wizard, different types are enforced on different devices at different locations in the network.

 Cisco SD-WAN Policy Types

• Centralized policies allow us to manipulate the whole overlay fabric in a centralized fashion and localized ones give us the ability to manipulate only a particular device or location. 
• Because the control and data plane are separated, centralized policies are also separated into centralized-control-policies that affect the control-plane operations and centralized-data-policies that directly affect the forwarding of packets.

Policy Key Points

A policy is processed in the following order of steps:

• All match–action clauses are processed in sequential order, starting from the lowest sequence number upwards.
• When a match occurs, the configured action is performed, and the sequential processing does NOT continue further (all other match-action pairings are skipped).
• If a match does not occur, the configured entity is subject to the default action configured (by default it is reject).

Centralized policies (the ones configured on vSmart) are always applied to a site-list.

• Only one of each type of policies can be applied to a site-list. For example, you can configure one control-policy in and one control-policy out but not two control policies in the outbound direction.
• Cisco does not recommend including a site in more than one site-list. Doing this may result in unpredictable behavior of the policies applied to these site-lists.
• Centralized-Control-policy is unidirectional applied either inbound or outbound. For example, If we need to manipulate omp routes that the controller sends and receives, we must configure two control policies.
• Centralized-Data-policy is directional and can be applied either to the traffic received from the service side of the vEdge router, traffic received from the tunnel side, or both.
• VPN membership policy is always applied to traffic outbound from the vSmart controller.

vEdge Order of Operations

The steps that a WAN edge router takes when forwarding a packet through:

1. IP Destination Lookup: WAN edge devices are in essence just routers, so the forwarding decision always starts with IP address lookup.
2. Ingress Interface ACL: Localized policies are typically used to create ACLs and tie them to vEdge interfaces. As in traditional networking, these ACLs can be used for filtering, marking, and traffic policing.
3. Application-Aware Routing:  If there is an Application-Aware Routing policy applied, it makes a routing decision based on the defined SLA characteristics such as packet loss, latency, jitter, load, cost, and bandwidth of a link.
4. Centralized Data Policy: The centralized data policy is evaluated after the Application-Aware Routing policy and can override the Application-Aware Routing forwarding decision.
5. Forwarding: At this point, the destination IP address is compared against the routing table, and the output interface is determined.
6. Security Policy: If there are security services attached to the WAN edge node, they are processed in the following sequence - Firewall, IPS (Intrusion Prevention), URL-Filtering, and lastly AMP (Advanced Malware Protection). The necessary tunnel encapsulations are performed, and VPN labels are inserted.
7. Egress Interface ACL: As with ingress ACLs, local policy is able to create ACLs that are applied on egress as well. If traffic is denied or manipulated by the egress ACL, those changes will take effect before the packet is forwarded.
8. Queueing and Scheduling: Egress traffic queueing services such as Low-Latency (LLQ) and Weighted Round Robin (WRR) queueing are performed before the packet leaves.

In Cisco SD-WAN, you can apply configurations to network devices with either one of the following two methods:

• Via the CLI - This is the well-known way of configuring network nodes in traditional networking. You connect to the device via TELNET/SSH or CONSOLE and modify the running configuration. As much as we network engineers love the CLI, it has not been designed to make massive scale configuration changes to multiple devices at the same time.
• Via the vManage GUI - This is the recommended centralized approach of configuring the devices in the Cisco SD-WAN solution. It is significantly less error-prone, can easily scale, and has support for automation, backups, and recovery.

Configuration Templates

• The actual process of configuring Cisco SD-WAN nodes via vManage is done by applying device templates to one or multiple devices. 
• A device template holds the whole operational config of a device. 
• When vManage provisions the configuration of a node, it acts as a single source of truth and "locks" the device in a configuration mode called "vManage mode". 
• That means that configuration changes can only be applied via vManage and changes via CLI are not allowed.  

A device template can be either Feature-based or CLI-based as is shown in figure. Something very important about templates is that when we create a CLI-based template for a specific device, the whole configuration of the device must be in the CLI template and not only a specific snippet of the configuration. The opposite is true about feature templates.

Creating a feature-based template is comparable to assembling a template of lego blocks where each block is a different technology feature. For example, OSPF is one lego block, BGP is another lego block, AAA is another, and so on. Let's highlight the main benefit of configuring devices using feature-based templates:

• Feature templates can be reused across multiple devices. This brings greater flexibility and scale.
• It is more granular than CLI-based templates. You can modify only a specific device feature such as AAA or BGP.
• You don't need to know the device-specific syntax of different platforms. You just apply the template and vManage handles the actual configuration behind the scenes.

Configuration Variables

Network engineers know very well that network devices have many device-specific parameters that are unique per device. For example, each one has a unique name, IP addresses, interface names, router-id, and so on. To account for that, Cisco SD-WAN gives us the ability to specify three different types of values when creating feature templates:

• Global - When we specify a value to be Global that means that it will be applied to all devices to which the feature template is attached. For example, this will most probably be the case for the SNMP communities, Syslog servers, or the company's banner message. At a later stage, when we want to change the Banner of all nodes, we would just update the feature template value and it will update every device template that is using this feature template. 
• Device-specific - When we know that a particular parameter will be unique for every different device, we specify a device-specific value. When we do that, the wizard will ask for a variable name. In the example in figure 2, this is [inet_if_name]. Upon applying a device template to a given device, the vManage Wizzard will ask us to provide the actual unique value for this variable.
• Default - The default value simply represents the factory default settings. It cannot be changed, that is why the textbox is always greyed-out and inactive. When we want to overwrite the default value, we change the value type to either Global or Device-specific.

Device Templates

It is important to understand that a device template defines a given device's complete operational configuration. The structure of a device template is shown in figure. It is made of several feature templates depending on the specific device, role, and so on.

• As you know, on the traditional Cisco networking devices, some essential features are mandatory and turned on by default (for example spanning-tree, vtp, etc). 
• In the same way in Cisco SD-WAN, when creating a device template, some features are mandatory, indicated with an asterisk (*). 
• That is why there are factory-default templates named Factory_Default_{Feature-Name}_Template that is applied by default in case you do not overwrite them with a more specific configuration.
• Upon attaching a configuration template to a cisco SD-WAN node, vManage requires all device-specific values to be filled in. This can be done through the vManage GUI directly, or by or by using a CSV file.
• Once a device template is applied to vEdge or vSmart device, the device is put in "vManaged mode" and its configuration can no longer be modified via CLI.
• In cases where we attach a device template to a WAN edge router and it for whatever reason loses control plane connectivity to the vManage controller, the vEdge will immediately start a 5-min rollback timer. If the control-plane connectivity does not come up within that 5 minute, the vEdge will revert back its configuration to the last-known working setup and will eventually reconnect to vManage.

A Cisco SD-WAN device can be either in one of these configuration modes at any given time:

• CLI mode – a template is not attached to the device by vManage and the device's configuration can be modified locally using the cli, for example via console or SSH. This is the default mode for all Cisco SD-WAN devices.
• vManage mode – a template is attached to the device by vManage and the device's configuration cannot be modified locally using the cli.

vManage Mode

• By default, all Cisco SD-WAN controllers are in "CLI mode". 
• That means that they allow configuration changes done using the CLI only. 
• However, as we have explained in our lesson for Cisco SD-WAN Policies when we activate a centralized policy through the vManage GUI, what happens behind the scenes is that the vManage is making configuration changes on the vSmart controller using NETCONF. 
• But by default, like all other devices, the vSmart controller is in CLI mode (allowing config changes via cli only) and thus it does not accept NETCONF transactions from vManage. 
• That is why the policy activation fails.
• To successfully activate a policy, we should change the configuration mode of the device, that the policy will be applied to, to be in “vManaged" mode. 
• This is done by applying a template from vManage to that device. 
• This tells the affected node that from now on it will not be configured manually via CLI but in a centralized fashion using templates and policies from vManage.
• You can check whether a device is in vManaged mode or not with the following command:

vSmart# show system status
Viptela (tm) vsmart Operating System Software
Copyright (c) 2013-2017 by Viptela, Inc.
Controller Compatibility:
Version: 18.4.4
Build: 82
## lines omitted
Personality:             vsmart
Model name:              vsmart
Services:                None
vManaged: false
Commit pending:          false
Configuration template: None
Policy template:         None
Policy template version: None

Or alternatively on vManage under Configuration > Devices > Controllers

Applying a template to vSmart

Applying a configuration template to vSmart allows vManage to have authoritative control of vSmart’s configuration. 
Any type of template does the job - it does not matter whether it is a CLI or Feature template. 
In typical production deployments, it is very common to use CLI templates for this use case, as they are very simple and quickly made, and do not require administration beyond the initial deployment.
Practically speaking, the easiest way to change a controller to be in vManaged mode is to create a CLI template and attach it to vSmart. 

• Let's create a CLI template by going to Configuration > Templates > Create Template > CLI.
• Then we select the device model (in our case vSmart) from the dropdown menu and specify the name and description for the template. At this point, we SSH to the controller, get the output of the show run command, and paste it in the CLI configuration section.

• Then we go to the additional options and select Attach Devices. In the next window, you are going to see all vSmart controllers that are known to vManage. You select the one, which you got the show run output from.

• In the next window, you will be prompted to validate the configuration that is going to be applied. Once you confirm it, vManage will push the configuration template to vSmart.
• At this point, the device is fully managed in a centralized fashion by vManage.