VEN Upgrade Using VEN Library in PCE

You can use your PCE cluster as a centralized mechanism for upgrading VENs in your environment.

From the 20.2.0 release on, you can upgrade one or more VENs by using the PCE web console. From the PCE web console menu, go to the Workloads and VENs > VENs > Upgrade.

You can also use the Illumio Core REST API to upgrade (but not install) VENs. See the REST API Developer Guide for information.

NOTE:

Before you use this feature, you must set up the VEN Library in the PCE. See VEN Library Setup in the PCE for information.

If you are an Illumio Cloud customer, Illumio Operations set up the VEN Library in the PCE.

IMPORTANT:

For Illumio Core Cloud customers, the VEN Library only provides the option to upgrade to VENs that Illumio designated as a Long Term Support (LTS) release. See Versions and Compatibility in the Illumio Support Portal (login required) for information.

About VEN Upgrade

You can upgrade all VENs, upgrade a selected subset of VENs, or upgrade all VENs that match a set of filters. After you confirm an upgrade from the PCE web console, the VEN will download the new VEN image from the PCE and upgrade itself. The upgrade on the workload host only takes on average a few minutes.

If the VEN does not successfully upgrade within a certain amount of time (approximately 1 hour), the upgrade will time out and the PCE will put the VEN in a warning state. However, in most cases, the upgrade will complete within this window. To clear this warning, just start another upgrade on the VEN.

You can upgrade up to 25,000 VENs in a single PCE region. Selecting this large a number of VENs in one upgrade will result in some CPU and memory spikes and increased network bandwidth because the VENs will be communicating with the PCE to request new firewalls and downloading new software versions.

The VEN upgrade feature includes upgrade validation; namely, the PCE will validate that the VEN upgrade path is allowed and that the version of the VEN you are upgrading to is compatible with the version of the PCE. If you attempt to upgrade VENs to a version incompatible with the PCE or Illumio does not support that upgrade path, the PCE web console provides feedback on which VENs can be upgraded.

VEN Package Format Changes

Starting with the 21.2.1 release, the Windows VEN installer switched from MSI to EXE package format. This package format change primarily affects Illumio Core On-Premises customers running older MSI-based Windows VENs. For information about using the VEN Library in the PCE to install Windows VENs on workloads, see Pair a Windows Workload.

Prerequisites and Limitations for VEN Upgrade

Prerequisites

The PCE must be a version 20.2.0 or later.
The VENs selected for upgrade must be version 18.2.0 or later.
The VEN Library must be set up in the PCE. See Upload VEN Software Bundle into PCE VEN Library for information.
The VEN upgrade compatibility matrix must be installed on the PCE. See Upload VEN Upgrade Compatibility Matrix for information.

Limitations

You must update the VENs in your environment using a supported upgrade path.
You cannot upgrade a VEN to a later version than the PCE's current version.
The VEN Upgrade feature does not support upgrading AIX or Solaris VENs or upgrading the C-VEN.
If a VEN has been installed with custom RPM installation options, you cannot use the VEN upgrade feature to upgrade it.

For example, you cannot upgrade VENs installed with a custom --prefix option because options like that aren't persisted when a VEN is upgraded from the PCE, and the VEN will be installed in the default directory. This outcome might cause data loss or operational issues with the VEN, depending on your environment.
You cannot use the feature to downgrade a VEN to an earlier version.
Using the PCE CLI to upgrade VENs is no longer supported in Illumio Core 21.2.0 and later releases.
If you installed the VEN with the VEN CTL and packaging CLI and customized installation options (such as, a custom installation directory or alternate VEN user), you cannot later upgrade the VEN by using the VEN Library in the PCE. You must upgrade the VEN using the workload's OS package upgrade process.

VEN Package Format Changes (Windows only)

Starting with the Illumio Core 21.2.1 release, the Windows VEN installer switched from MSI to EXE package format. This package format change affects Illumio Core On-Premises customers running older MSI-based Windows VENs.

Customers using the VEN Library in the PCE to install or upgrade VENs must take an extra step for the transition. Specifically, Illumio Core customers running older MSI-based Windows VENs must upgrade to Illumio Core 19.3.6+H1-VEN or 21.2.0+H2-VEN before upgrading to their VENs to 21.2.1 or a later version. The 21.2.0+H2-VEN release contained the necessary VEN changes to handle the transition in the VEN packaging from MSI to EXE.

For the detailed steps required for this transition, see New Windows VEN Installer Starting with 21.2.1, Knowledge Base Article 3561 (login required).

Upgrade All VENs to the Current Version

When you select “Upgrade All,” the PCE upgrades all your deployed VENs and not just the VENs that appear in that page of the list (for example, you have 10 pages in the list, the VENs in all 10 pages are upgraded.)

From the PCE web console menu, choose Workloads and VENs > VENs. The Workloads and VENs – VENs page appears.
From the Upgrade drop-down menu, choose Upgrade All.

The Upgrade VENs dialog box appears. By default, the most current VEN version in the VEN Library is selected.
(Optional) To upgrade to a version other than the current version, select the VEN version from the drop-down menu.

The PCE calculates the scope of the upgrade.

When the calculation is complete, the dialog box refreshes and informs you which VENs in your environment can be upgraded to the selected version. In this example, one VEN has an unsupported OS platform because it’s a VEN running on Solaris, which cannot be upgraded using this feature in this release.
Click Confirm Upgrade.

Upgrade Selective VENs

When upgrading, you can use all the filters available in the VEN list to manage the upgrade. For example, you could filter VENs by the location label to only upgrade VENs in a specific datacenter or you can filter the VENs in your environment by their operating systems and upgrade just those VENs.

You can filter and upgrade VENs based on specific conditions. For example, you can locate the VENs that weren't upgraded in your first attempt to upgrade all VENs to the default version; for example, these VENs might have been offline when you ran your first upgrade and the upgrade timed out for those VENs but succeeded on all the others in your environment.

You can filter and upgrade VENs based on their Health status in the PCE.

VENs can have one of three Health statuses that you can filter for:

Healthy: The VEN has no Health conditions.
Warning: The VEN has one or more Warning conditions.
Error: The VEN has one or more Error conditions or has both Error and Warning conditions.

View VEN Upgrade Events

Upgrading VENs using the PCE web console or the REST API generate events that you can view in the Events page.

From the PCE web console menu, choose Troubleshooting > Events.

You can find events indicating when upgrades succeeded and when they failed. If you are using the Upgrade All option to upgrade large numbers of VENs in your environment, the PCE aggregates the event for agent.upgrade_requested; however, the PCE still generates a separate event for each successful upgrade.