PCE Capacity Planning

Use these guidelines and requirements to estimate host system capacity based on typical usage patterns.

The exact requirements vary based on a large number of factors, including, but not limited to:

  • Whether you are using physical or virtual hardware
  • Number of managed workloads
  • Number of unmanaged workloads and other labeled objects, such as virtual services
  • Policy complexity, which includes the following factors:
    • Number of rules in your rulesets
    • Number of labels, IP lists, and other objects in your rules
    • Number of IP ranges in your IP lists
    • Number of workloads affected by your rules
  • Frequency at which your policies change
  • Frequency at which workloads are added or deleted, or workload context changes, such as, change of IP address

  • Volume of traffic flows per second reported to the PCE from all VENs

    See the “Maximum Flow Capacity” table for information about maximum flow capacity of the PCE.

  • Total number of unique flows reported to the PCE from all VENs

CPU, Memory, and Storage

The capacity planning tables in this section list the minimum recommended sizes for CPU, memory, and storage. This section provides two tables, one for physical hardware and one for virtual machines. Use these tables to plan your deployment.

NOTE:

Based on your actual usage and other factors, your capacity needs might be greater than the recommended sizes. For example, if you have installed additional software along with the PCE, such as application performance management (APM) software or an endpoint protection agent, this consumes additional system resources.

Data nodes are configured with a dedicated storage device for each database on the data nodes. This configuration accommodates growth in traffic data, which is used by Explorer. See Runtime Parameters for Traffic Datastore on Data Nodes.

For more than 150 IOPS, locally attached, spinning hard disk drives (HDD) are not sufficient. You will require either mixed-use Solid-State Disk (SSD) or Storage Area Network (SAN).

The PCE does not require that you set up swap memory, but it is permissible to enable swap memory. As long as the PCE nodes are provisioned with the recommended memory (RAM) as shown in the tables below, the use of swap memory should not cause any issues.

Physical Hardware

Use this table if you are installing the PCE on physical hardware. If you are using virtual machines, see the table Virtual Hardware.

MNC Type + Workloads/VENs

Cores/Clock Speed

RAM per Node

Storage Device Size and IOPS

Core Nodes

Data Nodes

SNC

  • 250 VENs1
  • 2500 workloads
  • 3 cores2
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or equivalent

16GB

A single node including both core and data:

  • 1 x 50GB4
  • 100 IOPS per device5

N/A

2x2 Small

  • 2,500 VENs1
  • 12,500 workloads

Cluster type: 4node_v0_small
  • 4 cores per node2
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or equivalent

32GB

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5

Minimum:

  • Disk 1: 250GB4
  • Disk 2: 250GB4
  • 600 IOPS per device5

2x2

  • 10,000 VENs1
  • 50,000 workloads

Cluster type: 4node_v0 or 4node_dx
  • 16 cores per node2, 6
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or equivalent
  • Recommended: 128GB6
  • Minimum: 64GB

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5

Minimum:

  • Disk 1: 1TB4
  • Disk 2: 1TB4
  • 1,800 IOPS per device5

4x2

  • 25,000 VENs1
  • 125,000 workloads

Cluster type: 6node_v0 or 6node_dx
  • 16 cores per node2, 6
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or equivalent.

128GB6

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5
  • Disk 1: 1TB4
  • Disk 2: 1TB4
  • 5,000 IOPS per device5

Footnotes:

1 Number of VENs/workloads is the sum of both the number of managed VENs and the number of unmanaged workloads.

2 CPUs:

  • The recommended number of cores is based only on physical cores from allocated CPUs, irrespective of hyper-threading.

3 This is the absolute minimum needed. In the future, other applications, support reports, or new features may require additional disk.

4 Additional disk notes:

  • Storage requirements for network traffic data can increase rapidly as the amount of network traffic increases.
  • Network File Systems (NFS) is not supported for Illumio directories specified in runtime; for example, data_dir, persistent_data_dir, ephemeral_data_dir.

5 Input/output operations per second (IOPS) are based on 8K random write operations. IOPS specified for an average of 300 flow summaries (80% unique src_ip, dest_ip, dest_port, proto) per workload every 10 minutes. Different traffic profiles might require higher IOPS.

6 In the case of fresh installs or upgrades of a 2x2 for 10,000 VENs or a 4x2 for 25,000 VENs, if you deploy a system without sufficient cores, memory, or both, then the PCE will automatically reduce the object limits to 2,500 workloads. Object limit is the number of VENs (agents) per PCE. Adding more than 2,500 workloads will fail and an event is logged indicating that object limits have been exceeded. The workaround is to increase the number of cores, memory, or both to the recommended specifications and then increase the object limits manually. See PCE Default Object Limits in the PCE Administration Guide.

Virtual Hardware

Use this table if you are installing the PCE on virtual machines. If you are using physical hardware, see the table Physical Hardware.

MNC Type + Workloads/VENs

Virtual Cores/Clock Speed

RAM per Node

Storage Device Size and IOPS

Core Nodes

Data Nodes

SNC

  • 250 VENs1
  • 2500 workloads
  • 6 virtual cores (vCPU)2
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or higher

16GB7

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5

N/A

2x2 Small

  • 2,500 VENs1
  • 12,500 workloads

Cluster type: 4node_v0_small
  • 8 virtual cores (vCPU) per node2
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or higher

32GB7

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5

Minimum:

  • Disk 1: 250GB
  • Disk 2: 250GB
  • 600 IOPS per device

2x2

  • 10,000 VENs1
  • 50,000 workloads

Cluster type: 4node_v0 or 4node_dx
  • 32 virtual cores (vCPU) per node2, 6
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or higher
  • Recommended: 128GB6, 7
  • Minimum: 64GB

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5

Minimum:

  • Disk 1: 1TB4
  • Disk 2: 1TB4
  • 1,800 IOPS per device5

4x2

  • 25,000 VENs1
  • 125,000 workloads

Cluster type: 6node_v0 or 6node_dx
  • 32 virtual cores (vCPU) per node2, 6
  • Intel® Xeon(R) CPU E5-2695 v4 at 2.10GHz or higher

128GB6, 7

Minimum:

  • Disk: 50GB3, 4
  • 150 IOPS per device5
  • Disk 1: 1TB4
  • Disk 2: 1TB4
  • 5,000 IOPS per device5

Footnotes:

1 Number of VENs/workloads is the sum of both the number of managed VENs and the number of unmanaged workloads.

2 Full reservations for vCPU. No overcommit.

3 This is the absolute minimum needed. In the future, other applications, support reports, or new features may require additional disk.

4 Additional disk notes:

  • Storage requirements for network traffic data can increase rapidly as the amount of network traffic increases.
  • Network File Systems (NFS) is not supported for Illumio directories specified in runtime; for example, data_dir, persistent_data_dir, ephemeral_data_dir.

5 Input/output operations per second (IOPS) are based on 8K random write operations. IOPS specified for an average of 300 flow summaries (80% unique src_ip, dest_ip, dest_port, proto) per workload every 10 minutes. Different traffic profiles might require higher IOPS.

6 In the case of fresh installs or upgrades of a 2x2 for 10,000 VENs or a 4x2 for 25,000 VENs, if you deploy a system without sufficient cores, memory, or both, then the PCE will automatically reduce the object limits to 2,500 workloads. Object limit is the number of VENs (agents) per PCE. Adding more than 2,500 workloads will fail and an event is logged indicating that object limits have been exceeded. The workaround is to increase the number of cores, memory, or both to the recommended specifications and then increase the object limits manually. See PCE Default Object Limits in the PCE Administration Guide.

7 Full reservations for vRAM. No overcommit.

Maximum Flow Capacity

The following table shows the maximum capacity of the PCE to accept flow data from all VENs.

MNC Type + Workloads/VENs

Flow Rate

(flow-summaries/second)

Equivalent Flow Rate

(flows/second)2

SNC

  • 250 VENs
  • 2500 workloads

100

1,030

2x2

  • 2,500 VENs
  • 12,500 workloads

1,000

10,300

2x2

  • 10,000 VENs
  • 50,000 workloads

4,100

422,000

4x2

  • 25,000 VENs
  • 125,000 workloads

10,4001

1,070,000

Footnotes:

1 The PCE might need to be tuned to achieve this rate. If you need to tune the PCE, please contact Illumio Support for assistance.

2 Real-world observation shows that 102 flows result in one flow summary on average.