The PCE enforces certain soft and hard limits to restrict the total number of system objects that you can create. These limits are set based on the tested performance and capacity limits of the PCE.
Types of Object Limits
This section describes the difference between soft and hard limits.
Soft Limits
Soft limits serve as an early warning for potential PCE scale and performance issues. When you see a soft limit warning, contact Illumio Customer Support to discuss the potential impact of this alert on your deployment.
When the PCE reaches a soft limit, the it logs an organization (audit) event that indicates the soft limit for that object has been reached:
soft_limit_exceededYou should investigate soft limit alerts on a non-emergency basis. When PCE services are functioning normally, but the PCE is generating a lot of soft limit alerts, consult Illumio Customer Support about altering or suppressing the soft limit alerts.
When you lower a soft limit below the current actual usage, the PCE does not generate an event.
Hard Limits
Hard limits protect the PCE from usage and performance overloads, such as creating too many workloads, or too large a security policy. When you receive a hard limit warning, Illumio recommends that you investigate it immediately. When a hard limit is reached in conjunction with a service outage, a PCE core capacity might be overloaded.
When a hard limit is reached, any attempt to create more objects of that type will fail and result in an error message in the PCE web console or a HTTP 406 error returned in REST API. In addition, the PCE logs this event:
hard_limit_exceededWhen you reach a hard limit, contact Illumio Customer Support to discuss your PCE deployment.
Check Object Limits and Usage
To check the status and usage of the current object limits, run the following command:
$ sudo -u ilo-pce <install_root>/illumio-pce-ctl obj-limits listThe CLI commands illumio-pce-db-management events-storage and illumio-pce-env show information about hard and soft limits and related events.
illumio-pce-db-management events-storageCLI commands list when the soft-cap reached, hard-cap reached, and hard-cap exited conditions were last observed.illumio-pce-db-management events-storageCLI commands list the current soft-cap and hard-cap limits.illumio-pce-envcommand displays a warning if a hard cap condition exists, but the command does not fail.
Example:
$ illumio-pce-db-management events-storage Reading /opt/pce_config/etc/runtime_env.yml. INSTALL_ROOT=/var/illumio_pce RENV=development Event limit conditions status Current events soft_limit, hard_limit (in MB): [7132, 8915] Events soft limit last exceeded at: Events hard limit last exceeded at: Last recovered from events hard limit exceeded condition at: Done.
Object Limits During Bulk Create
When you use the Illumio REST API to perform an asynchronous job, such as bulk creation of multiple workloads, and you reach the workload object limit during the job, the job will successfully create as many workloads within the limit, and fail to create more workloads.
The HTTP response shows that some workloads were successfully created, and includes a failure message for each workload that was not created due to the hard limit.
For example:
[
{
"token": "object_limit_hard_limit_reached",
"message": "Object limit hard limit reached"
}
]Object Limits and Concurrent Transactions
When multiple users create the same type of object simultaneously, the PCE can reach the hard object limit for that object concurrently during the parallel transactions. This type of “race” condition is atypical but can occur.
For example, a PCE has 900 rules. Two users each simultaneously add 100 rules in a single transaction. After their two transactions, the rule object count is 1100. When the two transactions occur simultaneously and the PCE reaches a hard limit for that object, both transaction can return an error after the PCE reaches the limit.
PCE Object Limits
The following table lists all PCE object limits, identified by each object name followed by the object's keyname in parentheses. The object keyname is displayed when you run the illumio-pce-ctl obj-limits list command on one of the nodes in your cluster.
|
Object |
Description |
Soft Limit |
Hard Limit |
|---|---|---|---|
|
VENS per PCE ( |
Total number of VENs that have been installed on managed workloads |
SNC: 8,000 2x2 (small): 2,000 2x2: 8,000 4x2: 20,000 |
SNC:10,000 2x2: 10,000 4x2: 25,000 |
|
Labels ( |
Total number of labels |
20,000 |
25,000 |
|
Label Groups ( |
Total number of label groups |
8,000 |
10,000 |
|
Label Group members ( |
Total number of labels in a label group, including nested label groups For example, you have label groups A and B, and each group contains 1000 labels. Label group C contains label groups A and B. The total number of |
8,000 |
10,000 |
|
IP List entries ( |
Total number of all IP list entries in all IP lists in the system |
8K |
10K |
|
Interfaces per Unmanaged Workload ( |
Total number of network interfaces supported per unmanaged workload An unmanaged workload does not have a VEN installed on it. |
102 |
128 |
|
Interfaces per VEN ( |
Total number of interfaces supported per managed workload A managed workload has a VEN installed on it. |
32 |
None (-1) |
|
Items per Rule ( |
Total number of items allowed per rule in the Providers and Consumers fields. A rule contains labels, workloads, and IP lists. When you have a rule that has two Provider items and two Consumer items, the rule has 4 items. |
50 |
200 |
|
Pairing Keys (active) ( |
Total number of active pairing keys A pairing key is active when you create a pairing profile, click Start Pairing, and generate the key. When you click Stop Pairing, the pairing key becomes inactive and is no longer counted in the object limit. |
1200 |
5K |
|
Pairing Profiles ( |
Total number of pairing profiles |
1200 |
5K |
|
RBAC Permissions ( |
Total number of RBAC permissions Each RBAC permission is a three tuple of an RBAC user or user group, role, and scope. |
10K |
35K |
|
Policy Services ( |
Total number of services that you have added to the PCE and provisioned to use in rules |
10K |
None (-1) |
|
Port ranges per Policy Service ( |
Total number of port ranges per service |
50 |
None (-1) |
|
Services per Rule ( |
Total number of services that can be associated with a single rule |
40 |
50 |
|
Ports per Rule ( |
Total number of ports that can be associated with a single rule. Each service has a certain number of ports or port ranges. |
400 |
500 |
|
Rules ( |
Total number of all rules in all rulesets |
40K |
50K |
|
Scopes and Rules ( |
Sum of the total number of rules times the total number of scopes in all rulesets For example, you have two rulesets: RuleSet1 (2 rules, 3 scopes) and RuleSet2 (2 rules, 1 scope). In this example, the total number of scopes and rules is (2 x 3) + (2 x 1) = 8. |
40K |
50K |
|
Total stateless Rules ( |
The total number of stateless rules in your organization |
80 |
100 |
|
RBAC Users and Groups ( |
Total number of all RBAC users and groups |
1600 |
2000 |
|
Adaptive User Segmentation (AUS) users ( |
Total number of Adaptive User Segmentation (AUS) users used in rules |
45K |
50K |
|
Service Bindings ( |
Total number of service bindings created between workloads and virtual services |
90K |
100K |
|
Services per VEN ( |
Total number of services on a managed workload that the VEN reports to the PCE When you add more than 200 services to a managed workload, the PCE ignores any services over the 200 limit. |
160 |
200
|
|
Workloads ( |
Total number of managed and unmanaged workloads A managed workload has a VEN installed on it, and unmanaged workloads do not. |
SNC: 2,000 2x2 (small): 10,000 2x2: 40,000 4x2: 100,000 |
SNC: 2,500 2x2(small): 12,500 2x2: 50,000 4x2: 125,000 |
|
User sessions ( |
Maximum number of user sessions on a single PCE cluster at the same time. When the limit is exceeded, anyone who tries to log in is refused with an explanatory message. |
100 |
125 |