CPU Limits

Aptible allocates 1 / 4 th of a CPU thread per GB of memory to Containers. In other words:

  • 512 MB containers are allocated 1 / 8th of a CPU thread.
  • 1 GB containers are allocated 1 / 4th of a CPU thread.
  • 2 GB containers are allocated half of a CPU thread.
  • 4 GB containers are allocated a full CPU thread.
  • Etc.

This ratio is not configurable.

Enforcement

CPU Limits are in place to enforce the aforementioned CPU allocation on Shared Stacks, but by default, that's not the case on Dedicated Stacks.

This has a few important implications for Containers deployed on Dedicated Stacks:

  • Your Containers will be allowed to use more CPU than their share
  • Your Containers may negatively impact other Containers you own if they use too much CPU.

In other words, in the absence of CPU limits, your Containers are granted extra CPU you didn't pay for, but their CPU share is not guaranteed. Conversely, if CPU limits are enabled, then your Containers are granted only what you paid for, but that CPU share is guaranteed.

The historical default for Dedicated Stacks has been to not enforce CPU Limits. This default changed for new Stacks in 2021; now all new Stacks enforce CPU Limits by default. You can check in the Dashboard whether your Stack shows CPU Limits "Enabled" or "Disabled". If you want to enable enforcement, this can be done via a simple request to Aptible Support.

The Problem With Disabling CPU Limit Enforcement

Containers are sized to an allocation of both memory and CPU based on the 4:1 ratio described above. A Stack's host machines have a fixed size and Aptible distributes containers for all of the Stack's resources across these hosts. For example, an 8GB App host might be running one 4GB container and two 1GB containers, and have space available but not yet used. Deploy only guarantees that a container will be placed on a host machine which has at least enough CPU and memory to fit the container's CPU and memory limits. The host may happen to have more, or it may be an exact fit.

Memory limits are always enforced, but CPU Limits are not enforced on all Dedicated Stacks as described above. If a container exceeds its memory allocation, it is restarted to ensure the host machine and other containers stay healthy. CPU Limits are enforced on all Shared Stacks which throttles containers to their CPU Limit as determined by the container's size. This ensures that malfunctioning or undersized containers do not affect other users' containers which happen to be located on the same host machine. However, the historical default for Dedicated Stacks has been to not enforce CPU Limits. Without enforcement, containers are not throttled, and can use up to the CPU Limit of the underlying host machine. Commonly this is equivalent to a 7GB or 15GB container, which gives 175% or 375% CPU available.

The benefit of not enforcing CPU Limits is surge capacity - available CPU on the host machine won't go unused if the container is taxed beyond its CPU Limit. Containers which require more CPU for only a brief period can be scaled small but still function well so long as there happens to be CPU available on the host machine when needed. The Aptible architecture spreads out containers among several availability zones, so often these CPU-hungry containers will not share a host machine with another CPU-hungry container of the same Service. This is especially true early on in App development, when Apps have not been scaled to many containers.

The downside of not enforcing CPU Limits is complexity, and visibility. When CPU limits are not enforced, a container can be starved of CPU if another container on the same host machine happens to exceed its limit to the point that the host machine is using all available CPU. In this case, an App can experience errors or timeouts (and set off your alerting) when the problem is with another App entirely. The triggering event might be a sudden load on another Service, a CPU-intensive cron job, or even a bad release of another App on the Stack. It can be difficult to narrow down the true cause of the problem, when containers are not guaranteed their allocated CPU.

Even worse, the triggering event might not be visible to you at all. The Aptible platform may restart containers on different hosts at any time, in response to a host machine failure or scheduled maintenance. Ephemeral SSH Sessions without CPU Limits enabled may cause CPU starvation of Stack resources which are colocated on the bastion host instance, including the NAT Gateway which affects all outbound traffic. App host machines may be replaced with larger or smaller machines, which should be a seamless transition, but may suddenly remove CPU an App was relying on beyond its allocated limit.

These failure modes are opaquely prevented by the Aptible platform when CPU Limits are enforced, but they can fail with CPU Limits not enforced. The transition of turning on CPU Limit enforcement is not trivial. It requires confirming ahead of time that you don't have important Apps relying on CPU beyond their allocation.


Did this page help you?