Auto-Scaling

The number of compute nodes in a cluster can auto-scale depending on the job queue depth. This behavior is controlled by the Jobs per Compute Node setting and constrained by the Minimum Nodes and Maximum Nodes settings.

Rules

Adding Compute Nodes

A cluster will add a new compute node if the following condition is met:

node_count < floor(queue_depth / jobs_per_compute_node)

Remove Compute Nodes

A cluster will remove a compute node if the following condition is met:

node_count > ceil(queue_depth / jobs_per_compute_node)

Example

If jobs_per_compute_node = 1000 and the queue starts empty, the cluster will use a single compute node until queue_depth reaches 2000, at which point it will add a second compute node.

As the queue shrinks, the cluster will attempt to remove a compute node when queue_depth drops below 1000.

Minimizing Costs

The cluster will attempt to minimize your costs by never removing a compute node that is processing a job, and only removing compute nodes that are near the end of a billed hour.