Narration transcript

AI-narrated article. Prose is preserved with pronunciation adjustments; code and image cues refer to the written article.

Read the original article

Kubernetes In-Place Pod Resource Updates And Vertical Pod Resize

This is the AI-narrated article. The prose follows the written post; pronunciation is adjusted, and code and images are announced separately.

Kubernetes can now resize a running Pod's CPU and memory without replacing it. This deep dive explains the resize API, kubelet behavior, failure states, VPA integration, and production limits.

TL;DR

Kubernetes in-place Pod resource updates make container CPU and memory requests and limits mutable through the Pod resize subresource. Kubernetes records the requested values as desired state, while kubelet reports the resources actually configured on the existing node. CPU changes can usually avoid restarts; memory changes need deliberate resize policies and remain riskier, especially when decreasing limits. The feature is stable from Kubernetes 1.35, but it does not reschedule an undersized Pod, change its QoS class, update its controller template, or guarantee that application runtimes adapt automatically.

Image. Kubernetes Pod changing CPU and memory resources in place while retaining its identity, network, and storage.

In-place Pod resize separates resource changes from Pod replacement: Kubernetes can change CPU and memory on the existing node while preserving Pod identity.

Vertical Scaling No Longer Has to Mean Pod Replacement

For most of Kubernetes' history, changing a container's CPU or memory meant replacing its Pod. A Deployment or StatefulSet controller created a new Pod from an updated template; the scheduler placed it; kubelet rebuilt the sandbox; and the application started again. That model is simple, but it is expensive for stateful services, long-running jobs, game servers, pre-warmed workers, and latency-sensitive processes.

In-place Pod resize breaks that coupling. CPU and memory requests and limits become mutable on the existing Pod. The Pod stays on its node and retains its UID, IP address, volumes, and sandbox. Depending on the container's policy and runtime behavior, even the container can continue running.

The feature, also known as In-Place Pod Vertical Scaling, entered alpha in Kubernetes 1.27, became beta and default-enabled in 1.33, and graduated to stable in Kubernetes 1.35. Stable does not mean every resize is immediate or restart-free. It means the API and core behavior are production-grade; capacity, runtime, QoS, and workload-controller constraints still matter.

Version and client boundaries:

Table. Columns: Component, Minimum or feature state, Why it matters.

Row 1. Component: Kubernetes server; Minimum or feature state: 1.33 or later for the current workflow; Why it matters: The feature was beta in 1.33 and stable in 1.35..

Row 2. Component: Kubernetes 1.35+; Minimum or feature state: Stable, enabled by default; Why it matters: Container-level CPU and memory resize is GA..

Row 3. Component: kubectl; Minimum or feature state: 1.32 or later; Why it matters: Older clients do not support --subresource=resize..

Row 4. Component: Kubernetes 1.36 Pod-level resize; Minimum or feature state: Separate beta feature; Why it matters: It resizes a Pod-level resource budget rather than only container fields..

End of table.

Start with the official resize task and confirm the versions you actually operate:

Code example. Refer to the written article for the complete code.

The Core Contract: Desired Resources Are Not Actual Resources

The most important architectural change is not that resources became mutable. It is that Kubernetes now exposes a deliberate gap between what you requested and what kubelet has applied.

spec.containers[*].resources is the desired CPU and memory state.

status.containerStatuses[*].resources is the resource state currently configured for the container.

status.containerStatuses[*].allocatedResources is kubelet-confirmed allocation data used mainly for internal scheduling decisions.

status.conditions explains whether a resize is pending or in progress.

An accepted API patch only proves that Kubernetes stored the desired state. It does not prove that the node had capacity, the runtime changed its cgroups, or the application adapted.

Image. Execution path from kubectl resize request through the API server, scheduler accounting, kubelet, container runtime, and cgroups.

Resize is asynchronous. The API server stores intent; kubelet on the existing node actuates it; container status reports what actually changed.

The request follows this path:

A user or autoscaler writes the Pod's /resize subresource.

Admission and API validation check the new values and preserve the Pod's QoS invariants.

Kubernetes stores the desired resources in the Pod specification.

Kubelet sees that desired and actual resources differ.

Kubelet checks node capacity and coordinates the change with the container runtime.

The runtime updates CPU and memory controls, typically Linux cgroups.

Kubelet updates container status and Pod resize conditions.

During an incomplete resize, the scheduler accounts for the maximum of desired, allocated, and actual requests. That conservative rule prevents Kubernetes from treating capacity as free while different components still observe different generations of the Pod.

Create a Pod with explicit CPU and memory policies:

Code example. Refer to the written article for the complete code.

This is a Guaranteed-QoS Pod because request equals limit for both resources. Apply it and inspect the initial state:

Code example. Refer to the written article for the complete code.

CPU and Memory Need Different Resize Policies

The per-container resizePolicy separates two decisions: which resource changes and whether that change requires a container restart.

Code example. Refer to the written article for the complete code.

NotRequired is the default. It tells kubelet to update the running container. RestartContainer restarts that container within the same Pod before applying the new value. It does not replace the Pod.

Image. Decision diagram for CPU and memory resize policies followed by completed, deferred, infeasible, and in-progress outcomes.

Policy determines whether a container restart is allowed; node capacity and runtime actuation determine the resulting resize state.

CPU is usually the safer no-restart case. The runtime can adjust CPU shares, weight, or quota while the process continues. A larger CPU envelope still does not force the application to create more workers, and a smaller limit can increase throttling.

Memory is harder. Raising a cgroup memory limit does not guarantee that a managed runtime expands its heap. Lowering a limit can place current usage above the new boundary and trigger an OOM kill. Since Kubernetes 1.35, kubelet allows memory-limit decreases and performs a best-effort usage check when the policy is NotRequired, but the GA announcement explicitly warns that the check is not race-free.

Increase CPU without restarting the container:

Code example. Refer to the written article for the complete code.

Increase memory with the policy above:

Code example. Refer to the written article for the complete code.

The CPU change should leave restartCount unchanged. The memory change should increment it because RestartContainer applies to memory. If both resources change in one request, the container restarts when either policy requires it.

A Pod with restartPolicy: Never cannot declare a resize policy that requires a restart. Every resizable resource in that Pod must use NotRequired.

A Resize Is an Asynchronous State Machine

Production automation must treat resize like reconciliation, not like a synchronous setter. Read the desired values, actual values, conditions, and generation together:

Code example. Refer to the written article for the complete code.

The main outcomes are:

Completed

Desired and actual resources match, the relevant resize condition has cleared, and status.observedGeneration shows that kubelet has acknowledged the current Pod generation.

PodResizePending with Deferred

The node cannot grant the request now, but the request might become feasible after another Pod terminates or shrinks. Kubelet retries deferred resizes. It prioritizes higher PriorityClass, then Guaranteed over Burstable at equal priority, then older deferred requests.

PodResizePending with Infeasible

The resize cannot be satisfied on the current node—for example, the requested CPU exceeds node capacity. Kubernetes does not automatically reschedule the Pod onto a larger node. Submit a feasible value or use higher-level automation that can replace the Pod or change cluster capacity.

PodResizeInProgress

Kubelet accepted the allocation, but runtime actuation is incomplete. The condition can include reason: Error and a message when the runtime cannot apply the change.

This distinction changes alerting. Do not page because spec and status differ for a few seconds. Alert when the gap persists beyond an operational threshold, when Infeasible appears, or when an in-progress resize reports an error.

The Limits That Matter in Kubernetes 1.36

The current Kubernetes documentation lists boundaries that determine whether in-place resize fits a workload:

Only CPU and memory are resizable.

Resize cannot change the Pod's original Guaranteed, Burstable, or BestEffort QoS class.

Guaranteed Pods must keep CPU and memory requests equal to limits.

BestEffort Pods cannot gain requests or limits through resize.

Non-restartable init containers and ephemeral containers cannot be resized; sidecar containers can.

A request or limit cannot be removed entirely after it is set; it can only change value.

Windows Pods do not support in-place resize.

Pods managed by static CPU Manager or static Memory Manager policies cannot be resized.

Pods using swap cannot resize memory requests unless memory uses RestartContainer.

Memory-limit reduction without restart remains best-effort and can still race with allocation.

There is also a critical controller boundary: resizing a Pod does not update its owning Deployment or StatefulSet template.

Code example. Refer to the written article for the complete code.

If the new size must survive Pod replacement, update the controller template as well or use a system that reconciles both lifecycle stages. Be deliberate when a GitOps controller owns the manifest; otherwise, live and declared state can diverge in ways your next rollout silently reverses.

Kubernetes 1.36 also exposes a separate beta workflow for in-place Pod-level resource resize. Do not confuse that Pod-wide resource budget with the stable container-level mechanism explained here.

VPA Turns the Mechanism Into an Autoscaling Policy

Core Kubernetes provides the resize actuator but does not decide the correct size. Vertical Pod Autoscaler supplies recommendations and update policy.

Its relevant modes are:

Table. Columns: VPA mode, Existing Pod behavior.

Row 1. VPA mode: Off; Existing Pod behavior: Produce recommendations only..

Row 2. VPA mode: Initial; Existing Pod behavior: Apply recommendations only when a Pod starts..

Row 3. VPA mode: Recreate; Existing Pod behavior: Evict the Pod so its controller creates a replacement..

Row 4. VPA mode: InPlaceOrRecreate; Existing Pod behavior: Attempt in-place resize, then fall back to eviction when necessary..

Row 5. VPA mode: InPlace; Existing Pod behavior: Never evict; wait and retry pending resize. Alpha in VPA 1.7.0..

End of table.

Code example. Refer to the written article for the complete code.

InPlaceOrRecreate is the pragmatic default when availability matters but eventual convergence matters more: it avoids disruption when possible and retains an eviction fallback. InPlace is stricter about continuity, but an infeasible resize can remain pending indefinitely because that mode will not evict the Pod.

Coordinate VPA with Horizontal Pod Autoscaler. HPA commonly evaluates CPU utilization as usage divided by requested CPU. Changing the request changes that denominator, so an in-place vertical resize can alter horizontal scaling decisions even when traffic and raw CPU consumption stay constant. Assign each autoscaler clear ownership of metrics and resources; otherwise, two valid control loops can work against each other.

Why Kubernetes Built It This Way

Mutating the Pod specification directly without an explicit resize contract would hide too much. The /resize subresource creates a narrower authorization and validation surface. Separate desired and actual fields make asynchronous node actuation observable. Per-resource restart policies acknowledge that CPU and memory have different runtime semantics. Resize conditions distinguish a temporary capacity problem from an impossible request.

The design also preserves the scheduler's fundamental safety model. A Pod remains bound to one node, so kubelet cannot promise resources that the node does not have. Conservative accounting protects the cluster while control plane, kubelet, and runtime converge. The trade-off is intentional: in-place resize minimizes disruption, but it is not rescheduling, node autoscaling, preemption, or application-level reconfiguration.

That boundary makes the feature composable. VPA can decide values and fallback behavior. A node autoscaler can add capacity. An application controller can adapt worker counts or heap settings. Kubernetes core provides a reliable actuator and an observable state machine instead of pretending one resize primitive solves the entire vertical-scaling problem.

A Production Rollout Checklist

Before enabling automated in-place resizing across a fleet:

Confirm API server, kubelet, container runtime, and kubectl versions.

Inventory QoS classes, static CPU/Memory Manager use, swap, Windows workloads, init containers, and sidecars.

Define CPU and memory resizePolicy explicitly rather than relying on defaults.

Test scale-up, scale-down, restart-required memory changes, Deferred, and Infeasible behavior.

Compare desired and actual resource fields in monitoring.

Alert on persistent generation gaps and resize error conditions.

Verify that application runtimes use newly available CPU or memory as expected.

Decide whether controller templates, GitOps state, or VPA own durable resource values.

Test HPA and VPA together under stable traffic and changing recommendations.

Keep disruption budgets and a recreate fallback for workloads where convergence outranks zero restart.

Clean up the manual example when finished:

Code example. Refer to the written article for the complete code.

Frequently Asked Questions

Q: What are Kubernetes in-place Pod resource updates?

A: They let Kubernetes change a running container's CPU and memory requests or limits through the Pod /resize subresource. The Pod remains on its current node and retains its identity, IP address, sandbox, and mounted volumes.

Q: Does an in-place Pod resize restart the container?

A: Not necessarily. resizePolicy can use NotRequired to update the running container or RestartContainer to restart that container. If no policy is specified for a resource, Kubernetes defaults to NotRequired.

Q: How do I verify that a Pod resize succeeded?

A: Compare desired values in spec.containers[*].resources with actual values in status.containerStatuses[*].resources. Inspect PodResizePending, PodResizeInProgress, status.observedGeneration, and the container restart count before declaring success.

Q: Does resizing a Pod update its Deployment or StatefulSet?

A: No. The resize subresource changes one Pod, not the owning controller's template. Update the template or use a reconciling autoscaler if replacement Pods must inherit the new resources.

Q: Can VPA resize Pods without evicting them?

A: Yes. InPlaceOrRecreate first attempts in-place resize and can fall back to eviction. VPA's alpha InPlace mode never evicts; it keeps waiting and retrying if a resize is deferred, in progress, or infeasible.

Primary Sources and Next Steps

Resize CPU and memory resources assigned to containers

Kubernetes 1.35: In-Place Pod Resize graduates to stable

Kubernetes 1.33: In-Place Pod Resize graduates to beta

KEP-1287: In-place update of Pod resources

Resource management for Pods and containers

Vertical Pod Autoscaling

Horizontal Pod Autoscaling

Pod lifecycle and resize status

Resize CPU and memory resources assigned to Pods

Audio library