# Automated resources sizing tool for containers in kubernetes


![drawing](https://github.com/unagex/kondense/raw/main/logo.png align="center")

Kondense is an automated resource sizing tool. It runs as a sidecar in kubernetes pods.

## Background

### Memory

Kondense uses memory pressure to apply just the right amount of memory on a container to page out the unused memory while not getting out-of-memory killed.  
[How is memory calculated ?](https://github.com/unagex/kondense/blob/main/doc/memory.md)

### CPU

Kondense resizes CPU based on CPU usage, default to 80%.  
[How is CPU calculated ?](https://github.com/unagex/kondense/blob/main/doc/cpu.md)

## Requirements

### On Kubernetes

* The Kubernetes cluster must run on Linux.
    
* Containerd version >= 1.6.9.
    
* Kubernetes should have the feature gate `InPlacePodVerticalScaling` enabled.
    

### On Containers

* Containers should include the linux kernel version >= 4.20. Ensure the file `/sys/fs/cgroup/memory.pressure` exists in the container to verify it.
    

You can build your containers from the image `alpine` to be sure.

## Example

Try the example on minikube:

```yaml
minikube start --kubernetes-version=v1.29.2 --feature-gates=InPlacePodVerticalScaling=true
kubectl apply -f https://raw.githubusercontent.com/unagex/kondense/main/example/nginx.yaml
```

Let's say we have a pod running `nginx` that we want to Kondense:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: kondense-test
spec:
  serviceAccountName: nginx-user
  containers:
  - name: nginx
    image: nginx:latest
    resources:
      limits:
        cpu: 100m
        memory: 100M
```

Add Kondense as a sidecar:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: kondense-test
spec:
  serviceAccountName: nginx-user
  containers:
  - name: nginx
    image: nginx:latest
    resources:
      limits:
        cpu: 0.1
        memory: 100M
  - name: kondense
    image: kondense/kondense:1.1.0
    resources:
      limits:
        cpu: 80m
        memory: 50M
```

After adding the kondense container, the nginx container resources are updated without any container restart.

**Notes:**

1. The pod should have a QoS of `Guaranteed`. In other words, we need to add resources limits for each containers.
    
2. The service account `nginx-user` should have the following rules:
    

```yaml
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
```

## Configuration

Kondense is configurable via environment variables in the kondense container.

#### Example

```yaml
    ...
    - name: kondense
      image: kondense/kondense:1.1.0
      resources:
        limits:
          cpu: 80m
          memory: 50M
      env:
      - name: <CONTAINER NAME>_MEMORY_MIN
        value: "100m"
```

If we have a container named `nginx` in our pod, the variable name should be `NGINX_MEMORY_MIN`.

### Environment variables

#### Global

| Name | Default value | Description |
| --- | --- | --- |
| EXCLUDE | "" | Comma separated list of containers to not kondense. |

#### Memory

| Name | Default value | Description |
| --- | --- | --- |
| &lt;CONTAINER NAME&gt;\_MEMORY\_MIN | 50M | Minimum memory of the container. Kondense will never resize below that limit. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_MAX | 100G | Maximum memory of the container. Kondense will never resize above that limit. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_TARGET\_PRESSURE | 10000 | Target memory pressure in microseconds. Kondense will take corrective actions to obtain it. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_INTERVAL | 10 | Kondense targets cumulative memory delays over the sampling period of this interval in seconds. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_MAX\_INC | 0.5 | Maximum memory increase for one correction. e.g. 0.5 is a 50% increase. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_MAX\_DEC | 0.02 | Maximum memory decrease for one correction. e.g. 0.02 is a 2% decrease. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_COEFF\_INC | 20 | Coeff to increase memory when the memory pressure is bigger then the target memory pressure. |
| &lt;CONTAINER NAME&gt;\_MEMORY\_COEFF\_DEC | 10 | Coeff to decrease memory when the memory pressure is smaller then the target memory pressure. |

#### CPU

| Name | Default value | Description |
| --- | --- | --- |
| &lt;CONTAINER NAME&gt;\_CPU\_MIN | 0.08 | Minimum CPU of the container. Kondense will never resize below that limit. |
| &lt;CONTAINER NAME&gt;\_CPU\_MAX | 100 | Maximum CPU of the container. Kondense will never resize above that limit. |
| &lt;CONTAINER NAME&gt;\_CPU\_MAX\_INC | 0.5 | Maximum CPU increase for one correction. e.g. 0.5 is a 50% increase. |
| &lt;CONTAINER NAME&gt;\_CPU\_MAX\_DEC | 0.1 | Maximum CPU decrease for one correction. e.g. 0.1 is a 10% decrease. |
| &lt;CONTAINER NAME&gt;\_CPU\_TARGET\_AVG | 0.8 | Target CPU average for the container. It is from 0 to 1. e.g. 0.8 means a target cpu usage of 80%. |
| &lt;CONTAINER NAME&gt;\_CPU\_INTERVAL | 6 | Target CPU average for the container. It is from 0 to 1. e.g. 0.8 means a target cpu usage of 80%. |
| &lt;CONTAINER NAME&gt;\_CPU\_COEFF | 6 | Used to calculate the new cpu limit when a cpu increase is needed. The higher the coeff, the higher the new cpu limit. |

### More

* Kondense memory resize is based on Meta [Transparent Memory Offloading (TMO)](https://www.cs.cmu.edu/~dskarlat/publications/tmo_asplos22.pdf)
    
* Kondense is active on himself by default