> For the complete documentation index, see [llms.txt](https://docs.konnex.world/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.konnex.world/supported-ai-models/pi0.md).

# PI-0

## 1. What is PI0 and why KNX needs it

PI0 is a Large Behavior Model (LBM) for robotic manipulation. It takes a history of observations (camera images, joint states) and predicts low-level actions (e.g., torques or joint deltas). It’s trained on large, diverse demonstrations across multiple labs and robot platforms.

In the LeRobot ecosystem, PI0 is distributed as a ready-to-use policy:

* Base model on Hugging Face: `lerobot/pi0_base`

Why this matters for KNX:

* An open-source “brain” for manipulators you can run locally.
* A convenient baseline to compare against other VLA/LBM models.
* Useful for benchmarking KNX-related simulation scenes.

***

## 2. GPU server prep (A10 on vast.ai or similar)

Recommended minimum:

* GPU: NVIDIA A10 (24 GB) or better
* CPU: 8+ vCPU
* RAM: 32 GB
* OS: Ubuntu 20.04 / 22.04
* NVIDIA drivers + CUDA 12.x installed (common on vast.ai images)

Assumptions:

* You have SSH access, e.g.:

```bash
ssh USERNAME@your-server-ip
```

* Working under `/data`.

***

## 3. Miniconda and the `lerobot` environment

```bash
cd /data

# Install Miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
bash miniconda.sh -b -p /data/miniconda

# Activate conda in this shell
eval "$(/data/miniconda/bin/conda shell.bash hook)"
conda init
exec $SHELL

# Create env
conda create -y -n lerobot python=3.10
conda activate lerobot
```

Set up Hugging Face cache:

```bash
export HF_HOME=/data/hf_home
mkdir -p "$HF_HOME"
```

(Add these to `~/.bashrc` if you want them persistent.)

***

## 4. System deps (MuJoCo/robosim, ffmpeg, OpenGL)

```bash
sudo apt update
sudo apt install -y \
  git \
  ffmpeg \
  libgl1-mesa-glx \
  libglib2.0-0 \
  libglfw3 \
  libosmesa6
```

MuJoCo is pulled via Python packages, but GL libraries are still needed.

***

## 5. Install LeRobot from source

```bash
cd /data
git clone https://github.com/huggingface/lerobot.git
cd lerobot

conda activate lerobot
# LeRobot + policies (includes PI0)
pip install -e ".[policies]"
```

If `hf-egl-probe` build fails due to missing `cmake`:

```bash
pip install cmake
pip install hf-egl-probe
```

***

## 6. Smoke test: run PI0 on Aloha (example)

Create `eval_pi0_aloha_live.py` under `/data/lerobot`:

```python
# file: eval_pi0_aloha_live.py
import torch

from lerobot.envs.factory import make_env
from lerobot.policies.pi0.policy import PI0Policy


def main():
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    # Example: Aloha insertion task (ensure gym_aloha is installed)
    env = make_env(
        env_type="gym_aloha",
        task_id="AlohaInsertion-v0",
        obs_type="image",
        render_mode="rgb_array",
    )

    policy = PI0Policy.from_pretrained("lerobot/pi0_base")
    policy.to(device)
    policy.eval()

    num_episodes = 5
    for ep in range(num_episodes):
        obs, info = env.reset()
        done = False
        step_idx = 0
        while not done:
            with torch.no_grad():
                action = policy.act(obs, device=device)
            obs, reward, terminated, truncated, info = env.step(action)
            done = bool(terminated or truncated)
            step_idx += 1
        print(f"Episode {ep} finished, steps={step_idx}, success={info.get('is_success', False)}")


if __name__ == "__main__":
    main()
```

Run:

```bash
cd /data/lerobot
conda activate lerobot
python eval_pi0_aloha_live.py
```

Expected:

* `lerobot/pi0_base` weights download from Hugging Face.
* Several episodes in simulation; success flag printed per episode.

***

## 7. KNX demo script: run and record a video

Save `pi0_run_and_record.py`:

```python
# file: pi0_run_and_record.py
import os
import cv2
import torch
import numpy as np

from lerobot.envs.factory import make_env
from lerobot.policies.pi0.policy import PI0Policy


def main():
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    env = make_env(
        env_type="gym_aloha",
        task_id="AlohaInsertion-v0",
        obs_type="image",
        render_mode="rgb_array",
        fps=30,
    )

    policy = PI0Policy.from_pretrained("lerobot/pi0_base")
    policy.to(device)
    policy.eval()

    out_path = "pi0_aloha_episode.mp4"
    fps = 30
    frames = []

    obs, info = env.reset()
    done = False
    step_idx = 0

    while not done:
        with torch.no_grad():
            action = policy.act(obs, device=device)
        obs, reward, terminated, truncated, info = env.step(action)
        done = bool(terminated or truncated)
        frame = env.render()
        if frame is not None:
            frames.append(frame)
        step_idx += 1

    print(f"Episode finished, steps={step_idx}, success={info.get('is_success', False)}")

    if len(frames) == 0:
        print("No frames collected, check env.render()")
        return

    h, w, _ = frames[0].shape
    writer = cv2.VideoWriter(
        out_path,
        cv2.VideoWriter_fourcc(*"mp4v"),
        fps,
        (w, h),
    )
    for f in frames:
        bgr = cv2.cvtColor(f, cv2.COLOR_RGB2BGR)
        writer.write(bgr)
    writer.release()
    print(f"Saved video to {out_path}")


if __name__ == "__main__":
    main()
```

Run:

```bash
cd /data/lerobot
conda activate lerobot
python pi0_run_and_record.py
```

Use the resulting `pi0_aloha_episode.mp4` in KNX docs or demos.

***

## 8. Pros and cons for KNX

Pros:

* Fully open model and weights (`lerobot/pi0_base`).
* Runs comfortably on a single A10.
* Integrated with LeRobot (multi-dataset, multiple policies).
* Strong baseline across different sim scenes (robosuite, LIBERO, gym\_aloha).

Cons:

* Not a text-conditioned VLA that follows arbitrary prompts; it’s a strong behavior policy for specific tasks.
* Without tailored integration to KNX scenes, looks more like a classic RL/IL baseline than a generalist KNX agent.
* Requires GPU and simulator setup, which is non-trivial for casual users.