Starting a simple SGLang instance

What's required?#

Set CUDA_HOME

Copyexport CUDA_HOME=/scratch/public/nvidia/cuda/cuda-12.9

An apropriate environment

For this we will use UV, more information on UV can be found here: UV Venv Documentation

Copyuv venv --python 3.12 py312

# '.' is a handy shortcut instead of typing out 'source'
. py312/bin/activate

uv pip install sglang[all]

From here you should be able to set