Skip to content

Instantly share code, notes, and snippets.

@mrseanryan

mrseanryan/gist:1f6ee66fde867ac37e9c57da2d7a3f09

Last active January 22, 2024 16:39
Show Gist options
  • Save mrseanryan/1f6ee66fde867ac37e9c57da2d7a3f09 to your computer and use it in GitHub Desktop.
Save mrseanryan/1f6ee66fde867ac37e9c57da2d7a3f09 to your computer and use it in GitHub Desktop.
Download ZIP
Install text-generation-webui with GPU support - via llama-cpp-python
Raw
gistfile1.txt
sudo apt update
sudo apt install python3-pip
conda create -n textgen python=3.10.9
conda activate textgen
conda init bash
pip install torch torchvision torchaudio
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt
pip uninstall -y llama-cpp-python
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment