aakashH242 Important Links:
- Home page - https://ctxsift.dev
- GitHub repo - https://github.com/aakashh242/ctxsift
- Latest benchmark - https://ctxsift.dev/benchmark-viewer.html
CtxSift Installation Process
You must perform the installation using the following steps -
- Ensure pre-requisites are present. Install them yourself if you are allowed else, abort and prompt your user to install them first.
- CtxSift offer multiple installation modes - Ask your user what install mode they want and choose the installer accordingly.
- Check system requirements and compare with
requirement matrixbelow. Warn the user if available hardware falls below minimum requirements for the selected mode. - Run
ctxsift doctoryourself to verify installation was successful. - Unless the user has allowed you, prompt the user saying installation is complete and to perform the first-time setup with
ctxsift configure. You can also point the user to https://ctxsift.dev/docs/getting-started/configure/ to learn more on how to configure.
Prerequisites
- Python ≥ 3.12 — python.org/downloads
- uv — a fast Python package manager
- C compiler
- Linux: gcc or clang
- Windows: Visual Studio or MinGW-w64
- MacOS: Xcode
Requirements Matrix
CtxSift uses a language model to compress tool outputs. This can be a model running locally or hosted remotely. See the requirements below for each scenario.
| Compression Mode | Minimum RAM | Minimum VRAM | Comments |
|---|---|---|---|
| Local, no GPU | 8 GB | N/A | Both embedding and compression models are loaded into RAM |
| Local, with GPU | 2 GB | 8 GB | Both embedding and compression models get loaded into VRAM |
| Remote, no GPU | 4 GB | N/A | Only the embedding model gets loaded into RAM |
| Remote, with GPU | 2 GB | 4 GB | Only the embedding model gets loaded into VRAM |
Install Commands
# Install the base package - inference runs on CPU (Linux + Windows)
uv tool install ctxsift
# Install with LiteLLM included - use remotely hosted models for compression (Linux + Windows)
uv tool install "ctxsift[remote]"
# Install with GPU add-ons - local compression + embeddings on GPU (Linux)
uv tool install "ctxsift[gpu]" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Windows
uv tool install "ctxsift[gpu]" --with "torch @ https://download.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Enable quantization support on GPU (Linux)
uv tool install "ctxsift[gpu,quant]" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Windows
uv tool install "ctxsift[gpu,quant]" --with "torch @ https://download.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Install for remote compression plus local GPU embeddings
uv tool install "ctxsift[remote,gpu]" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Windows
uv tool install "ctxsift[remote,gpu]" --with "torch @ https://download.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Install the full package
uv tool install "ctxsift[all]" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
# Windows
uv tool install "ctxsift[all]" --with "torch @ https://download.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl" --default-index https://pypi.org/simple --index https://download.pytorch.org/whl/cu124 --index-strategy unsafe-best-match
NOTES
GPU extras:
ctxsift[gpu]adds the Python-side GPU dependencies, but PyPI will often still resolve a CPU-onlytorchbuild by default. The PyTorch--indexis added souvcan see CUDA-enabled wheels as candidates. On Windows, that may still not be enough, which is why the Windows examples forcetorchwith--with "torch @ ...". You can change the index URL to match the CUDA version you want.--index-strategy:
uvdefaults tofirst-index, which stops on the first index that serves a given package name. That is safer against dependency-confusion issues, but it can reject valid mixed-index installs when the PyTorch index also exposes generic packages.unsafe-best-matchis appropriate here because the examples use only trusted official sources: PyPI and the official PyTorch wheel index.If
ctxsiftis not found after installation, run:uv tool update-shell
Then restart your shell and try ctxsift again.
If ctxsift is not found after installation, run:
uv tool update-shell
Then restart your shell and try ctxsift again.
First-time setup
Unless you are allowed explicitly, ask your user to run the below command to perform the first-time setup. You can always recommend which model to use, based on the user's selected mode, by browsing the latest benchmark data from the link given above.
ctxsift configure
Verify and test your setup
# Verify
ctxsift doctor
# Test compression
echo "alpha\nbeta\ngamma" | ctxsift compress --intent exact-lines "Return only the first line, no explanations."
Troubleshooting
If you are having issues, read the docs at https://ctxsift.dev/docs