Fails to install llama-cpp on windows. Latest releases from abetlen/llama-cpp-python don't include windows wheels

[o0Maikeru0o]

Hard coded the correct wheel url as a work around

# If pre-built wheels fail, try GitHub release wheels try: version = latest_lamacpp() platform_tag = system_info['platform_tag'] if platform_tag: wheel_url = f"https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.4-cu124/llama_cpp_python-0.3.4-cp312-cp312-win_amd64.whl" print(f"Attempting to install from {wheel_url}") install_package(wheel_url) print(f"Successfully installed llama-cpp-python v{version}") return True except Exception as e: print(f"GitHub wheel installation failed: {e}") print("Attempting source build with acceleration...")

[HurrsonBurrson]

#134

[LichAcademy]

Let me shed some light, for those of us who are not professional developers.

What is happening?

Basically, when you try to install llama-cpp-python, it installs the latest version, which is 0.3.5 (at the time of writing). However, this version does not have wheels for Windows or Linux. Only for Mac (labelled 'metal').

The script then tries to build the tools, but it fails. I don't know about you, but in my case, it is probably because I am using conda. It won't build wheels on Windows or Linux, amazingly! I think conda environment doesn't communicate with C++ compiler, which is installed externally.

Anyway... So, we activate environment and then install previous version of llama-cpp-python (v0.3.4), by specifying the version explicitly:

pip install llama-cpp-python==0.3.4 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124/

Huh? 'Activate Environment?' What?

If you are using Comfy Portable, entire Python distribution is embedded in the python_embeded folder, which means there is no traditional venv/Scripts/activate or conda activate setup for activating your virtual environment.

You'll need to manually pick the Python interpreter for each pip command. For example, to see what you currently have installed, instead of simply using pip list, you use:

python_embeded\python.exe -m pip list

This will list all the Python packages currently installed in your ComfyUI. If llama-cpp-python is missing, you need to install it. But instead of using pip install command (like what I listed above), we must manually pick the Python interpreter using:

python_embeded\python.exe -m pip install llama-cpp-python==0.3.4 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124/

And that should do it. Hopefully, this helps. 👍