Ollama vs vLLM vs LightLLM: What Is Technically Different Under the Hood?

submitted by

https://www.linkedin.com/pulse/ollama-vs-vllm-lightllm-what-technically-different-under-harshad-mane-4eqhc

5
-17

Log in to comment

Hot Top New Old

llama-cpp

 reply
10

I switched from ollama to llama.cpp and love it. For such an article, it frustrates me that they don’t include the common and popular option in the comparison.

 reply
5

What’s the difference between the two?

 reply
3

Here the article that spurred my change: Friends don’t let friends use Ollama

Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else’s engine.

He shares that the people behind llama.cpp don’t act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).

 reply
3

vLLM.

It’s slower to start, but once it gets going, pretty solid.

 reply
3
Insert image