mirror of
https://github.com/MindWorkAI/AI-Studio.git
synced 2026-10-15 12:33:48 +00:00
Add llmman as a self-hosted host for chat and embedding providers (#947)
Co-authored-by: Thorsten Sommer <SommerEngineering@users.noreply.github.com>
This commit is contained in:
1 parent
2bd3d41421
commit
88d1f4d96a
20 files changed
+270
-89
No files matched your search
@@ -8,8 +8,8 @@ namespace AIStudio.Provider.SelfHosted;
|
||||
/// <remarks>
|
||||
/// The context window is vLLM's addition to that route: it reports the window the operator started
|
||||
/// the engine with, which is the one number no rule about the weights could ever know. Ollama,
|
||||
/// LM Studio, and llama.cpp answer the same route without it, so it stays unknown there instead of
|
||||
/// being guessed.
|
||||
/// LM Studio, llama.cpp, and llmman answer the same route without it, so it stays unknown there
|
||||
/// instead of being guessed.
|
||||
///
|
||||
/// vLLM calls that field max_model_len, which reads like a limit on the model rather than on a
|
||||
/// conversation. The wire keeps their spelling, and this record says what the number means, so that
|
||||
|
||||
Reference in new issue
Block a user