Let a provider state its own window and image limits

This commit is contained in:
Thorsten Sommer committed 2026-09-12 20:50:38 +02:00
1 parent 8fdef4aa6c
commit bb22b3c10a
3 files changed
+506 -7

No files matched your search

@@ -110,16 +110,31 @@ CONFIG["LLM_PROVIDERS"] = {}
-- -- surfaces.
-- -- ["IconPath"] = "assets/project-icon.svg",
--
-- -- Optional: expert capability overrides.
-- -- Allowed keys are exactly:
-- -- Optional: expert overrides for the model behind this provider. Missing keys keep the
-- -- automatic answer, and each key contradicts only what it names.
-- --
-- -- What the model can do. Allowed keys are exactly:
-- -- AUDIO_INPUT, FUNCTION_CALLING, MULTIPLE_IMAGE_INPUT, SPEECH_INPUT, VIDEO_INPUT,
-- -- OPTIONAL_REASONING, ALWAYS_REASONING, REASONING_BY_DEFAULT
-- -- Allowed values are booleans only.
-- -- For default-on reasoning (thinking), set OPTIONAL_REASONING and REASONING_BY_DEFAULT to true.
-- -- ALWAYS_REASONING means the model cannot disable reasoning (thinking).
-- -- Missing keys keep the automatic capability detection result.
-- --
-- -- How much the model reads and how many images it takes. Allowed keys are exactly:
-- -- CONTEXT_WINDOW, MAX_IMAGES_PER_MESSAGE, MAX_IMAGES_PER_REQUEST
-- -- Allowed values are whole numbers: tokens greater than zero for the window, and images of
-- -- zero or more for the two limits, where zero means the model is configured to take none.
-- -- These are the same key names a model plugin uses for the same questions, but they say
-- -- something narrower here: a model plugin describes a model wherever it is reached, while
-- -- these describe this one installation of it. State what your deployment actually does --
-- -- for a self-hosted engine, the window your operator configured rather than the one the
-- -- model card advertises.
-- -- CONTEXT_WINDOW feeds the token counter AI Studio shows below the chat input, so a wrong
-- -- number here misleads users about how much room they have left.
-- -- ["CapabilityOverrides"] = {
-- -- ["VIDEO_INPUT"] = false,
-- -- ["CONTEXT_WINDOW"] = 32768,
-- -- ["MAX_IMAGES_PER_REQUEST"] = 4,
-- -- },
--
-- -- Optional: Hugging Face inference provider. Only relevant for UsedLLMProvider = HUGGINGFACE.