mirror of
https://github.com/MindWorkAI/AI-Studio.git
synced 2026-08-15 00:22:11 +00:00
Added Hetzner as an experimental LLM provider (#912)
Some checks are pending
Build and Release / Determine run mode (push) Waiting to run
Build and Release / Read metadata (push) Blocked by required conditions
Build and Release / Sync Flatpak repo (push) Blocked by required conditions
Build and Release / Collect Flatpak artifacts (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-apple-darwin, osx-arm64, macos-latest, aarch64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-pc-windows-msvc.exe, win-arm64, windows-latest, aarch64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-unknown-linux-gnu, linux-arm64, ubuntu-22.04-arm, aarch64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-apple-darwin, osx-x64, macos-latest, x86_64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-pc-windows-msvc.exe, win-x64, windows-latest, x86_64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-unknown-linux-gnu, linux-x64, ubuntu-22.04, x86_64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Prepare & create release (push) Blocked by required conditions
Build and Release / Publish release (push) Blocked by required conditions
Some checks are pending
Build and Release / Determine run mode (push) Waiting to run
Build and Release / Read metadata (push) Blocked by required conditions
Build and Release / Sync Flatpak repo (push) Blocked by required conditions
Build and Release / Collect Flatpak artifacts (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-apple-darwin, osx-arm64, macos-latest, aarch64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-pc-windows-msvc.exe, win-arm64, windows-latest, aarch64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-unknown-linux-gnu, linux-arm64, ubuntu-22.04-arm, aarch64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-apple-darwin, osx-x64, macos-latest, x86_64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-pc-windows-msvc.exe, win-x64, windows-latest, x86_64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-unknown-linux-gnu, linux-x64, ubuntu-22.04, x86_64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Prepare & create release (push) Blocked by required conditions
Build and Release / Publish release (push) Blocked by required conditions
This commit is contained in:
parent
bb8f6f13f0
commit
e026c03d14
@ -8896,6 +8896,9 @@ UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T1014558951"] = "The trust leve
|
||||
-- You or your organization operate the LLM locally or within your trusted network. In terms of data processing and security, this is the best possible way.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2124364471"] = "You or your organization operate the LLM locally or within your trusted network. In terms of data processing and security, this is the best possible way."
|
||||
|
||||
-- The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2930312134"] = "The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training."
|
||||
|
||||
-- The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T3010553924"] = "The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**."
|
||||
|
||||
|
||||
@ -5,11 +5,11 @@
|
||||
<MudTooltip Text="@T("Shows and hides the confidence card with information about the selected LLM provider.")" Placement="Placement.Top">
|
||||
@if (this.Mode is PopoverTriggerMode.ICON)
|
||||
{
|
||||
<MudIconButton Icon="@Icons.Material.Filled.Security" Class="confidence-icon" Style="@this.LLMProvider.GetConfidence(this.SettingsManager).SetColorStyle(this.SettingsManager)" OnClick="@(() => this.ToggleConfidence())"/>
|
||||
<MudIconButton Icon="@Icons.Material.Filled.Security" Class="confidence-icon" Style="@this.LLMProvider.GetConfidence(this.SettingsManager).SetColorStyle(this.SettingsManager)" OnClick="@this.ToggleConfidence"/>
|
||||
}
|
||||
else
|
||||
{
|
||||
<MudButton Variant="Variant.Filled" StartIcon="@Icons.Material.Filled.Security" IconClass="confidence-icon" Style="@this.LLMProvider.GetConfidence(this.SettingsManager).SetColorStyle(this.SettingsManager)" OnClick="@(() => this.ToggleConfidence())">
|
||||
<MudButton Variant="Variant.Filled" StartIcon="@Icons.Material.Filled.Security" IconClass="confidence-icon" Style="@this.LLMProvider.GetConfidence(this.SettingsManager).SetColorStyle(this.SettingsManager)" OnClick="@this.ToggleConfidence">
|
||||
@T("Confidence")
|
||||
</MudButton>
|
||||
}
|
||||
@ -28,7 +28,7 @@
|
||||
<MudText Typo="Typo.h6">
|
||||
@T("Description")
|
||||
</MudText>
|
||||
<MudMarkdown Value="@this.currentConfidence.Description" MarkdownPipeline="Markdown.SAFE_MARKDOWN_PIPELINE"/>
|
||||
<MudJustifiedMarkdown Value="@this.currentConfidence.Description" />
|
||||
|
||||
@if (this.currentConfidence.Sources.Count > 0)
|
||||
{
|
||||
@ -61,7 +61,7 @@
|
||||
</MudText>
|
||||
</MudCardContent>
|
||||
<MudCardActions>
|
||||
<MudButton Variant="Variant.Filled" OnClick="@(() => this.HideConfidence())">
|
||||
<MudButton Variant="Variant.Filled" OnClick="@this.HideConfidence">
|
||||
Close
|
||||
</MudButton>
|
||||
</MudCardActions>
|
||||
|
||||
@ -634,7 +634,7 @@ CONFIG["SETTINGS"] = {}
|
||||
-- Configure a custom confidence scheme.
|
||||
-- This is used when DataConfidence.ConfidenceScheme is set to CUSTOM.
|
||||
-- Allowed provider keys are: OPEN_AI, ANTHROPIC, MISTRAL, GOOGLE, X, DEEP_SEEK, ALIBABA_CLOUD,
|
||||
-- PERPLEXITY, OPEN_ROUTER, FIREWORKS, GROQ, HUGGINGFACE, SELF_HOSTED, HELMHOLTZ, GWDG
|
||||
-- PERPLEXITY, OPEN_ROUTER, HETZNER, FIREWORKS, GROQ, HUGGINGFACE, SELF_HOSTED, HELMHOLTZ, GWDG
|
||||
-- Allowed confidence values are: UNTRUSTED, VERY_LOW, LOW, MODERATE, MEDIUM, HIGH
|
||||
--
|
||||
-- Replaces, does not merge: a configuration with a higher priority replaces the whole
|
||||
@ -651,6 +651,7 @@ CONFIG["SETTINGS"] = {}
|
||||
-- ["ALIBABA_CLOUD"] = "LOW",
|
||||
-- ["PERPLEXITY"] = "MODERATE",
|
||||
-- ["OPEN_ROUTER"] = "MODERATE",
|
||||
-- ["HETZNER"] = "HIGH",
|
||||
-- ["FIREWORKS"] = "MODERATE",
|
||||
-- ["GROQ"] = "MODERATE",
|
||||
-- ["HUGGINGFACE"] = "MODERATE",
|
||||
|
||||
@ -8898,6 +8898,9 @@ UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T1014558951"] = "Das Vertrauens
|
||||
-- You or your organization operate the LLM locally or within your trusted network. In terms of data processing and security, this is the best possible way.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2124364471"] = "Sie oder ihre Organisation betreiben das LLM lokal oder innerhalb ihres vertrauenswürdigen Netzwerks. In Bezug auf Datenverarbeitung und Sicherheit ist dies die bestmögliche Lösung."
|
||||
|
||||
-- The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2930312134"] = "Der Anbieter betreibt seinen Dienst in der EU und unterliegt der **DSGVO** (Datenschutz-Grundverordnung). Er bietet Zugang zu **Open-Source-Modellen**. Der Dienst befindet sich jedoch derzeit in einer **experimentellen** Phase; Leistung und Verfügbarkeit werden nicht garantiert. Uns liegen keine anbieterspezifischen Informationen dazu vor, ob übermittelte Daten für das Training verwendet werden."
|
||||
|
||||
-- The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T3010553924"] = "Der Anbieter hat seinen Sitz in der EU und unterliegt der **DSGVO** (Datenschutz-Grundverordnung). Außerdem gibt der Anbieter an, dass **ihre Daten nicht zum Training verwendet werden**."
|
||||
|
||||
|
||||
@ -8898,6 +8898,9 @@ UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T1014558951"] = "The trust leve
|
||||
-- You or your organization operate the LLM locally or within your trusted network. In terms of data processing and security, this is the best possible way.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2124364471"] = "You or your organization operate the LLM locally or within your trusted network. In terms of data processing and security, this is the best possible way."
|
||||
|
||||
-- The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T2930312134"] = "The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training."
|
||||
|
||||
-- The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PROVIDER::CONFIDENCE::T3010553924"] = "The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**."
|
||||
|
||||
|
||||
@ -269,10 +269,10 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
{
|
||||
exception = new();
|
||||
|
||||
if (!line.StartsWith("data: ", StringComparison.InvariantCulture))
|
||||
if (!TryGetServerSentEventData(line, out var jsonData))
|
||||
return false;
|
||||
|
||||
var jsonData = line[6..].Trim();
|
||||
jsonData = jsonData.Trim();
|
||||
if (string.IsNullOrWhiteSpace(jsonData) || jsonData is "[DONE]")
|
||||
return false;
|
||||
|
||||
@ -304,6 +304,21 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
}
|
||||
}
|
||||
|
||||
private static bool TryGetServerSentEventData(string line, out string data)
|
||||
{
|
||||
const string DATA_PREFIX = "data:";
|
||||
data = string.Empty;
|
||||
|
||||
if (!line.StartsWith(DATA_PREFIX, StringComparison.InvariantCulture))
|
||||
return false;
|
||||
|
||||
data = line[DATA_PREFIX.Length..];
|
||||
if (data.StartsWith(' '))
|
||||
data = data[1..];
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool IsProviderStreamFailure(JsonElement root)
|
||||
{
|
||||
var eventType = TryGetString(root, "type");
|
||||
@ -661,13 +676,13 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
if (this.TryCreateProviderRequestExceptionFromStreamLine(providerName, line, out var providerRequestException))
|
||||
throw providerRequestException;
|
||||
|
||||
// Skip lines that do not start with "data: ". Regard
|
||||
// Skip lines that do not start with "data:". According
|
||||
// to the specification, we only want to read the data lines:
|
||||
if (!line.StartsWith("data: ", StringComparison.InvariantCulture))
|
||||
if (!TryGetServerSentEventData(line, out var jsonData))
|
||||
continue;
|
||||
|
||||
// Check if the line is the end of the stream:
|
||||
if (line.StartsWith("data: [DONE]", StringComparison.InvariantCulture))
|
||||
if (jsonData is "[DONE]")
|
||||
yield break;
|
||||
|
||||
//
|
||||
@ -681,10 +696,6 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
|
||||
try
|
||||
{
|
||||
// We know that the line starts with "data: ". Hence, we can
|
||||
// skip the first 6 characters to get the JSON data after that.
|
||||
var jsonData = line[6..];
|
||||
|
||||
// Deserialize the JSON data:
|
||||
providerResponse = JsonSerializer.Deserialize<TAnnotation>(jsonData, JSON_SERIALIZER_OPTIONS);
|
||||
|
||||
@ -713,10 +724,6 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
TDelta? providerResponse;
|
||||
try
|
||||
{
|
||||
// We know that the line starts with "data: ". Hence, we can
|
||||
// skip the first 6 characters to get the JSON data after that.
|
||||
var jsonData = line[6..];
|
||||
|
||||
// Deserialize the JSON data:
|
||||
providerResponse = JsonSerializer.Deserialize<TDelta>(jsonData, JSON_SERIALIZER_OPTIONS);
|
||||
|
||||
@ -866,20 +873,19 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
if (line.StartsWith("event: response.completed", StringComparison.InvariantCulture))
|
||||
yield break;
|
||||
|
||||
if (!TryGetServerSentEventData(line, out var jsonData))
|
||||
continue;
|
||||
|
||||
//
|
||||
// Find delta lines:
|
||||
//
|
||||
if (line.StartsWith("""
|
||||
data: {"type":"response.output_text.delta"
|
||||
""", StringComparison.InvariantCulture))
|
||||
if (jsonData.StartsWith("""
|
||||
{"type":"response.output_text.delta"
|
||||
""", StringComparison.InvariantCulture))
|
||||
{
|
||||
TDelta? providerResponse;
|
||||
try
|
||||
{
|
||||
// We know that the line starts with "data: ". Hence, we can
|
||||
// skip the first 6 characters to get the JSON data after that.
|
||||
var jsonData = line[6..];
|
||||
|
||||
// Deserialize the JSON data:
|
||||
providerResponse = JsonSerializer.Deserialize<TDelta>(jsonData, JSON_SERIALIZER_OPTIONS);
|
||||
|
||||
@ -903,18 +909,14 @@ public abstract class BaseProvider : IProvider, ISecretId
|
||||
//
|
||||
// Find annotation added lines:
|
||||
//
|
||||
else if (annotationSupported && line.StartsWith(
|
||||
else if (annotationSupported && jsonData.StartsWith(
|
||||
"""
|
||||
data: {"type":"response.output_text.annotation.added"
|
||||
{"type":"response.output_text.annotation.added"
|
||||
""", StringComparison.InvariantCulture))
|
||||
{
|
||||
TAnnotation? providerResponse;
|
||||
try
|
||||
{
|
||||
// We know that the line starts with "data: ". Hence, we can
|
||||
// skip the first 6 characters to get the JSON data after that.
|
||||
var jsonData = line[6..];
|
||||
|
||||
// Deserialize the JSON data:
|
||||
providerResponse = JsonSerializer.Deserialize<TAnnotation>(jsonData, JSON_SERIALIZER_OPTIONS);
|
||||
|
||||
|
||||
@ -64,6 +64,12 @@ public sealed record Confidence
|
||||
Level = ConfidenceLevel.MEDIUM,
|
||||
Description = TB("The provider is located in the EU and is subject to the **GDPR** (General Data Protection Regulation). Additionally, the provider states that **your data is not used for training**."),
|
||||
};
|
||||
|
||||
public static readonly Confidence GDPR_EXPERIMENTAL_OPEN_SOURCE = new()
|
||||
{
|
||||
Level = ConfidenceLevel.MEDIUM,
|
||||
Description = TB("The provider operates its service in the EU and is subject to the **GDPR** (General Data Protection Regulation). It provides access to **open source models**. However, the service is currently **experimental**, and performance and availability are not guaranteed. We have no provider-specific information about whether submitted data is used for training."),
|
||||
};
|
||||
|
||||
public static readonly Confidence SELF_HOSTED = new()
|
||||
{
|
||||
|
||||
93
app/MindWork AI Studio/Provider/Hetzner/ProviderHetzner.cs
Normal file
93
app/MindWork AI Studio/Provider/Hetzner/ProviderHetzner.cs
Normal file
@ -0,0 +1,93 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
|
||||
using AIStudio.Chat;
|
||||
using AIStudio.Provider.OpenAI;
|
||||
using AIStudio.Settings;
|
||||
|
||||
namespace AIStudio.Provider.Hetzner;
|
||||
|
||||
public sealed class ProviderHetzner() : BaseProvider(LLMProviders.HETZNER, new Uri("https://inference.hetzner.com/api/v1/"), ExternalHttpTrustPolicy.SYSTEM_TRUST_ONLY, LOGGER)
|
||||
{
|
||||
private static readonly ILogger<ProviderHetzner> LOGGER = Program.LOGGER_FACTORY.CreateLogger<ProviderHetzner>();
|
||||
|
||||
#region Implementation of IProvider
|
||||
|
||||
/// <inheritdoc />
|
||||
public override string Id => LLMProviders.HETZNER.ToSecretId();
|
||||
|
||||
/// <inheritdoc />
|
||||
public override string InstanceName { get; set; } = "Hetzner (Experimental)";
|
||||
|
||||
/// <inheritdoc />
|
||||
public override bool HasModelLoadingCapability => true;
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async IAsyncEnumerable<ContentStreamChunk> StreamChatCompletion(Model chatModel, ChatThread chatThread, SettingsManager settingsManager, [EnumeratorCancellation] CancellationToken token = default)
|
||||
{
|
||||
await foreach (var content in this.StreamOpenAICompatibleChatCompletion<ChatCompletionAPIRequest, ChatCompletionDeltaStreamLine, NoChatCompletionAnnotationStreamLine>(
|
||||
"Hetzner",
|
||||
chatModel,
|
||||
chatThread,
|
||||
settingsManager,
|
||||
async (systemPrompt, apiParameters) =>
|
||||
{
|
||||
var messages = await chatThread.Blocks.BuildMessagesUsingNestedImageUrlAsync(this.Provider, chatModel);
|
||||
|
||||
return new ChatCompletionAPIRequest
|
||||
{
|
||||
Model = chatModel.Id,
|
||||
Messages = [systemPrompt, ..messages],
|
||||
Stream = true,
|
||||
AdditionalApiParameters = apiParameters
|
||||
};
|
||||
},
|
||||
token: token))
|
||||
yield return content;
|
||||
}
|
||||
|
||||
#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously
|
||||
/// <inheritdoc />
|
||||
public override async IAsyncEnumerable<ImageURL> StreamImageCompletion(Model imageModel, string promptPositive, string promptNegative = FilterOperator.String.Empty, ImageURL referenceImageURL = default, [EnumeratorCancellation] CancellationToken token = default)
|
||||
{
|
||||
yield break;
|
||||
}
|
||||
#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<TranscriptionResult> TranscribeAudioAsync(Model transcriptionModel, string audioFilePath, SettingsManager settingsManager, CancellationToken token = default)
|
||||
{
|
||||
return Task.FromResult(TranscriptionResult.Failure());
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<IReadOnlyList<IReadOnlyList<float>>> EmbedTextAsync(Model embeddingModel, SettingsManager settingsManager, CancellationToken token = default, params List<string> texts)
|
||||
{
|
||||
return Task.FromResult<IReadOnlyList<IReadOnlyList<float>>>([]);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<ModelLoadResult> GetTextModels(string? apiKeyProvisional = null, CancellationToken token = default)
|
||||
{
|
||||
return this.LoadModelsResponse<ModelsResponse>(SecretStoreType.LLM_PROVIDER, "models", modelResponse => modelResponse.Data, token, apiKeyProvisional);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<ModelLoadResult> GetImageModels(string? apiKeyProvisional = null, CancellationToken token = default)
|
||||
{
|
||||
return Task.FromResult(ModelLoadResult.FromModels([]));
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<ModelLoadResult> GetEmbeddingModels(string? apiKeyProvisional = null, CancellationToken token = default)
|
||||
{
|
||||
return Task.FromResult(ModelLoadResult.FromModels([]));
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<ModelLoadResult> GetTranscriptionModels(string? apiKeyProvisional = null, CancellationToken token = default)
|
||||
{
|
||||
return Task.FromResult(ModelLoadResult.FromModels([]));
|
||||
}
|
||||
|
||||
#endregion
|
||||
}
|
||||
@ -16,6 +16,7 @@ public enum LLMProviders
|
||||
ALIBABA_CLOUD = 12,
|
||||
PERPLEXITY = 14,
|
||||
OPEN_ROUTER = 15,
|
||||
HETZNER = 16,
|
||||
|
||||
FIREWORKS = 5,
|
||||
GROQ = 6,
|
||||
|
||||
@ -6,6 +6,7 @@ using AIStudio.Provider.Google;
|
||||
using AIStudio.Provider.Groq;
|
||||
using AIStudio.Provider.GWDG;
|
||||
using AIStudio.Provider.Helmholtz;
|
||||
using AIStudio.Provider.Hetzner;
|
||||
using AIStudio.Provider.HuggingFace;
|
||||
using AIStudio.Provider.Mistral;
|
||||
using AIStudio.Provider.OpenAI;
|
||||
@ -56,6 +57,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => "Alibaba Cloud",
|
||||
LLMProviders.PERPLEXITY => "Perplexity",
|
||||
LLMProviders.OPEN_ROUTER => "OpenRouter",
|
||||
LLMProviders.HETZNER => "Hetzner (Experimental)",
|
||||
|
||||
LLMProviders.GROQ => "Groq",
|
||||
LLMProviders.FIREWORKS => "Fireworks.ai",
|
||||
@ -91,6 +93,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => "Alibaba Cloud",
|
||||
LLMProviders.PERPLEXITY => "Perplexity",
|
||||
LLMProviders.OPEN_ROUTER => "OpenRouter",
|
||||
LLMProviders.HETZNER => "Hetzner",
|
||||
|
||||
LLMProviders.GROQ => "Groq",
|
||||
LLMProviders.FIREWORKS => "Fireworks.ai",
|
||||
@ -144,6 +147,12 @@ public static class LLMProvidersExtensions
|
||||
|
||||
LLMProviders.OPEN_ROUTER => Confidence.USA_HUB.WithRegion("America, U.S.").WithSources("https://openrouter.ai/privacy", "https://openrouter.ai/terms").WithLevel(settingsManager.GetConfiguredConfidenceLevel(llmProvider)),
|
||||
|
||||
LLMProviders.HETZNER => Confidence.GDPR_EXPERIMENTAL_OPEN_SOURCE.WithRegion("Europe, Germany").WithSources(
|
||||
"https://experiments.hetzner.com/docs/inference",
|
||||
"https://www.hetzner.com/legal/privacy-policy/",
|
||||
"https://www.hetzner.com/legal/terms-and-conditions/"
|
||||
).WithLevel(settingsManager.GetConfiguredConfidenceLevel(llmProvider)),
|
||||
|
||||
LLMProviders.SELF_HOSTED => Confidence.SELF_HOSTED.WithLevel(settingsManager.GetConfiguredConfidenceLevel(llmProvider)),
|
||||
|
||||
LLMProviders.HELMHOLTZ => Confidence.GDPR_NO_TRAINING.WithRegion("Europe, Germany").WithSources("https://helmholtz.cloud/services/?serviceID=d7d5c597-a2f6-4bd1-b71e-4d6499d98570").WithLevel(settingsManager.GetConfiguredConfidenceLevel(llmProvider)),
|
||||
@ -180,6 +189,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.HUGGINGFACE => false,
|
||||
LLMProviders.PERPLEXITY => false,
|
||||
LLMProviders.OPEN_ROUTER => true,
|
||||
LLMProviders.HETZNER => false,
|
||||
|
||||
//
|
||||
// Self-hosted providers are treated as a special case anyway.
|
||||
@ -209,6 +219,7 @@ public static class LLMProvidersExtensions
|
||||
// Providers that do not support transcription:
|
||||
//
|
||||
LLMProviders.OPEN_ROUTER => false,
|
||||
LLMProviders.HETZNER => false,
|
||||
LLMProviders.GROQ => false,
|
||||
LLMProviders.ANTHROPIC => false,
|
||||
LLMProviders.X => false,
|
||||
@ -271,6 +282,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => new ProviderAlibabaCloud { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
LLMProviders.PERPLEXITY => new ProviderPerplexity { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
LLMProviders.OPEN_ROUTER => new ProviderOpenRouter { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
LLMProviders.HETZNER => new ProviderHetzner { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
|
||||
LLMProviders.GROQ => new ProviderGroq { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
LLMProviders.FIREWORKS => new ProviderFireworks { InstanceName = instanceName, ConfiguredProviderId = configuredProviderId, AdditionalJsonApiParameters = expertProviderApiParameter, IsEnterpriseConfiguration = isEnterpriseConfiguration },
|
||||
@ -302,6 +314,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => "https://account.alibabacloud.com/register/intl_register.htm",
|
||||
LLMProviders.PERPLEXITY => "https://www.perplexity.ai/account/api",
|
||||
LLMProviders.OPEN_ROUTER => "https://openrouter.ai/keys",
|
||||
LLMProviders.HETZNER => "https://experiments.hetzner.com",
|
||||
|
||||
LLMProviders.GROQ => "https://console.groq.com/",
|
||||
LLMProviders.FIREWORKS => "https://fireworks.ai/login",
|
||||
@ -327,6 +340,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.PERPLEXITY => "https://www.perplexity.ai/account/api/",
|
||||
LLMProviders.OPEN_ROUTER => "https://openrouter.ai/activity",
|
||||
LLMProviders.HUGGINGFACE => "https://huggingface.co/settings/billing",
|
||||
LLMProviders.HETZNER => "https://experiments.hetzner.com",
|
||||
|
||||
_ => string.Empty,
|
||||
};
|
||||
@ -345,6 +359,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.PERPLEXITY => true,
|
||||
LLMProviders.OPEN_ROUTER => true,
|
||||
LLMProviders.HUGGINGFACE => true,
|
||||
LLMProviders.HETZNER => true,
|
||||
|
||||
_ => false,
|
||||
};
|
||||
@ -422,6 +437,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => true,
|
||||
LLMProviders.PERPLEXITY => true,
|
||||
LLMProviders.OPEN_ROUTER => true,
|
||||
LLMProviders.HETZNER => true,
|
||||
|
||||
LLMProviders.GROQ => true,
|
||||
LLMProviders.FIREWORKS => true,
|
||||
@ -445,6 +461,7 @@ public static class LLMProvidersExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => true,
|
||||
LLMProviders.PERPLEXITY => true,
|
||||
LLMProviders.OPEN_ROUTER => true,
|
||||
LLMProviders.HETZNER => true,
|
||||
|
||||
LLMProviders.GROQ => true,
|
||||
LLMProviders.FIREWORKS => true,
|
||||
|
||||
@ -76,6 +76,16 @@ public static partial class ProviderExtensions
|
||||
//
|
||||
if (modelName.IndexOf("deepseek") is not -1)
|
||||
{
|
||||
if ((modelName.IndexOf("deepseek-v4-flash") is not -1 ||
|
||||
modelName.IndexOf("deepseek-v4-pro") is not -1) &&
|
||||
modelName.IndexOf("-base") is -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT, Capability.TEXT_OUTPUT,
|
||||
Capability.REASONING_BY_DEFAULT, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
if(modelName.IndexOf("deepseek-r1") is not -1 ||
|
||||
modelName.IndexOf("deepseek r1") is not -1)
|
||||
return [
|
||||
@ -101,6 +111,16 @@ public static partial class ProviderExtensions
|
||||
Capability.ALWAYS_REASONING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
// Check for the open-weight Qwen 3.8 checkpoint:
|
||||
if(modelName.IndexOf("qwen3.8-2.4t-a95b") is not -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT, Capability.TEXT_OUTPUT,
|
||||
|
||||
Capability.ALWAYS_REASONING, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
// Check for Qwen 3.5:
|
||||
if(modelName.IndexOf("qwen3.5") is not -1)
|
||||
@ -117,11 +137,10 @@ public static partial class ProviderExtensions
|
||||
if(modelName.IndexOf("qwen3.6") is not -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT, Capability.VIDEO_INPUT,
|
||||
Capability.MULTIPLE_IMAGE_INPUT,
|
||||
Capability.TEXT_INPUT, Capability.MULTIPLE_IMAGE_INPUT,
|
||||
Capability.TEXT_OUTPUT,
|
||||
|
||||
Capability.OPTIONAL_REASONING, Capability.FUNCTION_CALLING,
|
||||
Capability.REASONING_BY_DEFAULT, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
@ -139,6 +158,29 @@ public static partial class ProviderExtensions
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
}
|
||||
|
||||
//
|
||||
// Moonshot AI / Kimi models:
|
||||
//
|
||||
if (modelName.IndexOf("kimi-k3") is not -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT, Capability.MULTIPLE_IMAGE_INPUT, Capability.VIDEO_INPUT,
|
||||
Capability.TEXT_OUTPUT,
|
||||
|
||||
Capability.ALWAYS_REASONING, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
if (modelName.IndexOf("kimi-k2.7-code") is not -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT, Capability.MULTIPLE_IMAGE_INPUT,
|
||||
Capability.TEXT_OUTPUT,
|
||||
|
||||
Capability.ALWAYS_REASONING, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
//
|
||||
// Mistral models:
|
||||
@ -335,6 +377,16 @@ public static partial class ProviderExtensions
|
||||
//
|
||||
if (modelName.IndexOf("glm") is not -1)
|
||||
{
|
||||
if (modelName.IndexOf("glm-5.2") is not -1)
|
||||
return
|
||||
[
|
||||
Capability.TEXT_INPUT,
|
||||
Capability.TEXT_OUTPUT,
|
||||
|
||||
Capability.REASONING_BY_DEFAULT, Capability.FUNCTION_CALLING,
|
||||
Capability.CHAT_COMPLETION_API,
|
||||
];
|
||||
|
||||
if(modelName.IndexOf("v") is not -1)
|
||||
return
|
||||
[
|
||||
|
||||
@ -90,6 +90,7 @@ public static partial class ProviderExtensions
|
||||
GetQwenReasoningState(parameters)),
|
||||
|
||||
LLMProviders.OPEN_ROUTER or
|
||||
LLMProviders.HETZNER or
|
||||
LLMProviders.X or
|
||||
LLMProviders.DEEP_SEEK or
|
||||
LLMProviders.GROQ or
|
||||
|
||||
@ -54,6 +54,7 @@ public static partial class ProviderExtensions
|
||||
LLMProviders.ALIBABA_CLOUD => GetModelCapabilitiesAlibaba(model),
|
||||
LLMProviders.PERPLEXITY => GetModelCapabilitiesPerplexity(model),
|
||||
LLMProviders.OPEN_ROUTER => GetModelCapabilitiesOpenRouter(model),
|
||||
LLMProviders.HETZNER => GetModelCapabilitiesOpenSource(model),
|
||||
|
||||
LLMProviders.GROQ => GetModelCapabilitiesOpenSource(model),
|
||||
LLMProviders.FIREWORKS => GetModelCapabilitiesOpenSource(model),
|
||||
|
||||
@ -599,8 +599,9 @@ public sealed class SettingsManager
|
||||
{
|
||||
LLMProviders.SELF_HOSTED => ConfidenceLevel.HIGH,
|
||||
LLMProviders.DEEP_SEEK => ConfidenceLevel.LOW,
|
||||
LLMProviders.ALIBABA_CLOUD => ConfidenceLevel.LOW,
|
||||
|
||||
_ => ConfidenceLevel.MEDIUM,
|
||||
_ => ConfidenceLevel.MEDIUM,
|
||||
};
|
||||
|
||||
case ConfidenceSchemes.TRUST_USA:
|
||||
@ -610,7 +611,9 @@ public sealed class SettingsManager
|
||||
LLMProviders.MISTRAL => ConfidenceLevel.LOW,
|
||||
LLMProviders.HELMHOLTZ => ConfidenceLevel.LOW,
|
||||
LLMProviders.GWDG => ConfidenceLevel.LOW,
|
||||
LLMProviders.HETZNER => ConfidenceLevel.LOW,
|
||||
LLMProviders.DEEP_SEEK => ConfidenceLevel.LOW,
|
||||
LLMProviders.ALIBABA_CLOUD => ConfidenceLevel.LOW,
|
||||
|
||||
_ => ConfidenceLevel.MEDIUM,
|
||||
};
|
||||
@ -622,6 +625,7 @@ public sealed class SettingsManager
|
||||
LLMProviders.MISTRAL => ConfidenceLevel.MEDIUM,
|
||||
LLMProviders.HELMHOLTZ => ConfidenceLevel.MEDIUM,
|
||||
LLMProviders.GWDG => ConfidenceLevel.MEDIUM,
|
||||
LLMProviders.HETZNER => ConfidenceLevel.MEDIUM,
|
||||
|
||||
_ => ConfidenceLevel.LOW,
|
||||
};
|
||||
@ -631,6 +635,7 @@ public sealed class SettingsManager
|
||||
{
|
||||
LLMProviders.SELF_HOSTED => ConfidenceLevel.HIGH,
|
||||
LLMProviders.DEEP_SEEK => ConfidenceLevel.MEDIUM,
|
||||
LLMProviders.ALIBABA_CLOUD => ConfidenceLevel.MEDIUM,
|
||||
|
||||
_ => ConfidenceLevel.LOW,
|
||||
};
|
||||
|
||||
@ -1,4 +1,6 @@
|
||||
# v26.8.1, build 251 (2026-08-xx xx:xx UTC)
|
||||
- Added Hetzner's experimental inference API as an LLM provider. It runs open-source models in the EU and supports text and image chats through its OpenAI-compatible API.
|
||||
- Added support for the new open source models DeepSeek V4 Flash and Pro, GLM 5.2, Kimi K2.7 Code and K3, as well as Qwen 3.6 and 3.8.
|
||||
- Added a prototype Visual Briefing Assistant that turns documents, data, images, audio, and video into self-contained interactive HTML briefings. When you want to test it, you have to enable this preview feature in your app settings.
|
||||
- Added organization-configurable defaults and visibility controls for the Visual Briefing Assistant.
|
||||
- Added a share button for assistants, configurations, and language plugins. It uses the native share dialog on Windows and macOS. For Linux, we added an export option, which stores the plugin archive at a location of your choice. When you work on a translation for a new language, you can now hand your current state to testers or to us with one click.
|
||||
@ -6,7 +8,7 @@
|
||||
- Added a delete button for assistants, configurations, and language plugins you installed or placed yourself. Until now, such a plugin could only be removed from the data directory by hand, which was especially painful for configurations because they have no on/off switch. Before deleting a configuration, AI Studio lists what disappears with it, such as providers, data sources, and settings that return to their default. When you delete the language plugin you had chosen, AI Studio returns to choosing your language automatically. Plugins shipped with AI Studio and plugins deployed by your IT department cannot be deleted.
|
||||
- Added options for organizations to disable importing, sharing, and exporting plugins, with a separate option for configuration plugins. Organizations can now let people import assistants while keeping configurations to their IT department.
|
||||
- Added a priority for configuration plugins. Organizations that deploy several configurations can now decide which one wins: a configuration with a higher priority overrides the settings and providers of a lower one. This allows a company-wide base configuration that each department refines for itself.
|
||||
- Added the Batch Processing Assistant: process all documents of a folder in one run. Each document is sent to the AI along with your instructions - either a free prompt, one of your document analysis policies, or instructions you import from a file. You choose whether each answer is stored as its own Markdown file or whether all answers are collected in one CSV results table, which you can name yourself. Every run writes a log that lists each document with its processing time, the model, the status, and the reason for any error. When you start another run on the same output folder, we ask whether you want to continue it: documents that failed or are missing from the log are processed again, which is helpful after a crash or when documents exceeded the context window of your model. A single failing document never stops the run, and you can cancel at any time. The assistant was contributed by Jan Erler (`j-erler`) and marks his first contribution to AI Studio. Thank you, Jan, for this wonderful and useful contribution.
|
||||
- Added the Batch Processing Assistant: process all documents of a folder in one run. Each document is sent to the AI along with your instructions - either a free prompt, one of your document analysis policies, or instructions you import from a file. You choose whether each answer is stored as its own Markdown file or whether all answers are collected in one CSV results table, which you can name yourself. Every run writes a log that lists each document with its processing time, the model, the status, and the reason for any error. When you start another run on the same output folder, we ask whether you want to continue it: documents that failed or are missing from the log are processed again, which is helpful after a crash or when documents exceeded the context window of your model. A single failing document never stops the run, and you can cancel at any time. The assistant was contributed by Jens Erler (`j-erler`) and marks his first contribution to AI Studio. Thank you, Jan, for this wonderful and useful contribution.
|
||||
- Added a way for IT departments to try out a configuration before rolling it out. A configuration placed in the new `.config-tests` directory below the plugins directory acts like one your organization deployed, including the approval of assistant plugins, so a test shows exactly what colleagues will see later. No configuration server is needed for this. AI Studio empties that directory every time it starts, so a test configuration is valid for one session, and the information page reports it while it is active. The Enterprise IT documentation describes the whole procedure.
|
||||
- Added CSV and TSV files to the file types you can attach. AI Studio was already able to read them, but they could not be selected.
|
||||
- Improved how your organization's configuration behaves when a configuration plugin is present but cannot be loaded, e.g. because of an error in the plugin. Such a plugin still manages your app, so its settings, providers, data sources, profiles, and chat templates now stay in place instead of being removed.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user