mirror of
https://github.com/MindWorkAI/AI-Studio.git
synced 2026-09-19 12:03:38 +00:00
Show the details of both RAG databases on the information page (#980)
Some checks are pending
Build and Release / Determine run mode (push) Waiting to run
Build and Release / Read metadata (push) Blocked by required conditions
Build and Release / Sync Flatpak repo (push) Blocked by required conditions
Build and Release / Collect Flatpak artifacts (push) Blocked by required conditions
Build and Release / Verify (push) Waiting to run
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-apple-darwin, osx-arm64, macos-latest, aarch64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-pc-windows-msvc.exe, win-arm64, windows-latest, aarch64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-unknown-linux-gnu, linux-arm64, ubuntu-22.04-arm, aarch64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-apple-darwin, osx-x64, macos-latest, x86_64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-pc-windows-msvc.exe, win-x64, windows-latest, x86_64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-unknown-linux-gnu, linux-x64, ubuntu-22.04, x86_64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Prepare & create release (push) Blocked by required conditions
Build and Release / Publish release (push) Blocked by required conditions
Some checks are pending
Build and Release / Determine run mode (push) Waiting to run
Build and Release / Read metadata (push) Blocked by required conditions
Build and Release / Sync Flatpak repo (push) Blocked by required conditions
Build and Release / Collect Flatpak artifacts (push) Blocked by required conditions
Build and Release / Verify (push) Waiting to run
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-apple-darwin, osx-arm64, macos-latest, aarch64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-pc-windows-msvc.exe, win-arm64, windows-latest, aarch64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-aarch64-unknown-linux-gnu, linux-arm64, ubuntu-22.04-arm, aarch64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-apple-darwin, osx-x64, macos-latest, x86_64-apple-darwin, dmg,app,updater, dmg) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-pc-windows-msvc.exe, win-x64, windows-latest, x86_64-pc-windows-msvc, nsis,updater, nsis) (push) Blocked by required conditions
Build and Release / Build app (${{ matrix.dotnet_runtime }}) (-x86_64-unknown-linux-gnu, linux-x64, ubuntu-22.04, x86_64-unknown-linux-gnu, appimage,updater, appimage) (push) Blocked by required conditions
Build and Release / Prepare & create release (push) Blocked by required conditions
Build and Release / Publish release (push) Blocked by required conditions
This commit is contained in:
parent
b2f46a5611
commit
557d0b1409
40
AGENTS.md
40
AGENTS.md
@ -214,9 +214,45 @@ RAG integration is currently in development (preview feature). Architecture:
|
||||
- **Data Sources** - Local files and external data via ERI servers
|
||||
- **Agents** - AI agents select data sources and validate retrieval quality
|
||||
- **Embedding providers** - Support for various embedding models
|
||||
- **Vector database** - Planned integration with Qdrant for vector storage
|
||||
- **Vector database** - Qdrant Edge, embedded in the Rust runtime; see "Databases" below
|
||||
- **Index database** - SQLite, holding the file fingerprints and the chunk texts for full-text search; see "Databases" below
|
||||
- **File processing** - Extracts text from PDF, DOCX, XLSX via Rust runtime
|
||||
|
||||
## Databases
|
||||
|
||||
Local RAG runs on two databases, addressed through `DatabaseRole`:
|
||||
|
||||
- **`VECTOR_STORE`** — Qdrant Edge through the `qdrant-edge` crate, running **in-process inside the
|
||||
Rust runtime**. There is no sidecar process, no port 6333 and no Qdrant API key; .NET reaches it
|
||||
over the internal runtime API (`/system/qdrant-edge/*`, see `runtime/src/qdrant_edge_database.rs`),
|
||||
secured by the same TLS and API token as every other runtime call. One store per data source,
|
||||
named `rag_<data source guid>`, holding a single named vector `embedding` per point.
|
||||
- **`INDEX_STORE`** — SQLite at `<data directory>/databases/sqlite/rag-index.sqlite3`, reached
|
||||
through EF Core. It holds the data sources, the file fingerprints, the chunk texts and an FTS5
|
||||
index over them, plus the files which permanently failed to index.
|
||||
|
||||
`DatabaseClientProvider` is the only way to a client. It caches one per role and guards each role
|
||||
with its own semaphore, so never construct a client yourself.
|
||||
|
||||
When working on these, keep in mind:
|
||||
|
||||
- **`GetDisplayInfo()` feeds the information page.** A new diagnostic value belongs in the client
|
||||
that knows it, not in `Pages/Information.razor.cs`. The page renders whatever label-value pairs it
|
||||
receives and stays free of per-database knowledge.
|
||||
- **Let every probe in `GetDisplayInfo()` catch its own failure.** When the method throws, the page
|
||||
replaces the *entire* block with the fallback client, so one unreadable value costs all the others
|
||||
as well.
|
||||
- **Raw SQL against SQLite goes through `context.Database.GetDbConnection()`**, not through
|
||||
`SqlQueryRaw<T>`: that one expects a column named `Value` and wraps the statement, so a `PRAGMA`
|
||||
never works with it.
|
||||
- **A new EF Core migration needs a `[DynamicDependency]`** in `IndexStoreSchemaMigrator`, because
|
||||
`PublishTrimmed` is on and the migration type would otherwise be trimmed away. The "Schema version"
|
||||
line on the information page shows the applied and pending counts, so a forgotten entry becomes
|
||||
visible there.
|
||||
- **Counts in the UI go through `long.CompactCount()` / `int.CompactCount()`** (`Tools/LongExtensions.cs`),
|
||||
which shortens anything above 999 to `1.46k` or `4.51M` and formats it with the culture of the
|
||||
active language plugin. Storage sizes are the exception: they keep using the byte formatters.
|
||||
|
||||
## Enterprise IT Support
|
||||
|
||||
AI Studio supports centralized configuration for enterprise environments:
|
||||
@ -245,6 +281,7 @@ Multi-level confidence scheme allows users to control which providers see which
|
||||
- keyring - OS keyring integration
|
||||
- pdfium-render - PDF text extraction
|
||||
- calamine - Excel file parsing
|
||||
- qdrant-edge - Embedded vector database
|
||||
|
||||
**.NET:**
|
||||
- Blazor Server - UI framework
|
||||
@ -252,6 +289,7 @@ Multi-level confidence scheme allows users to control which providers see which
|
||||
- LuaCSharp - Lua scripting engine
|
||||
- HtmlAgilityPack - HTML parsing
|
||||
- ReverseMarkdown - HTML to Markdown conversion
|
||||
- EF Core Sqlite + SQLitePCLRaw - the local RAG index
|
||||
|
||||
## Security
|
||||
|
||||
|
||||
@ -9271,9 +9271,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T103551060"] = "The configured ro
|
||||
-- Browse AI Studio's source code on GitHub — we welcome your contributions.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1107156991"] = "Browse AI Studio's source code on GitHub — we welcome your contributions."
|
||||
|
||||
-- Vector store version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1124039623"] = "Vector store version"
|
||||
|
||||
-- Qdrant Edge is an embedded vector database and vector similarity search engine. We use it to realize local RAG—retrieval-augmented generation—within AI Studio. Thanks for the effort and great work that has been and is being put into Qdrant.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1126023000"] = "Qdrant Edge is an embedded vector database and vector similarity search engine. We use it to realize local RAG—retrieval-augmented generation—within AI Studio. Thanks for the effort and great work that has been and is being put into Qdrant."
|
||||
|
||||
@ -9376,6 +9373,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1924365263"] = "This library is
|
||||
-- Encryption secret: is configured
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1931141322"] = "Encryption secret: is configured"
|
||||
|
||||
-- Index database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1949702956"] = "Index database"
|
||||
|
||||
-- The objc2 project provides access to Apple's Objective-C frameworks from Rust. On macOS, we use the libraries objc2, objc2-app-kit, and objc2-foundation to open the native macOS share sheet, e.g., when you share a plugin with others.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1985806792"] = "The objc2 project provides access to Apple's Objective-C frameworks from Rust. On macOS, we use the libraries objc2, objc2-app-kit, and objc2-foundation to open the native macOS share sheet, e.g., when you share a plugin with others."
|
||||
|
||||
@ -9553,9 +9553,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3019585985"] = "Test configurati
|
||||
-- External HTTPS custom root certificates are configured but not active.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3021325354"] = "External HTTPS custom root certificates are configured but not active."
|
||||
|
||||
-- Vector store
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3046399223"] = "Vector store"
|
||||
|
||||
-- Enterprise configuration ID:
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3092349641"] = "Enterprise configuration ID:"
|
||||
|
||||
@ -9682,6 +9679,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3970230163"] = "Copies the allow
|
||||
-- Symphonia is used for media container demuxing and audio decoding. The exact MPL-covered source is available from the repository linked and is identified in the offline notices bundled with AI Studio.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3971563979"] = "Symphonia is used for media container demuxing and audio decoding. The exact MPL-covered source is available from the repository linked and is identified in the offline notices bundled with AI Studio."
|
||||
|
||||
-- Vector database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3977811115"] = "Vector database"
|
||||
|
||||
-- Installed Pandoc version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3983971016"] = "Installed Pandoc version"
|
||||
|
||||
@ -10699,23 +10699,50 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T10
|
||||
-- Starting
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T1233211769"] = "Starting"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T2608177081"] = "unknown"
|
||||
|
||||
-- Unavailable
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T3662391977"] = "Unavailable"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T563161633"] = "Process architecture"
|
||||
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T6222351"] = "Status"
|
||||
|
||||
-- Database path
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1100578143"] = "Database path"
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T634426545"] = "Native library"
|
||||
|
||||
-- no migration applied
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1014567907"] = "no migration applied"
|
||||
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1230141403"] = "Storage size"
|
||||
|
||||
-- Full-text search (FTS5)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1396060361"] = "Full-text search (FTS5)"
|
||||
|
||||
-- Wrapper version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1469427584"] = "Wrapper version"
|
||||
|
||||
-- available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1727918744"] = "available"
|
||||
|
||||
-- Indexed files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2235289713"] = "Indexed files"
|
||||
|
||||
-- Search chunks
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2333737457"] = "Search chunks"
|
||||
-- {0} ({1} applied)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2286846332"] = "{0} ({1} applied)"
|
||||
|
||||
-- Journal mode
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2411639995"] = "Journal mode"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2608177081"] = "unknown"
|
||||
|
||||
-- Database tables
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3279078157"] = "Database tables"
|
||||
|
||||
-- Indexed data sources
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3524534748"] = "Indexed data sources"
|
||||
@ -10723,9 +10750,27 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTI
|
||||
-- Reported version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3556099842"] = "Reported version"
|
||||
|
||||
-- not available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3574465749"] = "not available"
|
||||
|
||||
-- Permanently skipped files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3965853089"] = "Permanently skipped files"
|
||||
|
||||
-- Schema version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T424290830"] = "Schema version"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T563161633"] = "Process architecture"
|
||||
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T634426545"] = "Native library"
|
||||
|
||||
-- System architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T818259255"] = "System architecture"
|
||||
|
||||
-- {0} ({1} applied, {2} pending)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T983802795"] = "{0} ({1} applied, {2} pending)"
|
||||
|
||||
-- Reason
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::NODATABASECLIENT::T1093747001"] = "Reason"
|
||||
|
||||
@ -10750,6 +10795,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::NOVECTORSTORECLIENT::T
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T1230141403"] = "Storage size"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2608177081"] = "unknown"
|
||||
|
||||
-- Number of vector stores
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2785004838"] = "Number of vector stores"
|
||||
|
||||
@ -10759,6 +10807,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEM
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T6222351"] = "Status"
|
||||
|
||||
-- Stored vectors
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T701689894"] = "Stored vectors"
|
||||
|
||||
-- Qdrant Edge is not available.
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T744445696"] = "Qdrant Edge is not available."
|
||||
|
||||
|
||||
@ -22,25 +22,40 @@
|
||||
<MudListItem T="string" Icon="@Icons.Material.Outlined.Build" Text="@this.VersionRust"/>
|
||||
<MudListItem T="string" Icon="@Icons.Material.Outlined.Storage">
|
||||
<MudText Typo="Typo.body1">
|
||||
@this.VersionVectorStore
|
||||
@this.DatabaseHeaderText(this.vectorStoreSection)
|
||||
</MudText>
|
||||
<MudCollapse Expanded="@this.showVectorStoreDetails">
|
||||
<MudCollapse Expanded="@this.vectorStoreSection.ShowDetails">
|
||||
<MudText Typo="Typo.body1" Class="mt-2 mb-2">
|
||||
@foreach (var item in this.vectorStoreDisplayInfo)
|
||||
@foreach (var item in this.BuildDatabaseInfoItems(this.vectorStoreSection))
|
||||
{
|
||||
<div style="display: flex; align-items: center; gap: 8px;">
|
||||
<MudIcon Icon="@Icons.Material.Filled.ArrowRightAlt"/>
|
||||
<span>@item.Label: @item.Value</span>
|
||||
<MudCopyClipboardButton TooltipMessage="@(T("Copies the following to the clipboard")+": "+item.Value)" StringContent=@item.Value/>
|
||||
</div>
|
||||
<ConfigInfoRow Item="@item"/>
|
||||
}
|
||||
</MudText>
|
||||
</MudCollapse>
|
||||
<MudButton StartIcon="@(this.showVectorStoreDetails ? Icons.Material.Filled.ExpandLess : Icons.Material.Filled.ExpandMore)"
|
||||
<MudButton StartIcon="@(this.vectorStoreSection.ShowDetails ? Icons.Material.Filled.ExpandLess : Icons.Material.Filled.ExpandMore)"
|
||||
Size="Size.Small"
|
||||
Variant="Variant.Text"
|
||||
OnClick="@this.ToggleVectorStoreDetails">
|
||||
@(this.showVectorStoreDetails ? T("Hide Details") : T("Show Details"))
|
||||
OnClick="@(() => this.ToggleDatabaseDetails(this.vectorStoreSection))">
|
||||
@(this.vectorStoreSection.ShowDetails ? T("Hide Details") : T("Show Details"))
|
||||
</MudButton>
|
||||
</MudListItem>
|
||||
<MudListItem T="string" Icon="@Icons.Material.Outlined.ManageSearch">
|
||||
<MudText Typo="Typo.body1">
|
||||
@this.DatabaseHeaderText(this.indexStoreSection)
|
||||
</MudText>
|
||||
<MudCollapse Expanded="@this.indexStoreSection.ShowDetails">
|
||||
<MudText Typo="Typo.body1" Class="mt-2 mb-2">
|
||||
@foreach (var item in this.BuildDatabaseInfoItems(this.indexStoreSection))
|
||||
{
|
||||
<ConfigInfoRow Item="@item"/>
|
||||
}
|
||||
</MudText>
|
||||
</MudCollapse>
|
||||
<MudButton StartIcon="@(this.indexStoreSection.ShowDetails ? Icons.Material.Filled.ExpandLess : Icons.Material.Filled.ExpandMore)"
|
||||
Size="Size.Small"
|
||||
Variant="Variant.Text"
|
||||
OnClick="@(() => this.ToggleDatabaseDetails(this.indexStoreSection))">
|
||||
@(this.indexStoreSection.ShowDetails ? T("Hide Details") : T("Show Details"))
|
||||
</MudButton>
|
||||
</MudListItem>
|
||||
<MudListItem T="string" Icon="@Icons.Material.Outlined.DocumentScanner" Text="@this.VersionPdfium"/>
|
||||
|
||||
@ -4,7 +4,6 @@ using AIStudio.Components;
|
||||
using AIStudio.Dialogs;
|
||||
using AIStudio.Settings.DataModel;
|
||||
using AIStudio.Tools.Databases;
|
||||
using AIStudio.Tools.Databases.VectorStore;
|
||||
using AIStudio.Tools.Metadata;
|
||||
using AIStudio.Tools.PluginSystem;
|
||||
using AIStudio.Tools.Rust;
|
||||
@ -98,20 +97,38 @@ public partial class Information : MSGComponentBase
|
||||
|
||||
private string VersionPdfium => $"{T("Used PDFium version")}: v{META_DATA_LIBRARIES.PdfiumVersion}";
|
||||
|
||||
private string VersionVectorStore
|
||||
/// <summary>
|
||||
/// Builds the headline of one database block.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The vector store names the version from the build metadata, which is what this page always
|
||||
/// showed. The index store names the one its client read from the running database instead:
|
||||
/// there, which SQLite build actually got loaded is the whole point of showing it.
|
||||
/// </remarks>
|
||||
private string DatabaseHeaderText(DatabaseSection section)
|
||||
{
|
||||
get
|
||||
var nameLabel = section.Role switch
|
||||
{
|
||||
if (this.vectorStore is null)
|
||||
return $"{T("Vector store")}: {T("checking availability")}";
|
||||
|
||||
return this.vectorStore.Status switch
|
||||
{
|
||||
DatabaseClientStatus.AVAILABLE => $"{T("Vector store version")}: {this.vectorStore.Name} v{META_DATA_VECTOR_STORE.VectorStoreVersion}",
|
||||
DatabaseClientStatus.STARTING => $"{T("Vector store")}: {this.vectorStore.Name} - {T("starting")}",
|
||||
_ => $"{T("Vector store")}: {this.vectorStore.Name} - {T("not available")}"
|
||||
DatabaseRole.VECTOR_STORE => T("Vector database"),
|
||||
_ => T("Index database"),
|
||||
};
|
||||
|
||||
if (section.Client is null)
|
||||
return $"{nameLabel}: {T("checking availability")}";
|
||||
|
||||
var version = section.Role switch
|
||||
{
|
||||
DatabaseRole.VECTOR_STORE => META_DATA_VECTOR_STORE.VectorStoreVersion,
|
||||
_ => section.Client.Version
|
||||
};
|
||||
|
||||
return section.Client.Status switch
|
||||
{
|
||||
DatabaseClientStatus.AVAILABLE when !string.IsNullOrWhiteSpace(version) => $"{nameLabel}: {section.Client.Name} v{version}",
|
||||
DatabaseClientStatus.AVAILABLE => $"{nameLabel}: {section.Client.Name}",
|
||||
DatabaseClientStatus.STARTING => $"{nameLabel}: {section.Client.Name} - {T("starting")}",
|
||||
_ => $"{nameLabel}: {section.Client.Name} - {T("not available")}"
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
private string versionPandoc = TB("Determine Pandoc version, please wait...");
|
||||
@ -121,7 +138,6 @@ public partial class Information : MSGComponentBase
|
||||
|
||||
private bool showEnterpriseConfigDetails;
|
||||
|
||||
private bool showVectorStoreDetails;
|
||||
private bool showExternalHttpCustomRootCertificateDetails;
|
||||
|
||||
private List<IAvailablePlugin> configPlugins = [];
|
||||
@ -141,10 +157,30 @@ public partial class Information : MSGComponentBase
|
||||
|
||||
private sealed record MandatoryInfoPanelData(string HeaderText, string PluginName, DataMandatoryInfo Info, DataMandatoryInfoAcceptance? Acceptance);
|
||||
|
||||
private sealed record VectorStoreDisplayInfo(string Label, string Value);
|
||||
private readonly List<VectorStoreDisplayInfo> vectorStoreDisplayInfo = new();
|
||||
private DatabaseClient? vectorStore;
|
||||
private CancellationTokenSource? vectorStoreRefreshCancellationTokenSource;
|
||||
private sealed record DatabaseDisplayInfo(string Label, string Value);
|
||||
|
||||
/// <summary>
|
||||
/// Everything one database block on this page needs to show itself.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Both blocks work the same way, so they share their state and their methods and differ only
|
||||
/// in their role. Whoever adds a third database adds one field here, not another set of methods.
|
||||
/// </remarks>
|
||||
private sealed class DatabaseSection(DatabaseRole role)
|
||||
{
|
||||
public DatabaseRole Role => role;
|
||||
|
||||
public DatabaseClient? Client { get; set; }
|
||||
|
||||
public bool ShowDetails { get; set; }
|
||||
|
||||
public List<DatabaseDisplayInfo> DisplayInfo { get; } = [];
|
||||
|
||||
public CancellationTokenSource? RefreshCancellationTokenSource { get; set; }
|
||||
}
|
||||
|
||||
private readonly DatabaseSection vectorStoreSection = new(DatabaseRole.VECTOR_STORE);
|
||||
private readonly DatabaseSection indexStoreSection = new(DatabaseRole.INDEX_STORE);
|
||||
|
||||
private bool HasAnyActiveEnvironment => this.enterpriseEnvironments.Any(e => e.IsActive);
|
||||
|
||||
@ -191,9 +227,16 @@ public partial class Information : MSGComponentBase
|
||||
this.updatePolicyMode = this.UpdatePolicy.CurrentMode;
|
||||
this.logPaths = await this.RustService.GetLogPaths();
|
||||
|
||||
await this.RefreshVectorStoreInfo(CancellationToken.None);
|
||||
if (this.vectorStore?.Status is DatabaseClientStatus.STARTING)
|
||||
this.StartShortVectorStoreRefreshLoop();
|
||||
// The index store goes first: the vector store asks it for the number of stored vectors,
|
||||
// and this way that client is already cached when it does.
|
||||
await this.RefreshDatabaseInfo(this.indexStoreSection, CancellationToken.None);
|
||||
await this.RefreshDatabaseInfo(this.vectorStoreSection, CancellationToken.None);
|
||||
|
||||
if (this.indexStoreSection.Client?.Status is DatabaseClientStatus.STARTING)
|
||||
this.StartShortDatabaseRefreshLoop(this.indexStoreSection);
|
||||
|
||||
if (this.vectorStoreSection.Client?.Status is DatabaseClientStatus.STARTING)
|
||||
this.StartShortDatabaseRefreshLoop(this.vectorStoreSection);
|
||||
|
||||
// Determine the Pandoc version may take some time, so we start it here
|
||||
// without waiting for the result:
|
||||
@ -301,22 +344,31 @@ public partial class Information : MSGComponentBase
|
||||
this.showExternalHttpCustomRootCertificateDetails = !this.showExternalHttpCustomRootCertificateDetails;
|
||||
}
|
||||
|
||||
private void ToggleVectorStoreDetails()
|
||||
private void ToggleDatabaseDetails(DatabaseSection section)
|
||||
{
|
||||
this.showVectorStoreDetails = !this.showVectorStoreDetails;
|
||||
section.ShowDetails = !section.ShowDetails;
|
||||
}
|
||||
|
||||
private async Task RefreshVectorStoreInfo(CancellationToken cancellationToken)
|
||||
private IReadOnlyList<ConfigInfoRowItem> BuildDatabaseInfoItems(DatabaseSection section) => section.DisplayInfo
|
||||
.Select((item, index) => new ConfigInfoRowItem(
|
||||
Icons.Material.Filled.ArrowRightAlt,
|
||||
$"{item.Label}: {item.Value}",
|
||||
item.Value,
|
||||
$"{T("Copies the following to the clipboard")}: {item.Value}",
|
||||
index == 0 ? string.Empty : "margin-top: 4px;"))
|
||||
.ToList();
|
||||
|
||||
private async Task RefreshDatabaseInfo(DatabaseSection section, CancellationToken cancellationToken)
|
||||
{
|
||||
var refreshedClient = await this.DatabaseClientProvider.RefreshClientAsync(DatabaseRole.VECTOR_STORE, cancellationToken);
|
||||
this.vectorStore = refreshedClient;
|
||||
this.vectorStoreDisplayInfo.Clear();
|
||||
var refreshedClient = await this.DatabaseClientProvider.RefreshClientAsync(section.Role, cancellationToken);
|
||||
section.Client = refreshedClient;
|
||||
section.DisplayInfo.Clear();
|
||||
|
||||
try
|
||||
{
|
||||
await foreach (var (label, value) in refreshedClient.GetDisplayInfo().WithCancellation(cancellationToken))
|
||||
{
|
||||
this.vectorStoreDisplayInfo.Add(new VectorStoreDisplayInfo(label, value));
|
||||
section.DisplayInfo.Add(new DatabaseDisplayInfo(label, value));
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
@ -325,20 +377,24 @@ public partial class Information : MSGComponentBase
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
this.vectorStore = new NoVectorStoreClient(refreshedClient.Name, e.Message, DatabaseClientStatus.STARTING);
|
||||
await foreach (var (label, value) in this.vectorStore.GetDisplayInfo().WithCancellation(cancellationToken))
|
||||
// Drop whatever came in before the failure: those lines would otherwise stand next to
|
||||
// the status and reason of the stand-in client and read like current values.
|
||||
section.DisplayInfo.Clear();
|
||||
|
||||
section.Client = DatabaseClientProvider.CreateUnavailableClient(section.Role, refreshedClient.Name, e.Message, DatabaseClientStatus.STARTING);
|
||||
await foreach (var (label, value) in section.Client.GetDisplayInfo().WithCancellation(cancellationToken))
|
||||
{
|
||||
this.vectorStoreDisplayInfo.Add(new VectorStoreDisplayInfo(label, value));
|
||||
section.DisplayInfo.Add(new DatabaseDisplayInfo(label, value));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void StartShortVectorStoreRefreshLoop()
|
||||
private void StartShortDatabaseRefreshLoop(DatabaseSection section)
|
||||
{
|
||||
this.vectorStoreRefreshCancellationTokenSource?.Cancel();
|
||||
this.vectorStoreRefreshCancellationTokenSource?.Dispose();
|
||||
this.vectorStoreRefreshCancellationTokenSource = new CancellationTokenSource();
|
||||
var cancellationToken = this.vectorStoreRefreshCancellationTokenSource.Token;
|
||||
section.RefreshCancellationTokenSource?.Cancel();
|
||||
section.RefreshCancellationTokenSource?.Dispose();
|
||||
section.RefreshCancellationTokenSource = new CancellationTokenSource();
|
||||
var cancellationToken = section.RefreshCancellationTokenSource.Token;
|
||||
|
||||
Task.Run(async () =>
|
||||
{
|
||||
@ -350,11 +406,11 @@ public partial class Information : MSGComponentBase
|
||||
await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
|
||||
await this.InvokeAsync(async () =>
|
||||
{
|
||||
await this.RefreshVectorStoreInfo(cancellationToken);
|
||||
await this.RefreshDatabaseInfo(section, cancellationToken);
|
||||
this.StateHasChanged();
|
||||
});
|
||||
|
||||
if (this.vectorStore?.Status is not DatabaseClientStatus.STARTING)
|
||||
if (section.Client?.Status is not DatabaseClientStatus.STARTING)
|
||||
return;
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
@ -366,7 +422,14 @@ public partial class Information : MSGComponentBase
|
||||
return;
|
||||
}
|
||||
}
|
||||
}, cancellationToken).Observe($"{nameof(Information)}: refreshing the vector store info");
|
||||
}, cancellationToken).Observe($"{nameof(Information)}: refreshing the {section.Role} info");
|
||||
}
|
||||
|
||||
private void CancelDatabaseRefreshLoop(DatabaseSection section)
|
||||
{
|
||||
section.RefreshCancellationTokenSource?.Cancel();
|
||||
section.RefreshCancellationTokenSource?.Dispose();
|
||||
section.RefreshCancellationTokenSource = null;
|
||||
}
|
||||
|
||||
private IAvailablePlugin? FindManagedConfigurationPlugin(Guid configurationId)
|
||||
@ -525,8 +588,8 @@ public partial class Information : MSGComponentBase
|
||||
|
||||
protected override void DisposeResources()
|
||||
{
|
||||
this.vectorStoreRefreshCancellationTokenSource?.Cancel();
|
||||
this.vectorStoreRefreshCancellationTokenSource?.Dispose();
|
||||
this.CancelDatabaseRefreshLoop(this.vectorStoreSection);
|
||||
this.CancelDatabaseRefreshLoop(this.indexStoreSection);
|
||||
base.DisposeResources();
|
||||
}
|
||||
|
||||
|
||||
@ -9111,7 +9111,7 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::EMBEDDINGS::T1455637941"] = "Der Speicherort d
|
||||
-- Open the settings
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::EMBEDDINGS::T1582896271"] = "Einstellungen öffnen"
|
||||
|
||||
-- Datei {0} von {1} wird indexiert.
|
||||
-- File {0} of {1} is being indexed.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::EMBEDDINGS::T1616414701"] = "Datei {0} von {1} wird indexiert."
|
||||
|
||||
-- Tried again during the next run
|
||||
@ -9273,9 +9273,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T103551060"] = "Die konfigurierte
|
||||
-- Browse AI Studio's source code on GitHub — we welcome your contributions.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1107156991"] = "Sehen Sie sich den Quellcode von AI Studio auf GitHub an – wir freuen uns über Ihre Beiträge."
|
||||
|
||||
-- Vector store version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1124039623"] = "Vektordatenbankversion"
|
||||
|
||||
-- Qdrant Edge is an embedded vector database and vector similarity search engine. We use it to realize local RAG—retrieval-augmented generation—within AI Studio. Thanks for the effort and great work that has been and is being put into Qdrant.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1126023000"] = "Qdrant Edge ist eine eingebettete Vektordatenbank und ein Vektoraehnlichkeitssuchmaschine. Wir nutzen sie, um lokal RAG – retrieval-augmented generation – innerhalb von AI Studio zu realisieren. Vielen Dank für die Anstrengungen und die großartige Arbeit, die in Qdrant investiert wurde und weiterhin investiert wird."
|
||||
|
||||
@ -9378,6 +9375,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1924365263"] = "Diese Bibliothek
|
||||
-- Encryption secret: is configured
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1931141322"] = "Geheimnis für die Verschlüsselung: ist konfiguriert"
|
||||
|
||||
-- Index database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1949702956"] = "Indexdatenbank"
|
||||
|
||||
-- The objc2 project provides access to Apple's Objective-C frameworks from Rust. On macOS, we use the libraries objc2, objc2-app-kit, and objc2-foundation to open the native macOS share sheet, e.g., when you share a plugin with others.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1985806792"] = "Das Projekt objc2 ermöglicht den Zugriff auf die Objective-C-Frameworks von Apple aus Rust. Unter macOS verwenden wir die Bibliotheken objc2, objc2-app-kit und objc2-foundation, um den nativen macOS-Teilen-Dialog zu öffnen, beispielsweise wenn Sie ein Plugin mit anderen teilen."
|
||||
|
||||
@ -9555,9 +9555,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3019585985"] = "Testkonfiguratio
|
||||
-- External HTTPS custom root certificates are configured but not active.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3021325354"] = "Externe benutzerdefinierte Stammzertifikate sind konfiguriert, aber nicht aktiv."
|
||||
|
||||
-- Vector store
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3046399223"] = "Vektordatenbank"
|
||||
|
||||
-- Enterprise configuration ID:
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3092349641"] = "Unternehmenskonfigurations-ID:"
|
||||
|
||||
@ -9684,6 +9681,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3970230163"] = "Kopiert die zul
|
||||
-- Symphonia is used for media container demuxing and audio decoding. The exact MPL-covered source is available from the repository linked and is identified in the offline notices bundled with AI Studio.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3971563979"] = "Symphonia wird zum Demultiplexen von Mediencontainern und zur Audiodekodierung verwendet. Der genaue, unter der MPL lizenzierte Quellcode ist im verlinkten Repository verfügbar und in den mit AI Studio gebündelten Offline-Hinweisen angegeben."
|
||||
|
||||
-- Vector database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3977811115"] = "Vektordatenbank"
|
||||
|
||||
-- Installed Pandoc version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3983971016"] = "Installierte Pandoc-Version"
|
||||
|
||||
@ -10701,23 +10701,50 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T10
|
||||
-- Starting
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T1233211769"] = "Starten"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T2608177081"] = "Unbekannt"
|
||||
|
||||
-- Unavailable
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T3662391977"] = "Nicht verfügbar"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T563161633"] = "Prozessarchitektur"
|
||||
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T6222351"] = "Status"
|
||||
|
||||
-- Database path
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1100578143"] = "Datenbankpfad"
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T634426545"] = "Native Bibliothek"
|
||||
|
||||
-- no migration applied
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1014567907"] = "Keine Migration angewendet"
|
||||
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1230141403"] = "Speichergröße"
|
||||
|
||||
-- Full-text search (FTS5)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1396060361"] = "Volltextsuche (FTS5)"
|
||||
|
||||
-- Wrapper version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1469427584"] = "Wrapper-Version"
|
||||
|
||||
-- available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1727918744"] = "Verfügbar"
|
||||
|
||||
-- Indexed files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2235289713"] = "Indexierte Dateien"
|
||||
|
||||
-- Search chunks
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2333737457"] = "Suche in Blöcken"
|
||||
-- {0} ({1} applied)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2286846332"] = "{0} ({1} angewendet)"
|
||||
|
||||
-- Journal mode
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2411639995"] = "Journalmodus"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2608177081"] = "Unbekannt"
|
||||
|
||||
-- Database tables
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3279078157"] = "Datenbanktabellen"
|
||||
|
||||
-- Indexed data sources
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3524534748"] = "Indexierte Datenquellen"
|
||||
@ -10725,9 +10752,27 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTI
|
||||
-- Reported version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3556099842"] = "Gemeldete Version"
|
||||
|
||||
-- not available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3574465749"] = "Nicht verfügbar"
|
||||
|
||||
-- Permanently skipped files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3965853089"] = "Dauerhaft übersprungene Dateien"
|
||||
|
||||
-- Schema version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T424290830"] = "Schemaversion"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T563161633"] = "Prozessarchitektur"
|
||||
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T634426545"] = "Native Bibliothek"
|
||||
|
||||
-- System architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T818259255"] = "Systemarchitektur"
|
||||
|
||||
-- {0} ({1} applied, {2} pending)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T983802795"] = "{0} ({1} angewendet, {2} ausstehend)"
|
||||
|
||||
-- Reason
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::NODATABASECLIENT::T1093747001"] = "Grund"
|
||||
|
||||
@ -10752,6 +10797,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::NOVECTORSTORECLIENT::T
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T1230141403"] = "Speichergröße"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2608177081"] = "Unbekannt"
|
||||
|
||||
-- Number of vector stores
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2785004838"] = "Anzahl der Vektordatenbanken"
|
||||
|
||||
@ -10761,6 +10809,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEM
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T6222351"] = "Status"
|
||||
|
||||
-- Stored vectors
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T701689894"] = "Gespeicherte Vektoren"
|
||||
|
||||
-- Qdrant Edge is not available.
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T744445696"] = "Qdrant Edge ist nicht verfügbar."
|
||||
|
||||
|
||||
@ -9273,9 +9273,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T103551060"] = "The configured ro
|
||||
-- Browse AI Studio's source code on GitHub — we welcome your contributions.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1107156991"] = "Browse AI Studio's source code on GitHub — we welcome your contributions."
|
||||
|
||||
-- Vector store version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1124039623"] = "Vector store version"
|
||||
|
||||
-- Qdrant Edge is an embedded vector database and vector similarity search engine. We use it to realize local RAG—retrieval-augmented generation—within AI Studio. Thanks for the effort and great work that has been and is being put into Qdrant.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1126023000"] = "Qdrant Edge is an embedded vector database and vector similarity search engine. We use it to realize local RAG—retrieval-augmented generation—within AI Studio. Thanks for the effort and great work that has been and is being put into Qdrant."
|
||||
|
||||
@ -9378,6 +9375,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1924365263"] = "This library is
|
||||
-- Encryption secret: is configured
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1931141322"] = "Encryption secret: is configured"
|
||||
|
||||
-- Index database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1949702956"] = "Index database"
|
||||
|
||||
-- The objc2 project provides access to Apple's Objective-C frameworks from Rust. On macOS, we use the libraries objc2, objc2-app-kit, and objc2-foundation to open the native macOS share sheet, e.g., when you share a plugin with others.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T1985806792"] = "The objc2 project provides access to Apple's Objective-C frameworks from Rust. On macOS, we use the libraries objc2, objc2-app-kit, and objc2-foundation to open the native macOS share sheet, e.g., when you share a plugin with others."
|
||||
|
||||
@ -9555,9 +9555,6 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3019585985"] = "Test configurati
|
||||
-- External HTTPS custom root certificates are configured but not active.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3021325354"] = "External HTTPS custom root certificates are configured but not active."
|
||||
|
||||
-- Vector store
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3046399223"] = "Vector store"
|
||||
|
||||
-- Enterprise configuration ID:
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3092349641"] = "Enterprise configuration ID:"
|
||||
|
||||
@ -9684,6 +9681,9 @@ UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3970230163"] = "Copies the allow
|
||||
-- Symphonia is used for media container demuxing and audio decoding. The exact MPL-covered source is available from the repository linked and is identified in the offline notices bundled with AI Studio.
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3971563979"] = "Symphonia is used for media container demuxing and audio decoding. The exact MPL-covered source is available from the repository linked and is identified in the offline notices bundled with AI Studio."
|
||||
|
||||
-- Vector database
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3977811115"] = "Vector database"
|
||||
|
||||
-- Installed Pandoc version
|
||||
UI_TEXT_CONTENT["AISTUDIO::PAGES::INFORMATION::T3983971016"] = "Installed Pandoc version"
|
||||
|
||||
@ -10701,23 +10701,50 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T10
|
||||
-- Starting
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T1233211769"] = "Starting"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T2608177081"] = "unknown"
|
||||
|
||||
-- Unavailable
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T3662391977"] = "Unavailable"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T563161633"] = "Process architecture"
|
||||
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T6222351"] = "Status"
|
||||
|
||||
-- Database path
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1100578143"] = "Database path"
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::NOINDEXSTORECLIENT::T634426545"] = "Native library"
|
||||
|
||||
-- no migration applied
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1014567907"] = "no migration applied"
|
||||
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1230141403"] = "Storage size"
|
||||
|
||||
-- Full-text search (FTS5)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1396060361"] = "Full-text search (FTS5)"
|
||||
|
||||
-- Wrapper version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1469427584"] = "Wrapper version"
|
||||
|
||||
-- available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T1727918744"] = "available"
|
||||
|
||||
-- Indexed files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2235289713"] = "Indexed files"
|
||||
|
||||
-- Search chunks
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2333737457"] = "Search chunks"
|
||||
-- {0} ({1} applied)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2286846332"] = "{0} ({1} applied)"
|
||||
|
||||
-- Journal mode
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2411639995"] = "Journal mode"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T2608177081"] = "unknown"
|
||||
|
||||
-- Database tables
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3279078157"] = "Database tables"
|
||||
|
||||
-- Indexed data sources
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3524534748"] = "Indexed data sources"
|
||||
@ -10725,9 +10752,27 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTI
|
||||
-- Reported version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3556099842"] = "Reported version"
|
||||
|
||||
-- not available
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3574465749"] = "not available"
|
||||
|
||||
-- Permanently skipped files
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T3965853089"] = "Permanently skipped files"
|
||||
|
||||
-- Schema version
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T424290830"] = "Schema version"
|
||||
|
||||
-- Process architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T563161633"] = "Process architecture"
|
||||
|
||||
-- Native library
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T634426545"] = "Native library"
|
||||
|
||||
-- System architecture
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T818259255"] = "System architecture"
|
||||
|
||||
-- {0} ({1} applied, {2} pending)
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::INDEXSTORE::SQLITEINDEXSTORECLIENTIMPLEMENTATION::T983802795"] = "{0} ({1} applied, {2} pending)"
|
||||
|
||||
-- Reason
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::NODATABASECLIENT::T1093747001"] = "Reason"
|
||||
|
||||
@ -10752,6 +10797,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::NOVECTORSTORECLIENT::T
|
||||
-- Storage size
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T1230141403"] = "Storage size"
|
||||
|
||||
-- unknown
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2608177081"] = "unknown"
|
||||
|
||||
-- Number of vector stores
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T2785004838"] = "Number of vector stores"
|
||||
|
||||
@ -10761,6 +10809,9 @@ UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEM
|
||||
-- Status
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T6222351"] = "Status"
|
||||
|
||||
-- Stored vectors
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T701689894"] = "Stored vectors"
|
||||
|
||||
-- Qdrant Edge is not available.
|
||||
UI_TEXT_CONTENT["AISTUDIO::TOOLS::DATABASES::VECTORSTORE::QDRANTEDGECLIENTIMPLEMENTATION::T744445696"] = "Qdrant Edge is not available."
|
||||
|
||||
|
||||
@ -8,11 +8,20 @@ public abstract class DatabaseClient(string name, string path)
|
||||
|
||||
public virtual DatabaseClientStatus Status => DatabaseClientStatus.AVAILABLE;
|
||||
|
||||
/// <summary>
|
||||
/// The version the running database reports about itself.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Empty when the client cannot tell. Callers which want to show a version in a headline read it
|
||||
/// from here instead of picking it out of the label-value pairs the display info yields.
|
||||
/// </remarks>
|
||||
public virtual string Version => string.Empty;
|
||||
|
||||
public bool IsAvailable => this.Status is DatabaseClientStatus.AVAILABLE;
|
||||
|
||||
private string Path => path;
|
||||
|
||||
private ILogger<DatabaseClient>? logger;
|
||||
protected ILogger<DatabaseClient>? Logger;
|
||||
|
||||
public abstract IAsyncEnumerable<(string Label, string Value)> GetDisplayInfo();
|
||||
|
||||
@ -20,13 +29,13 @@ public abstract class DatabaseClient(string name, string path)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(this.Path))
|
||||
{
|
||||
this.logger!.LogError($"Error: Database path '{this.Path}' cannot be null or empty.");
|
||||
this.Logger!.LogError($"Error: Database path '{this.Path}' cannot be null or empty.");
|
||||
return "0 B";
|
||||
}
|
||||
|
||||
if (!Directory.Exists(this.Path))
|
||||
{
|
||||
this.logger!.LogError($"Error: Database path '{this.Path}' does not exist.");
|
||||
this.Logger!.LogError($"Error: Database path '{this.Path}' does not exist.");
|
||||
return "0 B";
|
||||
}
|
||||
var files = Directory.EnumerateFiles(this.Path, "*", SearchOption.AllDirectories)
|
||||
@ -52,7 +61,7 @@ public abstract class DatabaseClient(string name, string path)
|
||||
|
||||
public void SetLogger(ILogger<DatabaseClient> logService)
|
||||
{
|
||||
this.logger = logService;
|
||||
this.Logger = logService;
|
||||
}
|
||||
|
||||
public abstract void Dispose();
|
||||
|
||||
@ -69,6 +69,25 @@ public sealed class DatabaseClientProvider(RustService rustService, ILoggerFacto
|
||||
client.Status);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Builds the client which stands in for a database role that cannot serve right now.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Callers outside this namespace get their stand-in from here instead of naming the concrete
|
||||
/// type themselves, so a new role does not have to be spelled out in every one of them.
|
||||
/// </remarks>
|
||||
/// <param name="databaseRole">The role the stand-in has to fill.</param>
|
||||
/// <param name="name">The name to show for the database.</param>
|
||||
/// <param name="reason">Why the database is not available.</param>
|
||||
/// <param name="status">Whether the database is starting or unavailable.</param>
|
||||
/// <returns>A client which answers every operation without a database behind it.</returns>
|
||||
public static DatabaseClient CreateUnavailableClient(DatabaseRole databaseRole, string name, string? reason, DatabaseClientStatus status) => databaseRole switch
|
||||
{
|
||||
DatabaseRole.VECTOR_STORE => new NoVectorStoreClient(name, reason, status),
|
||||
DatabaseRole.INDEX_STORE => new NoIndexStoreClient(name, reason, status),
|
||||
_ => new NoDatabaseClient(name, reason, status)
|
||||
};
|
||||
|
||||
private DatabaseClient CacheIfAvailable(DatabaseRole databaseRole, DatabaseClient client)
|
||||
{
|
||||
if (!client.IsAvailable)
|
||||
@ -104,7 +123,7 @@ public sealed class DatabaseClientProvider(RustService rustService, ILoggerFacto
|
||||
|
||||
private async Task<DatabaseClient> CreateClientAsync(DatabaseRole databaseRole, CancellationToken cancellationToken) => databaseRole switch
|
||||
{
|
||||
DatabaseRole.VECTOR_STORE => await QdrantEdgeClientImplementation.CreateAsync(rustService, this.logger, this.databaseClientLogger, cancellationToken),
|
||||
DatabaseRole.VECTOR_STORE => await QdrantEdgeClientImplementation.CreateAsync(rustService, this.GetIndexStoreAsync, this.logger, this.databaseClientLogger, cancellationToken),
|
||||
DatabaseRole.INDEX_STORE => await SqliteIndexStoreClientImplementation.CreateAsync(this.logger, this.databaseClientLogger, cancellationToken),
|
||||
_ => new NoDatabaseClient(databaseRole.ToString(), "The requested database role is not supported.")
|
||||
};
|
||||
|
||||
@ -40,4 +40,17 @@ public abstract class IndexStoreClient(string name, string path) : DatabaseClien
|
||||
public abstract Task<IReadOnlyList<IndexStoreSearchResult>> SearchChunksAsync(string dataSourceId, string query, int maxMatches, CancellationToken token);
|
||||
|
||||
public abstract Task DeleteDataSourceAsync(string dataSourceId, CancellationToken token);
|
||||
|
||||
/// <summary>
|
||||
/// Counts the search chunks the index holds across all data sources.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// One chunk is one vector: every chunk becomes exactly one point carrying the single named
|
||||
/// vector "embedding". The vector store reports its vector count from here, because counting
|
||||
/// the points in Qdrant Edge would have to load every shard first and would hold the global
|
||||
/// database mutex against ongoing inserts and searches while doing so.
|
||||
/// </remarks>
|
||||
/// <param name="token">The cancellation token.</param>
|
||||
/// <returns>The number of chunks, or null when the index cannot tell. Null and zero mean different things here.</returns>
|
||||
public abstract Task<long?> GetTotalChunkCountAsync(CancellationToken token);
|
||||
}
|
||||
|
||||
@ -20,9 +20,19 @@ public sealed class NoIndexStoreClient(string name, string? unavailableReason, D
|
||||
if (!string.IsNullOrWhiteSpace(unavailableReason))
|
||||
yield return (TB("Reason"), unavailableReason);
|
||||
|
||||
//
|
||||
// Say which native library this process bound to even though the database itself is out of
|
||||
// reach. When SQLite cannot be loaded on a platform at all, this client is exactly what the
|
||||
// user sees, so this is the one place where those details matter most.
|
||||
//
|
||||
yield return (TB("Native library"), OrUnknown(SqliteRuntimeInfo.GetNativeLibraryName()));
|
||||
yield return (TB("Process architecture"), OrUnknown(SqliteRuntimeInfo.GetProcessArchitecture()));
|
||||
|
||||
await Task.CompletedTask;
|
||||
}
|
||||
|
||||
private static string OrUnknown(string value) => string.IsNullOrWhiteSpace(value) ? TB("unknown") : value;
|
||||
|
||||
public override Task<DataSourceEmbeddingManifest> GetManifestAsync(string dataSourceId, CancellationToken token) => Task.FromResult(new DataSourceEmbeddingManifest());
|
||||
|
||||
public override Task<DataSourceIndexState?> GetDataSourceStateAsync(string dataSourceId, CancellationToken token) => Task.FromResult<DataSourceIndexState?>(null);
|
||||
@ -55,6 +65,8 @@ public sealed class NoIndexStoreClient(string name, string? unavailableReason, D
|
||||
|
||||
public override Task DeleteDataSourceAsync(string dataSourceId, CancellationToken token) => Task.CompletedTask;
|
||||
|
||||
public override Task<long?> GetTotalChunkCountAsync(CancellationToken token) => Task.FromResult<long?>(null);
|
||||
|
||||
public override void Dispose()
|
||||
{
|
||||
}
|
||||
|
||||
@ -1,3 +1,4 @@
|
||||
using System.Data;
|
||||
using System.Globalization;
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
@ -11,7 +12,7 @@ namespace AIStudio.Tools.Databases.IndexStore;
|
||||
|
||||
public sealed class SqliteIndexStoreClientImplementation(string name, string databasePath, string basePath, string version) : IndexStoreClient(name, basePath)
|
||||
{
|
||||
private const string DATABASE_NAME = "Local RAG Index";
|
||||
private const string DATABASE_NAME = "SQLite";
|
||||
private const string DATABASE_FILENAME = "rag-index.sqlite3";
|
||||
private const int MAX_FTS_QUERY_TERMS = 32;
|
||||
private const int CHUNK_UPSERT_BATCH_SIZE = 500;
|
||||
@ -25,6 +26,8 @@ public sealed class SqliteIndexStoreClientImplementation(string name, string dat
|
||||
|
||||
public override string CacheKey => $"{this.Name}:{this.databasePath}:{version}";
|
||||
|
||||
public override string Version => version;
|
||||
|
||||
public static async Task<DatabaseClient> CreateAsync(
|
||||
ILogger logger,
|
||||
ILogger<DatabaseClient> databaseClientLogger,
|
||||
@ -58,15 +61,33 @@ public sealed class SqliteIndexStoreClientImplementation(string name, string dat
|
||||
|
||||
public override async IAsyncEnumerable<(string Label, string Value)> GetDisplayInfo()
|
||||
{
|
||||
await using var context = this.CreateContext();
|
||||
//
|
||||
// Read everything before yielding the first line: this is an iterator, and a try/catch
|
||||
// cannot wrap a yield return. Each probe therefore catches its own failure and answers
|
||||
// with an empty string, which shows up as "unknown" below. Without that, a single failing
|
||||
// PRAGMA would throw out of here and the information page would replace the entire block
|
||||
// with the fallback client.
|
||||
//
|
||||
var snapshot = await this.ReadDisplaySnapshotAsync();
|
||||
|
||||
yield return (TB("Reported version"), version);
|
||||
yield return (TB("Database path"), this.databasePath);
|
||||
yield return (TB("Native library"), OrUnknown(SqliteRuntimeInfo.GetNativeLibraryName()));
|
||||
yield return (TB("Wrapper version"), OrUnknown(SqliteRuntimeInfo.GetWrapperVersion()));
|
||||
yield return (TB("Process architecture"), OrUnknown(SqliteRuntimeInfo.GetProcessArchitecture()));
|
||||
|
||||
// Only worth a line when the process runs on a foreign architecture, Rosetta above all:
|
||||
var systemArchitecture = SqliteRuntimeInfo.GetSystemArchitecture();
|
||||
if (!string.IsNullOrWhiteSpace(systemArchitecture))
|
||||
yield return (TB("System architecture"), systemArchitecture);
|
||||
|
||||
yield return (TB("Full-text search (FTS5)"), OrUnknown(snapshot.FullTextSearch));
|
||||
yield return (TB("Journal mode"), OrUnknown(snapshot.JournalMode));
|
||||
yield return (TB("Schema version"), OrUnknown(snapshot.SchemaVersion));
|
||||
yield return (TB("Database tables"), OrUnknown(snapshot.TableCount));
|
||||
yield return (TB("Storage size"), this.GetStorageSize());
|
||||
yield return (TB("Indexed data sources"), (await context.DataSources.CountAsync(CancellationToken.None)).ToString(CultureInfo.InvariantCulture));
|
||||
yield return (TB("Indexed files"), (await context.EmbeddedFiles.CountAsync(CancellationToken.None)).ToString(CultureInfo.InvariantCulture));
|
||||
yield return (TB("Search chunks"), (await context.EmbeddingChunks.CountAsync(CancellationToken.None)).ToString(CultureInfo.InvariantCulture));
|
||||
yield return (TB("Permanently skipped files"), (await context.PermanentIndexingFailures.CountAsync(CancellationToken.None)).ToString(CultureInfo.InvariantCulture));
|
||||
yield return (TB("Indexed data sources"), OrUnknown(snapshot.DataSourceCount));
|
||||
yield return (TB("Indexed files"), OrUnknown(snapshot.FileCount));
|
||||
yield return (TB("Permanently skipped files"), OrUnknown(snapshot.FailureCount));
|
||||
}
|
||||
|
||||
public override async Task<DataSourceIndexState?> GetDataSourceStateAsync(string dataSourceId, CancellationToken token)
|
||||
@ -348,10 +369,167 @@ public sealed class SqliteIndexStoreClientImplementation(string name, string dat
|
||||
await transaction.CommitAsync(token);
|
||||
}
|
||||
|
||||
public override async Task<long?> GetTotalChunkCountAsync(CancellationToken token)
|
||||
{
|
||||
try
|
||||
{
|
||||
await using var context = this.CreateContext();
|
||||
|
||||
//
|
||||
// Sum the chunk counts the files carry instead of counting the rows of the chunk table:
|
||||
// embedded_files holds one row per file, embedding_chunks one per chunk. On a large index
|
||||
// that is a difference of two orders of magnitude, and the information page reads this on
|
||||
// every visit.
|
||||
//
|
||||
return await context.EmbeddedFiles.SumAsync(file => (long)file.ChunkCount, token);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
this.Logger?.LogWarning(exception, "Failed to count the search chunks of the local RAG index.");
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
public override void Dispose()
|
||||
{
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Everything the display info reads out of the database in one go.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Every property is empty when its probe could not answer. The caller turns that into "unknown".
|
||||
/// </remarks>
|
||||
private sealed record DisplaySnapshot
|
||||
{
|
||||
public string FullTextSearch { get; init; } = string.Empty;
|
||||
|
||||
public string JournalMode { get; init; } = string.Empty;
|
||||
|
||||
public string SchemaVersion { get; init; } = string.Empty;
|
||||
|
||||
public string TableCount { get; init; } = string.Empty;
|
||||
|
||||
public string DataSourceCount { get; init; } = string.Empty;
|
||||
|
||||
public string FileCount { get; init; } = string.Empty;
|
||||
|
||||
public string FailureCount { get; init; } = string.Empty;
|
||||
}
|
||||
|
||||
private static string OrUnknown(string value) => string.IsNullOrWhiteSpace(value) ? TB("unknown") : value;
|
||||
|
||||
private async Task<DisplaySnapshot> ReadDisplaySnapshotAsync()
|
||||
{
|
||||
var token = CancellationToken.None;
|
||||
try
|
||||
{
|
||||
await using var context = this.CreateContext();
|
||||
return new DisplaySnapshot
|
||||
{
|
||||
FullTextSearch = await GetFullTextSearchStateAsync(context, token),
|
||||
JournalMode = (await QueryScalarTextAsync(context, "PRAGMA journal_mode;", token)).ToUpperInvariant(),
|
||||
SchemaVersion = await GetSchemaVersionAsync(context, token),
|
||||
TableCount = await GetTableCountAsync(context, token),
|
||||
DataSourceCount = await FormatCountAsync(context.DataSources, token),
|
||||
FileCount = await FormatCountAsync(context.EmbeddedFiles, token),
|
||||
FailureCount = await FormatCountAsync(context.PermanentIndexingFailures, token),
|
||||
};
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
//
|
||||
// Opening the database failed altogether. The runtime details the caller shows next to
|
||||
// these values still say which library was loaded and for which architecture, which is
|
||||
// what a support case needs most in exactly this situation. So hand back an empty
|
||||
// snapshot instead of letting the whole block fall back.
|
||||
//
|
||||
this.Logger?.LogWarning(exception, "Failed to read the display details of the local RAG index.");
|
||||
return new DisplaySnapshot();
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task<string> QueryScalarTextAsync(IndexStoreDbContext context, string sql, CancellationToken token)
|
||||
{
|
||||
try
|
||||
{
|
||||
//
|
||||
// Go through the raw connection rather than through SqlQueryRaw: that one expects a
|
||||
// column named "Value" and wraps the statement, neither of which works for a PRAGMA.
|
||||
//
|
||||
var connection = context.Database.GetDbConnection();
|
||||
if (connection.State is not ConnectionState.Open)
|
||||
await connection.OpenAsync(token);
|
||||
|
||||
await using var command = connection.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
|
||||
var result = await command.ExecuteScalarAsync(token);
|
||||
return result?.ToString() ?? string.Empty;
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task<string> GetFullTextSearchStateAsync(IndexStoreDbContext context, CancellationToken token)
|
||||
{
|
||||
var compiledIn = await QueryScalarTextAsync(context, "SELECT sqlite_compileoption_used('ENABLE_FTS5')", token);
|
||||
return compiledIn switch
|
||||
{
|
||||
"1" => TB("available"),
|
||||
"0" => TB("not available"),
|
||||
_ => string.Empty
|
||||
};
|
||||
}
|
||||
|
||||
private static async Task<string> GetTableCountAsync(IndexStoreDbContext context, CancellationToken token)
|
||||
{
|
||||
//
|
||||
// Counts the migration history, the FTS5 virtual table and its shadow tables as well. That
|
||||
// is the point: a missing shadow table is a finding, not noise.
|
||||
//
|
||||
var tables = await QueryScalarTextAsync(context, "SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'", token);
|
||||
return int.TryParse(tables, NumberStyles.Integer, CultureInfo.InvariantCulture, out var tableCount) ? tableCount.CompactCount() : string.Empty;
|
||||
}
|
||||
|
||||
private static async Task<string> GetSchemaVersionAsync(IndexStoreDbContext context, CancellationToken token)
|
||||
{
|
||||
try
|
||||
{
|
||||
//
|
||||
// Reading the applied migrations only touches the history table, no assembly scan. The
|
||||
// pending ones do scan, but the schema migrator walks that same path on every start, so
|
||||
// the DynamicDependency attributes over there already keep the migration types alive.
|
||||
//
|
||||
var appliedMigrations = (await context.Database.GetAppliedMigrationsAsync(token)).ToList();
|
||||
if (appliedMigrations.Count == 0)
|
||||
return TB("no migration applied");
|
||||
|
||||
var pendingMigrations = (await context.Database.GetPendingMigrationsAsync(token)).ToList();
|
||||
return pendingMigrations.Count == 0
|
||||
? string.Format(I18N.I.Culture, TB("{0} ({1} applied)"), appliedMigrations[^1], appliedMigrations.Count.CompactCount())
|
||||
: string.Format(I18N.I.Culture, TB("{0} ({1} applied, {2} pending)"), appliedMigrations[^1], appliedMigrations.Count.CompactCount(), pendingMigrations.Count.CompactCount());
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task<string> FormatCountAsync<T>(IQueryable<T> query, CancellationToken token) where T : class
|
||||
{
|
||||
try
|
||||
{
|
||||
return (await query.CountAsync(token)).CompactCount();
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
private async Task InitializeAsync(CancellationToken token)
|
||||
{
|
||||
await using var context = this.CreateContext();
|
||||
|
||||
@ -0,0 +1,88 @@
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
namespace AIStudio.Tools.Databases.IndexStore;
|
||||
|
||||
/// <summary>
|
||||
/// What the running process can tell about the SQLite library it has loaded.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// These are methods instead of static fields on purpose: SQLitePCL.Batteries_V2.Init() runs when the
|
||||
/// index store client is created, and a type initializer could well run before that. Every member
|
||||
/// answers with an empty string when it cannot tell, so a single unavailable detail never costs the
|
||||
/// caller the rest of them.
|
||||
/// </remarks>
|
||||
internal static class SqliteRuntimeInfo
|
||||
{
|
||||
/// <summary>
|
||||
/// The name of the native library SQLitePCLRaw has bound to.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// We ship our own build through the bundle_e_sqlite3 package, so this reads e_sqlite3 on every
|
||||
/// platform. Anything else means the process bound to a different library than we shipped, which
|
||||
/// is exactly the kind of thing a support case needs to show.
|
||||
/// </remarks>
|
||||
public static string GetNativeLibraryName()
|
||||
{
|
||||
try
|
||||
{
|
||||
return SQLitePCL.raw.GetNativeLibraryName();
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The version of the managed SQLitePCLRaw wrapper, which is a different thing than the SQLite version.
|
||||
/// </summary>
|
||||
public static string GetWrapperVersion()
|
||||
{
|
||||
try
|
||||
{
|
||||
// Read the assembly name rather than an attribute: reflecting over members would not
|
||||
// survive trimming, the name does.
|
||||
var wrapperVersion = typeof(SQLitePCL.raw).Assembly.GetName().Version;
|
||||
return wrapperVersion is null ? string.Empty : $"SQLitePCLRaw.core {wrapperVersion.ToString(3)}";
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The architecture this process runs as, together with the runtime identifier it was built for.
|
||||
/// </summary>
|
||||
public static string GetProcessArchitecture()
|
||||
{
|
||||
try
|
||||
{
|
||||
return $"{RuntimeInformation.ProcessArchitecture} ({RuntimeInformation.RuntimeIdentifier})";
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The architecture of the machine, but only when it differs from the one of the process.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// A difference means the process runs through an emulation layer, Rosetta above all. That is a
|
||||
/// classic reason for a native library failing to load, so it earns its own line when it happens
|
||||
/// and stays out of the way when it does not.
|
||||
/// </remarks>
|
||||
public static string GetSystemArchitecture()
|
||||
{
|
||||
try
|
||||
{
|
||||
return RuntimeInformation.OSArchitecture == RuntimeInformation.ProcessArchitecture ? string.Empty : RuntimeInformation.OSArchitecture.ToString();
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,15 +1,24 @@
|
||||
using AIStudio.Tools.Databases.IndexStore;
|
||||
using AIStudio.Tools.PluginSystem;
|
||||
using AIStudio.Tools.Rust;
|
||||
using AIStudio.Tools.Services;
|
||||
|
||||
namespace AIStudio.Tools.Databases.VectorStore;
|
||||
|
||||
/// <param name="indexStoreAccessor">
|
||||
/// Resolves the index store, which is where the number of stored vectors comes from. Counting the
|
||||
/// points in Qdrant Edge itself would have to load every shard first and would hold the global
|
||||
/// database mutex against ongoing inserts and searches. The accessor is only called while building
|
||||
/// the display info, never while this client is created: creating it already holds the vector store
|
||||
/// lock, and the accessor takes the index store lock, so the two are never held at the same time.
|
||||
/// </param>
|
||||
public sealed class QdrantEdgeClientImplementation(
|
||||
string name,
|
||||
string path,
|
||||
string version,
|
||||
int storesCount,
|
||||
RustService rustService) : VectorStoreClient(name, path)
|
||||
RustService rustService,
|
||||
Func<CancellationToken, Task<IndexStoreClient>> indexStoreAccessor) : VectorStoreClient(name, path)
|
||||
{
|
||||
private const string DATABASE_NAME = "Qdrant Edge";
|
||||
private const string INFO_PATH = "/system/qdrant-edge/info";
|
||||
@ -28,6 +37,7 @@ public sealed class QdrantEdgeClientImplementation(
|
||||
|
||||
public static async Task<DatabaseClient> CreateAsync(
|
||||
RustService rustService,
|
||||
Func<CancellationToken, Task<IndexStoreClient>> indexStoreAccessor,
|
||||
ILogger logger,
|
||||
ILogger<DatabaseClient> databaseClientLogger,
|
||||
CancellationToken cancellationToken)
|
||||
@ -60,7 +70,7 @@ public sealed class QdrantEdgeClientImplementation(
|
||||
return CreateNoVectorStoreClient(DATABASE_NAME, $"Failed to get the {DATABASE_NAME} path from Rust.", DatabaseClientStatus.UNAVAILABLE, databaseClientLogger);
|
||||
|
||||
var name = string.IsNullOrWhiteSpace(qdrantEdgeInfo.Name) ? DATABASE_NAME : qdrantEdgeInfo.Name;
|
||||
var client = new QdrantEdgeClientImplementation(name, qdrantEdgeInfo.Path, qdrantEdgeInfo.Version, qdrantEdgeInfo.StoresCount, rustService);
|
||||
var client = new QdrantEdgeClientImplementation(name, qdrantEdgeInfo.Path, qdrantEdgeInfo.Version, qdrantEdgeInfo.StoresCount, rustService, indexStoreAccessor);
|
||||
client.SetLogger(databaseClientLogger);
|
||||
return client;
|
||||
}
|
||||
@ -77,9 +87,35 @@ public sealed class QdrantEdgeClientImplementation(
|
||||
if (!currentInfo.IsAvailable)
|
||||
yield return (TB("Status"), currentInfo.UnavailableReason ?? TB("Qdrant Edge is not available."));
|
||||
|
||||
var storedVectors = await this.GetStoredVectorCountAsync();
|
||||
|
||||
yield return (TB("Reported version"), displayVersion);
|
||||
yield return (TB("Storage size"), $"{this.GetStorageSize()}");
|
||||
yield return (TB("Number of vector stores"), displayStoresCount.ToString());
|
||||
yield return (TB("Number of vector stores"), displayStoresCount.CompactCount());
|
||||
yield return (TB("Stored vectors"), storedVectors?.CompactCount() ?? TB("unknown"));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads how many vectors the vector stores hold in total.
|
||||
/// </summary>
|
||||
/// <returns>The number of vectors, or null when the index store cannot tell.</returns>
|
||||
private async Task<long?> GetStoredVectorCountAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
//
|
||||
// One chunk is one vector: every chunk becomes exactly one point carrying the single
|
||||
// named vector "embedding". So the index store knows this number without Qdrant Edge
|
||||
// having to load a single shard for it.
|
||||
//
|
||||
var indexStore = await indexStoreAccessor(CancellationToken.None);
|
||||
return await indexStore.GetTotalChunkCountAsync(CancellationToken.None);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
this.Logger?.LogWarning(exception, "Failed to read the number of stored vectors from the index store.");
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
public override async Task<VectorStoreEnsureResult> EnsureVectorStoreExists(string storeName, string dataSourceName, int vectorSize, CancellationToken token) =>
|
||||
|
||||
@ -1,7 +1,46 @@
|
||||
using AIStudio.Tools.PluginSystem;
|
||||
|
||||
namespace AIStudio.Tools;
|
||||
|
||||
public static class LongExtensions
|
||||
{
|
||||
private static readonly string[] COUNT_SUFFIXES = ["k", "M", "B", "T"];
|
||||
|
||||
/// <summary>
|
||||
/// Formats a count so that large numbers stay readable: 1456 becomes 1.46k, 4512900 becomes 4.51M.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Counts scale in thousands, not in steps of 1024 — that is what FileSize is for, and storage
|
||||
/// sizes keep using it. Numbers below 1000 stay exact, because shortening them would hide the
|
||||
/// difference between 4 and 999. The decimal separator follows the culture of the active language
|
||||
/// plugin rather than the one of the thread, which never moves along with the app's language.
|
||||
/// </remarks>
|
||||
/// <param name="count">The number to format.</param>
|
||||
/// <returns>The formatted number.</returns>
|
||||
public static string CompactCount(this long count)
|
||||
{
|
||||
var culture = I18N.I.Culture;
|
||||
if (count is > -1_000 and < 1_000)
|
||||
return count.ToString("N0", culture);
|
||||
|
||||
var order = -1;
|
||||
double value = count;
|
||||
while (Math.Abs(value) >= 1_000 && order < COUNT_SUFFIXES.Length - 1)
|
||||
{
|
||||
order++;
|
||||
value /= 1_000;
|
||||
}
|
||||
|
||||
return $"{value.ToString("0.##", culture)}{COUNT_SUFFIXES[order]}";
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Formats a count so that large numbers stay readable.
|
||||
/// </summary>
|
||||
/// <param name="count">The number to format.</param>
|
||||
/// <returns>The formatted number.</returns>
|
||||
public static string CompactCount(this int count) => ((long)count).CompactCount();
|
||||
|
||||
/// <summary>
|
||||
/// Formats the file size in a human-readable format.
|
||||
/// </summary>
|
||||
|
||||
@ -21,6 +21,7 @@
|
||||
- Added a jump to the right page for the sources of your own documents (RAG), so a PDF opens directly where the passage was found, wherever your system and its program support it.
|
||||
- Added a way to show a source of your own documents (RAG) in your file manager.
|
||||
- Added the page of a passage to what the AI is told when it answers from your own documents (RAG), so it can name the page an answer rests on.
|
||||
- Added the details of the two databases behind local RAG to the information page: which versions they run, how much space they use on your disk, and how much they hold.
|
||||
- Added support for several drop areas on the same page. More complex assistants can now receive files or folders by drag and drop at more than one place.
|
||||
- Added drag and drop to the input and output folder of the Batch Processing assistant: drop a folder onto either field to choose it.
|
||||
- Added ways to load text from a file and drop zones for them, throughout the assistants and dialogs. We went through them one by one, so many fields that used to accept typed text only now take the content of a file as well.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user