Added mailboxes as local data sources (#1022)

AGENTS.md

@@ -186,12 +186,15 @@ When adding configuration plugin capabilities:
 When adding, changing, or removing model-driven tools, keep these parts in sync:
 - `app/MindWork AI Studio/Tools/ToolCallingSystem/ToolCallingImplementations/` for the `IToolImplementation` class, which states its own `ToolDefinition` through `GetDefinition()`, written with `ToolSettingsSchemaBuilder` for its settings and `ToolParameterSchemaBuilder` for the arguments the model passes. There are no tool definition files; a tool arriving from elsewhere brings an `IToolDefinitionSource` instead.
 - `app/MindWork AI Studio/Program.cs` for DI registration of the implementation. Registering it as an `IToolImplementation` is enough, because `CodeToolDefinitionSource` collects the definitions of all of them.
-- `app/MindWork AI Studio/Tools/ToolCallingSystem/ToolSelectionRules.cs` when the shared tool-call limits change. A tool's own minimum provider confidence belongs in its definition, not here.
+- An `IToolCollection` next to the tools, registered in `Program.cs`, when tools only make sense together. Selections, `DataTools.DisabledToolIds`, and the minimum provider confidence name tool collections; a tool outside a declared collection forms one under its own ID, and the ID of a tool inside one stands for its whole collection. Normalize a selection with `ToolRegistry.NormalizeSelection`, and expand it into tools with `ToolRegistry.ExpandSelection` only where the view of the model counts.
+- `app/MindWork AI Studio/Tools/ToolCallingSystem/ToolSelectionRules.cs` when the shared tool-call limits change. A tool's own minimum provider confidence belongs in its definition, or in the definition of its collection, not here.
 - `app/MindWork AI Studio/Tools/ToolCallingSystem/ToolSettingsOptionSources.cs` when a tool setting offers a fixed choice the app maintains, such as languages. Prefer this over spelling the values out in the settings schema; it keeps the list in one place and gives the user translated names.
 - `app/MindWork AI Studio/Plugins/configuration/plugin.lua` to document each setting's field name, meaning, and data type. Tool settings need no code to be centrally manageable: an organization addresses them by `"<toolId>.<fieldName>"` in `DataTools.LockedToolSettings` or `DataTools.DefaultToolSettings`.
 
 Tool implementations must treat model-provided arguments as untrusted input. Validate settings and arguments, protect secrets with `SensitiveTraceArgumentNames`, use `ToolExecutionBlockedException` for intentional policy blocks, and check provider confidence before returning sensitive data to the model.
 
+A tool which belongs to a preview feature returns false from `IToolImplementation.IsAvailable` while the preview is switched off; the registry then leaves it out of every list, every request, and the token count, so no component has to check that preview for the tool. Every tool declares in `IToolImplementation.OutboundData` where its arguments go: a chat which read from a mailbox keeps the tools whose data goes further than the mailbox allows from being offered and from running, see `ToolSelectionRules.IsOutboundDataAllowed`. A tool which brings content of a mailbox into the chat raises `ToolExecutionResult.RequiredOutboundDataRestriction`, next to `RequiredProviderConfidence` and `RequiredDataSecurity`. "Searching Mailboxes" in `documentation/Tools.md` explains the mail tools.
+
 A tool which offers itself from the context of a chat instead of being selected, such as `semantic_search`, sets `Activation = ToolActivation.CONTEXT` and tailors its function to each request in `ResolveFunctionAsync`. Code which decides something on behalf of a request — whether the classic RAG process steps back, say — asks `ToolRegistry.GetOfferBlockReasonAsync` or `ToolRegistry.GetEffectiveRetrievalModeAsync` with the provider settings of the request (`IProvider.CreateSettingsProvider`), never a check of its own: two answers which drift apart leave a chat searching nothing or twice.
 
 ## Model Capabilities
@@ -221,6 +224,56 @@ RAG is available as a beta preview feature. Architecture:
 - **Index database** - SQLite, holding the file fingerprints and the chunk texts for full-text search; see "Databases" below
 - **File processing** - Extracts text from PDF, DOCX, XLSX via Rust runtime
 
+### Indexed data sources
+
+Everything AI Studio embeds itself runs through one pipeline in `app/MindWork AI Studio/Tools/Services/Indexing/`,
+driven by `DataSourceEmbeddingService`, which queues the runs, prepares each one and owns the statuses. A new
+kind of data source plugs into this pipeline instead of building its own.
+
+The parts:
+- **`IIndexedDataSource`** (`Settings/`) - what indexing needs to know about a data source: confidence level,
+  embedding provider and chunk settings. `IDataSourceBase` is what every data source has. Implement
+  `IDataSource` on top only when classic RAG, Semantic Search and the agents should see the data source. A data
+  source kept in a list of its own implements `IIndexedDataSource` alone, and the compiler keeps it out of
+  `DataSources`.
+- **`IIndexedSourceIndexer`** - one per kind of data source. `Supports` claims the data sources, `ProcessAsync`
+  finds and reads the documents of one run, and `TrackChanges` / `StopTracking` notice changes on their own.
+  `FileSourceIndexer` is the reference: a file system watcher per data source, fingerprints over path, size and
+  write time.
+- **`IndexedRunContext`** - one prepared run: both stores, the embedding provider, the manifest and the
+  collection. `IndexDocumentAsync` embeds and stores one document; the cleanup methods remove what a failed
+  attempt left behind.
+- **`EmbeddingDocument`** - one document: its key, its index row, its display name, and how to read its chunks.
+- **`DocumentRunProgress`** - counts the documents, records indexed and failed ones in the stores, publishes the
+  status and completes the run.
+- **`TextChunker`** - cuts text into chunks the embedding provider accepts. Pick one of its strategies; do not
+  write a chunker of your own.
+
+To add a kind of data source:
+1. Write its indexer in `Tools/Services/Indexing/` and create it in `DataSourceEmbeddingService.CreateIndexers`,
+   which hands every indexer the same `TextChunker`.
+2. Gate it in `IsSupportedIndexedSource`, behind a preview feature of its own while it is new.
+3. When the data source is not kept in `DataSources`, add its list to `GetConfiguredIndexedSources`. Every lookup
+   by id and every pass over all data sources goes through it: the startup hash check,
+   `QueueAllInternalDataSourcesAsync` and `RefreshWatchers`.
+4. Keep whatever the kind has to remember beyond its documents in tables of its own in the index store, added
+   by an EF Core migration (see "Databases").
+5. Report every status through `DocumentRunProgress`, so all rows of the embedding page behave alike.
+
+Rules which are easy to break:
+- **A document key is not a path.** Only files use their full path as the key. Never pass a key through the
+  `Path` APIs: on Windows, `Path.GetFullPath` reads a key like `mail:…` as a file with an alternate data stream.
+- **Ids and signature are pinned.** The formats in `IndexedDocumentIds` and the embedding signature
+  (`DataSourceEmbeddingService.BuildEmbeddingSignature`) are fixed by tests, because every stored chunk and every
+  index depends on them. When the metadata stored next to a chunk changes, raise `CHUNK_METADATA_VERSION`
+  deliberately: that rebuilds every index.
+- **Every content path goes through the prompt injection filter.** Files pass the sanitizer of the runtime while
+  their text is extracted; a new kind of data source needs its own pass through `PromptInjectionGuardService`.
+- **The service decides when, the indexer decides how.** Whether changes are tracked at all depends on the
+  automatic refresh setting and the startup hash check, and only the embedding service decides that.
+- **Moving a `TB()` text into another class gives it a new I18N key**, so its translation is made anew during
+  the next localization run.
+
 ## Databases
 
 Local RAG runs on two databases, addressed through `DatabaseRole`: