Indexed large data sources in steps so the others get their turn (#1052)

This commit is contained in:
Thorsten Sommer authored and GitHub committed 2026-10-11 20:48:40 +02:00
1 parent fe4a64f382
commit a3c6b29313
26 files changed
+1932 -334

No files matched your search

@@ -0,0 +1,23 @@
namespace AIStudio.Tools.Services;
/// <summary>
/// The lanes of the queue of indexing runs, cf. DataSourceRunQueue.
/// </summary>
/// <remarks>
/// Every run in the front lane comes out before any run in the back lane. DataSourceEmbeddingService
/// picks the lane when it enqueues a run: a data source which carries on a pass of its indexing waits
/// in the back lane, cf. IndexingPass. A mailbox therefore waits for one step at most, rather than for
/// hours behind a large folder, and large data sources take turns with each other.
/// </remarks>
internal enum DataSourceQueueLane
{
/// <summary>
/// For a data source which begins a pass, e.g. a mailbox at its interval.
/// </summary>
FRONT,
/// <summary>
/// For a data source which carries on a pass, after it did its share of the work.
/// </summary>
BACK,
}