Improved the tool selection by showing unavailable tools on request (#1116)

This commit is contained in:
Thorsten Sommer authored and GitHub committed 2026-10-27 09:45:57 +01:00
1 parent 04546c25ac
commit 1a23479f33
9 files changed
+369 -62

No files matched your search

@@ -5989,9 +5989,21 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLDEFAULTSCONFIGURATION::T948842182"] =
-- Tool changes are locked while a response is running. Your current selection is shown below and applies again from the next message once the run is finished.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1688023907"] = "Tool changes are locked while a response is running. Your current selection is shown below and applies again from the next message once the run is finished."
-- Show 1 unavailable tool
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1852398429"] = "Show 1 unavailable tool"
-- Tools allow the LLM to perform targeted additional actions such as web searches or reading web pages.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1944689297"] = "Tools allow the LLM to perform targeted additional actions such as web searches or reading web pages."
-- Show {0} unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2227029327"] = "Show {0} unavailable tools"
-- Hide unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2707194504"] = "Hide unavailable tools"
-- Show {0} unavailable tools, {1} of them selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T30104720"] = "Show {0} unavailable tools, {1} of them selected"
-- Required settings are missing. Configure this tool before enabling it.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3119156561"] = "Required settings are missing. Configure this tool before enabling it."
@@ -6007,9 +6019,18 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3904490680"] = "No tools
-- This tool requires provider confidence {0}. The selected provider has {1}.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T4097602620"] = "This tool requires provider confidence {0}. The selected provider has {1}."
-- Show 1 unavailable tool, which is selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T641009275"] = "Show 1 unavailable tool, which is selected"
-- You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T732483684"] = "You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here."
-- Tool Selection
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T749664565"] = "Tool Selection"
-- No tool can be selected right now.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T953828816"] = "No tool can be selected right now."
-- Select tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T998515990"] = "Select tools"
@@ -1,3 +1,4 @@
@using AIStudio.Tools.ToolCallingSystem
@inherits MSGComponentBase
<div class="d-flex">
@@ -36,61 +37,49 @@
@if (this.SupportsTools && this.catalog.Count > 0)
{
@*
The striping sits on this wrapper: the rows share their parent with the
introduction and the occasional alert, which would shift the parity.
*@
<div class="tool-selection-rows">
@foreach (var item in this.catalog)
{
var isSelected = this.SelectedToolIds.Contains(item.Id);
var warningText = this.GetWarningText(item);
<div class="tool-selection-row">
<MudStack Row="true" AlignItems="AlignItems.Center" Justify="Justify.SpaceBetween" Spacing="1">
@*
Everything but the settings button switches the tool, so aiming for the
small switch is optional. The button spans that part of the row, which
keeps the settings button outside of it without any event plumbing.
*@
<MudButton Variant="Variant.Text" Color="Color.Default" Class="px-1 py-0 justify-start"
Style="min-width:auto; text-transform:none; flex-grow:1;"
Disabled="@this.IsRowDisabled(item)" OnClick="@(async () => await this.ToggleToolFromRow(item))">
<MudStack Row="true" AlignItems="AlignItems.Center" Spacing="1">
@*
A checkbox rather than a switch, because this row is one entry of a set the
user picks from, not a setting of its own -- the same question the data source
selection next to it asks, and it should not look like a different one.
var itemsShownFirst = this.GetItemsShownFirst();
var unavailableItems = this.GetUnavailableItems();
The checkbox only shows the state; the surrounding button does the switching.
It therefore takes no pointer events at all: its label reaches past the visible
box and would otherwise swallow the clicks landing in that strip.
*@
<MudCheckBox T="bool" Size="Size.Small" Dense="@true" Color="Color.Primary" Value="@isSelected" ReadOnly="@true" Disabled="@this.IsRowDisabled(item)" Style="pointer-events: none;" />
<MudIcon Icon="@item.Icon" Color="Color.Info" />
@if (!item.IsActive)
{
<MudTooltip Text="@T("This tool has been disabled by your organization.")">
<MudIcon Icon="@Icons.Material.Filled.Lock" Color="Color.Error" Size="Size.Small" />
</MudTooltip>
}
<MudTooltip Text="@item.Description">
<MudText Typo="Typo.body1">@item.DisplayName</MudText>
</MudTooltip>
</MudStack>
</MudButton>
<MudIconButton Icon="@Icons.Material.Filled.Settings" Size="Size.Small" OnClick="@(async () => await this.OpenSettings(item))" />
</MudStack>
@if (!string.IsNullOrWhiteSpace(warningText))
@*
Each list has a wrapper of its own, which gives its rows their compact padding:
the rows share their parent with nothing else, neither the introduction nor the
button between both lists.
*@
@if (itemsShownFirst.Count > 0)
{
<div class="tool-selection-rows">
@foreach (var item in itemsShownFirst)
{
@this.RenderRow(item)
}
</div>
}
else
{
<MudText Typo="Typo.body1">@T("No tool can be selected right now.")</MudText>
}
@if (unavailableItems.Count > 0)
{
<MudButton StartIcon="@(this.showUnavailableTools ? Icons.Material.Filled.ExpandLess : Icons.Material.Filled.ExpandMore)"
Size="Size.Small"
Variant="Variant.Text"
Class="mt-1"
OnClick="@this.ToggleUnavailableTools">
@this.GetUnavailableToolsButtonText(unavailableItems)
</MudButton>
<MudCollapse Expanded="@this.showUnavailableTools">
<div class="tool-selection-rows">
@foreach (var item in unavailableItems)
{
@*
A caption renders as an inline span by default, and an inline element
cannot be justified. The div makes it a block of its own.
*@
<MudJustifiedText Typo="Typo.caption" HtmlTag="div" Color="Color.Warning" Class="ml-2">@warningText</MudJustifiedText>
@this.RenderRow(item)
}
</div>
}
</div>
<MudJustifiedText Typo="Typo.caption" HtmlTag="div" Class="mt-2 ml-2">
@T("You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here.")
</MudJustifiedText>
</MudCollapse>
}
}
</MudCardContent>
<MudCardActions>
@@ -99,4 +88,58 @@
</MudCardActions>
</MudCard>
</MudPopover>
</div>
</div>
@code {
/// <summary>
/// One row of the selection, in either list.
/// </summary>
/// <remarks>
/// Both lists show the same row, so it is written once. It stays in this component, which keeps
/// the translations of its texts: a component of its own would give each of them a new key.
/// </remarks>
private RenderFragment RenderRow(ToolCatalogItem item) => @<div class="tool-selection-row">
<MudStack Row="true" AlignItems="AlignItems.Center" Justify="Justify.SpaceBetween" Spacing="1">
@*
Everything but the settings button switches the tool, so aiming for the
small switch is optional. The button spans that part of the row, which
keeps the settings button outside of it without any event plumbing.
*@
<MudButton Variant="Variant.Text" Color="Color.Default" Class="px-1 py-0 justify-start"
Style="min-width:auto; text-transform:none; flex-grow:1;"
Disabled="@this.IsRowDisabled(item)" OnClick="@(async () => await this.ToggleToolFromRow(item))">
<MudStack Row="true" AlignItems="AlignItems.Center" Spacing="1">
@*
A checkbox rather than a switch, because this row is one entry of a set the
user picks from, not a setting of its own -- the same question the data source
selection next to it asks, and it should not look like a different one.
The checkbox only shows the state; the surrounding button does the switching.
It therefore takes no pointer events at all: its label reaches past the visible
box and would otherwise swallow the clicks landing in that strip.
*@
<MudCheckBox T="bool" Size="Size.Small" Dense="@true" Color="Color.Primary" Value="@this.SelectedToolIds.Contains(item.Id)" ReadOnly="@true" Disabled="@this.IsRowDisabled(item)" Style="pointer-events: none;" />
<MudIcon Icon="@item.Icon" Color="Color.Info" />
@if (!item.IsActive)
{
<MudTooltip Text="@T("This tool has been disabled by your organization.")">
<MudIcon Icon="@Icons.Material.Filled.Lock" Color="Color.Error" Size="Size.Small" />
</MudTooltip>
}
<MudTooltip Text="@item.Description">
<MudText Typo="Typo.body1">@item.DisplayName</MudText>
</MudTooltip>
</MudStack>
</MudButton>
<MudIconButton Icon="@Icons.Material.Filled.Settings" Size="Size.Small" OnClick="@(async () => await this.OpenSettings(item))" />
</MudStack>
@if (this.GetWarningText(item) is var warningText && !string.IsNullOrWhiteSpace(warningText))
{
@*
A caption renders as an inline span by default, and an inline element
cannot be justified. The div makes it a block of its own.
*@
<MudJustifiedText Typo="Typo.caption" HtmlTag="div" Color="Color.Warning" Class="ml-2">@warningText</MudJustifiedText>
}
</div>;
}
@@ -45,6 +45,7 @@ public partial class ToolSelection : MSGComponentBase
private IDialogService DialogService { get; init; } = null!;
private bool showSelection;
private bool showUnavailableTools;
private IReadOnlyList<ToolCatalogItem> catalog = [];
protected override void OnParametersSet()
@@ -77,22 +78,72 @@ public partial class ToolSelection : MSGComponentBase
{
this.showSelection = !this.showSelection;
if (this.showSelection)
{
this.showUnavailableTools = false;
this.catalog = await this.ToolRegistry.GetCatalogAsync(this.Component);
}
}
private void Hide() => this.showSelection = false;
private void ToggleUnavailableTools() => this.showUnavailableTools = !this.showUnavailableTools;
/// <summary>
/// Whether the selection keeps the tools which cannot be selected right now apart, behind a button.
/// </summary>
/// <remarks>
/// Not while a response is running: no tool can be switched then, and the list shows the current
/// selection as a whole, as the note above it says.
/// </remarks>
private bool KeepsUnavailableToolsApart => !this.Disabled;
/// <summary>
/// The tools the selection lists first: those the user can select right now, or all of them while a response is running.
/// </summary>
private List<ToolCatalogItem> GetItemsShownFirst() => this.KeepsUnavailableToolsApart
? [..this.catalog.Where(this.IsSelectable)]
: [..this.catalog];
/// <summary>
/// The tools which cannot be selected right now, selected ones included, which the selection lists behind a button.
/// </summary>
private List<ToolCatalogItem> GetUnavailableItems() => this.KeepsUnavailableToolsApart
? [..this.catalog.Where(item => !this.IsSelectable(item))]
: [];
/// <summary>
/// The text of the button which shows or hides the tools that cannot be selected right now.
/// </summary>
/// <remarks>
/// A selected tool may end up behind the button, for example, when the user switches to a provider
/// with less confidence. The text then says so, because the selection would otherwise seem to have
/// lost it.
/// </remarks>
private string GetUnavailableToolsButtonText(IReadOnlyCollection<ToolCatalogItem> unavailableItems)
{
if (this.showUnavailableTools)
return T("Hide unavailable tools");
var selectedCount = unavailableItems.Count(item => this.SelectedToolIds.Contains(item.Id));
if (unavailableItems.Count == 1)
return selectedCount == 0 ? T("Show 1 unavailable tool") : T("Show 1 unavailable tool, which is selected");
return selectedCount == 0
? string.Format(T("Show {0} unavailable tools"), unavailableItems.Count.CompactCount())
: string.Format(T("Show {0} unavailable tools, {1} of them selected"), unavailableItems.Count.CompactCount(), selectedCount.CompactCount());
}
/// <summary>
/// Whether this tool can be switched at all right now.
/// </summary>
/// <remarks>
/// The switch and the row click share this, so both agree on when a tool is out of reach: the
/// organization disabled it, it is not configured, the provider lacks the confidence it needs,
/// the chat read a mailbox which keeps the tool back, a response is running, or the model
/// cannot call tools in the first place.
/// tool itself cannot be selected right now, see ToolSelectionRules.IsSelectable, a response is
/// running, or the model cannot call tools in the first place.
/// </remarks>
private bool IsRowDisabled(ToolCatalogItem item) => !item.IsActive || !item.ConfigurationState.IsConfigured || this.IsBlockedByProviderConfidence(item) ||
this.IsBlockedByOutboundDataRestriction(item) || this.Disabled || !this.SupportsTools;
private bool IsRowDisabled(ToolCatalogItem item) => !this.IsSelectable(item) || this.Disabled || !this.SupportsTools;
private bool IsSelectable(ToolCatalogItem item) => ToolSelectionRules.IsSelectable(item, this.ProviderConfidence, this.RequiredOutboundDataRestriction.Restriction);
/// <summary>
/// Switches a tool when the user clicks anywhere in its row.
@@ -134,11 +185,7 @@ public partial class ToolSelection : MSGComponentBase
/// <summary>
/// Whether the mailboxes the chat read from keep every tool of this entry back.
/// </summary>
/// <remarks>
/// Only then is there nothing left to select. Should a mailbox keep back some tools of a
/// collection, the others still run, and the request leaves out the rest on its own.
/// </remarks>
private bool IsBlockedByOutboundDataRestriction(ToolCatalogItem item) => item.Tools.All(tool => !ToolSelectionRules.IsOutboundDataAllowed(this.RequiredOutboundDataRestriction.Restriction, tool.Implementation));
private bool IsBlockedByOutboundDataRestriction(ToolCatalogItem item) => !ToolSelectionRules.IsOutboundDataAllowed(this.RequiredOutboundDataRestriction.Restriction, item);
private string? GetProviderConfidenceHint(ToolCatalogItem item)
{
@@ -5991,9 +5991,21 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLDEFAULTSCONFIGURATION::T948842182"] =
-- Tool changes are locked while a response is running. Your current selection is shown below and applies again from the next message once the run is finished.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1688023907"] = "Werkzeugänderungen sind gesperrt, während eine Antwort ausgeführt wird. Ihre aktuelle Auswahl wird unten angezeigt und gilt nach Abschluss der Ausführung ab der nächsten Nachricht wieder."
-- Show 1 unavailable tool
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1852398429"] = "1 nicht verfügbares Werkzeug anzeigen"
-- Tools allow the LLM to perform targeted additional actions such as web searches or reading web pages.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1944689297"] = "Werkzeuge ermöglichen es dem LLM, gezielte zusätzliche Aktionen auszuführen, wie z. B. Websuchen oder das Lesen von Webseiten."
-- Show {0} unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2227029327"] = "{0} nicht verfügbare Werkzeuge anzeigen"
-- Hide unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2707194504"] = "Nicht verfügbare Werkzeuge ausblenden"
-- Show {0} unavailable tools, {1} of them selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T30104720"] = "{0} nicht verfügbare Werkzeuge anzeigen, davon {1} ausgewählt"
-- Required settings are missing. Configure this tool before enabling it.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3119156561"] = "Erforderliche Einstellungen fehlen. Konfigurieren Sie dieses Werkzeug, bevor Sie es aktivieren."
@@ -6009,9 +6021,18 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3904490680"] = "Keine Wer
-- This tool requires provider confidence {0}. The selected provider has {1}.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T4097602620"] = "Dieses Werkzeug erfordert Anbieter-Vertrauen {0}. Der ausgewählte Anbieter hat {1}."
-- Show 1 unavailable tool, which is selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T641009275"] = "1 nicht verfügbares Werkzeug anzeigen, das ausgewählt ist"
-- You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T732483684"] = "Werkzeuge, die Sie nie benötigen, können Sie in den App-Einstellungen unter „Werkzeugeinstellungen“ deaktivieren. Sie erscheinen dann hier nicht mehr."
-- Tool Selection
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T749664565"] = "Werkzeugauswahl"
-- No tool can be selected right now.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T953828816"] = "Derzeit kann kein Werkzeug ausgewählt werden."
-- Select tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T998515990"] = "Werkzeuge auswählen"
@@ -5991,9 +5991,21 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLDEFAULTSCONFIGURATION::T948842182"] =
-- Tool changes are locked while a response is running. Your current selection is shown below and applies again from the next message once the run is finished.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1688023907"] = "Tool changes are locked while a response is running. Your current selection is shown below and applies again from the next message once the run is finished."
-- Show 1 unavailable tool
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1852398429"] = "Show 1 unavailable tool"
-- Tools allow the LLM to perform targeted additional actions such as web searches or reading web pages.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T1944689297"] = "Tools allow the LLM to perform targeted additional actions such as web searches or reading web pages."
-- Show {0} unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2227029327"] = "Show {0} unavailable tools"
-- Hide unavailable tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T2707194504"] = "Hide unavailable tools"
-- Show {0} unavailable tools, {1} of them selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T30104720"] = "Show {0} unavailable tools, {1} of them selected"
-- Required settings are missing. Configure this tool before enabling it.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3119156561"] = "Required settings are missing. Configure this tool before enabling it."
@@ -6009,9 +6021,18 @@ UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T3904490680"] = "No tools
-- This tool requires provider confidence {0}. The selected provider has {1}.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T4097602620"] = "This tool requires provider confidence {0}. The selected provider has {1}."
-- Show 1 unavailable tool, which is selected
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T641009275"] = "Show 1 unavailable tool, which is selected"
-- You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T732483684"] = "You can switch off tools you never need in the app settings, under Tool Settings. They then no longer appear here."
-- Tool Selection
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T749664565"] = "Tool Selection"
-- No tool can be selected right now.
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T953828816"] = "No tool can be selected right now."
-- Select tools
UI_TEXT_CONTENT["AISTUDIO::COMPONENTS::TOOLSELECTION::T998515990"] = "Select tools"
@@ -71,4 +71,38 @@ public static class ToolSelectionRules
restriction is OutboundDataRestriction.UNRESTRICTED ||
implementation.OutboundData is ToolOutboundData.NONE or ToolOutboundData.CONFIGURED_SERVICE ||
implementation.EnforcesOutboundDataRestriction;
/// <summary>
/// Whether an entry of the catalog has a tool left which may run in a chat with this outbound data restriction.
/// </summary>
/// <remarks>
/// Only when the restriction keeps every tool of a collection back is there nothing left to
/// select. Should it keep back some of them, the others still run, and the request leaves out
/// the rest on its own.
/// </remarks>
/// <param name="restriction">Where the chat may still send data.</param>
/// <param name="item">The entry of the catalog.</param>
/// <returns>True when at least one tool of the entry may run.</returns>
public static bool IsOutboundDataAllowed(OutboundDataRestriction restriction, ToolCatalogItem item) =>
item.Tools.Any(tool => IsOutboundDataAllowed(restriction, tool.Implementation));
/// <summary>
/// Whether the user can select an entry of the catalog right now.
/// </summary>
/// <remarks>
/// The tool selection shows the entries this denies apart from the others, together with the
/// reason, so the answer has to cover everything that lies with the entry itself: it must not be
/// switched off, its settings must be complete, the provider must have the confidence it needs,
/// and the outbound data restriction of the chat must leave one of its tools. What applies to all
/// entries alike, such as a running response or a model without tools, stays with the caller.
/// </remarks>
/// <param name="item">The entry of the catalog.</param>
/// <param name="providerConfidence">The confidence of the selected provider.</param>
/// <param name="restriction">Where the chat may still send data.</param>
/// <returns>True when the entry can be selected.</returns>
public static bool IsSelectable(ToolCatalogItem item, ConfidenceLevel providerConfidence, OutboundDataRestriction restriction) =>
item.IsActive &&
item.ConfigurationState.IsConfigured &&
IsProviderConfidenceAllowed(providerConfidence, item.MinimumProviderConfidence) &&
IsOutboundDataAllowed(restriction, item);
}
@@ -73,6 +73,7 @@
- Improved updating the index of your local data sources, thanks to Paul Koudelka. AI Studio no longer reads a file again that was saved without changes, and of a changed document, it sends only the changed parts to your embedding provider, which saves time and costs.
- Improved searching your local data sources while AI Studio updates their index. A document that changed stays searchable in its previous version until its new version is complete. This improvement comes from Paul Koudelka.
- Improved reading Word documents, for example, in your local data sources. AI Studio now leaves out their custom properties, such as the internal IDs of SharePoint, which told the AI nothing and made searches less precise. Thanks, Paul Koudelka.
- Improved the tool selection in chats and assistants, which now lists only the tools you can select right now. Tools you cannot select at the moment, for example, because a setting is missing, wait behind a button below the list, together with the reason. The button also tells you how many of them you have selected.
- Changed the banner on the start page to a fresh design that shows what AI Studio can do today.
- Changed when a chat you add to a workspace in the workspace list appears there. Like any other new chat, it now appears once there is something to keep, for example, your first message, instead of right away.
- Changed the name of the Slide Planner Assistant to Slide Assistant. With a PowerPoint template, it now creates the slides themselves, not just a plan for them.
@@ -0,0 +1,117 @@
using AIStudio.Provider;
using AIStudio.Settings.DataModel;
using AIStudio.Tools.ToolCallingSystem;
namespace AIStudio.Tests.Tools.ToolCalling;
/// <summary>
/// Checks which entries of the catalog the tool selection offers for selecting right now.
/// </summary>
/// <remarks>
/// The tool selection shows these entries first and keeps the others behind a button, together
/// with the reason. An entry wrongly counted as selectable would sit at the top without reacting to
/// a click; one wrongly counted as unavailable would hide a tool the user could have picked.
/// </remarks>
[TestFixture]
[NonParallelizable]
public sealed class ToolSelectableTests : ToolRegistryTestBase
{
private const string COLLECTION_ID = "test_collection";
private const string SEARCH_TOOL_ID = "test_search";
private const string READ_TOOL_ID = "test_read";
[Test]
public async Task NothingInTheWay()
{
var item = await this.GetItemAsync(new TestTool(Definition()));
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.True);
}
[Test]
public async Task ToolsSwitchedOffAltogether()
{
this.SettingsManager.ConfigurationData.Tools.EnableTools = false;
var item = await this.GetItemAsync(new TestTool(Definition()));
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.False, "The organization turned all tools off.");
}
[Test]
public async Task AToolSwitchedOffByTheOrganization()
{
this.SettingsManager.ConfigurationData.Tools.DisabledToolIds.Add(TOOL_ID);
var item = await this.GetItemAsync(new TestTool(Definition()));
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.False);
}
[Test]
public async Task AToolSwitchedOffByTheUser()
{
this.SettingsManager.ConfigurationData.Tools.UserDisabledToolIds.Add(TOOL_ID);
var item = await this.GetItemAsync(new TestTool(Definition()));
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.False, "The selection never lists such a tool, but the answer must not depend on that.");
}
[Test]
public async Task AToolMissingASetting()
{
var item = await this.GetItemAsync(new TestTool(Definition(requiresSetting: true)));
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.False, "The tool cannot work without a setting nobody filled in.");
}
[TestCase(ConfidenceLevel.UNKNOWN, false)]
[TestCase(ConfidenceLevel.MEDIUM, false)]
[TestCase(ConfidenceLevel.HIGH, true)]
public async Task TheProviderNeedsTheConfidenceOfTheTool(ConfidenceLevel providerConfidence, bool expected)
{
var item = await this.GetItemAsync(new TestTool(Definition(minimumConfidence: ConfidenceLevel.HIGH)));
Assert.That(ToolSelectionRules.IsSelectable(item, providerConfidence, OutboundDataRestriction.UNRESTRICTED), Is.EqualTo(expected), "The tool asks for high confidence.");
}
[Test]
public async Task ARestrictionKeepingSomeToolsOfACollectionBack()
{
var item = await this.GetCollectionItemAsync(readToolOutboundData: ToolOutboundData.CONFIGURED_SERVICE);
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.ONLY_CONFIGURED_SERVICES), Is.True, "The reader still runs, and the request leaves out the search on its own.");
}
[Test]
public async Task ARestrictionKeepingEveryToolOfACollectionBack()
{
var item = await this.GetCollectionItemAsync(readToolOutboundData: ToolOutboundData.MODEL_CHOSEN_ADDRESSES);
Assert.Multiple(() =>
{
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.ONLY_CONFIGURED_SERVICES), Is.False, "Nothing of the collection is left to run.");
Assert.That(ToolSelectionRules.IsSelectable(item, ConfidenceLevel.HIGH, OutboundDataRestriction.UNRESTRICTED), Is.True, "Without a restriction, both tools run.");
});
}
private async Task<ToolCatalogItem> GetItemAsync(TestTool tool)
{
var item = await this.CreateRegistry(tool).GetCatalogItemAsync(TOOL_ID);
Assert.That(item, Is.Not.Null);
return item!;
}
/// <summary>
/// A collection of a search which queries third parties and a reader whose outbound data the test chooses.
/// </summary>
private async Task<ToolCatalogItem> GetCollectionItemAsync(ToolOutboundData readToolOutboundData)
{
var registry = this.CreateRegistry(
[new TestCollection(COLLECTION_ID, ConfidenceLevel.NONE, SEARCH_TOOL_ID, READ_TOOL_ID)],
new TestTool(Definition(SEARCH_TOOL_ID)) { OutboundData = ToolOutboundData.THIRD_PARTY_QUERIES },
new TestTool(Definition(READ_TOOL_ID)) { OutboundData = readToolOutboundData });
var item = await registry.GetCatalogItemAsync(COLLECTION_ID);
Assert.That(item, Is.Not.Null);
return item!;
}
}
+2
View File
@@ -128,6 +128,8 @@ What the user switched off disappears; what the organization switched off stays
Switching a collection off only hides it. A chat, a chat template, a document analysis policy, the defaults of a component, or an assistant plugin which names it keeps naming it, and switching it on again brings it back there. A selection field therefore shows a selection through `ToolRegistry.LeaveOutSwitchedOffByUser` and stores a change through `ToolRegistry.KeepSwitchedOffByUser`, which puts the switched-off collections back. Where the rules of an assistant name a switched-off tool, `ManagedToolsWarning` says that the run goes without it. Switching off `semantic_search` has the data sources searched with every message, as when an organization does it.
A tool which merely cannot be selected right now is a different matter: the organization switched it off, its settings are missing, the provider lacks the confidence it needs, or the outbound data restriction of the chat keeps all of its tools back. The tool selection lists such a tool behind a button below the others, with the reason and its settings button: the user may fill in the settings, or the tool works again with another provider or in another chat. `ToolSelectionRules.IsSelectable` decides which list a tool belongs to; a selected tool which cannot be selected right now goes behind the button as well, and the button counts it. While a response is running, nothing can be switched at all, and the selection shows all tools in one list.
## Security
Treat model-provided tool arguments as untrusted input. Refuse a wrong one rather than guessing what it meant: a placeholder such as `0` is not a page, and reading it as "no page" does something the model did not ask for. The model reads the refusal and tries again, so the message has to name the argument and the value that arrived, say what would be valid, and, for an optional argument, that leaving it out is always possible. `ToolArgumentReader` reads strings, positive integers, and values out of a fixed choice, alone or as a list, and words the refusals so; `WebSearchTool` shows how a tool uses it.