Improved how the AI remembers its earlier tool calls (#1185)

Co-authored-by: Thorsten Sommer <SommerEngineering@users.noreply.github.com>

documentation/Tools.md

@@ -100,7 +100,7 @@ Whoever decides something on behalf of a request asks the registry rather than c
 
 A tool whose results come in pages takes a `page` argument starting at 1 and reports `has_more` with each result, never a total. A total is often unknown — a vector search has none, since every chunk matches, only less closely — and a number known for some sources and not for others invites the model to page through all of them. `semantic_search` works this way; `web_search` takes a `page` as well.
 
-Paging stays stateless: every call brings its query and its page again. It has to, because tool results do not travel into later turns. `ToolInvocationTrace.Result` is not saved, and the tool conversation of a request is gone once the answer stands. Cap how deep a model may page, since every page fetches its whole window again, and refuse a page beyond the cap with the last page there is in the message.
+Paging stays stateless: every call brings its query and its page again. It has to, because tool results do not travel into later turns. `ToolInvocationTrace.Result` is not saved, and the tool conversation of a request is gone once the answer stands. What does travel is a note in front of every earlier answer, see `ToolCallHistoryNote`: the calls with their arguments, written with the JSON types the model passed, which `ToolInvocationTrace.ArgumentsJson` keeps apart from the texts the user reads, the calls which failed or were blocked by name only, the sources by origin and web address, and when the answer was written, but never the content. A source without a web address, such as a local file, a passage of a data source, or a mail, stays out: no tool takes its address, and a data source builds it from text which the filter never saw. `ListContentBlockExtensions.BuildMessagesAsync` puts it there for the request only, so the stored answer stays as it was, and `ConversationParts` counts it the same way. The system prompt explains the note once a conversation holds one, without any web address, since those would count as given to the model. A model which needs the content again calls the tool again. Cap how deep a model may page, since every page fetches its whole window again, and refuse a page beyond the cap with the last page there is in the message.
 
 A page with a budget of characters ends right before the first result which no longer fits, and the next page starts with that result, see `CharacterPaging`. Never skip a long result to make room for shorter ones after it: the gap would sit in the middle of the ranking, and since every page is cut anew, no later page would ever bring the result back. A page is therefore worked out from the first one on, every time, and only a result too long for any page is shortened, then filtered again. `semantic_search` cuts its pages this way, see "Searching Data Sources" below.
 
@@ -180,9 +180,9 @@ Every successfully retrieved page with readable content is also returned as a st
 
 `read_web_page.freeAddressChoice` decides whether the model may read addresses it chose itself. `OFF`, the default, tells the model to read only URLs which appear word for word in the conversation: in the system prompt, in a user message with the documents and data source content it carries, or in a tool result. When none fits and no other tool can find one, the model asks the user. `ON` lets it choose addresses as well. The values are the members of `FreeAddressChoice`, offered through `ToolSettingsOptionSources.FREE_ADDRESS_CHOICE`, and the setting follows the usual precedence of tool settings: a locked organization value, then the user's saved value, then an organization default.
 
-`OFF` is enforced as well: `read_web_page` refuses an address which was not given to the model, see `ChatThread.IsWebAddressGivenToTheModel`. Given means that the address stands in the system prompt the last request was sent with, in a user message or a document attached to it, or in the result of a tool. What the model wrote itself never counts, its earlier answers included. Neither does what any model wrote when its text reaches a chat as a user message, such as an answer sent to an assistant or into a new chat: the chat keeps the request keys of such addresses in `ChatThread.WebAddressesWrittenByModels`, saves them along, and takes them over with the other restrictions, see `ChatRestrictions.WithWebAddressesWrittenByModelIn`. Each source is collected where its text exists in full: the system prompt in `PrepareSystemPrompt`, the attached documents in `ContentText.PrepareTextContentForAI`, since they are read from disk only when a message is sent, and the tool results in `ToolExecutor`. Relative links need no care of their own, because the page extraction makes every link absolute before the model sees it. Two addresses count as the same when they ask the server for the same, see `WebAddresses.CreateRequestKey`: scheme and host regardless of case, path and query exactly, the fragment not at all. A redirect may go anywhere, since the server rather than the model chose it.
+`OFF` is enforced as well: `read_web_page` refuses an address which was not given to the model, see `ChatThread.IsWebAddressGivenToTheModel`. Given means that the address stands in the system prompt the last request was sent with, in a user message or a document attached to it, or in the result of a tool. What the model wrote itself never counts, its earlier answers included. Neither does what any model wrote when its text reaches a chat as a user message, such as an answer sent to an assistant or into a new chat: the chat keeps the request keys of such addresses in `ChatThread.WebAddressesWrittenByModels`, saves them along, and takes them over with the other restrictions, see `ChatRestrictions.WithWebAddressesWrittenByModelIn`. Each source is collected where its text exists in full: the system prompt in `PrepareSystemPrompt`, the attached documents in `ContentText.PrepareTextContentForAI`, since they are read from disk only when a message is sent, and the tool results in `ToolExecutor`. Of the addresses a tool returned, the answer they came about for stores those which a later message still shows the model: in its text, among its sources, or in the arguments of a call which ran. They go into `ContentText.WebAddressesFromTools` when the request ends, see `ContentText.EndToolRun`, so that the model may read them again in a later message, also after a restart: a tool returned them, so the model neither made them up nor wrote anything of the chat into them. The other addresses, such as the hundreds of links a page may hold, would only make the stored chat grow: once the result is gone, the model cannot know them without reading the page again, which returns them once more. While the request runs, they wait in `ContentText.PendingWebAddressesFromTools`, which is never stored, since a chat is saved while its answer is still being written. They go with the answer: regenerating it removes them, and so does deleting it once the chat is opened again, and sending its text on, as a `TextLaunchPayload`, does not carry them along. A copy of the whole answer, such as an assistant's conversation carried into a chat, keeps them, since a tool returned them in that conversation. Relative links need no care of their own, because the page extraction makes every link absolute before the model sees it. Two addresses count as the same when they ask the server for the same, see `WebAddresses.CreateRequestKey`: scheme and host regardless of case, path and query exactly, the fragment not at all. A redirect may go anywhere, since the server rather than the model chose it.
 
-An address in a tool result counts only when it is no echo of the call: one which stands in the arguments, even as a part of one, is left out, because Semantic Search returns its query, and the model could otherwise turn any address it makes up into one a tool returned. The tool results are kept for the session only, like the results themselves, so after a restart a chat knows fewer addresses, never more. A refusal never repeats the address, for the same reason. On top of this, the network target restrictions of `WebPageRetrievalService` and the prompt-injection filter apply with both values.
+An address in a tool result counts only when it is no echo of the call: one which stands in the arguments, even as a part of one, is left out, because Semantic Search returns its query, and the model could otherwise turn any address it makes up into one a tool returned. Beyond what each answer stores, the chat keeps the addresses of all tool results for the session only, in `ChatThread.RuntimeWebAddressesFromTools`: those of runs without an answer in the chat, such as the runs of the assistants, those which no later message shows, and those of an answer deleted meanwhile. After a restart, a chat therefore knows fewer addresses, never more. A refusal never repeats the address, for the same reason. On top of this, the network target restrictions of `WebPageRetrievalService` and the prompt-injection filter apply with both values.
 
 Links in a tool result count as given with both values, a link on a page read before included. Searching and then reading what was found is what these tools are for, and `search_confluence` opens its hits that way. Before this setting existed, the instructions forbade following a link which only retrieved content mentioned; that rule was dropped on purpose. Following a link word for word cannot carry anything out of the conversation. Putting parts of the conversation into an address could, so the instructions forbid that with both values.