Added a semantic search tool (#1005)

app/MindWork AI Studio/wwwroot/changelog/v26.9.1.md

@@ -25,7 +25,7 @@
 - Added a warning when your conversation holds more images than the model accepts, wherever we know that limit. The Visual Briefing assistant stops before anything is uploaded, instead of letting the provider refuse it afterward.
 - Added the context window and the image limits to the expert provider settings, next to the abilities you could already state there. Leave a field empty, and AI Studio keeps its own answer, which you see as the placeholder. IT departments can state the same numbers for the providers they roll out.
 - Added model plugins, so IT departments can describe the models their organization runs itself.
-- Added local RAG as a beta feature, so the AI can answer from your own documents. You point AI Studio at a folder or at a single file, and it prepares those documents in the background so their contents can be found again later. Ask a question with such a data source selected, and AI Studio looks for the passages that fit your question and hands only those to the model, along with where each one came from. We will keep developing it together with the people who use it: to try it, open the app settings, allow preview features down to beta, and then enable the RAG feature. Many thanks to Paul Koudelka (`PaulKoudelka`) for around ten months of work on the concept and the implementation.
+- Added local RAG as a beta feature, so the AI can answer from your own documents. You point AI Studio at a folder or at a single file, and it prepares those documents in the background so they can be searched. Ask a question with such a data source selected, and only the passages that fit your question reach the model, along with where each one came from. We will keep developing it together with the people who use it: to try it, open the app settings, allow preview features down to beta, and then enable the RAG feature. Many thanks to Paul Koudelka (`PaulKoudelka`) for around ten months of work on the concept and the implementation.
 - Added the setup for local data sources. You pick an embedding provider, and AI Studio asks for your confirmation before any document goes to a cloud service. It keeps up with your files as they change, shows the progress on a page of its own, and checks every document for hidden instructions before indexing it. Documents without readable text, such as scanned pages, are remembered as such, so AI Studio does not work through them again after every start — it comes back to them once they change.
 - Added a way to open the sources of your own documents: click a source below an answer, and the document opens in the program your system uses for it.
 - Added a jump to the right page for the sources of your own documents (RAG), so a PDF opens directly where the passage was found, wherever your system and its program support it.
@@ -33,6 +33,9 @@
 - Added the details of the two databases behind local RAG to the information page: which versions they run, how much space they use on your disk, and how much they hold.
 - Added a repair for your local data sources. Should the index of a data source ever become unreadable, AI Studio now says so instead of quietly finding nothing and leaves that source out of your chats until it works again.
 - Added the repair itself as a button next to each of your data sources. Rebuilding an index sends your documents to your embedding provider once more, so AI Studio asks you first and never starts it on its own.
+- Added Semantic Search: when your model can use tools, it now searches your data sources itself — only when a question calls for it, as often as it takes, and with search terms it works out from your conversation.
+- Added a choice per chat of how your data sources are used: the AI searches them itself (Semantic Search), or AI Studio searches them with every message you send (classic RAG). Models that cannot use tools always use classic RAG, and the data source selection tells you when that happens.
+- Added organization-wide management for how the data sources of new chats are searched. IT departments can make it a default or lock it.
 - Added a check that recognizes your own tokenizer by its content rather than by its file name. Tokenizers are almost always named alike, so swapping one for another is now noticed, and you are asked about it.
 - Added support for several drop areas on the same page. More complex assistants can now receive files or folders by drag and drop at more than one place.
 - Added drag and drop to the input and output folder of the Batch Processing assistant: drop a folder onto either field to choose it.
@@ -86,6 +89,7 @@
 - Fixed data sources you picked for your chats vanishing from the selection without a word when they cannot be used. AI Studio now lists them by name, so you can see why an answer was created without them.
 - Fixed the data sources you picked for a chat being forgotten the moment you changed your selection while one of them could not be used. Such a source stays selected and is used again as soon as it is available.
 - Fixed the silence when the step that picks the fitting passages out of your documents fails. You are told that the answer rests on everything that was found.
+- Fixed passages from your data sources still reaching the AI after you switched the data sources of a chat off. Likewise, when a search in them found nothing, the AI kept receiving the passages an earlier message had brought up.
 - Fixed the regenerate button taking an answer away without producing a new one. This happened in chats started from a template that holds no question of your own.
 - Fixed the counter above an answer, which shows how many sources it rests on, doing nothing when you clicked it. It now takes you down to the sources.
 - Fixed the list of models staying empty at a server you host yourself, which made the model you had picked look as if it had vanished. Your key was there all along, it just was not read when the settings opened.