diff --git a/app/MindWork AI Studio/Plugins/assistants/README.md b/app/MindWork AI Studio/Plugins/assistants/README.md index 15e0d1d7..1e680814 100644 --- a/app/MindWork AI Studio/Plugins/assistants/README.md +++ b/app/MindWork AI Studio/Plugins/assistants/README.md @@ -2,12 +2,32 @@ This folder keeps the Lua manifest (`plugin.lua`) that defines a custom assistant. Treat it as the single source of truth for how AI Studio renders your assistant UI and builds the submitted prompt. +## Table of Contents + +## How to use this Documention +Fill our here + +## Directory Structure + +``` +. +└── com.github.mindwork-ai.ai-studio/ + └── data/ + └── plugins/ + └── assistants/ + └── your-assistant-directory/ + ├── assets/ + │ └── your-media-files.jpg + ├── icon.lua + └── plugin.lua +``` + ## Structure - `ASSISTANT` is the root table. It must contain `Title`, `Description`, `SystemPrompt`, `SubmitText`, `AllowProfiles`, and the nested `UI` definition. - `UI.Type` is always `"FORM"` and `UI.Children` is a list of component tables. - Each component table declares `Type`, an optional `Children` array, and a `Props` table that feeds the component’s parameters. -Supported types (matching the Blazor UI components): +#### Supported types (matching the Blazor UI components): - `TEXT_AREA`: user input field based on `MudTextField`; requires `Name`, `Label`, and may include `HelperText`, `HelperTextOnFocus`, `Adornment`, `AdornmentIcon`, `AdornmentText`, `AdornmentColor`, `Counter`, `MaxLength`, `IsImmediate`, `UserPrompt`, `PrefillText`, `IsSingleLine`, `ReadOnly`, `Class`, `Style`. - `DROPDOWN`: selects between variants; `Props` must include `Name`, `Label`, `Default`, `Items`, and optionally `ValueType` plus `UserPrompt`. @@ -27,99 +47,9 @@ Supported types (matching the Blazor UI components): Images referenced via the `plugin://` scheme must exist in the plugin directory (e.g., `assets/example.png`). Drop the file there and point `Src` at it. The component will read the file at runtime, encode it as Base64, and render it inside the assistant UI. -## Prompt Assembly -Each component exposes a `UserPrompt` string. When the assistant runs, `AssistantDynamic` recursively iterates over the component tree and, for each component that has a prompt, emits: +More information on rendered components can be found [here](https://www.mudblazor.com/docs/overview). -``` -context: - ---- -user prompt: - -``` - -For switches the “value” is the boolean `true/false`; for readers it is the fetched/selected content; for color pickers it is the selected color text (for example `#FFAA00` or `rgba(...)`, depending on the picker mode). Always provide a meaningful `UserPrompt` so the final concatenated prompt remains coherent from the LLM’s perspective. - -### Advanced: BuildPrompt (optional) -If you want full control over prompt composition, define `ASSISTANT.BuildPrompt` as a Lua function. When present, AI Studio calls it and uses its return value as the final user prompt. The default prompt assembly is skipped. - -#### Contract -- `ASSISTANT.BuildPrompt(input)` must return a **string**. -- If the function is missing, returns `nil`, or returns a non-string, AI Studio falls back to the default prompt assembly. -- Errors in the function are caught and logged, then fall back to the default prompt assembly. - -#### Input table shape -The function receives a single `input` table with: -- `input.fields`: values keyed by component `Name` - - Text area, dropdown, and readers are strings - - Switch is a boolean - - Color picker is the selected color as a string -- `input.meta`: per-component metadata keyed by component `Name` - - `Type` (string, e.g. `TEXT_AREA`, `DROPDOWN`, `SWITCH`, `COLOR_PICKER`) - - `Label` (string, when provided) - - `UserPrompt` (string, when provided) -- `input.profile`: selected profile data - - `Id`, `Name`, `NeedToKnow`, `Actions`, `Num` - - When no profile is selected, values match the built-in "Use no profile" entry - -#### Table shapes (quick reference) -``` -input = { - fields = { - [""] = "", - ... - }, - meta = { - [""] = { - Type = "", - Label = "", - UserPrompt = "" - }, - ... - }, - profile = { - Name = "", - NeedToKnow = "", - Actions = "", - Num = - } -} -``` - -#### Using `meta` inside BuildPrompt -`input.meta` is useful when you want to dynamically build the prompt based on component type or reuse existing UI text (labels/user prompts). - -Example: iterate all fields with labels and include their values -```lua -ASSISTANT.BuildPrompt = function(input) - local parts = {} - for name, value in pairs(input.fields) do - local meta = input.meta[name] - if meta and meta.Label and value ~= "" then - table.insert(parts, meta.Label .. ": " .. tostring(value)) - end - end - return table.concat(parts, "\n") -end -``` - -Example: handle types differently -```lua -ASSISTANT.BuildPrompt = function(input) - local parts = {} - for name, meta in pairs(input.meta) do - local value = input.fields[name] - if meta.Type == "SWITCH" then - table.insert(parts, name .. ": " .. tostring(value)) - elseif meta.Type == "COLOR_PICKER" and value and value ~= "" then - table.insert(parts, name .. ": " .. value) - elseif value and value ~= "" then - table.insert(parts, name .. ": " .. value) - end - end - return table.concat(parts, "\n") -end -``` +## Component References ### `TEXT_AREA` reference - Use `Type = "TEXT_AREA"` to render a MudBlazor text input or textarea. @@ -142,7 +72,7 @@ end - `ReadOnly`: defaults to `false`; disables editing. - `Class`, `Style`: forwarded to the rendered component for layout/styling. -Example: +#### Example: ```lua { ["Type"] = "TEXT_AREA", @@ -162,6 +92,7 @@ Example: } } ``` +--- ### `BUTTON` reference - Use `Type = "BUTTON"` to render a clickable action button. @@ -180,7 +111,7 @@ Example: - `IconSize`: one of the MudBlazor `Size` enum names; omitted values fall back to `Medium`. - `Class`, `Style`: forwarded to the rendered component for layout/styling. -#### `Action(input)` contract +#### `Action(input)` interface - The function receives the same `input` structure as `ASSISTANT.BuildPrompt(input)`. - Return `nil` for no state update. - To update component state, return a table with a `fields` table. @@ -190,7 +121,7 @@ Example: - `SWITCH`: boolean values - Unknown field names and wrong value types are ignored and logged. -Example: +#### Example: ```lua { ["Type"] = "BUTTON", @@ -211,12 +142,12 @@ Example: local output = email if translate then - output = output .. "\n\nTranslate this email." + output = output .. "\n\nTranslate this email:" end return { fields = { - outputBuffer = output + outputTextField = output } } end, @@ -225,6 +156,7 @@ Example: } } ``` +--- ### `BUTTON_GROUP` reference - Use `Type = "BUTTON_GROUP"` to render multiple `BUTTON` children as a single MudBlazor button group. @@ -240,7 +172,7 @@ Example: - `Class`, `Style`: forwarded to the rendered `MudButtonGroup` for layout/styling. - Child buttons use the existing `BUTTON` props and behavior, including Lua `Action(input)`. -Example: +#### Example: ```lua { ["Type"] = "BUTTON_GROUP", @@ -283,6 +215,193 @@ Example: } } ``` +--- + +### `SWITCH` reference +- Use `Type = "SWITCH"` to render a boolean toggle. +- Required props: + - `Name`: unique state key used in prompt assembly and `BuildPrompt(input.fields)`. + - `Label`: visible label for the switch field. + - `Value`: initial boolean state (`true` or `false`). +- Optional props: + - `Disabled`: defaults to `false`; disables user interaction while still allowing the value to be included in prompt assembly. + - `UserPrompt`: prompt context text for this field. + - `LabelOn`: text shown when the switch value is `true`. + - `LabelOff`: text shown when the switch value is `false`. + - `LabelPlacement`: one of `Bottom`, `End`, `Left`, `Right`, `Start`, `Top`; omitted values follow the renderer default. + - `Icon`: MudBlazor icon identifier string displayed inside the switch thumb. + - `IconColor`: one of the MudBlazor `Color` enum names such as `Primary`, `Secondary`, `Warning`; omitted values default to `Inherit`. + - `CheckedColor`: color used when the switch state is `true`; omitted values default to `Inherit`. + - `UncheckedColor`: color used when the switch state is `false`; omitted values default to `Inherit`. + - `Class`, `Style`: forwarded to the rendered component for layout/styling. + +#### Example: +```lua +{ + ["Type"] = "SWITCH", + ["Props"] = { + ["Name"] = "IncludeSummary", + ["Label"] = "Include summary", + ["Value"] = true, + ["Disabled"] = false, + ["UserPrompt"] = "Decide whether the final answer should include a short summary.", + ["LabelOn"] = "Summary enabled", + ["LabelOff"] = "Summary disabled", + ["LabelPlacement"] = "End", + ["Icon"] = "Icons.Material.Filled.Summarize", + ["IconColor"] = "Primary", + ["CheckedColor"] = "Success", + ["UncheckedColor"] = "Default", + ["Class"] = "mb-6", + } +} +``` +--- + +### `COLOR_PICKER` reference +- Use `Type = "COLOR_PICKER"` to render a MudBlazor color picker. +- Required props: + - `Name`: unique state key used in prompt assembly and `BuildPrompt(input.fields)`. + - `Label`: visible field label. +- Optional props: + - `Placeholder`: default color hex string (e.g. `#FF10FF`) or initial hint text. + - `ShowAlpha`: defaults to `true`; enables alpha channel editing. + - `ShowToolbar`: defaults to `true`; shows picker/grid/palette toolbar. + - `ShowModeSwitch`: defaults to `true`; allows switching between HEX/RGB(A)/HSL modes. + - `PickerVariant`: one of `DIALOG`, `INLINE`, `STATIC`; invalid or omitted values fall back to `STATIC`. + - `UserPrompt`: prompt context text for the selected color. + - `Class`, `Style`: forwarded to the rendered component for layout/styling. + +#### Example: +```lua +{ + ["Type"] = "COLOR_PICKER", + ["Props"] = { + ["Name"] = "accentColor", + ["Label"] = "Accent color", + ["Placeholder"] = "#FFAA00", + ["ShowAlpha"] = false, + ["ShowToolbar"] = true, + ["ShowModeSwitch"] = true, + ["PickerVariant"] = "STATIC", + ["UserPrompt"] = "Use this as the accent color for the generated design." + } +} +``` + +## Prompt Assembly - UserPrompt Property +Each component exposes a `UserPrompt` string. When the assistant runs, `AssistantDynamic` recursively iterates over the component tree and, for each component that has a prompt, emits: + +``` +context: + +--- +user prompt: + +``` + +For switches the “value” is the boolean `true/false`; for readers it is the fetched/selected content; for color pickers it is the selected color text (for example `#FFAA00` or `rgba(...)`, depending on the picker mode). Always provide a meaningful `UserPrompt` so the final concatenated prompt remains coherent from the LLM’s perspective. + +## Advanced Prompt Assembly - BuildPrompt() +If you want full control over prompt composition, define `ASSISTANT.BuildPrompt` as a Lua function. When present, AI Studio calls it and uses its return value as the final user prompt. The default prompt assembly is skipped. + +--- +### Interface +- `ASSISTANT.BuildPrompt(LuaTable input) => string` must return a **string**, the complete User Prompt. +- If the function is missing, returns `nil`, or returns a non-string, AI Studio falls back to the default prompt assembly. +- Errors in the function are caught and logged, then fall back to the default prompt assembly. +--- +### `input` table shape +The function receives a single `input` Lua table with: +- `input.fields`: values keyed by component `Name` + - Text area, dropdown, and readers are strings + - Switch is a boolean + - Color picker is the selected color as a string +- `input.meta`: per-component metadata keyed by component `Name` + - `Type` (string, e.g. `TEXT_AREA`, `DROPDOWN`, `SWITCH`, `COLOR_PICKER`) + - `Label` (string, when provided) + - `UserPrompt` (string, when provided) +- `input.profile`: selected profile data + - `Name`, `NeedToKnow`, `Actions`, `Num` + - When no profile is selected, values match the built-in "Use no profile" entry +``` +input = { + fields = { + [""] = "", + ... + }, + meta = { + [""] = { + Type = "", + Label = "", + UserPrompt = "" + }, + ... + }, + profile = { + Name = "", + NeedToKnow = "", + Actions = "", + Num = + } +} + +-- is the value you set in the components name property +``` +--- + +### Using `meta` inside BuildPrompt +`input.meta` is useful when you want to dynamically build the prompt based on component type or reuse existing UI text (labels/user prompts). + +#### Example: iterate all fields with labels and include their values +```lua +ASSISTANT.BuildPrompt = function(input) + local parts = {} + for name, value in pairs(input.fields) do + local meta = input.meta[name] + if meta and meta.Label and value ~= "" then + table.insert(parts, meta.Label .. ": " .. tostring(value)) + end + end + return table.concat(parts, "\n") +end +``` + +#### Example: handle types differently +```lua +ASSISTANT.BuildPrompt = function(input) + local parts = {} + for name, meta in pairs(input.meta) do + local value = input.fields[name] + if meta.Type == "SWITCH" then + table.insert(parts, name .. ": " .. tostring(value)) + elseif meta.Type == "COLOR_PICKER" and value and value ~= "" then + table.insert(parts, name .. ": " .. value) + elseif value and value ~= "" then + table.insert(parts, name .. ": " .. value) + end + end + return table.concat(parts, "\n") +end +``` +--- +### Using `profile` inside BuildPrompt +Profiles are optional user context (e.g., "NeedToKnow" and "Actions"). You can inject this directly into the user prompt if you want the LLM to always see it. + +#### Example: +```lua +ASSISTANT.BuildPrompt = function(input) + local parts = {} + if input.profile and input.profile.NeedToKnow ~= "" then + table.insert(parts, "User context:") + table.insert(parts, input.profile.NeedToKnow) + table.insert(parts, "") + end + table.insert(parts, input.fields.Main or "") + return table.concat(parts, "\n") +end +``` +## Advanced Layout Options ### `LAYOUT_GRID` reference - Use `Type = "LAYOUT_GRID"` to render a MudBlazor grid container. @@ -308,22 +427,32 @@ Example: { ["Type"] = "LAYOUT_ITEM", ["Props"] = { - ["Name"] = "leftItem", + ["Name"] = "contentColumn", ["Xs"] = 12, - ["Md"] = 6 - } + ["Lg"] = 8 + }, + ["Children"] = { + ["Type"] = "", + ["Props"] = {...}, + }, }, { ["Type"] = "LAYOUT_ITEM", ["Props"] = { - ["Name"] = "rightItem", + ["Name"] = "contentColumn2", ["Xs"] = 12, - ["Md"] = 6 - } - } + ["Lg"] = 8 + }, + ["Children"] = { + ["Type"] = "", + ["Props"] = {...}, + }, + }, + ... } } ``` +--- ### `LAYOUT_ITEM` reference - Use `Type = "LAYOUT_ITEM"` to render a MudBlazor grid item. @@ -344,9 +473,17 @@ Example: ["Name"] = "contentColumn", ["Xs"] = 12, ["Lg"] = 8 + }, + ["Children"] = { + { + ["Type"] = "", + ["Props"] = {...}, + }, + ... } } ``` +--- ### `LAYOUT_PAPER` reference - Use `Type = "LAYOUT_PAPER"` to render a MudBlazor paper container. @@ -369,9 +506,17 @@ Example: ["Elevation"] = 2, ["Width"] = "100%", ["IsOutlined"] = true + }, + ["Children"] = { + { + ["Type"] = "", + ["Props"] = {...}, + }, + ... } } ``` +--- ### `LAYOUT_STACK` reference - Use `Type = "LAYOUT_STACK"` to render a MudBlazor stack layout. @@ -403,94 +548,7 @@ Example: } ``` -### `SWITCH` reference -- Use `Type = "SWITCH"` to render a boolean toggle. -- Required props: - - `Name`: unique state key used in prompt assembly and `BuildPrompt(input.fields)`. - - `Label`: visible label for the switch field. - - `Value`: initial boolean state (`true` or `false`). -- Optional props: - - `Disabled`: defaults to `false`; disables user interaction while still allowing the value to be included in prompt assembly. - - `UserPrompt`: prompt context text for this field. - - `LabelOn`: text shown when the switch value is `true`. - - `LabelOff`: text shown when the switch value is `false`. - - `LabelPlacement`: one of `Bottom`, `End`, `Left`, `Right`, `Start`, `Top`; omitted values follow the renderer default. - - `Icon`: MudBlazor icon identifier string displayed inside the switch thumb. - - `IconColor`: one of the MudBlazor `Color` enum names such as `Primary`, `Secondary`, `Warning`; omitted values default to `Inherit`. - - `CheckedColor`: color used when the switch state is `true`; omitted values default to `Inherit`. - - `UncheckedColor`: color used when the switch state is `false`; omitted values default to `Inherit`. - - `Class`, `Style`: forwarded to the rendered component for layout/styling. - - -Example: -```lua -{ - ["Type"] = "SWITCH", - ["Props"] = { - ["Name"] = "IncludeSummary", - ["Label"] = "Include summary", - ["Value"] = true, - ["Disabled"] = false, - ["UserPrompt"] = "Decide whether the final answer should include a short summary.", - ["LabelOn"] = "Summary enabled", - ["LabelOff"] = "Summary disabled", - ["LabelPlacement"] = "End", - ["Icon"] = "Icons.Material.Filled.Summarize", - ["IconColor"] = "Primary", - ["CheckedColor"] = "Success", - ["UncheckedColor"] = "Default", - ["Class"] = "mb-6", - } -} -``` - -### `COLOR_PICKER` reference -- Use `Type = "COLOR_PICKER"` to render a MudBlazor color picker. -- Required props: - - `Name`: unique state key used in prompt assembly and `BuildPrompt(input.fields)`. - - `Label`: visible field label. -- Optional props: - - `Placeholder`: default color hex string (e.g. `#FF10FF`) or initial hint text. - - `ShowAlpha`: defaults to `true`; enables alpha channel editing. - - `ShowToolbar`: defaults to `true`; shows picker/grid/palette toolbar. - - `ShowModeSwitch`: defaults to `true`; allows switching between HEX/RGB(A)/HSL modes. - - `PickerVariant`: one of `DIALOG`, `INLINE`, `STATIC`; invalid or omitted values fall back to `STATIC`. - - `UserPrompt`: prompt context text for the selected color. - - `Class`, `Style`: forwarded to the rendered component for layout/styling. - -Example: -```lua -{ - ["Type"] = "COLOR_PICKER", - ["Props"] = { - ["Name"] = "accentColor", - ["Label"] = "Accent color", - ["Placeholder"] = "#FFAA00", - ["ShowAlpha"] = false, - ["ShowToolbar"] = true, - ["ShowModeSwitch"] = true, - ["PickerVariant"] = "STATIC", - ["UserPrompt"] = "Use this as the accent color for the generated design." - } -} -``` - -#### Using `profile` inside BuildPrompt -Profiles are optional user context (e.g., "NeedToKnow" and "Actions"). You can inject this directly into the user prompt if you want the LLM to always see it. - -Example: -```lua -ASSISTANT.BuildPrompt = function(input) - local parts = {} - if input.profile and input.profile.NeedToKnow ~= "" then - table.insert(parts, "User context:") - table.insert(parts, input.profile.NeedToKnow) - table.insert(parts, "") - end - table.insert(parts, input.fields.Main or "") - return table.concat(parts, "\n") -end -``` - +## Useful Functions ### Included lua libraries - [Basic Functions Library](https://www.lua.org/manual/5.2/manual.html#6.1) - [Coroutine Manipulation Library](https://www.lua.org/manual/5.2/manual.html#6.2) @@ -498,8 +556,9 @@ end - [Table Manipulation Library](https://www.lua.org/manual/5.2/manual.html#6.5) - [Mathematical Functions Library](https://www.lua.org/manual/5.2/manual.html#6.6) - [Bitwise Operations Library](https://www.lua.org/manual/5.2/manual.html#6.7) +--- -### Logging helpers (assistant plugins only) +### Logging helpers The assistant runtime exposes basic logging helpers to Lua. Use them to debug custom prompt building. - `LogDebug(message)` @@ -507,13 +566,14 @@ The assistant runtime exposes basic logging helpers to Lua. Use them to debug cu - `LogWarn(message)` - `LogError(message)` -Example: +#### Example: ```lua ASSISTANT.BuildPrompt = function(input) LogInfo("BuildPrompt called") return input.fields.Text or "" end ``` +--- ### Date/time helpers (assistant plugins only) Use these when you need timestamps inside Lua. @@ -524,7 +584,7 @@ Use these when you need timestamps inside Lua. - Members: `year`, `month`, `day`, `hour`, `minute`, `second`, `millisecond`, `formatted`. - `Timestamp()` returns a UTC timestamp in ISO-8601 format (`O` / round-trip), e.g. `2026-03-02T21:15:30.1234567Z`. -Example: +#### Example: ```lua local dt = DateTime("yyyy-MM-dd HH:mm:ss") LogInfo(dt.formatted) @@ -532,46 +592,15 @@ LogInfo(Timestamp()) LogInfo(dt.day .. "." .. dt.month .. "." .. dt.year) ``` -#### Example: simple custom prompt -```lua -ASSISTANT.BuildPrompt = function(input) - local f = input.fields - return "Topic: " .. (f.Topic or "") .. "\nDetails:\n" .. (f.Details or "") -end -``` +## General Tips -#### Example: structured prompt (similar to Coding assistant) -```lua -ASSISTANT.BuildPrompt = function(input) - local f = input.fields - local parts = {} - - if (f.Code or "") ~= "" then - table.insert(parts, "I have the following code:") - table.insert(parts, "```") - table.insert(parts, f.Code) - table.insert(parts, "```") - table.insert(parts, "") - end - - if (f.CompilerMessages or "") ~= "" then - table.insert(parts, "I have the following compiler messages:") - table.insert(parts, "```") - table.insert(parts, f.CompilerMessages) - table.insert(parts, "```") - table.insert(parts, "") - end - - table.insert(parts, "My questions are:") - table.insert(parts, f.Questions or "") - return table.concat(parts, "\n") -end -``` - -# Tips - -1. Give every component a unique `Name`— it’s used to track state. -2. Keep in mind that components and their properties are case-sensitive (e.g. if you write `["Type"] = "heading"` instead of `["Type"] = "HEADING"` the component will not be registered). Always copy-paste the component from the `plugin.lua` manifest to avoid this. +1. Give every component a _**unique**_ `Name`— it’s used to track state and treated like an Id. +2. Keep in mind that components and their properties are _**case-sensitive**_ (e.g. if you write `["Type"] = "heading"` instead of `["Type"] = "HEADING"` the component will not be registered). Always copy-paste the component from the `plugin.lua` manifest to avoid this. 3. When you expect default content (e.g., a textarea with instructions), keep `UserPrompt` but also set `PrefillText` so the user starts with a hint. 4. If you need extra explanatory text (before or after the interactive controls), use `TEXT` or `HEADING` components. 5. Keep `Preselect`/`PreselectContentCleanerAgent` flags in `WEB_CONTENT_READER` to simplify the initial UI for the user. + +## Useful Resources +- [plugin.lua - Lua Manifest](https://github.com/MindWorkAI/AI-Studio/tree/main/app/MindWork%20AI%20Studio/Plugins/assistants/plugin.lua) +- [Lua 5.2 Reference Manual](https://www.lua.org/manual/5.2/manual.html) +- [MudBlazor Documentation](https://www.mudblazor.com/docs/overview)