Commit Graph
47 Commits
Author SHA1 Message Date
PaulKoudelka 88999d985c refactored confidence levels and ensured checks work 2026-08-14 12:03:16 +02:00
PaulKoudelka 786b106096 made datasource selection more user friendly 2026-08-13 15:51:28 +02:00
Paul Koudelka 647c428ebf improved code and made sure components are reused if possible 2026-08-12 14:19:32 +02:00
Paul Koudelka d95487aa6e disabled images for RAG for now 2026-08-12 11:43:47 +02:00
Paul Koudelka c92e83ee59 made datasources button an icon 2026-08-12 11:27:27 +02:00
Paul Koudelka 987f493848 made check for vectordb more robust 2026-08-11 15:55:12 +02:00
Paul Koudelka 92e986d9b9 ensured data sources can be renamed and made it secure 2026-08-11 15:22:07 +02:00
Paul Koudelka 202e37410b moved datasource indexing to lower level 2026-08-10 23:12:55 +02:00
Paul Koudelka b4e916e166 fixed sqlite info 2026-08-10 22:26:06 +02:00
Paul Koudelka 2547701320 ensured consistent maximum token limit 2026-08-10 22:13:13 +02:00
Paul Koudelka f9263b02dd fixed the expert settings for datasource_file 2026-08-10 22:06:26 +02:00
Paul Koudelka c8c1e38475 made tokenizer concurrent and threadsafe 2026-08-10 21:36:01 +02:00
Paul Koudelka 6985678d2f changed DateTime to DateTimeOffset 2026-08-10 20:06:51 +02:00
Paul Koudelka 9ab196c13c ensured prompts fit embedding model 2026-08-10 19:42:39 +02:00
Paul Koudelka 3918b48409 fixed tokenization issue for long files 2026-08-10 18:21:31 +02:00
Paul Koudelka 185a99c9ed Added security checks for confidence levels 2026-08-05 15:05:17 +02:00
PaulKoudelka 3eca153f83 fixed merge errors and added language support for german 2026-08-04 18:52:16 +02:00
PaulKoudelka 9932b5870b Merge branch 'main' into chunk-data 2026-08-04 18:01:25 +02:00
PaulKoudelka 3d68d73767 changed to sqlite connection to ef core 2026-08-04 17:27:26 +02:00
PaulKoudelka 23416b21fc run optimise on qdrant_edge store 2026-08-04 15:59:13 +02:00
PaulKoudelka 2a3a82820c forced overlap on chunks 2026-08-04 15:08:47 +02:00
PaulKoudelka 010f35b08a added retrieval with vector and bm25 search 2026-08-03 20:56:19 +02:00
PaulKoudelka c78a8f05ef added metadata for bm25 and also confidence levels 2026-08-03 19:45:47 +02:00
PaulKoudelka b3ba945b98 made the embedding procedure more robust 2026-08-03 17:53:31 +02:00
PaulKoudelka 89f7156fdf improved chunking pipeline and handling of expert settings 2026-07-29 18:47:59 +02:00
PaulKoudelka 2829051e4c made only custom tokenizer visible in chat 2026-07-29 15:03:23 +02:00
PaulKoudelka fd2d99872a added warning and improved error handling 2026-07-28 20:22:37 +02:00
PaulKoudelka 759c7f9d89 improved stability of embedding chunks 2026-07-28 17:36:06 +02:00
PaulKoudelka 1bf9328fb9 made the chunking process more robust 2026-07-28 16:37:07 +02:00
PaulKoudelka 95ca74809b made file watchers more robust against multiple triggers 2026-07-28 15:52:59 +02:00
PaulKoudelka 2332e83da8 Added dependencies for sql 2026-07-28 15:26:45 +02:00
PaulKoudelka c59e33e1b1 added sql integration instead of json files 2026-07-28 15:25:10 +02:00
Paul Koudelka 2e06af13e0 Merge branch 'main' into chunk-data 2026-07-01 09:54:12 +02:00
Paul Koudelka 6989cbe3b2 changed qdrant to qdrant edge 2026-06-10 17:07:34 +02:00
Paul Koudelka b7b18aa45c Merge branch 'main' into chunk-data 2026-06-10 14:11:44 +02:00
Paul Koudelka ea87d79ca9 current main is now successfully merged into branch 2026-05-27 20:02:43 +02:00
PaulKoudelka 00b6c98f3b Merge branch 'main' into chunk-data 2026-05-26 17:19:32 +02:00
Paul Koudelka 637bf9701b added manual refresh and cleaned up code 2026-05-13 18:13:34 +02:00
PaulKoudelka ac677c5ac7 background embed 2026-05-08 17:37:34 +02:00
PaulKoudelka e3cb7e9734 Merge branch 'main' into custom-tokenizer 2026-04-24 12:07:22 +02:00
PaulKoudelka 54ef68466a improved visual representation of SelectFile component with errors 2026-04-24 11:48:42 +02:00
Paul Koudelka aa554d4a53 added possibility to add tokenizer via plugin and refactored earlier changes 2026-04-23 16:29:30 +02:00
Paul Koudelka 95fb213069 removed print statements 2026-04-15 10:19:50 +02:00
Paul Koudelka 4a8ab90173 cleanup to ensure uniform style of code 2026-04-15 10:17:14 +02:00
PaulKoudelka 41573406d5 token count now adapts to chosen provider tokenizer 2026-04-14 16:27:10 +02:00
PaulKoudelka e07ca378d4 added functionality to add tokenizer to LLM and embedding models 2026-04-10 18:34:10 +02:00
PaulKoudelka 0854debc00 added functionality to save/move tokenizer file 2026-04-10 18:31:20 +02:00