mirror of
https://github.com/MindWorkAI/AI-Studio.git
synced 2026-11-04 01:47:21 +00:00
Added file patterns and exclusions to folder data sources (#1057)
Co-authored-by: Thorsten Sommer <SommerEngineering@users.noreply.github.com>
This commit is contained in:
1 parent
249fc778a2
commit
031874cebf
75 files changed
+4808
-589
No files matched your search
@@ -953,6 +953,10 @@ async fn stream_data(file_path: &str, extract_images: bool, stream_id: &str) ->
|
||||
}
|
||||
|
||||
/// How many bytes we inspect for NUL bytes to tell binary content from text.
|
||||
///
|
||||
/// The .NET app reads the same number of bytes before it takes a file without a known extension for
|
||||
/// text, cf. `ReadableTextDetection.PROBE_SIZE`. Change both together, so that both agree on what is
|
||||
/// binary.
|
||||
const BINARY_PROBE_SIZE: usize = 8_192;
|
||||
|
||||
/// Reads a text file and decodes it, no matter which encoding it uses.
|
||||
|
||||
Reference in new issue
Block a user