Housekeeping (#379)

README.md

@@ -13,7 +13,7 @@ Things we are currently working on:
   - [x] ~~App: Configure embedding providers (PR [#224](https://github.com/MindWorkAI/AI-Studio/pull/224))~~
   - [x] ~~App: Implement an [ERI](https://github.com/MindWorkAI/ERI) server coding assistant (PR [#231](https://github.com/MindWorkAI/AI-Studio/pull/231))~~
   - [x] ~~App: Management of data sources (local & external data via [ERI](https://github.com/MindWorkAI/ERI)) (PR [#259](https://github.com/MindWorkAI/AI-Studio/pull/259), [#273](https://github.com/MindWorkAI/AI-Studio/pull/273))~~
-  - [ ] Runtime: Extract data from txt / md / pdf / docx / xlsx files
+  - [x] ~~Runtime: Extract data from txt / md / pdf / docx / xlsx files (PR [#374](https://github.com/MindWorkAI/AI-Studio/pull/374))~~
   - [ ] (*Optional*) Runtime: Implement internal embedding provider through [fastembed-rs](https://github.com/Anush008/fastembed-rs)
   - [ ] App: Implement external embedding providers
   - [ ] App: Implement the process to vectorize one local file using embeddings

app/MindWork AI Studio/Pages/About.razor

@@ -102,12 +102,17 @@
                     <ThirdPartyComponent Name="serde" Developer="Erick Tryzelaar, David Tolnay & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/serde-rs/serde/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/serde-rs/serde" UseCase="Now we have multiple systems, some developed in .NET and others in Rust. The data format JSON is responsible for translating data between both worlds (called data serialization and deserialization). Serde takes on this task in the Rust world. The counterpart in the .NET world is an integral part of .NET and is located in System.Text.Json."/>
                     <ThirdPartyComponent Name="keyring" Developer="Walther Chen, Daniel Brotsky & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/hwchen/keyring-rs/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/hwchen/keyring-rs" UseCase="In order to use any LLM, each user must store their so-called token for each LLM provider. This token must be kept secure, similar to a password. The safest way to do this is offered by operating systems like macOS, Windows, and Linux: They have mechanisms to store such data, if available, on special security hardware. Since this is currently not possible in .NET, we use this Rust library."/>
                     <ThirdPartyComponent Name="arboard" Developer="Artur Kovacs, Avi Weinstock, 1Password & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/1Password/arboard/blob/master/LICENSE-MIT.txt" RepositoryUrl="https://github.com/1Password/arboard" UseCase="To be able to use the responses of the LLM in other apps, we often use the clipboard of the respective operating system. Unfortunately, in .NET there is no solution that works with all operating systems. Therefore, I have opted for this library in Rust. This way, data transfer to other apps works on every system."/>
-                    <ThirdPartyComponent Name="tokio" Developer="Alex Crichton & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/tokio/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/tokio" UseCase="Code in the Rust language can be specified as synchronous or asynchronous. Unlike .NET and the C# language, Rust cannot execute asynchronous code by itself. Rust requires support in the form of an executor for this. Tokio is one such executor."/>
+                    <ThirdPartyComponent Name="tokio" Developer="Alex Crichton, Carl Lerche, Alice Ryhl, Taiki Endo, Ivan Petkov, Eliza Weisman, Lucio Franco & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/tokio/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/tokio" UseCase="Code in the Rust language can be specified as synchronous or asynchronous. Unlike .NET and the C# language, Rust cannot execute asynchronous code by itself. Rust requires support in the form of an executor for this. Tokio is one such executor."/>
+                    <ThirdPartyComponent Name="futures" Developer="Alex Crichton, Taiki Endo, Taylor Cramer, Nemo157, Josef Brandl, Aaron Turon & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rust-lang/futures-rs/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/rust-lang/futures-rs" UseCase="This is a library providing the foundations for asynchronous programming in Rust. It includes key trait definitions like Stream, as well as utilities like join!, select!, and various futures combinator methods which enable expressive asynchronous control flow."/>
+                    <ThirdPartyComponent Name="async-stream" Developer="Carl Lerche, Taiki Endo & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/async-stream/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/async-stream" UseCase="This library is used to create asynchronous streams in Rust. It allows us to work with streams of data that can be produced asynchronously, making it easier to handle events or data that arrive over time. We use this, e.g., to stream arbitrary data from the file system to the embedding system."/>
                     <ThirdPartyComponent Name="flexi_logger" Developer="emabee & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/emabee/flexi_logger/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/emabee/flexi_logger" UseCase="This Rust library is used to output the app's messages to the terminal. This is helpful during development and troubleshooting. This feature is initially invisible; when the app is started via the terminal, the messages become visible."/>
                     <ThirdPartyComponent Name="rand" Developer="Rust developers & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rust-random/rand/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/rust-random/rand" UseCase="We must generate random numbers, e.g., for securing the interprocess communication between the user interface and the runtime. The rand library is great for this purpose."/>
                     <ThirdPartyComponent Name="base64" Developer="Marshall Pierce, Alice Maz & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/marshallpierce/rust-base64/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/marshallpierce/rust-base64" UseCase="For some data transfers, we need to encode the data in base64. This Rust library is great for this purpose."/>
                     <ThirdPartyComponent Name="Rust Crypto" Developer="Artyom Pavlov, Tony Arcieri, Brian Warner, Arthur Gautier, Vlad Filippov, Friedel Ziegelmayer, Nicolas Stalder & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/RustCrypto/traits/blob/master/cipher/LICENSE-MIT" RepositoryUrl="https://github.com/RustCrypto" UseCase="When transferring sensitive data between Rust runtime and .NET app, we encrypt the data. We use some libraries from the Rust Crypto project for this purpose: cipher, aes, cbc, pbkdf2, hmac, and sha2. We are thankful for the great work of the Rust Crypto project."/>
                     <ThirdPartyComponent Name="rcgen" Developer="RustTLS developers, est31 & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rustls/rcgen/blob/main/LICENSE" RepositoryUrl="https://github.com/rustls/rcgen" UseCase="For the secure communication between the user interface and the runtime, we need to create certificates. This Rust library is great for this purpose."/>
+                    <ThirdPartyComponent Name="file-format" Developer="Mickaël Malécot & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/mmalecot/file-format/blob/main/LICENSE-MIT" RepositoryUrl="https://github.com/mmalecot/file-format" UseCase="This library is used to determine the file type of a file. This is necessary, e.g., when we want to stream a file."/>
+                    <ThirdPartyComponent Name="calamine" Developer="Johann Tuffe, Joel Natividad, Eric Jolibois, Dmitriy & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tafia/calamine/blob/master/LICENSE-MIT.md" RepositoryUrl="https://github.com/tafia/calamine" UseCase="This library is used to read Excel and OpenDocument spreadsheet files. This is necessary, e.g., for using spreadsheets as a data source for a chat."/>
+                    <ThirdPartyComponent Name="pdfium-render" Developer="Alastair Carey, Dorian Rudolph & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/ajrcarey/pdfium-render/blob/master/LICENSE.md" RepositoryUrl="https://github.com/ajrcarey/pdfium-render" UseCase="This library is used to read PDF files. This is necessary, e.g., for using PDFs as a data source for a chat."/>
                     <ThirdPartyComponent Name="Lua-CSharp" Developer="Yusuke Nakada & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/nuskey8/Lua-CSharp/blob/main/LICENSE" RepositoryUrl="https://github.com/nuskey8/Lua-CSharp" UseCase="We use Lua as the language for plugins. Lua-CSharp lets Lua scripts communicate with AI Studio and vice versa. Thank you, Yusuke Nakada, for this great library." />
                     <ThirdPartyComponent Name="HtmlAgilityPack" Developer="ZZZ Projects & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/zzzprojects/html-agility-pack/blob/master/LICENSE" RepositoryUrl="https://github.com/zzzprojects/html-agility-pack" UseCase="We use the HtmlAgilityPack to extract content from the web. This is necessary, e.g., when you provide a URL as input for an assistant."/>
                     <ThirdPartyComponent Name="ReverseMarkdown" Developer="Babu Annamalai & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/mysticmind/reversemarkdown-net/blob/master/LICENSE" RepositoryUrl="https://github.com/mysticmind/reversemarkdown-net" UseCase="This library is used to convert HTML to Markdown. This is necessary, e.g., when you provide a URL as input for an assistant."/>

app/MindWork AI Studio/Pages/Supporters.razor

@@ -79,6 +79,7 @@
                                 <Supporter Name="DevNullx64" Type="SupporterType.INDIVIDUAL" URL="https://github.com/DevNullx64" Acknowledgment="Thanks Luc for your build script contribution."/>
                                 <Supporter Name="SolsticeSpectrum" Type="SupporterType.INDIVIDUAL" URL="https://github.com/SolsticeSpectrum" Acknowledgment="Thanks for your build script contribution."/>
                                 <Supporter Name="peerschuett" Type="SupporterType.INDIVIDUAL" URL="https://github.com/peerschuett" Acknowledgment="Thanks for your contributions to the project."/>
+                                <Supporter Name="nilskruthoff" Type="SupporterType.INDIVIDUAL" URL="https://github.com/nilskruthoff" Acknowledgment="Thanks, Nils, for taking the time to learn Rust and build the foundation for local retrieval."/>
                             </MudList>
                         </MudPaper>
                     </MudItem>

app/MindWork AI Studio/wwwroot/changelog/v0.9.39.md

@@ -1,7 +1,8 @@
-# v0.9.39, build 214 (2025-03-xx xx:xx UTC)
+# v0.9.39, build 214 (2025-04-xx xx:xx UTC)
 - Added a feature flag for the plugin system. This flag is disabled by default and can be enabled inside the app settings. Please note that this feature is still in development; there are no plugins available yet.
 - Added the Lua library we use for the plugin system to the about page.
 - Added the plugin overview page. This page shows all installed plugins and allows you to enable or disable them. It is only available when the plugin preview feature is enabled.
 - Added hot reloading for plugins. When any plugin is changed, the app will automatically reload the plugin without needing to restart the app.
+- Added an API for streaming arbitrary local files to the embedding process. Thanks Nils `nilskruthoff` for this great contribution.
 - Fixed the preview tooltip component not showing the correct position when used inside a scrollable container.
 - Upgraded to Rust 1.85.1

runtime/Cargo.lock

@@ -131,9 +131,9 @@ checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
 
 [[package]]
 name = "async-stream"
-version = "0.3.5"
+version = "0.3.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "cd56dd203fef61ac097dd65721a419ddccb106b2d2b70ba60a6b529f03961a51"
+checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476"
 dependencies = [
  "async-stream-impl",
  "futures-core",
@@ -142,9 +142,9 @@ dependencies = [
 
 [[package]]
 name = "async-stream-impl"
-version = "0.3.5"
+version = "0.3.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "16e62a023e7c117e27523144c5d2459f4397fcc3cab0085af8e2224f643a0193"
+checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -442,7 +442,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "138646b9af2c5d7f1804ea4bf93afc597737d2bd4f7341d67c48b03316976eb1"
 dependencies = [
  "byteorder",
- "chrono",
  "codepage",
  "encoding_rs",
  "log",
@@ -476,6 +475,8 @@ version = "1.2.16"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "be714c154be609ec7f5dad223a33bf1482fff90472de28f7362806e6d4832b8c"
 dependencies = [
+ "jobserver",
+ "libc",
  "shlex",
 ]
 
@@ -1152,9 +1153,9 @@ dependencies = [
 
 [[package]]
 name = "flexi_logger"
-version = "0.29.8"
+version = "0.30.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "88a5a6882b2e137c4f2664562995865084eb5a00611fba30c582ef10354c4ad8"
+checksum = "6807d19113a0dac26d3dae81ef5859058c341b31c71d0f8d0298b17327f21011"
 dependencies = [
  "chrono",
  "log",
@@ -1241,12 +1242,13 @@ dependencies = [
 
 [[package]]
 name = "futures"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "645c6916888f6cb6350d2550b80fb63e734897a8498abe35cfb732b6487804b0"
+checksum = "65bc07b1a8bc7c85c5f2e110c476c7389b4554ba72af57d8445ea63a576b0876"
 dependencies = [
  "futures-channel",
  "futures-core",
+ "futures-executor",
  "futures-io",
  "futures-sink",
  "futures-task",
@@ -1255,9 +1257,9 @@ dependencies = [
 
 [[package]]
 name = "futures-channel"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "eac8f7d7865dcb88bd4373ab671c8cf4508703796caa2b1985a9ca867b3fcb78"
+checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
 dependencies = [
  "futures-core",
  "futures-sink",
@@ -1265,15 +1267,15 @@ dependencies = [
 
 [[package]]
 name = "futures-core"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dfc6580bb841c5a68e9ef15c77ccc837b40a7504914d52e47b8b0e9bbda25a1d"
+checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e"
 
 [[package]]
 name = "futures-executor"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a576fc72ae164fca6b9db127eaa9a9dda0d61316034f33a0a0d4eda41f02b01d"
+checksum = "1e28d1d997f585e54aebc3f97d39e72338912123a67330d723fdbb564d646c9f"
 dependencies = [
  "futures-core",
  "futures-task",
@@ -1282,15 +1284,15 @@ dependencies = [
 
 [[package]]
 name = "futures-io"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a44623e20b9681a318efdd71c299b6b222ed6f231972bfe2f224ebad6311f0c1"
+checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6"
 
 [[package]]
 name = "futures-macro"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "87750cf4b7a4c0625b1529e4c543c2182106e4dedc60a2a6455e00d212c489ac"
+checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -1299,21 +1301,21 @@ dependencies = [
 
 [[package]]
 name = "futures-sink"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9fb8e00e87438d937621c1c6269e53f536c14d3fbd6a042bb24879e57d474fb5"
+checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7"
 
 [[package]]
 name = "futures-task"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "38d84fa142264698cdce1a9f9172cf383a0c82de1bddcf3092901442c4097004"
+checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
 
 [[package]]
 name = "futures-util"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3d6401deb83407ab3da39eba7e33987a73c3df0c82b4bb5813ee871c19c41d48"
+checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81"
 dependencies = [
  "futures-channel",
  "futures-core",
@@ -1467,15 +1469,16 @@ dependencies = [
 ]
 
 [[package]]
-name = "gif"
+name = "getrandom"
-version = "0.13.1"
+version = "0.2.15"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3fb2d69b19215e18bb912fa30f7ce15846e301408695e44e0ef719f1da9e19f2"
+checksum = "c4567c8db10ae91089c99af84c68c38da3ec2f087c3f82960bcdbf3656b6f4d7"
 dependencies = [
- "color_quant",
+ "cfg-if",
- "weezl",
+ "libc",
+ "wasi 0.11.0+wasi-snapshot-preview1",
 ]
- 
+
 [[package]]
 name = "getrandom"
 version = "0.3.1"
@@ -1488,6 +1491,16 @@ dependencies = [
  "windows-targets 0.52.6",
 ]
 
+[[package]]
+name = "gif"
+version = "0.13.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3fb2d69b19215e18bb912fa30f7ce15846e301408695e44e0ef719f1da9e19f2"
+dependencies = [
+ "color_quant",
+ "weezl",
+]
+
 [[package]]
 name = "gimli"
 version = "0.29.0"
@@ -2647,13 +2660,14 @@ version = "0.9.38"
 dependencies = [
  "aes",
  "arboard",
+ "async-stream",
  "base64 0.22.1",
  "calamine",
  "cbc",
- "chrono",
  "cipher",
  "file-format",
  "flexi_logger",
+ "futures",
  "hmac",
  "keyring",
  "log",
@@ -2674,6 +2688,7 @@ dependencies = [
  "tauri-build",
  "tauri-plugin-window-state",
  "tokio",
+ "tokio-stream",
  "url",
 ]
 
@@ -5294,9 +5309,9 @@ dependencies = [
 
 [[package]]
 name = "tokio-stream"
-version = "0.1.15"
+version = "0.1.17"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "267ac89e0bec6e691e5813911606935d77c476ff49024f98abcea3e7b15e37af"
+checksum = "eca58d7bba4a75707817a2c44174253f9236b2d5fbd055602e9d5c07c139a047"
 dependencies = [
  "futures-core",
  "pin-project-lite",

runtime/Cargo.toml

@@ -16,7 +16,10 @@ serde_json = "1.0.134"
 keyring = { version = "3.6.2", features = ["apple-native", "windows-native", "sync-secret-service"] }
 arboard = "3.4.1"
 tokio = { version = "1.44", features = ["rt", "rt-multi-thread", "macros", "process"] }
-flexi_logger = "0.29.8"
+tokio-stream = "0.1.17"
+futures = "0.3.31"
+async-stream = "0.3.6"
+flexi_logger = "0.30.0"
 log = { version = "0.4.26", features = ["kv"] }
 once_cell = "1.20.3"
 rocket = { version = "0.5.1", features = ["json", "tls"] }
@@ -31,11 +34,8 @@ hmac = "0.12.1"
 sha2 = "0.10.8"
 rcgen = { version = "0.13.2", features = ["pem"] }
 file-format = "0.26.0"
-calamine = "0.22"
+calamine = "0.26.1"
-pdfium-render = "0.8.27"
+pdfium-render = "0.8.29"
-async-stream = "0.3"
-futures = "0.3"
-tokio-stream = "0.1"
 
 # Fixes security vulnerability downstream, where the upstream is not fixed yet:
 url = "2.5"

runtime/src/file_data.rs

@@ -10,7 +10,7 @@ use tokio::io::AsyncBufReadExt;
 use tokio::process::Command;
 use tokio::sync::mpsc;
 use tokio_stream::wrappers::ReceiverStream;
-use rocket::{State, Shutdown};
+use rocket::Shutdown;
 use rocket::response::stream::{EventStream, Event};
 use rocket::tokio::select;
 use rocket::serde::Serialize;
@@ -38,11 +38,10 @@ const ODT: &str = "odt";
 type Result<T> = std::result::Result<T, Box<dyn std::error::Error + Send + Sync>>;
 type ChunkStream = Pin<Box<dyn Stream<Item = Result<Chunk>> + Send>>;
 
-#[get("/system/file-data/extract?<path>")]
+#[get("/retrieval/fs/extract?<path>")]
 pub async fn extract_data(path: String, mut end: Shutdown) -> EventStream![] {
     EventStream! {
         let stream_result = stream_data(&path).await;
-        
         match stream_result {
             Ok(mut stream) => {
                 loop {
@@ -61,6 +60,7 @@ pub async fn extract_data(path: String, mut end: Shutdown) -> EventStream![] {
                     yield Event::json(&chunk);
                 }
             },
+
             Err(e) => {
                 yield Event::json(&format!("Error starting stream: {}", e));
             }
@@ -74,21 +74,21 @@ async fn stream_data(file_path: &str) -> Result<ChunkStream> {
     }
 
     let file_path_clone = file_path.to_owned();
-
     let fmt = tokio::task::spawn_blocking(move || {
         FileFormat::from_file(&file_path_clone)
     }).await??;
 
     let ext = file_path.split('.').last().unwrap_or("");
-
     let stream = match ext {
         DOCX | ODT => {
             let from = if ext == DOCX { "docx" } else { "odt" };
             convert_with_pandoc(file_path, from, TO_MARKDOWN).await?
         }
+        
         "xlsx" | "ods" | "xls" | "xlsm" | "xlsb" | "xla" | "xlam" => {
             stream_spreadsheet_as_csv(file_path).await?
         }
+        
         _ => match fmt.kind() {
             Kind::Document => match fmt {
                 FileFormat::PortableDocumentFormat => read_pdf(file_path).await?,
@@ -100,20 +100,24 @@ async fn stream_data(file_path: &str) -> Result<ChunkStream> {
                 }
                 _ => stream_text_file(file_path).await?,
             },
+            
             Kind::Ebook => return Err("Ebooks not yet supported".into()),
             Kind::Image => chunk_image(file_path).await?,
+            
             Kind::Other => match fmt {
                 FileFormat::HypertextMarkupLanguage => {
                     convert_with_pandoc(file_path, fmt.extension(), TO_MARKDOWN).await?
                 }
                 _ => stream_text_file(file_path).await?,
             },
+            
             Kind::Presentation => match fmt {
                 FileFormat::OfficeOpenXmlPresentation => {
                     convert_with_pandoc(file_path, fmt.extension(), TO_MARKDOWN).await?
                 }
                 _ => stream_text_file(file_path).await?,
             },
+            
             Kind::Spreadsheet => stream_spreadsheet_as_csv(file_path).await?,
             _ => stream_text_file(file_path).await?,
         },
@@ -156,7 +160,7 @@ async fn read_pdf(file_path: &str) -> Result<ChunkStream> {
         };
 
         for (i, page) in doc.pages().iter().enumerate() {
-            let content = match page.text().and_then(|t| Ok(t.all())) {
+            let content = match page.text().map(|t| t.all()) {
                 Ok(c) => c,
                 Err(e) => {
                     let _ = tx.blocking_send(Err(e.into()));
@@ -191,12 +195,11 @@ async fn stream_spreadsheet_as_csv(file_path: &str) -> Result<ChunkStream> {
 
         for sheet_name in workbook.sheet_names() {
             let range = match workbook.worksheet_range(&sheet_name) {
-                Some(Ok(r)) => r,
+                Ok(r) => r,
-                Some(Err(e)) => {
+                Err(e) => {
                     let _ = tx.blocking_send(Err(e.into()));
                     continue;
                 }
-                None => continue,
             };
 
             for (row_idx, row) in range.rows().enumerate() {
@@ -228,7 +231,7 @@ async fn convert_with_pandoc(
 ) -> Result<ChunkStream> {
     let output = Command::new("pandoc")
         .arg(file_path)
-        .args(&["-f", from, "-t", to])
+        .args(["-f", from, "-t", to])
         .output()
         .await?;