desktop/src-tauri/crates/core/src/adapters/jsonl.rs
Comment 1, line 1
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:1
//! Bounded JSONL reader (V07).Comment 2, line 2
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:2
//!Comment 3, line 3
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:3
//! Incremental input rules in architecture.md:Comment 4, line 4
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:4
//! - cursors keep file identity, generation, complete-line byte offset and parse context;Comment 5, line 5
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:5
//! - default line limit 8 MiB and chunk size 4 MiB; assemble lines across chunks;Comment 6, line 6
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:6
//! report oversized positions/reasons without silent dropping, and permit controlled retries;Comment 7, line 7
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:7
//! - leave incomplete lines for the next run without advancing cursors; assemble UTF-8 and skip its BOM;Comment 8, line 8
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:8
//! - isolate invalid lines with error codes/line numbers/byte offsets, without copying raw content;Comment 9, line 9
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:9
//! - detect truncation, same-size replacement and rename; file length alone does not establish continuity.Comment 10, line 14
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:14
/// Default line limit: 8 MiB.Comment 11, line 16
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:16
/// Default chunk size: 4 MiB.Comment 12, line 18
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:18
/// Bytes sampled at each end for content fingerprints.Comment 13, line 22
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:22
/// max_lines is a deterministic line limit for tests and controlled retries;Comment 14, line 23
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:23
/// time_budget is a wall-clock timeout, defaulting to 30 seconds per source run.Comment 15, line 43
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:43
/// Complete line with byte positions; end is the offset after its terminator.Comment 16, line 52
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:52
/// Invalid-line diagnostic without message content.Comment 17, line 60
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:60
/// Why reading stopped.Comment 18, line 63
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:63
/// Reached the current end of file.Comment 19, line 65
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:65
/// Oversized complete line: keep its start as the cursor and permit retry with a higher limit.Comment 20, line 67
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:67
/// Reached the configured line/byte limit.Comment 21, line 69
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:69
/// Reached the wall-clock timeout.Comment 22, line 73
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:73
/// Incremental read result.Comment 23, line 78
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:78
/// Next unread offset, after the last complete line; incomplete lines do not advance it.Comment 24, line 80
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:80
/// Next line number, starting at one.Comment 25, line 82
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:82
/// Incomplete bytes retained for the next read.Comment 26, line 87
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:87
/// Read from a complete-line start_offset; first_line_number identifies that position.Comment 27, line 119
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:119
// At offset zero, skip the BOM as line content while retaining it in byte offsets.Comment 28, line 159
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:159
// EOF leaves carry as an incomplete line for the next run.Comment 29, line 166
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:166
// Split complete lines by byte: 0x0A cannot occur within a UTF-8 multibyte sequence.Comment 30, line 215
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:215
// Incomplete data above the line limit is already oversized; do not wait for further growth.Comment 31, line 237
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:237
/// Probe length, mtime, creation time and prefix/suffix fingerprints, not length alone.Comment 32, line 243
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:243
/// Prefix fingerprint for head_len sampled bytes.Comment 33, line 245
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:245
/// Prefix length: 4096 for larger files, otherwise current length; changed lengths are not comparable.Comment 34, line 296
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:296
/// Stored source_files state and cursor.Comment 35, line 303
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:303
/// Prefix hashes are not comparable when sample lengths differ, such as a growing small file.Comment 36, line 309
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:309
/// Decide whether file continuity permits resuming or requires a new generation.Comment 37, line 312
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:312
/// Same generation: resume from the cursor for unchanged or appended content.Comment 38, line 314
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:314
/// Replacement/truncation/rebuilding requires generation+1, a full reread and discarded parse context.Comment 39, line 318
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:318
/// Compare stored/probed state: changed creation time indicates a recreated file;Comment 40, line 319
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:319
/// length below the cursor means truncation; comparable changed prefix or same-size suffix means replacement.Comment 41, line 320
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:320
/// When growing small files change sample length, rely on creation time and monotonic length instead.Comment 42, line 339
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:339
/// JSONL cursor persisted in ingestion_checkpoints.cursor_value.Comment 43, line 370
desktop/src-tauri/crates/core/src/adapters/jsonl.rs:370
// A growing small file changes prefix length; unchanged creation time and increasing length permit append.