Skip to content
AI usage dashboard

desktop/src-tauri/crates/core/src/ingest.rs

Comment 1, line 1

desktop/src-tauri/crates/core/src/ingest.rs:1

//! Commit event updates, cursors, parse context and affected daily aggregates in one ingest transaction.

Comment 2, line 2

desktop/src-tauri/crates/core/src/ingest.rs:2

//! Replay uncommitted interrupted batches without duplicates: update events by key and recompute daily partitions.

Comment 3, line 17

desktop/src-tauri/crates/core/src/ingest.rs:17

/// Existing usage_events metadata used to resolve duplicates.

Comment 4, line 28

desktop/src-tauri/crates/core/src/ingest.rs:28

/// Cursor and versioned parse-context updates: model state, cumulative baseline and unfinished requests.

Comment 5, line 37

desktop/src-tauri/crates/core/src/ingest.rs:37

/// Redacted batch diagnostics: field names, error codes and positions, without original row content.

Comment 6, line 47

desktop/src-tauri/crates/core/src/ingest.rs:47

/// One ingest batch; every event belongs to the same source instance.

Comment 7, line 51

desktop/src-tauri/crates/core/src/ingest.rs:51

/// Source instance for this batch.

Comment 8, line 53

desktop/src-tauri/crates/core/src/ingest.rs:53

/// Fixed IANA timezone for grouping; stored settings do not follow later system changes.

Comment 9, line 59

desktop/src-tauri/crates/core/src/ingest.rs:59

/// Associated running job; commit its batch statistics in the same transaction.

Comment 10, line 61

desktop/src-tauri/crates/core/src/ingest.rs:61

/// UTC-millisecond retention cutoff: skip older events with diagnostics instead of restoring deleted details.

Comment 11, line 65

desktop/src-tauri/crates/core/src/ingest.rs:65

/// V09 fault-injection points; failure must roll back the whole transaction and permit identical replay.

Comment 12, line 98

desktop/src-tauri/crates/core/src/ingest.rs:98

/// Local days affected by recomputation, formatted YYYY-MM-DD.

Comment 13, line 109

desktop/src-tauri/crates/core/src/ingest.rs:109

/// Commit in one transaction; the fault test hook forces complete rollback.

Comment 14, line 129

desktop/src-tauri/crates/core/src/ingest.rs:129

// Test-frozen schemas before v3 lack parse_basis; write using the historical schema

Comment 15, line 130

desktop/src-tauri/crates/core/src/ingest.rs:130

// without that field. Normal startup migrates to SCHEMA_VERSION first.

Comment 16, line 184

desktop/src-tauri/crates/core/src/ingest.rs:184

// 1. Insert or update events.

Comment 17, line 243

desktop/src-tauri/crates/core/src/ingest.rs:243

// The v1 content hash included observed_at; an observation-time change alone is duplicate content.

Comment 18, line 244

desktop/src-tauri/crates/core/src/ingest.rs:244

// Same-key final repeats differing only in occurrence/observation/native time are repeated reports, not conflicts.

Comment 19, line 271

desktop/src-tauri/crates/core/src/ingest.rs:271

// Full prior hashes permit only parser-metadata changes; retain original time, usage and source revision.

Comment 20, line 272

desktop/src-tauri/crates/core/src/ingest.rs:272

// Later metadata updates cannot clear a real conflict already observed in this batch.

Comment 21, line 318

desktop/src-tauri/crates/core/src/ingest.rs:318

// Explicit source-correction policies compare the complete old hash.

Comment 22, line 319

desktop/src-tauri/crates/core/src/ingest.rs:319

// Retain first observation, real conflict flags and all change history

Comment 23, line 320

desktop/src-tauri/crates/core/src/ingest.rs:320

// while correcting only verified fields.

Comment 24, line 386

desktop/src-tauri/crates/core/src/ingest.rs:386

// Keep the existing conflicting value rather than choose a larger one; diagnose and recompute its daily conflict count.

Comment 25, line 408

desktop/src-tauri/crates/core/src/ingest.rs:408

// Record mathematical contradictions as diagnostics instead of hiding them with max(0, ...).

Comment 26, line 427

desktop/src-tauri/crates/core/src/ingest.rs:427

// 2. Commit cursors and parse context together; separate stages make fault-injection points explicit.

Comment 27, line 463

desktop/src-tauri/crates/core/src/ingest.rs:463

// 3. Advance the data revision when days are affected; recomputed daily rows carry that revision.

Comment 28, line 472

desktop/src-tauri/crates/core/src/ingest.rs:472

// 4. Recompute affected timezone/day partitions without adding to sealed days; persist hours

Comment 29, line 473

desktop/src-tauri/crates/core/src/ingest.rs:473

// in the same transaction so hourly archives survive later deletion of details.

Comment 30, line 482

desktop/src-tauri/crates/core/src/ingest.rs:482

// 5. Store redacted diagnostics.

Comment 31, line 492

desktop/src-tauri/crates/core/src/ingest.rs:492

// 6. Commit job progress in the same transaction.

Comment 32, line 512

desktop/src-tauri/crates/core/src/ingest.rs:512

/// Compare complete prior events; a renamed parser is not a source revision or permission to change other fields.

Comment 33, line 520

desktop/src-tauri/crates/core/src/ingest.rs:520

// Match normal same-key final-repeat time handling; metadata updates still retain original timestamps.

Comment 34, line 537

desktop/src-tauri/crates/core/src/ingest.rs:537

/// Reconstruct the complete doc1 event for the verified native writer only.

Comment 35, line 538

desktop/src-tauri/crates/core/src/ingest.rs:538

/// All positive counters, identity and attribution must still match the old hash.

Comment 36, line 588

desktop/src-tauri/crates/core/src/ingest.rs:588

/// Only the verified gajae v5/OpenAI-completions policy permits these v2

Comment 37, line 589

desktop/src-tauri/crates/core/src/ingest.rs:589

/// unknown fields/start-time basis. Reconstruct every old field before comparing

Comment 38, line 590

desktop/src-tauri/crates/core/src/ingest.rs:590

/// canonical or legacy full hashes; resolve unrelated content changes normally.

Comment 39, line 678

desktop/src-tauri/crates/core/src/ingest.rs:678

/// Junie native uncached input and default-zero corrections compare at most 128

Comment 40, line 679

desktop/src-tauri/crates/core/src/ingest.rs:679

/// candidates restoring only missing/default-zero fields from the old parser;

Comment 41, line 680

desktop/src-tauri/crates/core/src/ingest.rs:680

/// every other field must match its full canonical or legacy hash.

Comment 42, line 761

desktop/src-tauri/crates/core/src/ingest.rs:761

/// MiMo required raw counters and client prices default to zero. Restore only those

Comment 43, line 762

desktop/src-tauri/crates/core/src/ingest.rs:762

/// absent/default fields and their old derived values; the complete prior hash

Comment 44, line 763

desktop/src-tauri/crates/core/src/ingest.rs:763

/// still preserves identity, every positive value, quality and attribution.

Comment 45, line 797

desktop/src-tauri/crates/core/src/ingest.rs:797

// The old parser required five raw counters. Reconstruct its complete summary;

Comment 46, line 798

desktop/src-tauri/crates/core/src/ingest.rs:798

// an unchanged part ID alone cannot authorize a correction.

Comment 47, line 1103

desktop/src-tauri/crates/core/src/ingest.rs:1103

/// Shared INSERT/UPDATE parameter order; ?41 is conflict (INSERT uses literal zero).

Comment 48, line 1104

desktop/src-tauri/crates/core/src/ingest.rs:1104

/// ?48 created_at_ms is used only by INSERT, not UPDATE.

Comment 49, line 1126

desktop/src-tauri/crates/core/src/ingest.rs:1126

// 1 event_id

Comment 50, line 1127

desktop/src-tauri/crates/core/src/ingest.rs:1127

// 2 source_instance_id

Comment 51, line 1128

desktop/src-tauri/crates/core/src/ingest.rs:1128

// 3 source_record_key

Comment 52, line 1129

desktop/src-tauri/crates/core/src/ingest.rs:1129

// 4 record_kind

Comment 53, line 1130

desktop/src-tauri/crates/core/src/ingest.rs:1130

// 5 schema_version

Comment 54, line 1131

desktop/src-tauri/crates/core/src/ingest.rs:1131

// 6 parser_version

Comment 55, line 1132

desktop/src-tauri/crates/core/src/ingest.rs:1132

// 7 origin_call_id

Comment 56, line 1133

desktop/src-tauri/crates/core/src/ingest.rs:1133

// 8 attempt_id

Comment 57, line 1134

desktop/src-tauri/crates/core/src/ingest.rs:1134

// 9 session_id

Comment 58, line 1135

desktop/src-tauri/crates/core/src/ingest.rs:1135

// 10 parent_session_id

Comment 59, line 1136

desktop/src-tauri/crates/core/src/ingest.rs:1136

// 11 host_application

Comment 60, line 1137

desktop/src-tauri/crates/core/src/ingest.rs:1137

// 12 agent

Comment 61, line 1138

desktop/src-tauri/crates/core/src/ingest.rs:1138

// 13 call_category

Comment 62, line 1139

desktop/src-tauri/crates/core/src/ingest.rs:1139

// 14 occurred_at_ms

Comment 63, line 1140

desktop/src-tauri/crates/core/src/ingest.rs:1140

// 15 observed_at_ms

Comment 64, line 1141

desktop/src-tauri/crates/core/src/ingest.rs:1141

// 16 source_time

Comment 65, line 1142

desktop/src-tauri/crates/core/src/ingest.rs:1142

// 17 time_basis

Comment 66, line 1143

desktop/src-tauri/crates/core/src/ingest.rs:1143

// 18 interval_start_ms

Comment 67, line 1144

desktop/src-tauri/crates/core/src/ingest.rs:1144

// 19 interval_end_ms

Comment 68, line 1145

desktop/src-tauri/crates/core/src/ingest.rs:1145

// 20 provider_id

Comment 69, line 1146

desktop/src-tauri/crates/core/src/ingest.rs:1146

// 21 model_raw

Comment 70, line 1147

desktop/src-tauri/crates/core/src/ingest.rs:1147

// 22 model_canonical

Comment 71, line 1148

desktop/src-tauri/crates/core/src/ingest.rs:1148

// 23 model_attribution

Comment 72, line 1149

desktop/src-tauri/crates/core/src/ingest.rs:1149

// 24 input_uncached

Comment 73, line 1150

desktop/src-tauri/crates/core/src/ingest.rs:1150

// 25 input_cache_read

Comment 74, line 1151

desktop/src-tauri/crates/core/src/ingest.rs:1151

// 26 input_cache_write

Comment 75, line 1152

desktop/src-tauri/crates/core/src/ingest.rs:1152

// 27 input_total

Comment 76, line 1153

desktop/src-tauri/crates/core/src/ingest.rs:1153

// 28 output_total

Comment 77, line 1154

desktop/src-tauri/crates/core/src/ingest.rs:1154

// 29 output_reasoning

Comment 78, line 1155

desktop/src-tauri/crates/core/src/ingest.rs:1155

// 30 total_tokens

Comment 79, line 1156

desktop/src-tauri/crates/core/src/ingest.rs:1156

// 31 source_total

Comment 80, line 1157

desktop/src-tauri/crates/core/src/ingest.rs:1157

// 32 quality_json

Comment 81, line 1158

desktop/src-tauri/crates/core/src/ingest.rs:1158

// 33 quality_bucket

Comment 82, line 1159

desktop/src-tauri/crates/core/src/ingest.rs:1159

// 34 lifecycle

Comment 83, line 1160

desktop/src-tauri/crates/core/src/ingest.rs:1160

// 35 source_revision

Comment 84, line 1161

desktop/src-tauri/crates/core/src/ingest.rs:1161

// 36 error_status

Comment 85, line 1162

desktop/src-tauri/crates/core/src/ingest.rs:1162

// 37 duration_ms

Comment 86, line 1163

desktop/src-tauri/crates/core/src/ingest.rs:1163

// 38 ttft_ms

Comment 87, line 1164

desktop/src-tauri/crates/core/src/ingest.rs:1164

// 39 attribution_status

Comment 88, line 1165

desktop/src-tauri/crates/core/src/ingest.rs:1165

// 40 exclusion_reason

Comment 89, line 1166

desktop/src-tauri/crates/core/src/ingest.rs:1166

// 41 conflict placeholder

Comment 90, line 1167

desktop/src-tauri/crates/core/src/ingest.rs:1167

// 42 content_hash

Comment 91, line 1168

desktop/src-tauri/crates/core/src/ingest.rs:1168

// 43 cost_minor

Comment 92, line 1169

desktop/src-tauri/crates/core/src/ingest.rs:1169

// 44 cost_currency

Comment 93, line 1170

desktop/src-tauri/crates/core/src/ingest.rs:1170

// 45 cost_kind

Comment 94, line 1171

desktop/src-tauri/crates/core/src/ingest.rs:1171

// 46 price_version

Comment 95, line 1172

desktop/src-tauri/crates/core/src/ingest.rs:1172

// 47 billing_scope

Comment 96, line 1173

desktop/src-tauri/crates/core/src/ingest.rs:1173

// 48 created_at_ms (INSERT)

Comment 97, line 1174

desktop/src-tauri/crates/core/src/ingest.rs:1174

// 49 updated_at_ms

Comment 98, line 1175

desktop/src-tauri/crates/core/src/ingest.rs:1175

// 50 parse_basis

Comment 99, line 1193

desktop/src-tauri/crates/core/src/ingest.rs:1193

/// Rebuild one local day from usage_events after deleting its unsealed rows.

Comment 100, line 1194

desktop/src-tauri/crates/core/src/ingest.rs:1194

/// Include only verified ownership in totals; transport_attempt contributes to attempt_count only.

Comment 101, line 1207

desktop/src-tauri/crates/core/src/ingest.rs:1207

// Since v4, daily_usage keeps each source-instance contribution; queries sum across sources (M1a).

Comment 102, line 1208

desktop/src-tauri/crates/core/src/ingest.rs:1208

// A v2 migration rebuilds before v4 partitioning, using the old shape without instance_id.

Comment 103, line 1256

desktop/src-tauri/crates/core/src/ingest.rs:1256

// input/output/total_unknown_count exclude quality_bucket='unknown' records:

Comment 104, line 1257

desktop/src-tauri/crates/core/src/ingest.rs:1257

// records with no known token field include failed calls and Copilot round markers.

Comment 105, line 1258

desktop/src-tauri/crates/core/src/ingest.rs:1258

// Calls remain counted while turn observations supply usage separately;

Comment 106, line 1259

desktop/src-tauri/crates/core/src/ingest.rs:1259

// include call/event counts without treating these as partially observed missing token fields.

Comment 107, line 1311

desktop/src-tauri/crates/core/src/ingest.rs:1311

/// Repair only the verified v1/v2 total omission; every other native field must match.

Comment 108, line 1342

desktop/src-tauri/crates/core/src/ingest.rs:1342

/// Recompute local days covering inclusive [from_ms, to_ms] in the requested timezone.

Comment 109, line 1343

desktop/src-tauri/crates/core/src/ingest.rs:1343

/// Use retained events, skip sealed days and commit recomputation with its revision in one transaction.

Comment 110, line 1344

desktop/src-tauri/crates/core/src/ingest.rs:1344

/// Return the new revision committed by this transaction.

Comment 111, line 1355

desktop/src-tauri/crates/core/src/ingest.rs:1355

// Limit recomputation to 750 days; reject larger ranges for the caller to split.

Comment 112, line 1377

desktop/src-tauri/crates/core/src/ingest.rs:1377

/// Rebuild hourly aggregates by local day for the hourly retention layer.

Comment 113, line 1378

desktop/src-tauri/crates/core/src/ingest.rs:1378

/// Use the same instance/Agent/provider/model/category/quality dimensions as daily totals.

Comment 114, line 1379

desktop/src-tauri/crates/core/src/ingest.rs:1379

/// Call with recompute_day in the same transaction; sealed days keep their frozen hourly rows.

Comment 115, line 1380

desktop/src-tauri/crates/core/src/ingest.rs:1380

/// Use the local offset at each event time, preserving 23/25-hour DST days.

Comment 116, line 1387

desktop/src-tauri/crates/core/src/ingest.rs:1387

// Test-frozen schemas before v5 have no hourly_usage table; skip hourly persistence.

Comment 117, line 1388

desktop/src-tauri/crates/core/src/ingest.rs:1388

// Migration enables it later without changing daily/detail meanings.