isphere-ai-bridge/docs/source-discovery/2026-07-10-receive-message-reconciliation-precheck.md

Receive Message Reconciliation Precheck

Date: 2026-07-10

Goal

Compare PacketReader and MsgLib receive-message sources using sanitized identifiers, counts, source refs, and booleans only.

This is a precheck node. It does not change MCP default routing, does not merge sources, and does not add pagination, send, download, hook, injection, or DB-write behavior.

Sources

PacketReader source

Current role: default isphere_receive_messages source when source_preference is empty, auto, or local_readonly.

Evidence:

• Decrypted PacketReader.ProcessPacket lines expose XMPP <message> stanzas.
• Current Go source: internal/isphere.EncryptedPacketLogSource.
• Current configuration:
  • ISPHERE_PACKET_LOG_FILE
  • ISPHERE_PACKET_LOG_DIR
• Current normalized fields include:
  • message_id
  • conversation_id
  • conversation_type
  • sender_id
  • receiver_id
  • content_text
  • subject
  • timestamp
  • receipt_id
  • read flag

MsgLib DB source

Current role: explicit copied-DB receive source only when source_preference="msglib_readonly" and full MsgLib env is configured.

Evidence:

• C26 proved copied MsgLib.db opens through 32-bit System.Data.SQLite with password 123.
• C38 added sidecar list_messages.
• C39 added internal/isphere.MsgLibMessageSource.
• C40 added explicit msglib_readonly tool selection.
• C41 wired env-configured MsgLib receive and optional copied-DB MCP smoke.

Current optional smoke prints sanitized evidence only:

{
  "db_receive_smoke": true,
  "db_receive_message_count": 5,
  "db_receive_sources": ["msglib:tblPersonMsg"],
  "message_body_values_printed": false,
  "file_paths_returned": false,
  "raw_rows_returned": false
}

Candidate match keys

Use these match keys for a future reconciliation helper:

strong: message_id exact match when both sources expose the same id
medium: same conversation id, same sender id, timestamp within a small window
medium: same group id, same sender id, same subject/body-present flag
weak: filename plus nearby timestamp for file-transfer messages
not allowed in committed evidence: message body values, local file paths, raw rows

Recommended future scoring:

Score	Meaning	Required sanitized evidence
strong	Same message id and compatible source refs.	message_id, raw_ref, source name.
medium	Same conversation/sender and close timestamp.	conversation_id, sender_id, timestamp bucket.
medium	Same group/sender and same subject/body-present state.	conversation_type=group, group id, sender id, body-present boolean.
weak	Same file name/size near the same time.	file name, size, timestamp bucket; no path values.
no match	Different ids and no close timestamp/participant overlap.	sanitized ids/booleans only.

Known mismatches

1. Completeness may differ. PacketReader sees decrypted transport/client message stanzas; MsgLib DB may not store every message.
2. Subject coverage differs. PacketReader exposes XML <subject> strongly; direct/group MsgLib tables do not have a clearly equivalent subject column.
3. Receipt semantics differ. PacketReader receipt ids and MsgLib receipt/read-state columns are not proven equivalent.
4. Timestamp format differs. PacketReader uses XMPP/log epoch-like values; MsgLib uses table-specific timestamp columns such as ActionTime, MsgTime, and system MsgTime.
5. Conversation id shape differs. PacketReader may use normalized sender|receiver for direct messages; MsgLib direct messages may prefer ObjPersonId.
6. Attachment mapping is incomplete. MsgLib can expose safe attachment metadata through TD_ReceiveFileRecord, but download_ref and local cache path remain unresolved.
7. Body output policy differs. PacketReader configured-source verification uses redacted/synthetic bodies. Real copied-DB optional smoke must not print body values even if the internal explicit DB receive call can detect body presence.

Safe evidence allowed in logs

Allowed in committed docs or verification output:

• Counts: message count, source count, table count.
• Source refs: packetlog_message, msglib:tblPersonMsg, msglib:tblMsgGroupPersonMsg, msglib:TD_SystemMessageRecord.
• Booleans:
  • body values present internally
  • body values printed externally
  • file paths returned
  • raw rows returned
  • sender/conversation display populated
• Sanitized field names and table names.
• Timestamp bucket or normalized time window when values are not personally revealing.

Not allowed in committed docs or verification output:

• Real message body values.
• Real local file paths.
• Raw DB rows.
• Raw decrypted logs.
• User credentials or login metadata values.
• Attachment contents.
• Production send/download side effects.

Decision for default routing

Do not change default routing yet.

Current rule remains:

source_preference	Receive source
empty	PacketReader/log-backed source
auto	PacketReader/log-backed source
local_readonly	PacketReader/log-backed source
msglib_readonly	Explicit MsgLib DB receive source, only when env is configured

Reason:

• PacketReader is still the safest default because it is the current parsed receive source and was implemented first with redacted fixtures.
• MsgLib DB is now a valid explicit receive source, but it needs reconciliation before it can become part of auto.
• The current safe evidence is enough to design a reconciliation helper, but not enough to automatically merge or replace sources.

Next implementation node

Recommended next business node: R2 contact/group search quality hardening.

Reason:

• Send-message and send-file both depend on reliable target resolution.
• Search results need deterministic ranking and de-duplication before write-side connector work.
• Receive-source reconciliation can continue later as a helper node before any auto merge, but it should not block target search hardening.

Future receive-specific node after R2/R3:

• Add a local-only reconciliation helper that compares PacketReader and MsgLib messages by sanitized keys and outputs only match counts, source refs, and mismatch categories.
• Do not implement pagination until reconciliation proves which source should drive ordering.
• Do not implement file download until R3 proves message-to-cache/download mapping.