kindle-zotero_importer/maschine/maschine_log_20260527.md

kind	project
maschine	kindle-zotero-importer

• Started project status review from repo files, generated artifacts, git status, README, and MEMORY.md.

• Found independent Git repo on main with no commits yet; tracked source/docs/config are uncommitted, generated pipeline outputs are ignored by .gitignore.

• Current generated pipeline state: 2,109 clippings parsed; 23,185 Zotero items indexed; 67 unique title matches reviewed; final writer plan contains 770 positioned annotations and skips unresolved match, attachment, and positioning cases.

• Created initial Git commit a38ac33 (Initial Kindle Zotero importer baseline) including source, docs, overrides, Zotero writer, and the maschine log.

• Extracted mismatch-review worklist from docs/mismatch-review.md: remaining work covers missing Coeckelbergh attachment, unmatched Kindle title mappings, missing PDF paths, EPUB positioning failures, PDF text lookup failures, and PDF rectangle recovery failures.

• Recovered stalled session state: inspected dirty repo, existing maschine log, README, core importer modules, and generated mismatch-review diff.

• Completed cleanup for ignore overrides: ignore: true now rejects all other resolution fields, ignored matches are excluded from generated override skeletons, ignored titles flow to ignored-title import-plan status, and README documents permanent skips.

• Verified recovery with PYTHONPATH=src python -m compileall -q src, match/plan/review CLI smoke checks using generated temp outputs under /var/folders/dz/bzxsk9tj3clf0cs7vg6r7h_80000gn/T/opencode/, and an ignore override validation smoke test.

• Current uncommitted work remains in README, docs/mismatch-review.md, importer modules, and new src/kindle_zotero_importer/__main__.py; no commit was made.

• Diagnosed Burroughs import failure: citekey override burroughs1992 matched successfully to Zotero item 39116 / key SGPP6ATY / PDF /Users/ubd/Library/Mobile Documents/com~apple~CloudDocs/03_Academia/ZoteroFiles/Burroughs_1992_.pdf; all 69 planned highlights failed at PDF positioning because pdftotext exits with permission error on an encrypted PDF where copying text is disallowed. Final writer plan therefore contained 0 Burroughs annotations.

• Implemented PDF permission-encryption fallback in src/kindle_zotero_importer/pdf_position.py: when Poppler text extraction reports copying is not allowed, the positioning step creates a temporary qpdf --decrypt copy and retries extraction/geometry against that copy without modifying the Zotero attachment.

• Regenerated import-plan.positioned.json, import-plan.final.json, and docs/mismatch-review.md after the fallback. Burroughs now has 58 positioned/final annotations out of 69 planned highlights; 10 remain pdf-rects-not-found and 1 remains pdf-text-not-found. Final writer plan annotation_count is now 859.

• Reviewed current mismatch-review structure for discard workflow. Remaining unmatched titles can be permanently skipped by adding ignore: true entries to match-overrides.json rather than editing docs/mismatch-review.md directly.

• Populated match-overrides.json with all 37 currently unmatched titles as editable ignore: true entries, preserving the Burroughs citation-key mapping. Validated overrides with a temp match run: 37 ignored, 24 matched, 6 ambiguous. Did not regenerate main pipeline so entries can be edited first.

• Changed override skeleton workflow so generated overrides always include review metadata with status and clipping_count. Unmatched generated entries now default to ignore: true for batch editing; ambiguous entries keep candidate resolution fields. Added the same review metadata to current match-overrides.json and regenerated match-overrides.generated.json.

• Refined generated override review ordering: unresolved entries sort first by clipping_count descending; already-entered overrides are retained at the end as an audit trail. Regenerated matches.json with current overrides and regenerated match-overrides.generated.json; current match statuses are 37 ignored, 24 matched, 6 ambiguous.

• Updated override editing workflow so ignored/unmatched entries include a blank citation_key field by default. Blank citation_key values are ignored by the loader, so entries remain ignored until edited; generated audit entries now preserve the blank citation_key for ignored items.

• Saved recovery/workflow changes in Git commit b713faa (Improve override review and PDF positioning). Commit includes PDF qpdf fallback, editable override review workflow, current match-overrides.json, regenerated mismatch review, README updates, and python -m entry point.

• Configured single Git remote origin with GitHub as fetch URL and two push URLs: GitHub plus Forgejo over ssh://git@git.utkubilen.de:2222/utku/kindle-zotero_importer.git. Initial push to GitHub succeeded; Forgejo required SSH URL syntax for the nonstandard port, then main pushed successfully there too.