diff --git a/.gitignore b/.gitignore
index e00d036..c05ccfb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,7 +2,7 @@ clippings.json
zotero-index.json
matches.json
match-overrides.generated.json
-!match-overrides.json
+match-overrides.json
import-plan*.json
*.import-plan.json
__pycache__/
diff --git a/README.md b/README.md
index c4384e5..8c4a4bb 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
# Kanzi — Kindle annotations to Zotero
-Import Kindle `My Clippings.txt` highlights into Zotero as native annotations, directly inside Zotero, no terminal needed. Formerly *Kindle Zotero Importer*.
+Import Kindle `My Clippings.txt` highlights into Zotero as native annotations, directly inside Zotero. Once the required host tools are installed, imports run from the manager without CLI commands. Formerly *Kindle Zotero Importer*.
@@ -13,22 +13,22 @@ Import Kindle `My Clippings.txt` highlights into Zotero as native annotations, d
# Install
-1. Download `kanzi.xpi` from [Releases](https://github.com/UtkuBilenDemir/kanzi/releases) (latest `0.6.20`, or `0.6.19-beta` for preview).
+1. Download `kanzi.xpi` from [Releases](https://github.com/UtkuBilenDemir/kanzi/releases) (latest `0.7.0`).
2. In Zotero: `Tools` → `Plugins` → `gear` → `Install Plugin From File…` → pick the `.xpi` → restart Zotero.
3. `Tools` → `Kanzi…` to open the manager.
-Works with Zotero 7 to 10, macOS, Windows and Linux, PDF and EPUB.
+Works with Zotero 7 to 10. Kanzi packages its importer source, but requires Python 3.11 or newer on the host. EPUB imports need no external positioning tools. PDF imports additionally require Poppler (`pdftotext`, `pdftohtml`, `pdfinfo`); `qpdf` is recommended for encrypted PDFs. macOS and Linux are tested. Windows requires setting the full Python executable path in `Settings` and installing the PDF tools separately.
# Use
1. **Choose your file.** Click `Choose My Clippings.txt` and pick your *cumulative* `My Clippings.txt` from Kindle (`Documents/My Clippings.txt`). Keep the window open. You will see live progress.
2. **Check Integrated.** After the run, `Integrated` shows what was imported. Columns are `Highlight Text`, `Citekey`, `Added On`, `Integrated` and `Page`. Use the filter to find anything.
3. **Fix Conflicts.** If a Kindle title did not match a Zotero item, `Conflicts` shows it with up to three suggestions `citekey · title (score%)`. Click `Use` on the right one. Or type any citation key, Zotero key or item ID under `Use Custom`. Or `Ignore Title` to skip it forever. If you map `chabot2013` for one `Simondon` variant, it will offer to apply the same mapping to the other variants with that candidate.
-4. **Re-import.** After you have fixed one or more titles, click `Re-import with saved overrides`. It reuses the last file, or asks for it. Only new or changed highlights are repositioned. Already integrated ones are skipped, so the second run is fast. Tick `Full re-import from scratch` only if you want to rebuild everything.
+4. **Re-import.** After you have fixed one or more titles, click `Re-import with saved overrides`. It reuses the last file, or asks for it. Only new or changed highlights are repositioned. Already integrated ones are skipped, so the second run is fast. Use `Retry unresolved attachments and positions` for prior failures without rebuilding successful annotations. Tick `Full re-import from scratch` only if you want to rebuild everything.
5. **Mappings.** See all titles you have approved or ignored, newest first, with date. Use `Delete` to send any entry back to `Conflicts` for rematching.
6. **Settings and Artifacts.** Change `Python` or `Zotero DB` paths and `Save Settings`. Or use `Open` or `Reveal` for any generated file, such as `mismatch-review.md` or `match-overrides.json`.
-Your choices are saved in `match-overrides.json` in the project folder. Back it up, share it, or delete an entry to undo a mapping.
+Your choices are saved in `match-overrides.json` in Kanzi's private runtime folder inside your Zotero profile. Back it up or delete an entry to undo a mapping; it is excluded from Git because it contains your document titles.
# Coloured highlights
@@ -64,13 +64,13 @@ Both `[o]` and `[orange]` work, case is not sensitive, and space after the brack
**Summary.** Shows `clippings`, `unique titles`, `final annotations` and `conflicts`. Also `Match Statuses` and `Plan Statuses`. The run panel shows `Choose My Clippings.txt`, progress, stage and detail, and the `Full re-import` checkbox.
-**Integrated.** The last `import-plan.final.json` annotations. Columns are `Kindle Title`, `Citekey`, `Highlight Text`, `Added On`, `Integrated` and `Page` with colour swatch. Filter at the top right, 400 shown, sortable and resizable.
+**Integrated.** All annotations remembered in `import-plan.final.json`. Columns are `Kindle Title`, `Citekey`, `Highlight Text`, `Added On`, `Integrated` and `Page` with colour swatch. Filter at the top right; results are resizable and paginated.
**Conflicts.** Unresolved titles. Type is `Title match`, `Override suggestion` or `Attachment/position`. `Candidates` shows `N suggestion(s) — top: citekey (score%)`. Each candidate has a `Use` button. Below that is `Use Custom` and `Ignore Title`. Saving shows the `Re-import` bar. If you use `chabot2013` for one Simondon variant, it offers to apply to other variants with the same candidate.
**Mappings.** Persistent `match-overrides.json`. Columns are `Kindle Title`, `Resolution`, `Status`, `Count` and `Updated`. Use `Delete` to send it back to `Conflicts`. Filterable.
-**Settings.** `Project directory`, `Python`, `Zotero DB`, `Zotero storage`, `Share anonymised annotations` (on by default, private, hashed only), and `Highlight colours` as above. `Save Settings` writes `plugin-config.json` and `colour-map.json` for the pipeline. No restart is needed.
+**Settings.** `Project directory`, `Python`, `Zotero DB`, `Zotero storage`, annotation sharing, and `Highlight colours` as above. Sharing is off until you explicitly opt in and can be disabled here at any time. `Save Settings` writes `plugin-config.json` and `colour-map.json` for the pipeline. No restart is needed.
**Artifacts.** `Mismatch review`, `Persistent overrides`, `Generated suggestions`, `Positioned plan`, `Final writer plan` and `Plugin summary`. Each has `Open` and `Reveal`.
@@ -79,12 +79,12 @@ Incremental is default. Only new, changed or colour-changed highlights are repos
# For developers / CLI
```sh
-python scripts/build_plugin.py # → dist/kanzi.xpi
+python scripts/build_plugin.py # → dist/kanzi.xpi, including the Python runtime
python -m kindle_zotero_importer run "/path/to/My Clippings.txt" --workdir . --pretty
python -m kindle_zotero_importer run ... --full # ignore incremental and reprocess all
```
-PDF positioning uses Poppler (`pdftotext`, `pdftohtml`, `pdfinfo`) plus `qpdf` fallback. EPUB uses CFI. Zotero writes are only via `Zotero.Annotations`. It never writes `zotero.sqlite` directly.
+PDF positioning uses host-installed Poppler (`pdftotext`, `pdftohtml`, `pdfinfo`) plus `qpdf` fallback. EPUB uses CFI. Zotero writes are only via `Zotero.Annotations`. It never writes `zotero.sqlite` directly.
# Releases
@@ -94,4 +94,4 @@ Stable releases are on `Releases` with `kanzi.xpi` and `updates.json` attached.
Issues and pull requests welcome at [Issues](../../issues).
-Anonymised sharing is on by default to help future development. It sends only citation key, hashed highlight, colour, has comment and note, added on and integrated at. It never sends raw highlight text. Opt out any time in `Manager` → `Settings` → uncheck `Share anonymised annotations`.
+Annotation sharing is off until you explicitly opt in. If enabled, it sends a random pseudonymous installation ID, citation key and publication metadata (including DOI/ISBN when available), raw highlight text and comments, normalised/fuzzy hashes, colour, and dates to `annotation.utkubilen.de`. The collector does not receive your Zotero account identity, but free text can contain identifying information and therefore is not anonymous. Change this any time in `Manager` → `Settings`.
diff --git a/collector/app.py b/collector/app.py
new file mode 100644
index 0000000..0a8b851
--- /dev/null
+++ b/collector/app.py
@@ -0,0 +1,357 @@
+from __future__ import annotations
+
+from datetime import datetime, timezone
+import hashlib
+import json
+import os
+from pathlib import Path
+import re
+import secrets
+import sqlite3
+import unicodedata
+
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.responses import HTMLResponse
+
+
+DB = Path(os.environ.get("KANZI_ANNOTATION_DB", Path(__file__).parent / "annotations.db"))
+MAX_PAGE_SIZE = 200
+MAX_REQUEST_BYTES = 2_000_000
+MAX_BATCH_SIZE = 5_000
+
+app = FastAPI(title="Kanzi Annotation Collector")
+
+
+def _normalise(text: str) -> str:
+ value = unicodedata.normalize("NFKC", text or "").lower()
+ value = re.sub(r"-\s*\n\s*", "", value)
+ value = re.sub(r"[\u2010-\u2015\u2018\u2019\u201c\u201d]", "'", value)
+ value = re.sub(r"[^a-z0-9]+", " ", value)
+ return re.sub(r"\s+", " ", value).strip()
+
+
+def _hash_norm(text: str) -> str:
+ return hashlib.sha256(_normalise(text).encode()).hexdigest()[:16]
+
+
+def _fuzzy_hash(text: str) -> str:
+ norm = _normalise(text)
+ if len(norm) < 3:
+ return _hash_norm(text)
+ trigrams = {norm[index : index + 3] for index in range(len(norm) - 2)}
+ words = sorted(norm.split())
+ word_trigrams = {
+ " ".join(words[index : index + 3])
+ for index in range(max(1, len(words) - 2))
+ }
+ return hashlib.sha256(" ".join(sorted(trigrams | word_trigrams)).encode()).hexdigest()[:16]
+
+
+def _connect() -> sqlite3.Connection:
+ DB.parent.mkdir(parents=True, exist_ok=True)
+ connection = sqlite3.connect(DB)
+ connection.row_factory = sqlite3.Row
+ connection.execute("PRAGMA busy_timeout = 5000")
+ connection.execute("PRAGMA journal_mode = WAL")
+ return connection
+
+
+def init_db() -> None:
+ with _connect() as connection:
+ connection.execute(
+ """CREATE TABLE IF NOT EXISTS highlights (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ anon_id TEXT,
+ citation_key TEXT,
+ doi TEXT,
+ isbn TEXT,
+ issn TEXT,
+ publication_title TEXT,
+ clipping_id_hash TEXT,
+ norm_hash TEXT,
+ fuzzy_hash TEXT,
+ text TEXT,
+ color TEXT,
+ has_comment INTEGER,
+ comment TEXT,
+ comment_hash TEXT,
+ clipping_added_on_iso TEXT,
+ integrated_at TEXT,
+ received_at TEXT,
+ raw TEXT
+ )"""
+ )
+ existing_columns = {
+ row["name"] for row in connection.execute("PRAGMA table_info(highlights)")
+ }
+ required_columns = {
+ "doi": "TEXT",
+ "isbn": "TEXT",
+ "issn": "TEXT",
+ "publication_title": "TEXT",
+ "clipping_id_hash": "TEXT",
+ "norm_hash": "TEXT",
+ "fuzzy_hash": "TEXT",
+ "text": "TEXT",
+ "comment": "TEXT",
+ "comment_hash": "TEXT",
+ "clipping_added_on_iso": "TEXT",
+ "integrated_at": "TEXT",
+ "received_at": "TEXT",
+ "raw": "TEXT",
+ }
+ for name, column_type in required_columns.items():
+ if name not in existing_columns:
+ connection.execute(f"ALTER TABLE highlights ADD COLUMN {name} {column_type}")
+ connection.execute(
+ """CREATE TABLE IF NOT EXISTS raw_posts (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ anon_id TEXT,
+ received_at TEXT,
+ body TEXT
+ )"""
+ )
+ connection.execute(
+ """CREATE TABLE IF NOT EXISTS installations (
+ anon_id TEXT PRIMARY KEY,
+ token_hash TEXT UNIQUE NOT NULL,
+ created_at TEXT NOT NULL
+ )"""
+ )
+ # Older collector versions appended every retry. Retain only the newest
+ # record for each installation/clipping before enforcing idempotency.
+ connection.execute(
+ """DELETE FROM highlights
+ WHERE clipping_id_hash IS NOT NULL
+ AND id NOT IN (
+ SELECT MAX(id) FROM highlights
+ WHERE clipping_id_hash IS NOT NULL
+ GROUP BY anon_id, clipping_id_hash
+ )"""
+ )
+ connection.execute(
+ """CREATE UNIQUE INDEX IF NOT EXISTS highlights_identity
+ ON highlights(anon_id, clipping_id_hash)
+ WHERE clipping_id_hash IS NOT NULL"""
+ )
+
+
+init_db()
+
+
+HTML = """
+
Kanzi annotations
+Kanzi annotations
Private pseudonymous collector. Raw highlight text and comments are stored.
+-highlights
-citekeys
-installations
-received today
+Recent
| anon | citekey | DOI / ISBN | text | colour | received |
|---|
+Top citekeys
+Common highlights
Same publication and normalised text across at least two installations.
| DOI / ISBN | text | norm | installations | citekeys |
|---|
+"""
+
+
+@app.get("/", response_class=HTMLResponse)
+def root() -> str:
+ return HTML
+
+
+@app.get("/health")
+def health() -> dict[str, str]:
+ return {"status": "ok"}
+
+
+def _token_hash(token: str) -> str:
+ return hashlib.sha256(token.encode()).hexdigest()
+
+
+@app.post("/register")
+def register() -> dict[str, str]:
+ token = secrets.token_urlsafe(32)
+ anon_id = secrets.token_hex(16)
+ with _connect() as connection:
+ connection.execute(
+ "INSERT INTO installations (anon_id, token_hash, created_at) VALUES (?, ?, ?)",
+ (anon_id, _token_hash(token), datetime.now(timezone.utc).isoformat()),
+ )
+ return {"anonId": anon_id, "token": token}
+
+
+def _authenticated_anon_id(request: Request) -> str:
+ authorization = request.headers.get("authorization", "")
+ scheme, separator, token = authorization.partition(" ")
+ if not separator or scheme.lower() != "bearer" or not token:
+ raise HTTPException(status_code=401, detail="Missing collector credential")
+ with _connect() as connection:
+ row = connection.execute(
+ "SELECT anon_id FROM installations WHERE token_hash = ?",
+ (_token_hash(token),),
+ ).fetchone()
+ if not row:
+ raise HTTPException(status_code=401, detail="Invalid collector credential")
+ return str(row["anon_id"])
+
+
+@app.get("/stats")
+def stats() -> dict[str, int]:
+ today = datetime.now(timezone.utc).date().isoformat()
+ with _connect() as connection:
+ row = connection.execute(
+ """SELECT COUNT(*) AS total_highlights,
+ COUNT(DISTINCT citation_key) AS unique_citekeys,
+ COUNT(DISTINCT anon_id) AS unique_anons,
+ SUM(CASE WHEN substr(received_at, 1, 10) = ? THEN 1 ELSE 0 END) AS today
+ FROM highlights""",
+ (today,),
+ ).fetchone()
+ return {key: int(row[key] or 0) for key in row.keys()}
+
+
+@app.get("/recent")
+def recent(limit: int = 50, offset: int = 0) -> dict:
+ limit = max(1, min(limit, MAX_PAGE_SIZE))
+ offset = max(0, offset)
+ with _connect() as connection:
+ total = connection.execute("SELECT COUNT(*) FROM highlights").fetchone()[0]
+ rows = connection.execute(
+ """SELECT anon_id, citation_key, doi, isbn, text, clipping_id_hash,
+ norm_hash, fuzzy_hash, color, has_comment, received_at
+ FROM highlights ORDER BY id DESC LIMIT ? OFFSET ?""",
+ (limit, offset),
+ ).fetchall()
+ return {"recent": [dict(row) for row in rows], "total": total}
+
+
+@app.get("/top")
+def top(limit: int = 20) -> dict:
+ limit = max(1, min(limit, 100))
+ with _connect() as connection:
+ rows = connection.execute(
+ """SELECT citation_key, COUNT(*) AS count FROM highlights
+ WHERE citation_key IS NOT NULL AND citation_key != ''
+ GROUP BY citation_key ORDER BY count DESC, citation_key LIMIT ?""",
+ (limit,),
+ ).fetchall()
+ return {"top": [dict(row) for row in rows]}
+
+
+@app.get("/common")
+def common(limit: int = 50, offset: int = 0) -> dict:
+ limit = max(1, min(limit, MAX_PAGE_SIZE))
+ offset = max(0, offset)
+ group_sql = """FROM highlights
+ WHERE norm_hash IS NOT NULL AND norm_hash != ''
+ AND COALESCE(NULLIF(doi, ''), NULLIF(isbn, ''), NULLIF(citation_key, '')) IS NOT NULL
+ GROUP BY COALESCE(NULLIF(doi, ''), NULLIF(isbn, ''), citation_key), norm_hash
+ HAVING COUNT(DISTINCT anon_id) > 1"""
+ with _connect() as connection:
+ total = connection.execute(f"SELECT COUNT(*) FROM (SELECT 1 {group_sql})").fetchone()[0]
+ rows = connection.execute(
+ f"""SELECT COALESCE(NULLIF(doi, ''), NULLIF(isbn, ''), citation_key) AS publication,
+ norm_hash, COUNT(DISTINCT anon_id) AS installations,
+ GROUP_CONCAT(DISTINCT citation_key) AS citekeys,
+ MAX(text) AS example_text
+ {group_sql}
+ ORDER BY installations DESC, publication, norm_hash LIMIT ? OFFSET ?""",
+ (limit, offset),
+ ).fetchall()
+ return {"common": [dict(row) for row in rows], "total": total}
+
+
+@app.post("/highlights")
+async def post_highlights(request: Request) -> dict:
+ anon = _authenticated_anon_id(request)
+ content_length = request.headers.get("content-length")
+ if content_length:
+ try:
+ if int(content_length) > MAX_REQUEST_BYTES:
+ raise HTTPException(status_code=413, detail="Request body is too large")
+ except ValueError:
+ raise HTTPException(status_code=400, detail="Invalid Content-Length")
+ body = await request.body()
+ if len(body) > MAX_REQUEST_BYTES:
+ raise HTTPException(status_code=413, detail="Request body is too large")
+ try:
+ payload = json.loads(body)
+ except (json.JSONDecodeError, UnicodeDecodeError) as error:
+ raise HTTPException(status_code=400, detail="Invalid JSON") from error
+ if not isinstance(payload, dict):
+ raise HTTPException(status_code=422, detail="Payload must be an object")
+ highlights = payload.get("data") or payload.get("highlights") or []
+ if isinstance(highlights, dict):
+ highlights = [highlights]
+ if not isinstance(highlights, list):
+ raise HTTPException(status_code=422, detail="data must be a list")
+ if len(highlights) > MAX_BATCH_SIZE:
+ raise HTTPException(status_code=413, detail="Too many highlights in one request")
+ now = datetime.now(timezone.utc).isoformat()
+ inserted = updated = 0
+ with _connect() as connection:
+ connection.execute("BEGIN IMMEDIATE")
+ for highlight in highlights:
+ if not isinstance(highlight, dict):
+ continue
+ text = str(highlight.get("text") or "")[:2000]
+ clipping_hash = str(highlight.get("clipping_id_hash") or "")[:128] or None
+ values = (
+ anon,
+ str(highlight.get("citation_key") or "")[:512] or None,
+ str(highlight.get("doi") or "")[:512] or None,
+ str(highlight.get("isbn") or "")[:512] or None,
+ str(highlight.get("issn") or "")[:512] or None,
+ str(highlight.get("publication_title") or "")[:1000] or None,
+ clipping_hash,
+ highlight.get("norm_hash") or _hash_norm(text),
+ highlight.get("fuzzy_hash") or _fuzzy_hash(text),
+ text,
+ str(highlight.get("color") or "")[:32] or None,
+ 1 if highlight.get("has_comment") else 0,
+ str(highlight.get("comment") or "")[:2000],
+ str(highlight.get("comment_hash") or "")[:256] or None,
+ str(highlight.get("clipping_added_on_iso") or "")[:64] or None,
+ str(highlight.get("integrated_at") or "")[:64] or None,
+ now,
+ json.dumps(highlight, ensure_ascii=False)[:8000],
+ )
+ existing = None
+ if clipping_hash:
+ legacy_hash = clipping_hash[:8]
+ existing = connection.execute(
+ """SELECT id FROM highlights
+ WHERE anon_id = ? AND clipping_id_hash IN (?, ?)
+ ORDER BY clipping_id_hash = ? DESC LIMIT 1""",
+ (anon, clipping_hash, legacy_hash, clipping_hash),
+ ).fetchone()
+ if existing:
+ connection.execute(
+ """UPDATE highlights SET citation_key=?, doi=?, isbn=?, issn=?, publication_title=?, clipping_id_hash=?,
+ norm_hash=?, fuzzy_hash=?, text=?, color=?, has_comment=?, comment=?, comment_hash=?,
+ clipping_added_on_iso=?, integrated_at=?, received_at=?, raw=? WHERE id=?""",
+ (*values[1:7], *values[7:], existing["id"]),
+ )
+ updated += 1
+ else:
+ connection.execute(
+ """INSERT INTO highlights (
+ anon_id, citation_key, doi, isbn, issn, publication_title, clipping_id_hash,
+ norm_hash, fuzzy_hash, text, color, has_comment, comment, comment_hash,
+ clipping_added_on_iso, integrated_at, received_at, raw
+ ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
+ values,
+ )
+ inserted += 1
+ return {
+ "status": "ok",
+ "received": len(highlights),
+ "inserted": inserted,
+ "updated": updated,
+ "anon": anon,
+ }
diff --git a/collector/nginx.conf.example b/collector/nginx.conf.example
new file mode 100644
index 0000000..9e8c06e
--- /dev/null
+++ b/collector/nginx.conf.example
@@ -0,0 +1,28 @@
+# Add these directives in the nginx http context.
+limit_req_zone $binary_remote_addr zone=kanzi_ingest:10m rate=10r/m;
+limit_conn_zone $binary_remote_addr zone=kanzi_connections:10m;
+
+# Keep dashboard and read APIs behind Authentik. Only plugin ingestion and
+# health monitoring are intentionally unauthenticated.
+location = /highlights {
+ client_max_body_size 2m;
+ limit_req zone=kanzi_ingest burst=10 nodelay;
+ limit_conn kanzi_connections 2;
+ proxy_pass http://127.0.0.1:8099;
+}
+
+location = /register {
+ limit_req zone=kanzi_ingest burst=10 nodelay;
+ limit_conn kanzi_connections 2;
+ proxy_pass http://127.0.0.1:8099;
+}
+
+location = /health {
+ proxy_pass http://127.0.0.1:8099;
+}
+
+location / {
+ auth_request /outpost.goauthentik.io/auth/nginx;
+ error_page 401 = @goauthentik_proxy_signin;
+ proxy_pass http://127.0.0.1:8099;
+}
diff --git a/match-overrides.example.json b/match-overrides.example.json
new file mode 100644
index 0000000..d52ed16
--- /dev/null
+++ b/match-overrides.example.json
@@ -0,0 +1,15 @@
+{
+ "format": "kindle-zotero-importer.match-overrides.v1",
+ "overrides": [
+ {
+ "clipping_title": "Example Kindle title",
+ "resolution": {
+ "citation_key": "example2026"
+ },
+ "review": {
+ "status": "manager",
+ "clipping_count": 1
+ }
+ }
+ ]
+}
diff --git a/plugin/README.md b/plugin/README.md
index 9b0210f..079f8af 100644
--- a/plugin/README.md
+++ b/plugin/README.md
@@ -1,16 +1,16 @@
-# Kindle Zotero Importer Zotero Plugin
+# Kanzi Zotero Plugin
This is a hybrid Zotero plugin wrapper around the current Python importer. Zotero owns the user interface and writes native annotations through Zotero APIs; the Python project still performs parsing, matching, planning, EPUB positioning, and PDF positioning.
## Use
-1. Install `kindle-zotero-importer.xpi` in Zotero through `Tools > Plugins`.
-2. Open `Tools > Kindle Zotero Importer...`.
+1. Install `kanzi.xpi` in Zotero through `Tools > Plugins`.
+2. Open `Tools > Kanzi...`.
3. Select the updated Kindle `My Clippings.txt` in the manager.
4. Follow the current stage, percentage, and elapsed time in the manager.
5. Review created, existing, updated, failed, and unresolved counts when it completes.
-The plugin uses `match-overrides.json` from the project directory as the persistent source of title mappings and ignored titles. Generated JSON artifacts are still written to the project directory for audit and debugging.
+The XPI contains the importer Python source. On startup it extracts that source into the Zotero profile, where it keeps private mappings and generated audit artifacts. Python 3.11 or newer must be installed on the host; PDF positioning also requires Poppler (`pdftotext`, `pdftohtml`, and `pdfinfo`) and optionally uses `qpdf` for encrypted files. The plugin uses `match-overrides.json` there as the persistent source of title mappings and ignored titles.
## Build XPI
@@ -20,6 +20,6 @@ From the repository root:
python scripts/build_plugin.py
```
-The output is `dist/kindle-zotero-importer.xpi`.
+The output is `dist/kanzi.xpi`.
The plugin manifest targets Zotero `6.999` through `10.0.*`.
diff --git a/plugin/bootstrap.js b/plugin/bootstrap.js
index 0d11091..a157319 100644
--- a/plugin/bootstrap.js
+++ b/plugin/bootstrap.js
@@ -1,10 +1,9 @@
var KindleZoteroImporter = {
id: "kindle-zotero-importer@utkubilen.de",
menuItems: [],
- defaultProjectDir:
- "/home/user/Library/Mobile Documents/iCloud~md~obsidian/Documents/rhizome/06_projects/UTI/kindle-zotero-importer",
- defaultClippingsPath:
- "/home/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/test/My Clippings.txt",
+ runtimeProjectDir: "",
+ defaultProjectDir: "",
+ defaultClippingsPath: "",
prefBranch: "extensions.kindleZoteroImporter.",
rootURI: null,
managerWindow: null,
@@ -20,9 +19,41 @@ var KindleZoteroImporter = {
[["content", "kindle-zotero-importer", ""]]
);
await Zotero.initializationPromise;
+ await this.ensureRuntime();
this.addToAllWindows();
},
+ async ensureRuntime() {
+ const runtimeDir = PathUtils.join(PathUtils.profileDir, "kanzi-runtime");
+ const files = [
+ "plugin_runner.py",
+ "src/kindle_zotero_importer/__init__.py",
+ "src/kindle_zotero_importer/__main__.py",
+ "src/kindle_zotero_importer/cli.py",
+ "src/kindle_zotero_importer/clippings.py",
+ "src/kindle_zotero_importer/epub_position.py",
+ "src/kindle_zotero_importer/final_plan.py",
+ "src/kindle_zotero_importer/import_plan.py",
+ "src/kindle_zotero_importer/matcher.py",
+ "src/kindle_zotero_importer/mismatch_review.py",
+ "src/kindle_zotero_importer/overrides.py",
+ "src/kindle_zotero_importer/pdf_position.py",
+ "src/kindle_zotero_importer/zotero_index.py",
+ ];
+ await IOUtils.makeDirectory(runtimeDir, { createAncestors: true, ignoreExisting: true });
+ for (const relativePath of files) {
+ const destination = PathUtils.join(runtimeDir, ...relativePath.split("/"));
+ await IOUtils.makeDirectory(PathUtils.parent(destination), {
+ createAncestors: true,
+ ignoreExisting: true,
+ });
+ const response = await fetch(this.rootURI + "runtime/" + relativePath);
+ if (!response.ok) throw new Error(`Missing Kanzi runtime file: ${relativePath}`);
+ await Zotero.File.putContentsAsync(destination, await response.text());
+ }
+ this.runtimeProjectDir = runtimeDir;
+ },
+
shutdown() {
this.removeMenuItems();
if (this.managerWindow && !this.managerWindow.closed) {
@@ -89,7 +120,8 @@ var KindleZoteroImporter = {
getPref(name, fallback) {
try {
- return Services.prefs.getCharPref(this.prefBranch + name);
+ const value = Services.prefs.getCharPref(this.prefBranch + name);
+ return value || fallback;
} catch (_error) {
return fallback;
}
@@ -100,19 +132,21 @@ var KindleZoteroImporter = {
},
getProjectDir() {
- return this.getPref("projectDir", this.defaultProjectDir);
+ return this.getPref("projectDir", this.runtimeProjectDir || this.defaultProjectDir);
},
getPythonPath() {
- return this.getPref("pythonPath", "/opt/homebrew/bin/python3");
+ return this.getPref("pythonPath", "python3");
},
getZoteroDbPath() {
- return this.getPref("zoteroDbPath", "/home/user/Zotero/zotero.sqlite");
+ const dataDir = Zotero.DataDirectory && Zotero.DataDirectory.dir;
+ return this.getPref("zoteroDbPath", dataDir ? PathUtils.join(dataDir, "zotero.sqlite") : "");
},
getZoteroStorageRoot() {
- return this.getPref("zoteroStorageRoot", "/home/user/Zotero/storage");
+ const dataDir = Zotero.DataDirectory && Zotero.DataDirectory.dir;
+ return this.getPref("zoteroStorageRoot", dataDir ? PathUtils.join(dataDir, "storage") : "");
},
getConfigPath() {
@@ -143,6 +177,17 @@ var KindleZoteroImporter = {
zoteroDbPath: this.getZoteroDbPath(),
zoteroStorageRoot: this.getZoteroStorageRoot(),
};
+ if (settings.pythonPath === "python3") {
+ const candidates = Services.appinfo.OS === "Darwin"
+ ? ["/opt/homebrew/bin/python3", "/usr/local/bin/python3", "/usr/bin/python3"]
+ : ["/usr/bin/python3", "/usr/local/bin/python3"];
+ for (const candidate of candidates) {
+ if (await IOUtils.exists(candidate)) {
+ settings.pythonPath = candidate;
+ break;
+ }
+ }
+ }
try {
const text = await Zotero.File.getContentsAsync(this.getConfigPath());
const parsed = JSON.parse(text);
@@ -382,7 +427,7 @@ var KindleZoteroImporter = {
}
},
- async runPipeline(clippingsPath, settings, isFull = false) {
+ async runPipeline(clippingsPath, settings, isFull = false, retryUnresolved = false) {
const python = settings.pythonPath;
const projectDir = settings.projectDir;
const runner = projectDir + "/plugin_runner.py";
@@ -409,6 +454,9 @@ var KindleZoteroImporter = {
if (isFull) {
args.push("--full");
}
+ if (retryUnresolved) {
+ args.push("--retry-unresolved");
+ }
await this.exec(python, args);
return JSON.parse(await Zotero.File.getContentsAsync(summaryOutput));
@@ -420,10 +468,11 @@ var KindleZoteroImporter = {
return;
}
const isFull = manager.getFullReimport ? manager.getFullReimport() : false;
- await this.runManagedImportWithPath(manager, clippingsPath, isFull);
+ const retryUnresolved = manager.getRetryUnresolved ? manager.getRetryUnresolved() : false;
+ await this.runManagedImportWithPath(manager, clippingsPath, isFull, retryUnresolved);
},
- async runManagedImportWithPath(manager, clippingsPath, isFullOverride) {
+ async runManagedImportWithPath(manager, clippingsPath, isFullOverride, retryUnresolvedOverride) {
try {
if (!clippingsPath) {
throw new Error("No clippings file selected");
@@ -432,6 +481,9 @@ var KindleZoteroImporter = {
const progressPath = settings.projectDir + "/plugin-progress.json";
// Determine full vs incremental from explicit arg or manager checkbox
const isFull = typeof isFullOverride === "boolean" ? isFullOverride : (manager.getFullReimport ? manager.getFullReimport() : false);
+ const retryUnresolved = typeof retryUnresolvedOverride === "boolean"
+ ? retryUnresolvedOverride
+ : (manager.getRetryUnresolved ? manager.getRetryUnresolved() : false);
manager.beginRun(clippingsPath);
try {
await Zotero.File.removeIfExists(progressPath);
@@ -450,7 +502,7 @@ var KindleZoteroImporter = {
const pollPromise = poll();
let summary;
try {
- summary = await this.runPipeline(clippingsPath, settings, isFull);
+ summary = await this.runPipeline(clippingsPath, settings, isFull, retryUnresolved);
} finally {
polling = false;
await pollPromise;
@@ -464,8 +516,9 @@ var KindleZoteroImporter = {
const percent = total ? 96 + Math.floor((completed / total) * 4) : 100;
manager.updateProgress(percent, "Writing annotations", `${completed} of ${total} checked`);
});
+ await this.reconcileWriterState(summary.outputs.final_plan, finalPlan, results);
const data = await this.loadManagerData();
- manager.completeRun(summary, results, data);
+ await manager.completeRun(summary, results, data);
} catch (error) {
Zotero.logError(error);
manager.failRun(String(error && error.stack ? error.stack : error));
@@ -548,42 +601,143 @@ var KindleZoteroImporter = {
},
mergeComments(existingComment, newComment) {
- const stripBracket = (s) => String(s || "").replace(/^\s*\[[^\]]+\]\s*/, "").trim();
const seen = new Set();
const merged = [];
for (const comment of [existingComment, newComment]) {
for (const part of String(comment || "").split(/\n\s*\n/)) {
- const cleaned = part.trim();
+ const cleaned = this.stripColourCode(part);
if (!cleaned) continue;
- // dedup on stripped version too, so "[r] note" and "note" are same
- const key = stripBracket(cleaned).toLowerCase() || cleaned.toLowerCase();
- if (seen.has(cleaned) || seen.has(key)) {
- // if we have stripped version already, prefer stripped
- if (seen.has(key) && cleaned !== stripBracket(cleaned)) {
- // replace existing bracket version with stripped if we now have stripped
- const idx = merged.findIndex(p => stripBracket(p).toLowerCase() === key);
- if (idx >= 0) merged[idx] = stripBracket(cleaned);
- seen.add(cleaned);
- seen.add(key);
- continue;
- }
- continue;
- }
- seen.add(cleaned);
+ const key = cleaned.toLowerCase();
+ if (seen.has(key)) continue;
seen.add(key);
- // store stripped version if it had bracket
- const stripped = stripBracket(cleaned);
- if (stripped !== cleaned) {
- merged.push(stripped);
- seen.add(stripped);
- } else {
- merged.push(cleaned);
- }
+ merged.push(cleaned);
}
}
return merged.join("\n\n");
},
+ stripColourCode(value) {
+ return String(value || "")
+ .replace(/^\s*\[(?:y|yellow|o|orange|r|red|e|grey|gray|g|green|b|blue|bl|p|purple|m|magenta|pink)\]\s*/i, "")
+ .trim();
+ },
+
+ normalizeAnnotationText(value) {
+ return String(value || "")
+ .toLowerCase()
+ .normalize("NFKC")
+ .replace(/-\s*\n\s*/g, "")
+ .replace(/[^\p{L}\p{N}\s]+/gu, " ")
+ .replace(/\s+/g, " ")
+ .trim();
+ },
+
+ annotationPositionAnchor(annotation) {
+ const raw = annotation.annotationPosition || annotation.position;
+ let position = raw;
+ if (typeof raw === "string") {
+ try { position = JSON.parse(raw); } catch (_error) { return raw; }
+ }
+ if (!position || typeof position !== "object") return "";
+ if (position.type === "FragmentSelector" && position.value) {
+ const value = String(position.value);
+ const finalComma = value.lastIndexOf(",");
+ return `epub:${finalComma > 0 ? value.slice(0, finalComma) : value}`;
+ }
+ if (Number.isInteger(position.pageIndex) && Array.isArray(position.rects) && position.rects.length) {
+ const rect = position.rects[0] || [];
+ const x = Number(rect[0]);
+ const y = Number(rect[1]);
+ return `pdf:${position.pageIndex}:${x.toFixed(1)}:${y.toFixed(1)}`;
+ }
+ return this.normalizePosition(position);
+ },
+
+ areImportedDuplicates(left, right) {
+ const leftAnchor = this.annotationPositionAnchor(left);
+ const rightAnchor = this.annotationPositionAnchor(right);
+ if (!leftAnchor || leftAnchor !== rightAnchor) return false;
+ const leftPosition = this.normalizePosition(left.annotationPosition || left.position);
+ const rightPosition = this.normalizePosition(right.annotationPosition || right.position);
+ if (leftPosition && leftPosition === rightPosition) return true;
+ const leftText = this.normalizeAnnotationText(left.annotationText || left.text);
+ const rightText = this.normalizeAnnotationText(right.annotationText || right.text);
+ if (!leftText || !rightText) return false;
+ if (leftText === rightText) return true;
+ const shorter = leftText.length < rightText.length ? leftText : rightText;
+ const longer = shorter === leftText ? rightText : leftText;
+ return shorter.length >= 20 && longer.startsWith(shorter);
+ },
+
+ async collectImportedAnnotations() {
+ const annotations = [];
+ let allItems = [];
+ if (Zotero.DB && Zotero.DB.columnQueryAsync) {
+ const ids = await Zotero.DB.columnQueryAsync("SELECT itemID FROM itemAnnotations");
+ allItems = Zotero.Items.getAsync ? await Zotero.Items.getAsync(ids) : Zotero.Items.get(ids);
+ } else if (Zotero.Items.getAll) {
+ allItems = await Zotero.Items.getAll();
+ }
+ for (const item of allItems || []) {
+ if (!item || !item.isAnnotation || !item.isAnnotation()) continue;
+ if (this.hasKindleImportTag(item)) annotations.push(item);
+ }
+ return annotations;
+ },
+
+ async maintainImportedAnnotations(annotations, dryRun, results) {
+ if (!results.maintenanceDeletedIds) results.maintenanceDeletedIds = [];
+ const byAttachment = new Map();
+ for (const annotation of annotations) {
+ const parentID = annotation.parentItemID || annotation.parentID;
+ if (!byAttachment.has(parentID)) byAttachment.set(parentID, []);
+ byAttachment.get(parentID).push(annotation);
+ }
+
+ for (const group of byAttachment.values()) {
+ group.sort((a, b) =>
+ this.normalizeAnnotationText(b.annotationText).length -
+ this.normalizeAnnotationText(a.annotationText).length
+ );
+ const kept = [];
+ for (const annotation of group) {
+ const canonicalComment = this.mergeComments(annotation.annotationComment, "");
+ if (String(annotation.annotationComment || "") !== canonicalComment) {
+ annotation.annotationComment = canonicalComment;
+ results.updatedComments += 1;
+ if (!dryRun) await this.saveExistingAnnotation(annotation);
+ }
+
+ const duplicateOf = kept.find((candidate) => this.areImportedDuplicates(candidate, annotation));
+ if (!duplicateOf) {
+ kept.push(annotation);
+ continue;
+ }
+ const mergedComment = this.mergeComments(duplicateOf.annotationComment, annotation.annotationComment);
+ if (duplicateOf.annotationComment !== mergedComment) {
+ duplicateOf.annotationComment = mergedComment;
+ results.updatedComments += 1;
+ if (!dryRun) await this.saveExistingAnnotation(duplicateOf);
+ }
+ if (!dryRun) await annotation.eraseTx();
+ const keptClippingIds = new Set(
+ this.existingTags(duplicateOf)
+ .filter((tag) => tag.startsWith("kindle-id:"))
+ .map((tag) => tag.slice("kindle-id:".length))
+ );
+ for (const tag of this.existingTags(annotation)) {
+ const clippingID = tag.startsWith("kindle-id:")
+ ? tag.slice("kindle-id:".length)
+ : "";
+ if (clippingID && !keptClippingIds.has(clippingID)) {
+ results.maintenanceDeletedIds.push(clippingID);
+ }
+ }
+ results.removedDuplicates += 1;
+ }
+ }
+ },
+
existingSortIndex(annotation) {
return annotation.annotationSortIndex || annotation._annotationSortIndex || annotation.sortIndex || "";
},
@@ -607,6 +761,23 @@ var KindleZoteroImporter = {
return this.existingTags(annotation).includes("kindle-import");
},
+ commentForExistingAnnotation(annotation, newComment, previousImportedComment) {
+ if (!this.hasKindleImportTag(annotation) || previousImportedComment === undefined) {
+ return this.mergeComments(annotation.annotationComment, newComment);
+ }
+ const previousParts = new Set(
+ String(previousImportedComment || "")
+ .split(/\n\s*\n/)
+ .map((part) => this.stripColourCode(part).toLowerCase())
+ .filter(Boolean)
+ );
+ const manualParts = String(annotation.annotationComment || "")
+ .split(/\n\s*\n/)
+ .filter((part) => !previousParts.has(this.stripColourCode(part).toLowerCase()))
+ .join("\n\n");
+ return this.mergeComments(manualParts, newComment);
+ },
+
confirmRecreateForSortIndex(entry, existingAnnotation, oldSortIndex, newSortIndex) {
const tags = this.existingTags(existingAnnotation).join(", ") || "(none)";
const message = [
@@ -643,6 +814,37 @@ var KindleZoteroImporter = {
throw new Error("Existing annotation cannot be saved by this Zotero build");
},
+ async reconcileWriterState(path, plan, results) {
+ const plannedIds = new Set((plan.annotations || []).map((entry) => entry.clipping_id));
+ const failedIds = new Set(
+ (results.failed || [])
+ .map((failure) => failure.clipping_id)
+ .filter((clippingID) => plannedIds.has(clippingID))
+ );
+ plan.write_failed_ids = Array.from(failedIds).sort();
+ plan.delete_failed_ids = Array.from(new Set(results.failedDeletionIds || [])).sort();
+ const maintenanceDeletedIds = new Set(results.maintenanceDeletedIds || []);
+ if (failedIds.size) {
+ plan.annotations = (plan.annotations || []).filter(
+ (entry) => !failedIds.has(entry.clipping_id)
+ );
+ plan.all_annotations = (plan.all_annotations || []).filter(
+ (entry) => !failedIds.has(entry.clipping_id)
+ );
+ }
+ if (maintenanceDeletedIds.size) {
+ plan.annotations = (plan.annotations || []).filter(
+ (entry) => !maintenanceDeletedIds.has(entry.clipping_id)
+ );
+ plan.all_annotations = (plan.all_annotations || []).filter(
+ (entry) => !maintenanceDeletedIds.has(entry.clipping_id)
+ );
+ }
+ plan.delta_annotation_count = (plan.annotations || []).length;
+ plan.annotation_count = (plan.all_annotations || plan.annotations || []).length;
+ await Zotero.File.putContentsAsync(path, JSON.stringify(plan, null, 2) + "\n");
+ },
+
async writeAnnotations(plan, dryRun, onProgress) {
if (plan.format !== "kindle-zotero-importer.zotero-writer-plan.v1") {
throw new Error(`Unsupported plan format: ${plan.format}`);
@@ -660,75 +862,31 @@ var KindleZoteroImporter = {
recreateDeclined: 0,
removedDuplicates: 0,
deletedForIncremental: 0,
+ failedDeletionIds: [],
+ maintenanceDeletedIds: [],
failed: [],
};
- // Incremental deletions: remove annotations whose clipping_id is no longer present
- if (!dryRun && plan.deletions && plan.deletions.length) {
- const deletionIds = new Set(plan.deletions);
- try {
- // Find all annotations with kindle-id tag
- const allAnnotations = [];
- // Collect via attachments to be safe
- const allItems = Zotero.Items.getAll ? await Zotero.Items.getAll() : [];
- for (const item of allItems) {
- if (item.isAnnotation && item.isAnnotation()) {
- allAnnotations.push(item);
- } else if (item.getAnnotations) {
- const anns = item.getAnnotations();
- for (const a of anns) allAnnotations.push(a);
- }
- }
- // Deduplicate
- const seen = new Set();
- for (const ann of allAnnotations) {
- if (!ann || seen.has(ann.id)) continue;
- seen.add(ann.id);
- const tags = this.existingTags(ann);
- for (const tag of tags) {
- if (tag.startsWith("kindle-id:")) {
- const id = tag.substring("kindle-id:".length);
- if (deletionIds.has(id)) {
- await ann.eraseTx();
- results.deletedForIncremental += 1;
- break;
- }
- }
- }
- }
- } catch (e) {
- Zotero.logError(e);
+ let importedAnnotations = [];
+ let maintenanceReady = false;
+ try {
+ importedAnnotations = await this.collectImportedAnnotations();
+ await this.maintainImportedAnnotations(importedAnnotations, dryRun, results);
+ if (!dryRun && results.removedDuplicates) {
+ importedAnnotations = await this.collectImportedAnnotations();
}
+ maintenanceReady = true;
+ } catch (error) {
+ Zotero.logError(error);
+ results.failed.push({ clipping_id: "maintenance", message: String(error) });
}
const existingByAttachment = new Map();
- const normForDedup = (s) => String(s || "").toLowerCase().normalize("NFKC").replace(/-\s*\n\s*/g, "").replace(/[^\w\s]+/g, " ").replace(/\s+/g, " ").trim();
const getExistingAnnotations = async (attachment) => {
if (!existingByAttachment.has(attachment.id)) {
const annotations = attachment.getAnnotations ? attachment.getAnnotations() : [];
const byFingerprint = new Map();
- const byNorm = new Map();
for (const annotation of annotations) {
- const norm = normForDedup(annotation.annotationText || "");
- const dupNorm = byNorm.get(norm);
- if (dupNorm && norm) {
- // collapse identical highlights (e.g. My Clippings duplicates "forged its might…" x5)
- const kept = dupNorm;
- const keptHasComment = Boolean(kept.annotationComment);
- const curHasComment = Boolean(annotation.annotationComment);
- const duplicate = keptHasComment || !curHasComment ? annotation : kept;
- const replacement = duplicate === annotation ? kept : annotation;
- byNorm.set(norm, replacement);
- // also update fingerprint map
- const fpDup = this.existingAnnotationFingerprint(duplicate);
- const fpKeep = this.existingAnnotationFingerprint(replacement);
- byFingerprint.delete(fpDup);
- byFingerprint.set(fpKeep, replacement);
- if (!dryRun) await duplicate.eraseTx();
- results.removedDuplicates += 1;
- continue;
- }
- if (norm) byNorm.set(norm, annotation);
const fingerprint = this.existingAnnotationFingerprint(annotation);
const kept = byFingerprint.get(fingerprint);
if (!kept) {
@@ -754,6 +912,7 @@ var KindleZoteroImporter = {
};
let completed = 0;
+ const successfulEntries = new Map();
for (const entry of plan.annotations) {
try {
const attachment = Zotero.Items.get(entry.attachment_item_id);
@@ -771,19 +930,32 @@ var KindleZoteroImporter = {
const existingAnnotations = await getExistingAnnotations(attachment);
const fingerprint = this.annotationFingerprint(annotation);
- const existingAnnotation = existingAnnotations.get(fingerprint);
+ const existingAnnotation = existingAnnotations.get(fingerprint) ||
+ Array.from(existingAnnotations.values()).find(
+ (candidate) => this.areImportedDuplicates(candidate, annotation)
+ );
if (existingAnnotation) {
- const mergedComment = this.mergeComments(
- existingAnnotation.annotationComment,
- annotation.comment
+ const mergedComment = this.commentForExistingAnnotation(
+ existingAnnotation,
+ annotation.comment,
+ entry.previous_imported_comment
);
let changed = false;
- if (mergedComment && existingAnnotation.annotationComment !== mergedComment) {
+ if (existingAnnotation.annotationComment !== mergedComment) {
existingAnnotation.annotationComment = mergedComment;
changed = true;
results.updatedComments += 1;
}
+ const existingTags = new Set(this.existingTags(existingAnnotation));
+ for (const tag of annotation.tags || []) {
+ const name = tag.name || tag.tag;
+ if (!name || existingTags.has(name) || !existingAnnotation.addTag) continue;
+ existingAnnotation.addTag(name);
+ existingTags.add(name);
+ changed = true;
+ }
+
// colour update (British spelling in code handles both)
const existingColour = existingAnnotation.annotationColor || existingAnnotation.color || "";
const newColour = annotation.color || "";
@@ -836,12 +1008,14 @@ var KindleZoteroImporter = {
});
results.recreatedForSortIndex += 1;
results.skippedExisting += 1;
+ successfulEntries.set(entry.clipping_id, entry);
continue;
}
} else if (sortIndexChanged && dryRun) {
results.updatedSortIndex += 1;
}
results.skippedExisting += 1;
+ successfulEntries.set(entry.clipping_id, entry);
continue;
}
@@ -856,6 +1030,7 @@ var KindleZoteroImporter = {
annotationText: annotation.text || "",
});
results.created += 1;
+ successfulEntries.set(entry.clipping_id, entry);
} catch (error) {
results.failed.push({
clipping_id: entry.clipping_id,
@@ -870,6 +1045,40 @@ var KindleZoteroImporter = {
}
}
+ if (!dryRun && plan.deletions && plan.deletions.length) {
+ const deletionIds = new Set(plan.deletions);
+ const alreadyDeletedIds = new Set(results.maintenanceDeletedIds);
+ if (!maintenanceReady) {
+ results.failedDeletionIds.push(...deletionIds);
+ } else {
+ for (const ann of importedAnnotations) {
+ for (const tag of this.existingTags(ann)) {
+ if (!tag.startsWith("kindle-id:")) continue;
+ const id = tag.slice("kindle-id:".length);
+ if (!deletionIds.has(id) || alreadyDeletedIds.has(id)) break;
+ const replacement = successfulEntries.get(id);
+ if (replacement) {
+ const sameAttachment = (ann.parentItemID || ann.parentID) === replacement.attachment_item_id;
+ const sameFingerprint = this.existingAnnotationFingerprint(ann) ===
+ this.annotationFingerprint(replacement.annotation);
+ if (sameAttachment && sameFingerprint) break;
+ } else if ((plan.annotations || []).some((entry) => entry.clipping_id === id)) {
+ results.failedDeletionIds.push(id);
+ break;
+ }
+ try {
+ await ann.eraseTx();
+ results.deletedForIncremental += 1;
+ } catch (error) {
+ results.failedDeletionIds.push(id);
+ results.failed.push({ clipping_id: id, message: `Deletion failed: ${error}` });
+ }
+ break;
+ }
+ }
+ }
+ }
+
return results;
},
diff --git a/plugin/manager.html b/plugin/manager.html
index c031f28..69b94dc 100644
--- a/plugin/manager.html
+++ b/plugin/manager.html
@@ -90,6 +90,9 @@
.muted { color: var(--muted); font-size: 12px; }
.highlight-text { max-width: 520px; white-space: pre-wrap; word-break: break-word; font-size: 12px; line-height: 1.45; font-family: inherit; }
.integrated-meta { font-size: 10.5px; color: var(--muted); font-family: 'JetBrains Mono', monospace; }
+ .pagination { display: flex; align-items: center; justify-content: flex-end; gap: 8px; margin: 10px 0; color: var(--muted); font-size: 11px; }
+ .pagination button, .pagination select { width: auto; padding: 5px 8px; border: 1px solid var(--line); border-radius: var(--radius); background: var(--panel); color: var(--text); font: inherit; }
+ .pagination button:disabled { color: var(--faint); background: var(--panel-soft); cursor: not-allowed; }
::selection { background: var(--accent); color: white; }