Keeping the index in sync

A full re-index is the right move once. After that, you want the vector store to track your commits without re-walking the whole tree. That is what sai sync is for: it re-indexes only the files that changed, so the index stays in lockstep with HEAD. Wire it into git hooks and you never have to think about it again.

This page is a hook recipe set. For the underlying flags see the CLI reference; for first-time and full re-indexing see the indexing guide.

How sync decides what changed

sync computes a changed-file set, then for each path it does the same thing: delete that file’s existing points, then upload the current on-disk content fresh. There are three ways to supply the changed set, in priority order:

--file wins outright: when one or more --file paths are given, git is not consulted at all. Otherwise --staged runs git diff --cached, and the default path runs git diff against --since, whose default is HEAD~1 — i.e. the diff HEAD~1..HEAD, the files in the most recent commit.

If the changed set is empty, sync prints sync: no changed files and exits cleanly. That makes it safe to fire from a hook on every commit, even commits that touch nothing indexable.

Each changed file: deleted, then re-uploaded

For every path in the changed set, sync first deletes that file’s existing points — always, unconditionally — and then decides whether to re-upload:

• File exists and is indexable (wanted extension, passes the sai-cfg.yml globs, not an autogenerated file, and produces at least one chunk): it is chunked, embedded, and upserted fresh. Reported as + <path> (N chunks).
• File is gone, excluded, or empty of indexable content: the delete stands and nothing is re-uploaded. Reported as - <path> (<reason>).

The removal reasons you will see come straight from the binary:

This is the key property: a changed file that has become a test file, a generated file, or has been deleted is dropped from the collection rather than re-added. sync honors the exact same extension, glob, and generated-marker rules as a full index — so the index never drifts to include files a full re-index would skip. (See the opt-out markers guide for the autogenerated marker, and configuration for the exclusion config.)

At the end you get a one-line summary:

sync: 3 file(s) re-indexed (47 chunks), 1 file(s) removed

Hook templates

All hooks below assume the binary is at an absolute path and that QDRANT_URL / QDRANT_API_KEY are set in the hook’s environment when you use the Qdrant backend (offline DuckDB needs no such env). Use --silent in hooks to suppress timing, progress, and dirty-state warnings — the noise-free mode built for hooks and CI.

Run hooks from the repo root. sync resolves git diff paths against the current directory and stores them as each chunk’s payload path. Git runs hooks from the repo root by default, so the paths git reports match the stored payload paths — keep it that way. If you indexed a subtree with --root, keep your hook’s working directory and flags consistent with how you first indexed.

pre-commit — sync the staged set

.git/hooks/pre-commit re-indexes exactly what is about to be committed:

#!/bin/sh
# Re-index staged changes before the commit lands.
SAI=/abs/path/to/semanticastindexer
QDRANT_URL="https://<id>.<region>.aws.cloud.qdrant.io:6334" \
QDRANT_API_KEY="$QDRANT_API_KEY" \
  "$SAI" --ext ts,tsx sync --staged --silent >/dev/null 2>&1 &
exit 0

The trailing & backgrounds the sync and exit 0 lets the commit proceed immediately (see Don’t block the commit below).

post-commit / post-merge — sync the new commit

.git/hooks/post-commit (and the identical .git/hooks/post-merge) keep the index in lockstep with the commit that just landed:

#!/bin/sh
# Re-index the files in the commit/merge that just completed.
SAI=/abs/path/to/semanticastindexer
QDRANT_URL="https://<id>.<region>.aws.cloud.qdrant.io:6334" \
QDRANT_API_KEY="$QDRANT_API_KEY" \
  "$SAI" --ext ts,tsx sync --since HEAD~1 --silent >/dev/null 2>&1 &

--since HEAD~1 diffs the last commit, which is exactly the set of files just committed or merged. sync respects the same --ext and sai-cfg.yml filters as a full index.

pre-push — sync everything since the upstream

.git/hooks/pre-push is a good catch-all if you commit without per-commit hooks and want the index correct before code leaves your machine. Diff against the upstream tracking branch:

#!/bin/sh
# Re-index everything not yet on the upstream branch, before pushing.
SAI=/abs/path/to/semanticastindexer
UPSTREAM=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null)
if [ -n "$UPSTREAM" ]; then
  "$SAI" --ext ts,tsx sync --since "$UPSTREAM" --silent >/dev/null 2>&1 &
fi
exit 0

--since <rev> accepts any revision git understands, so @{u} (the upstream of the current branch) gives you the full set of files in unpushed commits.

husky

With husky, put the same commands in the managed hook files. For example .husky/post-commit:

SAI=/abs/path/to/semanticastindexer
"$SAI" --ext ts,tsx sync --since HEAD~1 --silent >/dev/null 2>&1 &

And .husky/pre-commit:

SAI=/abs/path/to/semanticastindexer
"$SAI" --ext ts,tsx sync --staged --silent >/dev/null 2>&1 &
exit 0

lefthook

With lefthook, add the syncs to lefthook.yml. Background the command so lefthook continues without waiting:

pre-commit:
  commands:
    sai-sync:
      run: /abs/path/to/semanticastindexer --ext ts,tsx sync --staged --silent >/dev/null 2>&1 &

post-commit:
  commands:
    sai-sync:
      run: /abs/path/to/semanticastindexer --ext ts,tsx sync --since HEAD~1 --silent >/dev/null 2>&1 &

Don’t block the commit

Embedding takes time, and the Qdrant backend talks to the network. Two pieces keep a git commit from stalling on it:

• & — background the sync so the hook returns immediately and git proceeds.
• >/dev/null 2>&1 — discard stdout and stderr so the backgrounded process does not bleed output into your terminal after the prompt returns. Pair it with --silent to keep the process quiet in the first place.

Add exit 0 to hooks where the background process is the last statement, so the hook’s exit status does not depend on the still-running sync.

Trade-off. Backgrounding means the commit succeeds even if the sync later fails (bad credentials, backend down). The index will be momentarily behind, and the next sync (or a full re-index) repairs it. If you would rather a broken index fail the commit, drop the & and let sync run in the foreground — and accept the latency.

A note on cost (DuckDB HNSW)

sync always wraps its work in a bulk window (begin_bulk / end_bulk). On Qdrant this is a no-op. On the DuckDB backend it is required for correctness, and it is not free: every sync drops the experimental HNSW index up front and rebuilds it at the end. DuckDB’s HNSW returns too few candidates after in-place deletes — recall degrades — and hnsw_compact_index does not fix it; dropping and rebuilding the index is the only way to restore full recall.

The practical consequence: syncing a single file on DuckDB still pays a full index rebuild. For a small or medium index this is cheap; for a very large one it can dominate the hook’s runtime, which is another reason to background the call. See performance for the rebuild cost, and backends and embedders for the DuckDB vs. Qdrant trade-offs.

See also

• CLI reference — every sync flag and default.
• Indexing — first-time and full re-indexing.
• CI/CD — keeping the index fresh from pipelines instead of local hooks.
• Opt-out markers — how files are excluded and the autogenerated marker.
• Performance — the DuckDB HNSW rebuild cost.