Sync & Merge
Sync and merge are two related but distinct systems in Stoneforge. Data sync keeps project data (tasks, documents, dependencies) consistent across branches using JSONL files. Branch merge integrates completed agent work back into the main branch using the MergeStewardService. Together, they ensure your project stays coherent across parallel work streams.
Data sync: JSONL as source of truth
Stoneforge’s dual storage model uses SQLite for fast runtime queries and JSONL files for durable, git-friendly persistence. Sync is the bridge between them:
- Export (SQLite → JSONL): Serialize element state from the database to
.stoneforge/sync/files - Import (JSONL → SQLite): Read JSONL files and merge them into the SQLite database
The JSONL files are committed to Git and travel with your repository. When branches diverge and reconverge, the import process uses content hashing and merge strategies to reconcile differences.
Content hashing
Every element has a content_hash — a hash computed from its semantic content. Before merging two versions of the same element, Stoneforge compares their hashes:
- Identical hashes → skip (no conflict, no work needed)
- Different hashes → apply merge strategies to decide which version wins
The hash covers all fields that represent the element’s actual content. These fields are excluded from the hash because they don’t represent semantic changes:
id(identity, never changes)createdAt(set once at creation)updatedAt(changes on every write, not a content field)createdBy(set once at creation)contentHash(the hash itself)
Merge strategies
When two versions of the same element have different content hashes, Stoneforge applies a set of deterministic rules to decide which version wins.
Resolution rules
| Scenario | Winner | Rationale |
|---|---|---|
| Same content hash | Skip (identical) | No conflict exists |
| Fresh tombstone vs. live | Tombstone wins | Deletion should propagate |
| Expired tombstone vs. live | Live wins | Old deletions shouldn’t resurface |
| Closed vs. open | Closed wins | Completion is sticky |
| Both same status | Later updatedAt wins | Last-write-wins tiebreaker |
| Tags differ | Set union | Never lose tags in merge |
force: true | Remote always | Manual override for rebuilds |
Tombstone handling
Soft-deleted elements have a deletedAt timestamp. Tombstones are classified by age:
- Fresh tombstone (within TTL, default 30 days): Wins over a live element. The deletion should propagate to other branches.
- Expired tombstone (past TTL): Loses to a live element. An ancient deletion shouldn’t override an element that’s been recreated or kept alive elsewhere.
Status merge
Closed status is sticky — if one version is closed and the other is open, closed always wins regardless of timestamps. This prevents a scenario where an agent on a stale branch accidentally reopens a completed task.
Tag merge
Tags use set union — they’re combined, never removed. If the local version has tags ["api", "auth"] and the remote has ["auth", "testing"], the merged result is ["api", "auth", "testing"].
This means you can never lose tags through sync. If you need to remove a tag, do it explicitly after the merge.
Dependency merge
Dependencies use a removal-wins strategy. If one side removed a dependency that existed in the original, the removal is honored. If one side added a new dependency, it’s kept. This prevents deleted dependencies from “resurrecting” during sync.
Conflict records
Every merge decision is recorded as a ConflictRecord for auditability:
| Resolution type | Meaning |
|---|---|
IDENTICAL | Same content hash — no conflict |
LOCAL_WINS | Local version kept (newer or force-local) |
REMOTE_WINS | Remote version kept (newer or closed) |
TAGS_MERGED | Tags combined from both versions |
DEPENDENCY_ADDED | New dependency imported from remote |
DEPENDENCY_REMOVED | Dependency removal honored |
Each record includes both content hashes, both updatedAt timestamps, and the resolution timestamp. This lets you trace exactly what happened during any sync operation.
The sync loop
Here’s the full developer workflow for keeping data in sync across branches:
┌─────────────────────────────────────────────────────┐│ Sync Loop ││ ││ 1. Developer makes changes (SQLite modified) ││ │ ││ ▼ ││ 2. Export: SQLite → JSONL (sf sync export) ││ │ ││ ▼ ││ 3. Git add / commit / push ││ │ ││ ▼ ││ 4. Other developer: git pull ││ │ ││ ▼ ││ 5. Import: JSONL → SQLite (sf sync import) ││ (merge resolution happens here) ││ │ ││ ▼ ││ 6. Export: SQLite → JSONL (normalize) ││ │ ││ ▼ ││ 7. Git commit (clean state) ││ │└─────────────────────────────────────────────────────┘Steps 6–7 normalize the JSONL files after import, ensuring the exported state matches what’s in SQLite. This prevents drift between the two layers.
Git conflict resolution
When git pull or git merge produces conflicts in the JSONL files, don’t try to resolve them manually. Let Stoneforge handle the semantic merge:
-
Accept both versions — For the conflicted JSONL files, accept both sides by concatenating all lines. Duplicate entries are fine.
-
Import — Run
sf sync import. Stoneforge reads all lines, deduplicates by element ID, and applies the merge strategies described above. -
Export — Run
sf sync export. This writes the clean, merged state back to the JSONL files, removing any duplicates or conflicts. -
Commit — Commit the resolved JSONL files. The data is now cleanly merged.
Branch merge: the MergeStewardService
While data sync handles JSONL files, branch merge handles code. When a worker completes a task, its code changes need to be tested and merged into the main branch. This is the job of the MergeStewardService.
The merge flow
Task branch (agent/e-worker-1/el-3a8f-add-login) │ │ 1. Worker completes task │ (sf task complete el-3a8f) │ ▼┌──────────────────┐│ Task moves to ││ REVIEW status ││ mergeStatus: ││ "pending" │└────────┬─────────┘ │ │ 2. Daemon assigns to merge steward │ ▼┌──────────────────┐│ Steward creates ││ temp worktree │──── .stoneforge/.worktrees/_merge-el-3a8f/│ (detached HEAD │ checked out at origin/master│ at origin/master)│└────────┬─────────┘ │ │ 3. Run tests in temp worktree │ ┌────┴──────────┐ ▼ ▼┌────────┐ ┌──────────┐│ PASS │ │ FAIL │└───┬────┘ └────┬─────┘ │ │ │ │ Create fix task │ │ with test output │ ▼ │ Task stays │ in REVIEW │ │ 4. Squash merge in temp worktree │ git merge --squash <task-branch> │ │ 5. Commit: "Add login (el-3a8f)" │ │ 6. Push: git push origin HEAD:master │ │ 7. Remove temp worktree │ │ 8. Fast-forward local master │ │ 9. Delete task branch + worktree │ ▼┌──────────────────┐│ Task moves to ││ CLOSED status ││ mergeStatus: ││ "merged" │└──────────────────┘Why detached HEAD
The merge happens in a temporary worktree with a detached HEAD at origin/master (the merge steward auto-detects the repository’s default branch, and the target branch can be customized via config). This design has two important properties:
-
Main repo is never at risk — If the merge fails, crashes, or is interrupted, the main repository’s HEAD is untouched. The temporary worktree is cleaned up in a
finallyblock. -
No branch locking — Git prevents two worktrees from checking out the same branch. By using detached HEAD, the merge worktree doesn’t lock the default branch, so local sync operations can continue in the main repository.
Two-phase merge
The merge runs in two phases:
- Phase A — Merge and push in the temporary worktree. This is the critical path: squash merge, commit, push to remote.
- Phase B — After the temporary worktree is cleaned up, sync the local main branch with the remote via fast-forward. This is best-effort — if it fails, the remote is already updated.
Standard merge alternative
By default, Stoneforge uses squash merge — all commits from the task branch are combined into a single commit on main. This keeps the main branch history clean and linear.
If you prefer to preserve the full commit history from task branches, you can switch to standard merge:
| Strategy | Commit history | Main branch |
|---|---|---|
squash (default) | Single commit per task | Clean, linear |
merge | All original commits preserved | Merge commits present |
Standard merge uses git merge --no-ff instead of git merge --squash, creating a merge commit that preserves the full branch history.
Merge status tracking
Tasks in REVIEW status track their merge progress through a mergeStatus field in metadata:
| Status | Meaning | Next action |
|---|---|---|
pending | Awaiting merge steward | Steward will run tests |
testing | Tests currently running | Wait for results |
merging | Merge in progress | Wait for completion |
merged | Successfully merged | Task moves to CLOSED |
test_failed | Tests failed | Fix task created |
conflict | Merge conflict detected | Fix task created |
failed | Merge failed (other reason) | Fix task created |
not_applicable | No merge needed | Task moves to CLOSED |
When tests fail or a conflict is detected, the merge steward creates a fix task with the test output or conflict details, and assigns it back to the pool. The original task stays in REVIEW until the fix is merged.
Merge configuration
The MergeStewardService is configurable:
| Option | Default | Description |
|---|---|---|
mergeStrategy | 'squash' | 'squash' (single commit) or 'merge' (preserve history) |
autoPushAfterMerge | true | Push target branch to remote after merge |
autoCleanup | true | Clean up task worktree after merge |
deleteBranchAfterMerge | true | Delete source branch (local + remote) after merge |
External sync: connecting to external tools
While the internal JSONL sync keeps data consistent across git branches, external sync connects Stoneforge to the tools your team already uses — syncing tasks with GitHub Issues or Linear, and documents with Notion or a local folder.
Providers
External sync has a strict separation: each provider handles either tasks or documents, never both.
| Provider | Type | External system | Auth |
|---|---|---|---|
| GitHub | Task | GitHub Issues | Personal access token (ghp_...) |
| Linear | Task | Linear Issues | API key (lin_api_...) |
| Notion | Document | Notion Pages | Integration token (ntn_...) |
| Folder | Document | Local filesystem (Markdown) | None |
How it works
Each synced element stores an _externalSync metadata entry that tracks the provider, external ID, URL, and content hashes. The sync engine uses content hashing to detect changes on either side:
┌─────────────────────┐ ┌─────────────────────┐│ Stoneforge │ │ External System ││ │ push → │ ││ Tasks │ │ GitHub Issues ││ Documents │ ← pull │ Linear Issues ││ │ │ Notion Pages ││ │ sync ⇄ │ Local Folder │└─────────────────────┘ └─────────────────────┘- Push — Exports local changes to the external system. Compares the current content hash against
lastPushedHashto skip unchanged elements. - Pull — Imports changes from the external system. Uses a per-provider cursor to fetch only items modified since the last pull.
- Sync — Bidirectional: push first, then pull. Detects conflicts when both sides changed.
Task field mapping
Task sync maps Stoneforge fields to external issue fields:
| Stoneforge field | GitHub | Linear |
|---|---|---|
status | open/closed state + sf:status:* labels | Workflow state type (started, completed, etc.) |
priority | sf:priority:* labels | Native priority (0–4 scale mapped to 1–5) |
taskType | sf:type:* labels | sf:type:* labels |
tags | User labels (unprefixed) | User labels (unprefixed) |
| Description document | Issue body text | Issue description |
Document field mapping
Document sync maps Stoneforge documents to external pages:
| Stoneforge field | Notion | Folder |
|---|---|---|
title | Page title | Filename (slugified) |
content | Page blocks (Markdown ↔ Notion blocks) | Markdown file content |
category | Category property | YAML frontmatter |
tags | Multi-select property | YAML frontmatter |
libraryPath | Library select property | Subdirectory hierarchy |
System documents (task descriptions, message content) and untitled documents are excluded from sync.
Conflict resolution
When both sides change the same element between sync cycles, a conflict occurs. Four resolution strategies are available:
| Strategy | Behavior |
|---|---|
last_write_wins (default) | Most recent updatedAt timestamp wins |
local_wins | Stoneforge version always wins |
remote_wins | External version always wins |
manual | Element tagged sync-conflict for human resolution |
The conflict resolver also supports field-level merge — when different fields changed on each side, both changes can be applied automatically. Only same-field changes are true conflicts.
Linking elements
Before an element can sync, it must be linked to an external item:
# Link a single task to a GitHub issuesf external-sync link el-3a8f https://github.com/org/repo/issues/42
# Bulk-link all unlinked tasks to GitHubsf external-sync link-all -p github --project org/repo
# Auto-link new tasks as they're createdsf external-sync config set-auto-link githubBackground daemon
When external sync is enabled and at least one provider is configured, the orchestrator runs a background polling daemon. It calls sync() (push then pull) at a configurable interval (default: 60 seconds).
Recovery
If the SQLite database corrupts or gets out of sync with the JSONL files:
# Delete the cacherm .stoneforge/stoneforge.db
# Rebuild from JSONLsf sync importThe JSONL files are the source of truth. The database can always be regenerated from them.