Architecture

Saga-Scribe is a Tauri desktop application:

• Frontend: Svelte 5 + SvelteKit (TypeScript), with CodeMirror 6 as the editor.
• Backend: Rust + Tauri 2.9, including the shorthand compiler, the TEI importer and everything that touches files.
• XML: libxml2, accessed through a small wrapper that declares the document’s entities before parsing and disables network access.
• PDF: Vivliostyle, in an off-screen iframe, for CSS Paged Media layout.

Project layout

saga-scribe/
├── src/                          # Frontend
│   ├── lib/
│   │   ├── components/           # UI components
│   │   ├── editor/               # Editor support (theme, edit history, typing)
│   │   ├── parser/               # Editor highlighting grammar (Lezer)
│   │   ├── rendered/             # Rendered-text preview
│   │   ├── stores/               # Svelte stores (Svelte 5 runes)
│   │   ├── utils/                # XSLT, export, PDF pagination, …
│   │   └── tauri.ts              # Calls into the backend
│   └── routes/                   # Main window and the print window
├── src-tauri/src/                # Backend
│   ├── parser/                   # Shorthand compiler (lexer, AST, word tokenizer)
│   ├── importer/                 # TEI-XML and text import, round-trip patching
│   ├── commands/                 # Tauri commands
│   ├── annotations/, metadata/, dictionary/, entities/,
│   │   normalizer/, settings/, stylesheet/, template/, validator/
│   └── identity.rs               # Which word is which across edits
├── static/                       # Fonts, entity and dictionary data, schemas, XSLT
├── docs/                         # User guide, UI standards
├── flatpak/                      # Flatpak manifest and AppStream metadata
└── tests/fixtures/               # Shared test inputs (e.g. the shorthand corpus)

The compile pipeline

1. Lexing and parsing — src-tauri/src/parser/syntax.rs produces a lossless syntax tree with spans and error recovery. Every node records the byte range it came from.
2. Word tokenization — src-tauri/src/parser/wordtokenizer.rs groups text and inline constructs into words and punctuation, handling spaces, ~, | and breaks inside words.
3. Compilation — src-tauri/src/parser/compiler.rs writes the TEI body, applying the template’s options, lemmas and annotations.
4. Patching (imported mode) — src-tauri/src/importer/tei/patching.rs splices the compiled edits into the original document bytes, leaving everything else alone.

The editor’s highlighting grammar (src/lib/parser/tei-dsl.grammar, generated from src/lib/parser/tokens.ts) is a rule-for-rule copy of the Rust scanner’s logic. Change both together, run bun run build:grammar, and regenerate the shared corpus:

cd src-tauri; UPDATE_DSL_CORPUS=1 cargo test corpus

The corpus test (src/lib/parser/corpus.test.ts) catches drift: the editor must find the same constructs the compiler does.

Key files

• DSL parser: src-tauri/src/parser/syntax.rs, src-tauri/src/parser/compiler.rs
• Word tokenizer: src-tauri/src/parser/wordtokenizer.rs
• Editor grammar: src/lib/parser/tei-dsl.grammar, src/lib/parser/tokens.ts
• Entities: static/entities/menota.json, src-tauri/src/entities/registry.rs
• Editor: src/lib/components/Editor.svelte
• ONP dictionary: src-tauri/src/dictionary/onp.rs
• Compile session and renumbering: src/lib/stores/compileSession.ts, src-tauri/src/identity.rs

Keeping the UI responsive

Large manuscripts can have 50,000 or more tokens. Any synchronous operation over that much data freezes the window and blocks animations. The project has two rules:

Typing comes first. A keystroke must appear in the editor at once, whatever the document’s size. Work that redraws the previews waits for a pause in typing (whenTypingPauses). Per-word work in the rendered text must be a lookup, never a scan of all annotations.

Heavy backend work runs off the UI thread. Tauri commands that do heavy work are #[tauri::command(async)] and use tauri::async_runtime::spawn_blocking() for CPU-bound work.

The performance log (--perf-log) records how long each stage takes, from both sides, and logs late frames as ui.stall. When adding heavy work, time it there.

Word identity

Lemmas and annotations are stored by word number: the n-th <w> of the body. When words are inserted or deleted, those numbers shift. The backend therefore tracks which token is which across edits and reports, after each compile, where the previous compile’s words went (renumbering). The frontend applies that report before the next compile gathers its payload, so annotations sent to the backend are always numbered by its previous compile.

Anything new that stores word numbers must be renumbered in onRenumbering.

XSLT output

All XSLT output passes through sanitizeXsltHtml() (src/lib/utils/xslt.ts) before it is rendered or exported. The application runs under a strict content security policy.

Stylesheets that declare <?saga-scribe render="page"?> are applied one page at a time and re-run only on changed pages. Such a stylesheet’s output for an element may depend only on the element and its ancestors: no preceding::/following::, no keys, no document-wide counts. Document-wide facts are passed as parameters instead (for example lineBased).