Measured, not claimed

Validation engines compared

Beleggo, Mustangproject, phive and 21 more engines compared

24 validation engines check the same 722 e-invoices from the Beleggo data kit — XRechnung, ZUGFeRD / Factur-X, Peppol BIS 3 and national CIUS, valid and defective — on the same machine, one document at a time. The official KoSIT validator runs alongside as the reference. Mustangproject and phive, the most widely used open-source validators, then stand next to Beleggo in detail.

Measured on 30 September 2026 on linux x64, Intel(R) Xeon(R) Processor @ 2.10GHz × 4, 16 GiB, Node v22.22.2

All 24 engines at a glance

“Profiles”: how many of the kit’s 11 profiles the engine itself claims to check. “Correct”: the share of documents in those profiles whose verdict (valid or defective) matches the expected one; a document the engine cannot judge counts as wrong. As every engine covers different profiles, the percentages rest on different numbers of documents. The KoSIT validator is the official reference for XRechnung. Measured 25 – 30 September 2026.

“Rule named”: of the 338 defective samples across the kit, how many the engine rejects while naming the broken rule. “Median”: time per document after warm-up; “Memory”: peak of the whole process.

In detail: Beleggo, Mustangproject and phive

Correct verdicts

Speed and cost

Beleggo runs the official Schematron rules as code translated to JavaScript, read from the same stylesheets Saxon-JS runs and checked against Saxon-JS as the reference on every test corpus: the same findings, the same text, the same order. A document the translated rules cannot judge exactly goes to Saxon-JS. CPU time is above the time per document because Node.js collects garbage on threads of its own. The same engine runs in the browser.

What each engine does

Where and why they differ

Mustang departs from the expected verdict on 54 of 722 documents, 24 of them in profiles it does not check at all. The rest have five causes:

• Mustang does not apply the severities of the official XRechnung configuration: it misses a rule KoSIT raises to an error and rejects on one KoSIT lowers to a warning.
• It checks XRechnung in CII against the narrower Factur-X schema instead of CII D16B, and so reports errors no official validator sees.
• A ZUGFeRD MINIMUM file is valid Factur-X but not an e-invoice under § 14 UStG. Beleggo says so and Mustang does not — a question of law, not of technology.
• Mustang ships the French CTC rules in the older edition 1.3.0, in which almost all of them only warn, and for CII only. It accepts a French invoice that breaks them; from 1 October 2026 that invoice is defective.
• It checks XRechnung with Schematron 2.4.0. What has been tightened since — BR-TMP-2, an attachment link without https://, for one — it reports only as a warning.

By profile

How it was measured

Data kit 0.1.0 (969f6f3ec8fdccbc), 722 documents, judged against each sample’s expected verdict. That verdict comes from the Beleggo project; so that it confirms more than Beleggo, every Mustang disagreement was checked by hand against the specifications, and the official KoSIT validator runs alongside as an independent reference. Where it judges differently from Beleggo, it lacks the rule: the French CTC rules and one Factur-X code list.

Beleggo artefacts 2026-08-31, Mustangproject 2.26.0, phive 4.6.0 (with every phive-rules module; phive's own document detection picks the ruleset), KoSIT validator 1.6.3 · Validator Configuration XRechnung 3.0.2 2026-09-02. Each engine runs in its own process that lives for the whole measurement — so the times measure the engine, not a Java VM starting up.