Meta models at work
Which Meta models read your documents and answer your questions, whether each is up right now, and how they scored on our test sets.
The chain
Each AI call goes to the first provider that is up; if it times out, errors or is out of credit, the next one takes the call. All three serve Meta models. If you chose your own AI in Settings, your calls go only there.
Checking each provider…
Last AI calls
The last 50 calls this server made, from memory only (a restart empties it). No document or question content is kept.
How they scored
From the committed eval runs, on made-up documents: real phone photos will score lower.
Receipts and invoices, 200 clean
Muse Spark Open · hosted open weights
- Documents
- 200
- Every field right
- 91 %
- Proposed vs Needs a look right
- 94 %
- Mean per document
- 12.9 s
Receipts and invoices, 200 messy
Muse Spark Open · hosted open weights
- Documents
- 200
- Every field right
- 93 %
- Proposed vs Needs a look right
- 97 %
- Mean per document
- 13.12 s
Receipts and invoices, 200 clean
Muse Glimmer 30B · hosted GPU
- Documents
- 200
- Every field right
- 85 %
- Proposed vs Needs a look right
- 88 %
- Mean per document
- 17.72 s
Receipts and invoices, 200 messy
Muse Glimmer 30B · hosted GPU
- Documents
- 200
- Every field right
- 80 %
- Proposed vs Needs a look right
- 83 %
- Mean per document
- 24.14 s
Ask: 35 questions over the seeded books
Muse Glimmer 30B · host not recorded in the run
- Numbers exact
- 32/32
- Invented figures shown
- 0
- Citations present
- 32/32
- Median per question
- 16.4 s
Autopilot: documents posted without a person
Muse Glimmer 30B · our own server
- Documents
- 467
- Posted
- 149
- False posts
- 17 (4 %)
- Without the seeded copies
- 0 (0 %)