Sample run · Aircraft engineering reports
A playback of what Lakshana discovered when run on 30 aerospace documents across 3 types. You're seeing the real output, just pre-computed.
What's in this corpus
30 synthetic but realistic aerospace documents: Maintenance Inspection Reports (A/B/C/D-checks, ATA-chapter referenced), Flight Test Reports (CAR-23 / CS-25 acceptance tests), and Service Bulletins (manufacturer modification advisories). Fleet includes A320neo, A350-900, B787-8, ATR 72-600, and others.
Lakshana was given no labels. It clustered the documents into the 3 implicit types, named each one, inferred a schema per type, and verified field frequencies. The output you'll see across the next steps is what came out — UMAP coordinates, cluster cards, schemas, coverage, and the lot.
Lakshana was given no labels. It clustered the documents into the 3 implicit types, named each one, inferred a schema per type, and verified field frequencies. The output you'll see across the next steps is what came out — UMAP coordinates, cluster cards, schemas, coverage, and the lot.
Model used
—
Wall time
—
Documents
—
Clusters found
—
Browse the corpus
Click any document to read it. These are the raw files Lakshana saw.
Analyzing Documents
Extracting structure, embedding, clustering, and discovering schemas...
Starting...
Live Log
Document Clusters
Each point is a document. Colors represent discovered document types. Click any point or cluster card.
How clustering works:
1. Each document is converted into a numerical "fingerprint" combining structural features (dates, amounts, key-value pairs, headers, tables) with semantic meaning (sentence-transformer embeddings).
2. UMAP (Uniform Manifold Approximation and Projection) reduces these high-dimensional fingerprints to 2D/3D coordinates while preserving the relative distances — similar documents stay close.
3. HDBSCAN (Hierarchical Density-Based Spatial Clustering) automatically detects groups of documents that are close together, without needing to specify the number of clusters upfront. It handles outliers gracefully.
4. For each cluster, an LLM analyzes sample documents to generate a human-readable name, description, and list of distinguishing keywords.
Structural weight: 60% | Semantic weight: 40% — template structure matters more than content for grouping.
1. Each document is converted into a numerical "fingerprint" combining structural features (dates, amounts, key-value pairs, headers, tables) with semantic meaning (sentence-transformer embeddings).
2. UMAP (Uniform Manifold Approximation and Projection) reduces these high-dimensional fingerprints to 2D/3D coordinates while preserving the relative distances — similar documents stay close.
3. HDBSCAN (Hierarchical Density-Based Spatial Clustering) automatically detects groups of documents that are close together, without needing to specify the number of clusters upfront. It handles outliers gracefully.
4. For each cluster, an LLM analyzes sample documents to generate a human-readable name, description, and list of distinguishing keywords.
Structural weight: 60% | Semantic weight: 40% — template structure matters more than content for grouping.
Document Map
UMAP projection: documents are positioned so that structurally similar documents appear near each other. The axes are abstract coordinates — only relative distances matter, not the absolute position. Hover to see document names. Click a point to preview the document.
Click a document
📄
Click any point on the map to preview the document here.
Discovered Document Types
Each card represents a group of structurally similar documents. The name and description are generated by the LLM after analyzing representative samples from the cluster. Keywords are the most distinguishing terms. Click "View Schema" to see the discovered fields for that document type.
Discovered Schema
Review and edit the discovered fields. Toggle required, rename, or remove fields.
How schema discovery works:
The LLM analyzes 5-8 sample documents from each cluster using iterative schema discovery (ReDD pattern): it starts with an empty schema, processes documents one by one, and evolves the schema with each new document — adding new fields and refining descriptions.
Frequency measures how many documents in the cluster contain each field. It's computed by checking whether the field's name keywords or description keywords appear in each document's text.
• ● ≥80% = Required (appears in most documents)
• ● 40-79% = Common (appears often but not always)
• ● <40% = Rare (appears occasionally — consider removing)
All fields are editable. Rename, change types, toggle required, or remove noise fields before exporting.
The LLM analyzes 5-8 sample documents from each cluster using iterative schema discovery (ReDD pattern): it starts with an empty schema, processes documents one by one, and evolves the schema with each new document — adding new fields and refining descriptions.
Frequency measures how many documents in the cluster contain each field. It's computed by checking whether the field's name keywords or description keywords appear in each document's text.
• ● ≥80% = Required (appears in most documents)
• ● 40-79% = Common (appears often but not always)
• ● <40% = Rare (appears occasionally — consider removing)
All fields are editable. Rename, change types, toggle required, or remove noise fields before exporting.
20%
Frequency filter: Slide to hide fields that appear in fewer than X% of the cluster's documents. This helps focus on the most consistent fields and filter out noise. Fields with <20% frequency are often extraction artifacts.
Schema Coverage
0%
Coverage = percentage of all field×document cells that are filled. If you have 6 fields and 10 verified documents, there are 60 possible cells. If 52 of those cells have a value, coverage is 87%. Higher coverage means the selected fields comprehensively capture the information in your documents. Remove low-frequency fields to see how coverage changes — sometimes dropping rare fields barely affects coverage while simplifying the template.
| Field | Type | Frequency | Required | Description | Example |
|---|
Frequency = percentage of documents in this cluster where the field was detected. Computed by keyword matching: if the field name or description words appear in the document text, the field is considered present. This is an approximation — actual extraction may differ.
Field Frequency
Fields sorted by how consistently they appear across documents in this cluster. Green = required (≥80%). Orange = optional. Longer bars = more reliable fields.
Schema Map
Each rectangle represents a field. Size = frequency (bigger = appears more often). Color = data type: string, number, date, boolean. Hover for details.
Schema Tree
Interactive tree visualization of the discovered schema. The root is the template name, branches are fields, and leaves show properties (type, frequency, example). Click any node to expand or collapse. Colors indicate data type.
Create Extraction Project
Build a full Structure project from the discovered schema. Preview extraction, review annotations, and launch.
1
Enrich Schema
Adds descriptions, patterns, and enum values to each discovered field for better extraction accuracy.
2
Select Examples
Picks 3 diverse representative documents and auto-generates few-shot annotations for the extraction model.
3
Preview Extraction
Runs extraction on 2-3 sample documents so you can verify the results before committing.
4
Field Quality
Shows per-field quality metrics and confidence indicators based on the preview results.
5
Launch Extraction
Creates the Structure project with enriched schema, few-shot examples, and all cluster documents. Ready to run full extraction.
or press ⌈⌘⏎⌋
Export
Export the discovered schema or create a Structure project from it.
Schema Template Library
Loading templates...