What is intelligent document processing?

Intelligent document processing, or IDP, turns a document into text, structure and named values that software can use. A work order becomes a record with a number, a part, a date and the torque values in its table.

It sits between two older ideas. Optical character recognition, OCR, turns a picture of text into text and stops there. Template capture reads fixed boxes on a known form, and breaks when the form changes. IDP adds models that find the layout, recognize the kind of document and locate a field wherever it sits on the page.

IBM’s explainer names three activities that stay the same across vendors: classification, extraction and output. It calls data extraction “the core activity whereby important, relevant information is pulled off the page.”

A document passes through five steps. Not every document needs all of them.

• Read. A born digital file, such as a PDF exported from a word processor, already holds its text, and a parser reads it out. A scan or a phone photo holds only pixels, so OCR recognizes the characters, page by page.
• Find the layout. Headings, paragraphs, tables, figures and footers are marked as regions. This keeps a table’s rows together and a page footer out of the text.
• Classify. Each document is sorted by type: a work order, a supplier notice, a test report, an invoice. The type decides which fields to look for.
• Capture fields. Named values are read out: the work order number, the part, the date, the torque value. Each comes with a confidence.
• Check and hand over. Values the system is sure of go on to the next system. The rest go to a person first.

Email and archives add one more step. A message is a body plus attachments, and each attachment is a file that goes through the same steps on its own.

Many errors that reach a user start before any field is captured. If two columns are read as one, sentences from both interleave. If a table is read as lines of text, a value loses its column header. Then 40 Nm no longer says which bolt it belongs to.

Microsoft’s layout model documentation splits layout into geometric roles, such as text, tables and figures, and logical roles, such as titles, headings and footers. Both matter. A heading tells a reader which procedure a step belongs to.

Layout models also depend on what they were trained on. DocLayNet is a dataset of 80,863 pages annotated by hand. Its authors found that models trained only on scientific articles lost accuracy on more varied layouts. Manuals, forms and drawings from your own archive are the test that counts.

In an industrial company much of what is known sits in documents: maintenance logs, supplier notices, test reports, old scanned work orders. A document that was never read is invisible to search, and to every assistant built on search.

What comes out has to suit three readers. A search index wants plain text. A language model reads better when headings, lists and tables survive, so many systems keep a Markdown copy beside the text. Fields go a third way, into a system of record, a workflow or a graph.

In a graph, an extracted part number connects the document to the part, the supplier and the order it is about. Matching that number to the right part record is entity resolution, which What is entity resolution? explains.

• Silent character errors. A 0 read as an O, a 1 as an l. A part number with one wrong character still looks valid. It finds nothing, or the wrong part.
• Lost structure. Tables flattened into lines, two columns merged, a header row dropped. The text is all there and means something else.
• A confident wrong value. A model asked for fields in JSON returns well formed JSON either way. Nothing in the shape of the answer says whether a value is right, or whether the document had it at all.
• Drift. A supplier changes its form, a scanner is replaced, a new document type arrives. A setup that worked in the pilot quietly gets worse.

No extraction is right on every page, so the question is which values a person sees. Microsoft’s guidance on confidence scores says a confidence “can be used to determine whether to automatically accept the prediction or flag it for human review.” It also notes that the confidence of the text read and the confidence of the field are separate, and that both need checking.

Set the bar per field, by what an error costs. A wrong date on a work order is an annoyance. A wrong torque value is a safety issue. Route everything under the bar to a person who can see the page beside the value. Their corrections become the test set for the next change.

Test it on your own documents, the worst ones included: faxes, phone photos, stamped pages, scans turned on their side. Six checks cover most of it.

• Measure per field, against values a person checked: how often each field is right, wrong or missing.
• Check structure, not only text. Do tables keep their rows and columns, and do headings stay headings?
• Search for exact identifiers in what was read. A part number with one wrong character should show up in this test, not in production.
• Look at the review queue: what share goes to a person, and whether they see the page beside the value.
• Ask where each step runs, and which steps send a page to a hosted service.
• Ask how a value traces back. A user should be able to open the page a value came from.

In Curiosity Studio every file a connector brings in goes through a built in extractor. Text PDFs, Office files, HTML and email are parsed directly. Scans, photos and PDFs that hold only page images go through OCR, archives are unpacked, and audio and video are transcribed. The extraction documentation describes it. Extraction runs once, when a file is first indexed, and its result is stored with the file.

One extraction produces the plain text the indexes read and the same document as Markdown, with the same page numbers in both. Our post on X-Ray, the C# libraries that read the files, explains how. OCR runs inside Studio, on your infrastructure.

The OCR documentation describes how a page is read. NanoOCR, whose models are built in, reads every scanned page first and decides what kind of page it is. Printed text keeps its reading. Photos go to Florence2, and tables, forms and handwriting go to PaddleOCR-VL. Those two are optional downloads, and until they are installed, NanoOCR’s reading stands. Every engine writes Markdown as well as text, so a scanned table arrives as a table.

Field capture is a choice of method, and Studio offers several. Spotters match a curated list of names and their aliases, such as part numbers, suppliers or sites. Pattern spotters match identifiers by their shape, such as ticket or order numbers. Both are set up in the admin pages, run inside Studio and give the same answer every time.

For a field no list or pattern can catch, code you write asks a language model to read it out. That is the model you configured, hosted or on your own servers. Structured decisions answer typed questions with a confidence, on a local model or a hosted provider. Code can act on a clear answer and send an unclear one to a person.

The newest option is a class of specialized models built for one job: typed answers with probabilities, not free text. TypeSafe started it in September 2026 with Jev, “a new class of frontier models built to make fast, structured decisions that software can use directly.” Cloudflare’s Clairic followed a month later as an extraction model of that kind. Structured extraction can use it, and returns each value with the place in the text it came from, or null. Clairic is hosted and in preview, so its input goes to Cloudflare.

What is read lands in the same graph as the rest of your records. There, entity extraction links it to the parts and suppliers it names.