Document classification built on a real data audit

Context

The underwriting team of a commercial lender used several hours on document review for each application. The documents included financial statements, contracts, supplier agreements, and regulatory filings. Some were scanned PDFs of different quality, and some were native digital files. They were in many languages, and some had legacy formats. Earlier vendor proposals promised end-to-end automation, but they did not address the inconsistent source data.

Approach

Before we built anything, we did a six-week readiness assessment. The audit examined document quality, format consistency, classification taxonomy, and how documents moved through the team. The result was clear. Approximately 40% of new documents needed pre-processing or normalisation to make a classifier operate reliably.

We built the data preparation layer first. This layer had an optical character recognition (OCR) pipeline that we tuned for the actual document mix. It also had normalisation for the many less common formats, and a small human review step for ambiguous cases. After that, we built the classification model and trained it on the clean input.

Risk and compliance teams examined the audit findings before the model work started. As a result, the project was explainable from the start.

Outcome

• 4× throughput on document review for each analyst.
• Underwriters now use their time on cases that need judgement, such as covenant interpretation and risk assessment. They no longer use it on triage.
• The data foundation is reusable. The same pre-processing layer now supplies two more AI workflows in adjacent operations of the lender.

Principle

Readiness. Without the data audit, the result is a fragile classifier and a project that fails on edge cases. The audit made the model viable in production. It also made the model explainable to risk and compliance teams. A regulated workflow must have both of these before it can use AI.

We show this case study with permission from the client, as an example only. Specific commercial details stay confidential. If you are a qualified counterparty, you can ask for these details under a non-disclosure agreement (NDA).

← Back to Orchestron. · More case studies · Discuss an opportunity