VinUniversity
College of Business
and Management
MBA Capstone Project 2026 · Final Report

An AI Orchestration Framework for Automating Document-Intensive Banking Workflows with Human-in-the-Loop Governance

The Case of Credit Appraisal in Vietnamese Commercial Banks

Presented by
LTLe Quoc Trung
TPNguyen Thi Thu Phuong
DLDang Dam Linh
VHNguyen Viet Hung
Advisors
MLDr. Nguyen Thi Mai Lan
NTDr. Le Nhan Tam
Ha Noi, October 2026
Credit Orchestrator governed pipeline
01Read inside the bank

An in-house model extracts every field from the credit pack.

02Mask before it leaves

Only masked or public data crosses the boundary, and every call is logged.

03A named person decides

Every field is reviewed at the gate; approval stays with the officer.

Group 5

Advisors and team

ML
Capstone Advisor

Dr. Nguyen Thi Mai Lan

Assistant Professor of Finance
College of Business and Management, VinUniversity
NT
Advisor

Dr. Le Nhan Tam

Former CTO
Microsoft and IBM Vietnam
LT

Le Quoc Trung

AI Engineer | Techcombank
TP

Nguyen Thi Thu Phuong

Executive Assistant | Sheraton
DL

Dang Dam Linh

Business Analyst | BIDV
VH

Nguyen Viet Hung

Head of AI Product | Vinsmart Future
Contents
01
VinUniversityMBA Capstone 2026 · Group 5
Part 01 of 06

The problem

Context, problem statement, challenges and the gap

LT
Presented byLe Quoc TrungAI Engineer, Techcombank
01 Industry background

Vietnamese banks have digitised the front office; the credit middle office remains manual

A typical SME credit pack
Synthetic business registration certificate
Business registration certificate, as an appraiser receives it (synthetic example)
Financial statements, Circular 200Business registrationBank statements Collateral valuationTax filingsContracts

Scans, native files and paper. Each pack is read, reconciled and summarised, and the decision must stay traceable for audit.

01
Why now

Foundation models read unseen layouts

Removes the template dependence that limited OCR and RPA.

02
Why now

Open-weight models run in-house

Automation becomes feasible where data cannot leave the bank.

03
Why now

Regulation tightened

Where data is processed is now an architectural question.

The legal frame tightened, 2020–2026
2020SBV Circular 09/2020Information-system safety
2023Decree 13/2023First personal-data decree
2024Law 32/2024 on Credit InstitutionsCustomer confidentiality
2025PDPL 91/2025 + Decree 356/2025Data protection becomes statute
1 Jan 2026PDPL in forceConsent, purpose, cross-border rules
01 Problem identification

The binding constraint is architectural, not model capability

Vietnamese banks are not capturing AI productivity in document-heavy middle-office work, because no deployable architecture yet reconciles model capability with data protection, audit evidence and human accountability.

The proposition, in one line

A governed orchestration layer that decides what the AI may do, where it may run, what evidence it must leave, and when a human takes over.

Four observations
1

Awareness is high

Banks know about AI and want it.

2

A bigger model does not help

A model the bank cannot legally send data to adds nothing.

3

Point solutions do not pay back

One workflow cannot amortise the governance cost.

4

The problem recurs

Every new workflow reopens data, model, oversight and audit questions.

01 Four challenges, four theories

Four challenges, each addressed by a distinct component and grounded in theory

Challenge 1 · Organisation

Manual processing burden

Analysts spend their judgement on data assembly. Longer cycle times, more rework, and headcount that grows linearly with volume.

Challenge 2 · Environment

Data cannot go to external models

PDPL and banking secrecy make unrestricted frontier-API designs unimplementable. Fully on-premise designs accept a lower capability ceiling.

Challenge 3 · Technology

External intelligence unused

Sector news, litigation and registry changes are public, but nobody has a mechanism to gather, verify and attribute them quickly.

Challenge 4 · Technology

No way to pick a model tier

Banks swing between over-specifying (uneconomic or non-compliant) and under-specifying (a failed pilot).

TOE · Tornatzky & Fleischer, 1990

Adoption stalls on the environment

So the compliance guardrail is a first-class layer.

Task-Technology Fit · Goodhue & Thompson, 1995

Fit beats raw capability

Pick tiers per task and evaluate per task.

Levels of automation · Parasuraman et al., 2000

Automate acquisition and analysis

People keep decision selection.

Dynamic capabilities · Teece, 2007

Build a framework, not an app

Each new workflow is cheaper than the last.

01 Existing solutions

Document extraction is commoditised; the governance layer above it is not

RequirementExisting solutionsWhat this project adds
Extract structured data from mixed documentsYes international and domestic providersNothing. Adopted, not reinvented
Reason across documentsPartly rule-based aggregationSynthesis under inconsistency; reconciling conflicting sources
Data residency as an architectural propertyOption a deployment settingAn egress policy layer with per-event audit evidence; sovereign or masked-hybrid by design
Human authority over the decisionRarely an optional review stepA gate that cannot be bypassed, with supervisory evidence and an amendment measure
Pick a model tier for the taskNo vendor-chosen, opaqueA repeatable method for the minimum sufficient tier per task
Reuse across workflowsPartly templates tied to doc typesA harness with persistent memory and a reusable skill library
02
VinUniversityMBA Capstone 2026 · Group 5
Part 02 of 06

Our approach

Research question, framework, strategies and evaluation

LT
Presented byLe Quoc TrungAI Engineer, Techcombank
02 Research question, objectives, method

What deployable architecture allows a bank to use modern AI in a credit workflow while meeting data-protection obligations, producing audit evidence and retaining human accountability, and what minimum model capability does it require?

O1

Quantify the manual burden

Measured baseline from interviews and file sampling

Measured by: cycle-time baseline

O2

Specify the framework

Capabilities, harness, compliance guardrails

Measured by: boundary integrity (target 0)

O3

Build and evaluate a prototype

One end-to-end workflow: accuracy, cycle time, amendment rate

Measured by: field accuracy, cycle time, amendment rate

O4

Model-tiering method

Minimum sufficient model per task, then a roadmap

Measured by: capability–cost frontier

Design Science Research · Hevner et al., 2004

Problem characterisation → Artefact design → Build and demonstrate → Evaluate

Five measures

Field accuracy · Cycle-time reduction · Human amendment rate · Boundary integrity (target 0) · Capability–cost frontier

02 The artefact

A three-layer framework enforces the institutional data boundary

L3

Compliance guardrail

De-identification with a typed token vault inside the perimeter · re-identification · egress scanner that blocks on any hit · immutable call log with payload hashes · blocking human review gate · per-task routing policy

L2

Orchestration harness

Case store · parallel fan-out across documents · deterministic number and unit normaliser · reconciliation and credit-ratio skill (current ratio, D/E, interest cover, net margin, LTV, loan-to-revenue + 5 cross-document checks) · benchmark harness

L1

Task capabilities

Extraction against Circular 200 row codes · vision transcription in the perimeter · public-only sector research · credit-memo drafting that ends in a recommendation, never a decision

Design rule (iteration 3)

Code handles facts, policy and numbers.
The model reads and writes.
A named person decides.

Built on
Cloudflare Pages + FunctionsWorkers AID1Google sign-in381 automated tests
02 Strategies

Two deployment strategies, with task-level routing as the working hypothesis

Approach 1

Sovereign on-premise

All inference on bank infrastructure with open-weight models. Compliant by construction.

ComplianceLow risk
AccuracyAdequate; limited on synthesis
CostCapital-heavy, good at volume
LatencyDepends on capacity
ExtensibilityHigh once built
ReadinessNeeds in-house model ops
Approach 2

Hybrid gateway

External frontier models behind de-identification. Only masked content crosses; re-identified locally.

ComplianceModerate residual risk
AccuracyHigher on reconciliation
CostVariable, good at pilot scale
LatencyExternal + gateway overhead
ExtensibilityHigh; low marginal cost
ReadinessNeeds vendor governance
Implemented routing policies
PolicyExtractResearchMemo
SovereignS3offS3
HybridF1 maskedF1 publicF1 masked
Task-routedS2 insideF1 publicF1 fact sheet

Hypothesis: extraction (moderate capability, maximum sensitivity) stays inside; research and synthesis (higher capability, maskable or public) may go out. An all-or-nothing choice becomes a per-task routing decision.

02 Evaluation setting

A controlled experiment on a synthetic corpus with engineered heterogeneity

15
synthetic SME credit files, in Vietnamese
6
documents per file: registration, B01-DN, B02-DN, bank statement, collateral valuation, loan application
375
scored ground-truth fields (25 per file)
Five model tiers, all on Cloudflare Workers AI
TierModelRole$/M in / out
S1Llama 3.2 3BSovereign small0.051 / 0.335
S2Llama 3.1 8B FP8Sovereign medium0.152 / 0.287
S3Gemma 4 26B-A4BSovereign large (MoE)0.100 / 0.300
S4gpt-oss-120bSovereign XL (MoE)0.350 / 0.750
F1Llama 3.3 70B FP8External frontier stand-in, masked only0.293 / 2.253

List prices read 11 Oct 2026. Every call is logged with tier, route, data class, payload size, masked count, scanner hits, decision, latency, cost and payload hash.

3 layout styles

A table, VND, Vietnamese separators
B bilingual table, US separators
C narrative, in millions of VND

3 scan-noise levels

0%, 4% and 10% diacritic loss on letters (digits untouched)

11 red flags in 9 files

4 revenue mismatches · 2 charter-capital mismatches · 2 weak bank inflows · 3 LTV breaches

03
VinUniversityMBA Capstone 2026 · Group 5
Part 03 of 06

Seven findings

Model tiers, determinism, red flags, masking, cost and business depth

VH
Presented byNguyen Viet HungHead of AI Product, Vinsmart Future
F1 Finding 1

An 8B in-perimeter model is sufficient for extraction at 99.5% accuracy

Field accuracy, iteration 2 (375 fields per tier)
dashed = 99% threshold
S1
76.8%
S2
99.5%
S3
97.3%
S4
84.8%
F1
100%
TierCorrectMedian s / fileUSD / fileFailed docs
S12881.50.00040
S237310.50.00060
S336543.20.00330
S43188.20.00210
F13754.90.00240
2
fields of 375 separate S2 from the external frontier tier, at about ¼ of the cost
Non-monotonic frontier

The 120B model S4 loses to S2 on both accuracy and cost: it converted units itself despite instructions, inflating narrative amounts a thousandfold.

S1 is unreliable on numeric tables (59% on balance sheets) and returned no bank-statement fields.

F2 F3 Findings 2 and 3

Deterministic unit handling raised accuracy; layout, not scan noise, drives residual error

Accuracy before → after the deterministic unit skill
Axis 75% to 100%
Iteration 1 · model converts unitsIteration 2 · code converts units

Iteration 1's dominant error was mis-scaling ×1,000 on "Đơn vị tính: triệu đồng". S3 dipped because it returned nothing for one narrative balance sheet; S4 ignored the new instruction.

Design rule: if a transformation can be specified exactly, the harness should do it. Fewer errors, auditable, and a lower minimum tier.

Accuracy by layout style, iteration 2
100%
S2 on clean files
98.7%
S2 on the noisiest files

Residual error sits in style C (figures in running text, in millions). Scan noise has no consistent effect, so validate per document family, not on a generic benchmark.

F4 Finding 4

Red-flag recall was complete, but precision on the recommended tier was 50%

Recall and precision by extraction tier · 11 planted flags
RecallPrecision
Four deterministic checks
  • Declared revenue vs audited income statement
  • Charter capital: application vs registration
  • Bank inflows far below monthly revenue
  • Loan-to-value above 100%
Root cause of S2's 11 false alarms

S2 put owners' equity "vốn chủ sở hữu" into the charter-capital field "vốn điều lệ".

A semantic disambiguation error, not a reading error.

A false alarm costs seconds at the gate. A missed flag can cost a credit loss.

F5 Finding 5

The boundary held on the corpus, but de-identification is the weakest control

0 / 90
external calls carried a ground-truth identifier. No call needed blocking. Pass on the corpus.
Checked identifiers

Company names, tax codes, legal reps, national IDs, phones, addresses, bank accounts

Adversarial red-team: de-identification recall
Tuning · rules v1
55%
Tuning · rules v2
95%
Tuning · rules + NER
100%
Held-out · rules
23%
Held-out · rules + NER
73%

The held-out set of 20 cases was written before v2 was evaluated, to guard against overfitting. Hand-written rules overfit; a learned detector is needed, and even that is not enough to be the only control.

Missed on held-out (6 of 22)
ID number with dashes Hanoi landline Abbreviated company name Vehicle plate Shareholder list ×2

NER also over-masked the lending bank's own name, which degrades what the external model sees.

Prototype Inside vs outside the perimeter

Only tokenised content crosses the institutional perimeter

Masking view: original document inside the bank next to the masked payload

Synthetic file SME-009: 24 identifier occurrences masked, 17 entities in the vault, 7 entity classes, 0 of 2 external calls carrying an identifier. Click to enlarge.

F6 Finding 6

Inference cost is immaterial; cycle time is governed by human review

<1¢
automated path per credit file
$0.0006
extraction on S2 per file. Research and memo each under $0.001
13–17s
automated steps per file
~15s
credit memo draft
So the business case rests on

Analyst time recovered, plus governance and integration cost. Neither can be measured without a manual baseline and a reviewer study.

How the net effect will be measured

A within-subject reviewer study: matched files done manually and with the prototype, in counterbalanced order, reported as medians with IQR. The gate already logs review time and every decision.

F7 Finding 7 · iteration 3

Structural controls, not prompting, make the model usable in appraisal

Five components added
  • Business spec: process and roles, 12-family pack, Circular 200/133 forms, policy catalogue, credit-need method, 10-section report
  • 18 synthetic packs with 22 planted business issues
  • Credit skills: 3-year ratios, 10 early warnings, revenue triangulation, CIC summary, Decree 80/2021 sizing, policy engine, offer with binding constraint
  • Ten-section report where code renders every number
  • Roles, append-only overrides, four-eyes approval
Live run · pack SME-100 · 12 Oct 2026
0.4s
policy analysis, no model call
52s
ten-section memo
10/10
untraceable numbers flagged

Held at PENDING on sanctions, internal rating and owner screening until an officer added them, then ELIGIBLE. The model wrote 10 invented figures despite instructions; confirmation was blocked until they were acknowledged.

Policy view with entry-gate rules
Appraisal memo with version stamp and unsupported-number count
04
VinUniversityMBA Capstone 2026 · Group 5
Part 04 of 06

Results

Accuracy, acceptability, thresholds, the product and outcomes

TP
Presented byNguyen Thi Thu PhuongExecutive Assistant, Sheraton
04 Accuracy achieved

No stage is fully accurate; every error type is caught by a control

Stage and measureResultIn absolute termsControl that catches it
Extraction, S2 (recommended)99.5%2 wrong of 375Review gate shows each field with its source; reconciliation
Extraction, F1 (external, masked)100%0 of 375As above; external only through masking
Extraction, S1 / S3 / S476.8 / 97.3 / 84.8%87 / 10 / 57 wrongNot recommended
Red-flag recall, S2100%0 of 11 missedAnalyst disposes of every flag
Red-flag precision, S250%11 false alarms / 15 filesAnalyst dismisses at the gate
De-identification, held-out73%6 of 22 missedEgress scanner; raw docs stay inside
Memo figures (iter. 3, live)10 untraceable10 of 10 flaggedConfirmation blocked; four-eyes approval
Key figures, full packs (iter. 3, S2, 17 packs)79.5%median pack 94%; 3 packs ×1,000,000Review gate; magnitude check to add (R5)
Business red flags, full packs93.1% / 69.2%2 of 29 missed; 12 falseAnalyst reviews the whole pack
Entry-gate verdict, full packs94.1%1 of 17 eligible shown pendingOfficer completes pending items
Unknown facts invented / pending shown as passed0 / 0of 126 / of 16Policy engine: unknown → PENDING
04 Acceptability

Under maker–checker, the model acts as maker and named staff remain checkers

The question is not whether the model is perfect, but whether each error type is (a) rare enough that checking beats re-keying and (b) visible to the checker.

Acceptable with review

Field extraction

At 99.5% the analyst confirms rather than re-types. On full packs (79.5%), checking every decision-critical figure is mandatory. Accuracy here measures review workload, not appraisal accuracy.

Pilot thresholdDecision-critical fields ≥ 99% per family, 100% verified at the gate
Acceptable one way

Red flags

Recall must be complete, and it was (11 of 11). 50% precision is tolerable short-term but would erode trust in production, so it is the first development target.

Pilot threshold100% recall on a regression suite, ≥ 70% precision
Not as the only control

De-identification

An identifier that leaves the bank cannot be recalled. At 73% held-out recall, raw documents stay inside; only structured, harness-generated fact sheets go out.

Pilot threshold≥ 99% held-out recall, only if raw text leaves
04 Proposed thresholds for a bank pilot

Proposed acceptance thresholds for a bank pilot

Control pointProposed thresholdStatus on synthetic corpus (S2)
Decision-critical fields
revenue, assets, equity, debt, inflows, collateral, amount
≥ 99% per document family on the bank's own set, plus 100% human verificationIter. 2: 4 of 6 families Iter. 3: 79.5%
Descriptive fields
names, addresses, dates
≥ 97%, verified at the gateMet
Red-flag recall / precision100% recall on a regression suite / ≥ 70% precisionIter. 2: recall met, precision 50% Iter. 3: 93.1% / 69.2%
De-identification
only if raw text leaves
≥ 99% recall on a held-out suite refreshed by complianceNot met (73%) raw text kept inside
Figures in drafted reports100% traceable or acknowledged before confirmationMet by structural check

The group's proposal for calibration with a partner bank's risk and compliance functions, not an observed industry standard.

Prototype The governed workflow

The governed workflow halts at a mandatory human review gate

Governed pipeline
Human review gate
Credit memo

1 · Pipeline

Intake, masking, extraction, reconciliation and research run under the task-routed policy, each step with tier, time and cost.

2 · Review gate

Every field shown with its source and confidence. Accept, amend or reject before a memo can be drafted. Time and decisions logged.

3 · Memo and decision

Drafted from a masked fact sheet. Approve, approve with conditions, decline or refer stays with the credit officer.

04 Outcomes

Three of four objectives were achieved; the manual baseline remains outstanding

Objectives
O1Quantify the manual burdenNot met
No partner access. Interview protocol, sampling frame and baseline record prepared and placed first in the roadmap.
O2Specify the frameworkAchieved
O3Build and evaluate a prototypeSubstantially
Amendment rate and full cycle time are instrumented, waiting for real reviewers.
O4Model-tiering methodOn synthetic data
Four deliverables
  • A deployed working prototype
  • A reusable synthetic corpus with ground truth
  • A harness that reproduces every figure
  • A measured model-tiering recommendation
Success criteria
Quantified baselineNot met
End-to-end prototype, 15 files × 5 tiers, 0 failedMet
Zero boundary violationsMet on corpus
Frontier supports a tier choiceMet
05
VinUniversityMBA Capstone 2026 · Group 5
Part 05 of 06

Implementation

Roadmap, people and compliance clearance

DL
Presented byDang Dam LinhBusiness Analyst, BIDV
05 Implementation roadmap

A four-phase, gated roadmap from prototype to production over 24 months

Months 0–3

0 · Readiness

Sponsor and steering committee; interviews and manual baseline; DPIA; anonymised validation set per family; on-prem hosting of the open-weight model

ExitBaseline measured; compliance accepts routing and residual risk; validation set ready
Months 3–6

1 · Shadow pilot

One branch or SME desk, 3–5 analysts. Runs alongside the manual process on live files; output not used for decisions

ExitThresholds met on the bank's own documents; amendment rate and review time measured
Months 6–12

2 · Controlled production

Replaces manual data assembly for SME working-capital loans at pilot units; four-eyes unchanged; monthly control testing

ExitCycle-time reduction vs baseline; zero boundary violations; internal audit passed
Months 12–24

3 · Scale

More regions and adjacent workflows (renewals, trade finance, collateral revaluation) reusing harness skills; periodic re-tiering

ExitEach new workflow passes its own validation set
Technology workstream

Hosting. 8B-class model on bank GPUs or domestic private cloud; re-verify accuracy on that hardware

Integration. LOS, document management, CIC feed, core banking; write the stamped report back to the LOS

Evaluation as a control. Every model, prompt or format change passes the validation set first

Backlog. Disambiguation, learned de-ID, OCR, run variance, done in Phases 0–1

05 People and coordination

Most implementation effort lies in organisational change, not technology

Steering committee

Chaired by the CRO, monthly in Phases 0–2, owns every go / no-go.

Role redesign

Appraisers review a machine-assembled file and own the judgement sections. Saved time goes to analysis, not higher file counts.

Training

Known error types (unit scaling, adjacent terms, invented figures), the review gate, version stamps and override logs.

New capabilities

AI product owner from credit, 2–3 engineers, and an independent model-risk validator in risk.

Change management

Analysts help set thresholds. The signing officer, not the system, owns the appraisal.

RACI across departments
R responsible A accountable · C consulted · I informed
ActivityCreditRMsIT/SecCompl.RiskAuditHR
Baseline and requirementsARCCIII
Model hosting and integrationCIA/RCCI–
Routing policy and de-IDC–RACI–
Validation and thresholdsA–CCRI–
Training and role redesignACC–––R
Phase-gate decisionRCCCCII
Ongoing control testingC–RCAR–

A steering committee chaired by the CRO meets monthly in Phases 0–2 and is accountable for every go / no-go.

05 Compliance clearance

Compliance clearance is structured as five signable deliverables

Regulator and law

State Bank of Vietnam supervision, PDPL 91/2025, Law 32/2024 and Circular 09/2020 decide what the architecture may do.

1

Personal data protection

DPIA under PDPL 91/2025/QH15 and Decree 356/2025: which data, where, on what legal basis. Simpler because raw documents never leave.

2

Confidentiality and infosec

Routing consistent with Law 32/2024/QH15; system classified under Circular 09/2020/TT-NHNN with its required controls.

3

Third-party risk

If an external model is used: residency, no retention, no training, audit rights, plus a formally accepted residual de-ID risk statement.

4

Internal credit regulation

Machine data confirmed by a named officer; untraceable figures block confirmation; approval stays with the approver.

5

Audit evidence

Call log, payload hashes, override log and version stamps mapped to audit tests before Phase 2.

06
VinUniversityMBA Capstone 2026 · Group 5
Part 06 of 06

Conclusion

Evaluation, recommendations and key insights

DL
Presented byDang Dam LinhBusiness Analyst, BIDV
06 Evaluation of the seven findings

Confidence in each finding, and the limits common to all

FindingServesWhat it meansConfidence
1 Medium sovereign model sufficesO4Task-level tiering; the largest model is not requiredModerate synthetic, one run
2 Code beats model on unitsO2Generalisable design rule for the harnessHigh large effect, clear mechanism
3 Layout, not noise, drives errorO4Validate per document family, not per modelModerate simulated noise
4 Full recall, imperfect precisionO3Only acceptable with a human gateModerate 11 flags
5 Boundary held; masking weakestO2 O3Raw documents must stay insideHigh for the weakness
6 Model cost is immaterialO3 O4The case rests on analyst time and governanceHigh for cost; time unmeasured
7 Human role moves to judgementO2 O3Controls must be structural, not prompt-basedModerate 17 packs, one run
Limits common to all findings

Small synthetic corpus: S2 vs F1 is two fields

One run per configuration; outputs vary even at temperature zero

One provider's infrastructure: "sovereign" is a role, not a location

No interviews or baseline: no claim about time savings

06 Recommendations

Ten recommendations for model development and implementation

R1

Fix field disambiguation

Row-code anchors separate charter capital from equity; cross-check the registration certificate. Precision 50% → 70%.

R2

Learned de-identification

In-perimeter NER on Vietnamese banking text, a bank allow-list, and a red-team suite owned by compliance.

R3

OCR and real documents

In-perimeter OCR and vision; re-run tiering on the partner bank's anonymised documents per family.

R4

Measure variance, re-tier

At least 3 runs per configuration; re-run the frontier on every model, prompt or format change.

R5

Magnitude checks

Reject amounts inconsistent with size, VAT and inflows. Would have caught all 3 mis-scaled packs.

R6

Shadow mode, pre-agreed thresholds

Phase gates become mechanical, not negotiated.

R7

Harness = compliance evidence

One body of evidence for the DPIA, model-risk validation and audit.

R8

Reusable skill library

Units, row codes, ratios, policy rules and credit-need method as shared assets.

R9

Integrate rather than replace

Connect to LOS, DMS and core banking; keep the approval workflow.

R10

One cross-functional committee

CRO-chaired, so the workstreams move together.

06 Conclusion

Banks can adopt AI in credit appraisal within their obligations, provided people remain the checkers

Insight 1
1

Capability is not the constraint

An 8B sovereign model matched the frontier stand-in on extraction.

Insight 2
2

Determinism is a design asset

Whatever can be specified exactly belongs in the harness.

Insight 3
3

Statistical controls are the weak point

Measure de-identification; keep raw documents inside.

Insight 4
4

Do not trust models with figures

Even when told not to write them. Controls must be structural.

Still unproven: the size of the time saving in a real bank. Phase 0 of the roadmap is designed to measure it at the first gate.

Credit Orchestrator

Thank you

Questions and discussion

Public overview page

All slides