The Case of Credit Appraisal in Vietnamese Commercial Banks

An in-house model extracts every field from the credit pack.
Only masked or public data crosses the boundary, and every call is logged.
Every field is reviewed at the gate; approval stays with the officer.


MBA Capstone 2026 · Group 5Context, problem statement, challenges and the gap

Scans, native files and paper. Each pack is read, reconciled and summarised, and the decision must stay traceable for audit.
Removes the template dependence that limited OCR and RPA.
Automation becomes feasible where data cannot leave the bank.
Where data is processed is now an architectural question.
Vietnamese banks are not capturing AI productivity in document-heavy middle-office work, because no deployable architecture yet reconciles model capability with data protection, audit evidence and human accountability.
A governed orchestration layer that decides what the AI may do, where it may run, what evidence it must leave, and when a human takes over.
Banks know about AI and want it.
A model the bank cannot legally send data to adds nothing.
One workflow cannot amortise the governance cost.
Every new workflow reopens data, model, oversight and audit questions.
Analysts spend their judgement on data assembly. Longer cycle times, more rework, and headcount that grows linearly with volume.
PDPL and banking secrecy make unrestricted frontier-API designs unimplementable. Fully on-premise designs accept a lower capability ceiling.
Sector news, litigation and registry changes are public, but nobody has a mechanism to gather, verify and attribute them quickly.
Banks swing between over-specifying (uneconomic or non-compliant) and under-specifying (a failed pilot).
So the compliance guardrail is a first-class layer.
Pick tiers per task and evaluate per task.
People keep decision selection.
Each new workflow is cheaper than the last.
| Requirement | Existing solutions | What this project adds |
|---|---|---|
| Extract structured data from mixed documents | Yes international and domestic providers | Nothing. Adopted, not reinvented |
| Reason across documents | Partly rule-based aggregation | Synthesis under inconsistency; reconciling conflicting sources |
| Data residency as an architectural property | Option a deployment setting | An egress policy layer with per-event audit evidence; sovereign or masked-hybrid by design |
| Human authority over the decision | Rarely an optional review step | A gate that cannot be bypassed, with supervisory evidence and an amendment measure |
| Pick a model tier for the task | No vendor-chosen, opaque | A repeatable method for the minimum sufficient tier per task |
| Reuse across workflows | Partly templates tied to doc types | A harness with persistent memory and a reusable skill library |
MBA Capstone 2026 · Group 5Research question, framework, strategies and evaluation
Measured baseline from interviews and file sampling
Measured by: cycle-time baseline
Capabilities, harness, compliance guardrails
Measured by: boundary integrity (target 0)
One end-to-end workflow: accuracy, cycle time, amendment rate
Measured by: field accuracy, cycle time, amendment rate
Minimum sufficient model per task, then a roadmap
Measured by: capability–cost frontier
Problem characterisation → Artefact design → Build and demonstrate → Evaluate
Field accuracy · Cycle-time reduction · Human amendment rate · Boundary integrity (target 0) · Capability–cost frontier
De-identification with a typed token vault inside the perimeter · re-identification · egress scanner that blocks on any hit · immutable call log with payload hashes · blocking human review gate · per-task routing policy
Case store · parallel fan-out across documents · deterministic number and unit normaliser · reconciliation and credit-ratio skill (current ratio, D/E, interest cover, net margin, LTV, loan-to-revenue + 5 cross-document checks) · benchmark harness
Extraction against Circular 200 row codes · vision transcription in the perimeter · public-only sector research · credit-memo drafting that ends in a recommendation, never a decision
Code handles facts, policy and numbers.
The model reads and writes.
A named person decides.
All inference on bank infrastructure with open-weight models. Compliant by construction.
| Compliance | Low risk |
| Accuracy | Adequate; limited on synthesis |
| Cost | Capital-heavy, good at volume |
| Latency | Depends on capacity |
| Extensibility | High once built |
| Readiness | Needs in-house model ops |
External frontier models behind de-identification. Only masked content crosses; re-identified locally.
| Compliance | Moderate residual risk |
| Accuracy | Higher on reconciliation |
| Cost | Variable, good at pilot scale |
| Latency | External + gateway overhead |
| Extensibility | High; low marginal cost |
| Readiness | Needs vendor governance |
| Policy | Extract | Research | Memo |
|---|---|---|---|
| Sovereign | S3 | off | S3 |
| Hybrid | F1 masked | F1 public | F1 masked |
| Task-routed | S2 inside | F1 public | F1 fact sheet |
Hypothesis: extraction (moderate capability, maximum sensitivity) stays inside; research and synthesis (higher capability, maskable or public) may go out. An all-or-nothing choice becomes a per-task routing decision.
| Tier | Model | Role | $/M in / out |
|---|---|---|---|
| S1 | Llama 3.2 3B | Sovereign small | 0.051 / 0.335 |
| S2 | Llama 3.1 8B FP8 | Sovereign medium | 0.152 / 0.287 |
| S3 | Gemma 4 26B-A4B | Sovereign large (MoE) | 0.100 / 0.300 |
| S4 | gpt-oss-120b | Sovereign XL (MoE) | 0.350 / 0.750 |
| F1 | Llama 3.3 70B FP8 | External frontier stand-in, masked only | 0.293 / 2.253 |
List prices read 11 Oct 2026. Every call is logged with tier, route, data class, payload size, masked count, scanner hits, decision, latency, cost and payload hash.
A table, VND, Vietnamese separators
B bilingual table, US separators
C narrative, in millions of VND
0%, 4% and 10% diacritic loss on letters (digits untouched)
4 revenue mismatches · 2 charter-capital mismatches · 2 weak bank inflows · 3 LTV breaches
MBA Capstone 2026 · Group 5Model tiers, determinism, red flags, masking, cost and business depth
| Tier | Correct | Median s / file | USD / file | Failed docs |
|---|---|---|---|---|
| S1 | 288 | 1.5 | 0.0004 | 0 |
| S2 | 373 | 10.5 | 0.0006 | 0 |
| S3 | 365 | 43.2 | 0.0033 | 0 |
| S4 | 318 | 8.2 | 0.0021 | 0 |
| F1 | 375 | 4.9 | 0.0024 | 0 |
The 120B model S4 loses to S2 on both accuracy and cost: it converted units itself despite instructions, inflating narrative amounts a thousandfold.
S1 is unreliable on numeric tables (59% on balance sheets) and returned no bank-statement fields.
Iteration 1's dominant error was mis-scaling ×1,000 on "Đơn vị tính: triệu đồng". S3 dipped because it returned nothing for one narrative balance sheet; S4 ignored the new instruction.
Design rule: if a transformation can be specified exactly, the harness should do it. Fewer errors, auditable, and a lower minimum tier.
Residual error sits in style C (figures in running text, in millions). Scan noise has no consistent effect, so validate per document family, not on a generic benchmark.
S2 put owners' equity "vốn chủ sở hữu" into the charter-capital field "vốn điều lệ".
A semantic disambiguation error, not a reading error.
A false alarm costs seconds at the gate. A missed flag can cost a credit loss.
Company names, tax codes, legal reps, national IDs, phones, addresses, bank accounts
The held-out set of 20 cases was written before v2 was evaluated, to guard against overfitting. Hand-written rules overfit; a learned detector is needed, and even that is not enough to be the only control.
NER also over-masked the lending bank's own name, which degrades what the external model sees.

Synthetic file SME-009: 24 identifier occurrences masked, 17 entities in the vault, 7 entity classes, 0 of 2 external calls carrying an identifier. Click to enlarge.
Analyst time recovered, plus governance and integration cost. Neither can be measured without a manual baseline and a reviewer study.
A within-subject reviewer study: matched files done manually and with the prototype, in counterbalanced order, reported as medians with IQR. The gate already logs review time and every decision.
Held at PENDING on sanctions, internal rating and owner screening until an officer added them, then ELIGIBLE. The model wrote 10 invented figures despite instructions; confirmation was blocked until they were acknowledged.


MBA Capstone 2026 · Group 5Accuracy, acceptability, thresholds, the product and outcomes
| Stage and measure | Result | In absolute terms | Control that catches it |
|---|---|---|---|
| Extraction, S2 (recommended) | 99.5% | 2 wrong of 375 | Review gate shows each field with its source; reconciliation |
| Extraction, F1 (external, masked) | 100% | 0 of 375 | As above; external only through masking |
| Extraction, S1 / S3 / S4 | 76.8 / 97.3 / 84.8% | 87 / 10 / 57 wrong | Not recommended |
| Red-flag recall, S2 | 100% | 0 of 11 missed | Analyst disposes of every flag |
| Red-flag precision, S2 | 50% | 11 false alarms / 15 files | Analyst dismisses at the gate |
| De-identification, held-out | 73% | 6 of 22 missed | Egress scanner; raw docs stay inside |
| Memo figures (iter. 3, live) | 10 untraceable | 10 of 10 flagged | Confirmation blocked; four-eyes approval |
| Key figures, full packs (iter. 3, S2, 17 packs) | 79.5% | median pack 94%; 3 packs ×1,000,000 | Review gate; magnitude check to add (R5) |
| Business red flags, full packs | 93.1% / 69.2% | 2 of 29 missed; 12 false | Analyst reviews the whole pack |
| Entry-gate verdict, full packs | 94.1% | 1 of 17 eligible shown pending | Officer completes pending items |
| Unknown facts invented / pending shown as passed | 0 / 0 | of 126 / of 16 | Policy engine: unknown → PENDING |
The question is not whether the model is perfect, but whether each error type is (a) rare enough that checking beats re-keying and (b) visible to the checker.
At 99.5% the analyst confirms rather than re-types. On full packs (79.5%), checking every decision-critical figure is mandatory. Accuracy here measures review workload, not appraisal accuracy.
Recall must be complete, and it was (11 of 11). 50% precision is tolerable short-term but would erode trust in production, so it is the first development target.
An identifier that leaves the bank cannot be recalled. At 73% held-out recall, raw documents stay inside; only structured, harness-generated fact sheets go out.
| Control point | Proposed threshold | Status on synthetic corpus (S2) |
|---|---|---|
| Decision-critical fields revenue, assets, equity, debt, inflows, collateral, amount | ≥ 99% per document family on the bank's own set, plus 100% human verification | Iter. 2: 4 of 6 families Iter. 3: 79.5% |
| Descriptive fields names, addresses, dates | ≥ 97%, verified at the gate | Met |
| Red-flag recall / precision | 100% recall on a regression suite / ≥ 70% precision | Iter. 2: recall met, precision 50% Iter. 3: 93.1% / 69.2% |
| De-identification only if raw text leaves | ≥ 99% recall on a held-out suite refreshed by compliance | Not met (73%) raw text kept inside |
| Figures in drafted reports | 100% traceable or acknowledged before confirmation | Met by structural check |
The group's proposal for calibration with a partner bank's risk and compliance functions, not an observed industry standard.



Intake, masking, extraction, reconciliation and research run under the task-routed policy, each step with tier, time and cost.
Every field shown with its source and confidence. Accept, amend or reject before a memo can be drafted. Time and decisions logged.
Drafted from a masked fact sheet. Approve, approve with conditions, decline or refer stays with the credit officer.
| O1 | Quantify the manual burden | Not met |
| No partner access. Interview protocol, sampling frame and baseline record prepared and placed first in the roadmap. | ||
| O2 | Specify the framework | Achieved |
| O3 | Build and evaluate a prototype | Substantially |
| Amendment rate and full cycle time are instrumented, waiting for real reviewers. | ||
| O4 | Model-tiering method | On synthetic data |
| Quantified baseline | Not met |
| End-to-end prototype, 15 files × 5 tiers, 0 failed | Met |
| Zero boundary violations | Met on corpus |
| Frontier supports a tier choice | Met |
MBA Capstone 2026 · Group 5Roadmap, people and compliance clearance
Sponsor and steering committee; interviews and manual baseline; DPIA; anonymised validation set per family; on-prem hosting of the open-weight model
One branch or SME desk, 3–5 analysts. Runs alongside the manual process on live files; output not used for decisions
Replaces manual data assembly for SME working-capital loans at pilot units; four-eyes unchanged; monthly control testing
More regions and adjacent workflows (renewals, trade finance, collateral revaluation) reusing harness skills; periodic re-tiering
Hosting. 8B-class model on bank GPUs or domestic private cloud; re-verify accuracy on that hardware
Integration. LOS, document management, CIC feed, core banking; write the stamped report back to the LOS
Evaluation as a control. Every model, prompt or format change passes the validation set first
Backlog. Disambiguation, learned de-ID, OCR, run variance, done in Phases 0–1
Chaired by the CRO, monthly in Phases 0–2, owns every go / no-go.
Appraisers review a machine-assembled file and own the judgement sections. Saved time goes to analysis, not higher file counts.
Known error types (unit scaling, adjacent terms, invented figures), the review gate, version stamps and override logs.
AI product owner from credit, 2–3 engineers, and an independent model-risk validator in risk.
Analysts help set thresholds. The signing officer, not the system, owns the appraisal.
| Activity | Credit | RMs | IT/Sec | Compl. | Risk | Audit | HR |
|---|---|---|---|---|---|---|---|
| Baseline and requirements | A | R | C | C | I | I | I |
| Model hosting and integration | C | I | A/R | C | C | I | – |
| Routing policy and de-ID | C | – | R | A | C | I | – |
| Validation and thresholds | A | – | C | C | R | I | – |
| Training and role redesign | A | C | C | – | – | – | R |
| Phase-gate decision | R | C | C | C | C | I | I |
| Ongoing control testing | C | – | R | C | A | R | – |
A steering committee chaired by the CRO meets monthly in Phases 0–2 and is accountable for every go / no-go.
State Bank of Vietnam supervision, PDPL 91/2025, Law 32/2024 and Circular 09/2020 decide what the architecture may do.
DPIA under PDPL 91/2025/QH15 and Decree 356/2025: which data, where, on what legal basis. Simpler because raw documents never leave.
Routing consistent with Law 32/2024/QH15; system classified under Circular 09/2020/TT-NHNN with its required controls.
If an external model is used: residency, no retention, no training, audit rights, plus a formally accepted residual de-ID risk statement.
Machine data confirmed by a named officer; untraceable figures block confirmation; approval stays with the approver.
Call log, payload hashes, override log and version stamps mapped to audit tests before Phase 2.
MBA Capstone 2026 · Group 5Evaluation, recommendations and key insights
| Finding | Serves | What it means | Confidence |
|---|---|---|---|
| 1 Medium sovereign model suffices | O4 | Task-level tiering; the largest model is not required | Moderate synthetic, one run |
| 2 Code beats model on units | O2 | Generalisable design rule for the harness | High large effect, clear mechanism |
| 3 Layout, not noise, drives error | O4 | Validate per document family, not per model | Moderate simulated noise |
| 4 Full recall, imperfect precision | O3 | Only acceptable with a human gate | Moderate 11 flags |
| 5 Boundary held; masking weakest | O2 O3 | Raw documents must stay inside | High for the weakness |
| 6 Model cost is immaterial | O3 O4 | The case rests on analyst time and governance | High for cost; time unmeasured |
| 7 Human role moves to judgement | O2 O3 | Controls must be structural, not prompt-based | Moderate 17 packs, one run |
Small synthetic corpus: S2 vs F1 is two fields
One run per configuration; outputs vary even at temperature zero
One provider's infrastructure: "sovereign" is a role, not a location
No interviews or baseline: no claim about time savings
Row-code anchors separate charter capital from equity; cross-check the registration certificate. Precision 50% → 70%.
In-perimeter NER on Vietnamese banking text, a bank allow-list, and a red-team suite owned by compliance.
In-perimeter OCR and vision; re-run tiering on the partner bank's anonymised documents per family.
At least 3 runs per configuration; re-run the frontier on every model, prompt or format change.
Reject amounts inconsistent with size, VAT and inflows. Would have caught all 3 mis-scaled packs.
Phase gates become mechanical, not negotiated.
One body of evidence for the DPIA, model-risk validation and audit.
Units, row codes, ratios, policy rules and credit-need method as shared assets.
Connect to LOS, DMS and core banking; keep the approval workflow.
CRO-chaired, so the workstreams move together.
An 8B sovereign model matched the frontier stand-in on extraction.
Whatever can be specified exactly belongs in the harness.
Measure de-identification; keep raw documents inside.
Even when told not to write them. Controls must be structural.
Still unproven: the size of the time saving in a real bank. Phase 0 of the roadmap is designed to measure it at the first gate.
Questions and discussion
