The problem
Assessing a loan means reading months of bank statements and card transactions, checking them against the bank's criteria and writing up a decision. It is slow, paperwork-heavy and not always reliable, and a model that only outputs a risk score does not explain itself to the banker who has to sign off.
CLARA (Credit & Loan Assessment with Risk Agents) is a modular multi-agent system designed to aid financial institutions in their assessment of loan applications, driving decision-making and reducing cumbersome paperwork.
What I built
A banker fills in the application, uploads the applicant's bank and card statements, adds comments and sets the bank's current risk tolerance. Seven agents then work through the case, and the banker gets back a decision, a short motivation and a full report they can edit and regenerate. Applications and decisions are saved to a history.
- Predictive models and LLM agents together: classical models price the loan and score its risk, and LLM agents profile the applicant, reason about the decision and write it up.
- Grounded decisions: every decision is compared with the most similar real loans before it is made.
- Self-correcting: an evaluator can send work back to earlier agents, with a bounded number of revisions.
- Runs on paid or free models: Azure OpenAI GPT-4o, or open models through Hugging Face.
- Wrapped in a Streamlit app built on a LangChain pipeline.
Input
Loan application
Application form, bank and card transactions (CSV), the banker's comments and the bank's risk tolerance.
Agent 1
Behavioural agent
Turns three months of transactions into features and profiles, such as income stability, savings habit and spending concentration, and compares the applicant with the 10 most similar of 1,000 synthetic users.
Agents 2 to 5
Decision stage
- Interest-rate agentRandom-forest regressor trained on Lending Club loans
- Risk agentLogistic regression giving the probability that a loan goes bad
- Decision agentGPT-4o, grounded on the 10 most similar Lending Club loans retrieved from a Qdrant vector database
- RAG validation agentLLM judge scoring the decision's faithfulness, relevance and correctness against the retrieved loans
Agent 6
Evaluator agent
Reviews profiles, decision and terms against the bank's risk tolerance and the banker's comments, then routes the case:
- Approve: write the report
- Revise profiles: back to agent 1
- Revise decision or terms: back to the decision stage
- At most 3 revisions
Agent 7
Report generator
Writes the decision, a short motivation and a full report. The banker can edit and regenerate it.
The data
- Lending Club: the public Lending Club loan data from 2007 to 2018. Over 2.5 million loan applications train the interest-rate and risk models, and about 3 GB of historical loan records are embedded in a Qdrant vector database for retrieving similar cases.
- Synthetic applicants: real transaction data is private, so 1,000 synthetic users were generated, each with labelled bank and card transaction files. They serve as the reference cohort for the behavioural agent and as demo inputs.
- Features: from transactions, income and expense levels and volatility, savings rate, overdraft frequency, discretionary and top-category spending shares, credit-card repayment ratio and trends; from the form, loan amount and term, income, debt-to-income ratio, employment length, home ownership, delinquencies and bankruptcies, plus TF-IDF features of job title and loan purpose. Loans that were charged off, defaulted or ran late count as bad loans for the risk model.
Evaluation and safeguards
- Two independent checks on every decision. The RAG validation agent scores it for faithfulness, relevance and correctness against the retrieved loans. The evaluator then reviews the whole case against the bank's risk tolerance and the banker's comments, and can send the profiles or the decision back.
- It catches real mistakes. In a worked example, the first draft rejected a stable applicant. The validator scored its correctness 0.3 because the reasoning cited figures that were not in the data, the evaluator sent it back, and the revised decision approved the loan with adjusted terms, which the evaluator then signed off.
- Bounded loops. At most three revisions, then the report is written, so a disagreement between agents cannot run up cost indefinitely.
- Auditable and cheap. Every agent's output, every validation score and every API call's tokens and cost are logged. All development and testing used about 505,000 tokens, about $1.63, and under 2% of our budget.
What I would improve today
- Strengthen validation: a time-based split on loan issue dates, with discrimination and calibration reported for the risk model and error by loan grade for the interest-rate model.
- Measure decisions against people: a labelled set of applications with bankers' decisions, to measure agreement directly instead of relying only on LLM judges.
- Retrieve on structure, not text: replace the text embedding of each application with filters on purpose, term and grade plus numeric similarity.
- Review fairness and compliance before any real use: compare outcomes across applicant groups and make the explanations meet the requirements for explaining credit decisions.
- Make it faster: load the models once rather than on every call, and run the three validation checks in parallel. A full assessment currently takes a couple of minutes.
- idanh8/CLARALoan-assessment prototype combining predictive models, retrieval, LLM agents, and evaluation-driven revisions.