Building Web3 Credit Scoring Capability (Part 2): Lending Lab Highlights & Modeling

This article is Part 2 of “Building Web3 Credit Scoring Capability” series. Click here for Part 1.

First..some context

Before getting too deep into the details, here’s who we are and what we do.

Credit risk is an integral part of Atadia

Atadia is a Web3 collective analytics business with a goal to reinvent credit and commerce with the power of data science and collective wisdom.

We produce easy-to-consume Web3 profile data from billions of blockchain transactions.

One dimension of such profile data is someone’s creditworthiness 😇 🤝.

We believe that creditworthiness is an integral aspect of any credit system.

Success-case scenario -> Web3 credit risk enables new credit products along side trustless financial/DeFi products + offers another data point for IRL credit scoring.

Ok…so how does one capture “Web3 credit risk”?

Let’s start with IRL first.

Historically, credit scoring (the process to gauge the risk involved with a loan to someone) is a mix between cultural psychology and statistics. Age, gender, occupation, spending patterns, income slips, where the person lives,…you get the idea.

Recently, this process increasingly relies more on Big Data, particularly “alternative credit information” that we all leave behind as digital footprints (shopping behavior, social media usage, and telco/internet usage). With so much data available, almost anything could technically be used to see if it can help predict the likelihood that someone is going to go 90 days past due (this is a typical “default status” definition).

After ingesting enough data points and testing a sufficient amount of statistical models, machine learning engineers eventually land on a “formula” that hopefully can predict credit risk reasonably well.

A “good” credit scoring formula would be able to distinguish high risk borrowers from low risk borrowers even if it’s the first time they’re borrowing.

Modeling Web3 Credit Risk through Lending Lab

End-to-end zero-collateral lending process on Web3

There is no purer way to train a Web3 credit scoring model than disbursing money and collecting actual repayment data for real.

This is why we started “The Atadian Lending Lab,” a zero-collateral, zero KYC, lending service back in April 2022.

It was a pretty bold idea. Still is.

The goal was to build a repayment data set so we can feed over 500+ aspects of wallet owner behavioral data observed on-chain to the algorithm and see how these relate to repayment status.

Here are the terms & design (as of Oct 2022):

• 7-day tenure
• zero-collateral
• 5% interest for non-holders. 0% interest for holders
• Loan sizes are 1, 2, 3, or 4 SOL
• Submit loan application at https://product.atadia.io/
• Receive SOL in your wallet in 24–48 hours (or instantly for VIPs with high score)
• Pay back by sending SOL + interest to atadia.sol
• Minimal notification / debt collection efforts (Discord bot, Debt collection NFT reminder, no email yet)
• Below the line marketing

Lending Lab Highlights

We operated the Lending Lab in 2 phases.

Phase 1: 6 April, 2022 to 15 May, 2022.
Phase 2: 28 July 2022 to today.

From the two phases, here are some highlights

• ~ 1,600 loan applications approved
• ~ 3,800 SOL worth of zero-collateral loans disbursed (this was across multiple vintages and cycles)
• Default rate (volume basis, P+I, no write-off) from two phases = 24.6%
• Default rate (volume basis, P+I, no write-off) from Phase 2 only = 4.5% (on +1700 SOL volume)

👉 Our default rate on our mint-now-pay-later campaign was too low so we decided to be more adventurous in Phase 1 and marketed these zero-collateral loans as “mint money” to support new pre-mint projects.

👉 Phase 1 was pretty wild. We were liberal. Prior to the big crypto crash, we decided to shut down the Lending Lab. That was probably a good call 😂.

👉 Phase 2 was a more civil. The default rate also decreased dramatically due to a number of potential factors:

• Improved blacklist database
• Improved Lending Lab security
• Integration of Web of Wallets (WoW) algorithm in the underwriting process (i.e. if loan applicant has a strong association with a blacklister, even if they have a high credit score, he/she will be declined a loan)
• Emphasis on “credit building” through Atadian Pass (our data-backed soulbound token) rather than on the using the loan to mint or flip

👉 For our team, the highlight was of course the opportunity to build an end-to-end Web3 credit decision engine and the credit scoring models.

Each of the two figures in the bottom right corner above indicates the receiver operating characteristic (ROC), which is an indication of model performance. The straight, 45 degree black dotted line indicates 50% accuracy, while the ideal situation would be a vertically mirrored L shape with 100% accuracy. The area under the curve (AUC) is a common measure of model accuracy (out-of-sample in this case).

Key achievements:

1. We built a system to automatically underwrite and disburse tokens based on on-chain data-driven criteria (e.g. credit score > X or wallet isn’t blacklisted) 🤖
2. This system can connect with any loan portfolio monitoring admin dashboard to help with risk management 🖥
3. Our credit scoring models are improving each day 💪

On point 3, the Zero-collateral Model (AUC=0.86) is trained on data from the Lending Lab. While the out-of-sample performance is quite good, it will be more reliable and generalizable with a growing sample size.

The benefit of having a high performing Zero-collateral Credit Model is clear: it helps the lending entity do risk management more effectively, especially if we will be seeing traditional forms of credit on Web3.

The Collateralized Model (AUC=0.82) is trained on data from our partners’ to improve their user experience. The sample sizes are considerably larger but the nature of the risk is entirely different. The model picks up liquidation and malpractice risks more so than credit risk.

The benefit of having a high performing Collateralized Credit Model is also clear: it helps lending platforms monitor and gauge volatility as well as offers more appealing products that fit with characters of lenders/borrowers more effectively.

What’s Next for Atadia and Web3 Credit Risk?

We look forward to these 4 credit-related initiatives.

1. Instant zero-collateral loan service — we rolled out this feature a few days ago. If you’re a high credit score user, you may be eligible for getting the most convenient flash loan on earth.
2. Scaling zero-collateral Lending Lab — we will be scaling our Lending Lab by launching our own deposit pool with a partner. Funders interested in being part of this adventure can deposit SOL into the pool for Atadia to utilize in exchange for APY of a different risk profile.
3. Credit scores integrations — we look forward to integrate our credit scores to DeFi and NFT/lending platforms to help them enable a better user experience. Check out our latest partnership with Sharky here.
4. Expanding the types of “credit risk” models — we alluded earlier that there could be multiple definitions of credit risk on Web3. While zero-collateral product through the Lending Lab is the closest relative to IRL consumer credit, there are other aspects of risk that are crucial to risk management in the age of DeFi such as liquidation, volatility, exposure, and malpractice risks. These plan to fully support our lending partners.

There’s much to discover still. With our recently updated data engineering capability, we expect to improve our understanding of credit risk (in whatever definition) even more.

See you next time!

🥂🥂