Mar 16, 2022
🤝 Mint-Now-Pay-Later (MNPL) Post-Mortem Analysis 🕵️♀️
Finding trust in a “trustless” environment
Gm. We are very excited to share the details and results of our first set of experiments on trust in the Solana ecosystem.😎
To recap, we minted all 5,000 NFTs last Thursday, public sold out in seconds, and reached 10K SOL volume after 6 days 🎉🎉 🤗 🤗
During this madness period of mint hype, love, and attacks from all directions…we somehow thought it was a great idea to conduct some experiments. These were Mint-Now-Pay-Later (MNPL) and Pay-Now-Mint-Later (PNML) programs.
Our MNPL wasn’t a free mint and was probably the first of its kind in the ecosystem(correct us if we’re wrong). It’s a pretty simple exercise to test whether our core product concept about trust in a trustless environment is possible or not.
It was also there to provide a bit of reality-check for our team and for those who minted our NFTs that we all are not totally crazy 😅. And if it doesn’t work, we can still pivot to something else.
To be sure: trust, reputation, credit worthiness, frauds…these are related but they are not all the same. They require separate models. We use them interchangeably mostly just to keep things short.
How MNPL works:
- Participants of opt-in to the program by registering and providing a bit of their personal information e.g. wallet address, DiscordID and Twitter handle + answer short questions (psychometrics-inspired…but tweaked to crypto context)
- Selection happens (round 1 was quite random, round 2 we loosely picked based on our initial unsupervised learning model w/o any bad flag training whatsoever)
- After selection, we airdropped 1 OG Atadian NFT straight to the wallets of selected participants and asked them to pay back 1.5 SOL (cost of public mint) plus extra money they desire to pay for the convenience service by the deadline.
The process was rudimentary at best and not at all tech-elegant. But it’s simple, Hassle-free! Except for round 2 when Solana’s TPS wasn’t so kind to us.
Receive the NFT first (without any collateral), repay us later 🤝
After 2 rounds of MNPLs with 96 and 50 selected participants for each round, we are thrilled to find that the default rate is much lower than we expected.
Out of 146 participants, the overall default rate is approximately 11 percent.
Some folks loved the program so much they also “overpaid” to make the losses more bearable 🤗.
Using social incentives/nuances and some basic data science in the selection process, combined with an overly cautious opt-in process, we were able to generate some level of trust in an otherwise “trustless” world.
This is far from a solid experiment but it does derisk our product roadmap by quite a lot.
We also looked at “time till repayment.”
Changes in the number of non-payment individuals from round 1 and 2 are separately displayed in the above plot.
Clearly, most people pay back before the deadline for both rounds. Look at the steep decline after 24–48 hours later.
Well…another thing we learned is that that time zones suck 👎 and repayment can be tricker to achieve when everything seems like “24/7” but borrowers (and us) have to go to bed, doubling the time it takes to do debt collection. A more effective Payment Reminders will be key.
Other interesting things we noticed is that time untill repayment tends to positively correlate with the volume of SOL transactions from the web of wallets, e.g. total received and sent SOL during specific time periods. More volume => longer time till repayment.
Can we go deeper ?
Of course. But with a sample size of 146, huge selection bias (to derisk our mint), and barely anyone defaulting, it doesn’t make much sense to spend weeks or try to do “Big Data” on this.
But we are curious folks…and overachievers. Plus we already built the data layers that just want to be hit hard by these results.
So, we went ahead to do some deeper analysis, mostly doing some comparison between default and non-default groups.
- What kind or how frequent of transactions did they make?
- How long did they own the wallet / NFTs?
- Interactions with other wallets
- Social activities apart from just web of wallets
… and more
The differences between the two groups aren’t huge but here are some notable ones.
The above plot depicts the characteristics among non-default and default groups where the x-axis is scaled to 0 and 1 for an ease of comparison.
While it is too early to conclude anything, wallet age, intensity of wallet activities in terms of transactions, the avg. hold-time of NFTs ownership as well as social interactions (Twitter) appear to be related to default status.
To go a little further, we conduct another analysis for round 1 and 2 separately given that we launched the experiments at different time frames.
For this, we compute the Net Monetary Value (SOL) for each wallet, e.g. receiving minus transferring out.
Some interesting things we found:
- The Default group tends to be more of “the Taker” (NMV > 0) regardless of the experiment round.
- For Non-Default groups, round 1 participants are most likely the Taker but the opposite trend is observed in round 2. Among this Non-Default group of round 2, the NMV is relatively higher for those holding top-rank NFTs.
Default Cases
For these cases, we employ a statistical technique called “Wall of Shame.”
No…at least not until tomorrow.
We employ social graph, which is constructed based on the connections between wallet addresses, hopefully to reveal more meaningful insights.
The below network graph plot shows the interactions for someone from the Default group. The two connected wallets (the blue circle) exhibit close connection (99.9% same owner).
To compare among two groups, we also build social graphs for non-default borrowers. The below graph is a good example. Relatively stable and more spread out interactions are seen.
A few words on debt collection…
Debt collection process is a bit painful (like IRL), but we had to find out why people didn’t pay back on time.
Among late-payment folks, some of them just had a lot on their plate. Almost instant repayments were successfully received after one round of the debt collection attempt.
For non-successful cases, we DM’ed most of them again if possible. Some (true) default individuals had already left our Discord. NFTs were sold in the secondary market.
Even though some of this is disappointing, it’s the reality, a challenge, and a reminder that we will have to continuously improve each iteration. Not because the bad guys get away with it, but because most folks are normal, responsible folks 🤝 and they deserve good credit services.
Ok…So What’s Next?
First, MNPL Wall-of-Honor will soon be announced with great appreciation from our team and the whole Solana community. These folks and their DAOs will get a headstart with PFPscore for sure! 🤝🤝🤝🤝
On the opposite end, the Wall-of-Shame for those who default will also be featured on all of our communication channels as well as broadcasted by our media partners! 😅
In the days/weeks to come, we will be focusing on 4 things.
- Product-market fit ==> finding the right type of credit products to do experiments with
- Population ==> defining the right sort of population we try to get at and the type of “bad flags” we want to model against
- Process ==> designing the right process that fit with the 2P’s above
- PFPscore v0 ==> calculating the first set of PFPscore based on a few rounds of experiment results along with “existing bads” from data provided by our partners (rug-pulls and known scams)
In general, we will need to be more bold as well and may need to lose some money to gain insights in return.
Obtained data from future lending rounds will be analyzed in a similar, but grander way. The list of data features (characteristics about wallet/PFP NFT owners) is growing each day to better capture hidden behaviors thanks to our hardworking data scientists ❤️❤️❤️.
If you read until this far, I believe you either invested so heavily in us or could be someone we should hire…we have a job board in our Discord :)
Stay tuned for more activities and join our discord for further information if you haven’t done so already!
🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝🤝
