Insurance Claim Scraper: extract, transform & load
Auto Insurance Claim - Custom software development
- Weeks to 100% ROI
- 3
- Insurance provider support growth
- 33%
- Workflow cost reduction
- 150%
Background
When a person files a vehicle insurance claim, providers turn to a company for scheduling a damage assessment. That company— a K-Optional Software client— specializes in getting every vehicle appraised so that the insurance provider can cut a check for the repair.
Working with insurance providers means adopting long-lived industry conventions; our client receives claims to a dedicated email server in a pseudo-consistent format.
Before working with K-Optional Software, our client had written a script in-house for reading this inbox, attempting to interpret the data, and exporting it into a CSV file. This approach proved brittle: slight variations in claim emails confused the script causing entire CSVs to drop. Worse yet, the evolution of email authentication— actually a welcome improvement for IT security— broke the script entirely.
So K-Optional received an urgent request to quickly stand-up a robust process that reliably processed mangled emails while the client depended on it. This was a textbook “fix the plane while it’s flying” scenario.
The challenge
- Scraping email content is difficult. For one thing, the underlying SMTP protocol isn’t conducive to fetching data.
- Plus, unlike, say, websites, email content may consist of varying data formats. Suffice it to say that emails are even less machine readable than HTML web-pages.
- The predecessor script failed to handle mangled emails, a nontrivial proportion.
- Finally, this engagement demanded urgency; our client’s pipeline slowed to a trickle without automatic claim-intake.
Our assessment
- After investigating the premiere SMTP libraries in a host of programming languages, we concluded that we should find a way to minimize the use of email protocols.
- We also determined that a flexible parser would enable our client to support additional providers easily— that huge realization ultimately allowed them to sign a new contract after delivery without any follow-up engagement.
- We cataloged this business problem as an Extract, transform, load (ETL) problem and applied industry knowledge in our solution. A 4-step processing pipeline borrowed components from state-of-the-art compiler design, an esoteric field in computer science.
Results
K-Optional delivered a working MVP within a week. While the client employed this release, we solidified our full version 1 build in the span of two more weeks.
The client has noted an immediate elimination in dropped claims, a flexible and intuitive interface, and a boon to their business model.
Merely three weeks after delivery, the client seamlessly supported a new provider with no additional consulting necessary.