Cutting a clinical lab results pipeline from 3 hours to 3 minutes

Lead Backend Developer · Diag Laboratories · Dec 2020 – May 2024 · Team of 15

The problem

Diag Laboratories is a clinical diagnostics provider. Test results come off a variety of lab machines, get processed, and have to reach doctors and patients across both a web portal and a mobile app.

That pipeline ran through a large monolith and took roughly three hours end to end. In diagnostics that isn’t a performance metric — it’s a patient sitting in a waiting room, or a doctor making a call without the numbers in front of them. Worse, result PDFs were sometimes generated incorrectly or went missing entirely.

The obvious fix was a rewrite. The obvious problem with a rewrite is that this was a live clinical system that could not go down.

What I built

  • Incremental migration, not a big bang. I led the move from monolith to microservices using a lazy migration approach — services were carved out and cut over one at a time, with the monolith still serving everything not yet migrated. No flag day, no freeze.
  • An event-driven results pipeline. Kafka to collect examination results from lab machines, orchestrate processing, and fan out to the web and mobile delivery services.
  • The backend from scratch. New services written in Laravel and Python against PostgreSQL on GCP, plus a data warehouse for reporting.
  • Correctness as a first-class concern. The PDF generation path was rebuilt and hardened, because a missing or wrong result document is a clinical problem, not a bug ticket.
  • Standards for a team of 15. I defined the coding standards and review process, and made sure they actually held across the team rather than living in a wiki nobody read.

Result

  • Result processing went from ~3 hours to ~3 minutes.
  • 100% accuracy in PDF generation.
  • 99% uptime maintained throughout a live migration on a production clinical system.

Stack

Laravel, Python, Kafka, PostgreSQL, GCP, microservices, data warehouse, lazy migration. Agile/Scrum with Jira.

If this sounds like your situation

The recurring shape here: a system that works, is too slow or too fragile, and is too important to switch off while you fix it. That’s most of the rescue work I take on. Here’s how to reach me →