Impacting lives with a better data platform

Finding the needle in the haystack

Each year, some 12,000 people are diagnosed with blood cancers such as lymphoma and leukemia, and other diseases like sickle cell, for which a blood stem cell transplant from an unrelated donor might be their best or only chance for a cure. When customary treatments like radiation and chemotherapy fail, many patients pursue therapies that utilize donated stem cells from bone marrow and umbilical cords to restore the body’s ability to make healthy blood cells. In these cases, doctors will reach out to an organization like NMDP℠ to find a matching donor and facilitate a transplant. With its superior technology to match patients to donors, every search of the NMDP Registry℠ searches more than 41 million potential donors from worldwide registries to ensure patients find the best possible donor match. 

Finding suitable matches for bone marrow transplants is a highly complex task. Unlike “hard organs” such as kidneys, bone marrow transplants require an incredibly close fit in tissue typing to succeed. Only 30 percent of patients will find a fully matched donor in their families, the remaining 70 percent will turn to NMDP to find their best donor option. Without a sufficiently precise correlation of key genetic information, the transplant could be rejected, or worse yet, cause serious complications.

NMDP debuted its HapLogic search and match platform in 2006 to solve this complexity and has been incrementally improving it since then. The technology works by applying a complex algorithm to an extensive relational database made up of both NMDP Registry and the combined registries of prospective bone marrow donors around the world. This search both finds matches and predicts the quality of matches for all genetic components critical to a successful bone marrow graft. Over time, as the platform grew more sophisticated, BTM began facing challenges of scaling to continue meeting its compute- and data-intensive needs. 

A scalable solution to meet ongoing developments

In 2017, NMDP began investigating new data management solutions for its HapLogic platform. As a non-profit, it required a cost-effective solution that was open source and adaptable enough to handle ongoing developments in medical science. It also needed to be powerful: the HapLogic algorithm queries a slate of some 41 million bone marrow sources, and the organization runs as many as 700 searches a day—making speed and precision critical to helping patients find a lifesaving match. 

After considering various options, NMDP turned to Cloudera to modernize and scale its matching system. With Cloudera Data Platform (CDP) Public Cloud on AWS, NMDP has been able to execute its highly distributed application and deploy resources as needed to achieve a much higher search performance than was possible with its legacy system.

“In addition to a greatly improved service experience for our clients—shorter search times, higher quality searches with a better level of prediction—we have also used Cloudera technology innovation to reduce our IT expenditure as we mature,” said David Wroe, Principal Software Engineer & Solution Architect for NMDP. “Our move to CDP has resulted in significant cost savings for the organization and a reduction in the infrastructure maintenance expense measured in millions of dollars. As a non-profit, this affords us tremendous operational flexibility that was not previously possible.”

More precise performance gives doctors and patients more donor options

Since switching to Cloudera, NMDP’s search application has improved significantly. NMDP can perform extensive searches of worldwide donors in 35 seconds—down from an average of 60 seconds before adopting CDP—and with much greater precision as well. For the most difficult searches, NMDP has experienced a tenfold increase in speed.  As the organization utilizes research to improve outcomes for patients using mismatched donors to treat patients who do not have a fully matched donor available, the ability for the system to expand searches for an array of alternative transplant options is now made easy due to the new platform.

In particular, Cloudera’s technology enables NMDP to achieve much greater success in finding matches for ethnically diverse patients, especially those of South American and African American descent. These individuals have a more diverse genetic makeup than other populations and lower representation on the registry as result of historic and present-day inequities in healthcare. With a more complex tissue profile to match and fewer potential donors, the search process involves a far greater degree of computational complexity. Cloudera’s architecture easily accommodates these searches. 

Critically, as a non-profit organization, the move to CDP has enabled NMDP to both reduce and avoid costs while also improving services. Coming from an initial deployment on a proprietary licensed system that was cost-prohibitive to its plans to scale the platform, the move to CDP Public Cloud has both increased NMDP’s scaling flexibility and reduced its total cost of ownership for the service. The organization anticipates further reductions in operating costs through the ability to size its environments according to need. For example, it can scale up resources to accommodate peak search hours and can effectively pause the system when not in use.

“Cloudera CDP gives us the freedom and bandwidth to innovate more,” said Rob Hanson, Vice President, and Technology Officer, NMDP. “Pre-Cloudera, and before conversion to the cloud platform, we were often chasing ways to improve search times. Now we’re chasing after things that are way more important to the value of our business because it’s just a given that the platform is fast, capable of handling complex searches, and doing all the things we trust it to do.”