Blog: Healthcare Data Growth: An Exponential Problem?

Blog: Healthcare Data Growth: An Exponential Problem?

Health data was anticipated to grow at an alarming rate of 48% per year, from 153 Exabyte’s in 2013, to 2,314 Exabyte’s by 2020 with images and media forming the majority of that data.

The challenge facing healthcare organisations is how to manage the masses of data in a cost effective manner. Without this data organisations will not be able to leverage the improvements in patient care that technology advances deliver.

What is causing this wave of growth?

Diagnostic imaging is the most significant contributor of data growth with the number of studies increasing each year and the quality of image resolution improving in parallel. As 3D imaging becomes more common, file sizes are increasing to several MBs in size. There are now various modalities, applications and devices creating this kind of image data and video, and they are in operation across all departments.

Areas and impacts to consider include;

  • Pathology – the drive to digitise services creates potentially 100TBs of storage per healthcare organisation.
  • Genetic Sequencing – The 100k Genome Project is establishing a number of genomic centres in the UK. As this becomes more widely adopted as part of a personalised medicine strategy, PBs of storage will be required every year.
  • Big Data Analytics – Are healthcare organisations truly ready for this? With a traditional approach, this may require doubling up of capacity before an analytics application can even begin to analyse the data.
  • UK Government – As the push for EPR continues, this adds several TBs of data, particularly whilst paper records are still being scanned.

Managing 100TBs to PBs of data can be very difficult, requiring potentially several people, equipment and significant cost to perform the various tasks needed to maintain and protect that data.

A strategy to cope with the data deluge

Several factors need to be taken into account to build a strategy able to cope with the data growth whilst still maintaining control over costs.

Scalability to centralise all image stores:  Any solution must be scalable to several PBs in size using a scale out architecture. This facilitates a centralisation strategy to consolidate all image stores into one reducing the cost of managing multiple systems.

Simplicity of access: Implement a platform everyone can use and make it simple for all departments and applications to access that data. Use standard file sharing protocols such as CIFS (windows based) and NFS (UNIX based) which all applications can utilise.

Balancing performance with cost: This can be tricky to do; most clinicians will want lightening fast access to data. Typically most of that data will quickly become cold and not be accessed for a long time (if ever). Reduce your TCO with a strategy that can provide high performance but with integrated archiving to reduce cost.

Reduce the risk and cost of data migrations: Incorporate a methodology to enable simple low cost data migrations with no downtime for hardware replacements. Data should be simply moved internally on the system to newer storage with no downtime and little cost.

Eliminate the backup window and save costs combining backup and DR:  For critical clinical data there must be little to no data loss. Recovery should be simple and quick allowing services to be quickly brought back online after an incident. Traditional methods of backing up TBs to PBs of data are no longer practical, leaving your organisation at risk of data loss. A solution is needed that is robust enough to allow a more simple strategy that will lower the cost of backup and DR.

Future proof for big data: Though big data analytics has not made any significant progress in the NHS, it is on the horizon. Ideally the solution should analytics ready, i.e. able to integrate into Hadoop based systems to allow analytics. This is particularly important for organisations with strong links to research.

Reduce long term costs with Cloud integration:  It is simply inevitable that healthcare data will be stored in the cloud, particularly archive data. Any solution must be able to integrate with cloud providers to allow the flexibility of data to be moved to cloud in future.

What’s the solution?

From my experience in healthcare at EMC and Block, a strategy based on the proven technology of EMCs Isilon solves all the challenges of managing current and future healthcare imaging data growth, at an affordable cost for any Trust to implement.

Isilon is a scale out NAS solution built to scale to PBs in size and protect data at that scale.  Isilon has a single filesystem with several GB/s of bandwidth capable for high performance. As it is a NAS solution, it uses simple standard file sharing protocols for data access, such as CIFS for Windows and NFS for UNIX and Linux applications, connecting over standard Ethernet network.

Isilon is the perfect solution for meeting the challenges arising from the growth in imaging data as it provides the scalability, performance, low cost storage with archiving, simple file share access, robust architecture with inbuilt backup, is Hadoop big data ready and cloud integrated.

Isilon is already a proven solution with several organisations leveraging Isilon to manage and protect their data. So you can be safe in the knowledge it is a proven solution.

Having the right strategy now, based on Isilon, to manage your healthcare data will not only have massive cost savings, it will greatly benefit the agility of your organisation to be able to react quickly and become a facilitator for improving clinical imaging facilities and ultimately improving patient care.


Imran Arshad
Data Centre Consultant