Data Migration (Part 1)

As most existing clinical information systems are planned to be replaced by new 'NPfIT compliant' solutions over the next few years, organisations across the NHS are increasingly faced with the dilemma of what to do with the current and historic event data that resides on their existing systems.

In the first of two 5in5 articles, we discuss the issues associated with full migration of data.

In the next issue, we explore some of the alternatives to a full migration and details of our data transformation services.

Some Organisational Issues

Although some new systems will be 'greenfield' implementations, the majority will be replacements to existing systems. Organisations will need to ensure that there is a smooth transition between systems such that there is no detriment to operational services and no loss of clinical or management data. Although data migration from old to new is the obvious option, this process is often fraught with difficulty.

Remember that there are normally going to be at least two external parties involved - the 'owner' of the outgoing solution and the owner of the incoming solution. In the health context, the outgoing owner is quite likely to be going out of the market entirely and the incoming owner is going to be an LSP and their subcontractor(s). This makes for some interesting group dynamics, with the NHS organisation right in the middle!

Clearly, the incoming supplier should have the fullest understanding of the structure, logic and limitations of the new system and would therefore be in the best position to take ownership of the migration process. However, LSP contracts specifically exclude data migration from the core contract and define this as a client responsibility, with an expectation that client sites will do all the work necessary to provide a pre-formatted migration extract ready to load onto the new system.

Outgoing suppliers may be prepared to assist, but there is little incentive for them to expend a lot of effort in an activity that will result in the termination of their service and this often results in a price premium to pay for this assistance.

What Does Data Migration Involve?

For a migration to work successfully there needs to be a comprehensive data map to ensure that each existing data item has a potential and valid 'home' on the new system. This will involve:

On the source system, identifying each data item, its meaning and function, its format and its relationship with other data items;
On the receiving system, identifying a corresponding data item and identifying its format and its relationship with other data items;
Specifying where data transformations are required, for example where date formats differ between the two systems or field lengths are different;
Specifying 'value to value' mappings of reference data between the two systems, for example user code 'AJH' on the old system needs to become user code 'AJH01' on the new system. Note that this may require the creation of otherwise redundant reference entries on the new system to allow the mapping of historic data, for example for users who have retired or procedures that are no longer carried out; and
Ensuring that there are no logical differences between the two systems which would result in transferred data presenting a different meaning or triggering inappropriate workflow activity on the new system. Experience shows that this is often a problem and can result in an inability to transfer key data items.

OK, so you have your data map, now you need to:

- Set up all the new fields and code values on the new system (assuming that it is developed sufficiently for this to happen);
Commission trial data extracts from the old system supplier;
Find a way of achieving the necessary data mapping and transformations to produce a pre-formatted output file ready to load to the new system;
Undertake any essential data cleaning or de-duplication work to ensure that the data being transferred is valid and correctly referenced e.g. uniquely identified by NHS number - this may be a considerable task and may require external technical support;
Arrange with the new system supplier for the trial extract to be loaded onto the new system;
Painstakingly check that everything appears to be as it should on the new system, that no data has been lost or mistranslated and that no erroneous processes are triggered on the new system.

Assuming that this all runs successfully (miracles do happen occasionally), you will then need to commission another extract for your 'go-live' event, to ensure that current data is loaded to the new system at start-up. Remember that typically this go-live impacts on service users and has to be carefully timed.

Within all this, there is a distinct probability that some data items will have no logical home to go to on the new system and therefore cannot be transferred, or will have to be transferred as text items. In addition, some data may need to be manually re-entered because of differing processing logic between the two systems.

Tips for Migrating Data

There are a number of things you can do to make the transition less time critical and manageable in tranches rather than as a big bang migration.

Manually pre-entering sufficient information to prime current activity on the new system, e.g. by keying in details of outstanding appointments. This may require a lot of frenetic activity over a 'go live' weekend.

Entering data onto the new system as events occur, e.g. by booking in outstanding appointments on the new system as patients attend instead of trying to pre-load appointment details in advance.

Is Full Data Migration the Best Option?

Assuming the migration process is viable and safe and that the transferred data is useable on the new system after the changeover, full data migration is the cleanest option to take. However, it might also be the most expensive and, ultimately, not that useful.

There are other options that should be considered through a cost/ benefit analysis before you embark on this hazardous journey. Key questions to be asked are:

Do I need all my old data and why?
Does it all need to be on the new system or could it be held elsewhere?
How good is our data quality - how much needs to be done to clean it up?
How safe is the migration - how well do we understand the processing logic of the new system?
Can the migration work straight from the old system to the new or does there need to be significant transformation work?
What kind of help will I need/ get from the suppliers?
How much work is involved for me to do?
How accessible/ useable will the data be after migration?
What is going to happen to the old system?

Part two of this article will discuss alternative options to full migration and more of the technical implications of the migration process.

If you have any questions about the subjects covered in this white paper or you would like to find out more about how Oakleigh Consulting could help your organisation, please contact us on 0161 835 4100 or email us.

Copyright

You may publish, quote or reproduce any white papers on this website on the condition that Oakleigh Consulting Ltd is notified, properly credited (and linked to) as the source, including our URL: www.oakleigh.co.uk.

EMAIL THIS PAGE

Close this box

To:

Recipient email:

From:

Your name::

Your email:

Your message:

JCapture
Please enter the characters above:

This action will generate an email to the person above recommending this article. Your email address, and the email address of the person you are sending this article to, are not logged by our system.