Data Migration Project - Introduction

I would like to apologize for this, but first we are going to ask to Wikipedia. What´s Data Migration ?

In my particular case, two reasons  qualified me to join in this data migration project:

1. A successful CRM implementation that I managed in the past.I had been working with CRM System as system analyst, I had acquired a great knowledge about CRM data model and its main functionalities. I was responsible for maintenance of CRM System in the Company, I was able to understand where the data was and how I could access it. Its connections  with other corporate systems as GIS System and mainly the Billing System. Because of this, every time I had to deepen their understanding of what it was like and the flow of data between these systems,  I improved the capability of understanding their data model individually. So, in a certain moment I was in the middle of a new challenge that would be able to  qualify me to join in this project.
2.  Data Administration: I was a Data Administrator, one of my responsibilities were to understand and to manage the data model of all transactional systems, as I mentioned above.Apply a data quality management for all of them , manage data models both Transactional and Analytical.Specify data validation, verification and reconciliation strategies in both environments: Transactional and Analytical.

Provide support to developers in both systems CRM and Billing, among others. 

I have known from the beginning that I had a great opportunity to face a huge challenge in my career and I was joining both subjects  that I really liked to work, CRM and Data. From that moment , I started my preparation to face it…

The work has begun… From this point I suppose that all tasks before the kick-off meeting were made and below are the related important issues that you have to consider within the first 30 days of the project, at least. Remember, at the beginning you need to expect the functional decisions to start the data migration design, so, just after that, you could think to develop any extract program or ETL procedure.

Therefore, you must have planning after planning, take an advantage from that, develop  one of the most important document that you will have in our hand during the project life, that will give you all instructions to achieve your goals:

• Data Migration Strategy , which comprehends:
• The choice of the team members.
• The team organization.
• Roles and Responsibilities of each data migration member.
• The main activities that will be executed during  the project life related to data migration.
• The objects that will be target to migration, previously identify that SAP IS-U/CCS and CRM needs to start to working. They are provided in a SAP documentation.
• Conversion Tools, both legacy and SAP.
• IS Migration Performance in SAP Environment (SAP provides a document that helps you to enhancement the performance of  migration).
• Data Migration Performance in legacy system, based on choice of tools that will be
• Documentation, relate all the documents that will give a data migration team during the project life. Functional documents, standards, flat files, specifications, ETL programs, PL/SQL procedures, Binary files, Log files, Management documents,  Status, everything that you need.
• Data Cleaning strategies to support data migration
• Data Migration Cycles, test and simulation that will occur during the project, previously scheduled.
• Status Report and Metrics to follow the Data Migration progress.
• Risks and Issues about data migration
• Reconciliation rules in each data migration cycle.

Please, don´t worry, you will check this document, once, twice, a hundred times if you need, but I can sure you, you will be prepare to start any data migration activity after that.But...

Tip(s): There is important tip that I should give you during this planning period is: There is in SAP IS-U/CCS a specific transaction which was developed aiming of load of data, its name is EMIGALL. This transaction read just binary files. So, If you have choose it,  you will need to investigate how much your environment is prepared to generate this kind of file.

Sharing my experience with you.

Our data were stored in Oracle Database version 8 and 9i, obviously, we decided to use Oracle PL/SQL procedures as way to carry out the data extracting directly from it.

We spent a several time trying to understand how we could do extract at same time both files,
Text: it would help the extraction team to trace and find possible errors in data migration procedures.
Binary:  it was necessary and load it in SAP successfully.

After I many tests we found a way to generate a binary file through the  enablement of JAVA layer inside the Oracle Database, as showed in this picture ->  
...but, we had to pay attention:

• It was possible to be done just in the Oracle Database version 9 or upper.
• It was necessary the previous installation of components into Database. You should to check with your database administrator which are them.
• We had a additional step after that to handle with extract programs developed in Oracle Database 8, we were afraid of impact our extraction performance because that, but how we found this problem  in advance, we inserted this step in our migration window.  
• If the previous conditions are not possible to performing, also, you keep contact with Companies that had a similar issues when implemented SAP ISU. I talk this, because we spent a lot time to discuss with other Company at that time, we asked for help, every call we were able to understand the mechanism, even tough, they had other architecture in your migration environment. But, Every single tip, helps us. Don't you agree ?.

See you... Next post we will discuss about each topic related in the data migrations strategic document.

Sentence of the day: "If you change the way you look at things, the things you look at change." (Dr.Wayne Dyer)