SAP Data Migration – Dealing With Fallout (Part 3)

One of the inevitable aspects of data migration is dealing with fallout from automated data loads.  Typically, this process includes identifying the data that will not load, analyzing the error messages to determine the root cause, formatting a readable report that can be used as a tool in the cleanup process, and fixing the root cause of the problem so that it does not happen again.

Why the data will not load correctly.

There is a litany of reasons why some data records will load correctly while others will not.  Here is a list of some common root causes:

1. Poor quality legacy data. Legacy systems which are not as tightly integrated as SAP, and are not under master data control allow the end user a bit of freedom when entering data.  A zip code may contain too little or too many characters; the email address is not properly formatted; numeric fields have transposed digits; various forms of abbreviations (especially in the city field), a quantity of zero (0) permitted by the legacy system and uploaded into a field where SAP will not accept a quantity of 0 and even simple misspellings  all can cause stringent validation checks to trigger an error and prevent the record from loading at all.  A more sinister type of error occurs when the data is functionally incorrect, but good enough to pass all of the SAP validity checks.  In this case, the data record will technically load into SAP, but will not be functionally correct.  Duplicate customers, duplicate vendors, and the data entry error for a quantity of 1000 instead of 100, and the wrong pricing condition applied to a sales order line are examples of this scenario.
2. Functional configuration and supporting data effects. Many times I have watched the load statistics for a data object plummet from near 100% in the cycle two test load to near 0% in the cycle three test load.  This is very unnerving to the client because the cycle three test load is getting rather close to the go-live date, and “by the way, shouldn’t the statistics be getting better rather than worse?”  Functional configuration changes can wreak havoc on any data load.  Flipping the switch on a data field from optional to required; turning on batch management or serialization for materials for the first time; changes in the handling of tax, tax codes, and tax jurisdiction codes; that account determination entry that is missing or not set up correctly; a missing unit of measure or unit or measure conversion factor; the storage location in the upload file which does not exist in SAP – any of these can cause a load to drop mostly or completely onto the floor.

   While change is inevitable on any project, it is important to control and communicate the change so that the downstream impact can be recognized and understood.   Controlled change and communication always works better than total surprise.  Perhaps if we all know ahead of time about that data field that is now required, we can impose a requirement on the data extract side to make sure that the data field is populated before it enters the upload file.

3. Additional data in the upload file. Inserting a new field in the middle of the upload file data structure might be necessary for the business to close a gap, but if that change is not communicated to the technical team so that appropriate adjustments can be made to the load object’s input structures and processing logic, the new data will surely never load, and may cause misalignment of the data fields which follow it in the upload structure.

The Finger Pointing Game

It’s the load program!  No, it’s the data!  No, it’s the configuration!  No, it’s … (fill in your favorite finger pointing game excuse explaining why data will not load).

If you ever find yourself in the midst of this type of finger-pointing game, immediately stop the madness.  For this and similar situations, I apply a simple litmus test which has never failed me yet – manually enter the EXACT upload data into SAP for the transaction which has failed.  If one can be entered manually, then the program will be able to automatically load thousands with similar upload data and functional configuration.

I have lead many a functional analyst – kicking, screaming, and ranting about how terrible the load program is – to the terminal to play “let’s enter one manually”.  Typically, the result is that one cannot be entered manually due to configuration issues or the lack of supporting values data.  Once these issues are cleaned up, the load program “magically” begins to process thousands of records with no trouble at all.

Sometimes, the load object appears to be the cause, but is not the root cause.  The important data item not being handled by the load object (which was not called out in the functional specification document), the data item which turns blue (because the functional specification document explicitly stated “put this data item into the blue category”), the formula which is not calculating the desired result (but is indeed the exact formula found in the functional specification document) – all are examples of the load object adhering accurately to an incorrect functional specification document.  The root cause, then, is the functional specification document which must first be revised and checked into the controlled document repository before making any code modifications.

It’s the Program

Well, OK, sometimes the load object is at fault.  But it is extremely rare.

Collecting and Reporting the Technical Load Statistics

Load statistics are an important technical metric, indicating what percentage of the upload file has successfully posted a transaction into SAP.  This is a basic and simple record count check.  How many records were presented in the upload file, how many records posted successfully to SAP, and how many records failed to post.  Also, does the number of successful transaction plus the number of failed transactions equal the total number of records presented in the upload file.

Here is how I communicate this technical metric:

Total records in the upload file	100
Successful technical transactions	90	90% success
Failed technical transactions	10	10% fail

This technical metric indicates only that all of the SAP validation rules for posting the transaction have passed.  It does not indicate that the master or transactional data posted to SAP is actually correct in functional terms.

The Functional Review

It is very possible for an entire upload file to technically load at 100%; while at the same time, functionally fail at 100%.  The customer may be missing a partner, the material or article may be categorized incorrectly, the pricing conditions on a sales order may not have the desired validity date range, the GL posting may be to the wrong accounts.  Hence, the need a functional review and validation of the data transacted into SAP.  The data type – master or transactional – determines the type of functional review and metrics to be employed here.

For master data, such as of materials, customers, vendors, etc., it is impossible to individually validate the many thousands of entries in your SAP system.  But statistical methods can be employed which will guide you through a random sampling of the data, while at the same time assuring accuracy at a high degree of confidence levels.

For transactional data, such as inventory, open sales orders, open AR, etc., direct mathematical comparisons can be employed.  On a grand scale, if an inventory value of $15,246,321.44 is being moved from your legacy system to SAP, then that exact amount must arrive in SAP when the migration task is complete.  It may be a bit more difficult to do this mathematical comparison at a more granular level.   If, in the move to SAP you are also redesigning your material/storage location combinations, a direct comparison between the legacy system and SAP may not be possible without a translation factor.  The same scenario exists if you redesign your GL chart of accounts, where one legacy GL account now maps to several SAP GL accounts, or vice versa.

Reporting the Fallout

If an automated data load was not technically 100% successful, a clear set of error messages complete with a link back to the legacy data must be mined, formatted, and presented to the business for analysis.  Such a report really helps to facilitate the fallout cleanup.  The link back to the legacy data must be carefully designed into the load process to make sure that, for example, the legacy customer number, legacy vendor number, legacy sales order number, legacy purchase order number, etc. is included as part of the data being handled.  Without the link back to the legacy data, it becomes very difficult to identify which data record needs to be fixed.

The error message mining technique that I use depends on the load method.  I will describe two here – one for a BDC load method and one for an IDOC load method.

Mining Meaningful Error Messages from a BDC Session Log

At the completion of a batch input session, the batch input session overview screen (SM35) displays the technical load statistics.  In this example, out of a total of 229 transactions, 13 failed and 216 succeeded.

The session log shows the status of all 229 transactions.  This screen snapshot is a fragment of the complete session log for the batch input session.  It shows many successful transactions (Type = S) and one failed transaction (Type = E).  The error message here is clear – the article does not exist or is not activated.

But as you can imagine, the 13 failed transactions with error type = E are sprinkled throughout the many pages of this log file.  With only 229 transactions, this log file is quite easy to pick through to find the 13 errors.  But imagine if the number of transactions were in the thousands or tens of thousands.  How do we extract only the failed transactions and present a concise report of the failed transactions?

To do this, I use SAP transaction SM35P – Batch Input Log Overview.  This transaction has the ability to set a filter on any field in the batch input log file, display the filtered results, and then to export the results to a local file.

To enter the mode where this is possible, first press the PRINT icon.

Next, set the filter.  The appropriate filter field here is SESS. TYPE.

We only want the errors, so set the filter for field SESS. TYPE = E.

The display now shows only the 13 rows containing the error messages.  This can be exported directly to a local spreadsheet for further analysis.

Mining Meaningful Error Messages from IDOCs

Depending on the IDOC and the processing module, mining the error status messages from IDOCs can be very easy or somewhat challenging.  For the more difficult scenarios, you probably will need to hone your EXCEL skills to properly join several extracts together into one complete picture.

When creating IDOCs with a load object, I always note the date, time, and IDOC basic type.  I will use this information as the selection criteria for transaction WE05, which is going to locate the IDOCs and display the results information I need after the IDOCs are processed.  The results that I usually collect are the error status messages, and some data content from a segment or two to illustrate exactly where the problem is in the legacy data.

While the background job is busy processing the IDOCs, I usually take a peek, using transaction WE05, to see how the load is progressing.  If I see that most of the IDOCs are falling onto the floor (IDOC status 51) rather than moving into the database (IDOC status 53), I usually stop the background job and begin an immediate analysis of the fallout.  If the fallout solution does not require a change to the IDOC data content (e.g. a configuration change), then I can replay the fallout using SAP transaction BD87.  If the fallout solution does require a change to the IDOC data content, then the complete set of IDOCs must be regenerated again.

Let’s see what WE05 can tell us about a completed Article Master load.

On the WE05 screen below, we can see that of the total of 14,005 IDOCs, 11,412 have processed successfully and 2,593 have failed to process.  By double clicking the Status 51 folder, the display will show only the Status 51 IDOCs – the fallout.

By pressing the “status list” icon (shown above), the display will show the status messages for the fallout.    Once these messages are displayed, pressing the “export” icon allows me to save the screen contents to a spreadsheet.

Now it would be really nice if I could have the article number in the spreadsheet right next to the error message.  The article number in the ARTMAS IDOC is stored in segment E1BPE1MATHEAD.  The segment content for each IDOC can be displayed by pressing the “list specific segment” icon and entering the segment name in the box.

The segment display will show all fields in the segment, so I usually hide all of the columns that I don’t want to see.  Here is the E1BPE1MATHEAD segment display showing only the article number.  I can use the export icon to save the list of article numbers to another spreadsheet.

Here is a portion of the complete spreadsheet showing the error messages and the article numbers side by side.

A filter applied to the spreadsheet shows that the 2,593 errors are all grouped into one of three error status categories.  By selecting a single category, Excel will also show me the number of records within that failure category.

Sometimes it is easier to mine the status messages directly from the IDOC status table EDIDS.  This is especially true where the processing module is a BAPI which returns an error table rather than a single error message.  In this case, when you press the “status list” icon in WE05, only the first error status message of several is displayed for each IDOC.  I find that the first message is not very helpful (as shown below).  I also find that typically the second or third message in the return status table is usually the important one.  You won’t see it displayed on the WE05 screen, but you can mine it from the EDIDS table.

SAP transactions SE11 or SE16 both support this activity.  For the selection criteria I use the IDOC number range, status 51, and status type E.  On the display screen, choose only the relevant fields for display – the IDOC number (DOCNUM), IDOC status (STATUS), status message (STATXT), the four substitution parameters for the status message (STAPA1, STAPA2, STAPA3, STAPA4) and the message type (STATYP).  All of this can be exported into a spreadsheet.  If you really want to test your Excel skills, you can write code that will move the substitution parameters into their placeholders in the status text.

Preparing for the next data migration cycle – Let the fallout analysis and cleanup begin.

Presenting the fallout report to the business with a set of clear error reasons and links back to the legacy data is key to enabling the legacy data cleanup process to proceed.  In the iterative process of data migration cycles, cleansing the legacy data is a step in the right direction towards an improved next conversion cycle.

I hope you enjoyed this blog series on data migration.  Please feel free to send comments or questions.

SAP Solution Manager Service Desk Integration

Nowadays when you install SAP ECC 6.0 you get SAP Solution Manager (SOLMAN) as part of the deal – ostensibly for free (although it is really included in the purchase price).  SOLMAN provides a wealth of functionality to help manage the technical environment as well as project processes like testing.

Service Desk functionality is delivered to you for use as a ticketing system.  One of the features of it is that it can be used as a ticketing system for both SAP and non-SAP systems as well as in conjunction with other ticketing systems that may be in place already.  In this blog post I’ll briefly touch on some of the scenarios I have encountered and show that there are several ways to deploy Service Desk.

Using Service Desk is beneficial because it can automatically capture a wealth of information about what a user was doing when a problem occurred if the ticket is created directly from SAP.  Also, Service Desk can communicate directly with the SAP mother ship to log issues and manage OSS notes which obviously reduces the risk of transcription errors.  And Service Desk can be extended to include functional components from non-SAP systems which in turn leads to the possibility of one-stop-shopping for ticket management.

Here are some SAP Service Desk scenarios I’ve come across.

New SAP Installation

If you are a new SAP installation and have no ticketing in place Service Desk is a great place to start.  As mentioned the functionality available allows end users to create tickets directly from SAP and have workflow set up to manage ownership and eventual resolution.  Alternatively, a URL can be made available to allow trouble tickets to be logged via the SAP Service Desk web front end.

In this case end users and technical support personnel learn a new tool as part of their go-live experience.

Migrating Non-SAP System Support to SAP Service Desk

Rather than SAP being your sole IT application it is much more likely that you have an existing portfolio of systems and an associated ticketing tool and procedures for ticket management, escalation and resolution.  A conservative approach when bringing up Service Desk is to use it initially only for SAP related tickets and deploy a limited set of functionality.  This allows you to go through the growing pains and adjustments as you learn how to best make Service Desk work for you.

In the longer term migrating ticketing for non-SAP systems to Service Desk brings it all into one system.  Using the Service Desk capability to create custom components and to build an organization structure that fully supports your users and systems facilitates consolidated reporting and the ability to monitor Service Level Agreements (SLA).

This approach works best if you have a fairly simple environment where the level of disruption is limited when decommissioning an existing ticketing.  The impact on people, process and technology is carefully managed and the working environment has some tolerance for the time if and when things aren’t quite as expected at first.

SAP Service Desk Integration with an Existing Ticketing System

In situations where there is a well established ticketing system it is unlikely that that system will be easily displaced by SAP Service Desk and it may not make sense to do so.  In this situation you can choose to not use Service Desk and forgo the system capabilities and instead use the existing ticketing tool to manage SAP issues.  This gives up a lot of delivered SAP Service Desk capability.

Alternatively you can integrate the existing ticketing system and Service Desk and reap the benefits of both worlds.  In this case you need to decide which system is the primary ticketing system and which is the secondary.  DataXstream has worked with CA (formerly known as Computer Associates) to integrate the CA Service Desk offering with SAP Service Desk and now CA delivers a packaged solution for this integration.

SAP Service Desk is the Ticket Initiator

In the scenario where the SAP Service Desk (SAP SD) is primary tickets are initiated in SAP SD and categorized either as SAP related or as non-SAP related.

SAP related tickets are managed to resolution in SAP SD, including any communication with the SAP mother ship, and the secondary ticketing system, e.g. CASD, receives only informational updates on ticket progress, if required.  The complete ticket lifecycle is managed and closed in SAP SD.

Non-SAP related tickets are transferred to the secondary ticketing system, e.g. CASD, where they are worked to their resolution.  In this case ticket status changes and informational updates all occur in CASD and these updates are relayed to SAP SD.

In either case SAP SD and CASD can have a full picture of the ticket status throughout its lifecycle, the main difference being which system “owns” the ticket and has responsibility for working it and closing it out.

Overall the established ticketing system continues its original function and ticket resolution process, the key change being where a ticket is initially created.

SAP Service Desk is NOT the Ticket Initiator

In the scenario where SAP SD is the secondary ticketing system SAP-related incidents originate in the primary ticketing system, e.g. CASD, and are relayed to SAP SD.  Now the SAP related incidents are worked in SAP SD where ticket information can be supplemented with all the SAP unique information that would be useful for analysis and investigation.  Similar to the earlier scenario any significant status or informational updates in SAP SD are relayed to CASD and the ticket is ultimately closed out in SAP SD.

Any non-SAP related tickets are not sent from CASD, for example, to SAP SD.

A Natural Progression: from Island to Integration

SAP Service Desk can be deployed in a variety of ways in an organization: it can be a standalone ticketing and support system; it can be fully integrated with existing support applications.  The role played and the pace of adoption is very flexible and is driven by an organization’s ability to manage the change and adopt new tools and procedures.

The general progression I have seen is to bring up SAP Service Desk as a standalone tool and then after a settling in period one of two things happens:

• Existing ticketing tools and functionality are migrated into Service Desk
• SAP Service Desk is integrated with another ticket management tool

In either case the outcome usually allows SAP related tickets to be created directly from SAP in order to capture as much detail as possible at the time of the incident.

SAP PI CTS+: Letting CTS+ Out of the Bag to Get Better Change Management

Anybody familiar with older versions of SAP XI/PI understand that transporting interface development and configuration changes is often a prickly situation.  Standard change management in PI relies on the manual packaging and processing of changes into files.  These files have many issues:

• No documentation
• A different means of transport than standard SAP transports (need some training people used to ABAP transports)
• Manual audit accountability (what do you do if you lose an exported PI .tpz file)

To help resolve these issues, SAP released CTS+.  But, what is CTS+ and how can it help?

What is CTS+?

CTS (with out the plus) is stands for SAP’s Change and Transport System.  This is the transport capability of the SAP NetWeaver AS ABAP.   SAP has enhanced CTS with the capability to transport non-ABAP objects–hence CTS+.  Change management on SAP NetWeaver AS ABAP has been a strength of SAP Netweaver for a very long time. In SAP PI, CTS+ leverages this strength by allowing Enterprise Service Registry (ESR) development objects and Integration Directory (ID) objects on the Java stack to be exported and packaged into standard transports that reside on the ABAP stack.

How Can CTS+ Help?

Documentation

With CTS+ you have the ability to document your transport when you create your request. This is amazingly helpful when it’s time to go to production, as you can see what changes were imported and when they were imported. Documentation is also helpful for the Integration Directory (ID) transports, which require activation and input of values.  To get to transport organizer, export what you want to export (namespace, scenario, etc.) using CTS+ (which should be default once it is set up).  Click on create/modification of request and you will see a screen like this:

Here you can put in details of the transport in the description as well as configure various other parameters about the transport.

Additionally, if your project has transport naming standards that require RICEF ID, defect numbers, change request number, trouble ticket number, etc., you can add that information here.

Standardization of Transports

With CTS+ it is possible to configure a standard way of handling transports that is consistent with your ECC environment. This will improve the overall reliability and robustness of environment change management, and will make life a lot easier for the people responsible for handling transports.  Someone accustomed to ABAP transports in ECC but who hasn’t worked with PI can jump in and change management with CTS+ will make sense. CTS+ also allows SAP PI to be compliant with your standard SAP change management governance.

For More Information

CTS+ is available with AS ABAP and AS Java from SAP NetWeaver SPS 12 onwards, although you will have better results with SPS 14 or higher. The following OSS Notes will be helpful:

• 1003674 Enhancement for non-ABAP systems in CTS
• 1145268 CTS+: Changes from NW 7.0 SP12 -> NW 7.0 SP13
• 1146170 CTS+: Changes from NW 7.0 SP13 -> NW 7.0 SP14

In short, CTS+ is a great tool for transport management. It is totally integrated with SAP PI, it is easy to set up, and has huge advantages when it comes to documenting transports and maintaining consistency with ECC standards.

SAP Data Migration – The Data Migration Plan (Part 2)

If you are responsible for the success of data migration, you will want to build a detailed plan that will walk you through all of the three phases of data migration: pre-data migration preparation, the data migration itself, and post-data migration cleanup.  I like my data migration plan to contain detailed steps that ensure that I don’t forget anything.  Each step lists a specific named responsible person along with their area of responsibility and contact information.  Unless I am responsible for executing the task myself, I prefer the named person to be a client employee (i.e. the business owner of the process) rather than a project consultant.    This is where the responsibility should be, and it requires that the business process owners actually participate in the activity rather than sit on the sidelines and watch.

Before you embark on your project’s first data migration test cycle, your data migration plan will probably start out as a simple list of preparation steps, data load dependencies, and cleanup steps.  As you progress through your several data migration test cycles, your plan should evolve into a rather complex list containing load methods, responsible persons, historical data volumes, historical execution times, steps that need to be modified and additional steps that need to be added.

I always find it rather interesting to compare the very first plan to the plan that is actually used in the cutover process.  Many project factors will cause you to amend your data migration plan.  Newly-discovered gaps, changes in the business requirements, the decision to load all of the legacy customers instead of only those customers which have been active for the past two years, and toggling back and forth between “should this one be a manual or an automated load” are only a few.   Always keep in mind that the goal in amending the plan is to keep it compatible with the changing requirements of the project.

The Data Migration Preparation Phase

Before you actually start the data migration main course, you will probably want to execute a series of preparation steps.  Some of these preparation steps may have a significant lead time; while others may require a system reboot to take effect.  Make sure that you carefully plan accordingly.  Here are some data migration preparation steps that I have found useful:

• Obtain a Data Migration Login Obtain a special data migration login, with all of the authorizations needed to execute all of the required tasks.  This login is only to be used during the data migration process, and is to be deleted upon the completion of the data migration process. Using the data migration login makes identification of migrated data very easy both now and several years down the road.  It also facilitates implementing a general user lockout while the data migration user is performing data migration activities.
• Technical client preparation The BASIS team may want to configure the data migration target client to have less dialog processes and more background and update processes.  This configuration can significantly improve data migration performance, especially if you have the opportunity to parallel process IDOCs.  In a Virtual Machine environment, the infrastructure team can even provision additional processors and memory.  You may also need additional disk space to handle large upload files, and any intermediate files that a load program might need to create.
• Configuration for IDOC processing If a data migration process involves creating and processing IDOCs, you may need to configure the ports, partner profiles, message types, and processing modules.  My preference for IDOC triggering is by using the ABAP program RBDAPP01 rather than immediate triggering.  This allows me to exercise the most scheduling control over the background processing options.
• Change Pointers During a data migration turn off change pointers.  Creating what could potentially be hundreds of thousands of change pointer records that are never going to be processed is an overhead burden on system performance and disk space that you do not need.  If the migrated data needs to be sent to external systems, manually send the data after the data migration is complete.
• Set the Correct Open Posting Period Some of the transactional data will want to post into an open financial or manufacturing posting period.  You will want someone from the business to make sure that the correct posting periods are open for business.
• Configuration Settings Sometimes, special temporary functional configuration settings are made to facilitate the data migration process.  Several functional analysts may be responsible identifying and setting these prior to the start of data migration.  As these are temporary configuration settings, it is expected that they will be set to their production values after the data migration is complete.
• Basic Setup Data Some data migration objects are dependent on reference data – a template from which new data migration objects can obtain some of their default values.  You will want to make sure that those who are responsible for setting up this reference data have completed their task before you start the actual data migration.  Sometimes this data is moved from the Golden Client into the target client via ALE, so you may need to set up RFC destinations, ports, distribution models, etc.
• General user lockout All users except for the data migration user should be locked out of the data migration client during the actual data migration activity.  It is impossible to verify that we moved exactly $15,386,254.23 in inventory from the source system to the target system if users are performing material movements in either system during the conversion.
• Connections to External Systems Depending on what connections to external systems are doing, you may want to make sure that they are disconnected prior to the start of data migration.  This will ensure that inbound interfaces, which may either modify existing migrated data or add new data, are disabled.

The Data Migration Phase

When all of the preparation steps are complete, it is time to start the actual data migration.  Data must be loaded into the target system in a certain sequence that is dictated by their data dependencies.  Customers must exist before sales orders can be migrated, vendors must exist before purchase orders, materials must exist before either sales orders or purchase orders, etc.  This dependency exercise varies a bit from project to project, and should have already been determined well before you start loading the data.

Automated data migration tasks for a single object (e.g. customer master data) usually include some or all of the following:

1. Creation of an upload file from the source system
2. Technical validation of the upload file
3. Functional validation of the upload file
4. Execution of a technical object to load the data into the target system
5. Technical validation of the migrated data
6. Functional validation of the migrated data
7. Collection and reporting of load statistics
8. Analyzing the fallout
9. Correcting the fallout
10. Reprocessing  the fallout

Checkpoints

At a higher level, the data migration plan will string together an end to end list of all of data migration objects.  I strongly suggest that you study the list and judiciously intersperse several checkpoints at appropriate places in the plan.  A checkpoint is an activity where all interested parties meet to review status and metrics, and to determine whether or not to proceed forward.  Keep in mind that what fails to load early in the plan will geometrically cascade into larger fallout later in the plan.  That single vendor which failed to load may cause several hundred open purchase orders to also fail.  And while the business owner of the vendor master data may think the fallout of a single vendor to be quite insignificant, the business owner of open purchase orders may have an entirely different perspective.  It is important here to make sure that all of the right stakeholders are participating, and that all are in agreement about whether or not to proceed.

Backup or Restore Points

Another activity that you should judiciously sprinkle into the plan is several backup or restore points.  It is very painful to have successfully navigated seventy-five percent of the data migration plan, only to have the next load create a huge and unrecoverable mess in the target system.  Without a backup or restore point available, you may have to wipe the slate clean and restart the entire data migration process again.

Depending on the target system and the facilities available, the backup or restore point could be a full system backup, a client copy, or a virtual machine snapshot.  Whatever the means, make sure it is a part of your plan.  It’s really great if you don’t need it.  It’s also really great to have if you do need it!

The Data Migration Cleanup Phase

When the data migration loads are finally complete, it’s time to exercise a series of cleanup steps.  This is where we:

1. Undo the special temporary settings and configurations that were done prior to the start of the data migration activity.
2. Adjust the technical configuration of the system to a more user-oriented system with the right amount of dialog, background, and update processes.
3. Turn change pointers back on.
4. Allow the general user population to access the system.
5. Connect the system to its inbound and outbound communication partners.
6. Disable the data migration user.
7. Archive those massive upload files.
8. Delete any intermediate processing data files.
9. Clean up the IDOC tables if needed.  But first make sure that you do not need to keep the IDOCs  for legal reasons.

An Example Data Migration Plan

Click on the link below to access an example data migration plan.  This plan is not a complete data migration plan, but is intended to demonstrate format and content. Sample Data Migration Plan

Stay tuned!

In part 3 of this series on Data Migration, I will be discussing how to deal with fallout from automated data loads.

Coming Soon: SAP .NET Connector (NCo) 3.0

SAP is announcing a new version of SAP Connector for Microsoft .NET 3.0 (now called “NCo 3.0″). A beta program for selected customers and partners is currently underway (Q3, 2010) with the general release of the software coming soon thereafter.  I will highlight some of the major differences between the SAP Connector for Microsoft .NET 2.0 and NCo 3.0 (besides the obvious, and much-needed name-shortening).

NCo 3.0 Logon

One of the new features implemented to combat the malevolent for RFC client attacks is the interface IDestinationConfiguration. This interface contains method IDestinationConfiguration.GetParameters(string destinationName) which allows .NET programmers to provide their own secure method for storing and retrieving logon information, be it encrypted database, encrypted file, or even LDAP.  There is also a corresponding interface and method to secure NCo 3.0 RFC servers.

Additionally, there is updated functionality to support single sign-on (SSO) and X.509 certificates.

Say Goodbye to Data Containers and Generated Code

In SAP Connector 2.0, working with RFCs required the generation of proxy code via the SAP .NET Connector design-time tool.  This tool would convert the IMPORTING, EXPORTING, CHANGING, and TABLES parameters from the RFC you were calling/serving to .NET representations of the same.  With NCo 3.0 there is no longer any kind of generated code.  Instead of one generated proxy method for each function module, there is one single RfcFunction class, whose Invoke() method dynamically executes every given ABAP function module. Additionally, all ABAP parameters will be represented by the class RfcStructure instead of a dedicated generated class for every structure; All tables will be represented by class RfcTable.  So, instead of hard-coding all of the data and variable bindings statically at design time, NCo 3.0 now handles everything dynamically at runtime.

NCo 3.0 uses SAP’s data dictionary to determine the function interface for the called RFCs.  While this data is cached locally for performance reasons, NCo 3.0 is robust enough to detect changes in RFC interface signatures so your code will still execute if you’ve added, deleted, or changed RFC parameters.  If there arises a situation where the dynamic lookup of RFC interface signature is not wanted, there is also an option to hard code the RFC parameter metadata.

The biggest benefit of NCo 3.0′s new dynamic function handling is that NCo 3.0 no longer has a design-time component.  This releases the Visual Studio version restriction that .NET 2.0 had.  Let me rephrase that, NCo 3.0 will work with any version of Visual Studio you like! The only requirement is that at runtime, a .NET Framework version is installed that is compatible with the NCo 3.0 libraries.

Anxiously Waiting

NCo 3.0 figures to be a much-needed update to a vital SAP integration technology.  I, for one, cannot wait for the new functionality to be generally released.  Once NCo 3.0 is released we’ll be posting blogs highlighting the new features.  We’ll also include helpful information regarding a .NET Connector 2.0 to NCo 3.0 upgrade.  In the meantime, if you have any leave any questions or comments you have regarding NCo 3.0 below.  You may also email me directly at cstasila@dataxstream.com if you’d like to set up a detailed conversation.

Next Page »