For the element that’s text cannot be hardcoded or the element that’s visibility needs to change during runtime, the context needs to be used. The data that is seen on the screen is not stored within the web browser itself. The Web Dynpro application makes calls to the server to retrieve data to be displayed. This is where the context comes into play. The context is the memory on the server that stores data for the Web Dynpro application. When setup correctly a context can hold tables, structures or single element variables. Here is an example of how to build a context and then make method calls to read or set the context.

Select the ‘Context’ tab. Right click on the context node and select ‘Create’ -> ‘Node’ and fill out the fields according to the picture below.

Right click on the BUTTON_PROPERTIES node and select ‘Create’ -> ‘Attribute.’

Now we have created an attribute in the context that will hold a string value. By the names I have given to the node and the attribute you can guess that this node will hold any attributes related to the button we created. The attribute that we created will be used to hold text value of the button.

Now that the context attribute has been created, let’s fill it with a value. Go back to the button properties under the ‘Layout’ tab and find the ‘onAction’ properties under ‘Events.’ Click the create button, it looks like a white piece of paper with the top right edge folded down, and fill in the fields as below.

Double click on the name of the new action we just created, SHOW_TEXT and this will take you to the methods tab. Here we are going to use the Web Dynpro Code Wizard to set the value of the attribute.
Click on the Web Dynpro Code Wizard icon (or Ctrl +F7), select ‘Read’, click and select the attribute that was created, TEXT. (Illustrated Below)

Now repeat the same steps starting with the Web Dynpro Code Wizard except this time select ‘Set’ instead of ‘Read.’ Each time the Web Dynpro Code Wizard is used DATA variables are created.

Note: If there are multiple attributes within a node, all of the attributes can be read or set at once by selecting the node.

After using the Web Dynpro Code Wizard the second time we now have two copies of the data variables that will not compile. Select the second set and delete them and in place put the following code:

Now if the application was tested the value of the attribute TEXT would be changed  between ‘Hide Text’ and ‘Show Text’ every time the button is pushed.

That is the rundown of how to build, read and set the context’s nodes and attributes. Now the context needs to be linked or ‘bound’ to the UI elements. Go back to the Layout tab and select the button. Under the properties find the ‘Text’ property. Click the yellow box to the right of the property and select the attribute to create the binding .

Binding

Binding is linking the data in the context to the UI elements. Here we have created an attribute that holds text. The ‘Text’ property of the button is set to the value within that property. Activate the Web Dynpro Component, create an application and test it.

For a little extra practice, add a TextView by dragging and dropping into the layout or right clicking on ROOTUIELEMENTCONTAINER and selecting ‘Insert Element.’ In the properties section set the Text value to ‘Now you see me!’ Create another node and attribute for the TextView by going to the ‘Context’ tab and right clicking on the ‘Context’ node and selecting ‘Create’ -> ‘Node’. Then create the attribute with the properties below.

Go to the ‘Methods’ tab and go into the method we was worked on earlier, ONACTIONSHOW_TEXT. Place the cursor after the ‘if’ statement that was put in, approximately line 33. Use the code wizard to set the new context we made. Then cut and pasted the DATA declarations just created and move them up to the top of the code. Then edit the ‘if’ statement to look like this:

Return to the ‘Layout’ tab, select the TextView and set the ‘Visible’ property to the new attribute. Activate the Web Dynpro Component and test the application.

Discovering Direct Modifications of SAP Standard Objects – One Example

As I discussed in a previous post on this topic, direct modification of SAP standard objects within the SAP namespace carries a high risk in an upgrade project.  It would be very valuable to be able to identify some details about these objects, so that further analysis could be performed to determine their disposition in the upgrade.

Modification of standard SAP objects typically requires that a modification registration key be obtained for that object from the SAP SSCR facility.   The SSCR facility (SAP Software Change Registration) is a procedure which registers all manual changes to SAP sources and SAP Dictionary Objects.  With a valid and authorized OSS ID for an SAP system, you can go to the SAP Support Portal on the web and view the SAP objects that were registered for modification by that OSS ID.  If there are multiple OSS IDs for any SAP system, make sure that you check all of them to capture all objects registered by all of the authorized OSS IDs for an SAP system.  Press the “Objects Registered by Me” button to view a list of the registered objects.

Another place that you can look for this information is directly within SAP.  Table ADIRACCESS stores the registration keys which were obtained from SAP, and which were entered by the developer when the object was to be modified for the very first time.  This entry shown below from table ADIRACCESS shows that function group SCPRPS was registered with SAP for modification.

Registering an object for modification does not mean that the object was actually modified.
What additional evidence can we discover that would show that the object was, in fact, modified?   And, since a function group contains many objects, can we determine exactly which object or objects were modified?

With a little bit of forensic analysis, we can discover the specific object within the function group that was modified, the user name of the programmer who last modified the object, date of the last modification, and the line-by-line source code differences.

If you are familiar with the structure of function groups, you already know that they may contain many function modules, and that the source code for each function module resides within its own include file.  At the moment, we know that the function group name is SCPRPS, but we do not have the list of function modules within that function group.

To obtain that list, we can use the SAP function module FUNCTION_SELECT_TFDIR.
The result list is shown here:

So which, if any, of these has been modified?  To determine this, we need to find all of the rows in table TRDIR where the NAME field is equal to the PNAME entries listed above.

Here is an excerpt from SAP table TRDIR for SAP object names that begin with the characters LSCPRPSU:

Note that for all of these objects, which are not in the customer namespace, the Author Username field (TRDIR_CNAM) is ‘SAP’.  Also note that for most of the entries listed here, the Last Changed By user name field (TRDIR_UNAM) is also ‘SAP’, except for one row, where the Last Changed By username field is not ‘SAP’.  From this row, we can see that include file LSCPRPSU36 was last modified by user DXSDEV on 10/19/2009.

If we look back at the result list of FUNCTION_SELECT_TFDIR and find the row where the field PNAME is equal to LSCPRPSU36, we can see that the name of the modified function module is SCPR_PRSET_CT_IMPORT_INDUSTRY.

Exactly What Changes Were Implemented?

By going to the Version Management Utility for this function module, we can see what the various versions of the code contained.  The Version Management Utility will also show us the transport request numbers, date and time, and username for each version of the code.  The Version Management Utility also has a code comparison tool which will highlight the line-by-line differences between versions of the source code.

Here is an excerpt from the side-by-side source code comparison utility which shows the modified version on the left side and the original version on the right side.  Fortunately, in this instance, the programmer left excellent notes indicating that a correction instruction from an OSS note was being applied.

What Are the Discovery Possibilities?

The manual example described above showed how to discover and analyze customizations that had been applied to a single SAP standard object.  But what if the customizations number in the hundreds or even thousands?  What about all of the customizations that were built entirely within the customer namespace?

It is possible to discover hundreds or even thousands of customizations regardless of which namespace they were developed in.  The key is automation, understanding that this information already resides within SAP waiting to be discovered, and more importantly, understanding the meaning of the discovered information.  I have already developed an ABAP discovery program which will mine, analyze, and report data about the following customized objects:

More Discovery – Usage Statistics and Where-Used

Customization Usage Statistics

The goal of customization usage statistics analysis is to discover the usage frequency of customized objects.  The usage discovery may show usage anywhere from several times daily by key users, to none when analyzed over a long time period.   Since most companies really do not want to incur the cost and the risk of moving an unneeded customization into the upgrade system, objects which show no usage over long periods of usage history become targets.  Care must be used here, as some transactions are exercised only quarterly, semi-annually, or annually.  I always recommend letting the business process owner make the final decision to eliminate any customization object in the upgrade.

SAP keeps track of usage statistics, and this data is available for you to analyze.  You will need to check what facility is available for your particular SAP system, as the transaction codes and programs differ among various versions of SAP.   The amount of data stored within SAP is configurable, and is usually set to a small date range.  Some companies that want to analyze the statistics over longer time periods actually download and store the statistics data in their own external databases.

Customization Where-Used

Like usage statistics, the goal of customization where-used analysis is to discover customized objects that are orphan or not being used.  What about that custom data type that is not used in any table or structure?  What about that function module that shows an empty list when you click the where-used icon in the function builder?

To properly perform a where-used analysis, it is important to understand all of the various possible uses for an ABAP object.  For example, here is the selection screen of possible uses that is presented by the ABAP Dictionary when you click the where-used icon to determine the where-used list for a data table:

And what about that custom function module, which I alluded to earlier, which shows a completely empty where-used list?  Perhaps it is a remote-enabled function module that was installed by, and is currently being called by an external third-party package.   Or, perhaps it is linked to an SAP EDI inbound or outbound process code.  It is really important to be able to properly interpret the data being presented and to not draw any hasty conclusions.

ABAP Programs Where-Used Analysis Example

In the ABAP workbench, SAP provides a where-used facility, which reports the various areas where a single selected object is used.  For a discovery project, you may want to automate this facility to analyze groups of ABAP objects (e.g. all custom ABAP programs, all custom tables, etc.).

Here is an ABAP programs where-used analysis example to help illustrate how a where-used analysis might be utilized.

Your automated where-used analysis program to analyze custom ABAP programs reports fifteen custom programs which show no usage in any of the following areas:

A quick analysis shows that all of these fifteen programs are simple reports.  Since users typically execute their reports via transaction codes, you wonder how these fifteen programs could ever possibly be executed without transaction codes.

You further analyze the SAP job scheduler data as far back as possible and find that only ten of these ABAP programs were ever scheduled in a batch job.  Some of these were scheduled very recently, and some have not been scheduled for a long time (another thread of evidence that we will need to pursue).  This leaves five ABAP programs with no apparent means of being executed.

As further evidence, you perform a cross-check against the usage statistics report and find that two of these five ABAP programs show no usage.

Armed with this analysis, you are now ready to meet with the business process owners to discuss the disposition of the objects on your short list.

A salient point here is that you will need to gather your evidence from several different analysis sources to help paint the most accurate picture about the possible disposition of customized objects.

Summary

Throughout all three parts of this post, I have stressed that customizations, while necessary, add risk and cost to an upgrade project.  And customizations that are not necessary or built poorly without the constraints of best-practice controls can add high levels of risk and cost.  Here are some general guidelines to follow:

Many companies and their auditors consider a well-controlled and secure ERP system to be a fundamental component that is vital to the success of the business.   They recognize that an out-of-control ERP system can easily paralyze a company for extended periods of time.  With this in mind, implement and enforce best-practice controls and standards to make sure that future customizations are reviewed, approved, built, tested, documented, and implemented properly.  This may already be an internal or external audit requirement, so check with your inside and outside auditors to make sure that your controls meet the auditing requirements.

Periodically review, audit, and amend your best-practice controls to make sure that they are still effective in supporting the needs of the business; and that both internal staff and external consultants are in compliance of these controls.

Know your customizations.  Know where they are, why they are, and what they do.  Whenever feasible, it is most efficient to eliminate ABAP object customizations, especially any that were performed within the SAP namespace, and any that are not really needed because they can be implemented within standard SAP.

The degree of risk and cost depends on the specific type of customization that was performed.  Establish a formal review board to own the customization approval process.  Understand that approving a customization also means approving the risk and the cost burdens.  Some companies consider this to be so significant, that they include a high-level business executive as a default member of the review board.

Set and enforce a policy making ABAP object customization the last resort, only after all other options have been thoroughly researched and exhausted.  Here are some examples:

1. Research forums, blogs, become active in user groups, etc. to see how other companies might also be implementing the business function that you require.
2. Find and use customer and user exits instead of directly modifying objects in the SAP namespace.
3. Determine, for example, how to implement and maintain that complex pricing procedure using the pricing condition tables instead of writing custom code and building custom tables.

Share Your Experience.

Once again, I look forward to hearing from our readers to reply to this post with their own experience with customizations, best-practice standard and policies, and success stories illustrating how your company transitioned to best-practice controls.

[Editor's Note] This blog is third of a multi-part series:

1. Part 1
2. Part 2
3. Part 3

In this post, we will look at some of the customization areas which add risk and the cost to an upgrade project.

1. Direct Modifications to SAP Standard Objects

Direct modifications to SAP standard objects bear the highest risk.  During the upgrade, these modifications will be lose–either because they will be overwritten by their respective upgraded SAP standard objects; or because these objects no longer exist and are not used by standard SAP in the upgrade system.

Typically, there are two reasons for performing direct modifications to SAP standard objects:

1. The application of authorized individual OSS corrections to fix problems.
2. Direct customer modification to provide business process enhancements.

1.1. Application of Individual OSS Corrections and Support Packs

Individual OSS corrections are patches supplied by SAP to fix recognized problems with SAP objects.  Sometimes, individual OSS correction instructions are included and delivered within a specific level of support packs.   That means that applying support packs to the specified level will apply the desired correction, and manual intervention is not needed.  Sometimes OSS correction instructions are not yet included in any specific level of support packs, and these will need to be applied either manually or with the SAP Note Assistant.

It is important to carefully analyze the validity range of an OSS note to determine both how it was applied in the legacy system, and how it is to be applied in the upgrade system.  This analysis will help determine how far to patch the upgrade system to make sure that the desired corrections are included in the upgrade system.   Any needed correction that exists at a patch level beyond the highest patch level in the upgrade system will need to be addressed  manually or with the SAP Note Assistant.

The decision of which support pack levels should be applied to the upgrade system must be made carefully.  One approach is to apply the highest level of support packs that are available.    The rationale for applying the latest available support packs is to insure that the most recent problem fixes are applied to the upgrade system.  This is a two-edged sword, as the most recent  support packs may also introduce other problems into the upgrade system.

1.2. Direct Customer Modification of SAP Standard Objects to Provide Business Process Enhancements

Direct modifications of SAP standard objects to provide enhancements are most costly and bear the highest level of risk.  While direct modifications of SAP standard objects may have been technically expedient at the time of implementing the enhancement, they will always require additional work effort and analysis at upgrade time to determine:

1. Whether or not the functionality provided by the enhancement is still required by the business.
2. Whether or not the functionality provided by the enhancement is already included in the upgrade system as standard SAP.  You will need the help of functional subject matter experts here.
3. Whether or not the modified SAP object still exists in the upgrade system.
4. How to move these required enhancements into the customer namespace, using the SAP preferred enhancement procedures.  Moving these types of enhancements to the customer namespace also moves them from the very costly and highest risk category to a lower cost and lower risk category.

2. Custom Interfaces

Custom interfaces are loaded with opportunity for high upgrade risk.  Take, for example, the SAP standard IDOC Basic Type, which was copied as the baseline for an enhanced customized basic type containing additional custom segments and fields.  In the upgrade system, a new version of this SAP standard IDOC Basic Type may now contain additional segments and fields.

Likewise, the associated standard processing module for that standard basic type, which was copied as the baseline for an enhanced processing module to handle the custom segments and fields of the enhanced custom basic type, may also have changed in the upgrade to accommodate the new standard IDOC version.

So, in the upgrade system, especially if you need to implement the new upgrade IDOC version, it may be necessary to start with the new standard basic type and standard processing module, and reapply the customizations.

3. ABAP Code Objects

Of all customized objects discovered in most SAP systems, ABAP Code objects are the most numerous.  These objects include classes, methods, function modules, report programs, dialog programs, and include files which support the all of the other ABAP object types.

At greatest risk are the SAP standard ABAP code objects which were modified directly in the SAP namespace; and those that were copied into the customer namespace and subsequently modified. This is because the standard SAP code objects may have changed in the upgrade.    These changes can range anywhere from simple error corrections to complete process and code redesigns.  In these instances, each modified object must be compared to its counterpart in the upgrade system to determine its existence, and the nature and extent of what might have changed.   If the customization is still needed in the upgrade system, and to stay current with the enhanced standard code, it is probably best to start with the upgrade standard code and reapply the customizations.

ABAP code objects that were developed entirely in the customer namespace (not copied from standard SAP code) are usually low on the risk ladder.  Problems occur when data selected by these objects has moved elsewhere or behaves differently in the upgrade system, when ABAP constructs used are now obsolete in the upgrade system, or when standard called functions are changed in the upgrade system.

Ultimately, the business must decide why SAP standard code was directly modified, why SAP standard code was copied into the customer namespace and modified, why custom code was developed entirely within the customer namespace, whether or not any of these modifications are needed, and how they might be implemented differently in the upgrade system.  This evaluation presents another opportunity to remove some rocks from the customization rock bag.

4. Automated Standard SAP Transactions

Custom ABAP programs which automate standard SAP transactions bear an above-average risk.  This is because the behavior of some SAP transactions may have changed in the upgrade.  These changes can be manifest in the functional performance of the transaction, the technical location and existence of data fields on screens, screen field properties, IMG configuration specific to the transaction, the transaction code associated with the transaction, or even the existence of the transaction or an equivalent in the upgrade system.  The nature and the extent of the changes to the transaction in the upgrade system will determine the amount of effort needed to automate it.

Also, if an “old” SAP transaction has been replaced by a “new” upgraded equivalent, it is usually wise to make the change to the new transaction at upgrade time.  At some point in the future, the “old” version of the transaction will become obsolete and no longer useable or supported.

5. Data Dictionary Customizations

Many customizing projects within SAP require the storage and manipulation of data that is not included in the standard SAP dictionary.   One way in which these requirements can be addressed is by building complete custom transparent tables and data structures.  Another way is by adding append structures and append fields to existing standard SAP tables.  With either approach, it is always an excellent idea to evaluate the need to move these customizations into the upgrade; and another opportunity to lighten the customizing bag of rocks.

6. Customer and User Exits

Customer and user exits are provided by SAP as the SAP-approved mechanism for enhancing standard functionality.  Enhancements performed within these boundaries are guaranteed by SAP to bear low technical risk in an upgrade situation.  While SAP guarantees that the custom code will arrive technically intact in the upgrade system, functional analysis must still be performed to determine the continued need for the enhancement in the upgrade system.

Any enhancement implemented in a customer or user exit that is either functionally covered or determined to be unnecessary in the standard upgrade system should be retired.  This is consistent with the goal of lightening the load in the bag of rocks.  Since a customer or user exits may implement several different enhancements, take extra care to make sure that only the right code or other objects are retired.

7. The Environmental Effects – Operating System Change

OK, changing the operating system during an upgrade project is not customization of SAP objects.  But it does add risk and cost to the project.

Why?

If the upgrade operating system changes from UNIX to Windows or vice versa, then any ABAP program or interface which reads or writes a data file to an operating system folder must be changed.  The changes needed to these ABAP programs are very narrow in scope,  and reflect the manner in which the operating system specifies its file paths.  Additional BASIS work, such as determining the proper file system authorizations, may also be needed.

Using the SAP logical file name facility can help enormously by removing the need to make changes to hard-coded file names and path names in many different ABAP programs.

Share Your Experience

At the top of this post, I stated that I would discuss some of the customization areas which add risk and cost to an upgrade project.  I look forward to hearing from our readers to reply to this post with their own experience with customizations, how they were handled, and how this affected an upgrade project.

In my next post on this topic, I will reveal some of the techniques I use to “discover” customizations, even in the absence of documentation.

[Editor's Note] This blog is the second of a multi-part series:

1. Part 1
2. Part 2
3. Part 3

The sample program displays details about companies defined by SAP. There are two BAPI calls involved, BAPI_COMPANY_GETLIST and BAPI_COMPANY_GETDETAIL.
Along with the SAP .Net Connector 3.0, we are using Microsoft Visual Studio 2010 and the Microsoft .Net Framework 4.0 to build our sample.  Prior to starting, you will have to download and install NCo 3.0 (OSS login required).

Setting up the Project

Using Visual Studio 2010, create a new Windows Form project. In the project properties, be sure to set the Target Framework to .Net Framework 4.0.

Add references to the SAP .Net Connector 3.0. There are two DLLs, sapnco.dll and sapnco_utils.dll.

Design the Form

Add two controls to the form. The first is a List Box, which will contain the list of available SAP companies. The second control to add is a Property Grid control, which is used to display the details of a particular SAP company when the user selects one from the List Box.

Using App.Config to define the SAP Connections

There are several methods you can use in your solutions to define a particular SAP host. For this example, we are using an app.config file to define our SAP host.
The SAP .Net Connector 3.0 includes sample app.config files. For the RFC client, be sure there is a sectionGroup definition for <ClientSettings>.
The SAP host is defined by the destinations section. Within the destination, the Name field identifies this app.config entry to our program code. Specify the appropriate user name, password, SAP host name, client and system number in the destination section.

The BAPIs

Use the SAP transaction SE37 to determine the parameters for functions BAPI_COMPANY_GETLIST and BAPI_COMPANY_GETDETAIL. This will tell us what parameters need to be passed in and where to find the results that are returned to our code.

The Code

During the form load event, we want to populate our List Box with the defined companies from SAP.

1. Acquire a valid RfcDestination to use. Use method RfcDestinationManager.GetDestination() to do this. GetDestination() requires a parameter to determine which SAP host to refer to. Pass in the Name of the SAP host you previously defined in the app.config file. (Line 60)
2. Now that we have a valid RfcDestination object, we need access to the SAP Repository. The repository contains information about the BAPI calls we are going to make. The RfcRepository is an attribute of an RfcDestination object. (Line 62)
3. Using the RfcRepository object, acquire a reference to the SAP BAPI by calling method RfcRepository.CreateFunction() method. Pass in the name of the desired function as a string parameter to CreateFunction(). This method returns an object we can use to setup parameters, invoke the function, and retrieve results.
4. Now we can make the RFC calls into SAP. Using the IRfcFunction object returned by CreateFunction(), call the Invoke() method, passing in our RfcDestination object as parameter. This will make the RFC call into SAP, and provides the results via our IRfcFunction object. (Line 68)
5. SAP function BAPI_GET_COMPANYLIST returns a table of company records. We use an instance of an IRfcTable object that we retrieve by calling the GetTable() method on the IRfcFunction object. To get a desired table, pass in the table name to the GetTable() method. (Line 70)
6. We now have an instance of an IRfcTable object that contains company information through which we can iterate. It is possible to iterate through the data by setting the CurrentIndex property on the returned table. For a particular row in the table, it is possible to access values using the GetString() method. (Lines 75-79)
7. For each company, get the CompanyNumber, and use that as a parameter to SAP function BAPI_COMPANY_GETDETAIL. BAPI_COMPANY_GETDETAIL returns detailed information about that particular SAP company. The techniques are the same as before, with the additional step of using the SetValue() method on the IRfcFunction object to pass in the company number as a parameter. (Lines 78-84)
8. BAPI_COMPANY_GETDETAIL returns data as an instance of the IRfcStructure interface. Access individual field data of the company record, using the GetString() method of the IRfcStructure object instance. (Lines 88-99)
9. Extract the company details as desired. Add them to some object you create and then add that object to the List Box control. (Line 101)

The rest of the code is regular Windows Forms programming. Your code will need to wire up the List Box changed events so that the Property Grid will display the details of the selected company.

Additional Information

Follow these links for more information about NCo 3.0 programming:

• Webinar: SAP NCo 3.0 – Best Practice Guide for Upgrade from .NET Connector 2.0
• Source Code: RFC Client with NCo 3.0

Why the data will not load correctly.

There is a litany of reasons why some data records will load correctly while others will not.  Here is a list of some common root causes:

1. Poor quality legacy data.
   Legacy systems which are not as tightly integrated as SAP, and are not under master data control allow the end user a bit of freedom when entering data.  A zip code may contain too little or too many characters; the email address is not properly formatted; numeric fields have transposed digits; various forms of abbreviations (especially in the city field), a quantity of zero (0) permitted by the legacy system and uploaded into a field where SAP will not accept a quantity of 0 and even simple misspellings  all can cause stringent validation checks to trigger an error and prevent the record from loading at all.  A more sinister type of error occurs when the data is functionally incorrect, but good enough to pass all of the SAP validity checks.  In this case, the data record will technically load into SAP, but will not be functionally correct.  Duplicate customers, duplicate vendors, and the data entry error for a quantity of 1000 instead of 100, and the wrong pricing condition applied to a sales order line are examples of this scenario.

1. Functional configuration and supporting data effects.
   Many times I have watched the load statistics for a data object plummet from near 100% in the cycle two test load to near 0% in the cycle three test load.  This is very unnerving to the client because the cycle three test load is getting rather close to the go-live date, and “by the way, shouldn’t the statistics be getting better rather than worse?”  Functional configuration changes can wreak havoc on any data load.  Flipping the switch on a data field from optional to required; turning on batch management or serialization for materials for the first time; changes in the handling of tax, tax codes, and tax jurisdiction codes; that account determination entry that is missing or not set up correctly; a missing unit of measure or unit or measure conversion factor; the storage location in the upload file which does not exist in SAP – any of these can cause a load to drop mostly or completely onto the floor.While change is inevitable on any project, it is important to control and communicate the change so that the downstream impact can be recognized and understood.   Controlled change and communication always works better than total surprise.  Perhaps if we all know ahead of time about that data field that is now required, we can impose a requirement on the data extract side to make sure that the data field is populated before it enters the upload file. 
2. Additional data in the upload file.
   Inserting a new field in the middle of the upload file data structure might be necessary for the business to close a gap, but if that change is not communicated to the technical team so that appropriate adjustments can be made to the load object’s input structures and processing logic, the new data will surely never load, and may cause misalignment of the data fields which follow it in the upload structure.

The Finger Pointing Game

It’s the load program!  No, it’s the data!  No, it’s the configuration!  No, it’s … (fill in your favorite finger pointing game excuse explaining why data will not load).

If you ever find yourself in the midst of this type of finger-pointing game, immediately stop the madness.  For this and similar situations, I apply a simple litmus test which has never failed me yet – manually enter the EXACT upload data into SAP for the transaction which has failed.  If one can be entered manually, then the program will be able to automatically load thousands with similar upload data and functional configuration.

I have lead many a functional analyst – kicking, screaming, and ranting about how terrible the load program is – to the terminal to play “let’s enter one manually”.  Typically, the result is that one cannot be entered manually due to configuration issues or the lack of supporting values data.  Once these issues are cleaned up, the load program “magically” begins to process thousands of records with no trouble at all.

Sometimes, the load object appears to be the cause, but is not the root cause.  The important data item not being handled by the load object (which was not called out in the functional specification document), the data item which turns blue (because the functional specification document explicitly stated “put this data item into the blue category”), the formula which is not calculating the desired result (but is indeed the exact formula found in the functional specification document) – all are examples of the load object adhering accurately to an incorrect functional specification document.  The root cause, then, is the functional specification document which must first be revised and checked into the controlled document repository before making any code modifications.

It’s the Program

Well, OK, sometimes the load object is at fault.  But it is extremely rare.

Collecting and Reporting the Technical Load Statistics

Load statistics are an important technical metric, indicating what percentage of the upload file has successfully posted a transaction into SAP.  This is a basic and simple record count check.  How many records were presented in the upload file, how many records posted successfully to SAP, and how many records failed to post.  Also, does the number of successful transaction plus the number of failed transactions equal the total number of records presented in the upload file.

Here is how I communicate this technical metric:

This technical metric indicates only that all of the SAP validation rules for posting the transaction have passed.  It does not indicate that the master or transactional data posted to SAP is actually correct in functional terms.

The Functional Review

It is very possible for an entire upload file to technically load at 100%; while at the same time, functionally fail at 100%.  The customer may be missing a partner, the material or article may be categorized incorrectly, the pricing conditions on a sales order may not have the desired validity date range, the GL posting may be to the wrong accounts.  Hence, the need a functional review and validation of the data transacted into SAP.  The data type – master or transactional – determines the type of functional review and metrics to be employed here.

For master data, such as of materials, customers, vendors, etc., it is impossible to individually validate the many thousands of entries in your SAP system.  But statistical methods can be employed which will guide you through a random sampling of the data, while at the same time assuring accuracy at a high degree of confidence levels.

For transactional data, such as inventory, open sales orders, open AR, etc., direct mathematical comparisons can be employed.  On a grand scale, if an inventory value of $15,246,321.44 is being moved from your legacy system to SAP, then that exact amount must arrive in SAP when the migration task is complete.  It may be a bit more difficult to do this mathematical comparison at a more granular level.   If, in the move to SAP you are also redesigning your material/storage location combinations, a direct comparison between the legacy system and SAP may not be possible without a translation factor.  The same scenario exists if you redesign your GL chart of accounts, where one legacy GL account now maps to several SAP GL accounts, or vice versa.

Reporting the Fallout

If an automated data load was not technically 100% successful, a clear set of error messages complete with a link back to the legacy data must be mined, formatted, and presented to the business for analysis.  Such a report really helps to facilitate the fallout cleanup.  The link back to the legacy data must be carefully designed into the load process to make sure that, for example, the legacy customer number, legacy vendor number, legacy sales order number, legacy purchase order number, etc. is included as part of the data being handled.  Without the link back to the legacy data, it becomes very difficult to identify which data record needs to be fixed.

The error message mining technique that I use depends on the load method.  I will describe two here – one for a BDC load method and one for an IDOC load method.

Mining Meaningful Error Messages from a BDC Session Log

At the completion of a batch input session, the batch input session overview screen (SM35) displays the technical load statistics.  In this example, out of a total of 229 transactions, 13 failed and 216 succeeded.

The session log shows the status of all 229 transactions.  This screen snapshot is a fragment of the complete session log for the batch input session.  It shows many successful transactions (Type = S) and one failed transaction (Type = E).  The error message here is clear – the article does not exist or is not activated.

But as you can imagine, the 13 failed transactions with error type = E are sprinkled throughout the many pages of this log file.  With only 229 transactions, this log file is quite easy to pick through to find the 13 errors.  But imagine if the number of transactions were in the thousands or tens of thousands.  How do we extract only the failed transactions and present a concise report of the failed transactions?

To do this, I use SAP transaction SM35P – Batch Input Log Overview.  This transaction has the ability to set a filter on any field in the batch input log file, display the filtered results, and then to export the results to a local file.

To enter the mode where this is possible, first press the PRINT icon.

Next, set the filter.  The appropriate filter field here is SESS. TYPE.

We only want the errors, so set the filter for field SESS. TYPE = E.

The display now shows only the 13 rows containing the error messages.  This can be exported directly to a local spreadsheet for further analysis.

Mining Meaningful Error Messages from IDOCs

Depending on the IDOC and the processing module, mining the error status messages from IDOCs can be very easy or somewhat challenging.  For the more difficult scenarios, you probably will need to hone your EXCEL skills to properly join several extracts together into one complete picture.

When creating IDOCs with a load object, I always note the date, time, and IDOC basic type.  I will use this information as the selection criteria for transaction WE05, which is going to locate the IDOCs and display the results information I need after the IDOCs are processed.  The results that I usually collect are the error status messages, and some data content from a segment or two to illustrate exactly where the problem is in the legacy data.

While the background job is busy processing the IDOCs, I usually take a peek, using transaction WE05, to see how the load is progressing.  If I see that most of the IDOCs are falling onto the floor (IDOC status 51) rather than moving into the database (IDOC status 53), I usually stop the background job and begin an immediate analysis of the fallout.  If the fallout solution does not require a change to the IDOC data content (e.g. a configuration change), then I can replay the fallout using SAP transaction BD87.  If the fallout solution does require a change to the IDOC data content, then the complete set of IDOCs must be regenerated again.

Let’s see what WE05 can tell us about a completed Article Master load.

On the WE05 screen below, we can see that of the total of 14,005 IDOCs, 11,412 have processed successfully and 2,593 have failed to process.  By double clicking the Status 51 folder, the display will show only the Status 51 IDOCs – the fallout.

By pressing the “status list” icon (shown above), the display will show the status messages for the fallout.    Once these messages are displayed, pressing the “export” icon allows me to save the screen contents to a spreadsheet.

Now it would be really nice if I could have the article number in the spreadsheet right next to the error message.  The article number in the ARTMAS IDOC is stored in segment E1BPE1MATHEAD.  The segment content for each IDOC can be displayed by pressing the “list specific segment” icon and entering the segment name in the box.

The segment display will show all fields in the segment, so I usually hide all of the columns that I don’t want to see.  Here is the E1BPE1MATHEAD segment display showing only the article number.  I can use the export icon to save the list of article numbers to another spreadsheet.

Here is a portion of the complete spreadsheet showing the error messages and the article numbers side by side.

A filter applied to the spreadsheet shows that the 2,593 errors are all grouped into one of three error status categories.  By selecting a single category, Excel will also show me the number of records within that failure category.

Sometimes it is easier to mine the status messages directly from the IDOC status table EDIDS.  This is especially true where the processing module is a BAPI which returns an error table rather than a single error message.  In this case, when you press the “status list” icon in WE05, only the first error status message of several is displayed for each IDOC.  I find that the first message is not very helpful (as shown below).  I also find that typically the second or third message in the return status table is usually the important one.  You won’t see it displayed on the WE05 screen, but you can mine it from the EDIDS table.

SAP transactions SE11 or SE16 both support this activity.  For the selection criteria I use the IDOC number range, status 51, and status type E.  On the display screen, choose only the relevant fields for display – the IDOC number (DOCNUM), IDOC status (STATUS), status message (STATXT), the four substitution parameters for the status message (STAPA1, STAPA2, STAPA3, STAPA4) and the message type (STATYP).  All of this can be exported into a spreadsheet.  If you really want to test your Excel skills, you can write code that will move the substitution parameters into their placeholders in the status text.

Preparing for the next data migration cycle – Let the fallout analysis and cleanup begin.

Presenting the fallout report to the business with a set of clear error reasons and links back to the legacy data is key to enabling the legacy data cleanup process to proceed.  In the iterative process of data migration cycles, cleansing the legacy data is a step in the right direction towards an improved next conversion cycle.

I hope you enjoyed this blog series on data migration.  Please feel free to send comments or questions.

EDITOR’S NOTE: NCo 3.0 has now been released.  Read more details here.

NCo 3.0 Logon

One of the new features implemented to combat the malevolent for RFC client attacks is the interface IDestinationConfiguration. This interface contains method IDestinationConfiguration.GetParameters(string destinationName) which allows .NET programmers to provide their own secure method for storing and retrieving logon information, be it encrypted database, encrypted file, or even LDAP.  There is also a corresponding interface and method to secure NCo 3.0 RFC servers.

Additionally, there is updated functionality to support single sign-on (SSO) and X.509 certificates.

Say Goodbye to Data Containers and Generated Code

In SAP Connector 2.0, working with RFCs required the generation of proxy code via the SAP .NET Connector design-time tool.  This tool would convert the IMPORTING, EXPORTING, CHANGING, and TABLES parameters from the RFC you were calling/serving to .NET representations of the same.  With NCo 3.0 there is no longer any kind of generated code.  Instead of one generated proxy method for each function module, there is one single RfcFunction class, whose Invoke() method dynamically executes every given ABAP function module. Additionally, all ABAP parameters will be represented by the class RfcStructure instead of a dedicated generated class for every structure; All tables will be represented by class RfcTable.  So, instead of hard-coding all of the data and variable bindings statically at design time, NCo 3.0 now handles everything dynamically at runtime.

NCo 3.0 uses SAP’s data dictionary to determine the function interface for the called RFCs.  While this data is cached locally for performance reasons, NCo 3.0 is robust enough to detect changes in RFC interface signatures so your code will still execute if you’ve added, deleted, or changed RFC parameters.  If there arises a situation where the dynamic lookup of RFC interface signature is not wanted, there is also an option to hard code the RFC parameter metadata.

The biggest benefit of NCo 3.0′s new dynamic function handling is that NCo 3.0 no longer has a design-time component.  This releases the Visual Studio version restriction that .NET 2.0 had.  Let me rephrase that, NCo 3.0 will work with any version of Visual Studio you like! The only requirement is that at runtime, a .NET Framework version is installed that is compatible with the NCo 3.0 libraries.

Anxiously Waiting

NCo 3.0 figures to be a much-needed update to a vital SAP integration technology.  I, for one, cannot wait for the new functionality to be generally released.  Once NCo 3.0 is released we’ll be posting blogs highlighting the new features.  We’ll also include helpful information regarding a .NET Connector 2.0 to NCo 3.0 upgrade.  In the meantime, if you have any leave any questions or comments you have regarding NCo 3.0 below.  You may also email me directly at csta...@dataxstream.com if you’d like to set up a detailed conversation.

How much data to move? How much data to leave behind? What to automate, and what to execute manually?  How to gracefully orchestrate and execute a data migration cutover from one system to another?  Where and how to fit the data migration plan into the overall business implementation plan?  How to continue to run the business during the data migration phase of the business project implementation? These questions are all part of the planning fun!

Data Migration Testing

Processes that we have exercised and continue to exercise on a regular basis are no big deal, do not cause anxiety, or raise any blood pressures.  The daily report, the custom transaction, the interface that runs several hundred times a week – they are all second nature now.  They happen, and no one notices.

But reflect back to the very first time these processes went live.  This is precisely where you are with data migration.  For most, it is a once in a lifetime event.  It is important, then, to raise confidence levels, and reduce anxiety levels surrounding this activity.  This is achieved by practice, practice, practice.

Typically, I like to see at least three data migration test cycles executed in an isolated data migration client.  This allows for several attempts at exercising and fine-tuning the data migration plan; collecting nominal run time statistics to have some idea of how long the data migration might take; identifying and fixing any data migration program object defects; identifying and fixing any data mapping and content defects; and identifying and fixing any functional configuration issues. Each of these tasks are done with the goal of significantly improving the fallout rate with each data migration test cycle.  These data migration test cycles also give us the opportunity to practice our legacy extract skills, our fallout analysis skills, and our fallout manual cleanup skills.

Source Data Quality

Each data migration test cycle begins with the legacy or source data extract.  An important activity between data migration test cycles is the cleanup of the legacy or source data.  The data migration process will most likely discover many data problems.  Customer and vendor address data alone are enough to bring your bolt-on tax jurisdiction determination software to its knees – that zip code that is too short, too long, or just plain incorrect when combined with the associated city and state; the various abbreviations used in place of the actual city name; the city and state entered in the same field when they should have been entered into separate fields; etc.   If the source data is not fixed, your data migration test cycles will begin to look like the movie Groundhog Day.

Source data provisioning and cleanup could very well be problematic, especially in an acquisition or merger scenario where the providers are part of a different organization and have no incentive to participate in your project.  If you encounter this scenario, you will most likely need to engage the appropriate management level from your organization to have a serious discussion with the appropriate management level from the providing organization.  Recognize this, be in control, and raise that flag early.    Do not sit around for several days waiting for data to arrive, as this will only set your project behind schedule.

Data Migration Support for Other Testing

In any business project scenario, the development team will salivate at the thought of testing their customizations against your real data in the conversion test cycle box.  Likewise, the functional team will be chomping at the bit to use your real data to test configuration scenarios.  And the interface team and the data warehouse team can’t ever get enough data to play with.

While working within your three data migration test cycles, just say NO.  You absolutely need a controlled environment for data migration test cycles, and these other teams will not respect that.  They have a different focus and purpose which requires changing and manipulating degrees of freedom that you need held constant.

But, we are all on the same team trying to move the project to the finish line within the expected project timeline.  So to help your other team members, plan for additional data migration cycles to provide this data to these teams.  That’s more data migration practice for you, and it gives everyone else what they need.

The Perfect World of Data Migration

In a perfect world, for each data object to be converted:

1. The functional specification and mapping documents are well-written and clear.
2. A data load file is built exactly in conformity to the well-written functional specification and mapping documents.
3. The data migration ABAP objects are built exactly as specified by the well-written functional specification and data mapping documents.
4. No one is insisting that we move a square peg into a round hole.

In a perfect world, for each set of data that is to be migrated:

1. The legacy system data has been thoroughly cleansed.
2. The providers of the legacy data are genuinely interested in providing accurate data on time.
3. Server-resident load files in a Unicode system are correctly encoded to UTF-8.
4. A delimiter other than comma has been specified for the load file.

In a perfect world, the data migration test cycle client:

1. Is built in a box that is not the development box.
2. Is configured exactly like the client where the data migration ABAP objects were built and tested.
3. Is not open for configuration, unless by design.
4. Is not part of a transport path. to prevent the current cycle of conversion testing from being blindsided by any new configuration changes.
5. Is locked down so that only data migration and data validation tasks can be performed.
6. Is configured to handle a more background and update processes and fewer dialog processes.
7. Is built in a box that has enough disk space to be the repository for the primary load files and any intermediate processing files needed.

In a perfect world, at the project level:

1. There is an overall business cutover or implementation plan into which you can assimilate your data migration plan.
2. The data migration plan integrates nicely into the overall business cutover plan.
3. There is a strategy in place to bring the data current between the time the cutover freeze is enforced and the implementation date.

Stay Tuned!

In the real world, the fun is just beginning.  Those perfect world scenarios just never seem to happen.   But stay tuned!  In my next blog post in this series, I will discuss the details of the Data Migration Plan.  After that, in subsequent posts, I will drill down into some real world scenarios that I have encountered and discuss how I dealt with them.

• How will I ever match the inbound interface parameter “Tom” with “TOM”, or “tom”?
• How will I ever match the inbound interface parameter “Smith” with “SMITH” or “smith”?
• The ABAP WHERE clause is not case-INsensitive.
• There could be hundreds of customers named Tom Smith.
• KNA1-NAME1 and KNA1-NAME2 are not indexed fields.
• And no, we are not storing any portion of either first or last name in an existing indexed field like SORTL.
• There are well over one million customers in the database.
• We have already decided to use PI for all interfaces.
• I will have to buy the BASIS team a case of beer to get them to agree to create indices on the fields KNA1-NAME1 and KNA1-NAME2 in a table with over one million records.

I arrived at the conclusion that I need a case-insensitive database query, along with database indices created for the fields KNA1-NAME1 and KNA1-NAME2.

But, what is a case-insensitive WHERE clause?  A little research and help from colleagues revealed that many had gone before me, and this was nothing new.  To implement a case-insensitive WHERE clause in ABAP, you simply needed to use the native SQL UPPER() construct. The database system that is being used is Microsoft SQL Server, but the UPPER() function and its syntax is similar across different database platforms. This seemed like an easy nut to crack. But, as I soon found out, I actually had a lot to learn.

Take 1

The examples I found were very explicit, showing actual code snippets.  Using my research examples as a template, I built what was to be my first attempt at using native SQL in a ZTEST ABAP program:

exec sql.
  select kunnr name1 name2
    from kna1
    into table i_cust
    where upper(name1) like :l_name1
      and upper(name2) like :l_name2
endexec.

Well, as a native SQL rookie, I was stopped dead in my tracks.  Native SQL does not allow a select into a table – you can only select into a work area.

Take 2

In my ZTEST ABAP program, I changed the code to SELECT into a work area and append the work area to an internal table in a subroutine:

exec sql performing append_cust.
  select kunnr, name1, name2
    from kna1
    into :wa_cust
    where upper(name1) like :l_name1
      and upper(name2) like :l_name2
endexec.

form append_cust.
  append wa_cust to i_cust.
endform.

I really thought that this would work.  But, I was getting a syntax error from native SQL – the table name was not being recognized.  A little more research revealed that, in some cases, only UPPER CASE is valid for table names and field names.  OK.  I can handle that.

Take 3

I changed the native SQL statement to all UPPER CASE in my ZTEST ABAP program:

EXEC SQL PERFORMING APPEND_CUST.
  SELECT KUNNR, NAME1, NAME2
    FROM KNA1
    INTO :WA_CUST
    WHERE UPPER(NAME1) LIKE :L_NAME1
      AND UPPER(NAME2) LIKE :L_NAME2
ENDEXEC.

form append_cust.
  append wa_cust to i_cust.
endform.

This appearedto work just fine…  Except, I noticed that too many records were being returned.  A little more analysis showed that I was getting records from ALL clients, not just my logon client.

Take 4

Again, in my ZTEST ABAP program, specify the client:

EXEC SQL PERFORMING APPEND_CUST.
  SELECT KUNNR, NAME1, NAME2
    FROM KNA1
    INTO :WA_CUST
    WHERE UPPER(NAME1) LIKE :L_NAME1
      AND UPPER(NAME2) LIKE :L_NAME2
      AND MANDT = :L_MANDT
ENDEXEC.

form append_cust.
  append wa_cust to i_cust.
endform.

Now I was getting the results that I wanted!!!!  So, I pasted this ZTEST ABAP code into the ABAP proxy.

A syntax check of the code immediately rejected the use of subroutines – they are not allowed in OO ABAP. I suppose I should have known that.  The error message indicated that I needed to use a cursor instead of the subroutine.  So, I started cursing – a lot!!!

Take 5 (The Final Version)

Here is the final version of the code which works for OO ABAP, and, hence, the ABAP proxy:

EXEC SQL.
  OPEN CUSTCURSOR FOR
    SELECT KUNNR, NAME1, NAME2
      FROM KNA1
      WHERE UPPER(NAME1) LIKE :L_NAME1
        AND UPPER(NAME2) LIKE :L_NAME2
        AND MANDT = :L_MANDT
ENDEXEC.

DO.
  EXEC SQL.
    FETCH NEXT CUSTCURSOR INTO :WA_CUST
  ENDEXEC.

  IF SY-SUBRC NE 0.
    EXIT.
  ELSE.
    APPEND WA_CUST TO I_CUST.
  ENDIF.
ENDDO.

EXEC SQL.
  CLOSE CUSTCURSOR
ENDEXEC.

Well, I finally got it right after five tries.  As mentioned before, I found many explicit code examples, but none were for OO ABAP with a database  requiring UPPER CASE be used, so that the table and field names would be recognized in the native SQL SELECT statement.

I learned something new about SAP.  I hope that I never stop learning.  I also hope that this blog might save someone a bit of distress in trying to work through a similar situation.  Now, I just need to figure out what kind of beer to buy the BASIS team to get those indexes created…

I am currently working on an SAP implementation project that is just starting its realization phase.  One of my first tasks, as a member of the technical implementation team, is to review completed functional specification documents for RICEF objects.  These documents, written by functional subject matter experts, are supposed to detail business requirements that address gaps, and which need to be incorporated into the system being implemented. The purpose of the review is to make sure that the functional specification documents are complete, accurate, and contain the approval signatures required to move on to the technical design phase.

In my career, I have had the pleasure of working with some first-rate functional analysts who know how to draft an excellent functional specification document in a timely manner.  It is this type of performance that helps to move a project along in the right direction, on schedule, and within budget.  Likewise, I have had the not-so-pleasant task of working with not-so-first-rate functional analysts, who draft functional specification documents that are not clear, inaccurate, and incomplete.  The risks here are ultimately manifest as project delays and cost overruns.

The Good…

A really good functional specification document contains enough detailed information about the business process to enable a technical designer to use it as the foundation for drafting a complete and accurate technical design document.  The functional specification document should not only highlight the presence of a gap, but should demonstrate how the business process, accompanied by automation, will close the gap.  This document must also indicate the abnormal processing requirements – what should happen when that report or interface does not run, what are the recovery steps, how are key employees notified of the problem, etc.  The content of a functional specification document must be tuned to the flavor of the RICEF object that it is describing.  Since they perform very different tasks, a report specification document should be very much different from an interface, conversion, enhancement, or form functional specification document.  Using functional specification templates helps to insure the appropriate content for each type of RICEF object.

… and the Not-So-Good.

I am sometimes astonished by the sparse content that is actually offered up for review.  “We need a report” really does not tell me a whole lot about the business process that I am supposed to automate.  Nor does it even hint at the report purpose, content, layout, user interface, execution mode, authorization requirements, or error handling.  And likewise, “Build me an interface” does not even begin to describe the direction, payload content, mapping, frequency, error handling and recovery steps.  It would be so wrong for me to attempt to build a technical design on such meager functional definitions.   One of my favorite cartoons shows a development manager standing in front of rows of programmers saying “You guys start coding.  I’ll go and find out what they want”.

I am further astonished by:

a)      the project managers who apply pressure to accept inaccurate and incomplete functional specification documents, to give the impression that the project is actually moving forward and making meaningful progress.

b)      the functional analysts who whine incessantly when their paltry functional specification document is not accepted.

A functional specification document that does not meet expectations must be upgraded until it does.  But bouncing a functional specification document back and forth like a ping pong ball between the functional team and a technical reviewer is inefficient and wasteful.  I find that the best way to quickly firm up a weak functional specification document is to thoroughly research all of the issues that I found in the document, formulate proposed solutions where possible, and then schedule a face to face collaborative meeting with the functional analyst and the business process owner(s).  This type of collaboration can save hours, days, or even weeks of wasted ping-pong posturing, and that is always best outcome for the project.

Off-Shore Technical Resources

This face-to-face quick resolution scenario typically cannot happen if you have an off-shore technical contingent in play.  In this case, it is absolutely imperative that the functional specification document be most accurate and complete to mitigate the risk of excessive time loss.   Why is that?

Off-shore resources are sometimes time zone shifted eight or more hours ahead of where the project is located.  If a functional specification document is released for review, it will not be analyzed until we have left for the day.  If the off-shore reviewer has questions or raises issues, we will not see these questions or issues until the next day when we arrive at the project site.  When we respond to the questions or issues, the off-shore team will not see our response until we have left for the day.  And so on.

Under these conditions, a poorly written functional specification document with issues takes days instead of hours to resolve.  This leads to unnecessary project delays and cost overruns.

When One is Really Many

That 3PL interface, which was scoped and planned by the business process owners as a single RICEF object named “The 3PL Interface”; and for which only one interface functional specification document is written, is actually many RICEF objects.  We need to move purchase orders, inventory receipts, advance ship notices, inventory picks, and cycle counts between the two interfacing partners.  Each of these represents a different payload, different mapping, is triggered by a different point in the business process, has separate error handling and recovery procedures, and requires a separate RICEF development object.

That single enhancement functional specification document, which addresses all of SD pricing, has the potential to extend into many different user exits.  I just finished coding an ABAP proxy that was functionally specified as one interface.  In fact it was four.  The requirement was to search the database for sales and invoice data starting with either an invoice number, sales order number, customer name, or company name.  Each of these search techniques required the development of a separate method.  The only pieces of code that were shared among the four search techniques were the input parameters and the output return table.

The point here is to make sure that the project planners understand the real complexity and effort required on the development side, and to make sure that the project plan and budget reflects these more realistic metrics.  This really goes a long way to stop everyone from wondering, “It’s only one interface!  What is taking development so long?”

Great Expectations

So what is a reasonable set of expectations for a really good functional specification document?  What is it that we are asking the business analyst to do?

First, let me describe what I do not expect.  I do not expect a business analyst to write code, build tables, design efficient database retrievals, or to decide that one BAPI, function module, class, or IDOC is better than another.

Here is what I do expect:

A clear definition of a business process that is repeatable, and which actually works.  As a pre-automation test for data conversions, I always require the functional analyst to manually enter one of whatever, using the standard SAP transaction for which a conversion program is to be built. Many times, they can’t because the system is not configured correctly, the supporting data is not present, or any number of other reasons which cause the transaction to fail.  An interesting observation is that there is much indignant huffing and puffing during this manual entry “test” process.  But when the manual test fails, I simply remind them that I cannot automate a broken or non-existent business process.

A clear definition of what should happen under abnormal or failure circumstances.  This must include error handling, notification, recovery and reprocessing steps.

A business process that can efficiently be automated.  Requiring a search of sales order header text for the phrase “This is a red order” is a very bad design for automation purposes.  While such a design is technically possible to build, it will certainly be inefficient at run time, and may not always produce all of the red orders.  This is because the key value is a free-form phrase that can and will be misspelled, and abbreviated, along with countless other mutilations of the key phrase “This is a red order”.  There are much better business processes and technical implementations that will more efficiently and more accurately find all of the red orders in your system.

An explanation of the need for development.  Exactly what is the gap, and how will automating the business process close the gap?

Screen shots from SAP transactions depicting data that is to be retrieved or stored.  From the screen shot in the transaction, I can usually determine the exact table and field in the SAP database.  Note that some business analysts are very adept at identifying the actual underlying table and field name.

Clear and concise details with respect to data mappings, formulae, data transformations, conditional processing, etc.  If I come to an intersection and it is unclear whether I should continue to go straight, turn left, or turn right, then the functional specification document needs a bit more detail behind it.

How to insure Consistency in Functional Specification Document Review

Design a separate functional specification document review checklist for each flavor of RICEF object.  Distribute these checklists to the functional specification writers so that they know what the expectations are.  Using a checklist will help to make sure that your review process is consistent and accurate.  Improve these checklists over time.  My Form functional specification checklist document now includes the following check:

Is it physically possible to print the specified content on the specified form using the specified font style and size?  Was an actual printed mock-up provided as proof?

- but only after I had received a functional spec for a form that required four inches of print content on a one inch label.  And somehow, the business analyst who wrote this particular request erroneously thought that it was my problem to solve.   After all, writing code is magic!  Isn’t it?  In this case, I pushed back and insisted that an actual printed mock-up be produced – one that I would then agree to automate.

Summary

A good functional specification document will help tremendously in moving a project forward in the right direction with minimal cost and risk.  A poor functional specification document has serious potential to cause project delays, and  schedule and cost overruns.  The best goal for the project is to achieve a good functional specification document, using whatever means required.

Too many cooks spoil the broth

The common meaning extracted from this saying is that too many people working on a single task tend to make a mess of it.  But, what happens when you have too many developers working in a single piece of ABAP code?  That’s right, you get a big mess.  This issue is especially difficult to deal with when there are multiple functional requirements leveraging the same custom code object, form-based user-exit, or function-based user-exit.

While current releases of SAP (those built on SAP NetWeaver 2004s and later) have good built-in handling of enhancements and customizations via implicit and explicit enhancement points and BADIs, there still exists many old-school user-exits.

Multiple Developers; One Code Object

I recently worked on a project where three separate developers were creating three separate interfaces based on the outbound delivery IDOC.  While the development for all three interfaces was occurring at the same time, the go-live date for each of the interfaces were different (we’ll discuss that project management glitch at another time).  Each interface required a separate set of custom fields and, therefore, it’s own IDOC extension.  The problem is there is only one appropriate user-exit in IDOC_OUTPUT_DELVRY and three developers needed to be developing in it at the same time!

How did we solve this problem?

An Object To Call One’s Own

To allow each developer his own development object and transport timeline, I decided to implement a custom multiple-use, filtered BADI inside the SAP user-exit function EXIT_SAPLV56K_002.  Actually, you can implement your own BADI almost anywhere you need alternate logic execution–even custom code!

So you might be saying, “Create my own multi-use, filtered BADI definition? Sounds complicated.” Well, it’s easier than you think!

NOTE: This solution was done on an ECC6 system based on SAP NetWeaver 2004s.  The steps to implement this type of solution in ECC5 (SAP NetWeaver 2004) are very similar.

Create A Custom BADI Definition

1. To create a new enhancement spot definition, use transaction SE18.  I named my enhancement spot Z_EXIT_SAPLV56K_002 because the enhancement spot will be used in user-exit function EXIT_SAPLV56K_002.
   
2. In the ensuing dialog box, enter a description and choose BAdi Definition for the Technology
   
3. Next, create a new BADI definition.
   
4. Specify that the new BADI definition will be multiple use and limited to filtered use.
   
   Making the BADI definition multiple use means that each developer can create his or her own BADI implementation.  Each BADI is implemented in its own class.  This promotes code separation and the proper reuse of shared development objects.
   Making the BADI definition limited to filter use ensures that the BADI implementations are separate and cannot affect every interface using the user-exit.
5. Now, create a new filter.
   
   BADI filtering in SAP NetWeaver 2004s is much more powerful than in previous versions of SAP.  Read more about BADI filtering here.
6. The BADI we are defining will be used to populate IDOC extensions.  Each IDOC extension has its own message type (e.g. ZMHL_DESADV, ZQTC_DESADV, ZMHL_SHPORD), therefore the message type of the IDOC being processed is a good value to filter on.  I named my BADI filter EDI_MESTYP.
   
   SAP allows for the automatic checking of BADI implementation filter values.  The standard domain EDI_MESTYP has a value range of table EDMSG attached to it.  This is exactly the range of values I want as allowable values, so I configure the BADI filter to automatically check the filter values against dictionary object EDI_MESTYP.
7. Next, create the BADI interface that will be inherited by each implementing class
   
8. The original user-exti function EXIT_SAPLV56K_002 is executed after each segment is added.  Our BADI will do the same.  Create a new public interface method SEGMENT_ADDED.
   
9. The BADI we are creating is replicating user-exit function EXIT_SAPLV56K_002.  Here is the function interface for EXIT_SAPLV56K_002:
   
   Here is the same data being passed via the interface method.
   
10. Lastly, the BADI has to be called from the existing user-exit.  I will not cover the steps required for implementing user-exit function EXIT_SAPLV56K_002 here, but once you have the user-exit implemented, add the following code.  This code should be the only logic directly inside the user-exit.  Any data processing logic should be implemented via a BADI implementation:

    DATA: GO_BADI TYPE REF TO Z_EXIT_SAPLV56K_002.
    DATA: L_STR TYPE STRING.
    L_STR = MESSAGE_TYPE.
    
    * Get BADI class based on IDOC message type
    GET BADI GO_BADI
       FILTERS EDI_MESTYP = L_STR.
    
    * Execute SEGMENT_ADDED BADI method
    CALL BADI GO_BADI->SEGMENT_ADDED
       EXPORTING
         MESSAGE_TYPE       = MESSAGE_TYPE
         SEGMENT_NAME       = SEGMENT_NAME
         DATA               = DATA
         TAB_IDOC_REDUCTION = TAB_IDOC_REDUCTION
         FLT_VAL            = MESSAGE_TYPE
       CHANGING
         CONTROL_RECORD_OUT = CONTROL_RECORD_OUT
         IDOC_DATA          = IDOC_DATA[].

We’re done with creating the BADI definition!  Save and activate your work. I bet you’re saying to yourself, “Wow, that was easy!”

BADI Implementation

Implementing the BADI you just created in the steps above is just like implementing any other BADI in SAP, so here are the Cliff Notes version of implementation instructions.

1. Using transaction SE19, create a new enhancement spot implementation for Z_EXIT_SAPLV56K_002.
   
2. Enter the BADI implementation details.  This particular BADI implementation is for RICEF ID MHL-IO-007, so I added that identification to the BADI implementation name and implementing class name.
   
3. In the BADI definition, we required each BADI implementation to be filtered on message type. This particular BADI implementation is for message type ZMHL_SHPORD.  Create the filter combination for the BADI implementation.
   
4. This ensures that the other interfaces using this BADI (ZMHL_DESADV, ZQTC_DESADV) will not execute the code implemented with this BADI.
   
5. The final steps are to add the code to the SEGMENT_ADDED method just like you would in user-exit function EXIT_SAPLV56K_002.  Once you are done, save and activate your changes!

Summary

As you can see, with a few extra steps, it is possible to have your cake and eat it, too, by create a custom BADI definition that can be used by other developers.  All this talk of food is making me hungry.  Is it lunchtime yet?