IDoc Has Left The Building
SAP IDocs are a tried and true interface methodology. But as with any interfacing technology the most brittle component is the connection between the two systems (in the case, SAP and the subsystem). In scenarios where SAP is sending IDocs to a subsystem via the transactional RFC, one question is paramount:
How do you know when an Outbound IDoc has physically left SAP?
It’s a simple fact that if your subsystem never receives an IDoc from SAP, the subsystem will not be able to route or process the IDoc. I have always gone by the notion that once the subsystem has successfully received the IDoc, the subsystem now has all the responsibility to process and route the IDoc, but before that time, SAP has the responsibility to ensure that the IDoc is fully dispatched. The subsystem should have processes in place to determine if the IDoc has been received at the target system successfully or not. In this blog I will focus on the handshake between SAP and the subsystem. This will entail Outbound IDoc status and a couple of batch jobs that should be setup to ensure a better chance of a successful handoff between SAP and the subsystem.
How To Implement Field-Level HRMD_A Reduction
I love ALE. It is super-powerful and, once you get the hang of it, is a snap to configure. Recently, I was setting up an HRMD_A interface for my client. Everything was going smoothly until I ran into a requirement to filter out the social security number (PERID), birthdate (GBDAT), and gender (GESCH) for privacy reasons. All of these fields are in segment E1P0002. Initially, I thought that this requirement was easy enough to accomplish. I just created a IDOC reduction in transaction BD53 and filtered out the three fields. I soon found out that while entire segments were getting reduced from the IDOC as configured, the individually reduced fields were still showing up in the IDOC. What’s going on?!? Read more
SAP Standard Sale Area Cross Reference For Inbound IDOC Processing
Cross Reference To SAP Sales Area From External Customer Number – SAP EDSDC Table
If you have ever converted an external file into a SAP Sales Document IDOC (ex: ORDERS05) you may have hard coded the E1EDK14 segments of the IDOC which specify Sales Area and order type for the SAP Sales Document. However, these segments are not required if you populate the E1EDKA1-LIFNR segment of the IDOC with the External Customer Number and populate the EDSDC table with the Sales Area and Order Type for the customer. Your SAP system contains a standard utility that converts the External Customer Number to the Sales Area values and Order Type value contained in this table.
Sales Area Cross Reference From IDOCs ORDERS05 and DELFOR02
This utility is available in the standard SAP package to convert External Customer Number to SAP Sales Area when processing inbound IDOCs (ex: ORDERS05, DELFOR02) from non-SAP systems. It eliminates the need to add E1EDK14 segments to the IDOC as it populates the Sales Organization, Distribution Channel, Division, and Sales Document Type using the entries in the EDSDC table which is populated using transaction VOE2. In order for this process to work, you must have the Customers External number populated in the E1EDKA1-LIFNR field of the IDOC where E1EDKA1-PARVW is ‘AG’ and the EDPAR Customer Conversion table set up to convert the Customer External number to the SAP Customer number. See my blog ‘SAP EDPAR Table – External Customer Number cross reference to SAP Customer Number’ for instructions on the External Customer Conversion. By properly doing this, the standard SAP function module will use the EDSDC table to convert the External Customer Number to the SAP Sales Area and Document Type. The following instructions will explain how to populate the inbound IDOC and the EDSDC table.
SAP Standard Customer Cross Reference For Inbound IDOC Processing
Converting External Customer Number To Internal Customer Number – SAP EDPAR Table
Have you ever had a project where you are converting a customer file into an SAP Sales Document IDOC (ex: ORDERS05) and the customer file contains their Internal Customer Number? So you need to determine what the SAP Customer Number will be on the IDOC. If you are not familiar with the SAP interface process you may hardcode the SAP Customer Number or create external tables in your third party mapping software to accomplish this conversion process. However, this involves customer specific logic or updating the tables with your SAP Customer Numbers and having a process to maintain this table as new SAP Customer Numbers are created. There is a much easier way to perform this conversion with no additional coding as it is a standard SAP utility.
Customer Conversion For IDOCS ORDERS05 and DELFOR02
This utility is available in the standard SAP package to convert External Customer Numbers to SAP Customer numbers (Sold-To, Ship-To, etc.) when processing inbound IDOCs (ex: ORDERS05, DELFOR02) from non-SAP systems. When working with data that will be imported into SAP via IDOC format the data usually contain your customers external sold-to and ship-to numbers and not the associated SAP customer numbers. Therefore, you would need to convert this External Customer Number to your internal SAP Customer Number. Instead of creating cross-reference tables or conversion routines in your third-party translation software you can populate the E1EDKA1-LIFNR field of the inbound IDOC with the External Customer Number and use transaction VOE4 to populate the EDPAR table. By properly doing this, the standard SAP function module will use the EDPAR table values to convert the External Customer Number to the SAP Customer number. The following instructions will explain how to populate the inbound IDOC and the EDPAR table.
Changing SAP IDOCs Status In Mass / Mass Deletion Of IDOCs
Mass Change of SAP IDOC Status
From time to time it becomes necessary to change the status of SAP IDOCs in SAP. The most common scenario is the requirement to mark SAP IDOCs for deletion. There is no good way to mass mark IDOCs for deletion via the standard IDOC processing transaction BD87. However there is a program that will let you change status.
RC1_IDOC_SET_STATUS
CAUTION: This program should be used with great care and consideration. Improper use of this program can result in data consistency issues. Make sure you know what you are deleting, why you are deleting it, and what is required to correctly update you system after deleting.
Are You Managing Your Change Pointers Properly Part 5 – Going Forward
In my last post, I discussed a collaborative effort with functional business owners to devise and execute a proper cleanup plan. I also discussed a functional review of the configuration to make sure that change pointers are being created only when needed.
So now that you have achieved control over your change pointer data, how do you make sure that it does not go out of control again?
Are You Properly Managing Your SAP Change Pointers Part 4 – The Cleanup
In my last post on this topic, I discussed several change pointer data forensic methods that I use to point me in the right cleanup direction. The change pointer data forensics will determine how to design the cleanup process. Armed with meaningful statistics, I usually approach the functional business owners to collaborate on a solution. Typical of the many questions that I ask are “Why are change pointers configured for this message type if we are never distributing the IDOCs?”
Discussion points and procedures for the cleanup process.
1. Unless there is custom ABAP code depending on the continued existence of processed change pointers, they serve no further purpose and should be deleted. ABAP Program RBDCPCLR should be scheduled on a regular basis so that recent “processed” change pointers are purged.
If there is a large quantity of very old “processed” change pointer records, then consider special executions of RBDCPCLR where the selected date range is old enough to collect and purge these records.
To cleanup processed change pointers, use the Processed Change Pointers section on the select option screen of program RBDCPCLR. Make sure that the “Processed Change Pointers” check box is checked, and enter the proper date range. You can further limit the data selection by entering a message type in the message type box.
Note that on the selection screen there is also a “Test Run” check box that will list which records would be selected for purging, without actually purging them. I find this feature very useful, and I strongly recommend using it prior to actual purging.
2. “Unprocessed” change pointer records that are very old are most likely not useable. If the IDOCs that they represent were triggered now, the data distributed might no longer be valid. The important discussion point here is the definition of “very old”. Is it one year, six months, two weeks? This decision definitely needs input from the functional business owners.
Once the definition of “very old” has been established, schedule executions of RBDCPCLR with the “Obsolete Change Pointers” box checked, and the agreed-to “very old” date entered in the date box. This execution will remove both “processed” and “unprocessed” change pointer records up to the specified date.
Note that on the selection screen there is also a “Test Run” check box that will list which records would be selected for purging, without actually purging them. I find this feature very useful, and I strongly recommend using it prior to actual purging.
3. “Unprocessed” change pointers that are deemed recent must be analyzed to determine why these records are configured for creation if they are never actually processed. Perhaps, they were needed at one time, and are no longer needed now. What I need from the functional business owners here is an agreement to turn off, in configuration, the creation of these change pointers. Not turning them off will only continue to add unnecessary records to the change pointer tables.
After the creation of these change pointers is turned off in configuration, then schedule an execution of RBDCPCLR using the obsolete section on the selection screen. Enter the message type in the message type box and the current date. This will purge all change pointers of the specified message type, regardless of their status, up to the specified date. Special care must be taken here to make sure that the selection options are entered correctly.
Note that on the selection screen there is also a “Test Run” check box that will list which records would be selected for purging, without actually purging them. I find this feature very useful, and I strongly recommend using it prior to actual purging.
4. Compare the list of message types that are configured to create change pointers (use SAP transaction BD50), to the list of message types in RBDMIDOC executions and the list of message types RBDCPCLR executions. Make sure that all three lists agree. Review these lists with the functional business owners. Make sure that what is configured is meeting the needs of the current business processes.
I also ask the functional business owners to review the selected table-field values that, when changed or created new, will trigger the creation of change pointer records (use SAP transaction BD52). The goal, again, is to make sure that what is configured is meeting the needs of the current business processes.
Now that we have cleaned up, how do we keep the size of the change pointer tables in check? In my next post, I will discuss items that I recommend monitoring.
Are You Properly Managing Your SAP Change Pointers Part 3 – Using Change Pointer Data Forensics
Your batch job analysis shows that you are currently running the change pointer processing program RBDMIDOC, and the change pointer cleanup program RBDCPCLR; but you notice that the size of your change pointer tables continues to increase.
Now, it is time to analyze the change pointer data directly. Here are some targets that I pursue:
1. Count. How many change pointer records are in your tables – thousands, hundreds of thousands, millions, billions? This simple statistic alone reveals an interesting story.
While I typically try to use transaction SE11 to count the number of records in table BDCP, I have found, on occasion, that this method would time out due to an excessively large number of records in this table. In this case, you can write a simple ABAP which can be scheduled to run in background.
2. Age and status. Using the view BDCPV, determine the age and the status of your change pointer records. The age of a change pointer record is indicated by the field BDCPV-CRETIME, and the status of a change pointer record is indicated by the field BDCPV-PROCESS.
In general, are the change pointer records days, weeks, months, or years old? Are different ancient records found to be in both the ‘processed’ and ‘unprocessed’ status? Are different recent records found to be in both the ‘processed’ and ‘unprocessed’ status?
3. Message type and status. Using the BDCPV view, determine the message type and status of your change pointer records. The message type of a change pointer record is indicated by the field BDCPV-MESTYPE.
Are there different change pointer records for some message types that are never found in the ‘processed’ state? Are there different change pointer records for a particular message type that are in both the ‘processed’ and the ‘unprocessed’ status?
Custom Reporting For a more sohpisticated data analysis, you might consider writing an ABAP program to categorize your change pointer records into ageing buckets, message types, and by status. The cross-tab report shown here is one that I use to analyze change pointer records. It contains three sections - Unprocessed Change Pointers by Message Type Within Ageing Category, Processed Change Pointers by Message Type Within Ageing Category, and a Grand Totals Summary.
Example Scenario Leading to Runaway Change Pointer Records Of course, there are many reasons why you might observe some of the scenarios described above. Here is one example:
Many years ago, your company’s business processes required the daily distribution of material master data additions and changes to a remote plant. Your functional analysts correctly configured the change pointers and the ALE process for message type MATMAS to send this data to your remote plant. RBDMIDOC and RBDCPCLR were both running daily for the material master data message type MATMAS. Change pointer records for material master data were created whenever users added or changed certain data fields, IDOCs were created and sent, the change pointer records were marked as ‘processed’, and were then purged – all on a daily basis.
Last year, business process changed, and your remote plant no longer required the distribution of material master data changes. The batch jobs running RBDMIDOC and RBDCPCLR for the material master data message type MATMAS were cancelled, so that IDOCs would no longer be sent to the plant, and because there was no need to clean up after these IDOCs for this message type.
But…
No one remembered to turn off the creation of change pointers in configuration for the material master data message type MATMAS. So, every time a user added or changed a material, new change pointer records were still being created, causing the change pointer tables to needlessly increase in size. And the status of these change pointer records will forever remain ‘unprocessed’.
Do you have custom code that is creating change pointer records? There is another source of runaway change pointer table size that I would like to discuss here. In some SAP systems that I have worked on, I have found that change pointer records were being created via custom ABAP programs. This was done to support a custom process, having nothing to do with the SAP intended use of the change pointer records – the support of ALE distribution of master data. Unfortunately, the system designers also forgot to write a custom program to either clean up their “custom-purposed” change pointer records, or to allow these records to be recognized by the SAP cleanup program.
Next Steps So, now that we have discovered unnecessary change pointer records cluttering up our SAP database, what do we do next? What is the recovery cleanup process? How do we stop the creation of change pointer records that are no longer needed?
This will be discussed in my next post.
A Single Mistake Can Cost You $$$$$
While working with a client on developing EDI-856 transactions with one of their trading partners, an error occurred that cost my client thousands of dollars in charge back fines.
In their SAP environment they produce a single DESADV IDoc per order/purchase order. This could amount to several hundred DESADV IDocs being generated. The initial development was a 1 to 1 translation in that each DESADV IDoc that was produced would be translated into a single EDI-856 document.
Because of a single data error, due to department procedures not being followed in the warehouse in the closing of orders, the bad data element resulted and was repeated throughout all of the DESADV IDocs that were generated. Translation of the IDocs was successful because this error was not conceived at the point of map development. The hundreds of translated documents were sent to the trading partner and subsequently resulted in failed documents.
Because the trading partner generated a charge back for each document sent, my client was billed thousands of dollars in charge back fines. This simple example shows how a single error repeated in each document can turn out to be a very costly problem.
I hope to show you in the next few postings how to bundle the IDocs together to:
- Bundle and reduce the number of IDocs being translated
- Reduce the number of EDI documents being sent to the trading partner
- Minimize the charge back cost, should and error occur
Eli Lilly: Leveraging SAP XI In You Landscape
 |
 |
Title: Eli Lilly: Leveraging SAP XI in Your Landscape: Costs, Benefits, Lessons Learned
By: Timothy Yates and Scott Sanneman
Key Learning Points:
- This presentation will give you a solid starting point for determining / cost justifying an SAP XI Implementation.
- This presentation will give you specific considerations that need to addressed up front in order to control scope creep and cost.
- This presentation will give you specific technical lessons learned that can potentially impact implementation timeline, delivery quality and overall implementation cost.
