NCo 3.0 Logon

One of the new features implemented to combat the malevolent for RFC client attacks is the interface IDestinationConfiguration. This interface contains method IDestinationConfiguration.GetParameters(string destinationName) which allows .NET programmers to provide their own secure method for storing and retrieving logon information, be it encrypted database, encrypted file, or even LDAP.  There is also a corresponding interface and method to secure NCo 3.0 RFC servers.

Additionally, there is updated functionality to support single sign-on (SSO) and X.509 certificates.

Say Goodbye to Data Containers and Generated Code

In SAP Connector 2.0, working with RFCs required the generation of proxy code via the SAP .NET Connector design-time tool.  This tool would convert the IMPORTING, EXPORTING, CHANGING, and TABLES parameters from the RFC you were calling/serving to .NET representations of the same.  With NCo 3.0 there is no longer any kind of generated code.  Instead of one generated proxy method for each function module, there is one single RfcFunction class, whose Invoke() method dynamically executes every given ABAP function module. Additionally, all ABAP parameters will be represented by the class RfcStructure instead of a dedicated generated class for every structure; All tables will be represented by class RfcTable.  So, instead of hard-coding all of the data and variable bindings statically at design time, NCo 3.0 now handles everything dynamically at runtime.

NCo 3.0 uses SAP’s data dictionary to determine the function interface for the called RFCs.  While this data is cached locally for performance reasons, NCo 3.0 is robust enough to detect changes in RFC interface signatures so your code will still execute if you’ve added, deleted, or changed RFC parameters.  If there arises a situation where the dynamic lookup of RFC interface signature is not wanted, there is also an option to hard code the RFC parameter metadata.

The biggest benefit of NCo 3.0′s new dynamic function handling is that NCo 3.0 no longer has a design-time component.  This releases the Visual Studio version restriction that .NET 2.0 had.  Let me rephrase that, NCo 3.0 will work with any version of Visual Studio you like! The only requirement is that at runtime, a .NET Framework version is installed that is compatible with the NCo 3.0 libraries.

Anxiously Waiting

NCo 3.0 figures to be a much-needed update to a vital SAP integration technology.  I, for one, cannot wait for the new functionality to be generally released.  Once NCo 3.0 is released we’ll be posting blogs highlighting the new features.  We’ll also include helpful information regarding a .NET Connector 2.0 to NCo 3.0 upgrade.  In the meantime, if you have any leave any questions or comments you have regarding NCo 3.0 below.  You may also email me directly at cstasila@dataxstream.com if you’d like to set up a detailed conversation.

How much data to move? How much data to leave behind? What to automate, and what to execute manually?  How to gracefully orchestrate and execute a data migration cutover from one system to another?  Where and how to fit the data migration plan into the overall business implementation plan?  How to continue to run the business during the data migration phase of the business project implementation? These questions are all part of the planning fun!

Data Migration Testing

Processes that we have exercised and continue to exercise on a regular basis are no big deal, do not cause anxiety, or raise any blood pressures.  The daily report, the custom transaction, the interface that runs several hundred times a week – they are all second nature now.  They happen, and no one notices.

But reflect back to the very first time these processes went live.  This is precisely where you are with data migration.  For most, it is a once in a lifetime event.  It is important, then, to raise confidence levels, and reduce anxiety levels surrounding this activity.  This is achieved by practice, practice, practice.

Typically, I like to see at least three data migration test cycles executed in an isolated data migration client.  This allows for several attempts at exercising and fine-tuning the data migration plan; collecting nominal run time statistics to have some idea of how long the data migration might take; identifying and fixing any data migration program object defects; identifying and fixing any data mapping and content defects; and identifying and fixing any functional configuration issues. Each of these tasks are done with the goal of significantly improving the fallout rate with each data migration test cycle.  These data migration test cycles also give us the opportunity to practice our legacy extract skills, our fallout analysis skills, and our fallout manual cleanup skills.

Source Data Quality

Each data migration test cycle begins with the legacy or source data extract.  An important activity between data migration test cycles is the cleanup of the legacy or source data.  The data migration process will most likely discover many data problems.  Customer and vendor address data alone are enough to bring your bolt-on tax jurisdiction determination software to its knees – that zip code that is too short, too long, or just plain incorrect when combined with the associated city and state; the various abbreviations used in place of the actual city name; the city and state entered in the same field when they should have been entered into separate fields; etc.   If the source data is not fixed, your data migration test cycles will begin to look like the movie Groundhog Day.

Source data provisioning and cleanup could very well be problematic, especially in an acquisition or merger scenario where the providers are part of a different organization and have no incentive to participate in your project.  If you encounter this scenario, you will most likely need to engage the appropriate management level from your organization to have a serious discussion with the appropriate management level from the providing organization.  Recognize this, be in control, and raise that flag early.    Do not sit around for several days waiting for data to arrive, as this will only set your project behind schedule.

Data Migration Support for Other Testing

In any business project scenario, the development team will salivate at the thought of testing their customizations against your real data in the conversion test cycle box.  Likewise, the functional team will be chomping at the bit to use your real data to test configuration scenarios.  And the interface team and the data warehouse team can’t ever get enough data to play with.

While working within your three data migration test cycles, just say NO.  You absolutely need a controlled environment for data migration test cycles, and these other teams will not respect that.  They have a different focus and purpose which requires changing and manipulating degrees of freedom that you need held constant.

But, we are all on the same team trying to move the project to the finish line within the expected project timeline.  So to help your other team members, plan for additional data migration cycles to provide this data to these teams.  That’s more data migration practice for you, and it gives everyone else what they need.

The Perfect World of Data Migration

In a perfect world, for each data object to be converted:

1. The functional specification and mapping documents are well-written and clear.
2. A data load file is built exactly in conformity to the well-written functional specification and mapping documents.
3. The data migration ABAP objects are built exactly as specified by the well-written functional specification and data mapping documents.
4. No one is insisting that we move a square peg into a round hole.

In a perfect world, for each set of data that is to be migrated:

1. The legacy system data has been thoroughly cleansed.
2. The providers of the legacy data are genuinely interested in providing accurate data on time.
3. Server-resident load files in a Unicode system are correctly encoded to UTF-8.
4. A delimiter other than comma has been specified for the load file.

In a perfect world, the data migration test cycle client:

1. Is built in a box that is not the development box.
2. Is configured exactly like the client where the data migration ABAP objects were built and tested.
3. Is not open for configuration, unless by design.
4. Is not part of a transport path. to prevent the current cycle of conversion testing from being blindsided by any new configuration changes.
5. Is locked down so that only data migration and data validation tasks can be performed.
6. Is configured to handle a more background and update processes and fewer dialog processes.
7. Is built in a box that has enough disk space to be the repository for the primary load files and any intermediate processing files needed.

In a perfect world, at the project level:

1. There is an overall business cutover or implementation plan into which you can assimilate your data migration plan.
2. The data migration plan integrates nicely into the overall business cutover plan.
3. There is a strategy in place to bring the data current between the time the cutover freeze is enforced and the implementation date.

Stay Tuned!

In the real world, the fun is just beginning.  Those perfect world scenarios just never seem to happen.   But stay tuned!  In my next blog post in this series, I will discuss the details of the Data Migration Plan.  After that, in subsequent posts, I will drill down into some real world scenarios that I have encountered and discuss how I dealt with them.

Object Name Prefixes

I have seen a lot of places use prefixes or suffixes before objects such as or dt_ or _mt. I have never really been a fan of these prefixes and suffixes, and apparently neither is the above linked naming convention guide. The reason being is that they are unnecessary. It would be difficult to confuse a message type with a data type in a real interface scenario, even if troubleshooting an unknown broken object. Plus it makes message mapping unnecessarily confusing and long since it’s not clear whether to add the prefix, i.e. MT_one_to_MT_two. Good descriptive names are usually all that you need e.g. DEBMAS_to_Customer. The only possible exception to a no-suffix-or-prefix-policy is the service interface, as sometimes it is useful to know which direction and type a service interface is. An argument for a prefix or a suffix to describe a service interface would be to assist in understanding the flow from an SAP perspective in the event that someone who didn’t develop the interface had to come behind and troubleshoot. A argument against would be the fact that it looks silly if you use it for a web service, because you have a name that doesn’t mean anything to a third party user (note operation mappings are told to omit the prefix of direction and mode in the SAP guide).

One interesting thing that I noticed in my investigation of PI 7.1 EHP1 is that it appears that naming conventions on PI can be validated. If the object names do not conform to a naming convention a message will appear. Ter perform this check in the ESR go to menu Tools>Component Check.

Select “Governance” and “Interface Name Checks”:

If the service interface does not end with (In/Out)(SYNC/ASY) the interface check will show as an error (does not impact interface processing). I created 2 interfaces: one good and one bad to show this error.

My suspicion is that SAP will put more in place to force consistent naming standards depending on the service interface pattern in future releases.

Using a software component and Namespace for each “side” of an interface

All objects of an interface should not be grouped in a single namespace. They should to be split among the Software Component Versions of the systems being interfaced. Otherwise when you go to configure, you will not be able to see your operational mapping (OM) in the dropdown box without having to select all in the dropdown menu. A general rule of thumb is: if it’s not easy to configure, odds are you have probably done something wrong. Another reason why an object might not appear in the dropdown menu (for example for an operations mapping on an interface mapping) would be if the installed checkbox is not clicked on the SLD. When done correctly, most interfaces should be able to be configured quickly and intuitively in the integration directory (ID) without the need to select from all SWCV from dropdown menus on the integration builder.

Whatever naming convention you choose to use for the ESR, the important thing to remember is that adhering to the standard makes production support and troubleshooting faster and easier.

• How will I ever match the inbound interface parameter “Tom” with “TOM”, or “tom”?
• How will I ever match the inbound interface parameter “Smith” with “SMITH” or “smith”?
• The ABAP WHERE clause is not case-INsensitive.
• There could be hundreds of customers named Tom Smith.
• KNA1-NAME1 and KNA1-NAME2 are not indexed fields.
• And no, we are not storing any portion of either first or last name in an existing indexed field like SORTL.
• There are well over one million customers in the database.
• We have already decided to use PI for all interfaces.
• I will have to buy the BASIS team a case of beer to get them to agree to create indices on the fields KNA1-NAME1 and KNA1-NAME2 in a table with over one million records.

I arrived at the conclusion that I need a case-insensitive database query, along with database indices created for the fields KNA1-NAME1 and KNA1-NAME2.

But, what is a case-insensitive WHERE clause?  A little research and help from colleagues revealed that many had gone before me, and this was nothing new.  To implement a case-insensitive WHERE clause in ABAP, you simply needed to use the native SQL UPPER() construct. The database system that is being used is Microsoft SQL Server, but the UPPER() function and its syntax is similar across different database platforms. This seemed like an easy nut to crack. But, as I soon found out, I actually had a lot to learn.

Take 1

The examples I found were very explicit, showing actual code snippets.  Using my research examples as a template, I built what was to be my first attempt at using native SQL in a ZTEST ABAP program:

exec sql.
  select kunnr name1 name2
    from kna1
    into table i_cust
    where upper(name1) like :l_name1
      and upper(name2) like :l_name2
endexec.

Well, as a native SQL rookie, I was stopped dead in my tracks.  Native SQL does not allow a select into a table – you can only select into a work area.

Take 2

In my ZTEST ABAP program, I changed the code to SELECT into a work area and append the work area to an internal table in a subroutine:

exec sql performing append_cust.
  select kunnr, name1, name2
    from kna1
    into :wa_cust
    where upper(name1) like :l_name1
      and upper(name2) like :l_name2
endexec.

form append_cust.
  append wa_cust to i_cust.
endform.

I really thought that this would work.  But, I was getting a syntax error from native SQL – the table name was not being recognized.  A little more research revealed that, in some cases, only UPPER CASE is valid for table names and field names.  OK.  I can handle that.

Take 3

I changed the native SQL statement to all UPPER CASE in my ZTEST ABAP program:

EXEC SQL PERFORMING APPEND_CUST.
  SELECT KUNNR, NAME1, NAME2
    FROM KNA1
    INTO :WA_CUST
    WHERE UPPER(NAME1) LIKE :L_NAME1
      AND UPPER(NAME2) LIKE :L_NAME2
ENDEXEC.

form append_cust.
  append wa_cust to i_cust.
endform.

This appearedto work just fine…  Except, I noticed that too many records were being returned.  A little more analysis showed that I was getting records from ALL clients, not just my logon client.

Take 4

Again, in my ZTEST ABAP program, specify the client:

EXEC SQL PERFORMING APPEND_CUST.
  SELECT KUNNR, NAME1, NAME2
    FROM KNA1
    INTO :WA_CUST
    WHERE UPPER(NAME1) LIKE :L_NAME1
      AND UPPER(NAME2) LIKE :L_NAME2
      AND MANDT = :L_MANDT
ENDEXEC.

form append_cust.
  append wa_cust to i_cust.
endform.

Now I was getting the results that I wanted!!!!  So, I pasted this ZTEST ABAP code into the ABAP proxy.

A syntax check of the code immediately rejected the use of subroutines – they are not allowed in OO ABAP. I suppose I should have known that.  The error message indicated that I needed to use a cursor instead of the subroutine.  So, I started cursing – a lot!!!

Take 5 (The Final Version)

Here is the final version of the code which works for OO ABAP, and, hence, the ABAP proxy:

EXEC SQL.
  OPEN CUSTCURSOR FOR
    SELECT KUNNR, NAME1, NAME2
      FROM KNA1
      WHERE UPPER(NAME1) LIKE :L_NAME1
        AND UPPER(NAME2) LIKE :L_NAME2
        AND MANDT = :L_MANDT
ENDEXEC.

DO.
  EXEC SQL.
    FETCH NEXT CUSTCURSOR INTO :WA_CUST
  ENDEXEC.

  IF SY-SUBRC NE 0.
    EXIT.
  ELSE.
    APPEND WA_CUST TO I_CUST.
  ENDIF.
ENDDO.

EXEC SQL.
  CLOSE CUSTCURSOR
ENDEXEC.

Well, I finally got it right after five tries.  As mentioned before, I found many explicit code examples, but none were for OO ABAP with a database  requiring UPPER CASE be used, so that the table and field names would be recognized in the native SQL SELECT statement.

I learned something new about SAP.  I hope that I never stop learning.  I also hope that this blog might save someone a bit of distress in trying to work through a similar situation.  Now, I just need to figure out what kind of beer to buy the BASIS team to get those indexes created…

Part One – The Integration Directory (for PI 7.1 or greater)

I decided to start with the Integration Builder because it’s the easier component with which to bring everything into alignment (versus the ESR). Most of the time there are only two things to name: the scenario and the communication channel.  Unfortunately, it is still incredibly easy for objects to become disorganized.

Configuration Scenarios

The whole purpose of a configuration scenario is for organization. An interface will run whether it is in a scenario or not, however troubleshooting is much more difficult when objects are not properly organized. It is important to use a naming convention that addresses the business purpose of the interface and is specific to the flow of the interface (to help with troubleshooting issues and make flows easily traceable).  To accomplish all of these requirements, DataXstream recommends a naming convention that includes the specification number which originated the development request and a route number. The route number is used in interface diagrams and documentation and assists production support with identifying objects in the event that a business scenario involves multiple destinations.  We also recommend including a statement to describe the flow for the route, as well as the description of what the interface does. Some examples of a naming convention we might use would be:

0001.001_ECC_to_Partner1_TransactionalDataDistribution 0001.002_ECC_to_Partner2_TransacationalDataDistribution 0002.001_Partner3_to_ECC_SomeInboundInterface 0003.001_Partner4_to_Partner5_SomeNonSAPInterface

The other thing of note is that all objects should exist in a scenario, even if in a test_DONOTTRANSPORTTOPRODUCTION_prototype_interfacedescription scenario. Proper organization makes it a lot easier to identify, modify, turn on and off, and troubleshoot interfaces. Fully configured interfaces running in objects view take more time to understand and are more time consuming to fix.

Communication Channels

Communication channels should describe the purpose, type, and direction of communication. However a lot of people include unnecessary components in the communication channel. For example, since there are usually only two objects with names in the IB, it is not necessary to prefix a communication channel with cc_ as it is unlikely to ever be confused for a scenario. A lot of people also include the communication component in the name of the communication channel, but that also seems redundant, since only communication channels exist under communication components. Some things that are valuable in looking at communication channel naming convention are knowing a brief description of the process it involves, a direction of the communication channel, and the type of adapter being used. Some examples of naming conventions we might use are:

Logistics Company

• PO_file_sender

Database System

• materialSqlXml_jdbc_receiver

ECC system

• IDoc_receiver

As you can see, it is possible to keep the name descriptive and helpful without making the name overly verbose when it comes to communication channels.

The order in which the naming components are ordered depend as much on personal preference as anything else.  For example, let’s examine a sender file channel that pertains to purchase order data.  Some of the naming convention options are:

• Sender_File_PO: groups all sender and receiver channels together, then groups by adapter type
• File_Sender_PO: groups communication channels by adapter type, then by sender and receiver
• PO_File_Sender: groups by logical interface payload, then adapter type

Other Things To Watch Out For

• Configure the unnecessary- We all know that no sender agreement or channel is needed for certain communication methods like iDoc or SOAP to the IE, but if you can, add them to the scenario anyways. It makes it easier for a new developer (think of someone new to production support) to understand what’s happening with an interface, when they can see a consistent flow of the interface sender agreement to receiver agreement on all interfaces in a landscape. I know from working with working with soap, it’s kind of nice being able to grab the wsdl from the sender agreement, even though that component is technically optional.

• Don’t force an interface to work- If you are configuring a determination or agreement and are not seeing what you expect from the drop down menu, it may be because you haven’t properly defined the SLD or IR. Don’t be lazy and select the interface from all namespaces or SWCVs just to make the interface work; go back and figure out why its not dropping down on the drop down box and correct it.

Establishing and adhering to a well thought-out and consistent naming convention can save you a lot of time during the support of your SAP PI interfaces.  In Part II of this blog, I will discuss naming conventions in SAP PI’s Enterprise Service Repository (ESR).

This would, in turn, generate a short dump:

My initial thought was that this was a TemSe issue, meaning it was a true BASIS issue.  Upon further examination, I came to the conclusion that it was actually a security/access issue.  The user did not have create rights for TemSe.  Find the Role path Basis: Administration -> TemSe: Actions on TemSe objects -> TemSe: Action ID for Authorization and make sure that the user has the value “CRE” (for create).

Overview

I am currently working on an SAP implementation project that is just starting its realization phase.  One of my first tasks, as a member of the technical implementation team, is to review completed functional specification documents for RICEF objects.  These documents, written by functional subject matter experts, are supposed to detail business requirements that address gaps, and which need to be incorporated into the system being implemented. The purpose of the review is to make sure that the functional specification documents are complete, accurate, and contain the approval signatures required to move on to the technical design phase.

In my career, I have had the pleasure of working with some first-rate functional analysts who know how to draft an excellent functional specification document in a timely manner.  It is this type of performance that helps to move a project along in the right direction, on schedule, and within budget.  Likewise, I have had the not-so-pleasant task of working with not-so-first-rate functional analysts, who draft functional specification documents that are not clear, inaccurate, and incomplete.  The risks here are ultimately manifest as project delays and cost overruns.

The Good…

A really good functional specification document contains enough detailed information about the business process to enable a technical designer to use it as the foundation for drafting a complete and accurate technical design document.  The functional specification document should not only highlight the presence of a gap, but should demonstrate how the business process, accompanied by automation, will close the gap.  This document must also indicate the abnormal processing requirements – what should happen when that report or interface does not run, what are the recovery steps, how are key employees notified of the problem, etc.  The content of a functional specification document must be tuned to the flavor of the RICEF object that it is describing.  Since they perform very different tasks, a report specification document should be very much different from an interface, conversion, enhancement, or form functional specification document.  Using functional specification templates helps to insure the appropriate content for each type of RICEF object.

… and the Not-So-Good.

I am sometimes astonished by the sparse content that is actually offered up for review.  “We need a report” really does not tell me a whole lot about the business process that I am supposed to automate.  Nor does it even hint at the report purpose, content, layout, user interface, execution mode, authorization requirements, or error handling.  And likewise, “Build me an interface” does not even begin to describe the direction, payload content, mapping, frequency, error handling and recovery steps.  It would be so wrong for me to attempt to build a technical design on such meager functional definitions.   One of my favorite cartoons shows a development manager standing in front of rows of programmers saying “You guys start coding.  I’ll go and find out what they want”.

I am further astonished by:

a)      the project managers who apply pressure to accept inaccurate and incomplete functional specification documents, to give the impression that the project is actually moving forward and making meaningful progress.

b)      the functional analysts who whine incessantly when their paltry functional specification document is not accepted.

A functional specification document that does not meet expectations must be upgraded until it does.  But bouncing a functional specification document back and forth like a ping pong ball between the functional team and a technical reviewer is inefficient and wasteful.  I find that the best way to quickly firm up a weak functional specification document is to thoroughly research all of the issues that I found in the document, formulate proposed solutions where possible, and then schedule a face to face collaborative meeting with the functional analyst and the business process owner(s).  This type of collaboration can save hours, days, or even weeks of wasted ping-pong posturing, and that is always best outcome for the project.

Off-Shore Technical Resources

This face-to-face quick resolution scenario typically cannot happen if you have an off-shore technical contingent in play.  In this case, it is absolutely imperative that the functional specification document be most accurate and complete to mitigate the risk of excessive time loss.   Why is that?

Off-shore resources are sometimes time zone shifted eight or more hours ahead of where the project is located.  If a functional specification document is released for review, it will not be analyzed until we have left for the day.  If the off-shore reviewer has questions or raises issues, we will not see these questions or issues until the next day when we arrive at the project site.  When we respond to the questions or issues, the off-shore team will not see our response until we have left for the day.  And so on.

Under these conditions, a poorly written functional specification document with issues takes days instead of hours to resolve.  This leads to unnecessary project delays and cost overruns.

When One is Really Many

That 3PL interface, which was scoped and planned by the business process owners as a single RICEF object named “The 3PL Interface”; and for which only one interface functional specification document is written, is actually many RICEF objects.  We need to move purchase orders, inventory receipts, advance ship notices, inventory picks, and cycle counts between the two interfacing partners.  Each of these represents a different payload, different mapping, is triggered by a different point in the business process, has separate error handling and recovery procedures, and requires a separate RICEF development object.

That single enhancement functional specification document, which addresses all of SD pricing, has the potential to extend into many different user exits.  I just finished coding an ABAP proxy that was functionally specified as one interface.  In fact it was four.  The requirement was to search the database for sales and invoice data starting with either an invoice number, sales order number, customer name, or company name.  Each of these search techniques required the development of a separate method.  The only pieces of code that were shared among the four search techniques were the input parameters and the output return table.

The point here is to make sure that the project planners understand the real complexity and effort required on the development side, and to make sure that the project plan and budget reflects these more realistic metrics.  This really goes a long way to stop everyone from wondering, “It’s only one interface!  What is taking development so long?”

Great Expectations

So what is a reasonable set of expectations for a really good functional specification document?  What is it that we are asking the business analyst to do?

First, let me describe what I do not expect.  I do not expect a business analyst to write code, build tables, design efficient database retrievals, or to decide that one BAPI, function module, class, or IDOC is better than another.

Here is what I do expect:

A clear definition of a business process that is repeatable, and which actually works.  As a pre-automation test for data conversions, I always require the functional analyst to manually enter one of whatever, using the standard SAP transaction for which a conversion program is to be built. Many times, they can’t because the system is not configured correctly, the supporting data is not present, or any number of other reasons which cause the transaction to fail.  An interesting observation is that there is much indignant huffing and puffing during this manual entry “test” process.  But when the manual test fails, I simply remind them that I cannot automate a broken or non-existent business process.

A clear definition of what should happen under abnormal or failure circumstances.  This must include error handling, notification, recovery and reprocessing steps.

A business process that can efficiently be automated.  Requiring a search of sales order header text for the phrase “This is a red order” is a very bad design for automation purposes.  While such a design is technically possible to build, it will certainly be inefficient at run time, and may not always produce all of the red orders.  This is because the key value is a free-form phrase that can and will be misspelled, and abbreviated, along with countless other mutilations of the key phrase “This is a red order”.  There are much better business processes and technical implementations that will more efficiently and more accurately find all of the red orders in your system.

An explanation of the need for development.  Exactly what is the gap, and how will automating the business process close the gap?

Screen shots from SAP transactions depicting data that is to be retrieved or stored.  From the screen shot in the transaction, I can usually determine the exact table and field in the SAP database.  Note that some business analysts are very adept at identifying the actual underlying table and field name.

Clear and concise details with respect to data mappings, formulae, data transformations, conditional processing, etc.  If I come to an intersection and it is unclear whether I should continue to go straight, turn left, or turn right, then the functional specification document needs a bit more detail behind it.

How to insure Consistency in Functional Specification Document Review

Design a separate functional specification document review checklist for each flavor of RICEF object.  Distribute these checklists to the functional specification writers so that they know what the expectations are.  Using a checklist will help to make sure that your review process is consistent and accurate.  Improve these checklists over time.  My Form functional specification checklist document now includes the following check:

Is it physically possible to print the specified content on the specified form using the specified font style and size?  Was an actual printed mock-up provided as proof?

- but only after I had received a functional spec for a form that required four inches of print content on a one inch label.  And somehow, the business analyst who wrote this particular request erroneously thought that it was my problem to solve.   After all, writing code is magic!  Isn’t it?  In this case, I pushed back and insisted that an actual printed mock-up be produced – one that I would then agree to automate.

Summary

A good functional specification document will help tremendously in moving a project forward in the right direction with minimal cost and risk.  A poor functional specification document has serious potential to cause project delays, and  schedule and cost overruns.  The best goal for the project is to achieve a good functional specification document, using whatever means required.

How do you know when an Outbound IDoc has physically left SAP?

It’s a simple fact that if your subsystem never receives an IDoc from SAP, the subsystem will not be able to route or process the IDoc.   I have always gone by the notion that once the subsystem has successfully received the IDoc, the subsystem now has all the responsibility to process and route the IDoc, but before that time, SAP has the responsibility to ensure that the IDoc is fully dispatched.  The subsystem should have processes in place to determine if the IDoc has been received at the target system successfully or not.  In this blog I will focus on the handshake between SAP and the subsystem.  This will entail Outbound IDoc status and a couple of batch jobs that should be setup to ensure a better chance of a successful handoff between SAP and the subsystem.

First, let’s cover the normal, expected progression of statuses for outbound IDocs.  The following graphic shows the collection of status records for a random outbound IDoc.

The status records at the bottom of the list depict the IDoc status before the status records at the top of the list.  So, in this example, the IDoc started in status “01 – IDoc generated”, then progressed to status “30 – IDoc ready for dispatch (ALE service).”  These two statuses usually are created in rapid succession for outbound IDocs and mean that the IDoc has been created in the SAP system and is ready to be sent via tRFC to the subsystem.  The next two IDoc status, 03 and 12, can be a little confusing.  They both seem to mean that the IDoc has been dispatched from SAP.  What is the difference?  I have found that sometimes there is some confusion on weather the IDoc has left SAP or not.  After I explain the difference, I will show 2 batch jobs that should be setup in order to update the status if the Outbound IDoc has physically left SAP.

So what do statuses ‘03’ and ‘12’ really mean?

The status ‘03’ actually means “Data passed to port OK” which basically means that the IDoc has been dispatched to the tRFC queue and an attempt was made to send the IDoc to the subsystem.  However, that does not necessarily mean that the IDoc has left SAP.   There could be many reasons why SAP would fail to send the IDoc.  One example could be that the subsystem was down during the transmission of the IDoc.  Errors during data transmission would cause the IDoc to remain in the tRFC queue.  Thus, the IDoc would still remain in SAP and remain with a status of ‘03’.

When the Status changes to ‘12’; a successful connection was made to the subsystem and the IDoc was sent.  This status represents that the Outbound IDoc is no longer in the tRFC queue  and, therefore, has physically left SAP.  The IDoc is in the hands of the subsystem which is now responsible for processing or routing the IDoc.

But what if I never see a status of ‘12’ on IDocs that were successfully sent?

For one reason or another, sometimes the batch job to update the status to ‘12’ is not scheduled.  In Production this batch job is usually set up, however in the QA or Dev environment this may or may not be setup depending on the client.

1)  The following job should be setup in order to process a status ‘12’.  I typically set this to every 10 minutes depending on the client.

Program:  RBDMOIND – Schedule this program in order to check whether the IDoc have been successfully sent out of the tRFC queue.  If the IDoc has been sent successfully to the subsystem, the program RBDMOIND will change the status of the IDoc from status ‘03’ to status ‘12’.  This means that the handshake with subsystem was successfully and the IDoc has physically left SAP.

2)  There may be times when the subsystem is unavailable at the time the SAP is trying to send the Outbound IDoc.  By default, SAP will only make one attempt to transmit the IDoc.  By scheduling the following program in a batch job, SAP will try to resend the Outbound IDoc again at a specified time interval.  I also usually set this at job every 10 min, depending on the client.

Program:  RSARFCEX – Schedule this program in a batch job in order to resend any IDocs that have are stuck on the tRFC queue.

Ok I have set up these jobs, but the status does not change.  Now what?

Sometimes an Outbound IDoc could remain stuck on the tRFC queue and never get sent out successfully.  Some issues could be that the RFC Destination for the subsystem is configured incorrectly or there could be an authorization issue.  These types of errors need to be fixed before the batch job for RSARFCEX will send the IDoc out successfully.  You may need to get some additional detail of the problem.  Execute the following transaction below.

Transaction SM58 – This will check what is currently on the tRFC queue.   Once you display the list, you also have the ability to drill down and get some additional detail of what has caused the failed transmission.

Conclusion

Typically you won’t see many issues with Outbound IDocs.  From time to time, IDocs will get stuck on the tRFC queue and if not managed properly, could be stuck there a long time.  This blog should give you more visibility in respect to IDoc status to ensure that the Outbound IDoc has successfully left SAP for processing.

Too many cooks spoil the broth

The common meaning extracted from this saying is that too many people working on a single task tend to make a mess of it.  But, what happens when you have too many developers working in a single piece of ABAP code?  That’s right, you get a big mess.  This issue is especially difficult to deal with when there are multiple functional requirements leveraging the same custom code object, form-based user-exit, or function-based user-exit.

While current releases of SAP (those built on SAP NetWeaver 2004s and later) have good built-in handling of enhancements and customizations via implicit and explicit enhancement points and BADIs, there still exists many old-school user-exits.

Multiple Developers; One Code Object

I recently worked on a project where three separate developers were creating three separate interfaces based on the outbound delivery IDOC.  While the development for all three interfaces was occurring at the same time, the go-live date for each of the interfaces were different (we’ll discuss that project management glitch at another time).  Each interface required a separate set of custom fields and, therefore, it’s own IDOC extension.  The problem is there is only one appropriate user-exit in IDOC_OUTPUT_DELVRY and three developers needed to be developing in it at the same time!

How did we solve this problem?

An Object To Call One’s Own

To allow each developer his own development object and transport timeline, I decided to implement a custom multiple-use, filtered BADI inside the SAP user-exit function EXIT_SAPLV56K_002.  Actually, you can implement your own BADI almost anywhere you need alternate logic execution–even custom code!

So you might be saying, “Create my own multi-use, filtered BADI definition? Sounds complicated.” Well, it’s easier than you think!

NOTE: This solution was done on an ECC6 system based on SAP NetWeaver 2004s.  The steps to implement this type of solution in ECC5 (SAP NetWeaver 2004) are very similar.

Create A Custom BADI Definition

1. To create a new enhancement spot definition, use transaction SE18.  I named my enhancement spot Z_EXIT_SAPLV56K_002 because the enhancement spot will be used in user-exit function EXIT_SAPLV56K_002.
2. In the ensuing dialog box, enter a description and choose BAdi Definition for the Technology
3. Next, create a new BADI definition.
4. Specify that the new BADI definition will be multiple use and limited to filtered use. Making the BADI definition multiple use means that each developer can create his or her own BADI implementation.  Each BADI is implemented in its own class.  This promotes code separation and the proper reuse of shared development objects. Making the BADI definition limited to filter use ensures that the BADI implementations are separate and cannot affect every interface using the user-exit.
5. Now, create a new filter. BADI filtering in SAP NetWeaver 2004s is much more powerful than in previous versions of SAP.  Read more about BADI filtering here.
6. The BADI we are defining will be used to populate IDOC extensions.  Each IDOC extension has its own message type (e.g. ZMHL_DESADV, ZQTC_DESADV, ZMHL_SHPORD), therefore the message type of the IDOC being processed is a good value to filter on.  I named my BADI filter EDI_MESTYP. SAP allows for the automatic checking of BADI implementation filter values.  The standard domain EDI_MESTYP has a value range of table EDMSG attached to it.  This is exactly the range of values I want as allowable values, so I configure the BADI filter to automatically check the filter values against dictionary object EDI_MESTYP.
7. Next, create the BADI interface that will be inherited by each implementing class
8. The original user-exti function EXIT_SAPLV56K_002 is executed after each segment is added.  Our BADI will do the same.  Create a new public interface method SEGMENT_ADDED.
9. The BADI we are creating is replicating user-exit function EXIT_SAPLV56K_002.  Here is the function interface for EXIT_SAPLV56K_002: Here is the same data being passed via the interface method.
10. Lastly, the BADI has to be called from the existing user-exit.  I will not cover the steps required for implementing user-exit function EXIT_SAPLV56K_002 here, but once you have the user-exit implemented, add the following code.  This code should be the only logic directly inside the user-exit.  Any data processing logic should be implemented via a BADI implementation:

    DATA: GO_BADI TYPE REF TO Z_EXIT_SAPLV56K_002.
    DATA: L_STR TYPE STRING.
    L_STR = MESSAGE_TYPE.
    
    * Get BADI class based on IDOC message type
    GET BADI GO_BADI
       FILTERS EDI_MESTYP = L_STR.
    
    * Execute SEGMENT_ADDED BADI method
    CALL BADI GO_BADI->SEGMENT_ADDED
       EXPORTING
         MESSAGE_TYPE       = MESSAGE_TYPE
         SEGMENT_NAME       = SEGMENT_NAME
         DATA               = DATA
         TAB_IDOC_REDUCTION = TAB_IDOC_REDUCTION
         FLT_VAL            = MESSAGE_TYPE
       CHANGING
         CONTROL_RECORD_OUT = CONTROL_RECORD_OUT
         IDOC_DATA          = IDOC_DATA[].

We’re done with creating the BADI definition!  Save and activate your work. I bet you’re saying to yourself, “Wow, that was easy!”

BADI Implementation

Implementing the BADI you just created in the steps above is just like implementing any other BADI in SAP, so here are the Cliff Notes version of implementation instructions.

1. Using transaction SE19, create a new enhancement spot implementation for Z_EXIT_SAPLV56K_002.
2. Enter the BADI implementation details.  This particular BADI implementation is for RICEF ID MHL-IO-007, so I added that identification to the BADI implementation name and implementing class name.
3. In the BADI definition, we required each BADI implementation to be filtered on message type. This particular BADI implementation is for message type ZMHL_SHPORD.  Create the filter combination for the BADI implementation.
4. This ensures that the other interfaces using this BADI (ZMHL_DESADV, ZQTC_DESADV) will not execute the code implemented with this BADI.
5. The final steps are to add the code to the SEGMENT_ADDED method just like you would in user-exit function EXIT_SAPLV56K_002.  Once you are done, save and activate your changes!

Summary

As you can see, with a few extra steps, it is possible to have your cake and eat it, too, by create a custom BADI definition that can be used by other developers.  All this talk of food is making me hungry.  Is it lunchtime yet?

Now, put me behind a keyboard in front of a computer application which I have never seen before. My user experience is all over the map – somewhere in the continuum between most excellent and very poor.  Some application user interfaces are extremely intuitive, well-designed and easy to navigate, logically follow the business process flow, and provide real meaningful help when needed.  Other application user interfaces are extremely difficult to navigate, are not intuitive, do not follow a logical business process flow, and offer little or no meaningful help. And sometimes in these difficult user interfaces, not only has the location of the steering wheel been moved to a totally unsuspecting location, but its appearance has been changed so that, even when I see it, I do not even recognize it as being the application’s steering wheel.

A well-engineered user interface is no accident.  It doesn’t just magically happen.  It must be woven into the fabric of the design and the code; and it should never be shoe-horned into the application as an after-thought.   It takes a lot of up front planning, designing, testing, functional effort and technical effort to produce a really good application user interface.  And yes, designing, building, testing, and implementing a good user interface for your application will extend the delivery time of whatever it is that you are building.

Why is a well-designed and ergonomic user interface so important?  You could have built the best application ever developed.  But if it is unusable, it will never get very far.   Countless hours are lost every day as thousands of frustrated users spend extra time and effort wrestling with poorly designed user interfaces, rather than focusing on their jobs.  And when the frustration levels reach a certain trigger point, the users will seek out and find alternative ways to perform their duties.

Here are a few examples of some very interesting user interface experiences that I have personally encountered.

Let’s Punish the User

I thought we had gotten over this one a long time ago, but I was bitten by it recently while I was completing a web form.  Here is the scenario:

I am completing a screen form containing several fields, and I make an entry error in one of the fields.  When I try to submit the data, the application rejects the submission, and sends me a message indicating where the error field is, and then clears all of the fields on the screen and makes me re-enter all of the data again.

Bad User!  And, because you made a mistake, you get to start the entire form all over again …   and again … and again … and again …

When Help is not Help

I am especially fond of this one.  I need help, so I look for the HELP ICON.  I find the HELP ICON (because it looks like a help icon should look, and it is located where a help icon is typically located … well, sometimes it is) and I press it.  The anticipation of actually getting the help I need is overwhelming.     And the help information finally appears on my screen and reads something like this:

The exact details of what you want to know are located somewhere in our behemoth website.   Now, we could tell you exactly where and actually help you to solve your problem very quickly,    but we’re not going to, because then you won’t have spent hours wandering aimlessly through our website’s colossal labyrinth.  And while you are in there, don’t let the Minotaur get you!   Please go to www.needleinthehay.com.

Well, at least they said “please”.

The Errant Error Message

I can’t tell you how many times I have made a data entry mistake, only to be “helped” by this not-so-helpful error message:

Incorrect Entry

This is irritating.  If I don’t know why it is incorrect, I can’t fix it!  If the application is intelligent enough to determine that the entry is incorrect, it certainly is intelligent enough to tell me which of its check rules were violated (in plain user-oriented language please).

I really like this one:

Program Error 1776

This truly tells me everything I need to know!!!  Maybe the developer knows what error 1776 is, but I don’t.  Maybe I can find the meaning of error 1776 at www.needleinthehay.com. (I hope the answer is not too close to the Minotaur’s lair.)

Hey developer – can I have your phone number so I can call you every time I see one of these messages?  Maybe then you will get the hint and stop the madness!

I’ve fallen in and I can’t get out

Sometimes, I manage to navigate to a screen that has no obvious means of escape.  So I’m trapped until I can figure out the top-secret mouse click or keystroke combination.  Should I place the blue orb on the green pedestal?  I guess I really should have left a breadcrumb trail.  Or maybe the developer was a ZORK player!  Let’s try XYZZY!!!

Just Give me a Good Example

Like pictures, a good example is worth a thousand words.  Good examples are worth their weight in gold.  A really good example can get me to where I want to be – understanding what I need to know – much more quickly than an entire chapter of words.  Good examples are hard to come by.  They require lots of thought and planning.

Here is an example of a bad example:

2  @  2   =  4         where ‘@’ is some arithmetic operation.

OK – did we add or multiply here?  Who knows?  This is totally ambiguous, and a bad example.

Here is an example of a better example:

2  @  3   =  5         where ‘@’ is some arithmetic operation.

It is obvious here that the “arithmetic operation” was addition.

The Cluttered Screen

One of my favorite games as a child was the type of puzzle where a group of objects is hidden somewhere in a very busy picture.  The names of the hidden objects are listed on the side of the page, and the object of this type of puzzle is to “find the hidden objects” that are on the list.

Some user interface designers and builders are very adept at hiding items on their incredibly busy user interface screens.   Perhaps in their youth, like me, they liked to play with “find the hidden objects” puzzles also.  The problem here is that they are designing “find the hidden objects” puzzles into their user interfaces!  Is it really necessary to squeeze in every data field and push button onto a single screen?  Is the perception here that a single screen, no matter how extremely busy, is somehow easier to use?

My end-user work instructions tell me that if I push the “show details” button, I will then know the meaning of life, the universe, and everything.  But where is the “show details” button?  I am really good at playing “Where’s Waldo”!  Just give me an hour or two and I will find it!

The Plethora of Scattered Screens

The opposite of the tightly packing everything onto a single cluttered screen is the scattering of the user interface among way too many screens.  It becomes a memory exercise to remember the user interface elements that I have already touched, and a scavenger hunt to locate the interface elements that I haven’t found yet.  Let’s see … what was that part number that I entered on screen 152?  Or was it on screen 376?  And where do I go from here to get the order number?  Does anyone have a roadmap that I can follow?

Maybe if we could consolidate the enormous number of screens down to a few that represented logical work flow groupings, I could spend less time screen hopping and be more productive.

Let’s Get Colorful and Fontsy

They use several different font styles, sizes, and colors.

I suppose that’s OK, isn’t it?  It really wasn’t too difficult to read the sentence above, was it?

Consistency is Key

Let’s examine the rental car scenario that I posed at the beginning of this blog.  The reason that I am able to quickly enter the car and drive away is because a recognized user interface standard has been established and consistently applied to all automobile makes and models.  Among the various cars that I am faced with, the gas pedal always looks like a gas pedal, and is always located on the driver’s side floor to the right.  Likewise for the steering wheel, the brake pedal, and most other major user interface elements found in a car – they all adhere to standards of look and feel and placement.

It should not be too surprising then, that the not-so-ergonomic user interfaces that I have encountered do not even recognize that a standard might even exist. The really difficult user interfaces mix things up, and change the appearance, behavior, and location of controls from screen to screen.

Summary

The ergonomic user interface – the one that is intuitive, provides real help when needed, follows an organized logical workflow, and is easy to use are important components of a really great application that end users will want to use because it actually will be the best way to do their job.

Here are a few guidelines that I use whenever I build a user interface:

1)      Totally and completely understand what the business requirements are, and what the user needs to accomplish.  This is the first and most important step to building a good user interface.

2)      Design the user interface to follow the workflow.

Organize and logically group functions together.

3)      Determine if standards exist and consistently use them everywhere.

Position controls in expected and predictable locations.

4)      Give feedback to the user in a timely manner.

Don’t wait until screen 3 to inform the user that there is a problem back on screen 1.

5)      Craft meaningful error messages in user-oriented language.

Temper the messages appropriately – if everything is hot, then nothing is hot.

6)      Make sure that the “help” you provide really helps.

Ask the user to confirm before deleting anything.

Use list of values help whenever possible.

7)      Use default values whenever possible.

This avoids potential data entry errors and makes the user interface more efficient.

For example, if I work in warehouse 2, I should not have to enter that information every time I ship an item.

Display default values in pre-populated, grayed-out fields; but add a pushbutton to allow for the rare occasion when the user must override the default value.

8)      Keep training documents current.

As a developer, you may not own the training documents – but someone does.

There is nothing more frustrating than a training document that does not match the actual program behavior screen for screen and mouse click for mouse click.

I hope that this blog can provide some useful hints for building effective and efficient user interfaces.  Please reply to this blog if you have any most interesting user interface stories that you would like to share with our readers.