Siebel to SAP R/3 Integration Approaches

I was recently evaluating various approaches to integrate Siebel and SAP R/3 systems especially using Oracle SOA Suite. Siebel already provides a Siebel EAI Connector for SAP R/3. This is a tight coupling between Siebel and SAP R/3. If you are planning to use Oracle SOA Suite, it provides the iway adapter for both Siebel and SAP R/3 end systems which makes it possible to decouple the 2 end systems and integrate them. Both approaches are explained below.

Siebel EAI Connector for SAP R/3

The Siebel EAI Connector for SAP R/3 provides connectivity using BAPI and IDOC transport adapters, and predefined business processes. Using the connectors, you can exchange customer, order, and product information between a Siebel application and SAP. This leverages Siebel Workflow and Business Service data maps which are the transformations for the data entities like (Account in Siebel to Customer in SAP or Product in Siebel to Material in SAP). There are some pre-built integrations between common business processes and it also allows custom integrations to be built.

More details can be found in below link:
http://download.oracle.com/docs/cd/B31104_02/books/PDF/ConnSAP.pdf

Integrating using Oracle Siebel Adapter

We can also leverage Oracle Fusion Middleware and the available iway adapters for Siebel and SAP end systems. Ensure that the Siebel Business Services (WSDL) or Business Objects (XSD) are available from Application Explorer for WSDL/XSD generation. Similarily ensure that the SAP BAPIs/IDocs are available in Application Explorer for WSDL/XSD generation. After that you can create the fusion processes to integrate the 2 end systems.

Below is a screenshot from Application Explorer once you create a target and connect to Siebel system.

More details about this approach can be found in below link:
http://download.oracle.com/docs/cd/E14571_01/doc.1111/e17056/intro.htm#i1013615

I have covered details about SAP integration with SOA Suite in my earlier posts
Receiving idocs in BPEL
Invoking BAPIs from BPEL

JMS Messages lost after server restart

Recently I came across an issue where JMS messages(unread) were lost from the queue when Weblogic server was restarted. This was kind of strange because Weblogic JMS provides a pretty stable solution around message delivery. Did some more analysis around this and below are the findings.

Weblogic JMS supports message retention in form of both Oracle AQ or an out-of-the-box DB persistence (file based persistence is default). In either case it should not lose message. The messages can be removed from the queues ONLY if:

a.      They are consumed by some process.

b.      If the messages gets expired, the queue handler will remove them

c.      If the delivery mode of messages coming from JMS Provider is Non-Persistent.

Which brings to the question of "What is JMSDeliveryMode ?" JMSDeliveryMode specifies PERSISTENT or NON_PERSISTENT messaging.

• When a persistent message is sent, WebLogic JMS stores it in the JMS file or JDBC Store (database).
• WebLogic JMS does not store non-persistent messages in the JMS database (prefix_wlstore). These messages are guaranteed to be delivered at least once unless there is a system failure, in which case messages may be lost.

You can check the JMSDeliverMode on incoming messages by looking at the JMS Header section.

<mes:WLJMSMessage xmlns:mes="http://www.bea.com/WLS/JMS/Message">
<mes:Header>
<mes:JMSMessageID>ID:xxxxxxx</mes:JMSMessageID>
<mes:JMSDeliveryMode>NON_PERSISTENT</mes:JMSDeliveryMode>
<mes:JMSExpiration>0</mes:JMSExpiration>
<mes:JMSPriority>4</mes:JMSPriority>
<mes:JMSRedelivered>false</mes:JMSRedelivered>
..
..
</mes:Header>

The JMS Provider should set this JMS Delivery Persistence to PERSISTENT mode and send it to consumer (JMS Queues in Fusion). No additional configuration is needed on Fusion– BY DEFAULT. Incase the messages are still being sent as NON-PERSISTENT then Weblogic provides a property for the Queue called "Delivery Mode Override" which can be used to override the delivery mode of incoming messages.

Controlling the Size and Number of OPMN Debug logs generated

Incase you want to control the number and size of the files generated under SOA_HOME/opmn/logs directory, you can do that by adding few start up parameters in opmn.xml. Usually if these are not added, the debug files will grow over a period of time and will run out of space. (use du -csh * under logs directory to see space occupied by each sub-directory).

Usually the directory OC4J_SOA_xxxx which holds the *.out and *.err files consumes the most space. So to control that you can add below startup parameters for automatic recycling of these files.

<ias-component id="SOA" status="enabled">

            <process-type id="OC4J_SOA" module-id="OC4J" status="enabled">

               <module-data>

                  <category id="start-parameters">

                     <data id="java-options" value="-server -XX:MaxPermSize=2048M -ms4096M -mx8192M -XX:AppendRatio=3 -Djava.security.policy=$ORACLE_HOME/j2ee/OC4J_SOA/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doc4j.userThreads=true -Doracle.mdb.fastUndeploy=60 -Doc4j.formauth.redirect=true -Djava.net.preferIPv4Stack=true -Dorabpel.home=/soa/OracleAS_1/bpel -Xbootclasspath^/p:/soa/OracleAS_1/bpel/lib/orabpel-boot.jar -Dhttp.proxySet=false -Doraesb.home=/soa/OracleAS_1/integration/esb -DHTTPClient.disableKeepAlives=true -Dhttp.session.debug=false -Dfile.encoding=UTF-8 -Dstdstream.filesize=10 -Dstdstream.filenumber=10"/>

                     <data id="oc4j-options" value="-out /soa/OracleAS_1/opmn/logs/OC4J_SOA.out -err /soa/OracleAS_1/opmn/logs/OC4J_SOA.err "/>

                  </category>

These 2 highlighted parameters control the size of each file as 10MB and number to 10. Older files are overwritten. Restart the server for changes to take affect.

Weblogic Admin Server Unable to Start after ip change of host.

After changing the ip address of the App Server host (say from xx.xx.xx.xx to yy.yy.yy.yy), the weblogic admin server is unable to start. Following error message is seen in log file.

<Error> <Server> <AdminServer> <DynamicListenThread[Default]> <<WLS Kernel>> <> <> <1305039521705> <BEA-002606> <Unable to create a server socket for listening on channel "Default". The address xx.xx.xx.xx might be incorrect or another process is using port 7001: java.net.BindException: Cannot assign requested address.>

After ip change make sure you have changed references to the ip address in below places:

• If you have used the IP address, instead of the hostname, as the listen address of the WebLogic Server Administration Server. Make sure you change it in config.xml under $MW_HOME/user_projects/domains/domain_name/config directory.
• Also ensure that the /etc/hosts or C:\Windows\system32\drivers\etc\hosts file is modified to point to the new ip address.

Restart the Admin Server and it should start up successfully now.

Using SFTP with Oracle SOA

A common requirement in integration projects is to transfer files in/out of a system in a secured manner. FTP is the usual protocol for transferring files and if additional security is required then SFTP (Secured FTP) is the way to go. In this post I would cover some of the ways FTP adapter can be configured in SOA 10g and 11g to make use of SFTP.

For additional details on FTP adapter configuration you can refer to the Oracle link
http://download.oracle.com/docs/cd/B31017_01/integrate.1013/b28994/adptr_file.htm

SFTP supports couple of authentication mechanisms to ensure additional security on top of the FTP protocol.

• Password authentication
• Public Key authentication

In password authentication, the external site/vendor which hosts the FTP server shares a username/password combination which has to be configured on the SOA server. At runtime when a SFTP connection is attempted, the username/password is made use of for establishing the connection.

Similarly in Public key authentication, a private-public key pair is generated. The public key is shared with the external site/vendor which hosts the FTP server. At runtime when a SFTP connection is attempted, Fusion process will try to match the private key stored locally on SOA server with the public key on Remote FTP Server and do the authorization first before sending/posting the files.

The configuration information in either case is stored in SOA_HOME/j2ee/OC4J_SOA/application-deployments/default/FtpAdapter/oc4j-ra.xml  (SOA 10g) or MW_HOME/Oracle_SOA1/soa/connectors/FtpAdapter.rar/weblogic-ra.xml (SOA 11g).

For Password authentication below are the properties which you need to set (oc4j-ra.xml sample shown below)

<config-property name="host" value="XXXXX"/>
<config-property name="port" value="22"/>
<config-property name="username" value="xxxxx"/>
<config-property name="password" value="xxxxx"/>
<config-property name="useSftp" value="true"/>
<config-property name="authenticationType" value="password"/>

For Public Key authentication below are the properties which you need to set (weblogic-ra.xml sample shown below)

<wls:property>
<wls:name>host</wls:name>
<wls:value>XXXX</wls:value>
</wls:property>

<wls:property>
<wls:name>port</wls:name>
<wls:value>22</wls:value>
</wls:property>

<wls:property>
<wls:name>useSftp</wls:name>
<wls:value>true</wls:value>
</wls:property>

<wls:property>
<wls:name>authenticationType</wls:name>
<wls:value>publickey</wls:value>
</wls:property>

<wls:property>
<wls:name>privateKeyFile</wls:name>
<wls:value>path of private key file</wls:value>
</wls:property>

Apart from above configuration steps incase of Public Key authentication we can follow below additional steps to generate the private-public key pair and do corresponding setup for that.

1. On Remote FTP Server ensure that /etc/ssh/sshd_config has below parameters set
              RSA Authentication Yes
              PubKey Authentication Yes
2. On SOA server, generate the Public/Private Key pair using below command
            ssh -keygen –t  rsa
3. Once the public and private key are generated make a note of file path, file name etc.
4. Then copy the public key content to the Remote FTP Server . Login as the account with which the FTP has to be performed and put the public key content into file ~/.ssh/authorized_keys.
5. For modifying the weblogic-ra.xml in SOA 11g, extract the file from the FtpAdapter.rar file and after making changes repackage it using command (jar cvf FtpAdapter.rar .)

Incase you run into errors like below work with your network administrator to unblock the port 22 at firewall.
sftp xxxxx
Connecting to xxxxx...
ssh: connect to host xxxx port 22: Connection refused
Couldn't read packet: Connection reset by peer

Thats it ! Now you should be able to securely transfer files..

Configuring FTP Adapter in SOA 10g Cluster for High Availability

If you have a 2 Node cluster environment for SOA 10g, it is essential that the FTP adapter is configured in an Active/Passive manner. Otherwise you may run into situations where both nodes are trying to read the same file from the remote FTP server and this will lead to duplicate files entering the system.

The reason for duplicates is because FTP adapter on each node will maintain its own control file(locally) where it stores the last read time of the file. So if a file has already been read by Node1, there are chances that after server restart the same file may be picked by Node2 as well because the time maintained in control files is out-of-sync on both nodes.

There are couple of good articles published by Oracle for this specific configuration. The base idea of the solution is to have the control file on a shared folder which is accessible by both nodes.

Step1:
=====
Make sure the FTP adapters are configured in Singleton mode i.e BPEL clusterName value specified in $ORACLE_HOME/bpel/system/config/collaxa-config.xml should be different from the Adapters clusterGroupId property set inside bpel.xml of the BPEL project.

Also the Multicast host and port in jgroups-properties.xml file should be same on both nodes.

Step2:
=====
Once step1 has been ensured, you should create a folder on a shared file system. Either use an external share storage or you can create a shared directory on one node and use NFS share to mount it on the other node. Either ways make sure the folder has write permissions from both nodes. This folder will store the control files.

Next backup and edit the $ORACLE_HOME/bpel/system/service/config/pc.properties file on each node and set the below property oracle.tip.adapter.file.controldirpath to the shared folder name

Restart the servers for the change to take effect and test the scenario.

ORABPEL-05215 Error while deploying BPEL processes

Was working on some SOA 10g deployments recently and came across this error.

ORABPEL-05215

Error while loading process. The process domain encountered the following errors while loading the process "XXXXXX" (revision "1.0"): null.

If you have installed a patch to the server, please check that the bpelcClasspath domain property includes the patch classes.

Tried looking into the opmn logs and nothing additional was mentioned there. This was a simple BPEL process and didn't have any embedded java code. So definitely wasn't an issue with classpath settings.

On further checking found that the ORACLE_HOME and PATH used by my build scripts were pointing to Jdev 10.1.3.5 directory. However the SOA Suite version was 10.1.3.4. 

Downloading 10.1.3.4 Jdev and pointing ORACLE_HOME and PATH to this directory worked ! So when you encounter this error ensure that SOA Suite version matches your JDeveloper version.