Wednesday, July 28, 2010

Dynamic PartnerLinks in BPEL

I recently came across a requirement where I had to integrate the SAP R/3 system with a 3rd party vendor using SOA 11g. The requirement simple...
  1. Schedule the BPEL to call the SAP BAPI
  2. Transform the results from BAPI and call the vendor web service. We do not have the target web service yet !

Wait...did it mention "do not have the target web service yet". My experience so far in BPEL was with static partner links where I was aware of the target WSDL location which I could use for the configuration. But here I didn't have that information available. Fortunately I was also provided another vital piece of information : an XML Schema (.xsd) which was the data model for the vendor WSDL. That helped me to go ahead and complete the integration.

I am sure many developers have come across similar scenarios where the end service(WSDL) we are going to call is unknown during design/development phase. So it becomes a challenge to provide an endpoint to the BPEL process or configure the partner links. And this is where dynamic partnerlinks come in handy. They use WS-Addressing feature of BPEL PM and enable us to pass values to the dynamic end points at runtime.


A very good example of this has been provided in the BPEL Cookbook on OTN ,where they have explained Dynamic PartnerLink usage in the Sample Loan Approval BPEL flow.


In this post I would like to highlight some key points while configuring dynamic partnerlinks in BPEL. The most important point to be remembered when using dynamic partnerlinks is that you need to ensure each process that you wish to call has to be based on exactly the same WSDL. The message types, namespaces, port types, bindings etc...everything have to be same just the endpoint url will be different.

If the above is taken care of then configuration wise things are simple :

a. First create the patnerlink with a template WSDL file.

b. Then add the following namespace to your bpel process
xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing"

c. Import the schema from above namespace URL to your project.

d. Based on this namespace, create an “EndpointReference” variable.

<variable name="partnerReference" element="wsa:EndpointReference"/>

e.Initialize the endpoint-reference by copying the following xml-fragment to the endpoint reference.

<EndpointReference xmlns="http://schemas.xmlsoap.org/ws/2003/03/addressing">
<Address/>
</EndpointReference>

f. You can use a bpel preference to store the end point URL so that at runtime this can be changed easily. Copy the determined endpoint url into the address node of the EndpointReference variable.

<copy>
<from expression='ora:getPreference("EndPointURL")'/>
<to variable="partnerReference" query="/wsa:EndpointReference/wsa:Address"/>
</copy>

g. Finally before invoking the partnerlink copy the Endpoint Reference to the partnerlink

<copy>
<from variable="partnerReference" query="/wsa:EndpointReference"/>
<to partnerLink="DummyService"/>
</copy>

Saturday, July 24, 2010

Oracle SOA Suite integration with Siebel

I will be shortly starting work on SOA Suite 11G integration with Siebel and was doing some initial study on the various ways to integrate with Siebel system.

  • For inbound integrations, Siebel CRM does support native webservices and exposes a number of functions as webservice end points (also know as Application Service Interfaces or ASIs). It provides tool for generating these WSDLs which can be consumed by SOA Suite components like BPEL.
  • For outbound integrations, we can use JMS and Oracle AQs. Basically Siebel publishes the message to JMS queue and a consumer picks up the message to process it onwards.

A couple of good reference posts from OTN are:

Will be sharing more details about Integration with Siebel systems in coming weeks...stay tuned.

Wednesday, July 21, 2010

SAP Adapter : JCO_ERROR_RESOURCE Connection pool is exhausted

I came across an interesting issue recently faced by one of my colleagues. He was trying to integrate between MSSQL server and SAP R/3 system using SOA 11G. He used to get a batch of order data from MSSQL server and had to post these to SAP BAPI and receive order number in response. Things were working fine as long as the batch size was small. But when he started getting batch size >1000, the SAP adapter started throwing errors.

Exception occured when binding was invoked. Exception occured during invocation of JCA binding: "JCA Binding execute of Reference operation failed due to: JCA Binding Component connection issue. JCA Binding Component is unable to create an outbound JCA (CCI) connection. The JCA Binding Component was unable to establish an outbound JCA CCI connection due to the following issue: java.lang.IllegalStateException: Problem activating adapter. (java.lang.IllegalArgumentException: com.ibi.sap.SapAdapterException: com.sap.mw.jco.JCO$Exception: (106) JCO_ERROR_RESOURCE: Connection pool c22bbdb28dd27119d92c36c8ce75ad9e_p1 is exhausted. The current pool size limit (max connections) is 10 connections.). Check logs for more information ".

The default connection pool size for SAP adapter is 2 and this can be changed from the Application explorer. As can be seen above the connection pool size was increased to 10 but still this had failed. Basically when 10 concurrent threads are running and 11th thread tries to fetch the connection and unable to get it, that particular instance of BPEL fails with above error.

We tried a couple of things:

1. Tried controlling the number of records being polled from SQL server using distributed polling feature and controlling the number of records read per transaction. However incase of high load there were some BPEL instances which were still failing.

2. Second option tried was instead of directly connecting to the SAP R/3 system we tried connecting to a Load balancer infront of the SAP servers. This helped in distributing the load across SAP servers. For doing this configure the SAP target as shown below:



Fill the fields marked with *. Message Server is the load balancer name. Also note you need to edit the /etc/services file using root user and add below entry.
sapms<R/3 name> <port>/tcp

This ensured that the message server will share the load across SAP servers at runtime.








3. Incase the requirement is to control the concurrency on SAP side and allow only a single connection i.e prevent   parallel BPEL instances submitting concurrent requests to SAP R/3 we can create a Singleton BPEL process. More details on how to do this is well explained in Matt's Blog

Monday, July 19, 2010

Configuring MDS in AIA 3.0

AIA 3.0 or AIA 11gR1(rechristened name) brings along with it a lot of changes in comparison to earlier AIA 2.x releases, the most prominent being the way to access AIA artifacts. I have worked on AIA FP 2.x and accessing artificats like WSDLs , XSDs inside the code from Enterprise Object Library or Enterprise Business Service Library was done using http urls eg: http://hostname:port/AIAComponents/EnterpriseObjectLibrary/core/EBO/Invoice/V2/InvoiceEBO.xsd. When migrating between instances i.e Dev to QA to Prod we had to make sure the deployment scripts changed the partner link bindings in bpel.xml to change the http urls.


Now when I started development on AIA 3.0 the first thing I wondered was how to access these artifacts inside code because the AIAComponents folder was no longer hosted on http. I hadn't explored the new concept of MDS(Metadata Service) yet and hence thought of using a workaround. I created a war file out of the AIAComponents folder jar -vcf AIAMetaData.war $AIA_HOME/AIAMetaData/AIAComponents
I then deployed this war file into my weblogic server as an application and Enabled Index Directory. Now I was able to access the entire AIAComponents folder from URL again. 


However I started to think about the maintenance aspect of the above approach. If I had to modify any artifact or add a new one later then I had to place the file in relevant location under AIAComponents, regenerate the war and redeploy it to WLS. Secondly during code migration between instances I had to repeat this task on each server and also make sure my deployment scripts ensure proper URL replacement. Definitely doesn't sound like an Oracle recommended approach which made me search for better alternatives and I then stumbled across the concept of centralised repository called MDS. The MDS is metadata store which stores all AIA artifacts under AIAComponents folder and these are accessed inside code using oramds protocol.


In this post I would like to cover the details of how to setup Jdeveloper to create SOA-MDS connection, how to go about accessing the artifcats inside code and how to update/add artifacts into MDS.


Setup in Jdeveloper :


1. Create a database connection to connect to the repository with either DEV_MDS or equivalent user (prefix_MDS).
2. Then from resource palette create a new SOA-MDS connection. Make sure it uses the DB connection in step1 and also the partition is soa_infra.
3. Once completed you should be able to browse and choose the required AIAComponent files from MDS. These will be shown under /Apps/MetaData.
4. Next you have to update the adf-config.xml file in your Jdeveloper, application home (directory where .jws file exists). This will be used at design time to access files from MDS using oramds protocol. This is located in directory named .adf/META-INF.
    
  Make sure you add the below sections:
      <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_2" path="/apps" />
      </metadata-namespaces>

     <metadata-store-usage id="mstore-usage_2">
-        <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
         <property value="DEV_MDS" name="jdbc-userid" />
         <property value="welcome1" name="jdbc-password" />
         <property value="jdbc:oracle:thin:@hostname:port:sid" name="jdbc-url" />
         <property value="soa-infra" name="partition-name" />
     </metadata-store>
     </metadata-store-usage>

Thats it! You can now start accessing the artifacts like WSDLs, XSDs etc using oramds. Eg: oramds:/apps/AIAMetaData/AIAComponents/EnterpriseObjectLibrary/core/EBO/Invoice/V2/InvoiceEBO.xsd

How to update/add artifacts into MDS:

These have to be repeated every time new/modified files are to be loaded to MDS.
1. source AIA_HOME/aia_instances/instance_name/bin/aiaenv.sh

2. Modify the AIA_HOME/aia_instances/instance_name/config/UpdateMetaDataDP.xml

<?xml version="1.0" standalone="yes"?>
<!-- Do not modify the contents of this file manually -->
<DeploymentPlan component="Metadata" version="3.0">
        <Configurations>
                <UpdateMetadata wlserver="fp" >
                        <fileset dir="${AIA_HOME}/AIAMetaData">
                             <include name="AIAComponents/EnterpriseObjectLibrary/Core/Custom/EBO/Invoice/V2/*"/>
                        </fileset>
                </UpdateMetadata>
        </Configurations>
</DeploymentPlan>

NOTE:
 The path in "include" tag has to be relative to what is given in "fileset" tag. 

3.Run the below command to load the files to MDS from AIA_HOME/Infrastructure/Install/scripts

ant -f $AIA_HOME/Infrastructure/Install/AID/AIAInstallDriver.xml  -DDeploymentPlan=AIA_HOME/aia_instances/instance_name/config/UpdateMetaDataDP.xml -DPropertiesFile=AIA_HOME/aia_instances/instance_name/config/AIAInstallProperties.xml

Friday, July 16, 2010

Receiving SAP iDocs in Oracle BPEL process

This post is an extension to my earlier post on invoking-sap-bapis-from-oracle-bpel . Here I will be covering detailed steps involved in receiving SAP iDocs into Oracle BPEL process. Unlike Service adapters in case of BAPI we will be dealing with Event adapters incase of SAP iDocs. We can use events generated in SAP system due to some DB activity/application process to trigger a BPEL process. For doing this we need to create a SAP channel in Application explorer and select this channel to create a JCA Inbound WSDL(Event Notification – BPEL receive). Few things to be noted about SAP Channel:
  • It only applies to J2CA configuration
  • You must create channels for every different event object. Also each channel must be associated with a unique SAP R/3 program id.
  • After creating/modifying channels weblogic server must be restarted for the changes to take effect at runtime.
                

In above screenshot showing Channel creation steps, note the value of Program Id of server (BPEL01). This will be registered in the SAPGUI as a RFC destination. This helps SAP to direct the events to the target system. For detailed steps on how to register Program Id on SAPGUI please check the Appendix section Configuring SAP R/3 Outbound Processing from SAP R/3 User's Guide for Weblogic server


Once the SAP Channel has been created under Events section goto the Adapters section and generate the Inbound WSDL for your iDoc (you will have to use the SAP channel while creating inbound WSDL). This process generates 3 files: *.wsdl, *.jca and *.xsd under the $SOA_HOME/soa/thirdparty/ApplicationAdapters/wsdls directory.


Next copy these files to your local machine and in your Jdeveloper create a BPEL process with a third party adapter service which uses the *.wsdl and *.jca files for configuration. Complete the BPEL process to receive the idoc from the third party adapter service and do further processing.


Restart your Weblogic server and now when a transaction/action is performed on SAP server like generating a sales order that should trigger the BPEL process.

A few good reference posts...

Its really great to see people from Oracle SOA community sharing their views and experiences in blogs on a regular basis. There are many articles which have helped me understand issues/configurations/concepts during development and I would like to share a few I found useful...

SOA 10g & 11g
Transaction Control in 11g - Part1
Transaction Control in 11g - Part2
Email Notifications in 11g
Configuring MSSQL Server with SOA 11g
Sending Email notifications based on error messages in Weblogic Server
Passing BPEL variables into XSLT

Saturday, July 10, 2010

Some FAQs about SOA based integration with Oracle Ebusiness Suite

I started my SOA journey with Oracle products like IREP, EBS Adapters and Integrated SOA Gateway and having interacted with multiple Oracle Apps customers who were planning to venture into the world of SOA, I thought of putting some frequently asked questions about SOA integrations with Oracle Apps. This post will be more helpful for beginners rather than SOA practioners already involved with these kind of integrations.


1. Do we need to install Oracle Fusion middleware separately and then integrate with R12 or is it included in R12 ?? Also do we need a separate license for that ?

Incase you are planning an upgrade of your existing Ebusiness version from 11i to R12 and also want to integrate with Oracle Fusion Middleware(OFMW), please note that you have to separately install OFMW. The integration with Ebusiness is done using EBS Adapters. The separate installation of OFMW contains the SOA Suite components of BPEL Process Manager, ESB server, adapters etc which arent shipped with the scaled down version of R12 techstack (Oracle application server). And Yes OFMW is licensed separately.

2. Do we have any training links for learning more about SOA Integrations with EBS ?

3. What is Application Context and how is it set for EBS Adapters ?
In order to support application-level security to authorize use of its APIs,the Oracle E-Business Suite Adapter allows the use of a header variable in the BPEL process. At a minimum, the header variable must contain the application username and responsibility to use to pass application level authorization. Additionally it can contain the organization to use for the username and responsibility, depending on the version of E-Biz and whether E-Biz is configured to use multiple organization units. Since a BPEL process can invoke multiple PL/SQL using this header variable, this is usually declared as a global variable of the header type and assigned the username and responsibility early in the BPEL process.
  • When we create a partner link service,JDeveloper creates a WSDL named procedureAdapterHeader.wsdl that defines the message type for the Oracle Applications header, so we only need to create a global variable of this type.
  • Then, an assign activity is added to the BPEL process that allows the global header variable to be initialized for use later.
  • Then, when the Invoke activity in the BPEL process is added for adding a task in E-Biz, we select the variable storing our E-Biz credentials to use in the API using the “Browse” flashlight icon. That’s it! And now we can access the APIs we select without errors from Oracle.



Also please note that JDeveloper does embed a standard username and responsibility in the partner link WSDL it creates for E-Biz APIs. By default it will embed a username of “sysadmin” and responsibility of “System Administrator” in the
in the WSDL, so if that works fine for your installation and the APIs you need to access, then you won’t need to even use a header variable to setting your application context explicitly in the API call.




4. What is WF_BPEL_Q and where is it used ? Do we have a procedure or concurrent program to clean up/purge records from table WF_BPEL_QTAB?


WF_BPEL_Q is an advanced queue which comes into picture when we are doing integration with Oracle Apps business events system. Apps is preconfigured with numerous Business Events and each Business Event represents a ready to use Integration or extension point.When a user from Jdeveloper creates/configures an EBS adapter as a partner link and selects the APPS event to be listened , following activities happen in backend:
  • WF_BPEL_QTAB queue table created, if it does not exist.
  • WF_BPEL_Q queue created, if it does not exist.
  • Start WF_BPEL_Q
  • WF_BPEL_QAGENT , if it does not exist
  • Create and start wf_bpel_qagent listener, which listens on WF_BPEL_Q.
  • Event subscription for that particular event in WF_EVENT_SUBSCRIPTIONS table , if it does not exist.Note that for a particular event , only 1 event subscription is created irrespective of number of BPEL processes (you can have more than 1) subscribing to it.
  • AQ Subscriber to uniquely identify the BPEL process consuming the event. This subscriber is created in APPS.AQ$_WF_BPEL_QTAB_S table. This subscriber is needed since consuming business event is dequeue operation and each BPEL process consuming the event should identify itself with unique subscriber name. So ,say bpel1 and bpel2 subscribe to event1 ,two subscribers would be created in APPS.AQ$_WF_BPEL_QTAB_S table so that AQ knows it has to provide event message to bpel1 and bpel2.
  • When the BPEL process is deployed in the BPEL PM, the listener keeps polling the WF_BPEL_Q with whatever freqeuncy we have defined.
  • Once the corresponding event is raised , it is enqueued into the WF_BPEL_Q and the listener(spawned by BPEL PM) which is listening to this event, would dequeue this and starts the BPEL process.
A common maintenance question is how to purge records from the table WF_BPEL_QTAB if they are huge in number .Or whether there are any seeded concurrent programs or procs doing this job.

Well there are no purge programs which does purging of messages from this queue table. This being an AQ table,it should work like any other AQ where once the retention time is reached the record will be cleaned up by the background DB processes.For WF_BPEL_Q retention time is 86400 seconds.. ie 1 day; message would be retained for 1 day after successful dequeue of the message by all the registered subscribers.

However if there are discrepancies observed in removal of records there can be 2 reasons:

  • Say if subscriber(BPEL Process) A and B are registered to receive message, and only A has dequeued it, the message would sit indefinitely unless B too dequeues it. After both A and B dequeue, message would then be removed.
  • Another reason maybe when there are some wrong BPEL processes( subscribers here) registered for the business event and never dequeued it. In that case you can check out the subscribers table for WF_BPEL_Q i.e aq$wf_bpel_qtab_s and remove all the stale subscribers from it. Removing subscribers automatically removes the messages meant for it.
5. Does EBS adapter support custom PL/SQL APIs created in schemas other than APPS ? Also can Custom objects like PL/SQL APIs or Concurrent Programs be exposed to IREP UI by customers ?

Almost 90% of Oracle Apps customers have some or the other kind of customizations existing in their implementations. They would definitely like to have these custom objects exposed to the IREP UI so that they can be used for integration scenarios.

EBS Adapter does support custom PLSQL APIs created under Apps schema and shows up these in the Application Module Browser screen of Jdev while configuring the adapter. However this support is limited to objects created only under Apps schema and doesnt extend to custom objects under non-Apps schema.

Also starting with Release 12.1.1, we have a standalone IREP parser script being made available to customers (after application of some one-off patch) which lets them expose their custom objects to the IREP UI. This support doesnt exist in IREP of R12 or 11i. To learn more about How to Customize IREP please check the blog
http://blogs.oracle.com/integration_repository_customization

Thursday, July 8, 2010

weblogic.transaction.internal.TimedOutException: Transaction timed out after 299 seconds

I have come across couple of scenarios where my BPEL processes(SOA 11g) fail with timeout errors. The corresponding error messages shown in the soa server diagnostic log of managed server is as follows:

JTA transaction is not present or the transaction is not in active state.

In SOA 10g (OC4J) we could fine tune the timeout settings by modifying transaction-timeout parameter in below files:
$Oracle_Home\j2ee\home\config\transaction-manager.xml
$Oracle_Home\j2ee\home\application-deployments\orabpel\ ejb_ob_engine\orion-ejb-jar.xml

However in SOA 11g (weblogic) this can be set via the Oracle WebLogic Administration Console.
Go to Deployments->soa-infra ->scroll down to EJBs and find "BPELEngineBean" ->Under configuration tab all the way down you find the transaction timeout - set to 300 seconds by default. You can change it here.


Once done Save, Update the soa-infra application and restart the Weblogic server.

Tuesday, July 6, 2010

Modifying BPEL preferences in 11G as compared to 10G

BPEL preferences help a lot in setting dynamic values in BPEL processes. The values for these preferences can be changed during deployment to different servers (DEV,QA, PROD) or post-deployment (where they take effect at runtime without having to redeploy code or restart servers). The way preference values can be modified in SOA 10G is different from the way it can be done in 11G . In this post we will take a look at this difference.

In BPEL 10G the preferences were added to bpel process by adding them into the bpel.xml file. Later these could be changed from BPEL console as shown below :
<preferences>
<property name="SOURCE_CONTACT">XXXXX</property>
<property name="TARGET_CONTACT">XXXXX</property>
</preferences>


In BPEL 11G the preferences are added to bpel process by adding them into the composite.xml file. Later these can be changed by changing their value from EM console as shown below:
<component name="customBPELProcess">
<implementation.bpel src="CustomBPELProcess.bpel"/>
<property name="bpel.preference.customPref">CustomPrefValue</property>
</component>
Open EM Console from http://adminserverhost:port/em

On the left hand navigation bar go to :

Farm_xxx_domain > Weblogic Domain > xxx_domain > right mouseclick and select ‘System MBean Browser’.

Navigate to Application Defined MBeans > oracle.soa.config > Server : soa_server1 > SCAComposite > your_project > SCAComposite.SCAComponent > your bpel_process. Select the Attribute ‘Properties’.

Change the value of your preference and click apply.Run the bpel again to see the new value get used in the process.