Tuesday, August 31, 2010

Date formats in SAP and Inverted date format

People doing integrations with SAP R/3 system must be aware that SAP R/3 takes in/out date fields with format of YYYYMMDD (This is their internal date format). However there is a special date format called Inverted date which is used to store some date fields in SAP tables.

If you work on Currency Exchange Conversions (TCURR table) the Date Field (GDATU) stores date in Inverted date format. So what exactly is this Inverted date format and how can you convert it to normal date format of YYYYMMDD which can be later used in your integration.

Inverted date is nothing but the 9's complement of the real date. In other words the conversion logic will be something as shown below

99999999 – (GDATU) = actual date in yyyymmdd format
Example: 99999999 – 79899168 = 20100831

There are conversion exits in ABAP which does this date conversion like CONVERSION_EXIT_INVDT_OUTPUT -- Conversion exit routine for inverted date (OUTPUT)

However if your SAP BAPI gives you data in inverted date format you can use the logic mentioned above in your BPEL/XSL at Fusion layer to convert it into actual date format.

Monday, August 30, 2010

SOA 11g : EM Console fails with HTTP 404 error ; JspServlet error

After restarting servers (admin and managed) found that EM console isn't coming up and it shows below error either on browser or Admin server log files:
"JspServlet error: Servlet unable to dispatch to the following requested page: java.io.FileNotFoundException: /targetauth/asLogin.jspx"

On further checking the directory targetauth was missing from <MW_HOME>/user_projects/domains/<domain_name>/servers/AdminServer/tmp/_WL_user/emcore/28c293 location. This directory has the asLogin.jspx file.

You can copy this directory from a different domain but that would still give errors like 
"JspServlet error: Servlet unable to dispatch to the following requested page: java.io.FileNotFoundException: MDS-00013: no metadata found for metadata object "/targetauth/asLogin.jspx" 

Solution :
1. Stop the Admin server
2. Rename <MW_HOME>/user_projects/domains/<domain_name>/servers/AdminServer/tmp/ directory to tmp-old
3. Restart the Admin server. The tmp directory will be recreated again.

Open EM Console.It should work now !

Monday, August 23, 2010

Dependent SCA composites fail with "503: Service Unavailable for url" after server restart

We often design composite applications that invoke other composite applications. A typical use case would be AIA artifacts where RequestorABCS calls EBS which in turn calls ProviderABCS.

If these are not configured correctly then after restarting SOA server the one having dependency does not work any more. Basically issue is with the order in which SOA server re-deploys these composites. You may see the below error message in EM console

The composite SalesOrderRequestorABCS (1.0) is not available. This could happen because either the composite has been undeployed or soa-infra has not yet loaded this composite.
[JCABinding] [SalesOrderRequestorABCS/1.0]Unable to complete load due to: Error in getting XML input stream: http://hostname:port/soa-infra/services/default/SalesOrderEBS/SalesOrderEBS_ep?WSDL: Response: '503: Service Unavailable' for url


The recommendations to avoid dependencies between composites are :

1. In the composite.xml of the calling composite have the import refer to an abstract wsdl and
not a concrete one to avoid loading issues.
2. Make sure you refer to the MDS location for common XSDs, WSDLs and deployed composites.

For example in case of RequestorABCS if you are having references to end point URL of EBS(obtained from EM console after deployment of EBS) then make sure you change that to oramds location for abstract WSDL.

FROM
<import namespace="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2" location="http://hostname:port/soa-infra/services/default/SalesOrderEBS/SalesOrderEBS_ep?WSDL" importType="wsdl"/>
<reference name="SalesOrderEBS" ui:wsdlLocation="http://hostname:port/soa-infra/services/default/SalesOrderEBS/SalesOrderEBS_ep?WSDL">
<interface.wsdl interface="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2#wsdl.interface(SalesOrderEBS)"/>
<binding.ws port="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2#wsdl.endpoint(SalesOrderEBS_ep/SalesOrderEBS_pt)" location="http://hostname:port/soa-infra/services/default/SalesOrderEBS/SalesOrderEBS_ep?WSDL"/>
</reference>

TO

<import namespace="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2" location="oramds:/apps/AIAMetaData/AIAComponents/EnterpriseBusinessServiceLibrary/Core/EBO/SalesOrder/V2/SalesOrderEBSV2.wsdl" importType="wsdl"/>
<reference name="SalesOrderEBS" ui:wsdlLocation="oramds:/apps/AIAMetaData/AIAComponents/EnterpriseBusinessServiceLibrary/Core/EBO/SalesOrder/V2/SalesOrderEBSV2.wsdl">
<interface.wsdl interface="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2#wsdl.interface(SalesOrderEBS)"/>
<binding.ws port="http://xmlns.oracle.com/EnterpriseServices/Core/SalesOrder/V2#wsdl.endpoint(SalesOrderEBS_ep/SalesOrderEBS_pt)" location="http://hostname:port/soa-infra/services/default/SalesOrderEBS/SalesOrderEBS_ep?WSDL"/>
</reference>

Similarily if you are referring to ProviderABCS from EBS composite.xml of EBS will look something as below

<reference name="SalesOrderProviderABCS" ui:wsdlLocation="oramds:/deployed-composites/SalesOrderProviderABCS_rev1.0/SalesOrderProvABCSImpl.wsdl">
..
..

Saturday, August 21, 2010

HTTP Binding Adapter in SOA 11g

With the release of SOA Suite 11gR1 Patch Set 2 (PS2)! Oracle has added support for HTTP binding (helpful when we want to do away with SOAP and looking for simple GET and POST). 
I had a requirement recently where I had to integrate with an external vendor which supported HTTP(s) posting of files. So I started looking at ways to post a XML file to the HTTP(s) location. Luckily with Jdeveloper 11g (11.1.1.3) version we have a HTTP binding adapter available which can be made use of.


You can drag the HTTP binding from component palette into your composite and configure it to post XML data to a HTTP(s) endpoint. Incase you are using HTTPS you would require the certificates to be installed on the SOA server if it is server authentication and your certificates need to be installed on vendor side if its mutual authentication. Apart from that it should be similar to HTTP.


For more details on how to configure the HTTP binding adapter wizard please follow the below blogpost from Edwin Biemond where he has illustrated this with screenshots.
Http Binding Adapter Configuration In Soa Suite 11g

Friday, August 20, 2010

Unit testing Oracle SCA Composites

With Oracle SOA 11g we have a unit testing framework available for testing the individual components. This enables us to test the components in isolation without any external dependencies like availability of target services. This in turn helps during integration testing as we already have well tested low level components.

The framework allows :
1. Definition of Tests, assertions and emulations via Jdeveloper
2. Deployment of test cases along with deployment of SCA composite
3. Execution of test cases from EM console.

A typical testcase may contain an initiated message, an emulation which defines message/error
returned by an external reference/service and/or an assertion which is primarily used for comparison against expected results.

Below I have shown an example where I tested a component (AIA RequestorABCS) which had a DB adapter as initiator of process and called an external target service like SAP R/3. Now for unit testing this piece I used an initiator message in my test suite and an emulator to define a response message from SAP R/3. This way I don't have to depend on availability of actual target system at time of unit testing my ABCS code.

The test suite can be created from Jdeveloper :







It creates a copy of the composite.xml as shown below.














Once completed, deploy the test suite along with normal SCA deployment from Jdev/ant scripts and execute it from EM Console as shown below.

Count(bpws:getVariableData(....)) results in SelectionFailure in SOA 11g

This perhaps will be a short post about a known issue on SOA 11g. I was trying to use count XPATH function on getVariableData to find count of dataarea in EBM. This worked fine in SOA 10g but in 11g it failed with SelectionFailure.

Found that this is a known issue in SOA 11g.
Oracle Release Notes for 11g

16.1.8 getVariableData Function Throws a selectionFailure if the Result Node Set is of a Size Other Than One During Execution

According to the Business Process Execution Language for Web Services Specification, if the the locationPath argument of the bpws:getVariableData() function selects a node set of a size other than one during execution, the standard fault bpws:selectionFailure must be thrown by a compliant implementation.


To get around this issue simply use ora:getNodes() which acts as an alternate to getVariableData to obtain the same functionality. getVariableData is striclty for one node result, where as getNodes would return a nodeset.

Wednesday, August 18, 2010

BPEL Scheduling: Quartz and Database

A common requirement in many SOA projects is to schedule BPEL processes to run at specific time. Now there are various scheduling mechanisms which can be used for doing this. In this post I will be discussing about 2 approaches which I have used :
1. Using Open source Quartz scheduler.
2. Scheduling your process from Database using dbms_scheduler apis.

In Oracle BPEL open source job scheduler, Quartz is implemented as part of java class called DefaultSchedulerCalloutImpl. Incase your BPEL process is triggered by a file/ftp adapter you can modify the bpel.xml file to make use of the quartz scheduler.

<activationAgents>
<activationAgent className="oracle.tip.adapter.fw.agent.jca.JCAActivationAgent"
partnerLink="ReadFile" heartBeatInterval="10">
<property name="portType">Read_ptt</property>
<property name="schedulerCallout">DefaultSchedulerCalloutImpl</property>
<property name="endpointScheduleOn">0 0 8 * * ?</property>
<property name="endpointScheduleOff">0 0 10 * * ?</property>
</activationAgent>
</activationAgents>

The above piece of code is direction to the activation agent to activate the BPEL process between 8 and 10 am everyday and heartbeat interval of 10 seconds indicates how frequently the process is active for polling.

The 2nd approach is a simple solution based on Oracle dbms_scheduler api. Basically you need to have your BPEL process read from a DB table which gets populated by the DBMS job at a particular frequency. I am sharing some sample code below which can be reused after making modifications.

Create table bpelschedulertable (process_name varchar2(20));

Script to create the scheduler job.

/* Drop Jobs if they exist */
BEGIN
DBMS_SCHEDULER.DROP_JOB(JOB_NAME => 'BpelScheduledJob');
END;
/

/* Create Scheduled Job */
BEGIN
 DBMS_SCHEDULER.CREATE_JOB(
 JOB_NAME => 'BpelScheduledJob',
 JOB_TYPE => 'PLSQL_BLOCK',
 JOB_ACTION => 'BEGIN INSERT INTO bpelschedulertable (PROCESS_NAME) VALUES (''samplebpel'');COMMIT;END;',
 START_DATE => TO_DATE('2010-08-18 12:00:00','YYYY-MM-DD HH24:MI:SS'),
 REPEAT_INTERVAL => 'FREQ=MINUTELY;INTERVAL=15',
 COMMENTS => 'Scheduled for every 15 minutes');
 DBMS_SCHEDULER.ENABLE(NAME => 'BpelScheduledJob');
 END;
/

That's it ! Your BPEL process(samplebpel) should trigger every 15 minutues now.

Scripts to monitor/manage the scheduled job.

select * from dba_scheduler_jobs ;

/*Command to enable job*/
Begin    
dbms_scheduler.enable('BpelScheduledJob');  
end;

/*Command to disable job*/
Begin
Dbms_scheduler.Disable('BpelScheduledJob');
end;

/*Command to change the repeat interval of the job*/
Begin
Dbms_scheduler.set_attribute(name=>'BpelScheduledJob',attribute=> 'REPEAT_INTERVAL', value=> 'FREQ=HOURLY;INTERVAL=1');
end;

Friday, August 6, 2010

Common SOA composite deployment errors from Jdeveloper

In this post I would like to share few deployment errors I faced with SOA composites from Jdeveloper and their resolutions. This will be helpful to people starting work on SOA 11g.

Error 1 :

Preparing to send HTTP request for deployment
Creating HTTP connection to host:xxxx, port:yyyy
Sending internal deployment descriptor
Sending archive - sca_HelloWorld_rev1.0.jar
Received HTTP response from the server, response code=500
Error deploying archive sca_HelloWorld_rev1.0.jar to soa_server1 [xxxx:yyyy]  
HTTP error code returned [500]
Error message from server: 
Unknown Host
Description: Unable to locate the server named "xxxx" --- the server does not have a DNS entry.  Perhaps there is a misspelling in the server name, or the server no longer exists.  Double-check the name and try again.

OR

Error 2 :

Problem in sending HTTP request to the server. Check standard 
HTTP response code for 502
No error message is returned from the server.
#### Deployment incomplete. ####
Error deploying archive file:/C:/JDeveloper/mywork/Sample/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar
 (oracle.tip.tools.ide.fabric.deploy.common.SOARemoteDeployer)


Solution :

1. Go to weblogic console. Server->click on managed Server and select configuration->General tab
    There in listen address provide the listen address to the ip address and restart the server.

2. Go to jdeveloper 11g and disable the proxy settings there.
    Go to tools->Preferences->Web Browser and proxy and disable the use proxy http server

3. Exit and start the jdeveloper once again.


Error 3 :

Received HTTP response from the server, response code=500
Error deploying archive sca_HelloWorld_rev1.0.jar to soa_server1 [xxxx:yyyy]  
HTTP error code returned [500]
 Error message from server:


javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:333)
 at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
 at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:94)
 at java.security.AccessController.doPrivileged(Native Method)
 at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:313)
 at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)
 at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:138)
 at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:70)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
 at oracle.dms.wls.DMSServletFilter.doFilter(DMSServletFilter.java:326)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
 at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3592)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
 at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2202)
 at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2108)
 at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1432)
 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
 at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Caused by: java.lang.OutOfMemoryError: Java heap space


Solution :

The JVM Memory arguments are in the file " setDomainEnv.sh " in the folder $MW_HOME/user_projects/domains/<domain_name>/bin

You can open the file in any Text Editor and add the below line :-
     USER_MEM_ARGS="-Xms512m -Xmx1024m" (alter as per need)


Error 4 :

Preparing to send HTTP request for deployment
Creating HTTP connection to host:xxxx, port:yyyy
Sending internal deployment descriptor
Sending archive - sca_HelloWorld_rev1.0.jar
Received HTTP response from the server, response code=401
Problem in sending HTTP request to the server. Check standard HTTP response code for 401
Error deploying archive sca_HelloWorld_rev1.0.jar to soa_server1 [xxxx:yyyy]  
HTTP error code returned [401]
No error message is returned from the server.
####  Deployment incomplete.  ####
Error deploying archive file:/C:/JDeveloper/mywork/Sample/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar 
 (oracle.tip.tools.ide.fabric.deploy.common.SOARemoteDeployer)

Solution :


HTTP 401 Unauthorized is similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. Double-check the weblogic username/password prompted during deployment from Jdeveloper.

Oracle Application Explorer : Could not initialize JCA Container

While trying to connect to the JCA config on application explorer you may hit the below error message

Error stack is as mentioned below :

com.iwaysoftware.iwrepository.RepositoryException
        at com.iwaysoftware.iwrepository.xml.XMLRepository.end(XMLRepository.java:272)
        at com.iwaysoftware.iwrepository.xml.file.FileRepository.end(FileRepository.java:436)
        at com.iwaysoftware.iwrepository.xml.XMLQuery.executeQuery(XMLQuery.java:863)
        at com.iwaysoftware.af.container.IWAFRepositoryOP.getConfigs(IWAFRepositoryOP.java:167)
       ....
       ....
To fix this error go to $SOA_HOME/soa/thirdparty/ApplicationAdapters/config/<config_name> , Take a backup of the st_repository.xml file and delete it. Now restart Application explorer and try connecting. The error should be gone .....

Tuesday, August 3, 2010

VisualGC: Performance Monitoring tool for Oracle SOA Suite

Oracle Fusion Middleware supports various performance monitoring tools for helping us debug performance issues with SOA Suite. Performance bottlenecks can happen at many places in an integration solution. It can either take place at DB level, or the BPEL/ESB code level or the JVM/Middleware level. Identifying these bottlenecks is a daunting task and often requires analysis from multiple aspects. 

VisualGC tool from Sun which comes with jvmstat 3.0 (for JDK 1.5.0 and higher) is an excellent tool for monitoring JVM memory usage. This tool helps in investigating issues where the applications tends to slow down after a while/sees degraded performance over a period of time.

Links for downloading and using this tool on Linux environments is provided below:

VisualGC download link
Description of the tool 
Installation instructions on Linux/Solaris

Once the tool has been installed you can use the below commands to run it and monitor the performance:

I have unzipped the jvmstat3.0 zip file at /opt/apps/Java/ on my linux server and using JDK6.

$ export JVMSTAT_HOME=/opt/apps/Java/jvmstat 
$ export JAVA_HOME=/opt/apps/Java/jdk1.6.0_18 
$ export PATH=$JVMSTAT_HOME/bin:$JAVA_HOME/bin:$PATH 

$ jps 
5755 Server 
2885 Jps 
5930 Server 

jps command gives you the JVM ids which are required as input parameter for the visualgc command.

$ visualgc 5755
As can be seen from the adjacent image, the memory usage of different parts of the JVM are shown graphically. The PermGen, OldGen and YoungGen (Eden Space, Survivor0 and Survivor1) usage can be monitored. When you observe a flat line in the Eden Space (right hand graph window) then its a cause of concern and indicates memory leaks/deadlock scenarios and requires immediate investigation.





There are other performance monitoring tools which can be used with Oracle SOA Suite. The EM Console in SOA 11G has a nice Summary screen which shows several important performance tracking statistics. For details on few of these tools please check the below link.
Overview of Performance Tuning Tools in Oracle Fusion Middleware

Weblogic domain creation error "Unable to invoke parse in InvokeStaticMethodTask"

While creating weblogic domain on linux server below error is encountered.

Steps:
export WLS_HOME=/opt/apps/Oracle/Middleware
export WLS_SERVER=/opt/apps/Oracle/Middleware/wlserver_10.3
export ORACLE_HOME=/opt/apps/Oracle/Middleware/Oracle_SOA1
export JAVA_HOME=/opt/apps/Java6/jdk1.6.0_20
export WLS_DOMAIN=/opt/apps/Oracle/Middleware/user_projects/domains
cd $ORACLE_HOME/common/bin
./config.sh




Here is the full stack trace of the error.

2010-08-02 09:56:28,142 ERROR [invokeStaticMethod] com.bea.plateng.wizard.silent.tasks.InvokeStaticMethodTask - Unable to invoke parse on class com.bea.plateng.plugin.PlugInExecutionPlanParser>
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.bea.plateng.wizard.silent.tasks.InvokeStaticMethodTask.execute(InvokeStaticMethodTask.java:304)
at com.bea.plateng.wizard.silent.tasks.AbstractSilentTask.run(AbstractSilentTask.java:28)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:103)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at com.bea.plateng.plugin.PlugInExecutionPlanParser.parse(PlugInExecutionPlanParser.java:111)
at com.bea.plateng.plugin.PlugInExecutionPlanParser.parse(PlugInExecutionPlanParser.java:90)
... 7 more
2010-08-02 09:56:28,145 ERROR [invokeStaticMethod] com.bea.plateng.wizard.silent.tasks.InvokeStaticMethodTask - parameterValue 0 = <null>

The problem is the classpath for launching the configuration wizard.

/opt/apps/Oracle/Middleware/patch_wls1032/profiles/default/sys_manifest_classpat
h/weblogic_patch.jar:/opt/apps/Java6/jdk1.6.0_20/lib/tools.jar:/opt/apps/Oracle/
Middleware/utils/config/10.3.1.0/config-launch.jar
:.......

The source of this classpath entry is from $WLS_HOME/common/bin/commEnv.sh
The highlighted entry is invalid. It should point to 10.3 and not 10.3.1.0. Once this is changed the Configuration Wizard can be launched successfully for domain creation

Sunday, August 1, 2010

New updates to Oracle E-Business Suite Integrated SOA Gateway(ISG) with R12.1 ATG RUP3

Oracle has recently released the R12.1 ATG RUP3 patch which brings along with it many new features to the Integrated SOA Gateway(ISG) product. I had talked about ISG in one of my earlier posts soa-integrations-with-oracle-ebusiness. Its a nice product which allows system integrators to design solutions surrounding integrations with Oracle Ebusiness Suite using out-of-the-box features. Most of the updates coming along with this patch have been around enhanced administration capabilities for the product.


More details about the new added features can be found from below links:
New features overview
Enhanced logging in ISG
Enhanced Service Monitoring 
Securing WebServices with ISG