Empty string returned when fetching properties stored in WSO2 ESB?

Once I encountered this scenario where I have set of properties stored in registry and I wanted to fetch them in my ESB mediation sequence.

I had set of properties stored in /_system/config/myConfig. So I used get-property function as follows in my mediation sequence to read the value of stored properties,

get-property('registry','conf:/myConfig@myProperty1')

Problem

The problem was, get-property function was keep returning empty value.

Then I debugged org.apache.synapse.GetPropertyFunction.java to find out the issue and it turned out that registry stored in the synapse context was null.

Fix

The reason for the registry to be not loaded into the context was, I have missed following in my synapse definition,

<registry provider="org.wso2.carbon.mediation.registry.WSO2Registry">

<parameter name="cachableDuration">15000</parameter>

</registry>

When browsed to the soruce view of WSO2 ESB, I noticed that above entry is not present in the configuration and I prevents registry being loaded into the synapse context. After adding the above to the synapse definition everything started working as expected.

Reason for WSO2 ESB returns 202 response when an API is called!

Scenario

There's a REST API hosted WSO2 ESB. And when you invoke it, ESB only returns a 202 Accept response similar to follows and no processing is done to the request. And there are no errors printed in the wso2carbon.log too.

HTTP/1.1 202 Accepted
Date: Wed, 25 Feb 2015 13:43:14 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked
Connection: keep-alive

Reason

The reason for this is, there's no API or a resource that matches the request URL. Let me elaborate more on this. Let's say the request URL is as follows,

https://esb.api.host:8243/myapicontext/myresource/myvar/

If this request returns a response similar to above, following are the possible causes,

1. There's no API with the context="/myapicontext"
2. If there's an API with  context="/myapicontext", it has no resource with uri-template or a url-mapping which matches /myresource/myvar/ portion of the request URL.

Therefore, to fix this issue we should make sure target API and the resource exists in ESB.

In order for the ESB to send  a more meaning full response in case 2 ONLY, add the following sequence to the ESB.

<sequence xmlns="http://ws.apache.org/ns/synapse" name="_resource_mismatch_handler_">
   <payloadFactory media-type="xml">
      <format>
         <tp:fault xmlns:tp="http://test.com">
            <tp:code>404</tp:code>
            <tp:type>Status report</tp:type>
            <tp:message>Not Found</tp:message>
            <tp:description>The requested resource (/$1) is not available.</tp:description>
         </tp:fault>
      </format>
      <args>
      <arg xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns3="http://org.apache.synapse/xsd" expression="$axis2:REST_URL_POSTFIX" evaluator="xml"></arg>
      </args>
   </payloadFactory>
   <property name="NO_ENTITY_BODY" action="remove" scope="axis2"></property>
   <property name="HTTP_SC" value="404" scope="axis2"></property>
   <respond/>
   <drop></drop>
</sequence>

So that the ESB will return a response as follows if there's no matching resource in API,

<tp:fault xmlns:tp="http://test.com">
    <tp:code>404</tp:code>
    <tp:type>Status report</tp:type>
    <tp:message>Not Found</tp:message>
    <tp:description>The requested resource (//myresource/myvar/) is not available.</tp:description>
</tp:fault>

Sample on using Apache Cassandra with WSO2 CEP

In this post I hope to explain how you can configure WSO2 CEP to store events in Apache Cassandra.This don't intent to describe how you can configure  WSO2 CEP to receive events . Hence, I'm using configurations of sample 0001 and extend it to publish events to Cassandra as well in addition to publishing as WSO2Event*

In this sample,
- CEP receives events as WSO2Events
- Store received events in Cassandra.
For the sake of simplicity there's no any processing done and events are just passed through.

*WSO2Event is a Thrift based format for events supported by WSO2 CEP.

Configuring WSO2 CEP Server

1. Bring up CEP server with sample configuration 0001 by executing following command in <CEP_HOME>/bin,

./wso2cep-samples.sh -sn 0001 -Ddisable.cassandra.server.startup=false

By setting disable.cassandra.server.startup=false we bring up the embedded Cassandra in CEP server and event's will be stored in it. But, note that this embedded cassandra is only to be used for testing purposes and it's recommended to DISABLE IN PRODUCTION deployments.

2.  Add an Cassandra output event Adaptor[1] by navigating to "Configure" -> "Output Event Adaptors". Output adaptor is the component which communicate with Cassandra server and carry out the transport.

Note that I have configured the output adaptor to connect to  embedded Cassandra server @ localhost:9160 and used both user name and password as "admin"

3. Create an event formatter using the "CassandraOutoutEventAdaptor" by navigating to "Event Streams" -> "OutFlows"  of org.wso2.sample.service.data:1.0.0 stream.  

Event builder is the component which converts the outbound events to a format which can be stored in Cassandra. Here you have to enter keyspace name and column family name to be used when storing events in Cassandra. 

Only 'map' "Output Event Type" can be used with Cassandra, it will map value of each attribute in an event to key value pairs. Please refer [2] for more on map event formatters.

Now you have configured CEP server to store/publish events to Cassandra.

Verifying if it works

To verify if events have really stored in Cassandra server let's use "cassandra-cli".

1. Login to Cassandra server with using user name and password as "admin".

./cassandra-cli -u admin -pw admin

2. Execute the following commands,

USE TestEventKeyspace; 

LIST TestEventColumnFamily; <- Listing all rows in the Keyspace

If events are successfully stored something similar to following should be shown.

References 

[1] - Output cassandra event adaptor - WSO2 CEP Documentation
[2] - https://docs.wso2.com/display/CEP310/Map+Event+Formatter 

Performing a simple load test on WSO2 ESB proxy using Apache Jmeter

In this post I'm  explaining how you could perform a simple load test on a WSO2 ESB proxy or any Web service using Apache Jmeter which is a free and very easy to use tool. Also, please note that this post describes only how to setup a very basic test case and it's intended audience is people who has no experience in Jmeter (I'm not an expert too).

In this example I will described how you can create a test plan where you do a simple load test by sending same message over and over again to a ESB proxy or a web service.

Bear minimum required to perform a test

After launching Jmeter the following screen will appear. On the left pane you will see an icon named “Test Plan”.

• Right click on “Test plan” → Select “Add” → Select “Threads (Users)” → Select “Thread Group”. Here you can configure the number of threads that you going to use for the test. The number of threads are analogues to number of simulated users. Also you can specify the Loop Count or number of requests sent by each thread. This where you decide on the load that you going to use for the test
• Therefore, if you set “Number of Threads (users)” = 5 and “Loop Count” = 100. The test will send 5 * 100 = 500 requests to the web service.

• Now we have to set the SOAP request that's going to be used in the test. For this example we will be sending the same hard coded request over and over again. To add the request right click on the “Thread Group” → Select “Add” → Select “Sampler” → Select “SOAP/XML-RPC Request”

Above two  are the basic steps you need carry out to perform a test using Jmeter for a SOAP service. Now if you click "Start", JMeter will send the specified number of requests to the proxy or the web service. In addition to above, below I have described couple of JMeter components that may also be useful when building a real world test plan.

Setting HTTP headers

Let's say the proxy service you are trying to test is secured, so that you need to send the “Authorization” header. For fulfilling this requirement you have to use the “HTTP Header Manager”. To add a HTTP header manager right click on the “SOAP/XML-RPC Request” → select “Config Element” → select “HTTP Header Manager” . In this component you can set the headers as required.

Viewing and testing responses

You can add listeners to see the responses form the service. To add a Listener, right click on the “SOAP/XML-RPC Request” → Select “Listener” and from there you can select various types of listeners. Following is a screen shot of “View Result Tree” Listener which is a very primitive but still a very useful listner.

From this listener you can view the request and response payloads for each message along with the status of the message.

In order to validate the response coming from the service you can use “Assertion”. In an assertion you can set various validations on the response. Following is a “Response Assertion” where I'am testing if the “Text” elements in the response “Contains” the text “IBM”.

If a certain response does not meet the condition in Assertion, in the “Listener” the response status will be marked as "error" with a red exclamation mark. To add an “Assertion” right click on the “SOAP/XML-RPC Request” → Select “Assertion” and from here select the type of assertion you want.

Determining the size of a SOAP message inside a proxy service of WSO2 ESB

Below are two methods that can be used to determine the SOAP message size inside a WSO2 ESB proxy service.

Method 1 - Using script mediator

you can use the scrip mediator to find the size of the complete message. Following is example how you can do it,

 <script language="js">var msgLength = mc.getEnvelopeXML().toString().length;
         mc.setProperty("MSG_LENGTH", msgLength);</script>

<log level="custom">
        <property name="MSG_LENGTH" expression="get-property('MSG_LENGTH')"/>
</log>

In this sample it gets the string length of message, and assign the value to a property. And then read the value from outside the script mediator and log it.

Also you can only get the length of the payload of the message by  calling mc.getPayloadXML() inside script mediator.

Refer [1] for more information on script mediator.

Method 2 - Read the Content-Length header

Note that this method can be only used if the header is since it's not a required header. Value of the Content-Length header can be read as follows,

<property name="Lang" expression="get-property('transport', 'Content-Length')"/>

Please reply if you have any alternative or better methods other than these to find the SOAP message size inside a proxy service.

[1]-https://docs.wso2.com/display/ESB480/Script+Mediator

Increasing Timeout period of the Callout Mediator/Message Forwarding Processor of WSO2 ESB

When you use Callout mediator and Message Forwarding Processor  in WSO2 ESB it makes a blocking call to the  back end service using  org.apache.axis2.client.ServiceClient. If the service takes too much time to respond it will cause a timeout at ESB and will produce the following error in ESB,

java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
        at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
        at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
        at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
        at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
        at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
        at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
        at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
        at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:622)
        at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
        at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
        at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
        at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
        at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
        at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:398)
        at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:224)
        at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
        at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:554)
        at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:530)
        at org.apache.synapse.mediators.builtin.CalloutMediator.mediate(CalloutMediator.java:221)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:114)
        at org.apache.synapse.mediators.eip.Target.mediate(Target.java:121)
        at org.apache.synapse.mediators.eip.splitter.IterateMediator.mediate(IterateMediator.java:132)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.config.xml.AnonymousListMediator.mediate(AnonymousListMediator.java:30)
        at org.apache.synapse.mediators.filters.FilterMediator.mediate(FilterMediator.java:143)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:114)
        at org.apache.synapse.mediators.eip.Target.mediate(Target.java:121)
        at org.apache.synapse.mediators.eip.splitter.IterateMediator.mediate(IterateMediator.java:132)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:114)
        at org.apache.synapse.mediators.eip.Target.mediate(Target.java:121)
        at org.apache.synapse.mediators.eip.splitter.IterateMediator.mediate(IterateMediator.java:132)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:114)
        at org.apache.synapse.mediators.eip.Target.mediate(Target.java:121)
        at org.apache.synapse.mediators.eip.splitter.IterateMediator.mediate(IterateMediator.java:132)
        at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:71)
        at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:114)
        at org.apache.synapse.core.axis2.ProxyServiceMessageReceiver.receive(ProxyServiceMessageReceiver.java:162)
        at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
        at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
        at org.apache.synapse.transport.nhttp.ServerWorker.processEntityEnclosingMethod(ServerWorker.java:455)
        at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:275)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

To avoid this error you must increase the timeout period of org.apache.axis2.client.ServiceClient used by ESB. By default it has a timeout period of 30000ms (30s).

In order to increase the timeout period you have to set values(in milliseconds) of ,

• SO_TIMEOUT 
• CONNECTION_TIMEOUT 

parameters of the transport sender.

Your transport sender must look like this after you setting the parameters,

1 2 3 4 5 6 7 8

<transportSender name="http"> ... ... <parameter name="SO_TIMEOUT">45000</parameter> <parameter name="CONNECTION_TIMEOUT">45000</parameter> ... ... </transportSender>

If you are using HTTPS same parameters have to be set in the https transport sender as well.

For WSO2 ESB 4.8.0 or higher you must set these values in <ESB_HOME>/repository/conf/axis2/axis2_blocking_client.xml. For other versions you have to change <ESB_HOME>/samples/axis2Client/client_repo/conf/axis2.xml

Avoiding "Provider net.sf.saxon.TransformerFactoryImpl not found" error in Axis2

Scenario

When I'm trying to access the WSDL's of  web services deployed in a Axis2 SOAP engine the following error is occurring,

javax.xml.transform.TransformerFactoryConfigurationError: Provider net.sf.saxon.TransformerFactoryImpl not found
    at javax.xml.transform.TransformerFactory.newInstance(Unknown Source)
    at org.apache.ws.commons.schema.XmlSchema.serialize_internal(XmlSchema.java:505)
    at org.apache.ws.commons.schema.XmlSchema.write(XmlSchema.java:478)
    at org.apache.axis2.description.AxisService2WSDL11.generateOM(AxisService2WSDL11.java:215)
    at org.apache.axis2.dataretrieval.WSDLDataLocator.outputInlineForm(WSDLDataLocator.java:131)
    at org.apache.axis2.dataretrieval.WSDLDataLocator.getData(WSDLDataLocator.java:73)
    at org.apache.axis2.dataretrieval.AxisDataLocatorImpl.getData(AxisDataLocatorImpl.java:81)
    at org.apache.axis2.description.AxisService.getData(AxisService.java:2980)
    at org.apache.axis2.description.AxisService.getWSDL(AxisService.java:1653)
    at org.apache.axis2.description.AxisService.printWSDL(AxisService.java:1421)
    at org.wso2.carbon.core.transports.util.Wsdl11Processor$1.printWSDL(Wsdl11Processor.java:43)
    at org.wso2.carbon.core.transports.util.AbstractWsdlProcessor.printWSDL(AbstractWsdlProcessor.java:86)
    at org.wso2.carbon.core.transports.util.Wsdl11Processor.process(Wsdl11Processor.java:57)
    at org.wso2.carbon.transport.nhttp.api.NHttpGetProcessor.processWithGetProcessor(NHttpGetProcessor.java:137)
    at org.wso2.carbon.transport.nhttp.api.NHttpGetProcessor.process(NHttpGetProcessor.java:277)
    at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:256)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
[2014-05-19 17:56:10,980] ERROR - DefaultHttpGetProcessor Error processing request
org.apache.axis2.dataretrieval.DataRetrievalException: Provider net.sf.saxon.TransformerFactoryImpl not found
    at org.apache.axis2.dataretrieval.AxisDataLocatorImpl.getData(AxisDataLocatorImpl.java:85)
    at org.apache.axis2.description.AxisService.getData(AxisService.java:2980)
    at org.apache.axis2.description.AxisService.getWSDL(AxisService.java:1653)
    at org.apache.axis2.description.AxisService.printWSDL(AxisService.java:1421)
    at org.wso2.carbon.core.transports.util.Wsdl11Processor$1.printWSDL(Wsdl11Processor.java:43)
    at org.wso2.carbon.core.transports.util.AbstractWsdlProcessor.printWSDL(AbstractWsdlProcessor.java:86)
    at org.wso2.carbon.core.transports.util.Wsdl11Processor.process(Wsdl11Processor.java:57)
    at org.wso2.carbon.transport.nhttp.api.NHttpGetProcessor.processWithGetProcessor(NHttpGetProcessor.java:137)
    at org.wso2.carbon.transport.nhttp.api.NHttpGetProcessor.process(NHttpGetProcessor.java:277)
    at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:256)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
Caused by: javax.xml.transform.TransformerFactoryConfigurationError: Provider net.sf.saxon.TransformerFactoryImpl not found
    at javax.xml.transform.TransformerFactory.newInstance(Unknown Source)
    at org.apache.ws.commons.schema.XmlSchema.serialize_internal(XmlSchema.java:505)
    at org.apache.ws.commons.schema.XmlSchema.write(XmlSchema.java:478)
    at org.apache.axis2.description.AxisService2WSDL11.generateOM(AxisService2WSDL11.java:215)
    at org.apache.axis2.dataretrieval.WSDLDataLocator.outputInlineForm(WSDLDataLocator.java:131)
    at org.apache.axis2.dataretrieval.WSDLDataLocator.getData(WSDLDataLocator.java:73)
    at org.apache.axis2.dataretrieval.AxisDataLocatorImpl.getData(AxisDataLocatorImpl.java:81)
    ... 13 more

Reason

In one of the web service deployed in my server, it has used Saxon XML transformer implementation instead of default java XML transformer implementation.  Therefore, in it's code it has added the following static initialization block

1 2 3

static{ System.setProperty("javax.xml.transform."net.sf.saxon.TransformerFactoryImpl"); }

The setting of this system property is the reason for this error.  Also some can set this system property from command line using -Djavax.xml.transform.TransformerFactory=cnet.sf.saxon.TransformerFactoryImpl

Workaround

Remove the above static initialization block which sets the javax.xml.transfrom system property. Then, when instantiating TransfromFactory in the code, explicitly instantiate the saxon trasform factory with the full qualified class name as follows,

1	TransformerFactory fact = new net.sf.saxon.TransformerFactoryImpl()

In this way still you can use the  Saxon transform implementaion without casuing any complications. When the system property is set, all the instantiations of transform factories which not uses  the full qualified name of the transform factory class will be initialized with net.sf.saxon.TransfromerFactoryImpl and then it might lead to errors if they assume the default java xml transform implementation.

After recompiling and deployed the service the exception was not thrown