XML Schema in REST Webservices

Regarding the Usage of XML Schema for REST webservices to create an Open API does not seem to have common practice.

for eg
Amazon,Ebay,Yahoo exposes a xsd for their services.
netflix,best buy does not have a xml schema exposed.

The only reason which i could think about the above API's are netflix,best buy API have very less input parameters,and most of them are GET requests. But Ebay/Amazon requests have many input Parameters and POST methods are used in Many services.

As we all know most of the enterprises are tying to adopt/already adopted the REST architecture and expose atleast a minimum business function as an Open API.Request/Response complexity depends upon the nature of service exposed by the enterprise.

when to use XML Schema in REST?

If your service has more input parameters.
if you expect a many versions of your service in future,backaward compatability for the services exposed.
To Avoid malformed XML inputs.
To Detect the XML Errors Early and return Error Response

why to use XML Schema in REST?

The common reason is to validate our XML input Structures.
Most of the Java based REST webservices Frameworks uses JAXB to bind the Data to Objects.it would be better when you detect errors before allowing the xml input to bind java objects.

Apart from these reasons the level of validation done on the XML input is purely based on the design and requirement of your Service.

I prefer the below method.

Have a Filter/Interceptor class ,Register an error handler to it, and validate for error,fatal error messages and return an user friendly error message.

Do not have heavy patterns on your XML Schema, validate the XML Request structure with the Schema and user java regular expressions to validate the actual data.

This way we can stop the malformed xml inputs in the interceptor rather than allowing them to reach the resource for validations.

Overall it should not be a very strict XML Validation but we should make sure our system accepts only a valid input. When it comes to performance Vs security , it is worth to consider Security.

Thanks
Prem.

ConnectionPooling and Transaction

In websphere 6.1 if you see the below error messages they are logged because a commit or rollback was not performed in the transaction boundary for which we will be able to find many posts from google.

LocalTranCoor W CWWLT0033W: Resource jdbc/datasource rolled back in cleanup of LocalTransactionContainment.
LocalTranCoor W CWWLT0032W: One or more local transaction resources were rolled back during the cleanup of a LocalTransactionContainment.

Debugging gets complicated when Transaction layer is not tested thoroughly in local environments.

In My case a Method which performs the Transaction was defined with a definition "PROPAGATION_REQUIRED".One of the DAO classes did not commit or rollback(Yes ,thats a Code issue). So the Transaction manager rolled back the transaction. This situation described above starts to get worse when we use connection pooling.

Basics to recollect before we implement transaction

Transaction is always associated with a connection.

A transaction cannot span Connections.

Any number of Threads can have access to a Single Connection.


When multiple threads get access to a connection associated with an incomplete transaction fails to create new Transactions(for definition PROPAGATION_REQUIRED) which causes intermittent issues.

This below link will give more information about Transaction definitions.

http://stackoverflow.com/questions/4944349/spring-transaction-management-for-execution-service-methods-from-same-service


Thanks
Prem

API Management and Security

Many companies are creating an API and they like to have a developer community.

Apart from the API Design & Development there is another important activity
API Management. As of this Writing the Three major companies in the API Management are

1. Mashery
2. apiGee
3. SONOA

Almost the business model of all the above mentioned companies are very similar.
like providing the below API Management Capabilities

Throttling, alerts , Usage graphs,API key provisioning and Management ,a dedicated portal to host the documentation,blogs,forums etc..

There is no doubt these are very essential features for a corporate to support their API's and developers.

SO does all the companies who have an API are dependent on a separate vendor for API Management?

Let's forget about the cost involved for having a dedicated Managment Vendor. Apart from the Cost there is also a significant impact on the security

API's which deals with Payments/credit card information/restricted Data(restriction levels are classified by the company which exposes an API). are not considering a dedicated vendor for API Management

eg: Paypal,Master card Payment Gateway,Amazon s3,e bay

As long as the Data produced and Consumed by the API are not required to be very secure this API Management Model seems to be very good.

eg: Bestbuy API which exposes the catlogs,products etc..Netflix,NYTimes are exposing their API through a separate API management vendor.

Not To say that Security is compromised when we have an API Management vendor outside the Network.

But it does have an impact on the security Design of the API being Exposed through the API Management Vendor.

eg: https (point to point connection are not enough to secure when you have a management proxy.So security has to be end to end.)

XML Encryption is an option to Provide the End to end Security.

Engaging an API Management vendor outside the corporate network should also depend on the 'Security of the Data flowing through the API'.

While Architecting a solution for this model surely Architect will have to take a critical decision by listing the tradeoffs .

Please let me know if i am not correct in my Views.

Thanks
Prem

Deep Dive into Asymmetric Encryptrion

I came across an issue while encrypting a xml using Apache XML Securty.

Error Occured when the code Reached the below line.

//cipher.doFinal(ContainerDoc,PayloadDoc.getRootElement());

When i digged in to that issue a lot of things got cleared in a much better way .

I am not going to write all basics of Asymmetric Encryption.I did an Asymmetric Encryption as two Step Process as Described by many Articles on Web.

a. Generated a AES128 Symmetric key and encrypted the Data
b. Encrypted the Symmetric key with a Public Key.

I was aware of only one Reason for the above Steps. As the Asymmetric Encryption is Expensive the symmetric and Asymmetric Technique described above will Increase the Performance and will make the System More scalable.

But that's not the Only One Reason.

Another Very Important Reason. The x.509V3 Certificates which has RSA 1024 bits key, cannot Encrypt more than 117 bytes.

So,Use an Asymmetric key to split the data into blocks and encrypt it(this could be done on data of any size)

How this 117 bytes is calculated is given in the below link?

http://www.owasp.org/index.php/Digital_Signature_Implementation_in_Java

Error Handling REST ful WebServices

Looks to be a Easy One but Really Not.

Like Anyone before doing something i read many documents and referrred many REST webserivces api in the market by leading companies.

And there is no common practice.It purely depends on the Architect/Designer of the System.

for eg: Ebay returns a status code HTTP 200 OK in case of request error(eg :request element is not formatted as expected) and then a Custom Error Code. which is not so correct in my View.

Anything which is 200 OK is considered as SUCCESS as per HTTP Spec, and for application which are created with HTTP Awareness. the above apporoach may defeat that. Still i understand there should be a strong reason why ebay did that ...

OK coming back to Error Handling..

a. Create a error schema or object ( with complex type customcode, error classification,Message) you can keep it simple like this or you can even return the error param.

b. inside the catch block set the appropriate values for that error conditions.

c. Set the HTTP Status to appropriate status, if the Error Classification is Request ERROR.(i use http 400 for this)

How to Classify ?

Any Problem in the Request Data can be considered as REQUEST ERROR

if valid Request data, Any Problem while processing the Request is a APPLICATION ERROR. (this error includes business rule violations)


if Required Have another classification for SYSTEM ERROR

A Very important Rule is Never /Ever Let your application to expose a stack trace to client. Your application should handle and all the Errors and Exceptions and return specific code/Generic code . Make Sure you Log the Details on the Server.


Please let me know if you have any questions.

Thanks
Premkumar.

RestEasy - Not Easy for Validation

After a Long time i am back to my Blog to post my findings on REST easy and Rest ful web services implementation practices.

Though my Preference is' Jersey ' i was in a position to select Resteasy for the implementation

The documentation explains about Decorators for Validation. it applies only for Marhsalling. for unmarshaling there are no decorators available in the framework.
Writing a New Decorator for unmarshaling is not an easy task.

So i used a Validator class from javax.xml.bind.* which worked perfectly.

But the Trick Here is you cannot Validate and let the framework bind on the same input Stream .so you need to clone the input stream ,one for validation and the other one for binding. (this binding will be done by Rest easy framework)

This worked really well.

Code to clone Input Stream

org.apache.commons.io.IOUtils;

private List cloneInputStream(InputStream iStream) {
List inArray = null;
List errorList = new ArrayList();
try {

if (iStream != null) {
String base = org.apache.commons.io.IOUtils.toString(iStream);
logger.debug("called cloneInputStream: input stream content -> {}",
base);
inArray = new ArrayList();
for (int i = 0;
i < APIConstants.INPUTSTREAM_CLONES; i++) {
inArray.add(IOUtils.toInputStream(base));
}
}
return inArray;
}
Please let me know if you have any questions.

Thanks
Premkumar

REST / JAMON Integration

I liked to Monitor all the Calls to Made to My REST ful webservices.

JAMON is Considered to Be a better Monitoring Tool i liked it very much. It is easy to integrate easy to monitor .I downloaded it from

http://sourceforge.net/projects/jamonapi/files/

Jamon.jar (Required)
Jamon.war (optional)

But i suggest to integrate this to the web app. It has a nice admin screen to view the calls made to the services.

a. Added jamon.jar to the Project and added the monitors to My REST Services.

//Code
mon = MonitorFactory.start("EmpDetailsMonitor");

//My Business Logic

mon.stop();

b. I modified my application.xml to include the jamon.war file.

c. Built the .ear file again Redeployed it.

Call the Webservice in which you added the EmpDetails Monitor

Navigate to the admin screen .you should be able to access like below

http://hostname:port/jamon/jamonadmin.jsp

You should see the monitor Which you added with all other Call Specific Metrics

-Prem