Web caching in the Cloud

This is the buzz word now in enterprise if you are trying to implement private cloud. I think this can be confusing to an extend, is it application caching side we are talking about or are we talking about caching in cloud?

For Applications there are solution like Eh-cahce or open terracotta or Extreme scale from IBM are some good solution.

Web application performance can be improved by caching frequently‐accessed data in high‐speed memory, instead of in databases, they can dramatically reduce response times, economically scale their sites, and substantially reduce the load placed on their databases, file servers, and other information sources.

Now lets talk about web caching in the cloud and some player in this space. Memchached and Gear6 (which provides enhanced solution)

The open‐source distributed cache system Memcached has quickly become the standard for web caching, particularly for operators of dynamic, high‐growth web sites. Memcached stores frequently‐used data in DRAM instead of in databases, providing response time improvements of 100x or more.

It is not surprising that many large organizations are interested in both cloud computing and Memcached web caching, since they offer complementary advantages in cost savings, flexibility and ease of management. But they too have evolved in different communities, largely in isolation from one another, and they do not operate together as seamlessly as might be desired.

As an example, the ability to rapidly scale applications up or down in size is a key advantage of cloud computing, but standard Memcached does not accommodate it very well. Changing the size of a Memcached cache tier causes the cache data to be flushed and then rebuilt, this typically causes degradation in site performance, and causes spikes in the demands on source databases and file servers. Rebuilding the contents of a large cache tier can take hours, and site performance may not be restored to normal until the end of the rebuilding process.

You can read more about Memcahed and Gear6 and MemCahed can be dowloaded from

Download MemChached

MapReducer

Almost a year back I started looking in to multiple options as how to provide better search in quick time against large sets of data in enterprise. for example you might want to search some keyword in exceptions, it could be like sysOut or error logs for enterprise applications.

I think MapReducer is an answer to these problem, here's some info about MapReducer

MapReduce is an parallel and distributed solution approach developed by Google for processing large datasets. MapReduce is utilized by Google and Yahoo to power their websearch. MapReduce was first describes in a research paper from Google .

MapReduce has two key components. Map and Reduce. A map is a function which is used on a set of input values and calculates a set of key/value pairs. Reduce is a function which takes these results and applies another function to the result of the map function. Or with other words: Map transforms a set of data into key value pairs and Reduce aggregates this data into a scalar. A reducer receives all the data for a individual "key" from all the mappers.

The approach assumes that their are no dependencies between the input data. This make it easy to parallelize the problem. The number of parallel reduce task is limited by the number of distinct "key" values which are emitted by the map function.

MapReduce incorporates usually also a framework. A master controls the whole MapReduce process. The MapReduce framework is responsible for load balancing, re-issuing task if a worker as failed or is to slow, etc. The master divides the input data into separate units, send individual chunks of data to the mapper machines and collects the information once a mapper is finished. If the mapper are finished then the reducer machines will be assigned work. All key/value pairs with the same key will be send to the same reducer.

Here's nice video about MapReducer :

http://code.google.com/edu/submissions/mapreduce-minilecture/listing.htm

Apache Hadoop which start with one large open source project is getting very popular around this space.

Here's link if you want to know more about Hadoop:

Apache Hadoop



If you want paid solution and dont have time to develop, take a look at SPLUNK, its nice and easy and very powerful tool, you can feed any data against this. I have played with splunk and I believe Splunk usages Hadoop APIs.

Enterprise Architect vs Solution Architect

I find this subject very interesting and at least i didn't find any concrete definitions around these two different roles in IT, what are the main differences (in term of responsibility) between two practices/roles. I think TOGAF and Zackman architectural framework provide in detailed explanation about roles and responsibility for each role in and enterprise. I would like to share what i have got so far....

So what exactly is an Enterprise Architect vs a Solution Architect?

A Solution Architect may have a number of different types of architects working for him/her to help accomplish their task for delivering a high quality solution. For example, s/he might have an Infrastructure Architect to manage the architecture of the solution's hardware configuration so that the solution meets the Quality of Service business and IT requirements while at the same time represents the optimum way to deploy the solution into the target production environments.

There also might be a need for 'specialist' architects depending on the complexities of the business requirements or the target production environment. Such 'specialist' architects usually are called Domain Architects and examples include; Security Architect, Technology Architect (ie architects that specialize in a specific product or technology), Vertical Architects (ie architects that specialize in systems that are specialized in particular industry verticals such as Financial Services Industry, Telecommunicaitons, Public Sector, etc).

An Enterprise Architect may have also have a number of different types of architects in order to piece together a coherent, actionable future state architecture which can easily map to business strategy, be consumed by project teams and be a major contribution in governance activities.

For example, an Enterprise Architect may have Business Architects which document Business Strategies, Business Capabilities, Business Processes and Roles as well as a number of other artifacts.

Lastly, an Enterprise Architect may also have Solution Architects (aka Enterprise Solution Architects and Enterprise Application Architects) who focus on pulling all of the other Enterprise Architecture information together to shape the future state application system architecture.

Here is one place where I think there is confusion by many. You see, I described a Solution Architect at the project-level and an Enterprise Solution Architect across the enterprise and both share the words 'Solution Architect' in their role. They have VERY different purposes but are very complimentary. Could it be that simple of a reason why confustion exists? It just might be.

An Enterprise Solution Architect and project Solution Architect roles are very complimentary and because this relationship is of particular interest to me I'd like to take a moment and propose how they are to work together.

I think that the Enterprise Solution Architect be involved at the earliest point a new business initiative is created and do very high-level 'solutioning' via whiteboard and reference to future state architecture elements. Then, when the business initiative gains momentum is backed by Enterprise Architecture future state analysis the core project team is formed and the project Solution Architect takes over to be responsible for the solution. The Enterprise Solution Architect becomes a member of the project Solution Architects team and is responsible for the system integration architecture where the solution requires integration between enterprise systems as well as be responsible to provide future state system architecture guidance to help the project Solution Architect make decisions on which systems to commit to using in the solution. The Enterprise Solution Architect then is involved in governance boards to inform decision-makers with future-state architecture artifacts to help justify major technology, system, or business value decisions. Thats it.

As a quick summary, project Solution Architects and Enterprise Architects are different in that they have very different purposes. They are highly complimentary in that Solution Architects focus on delivery of solutions and Enterprise Architects focus on supporting them by documenting future state, participating on their teams and being involved in governance activities.

Groovy: Superset of Java

Groovy is a bit like scripting version of Java. Groovy is a superset of Java. It is no way means to replace java, Long time Java
programmers feel at home when building agile applications backed by big architectures. One nice thing about Groovy is that it
reduces the amount of code needed to do common tasks such as parsing XML files and accessing databases. Groovy on Grails is
as the name suggests similar to the Rails framework. For people who don't want to bother with Java's verbosity all the time.

The big advantage is speed, brevity and readability. For instance here is some java code which implements a bag:


public class Bag {

private Map counter = new HashMap();
public void add(Object obj) {
add(obj, 1);

}

public void add(Object obj, Integer count) {
Integer value = counter.get(obj);
if (value == null)
counter.put(obj, count);
else
counter.put(obj, value+count);
}

public Integer getCount(Object obj) {
return counter.containsKey(obj) ? counter.get(obj) : 0;
}

}

And here is the same implementation in Groovy:

class Bag {
def counter = [:]
def add(term, count=1) {
if (counter[term] == null)
counter[term] = count
else
counter[term]+=count
}

def getCount(term) {
return counter[term] != null ? counter[term] : 0;
}

}

So you can see how intuitive the Groovy code is, you don't need to define anything regarding types or default parameter values.

That said, Groovy is essentially Java, just wrapped around a different compiler, class loader and with many added methods and
members. For that reason it is unlikely that it will ever replace Java, not only that but because it IS Java it is bound by the same
limitations and rules and doesn't provide any functionality which is really unique.


Semicolons are optional. Use them if you like (though you must use them to put several statements on one line).
The return keyword is optional.
You can use the this keyword inside static methods (which refers to this class).
Methods and classes are public by default.
Protected in Groovy has the same meaning as protected in Java, i.e. you can have friends in the same package and derived classes can also see protected members.
Inner classes are not supported at the moment. In most cases you can use closures instead.
The throws clause in a method signature is not checked by the Groovy compiler, because there is no difference between checked and unchecked exceptions.
You will not get compile errors like you would in Java for using undefined members or passing arguments of the
wrong type. See Runtime vs Compile time, Static vs Dynamic.
Java can always be made faster, more memory efficient and more robust than Groovy if you are willing to put in the time and
effort.

So where would you use Groovy?

I think typically in places where readability, flexibility and speed of development is more important than performance,
extensibility and a large user base. For instance in deployment you can use the Groovy equivalent of Ant called Gant which
allows better flexibility and better readability. Other places might be in a web application framework such as Grails.

You can use Groovy all the time for utilities, both on the command line and on the web. Often, the utilities use jars/class files
from my project, since it is all on the JVM.

For web utils, take a look at Groovlets. You can come up to speed with Groovlets in a couple of hours. A groovlet is simply a
servlet distilled down to its essence.

Also, it does have integration for other open source framework like Spring framework. Groovy is great for writing unit tests for existing Java code and could save lots of development time...

I hope this brief introduction will help in understanding as what's groovy and how it can help in your project.

Scala Vs Java

Form last few days i'm trying to look in to some other dynamic language and trying to compare against java, Scala seems to be the better choice in compare to ruby and others...lets talk about Scala in this post and a small code comparison between Scala and java .. to start with, Scala is a modern multi-paradigm programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages.

Scala is object-oriented: Scala is a pure object-oriented language in the sense that every value is an object. Types and behavior of objects are described by classes and traits. Class abstractions are extended by subclassing and a flexible mixin-based composition mechanism as a clean replacement for multiple inheritance.

Scala is functional: Scala is also a functional language in the sense that every function is a value. Scala provides a lightweight syntax for defining anonymous functions, it supports higher-order functions, it allows functions to be nested, and supports currying. Scala's case classes and its built-in support for pattern matching model algebraic types used in many functional programming languages.

Scala is statically typed: Scala is equipped with an expressive type system that enforces statically that abstractions are used in a safe and coherent manner.

Scala is extensible: The design of Scala acknowledges the fact that in practice, the development of domain-specific applications often requires domain-specific language extensions. Scala provides a unique combination of language mechanisms that make it easy to smoothly add new language constructs in form of libraries:

any method may be used as an infix or postfix operator, and closures are constructed automatically depending on the expected type (target typing).

A joint use of both features facilitates the definition of new statements without extending the syntax and without using macro-like meta-programming facilities.

Scala interoperates with Java and .NET: Scala is designed to interoperate well with popular programming environments like the Java 2 Runtime Environment (JRE) and the .NET Framework (CLR). In particular, the interaction with mainstream object-oriented languages like Java and C# is as smooth as possible. Scala has the same compilation model (separate compilation, dynamic class loading) like Java and C# and allows access to thousands of high-quality libraries.

Advantages of Scala:

Scala language has been designed to be scalable. It scales according to user needs. The object oriented concepts allows this language to be used for larger and more complex projects while the functional constructs allows programmer to develop short and concise code.

Portability: Scala runs on Java Virtual machine (JVM) this ensures the portability to all the underlying platforms. Also there is an initiative for building Scala on top of .NET Common language Runtime (CLR).

Performance: Scala generate more efficient java code which leads to better runtime performance.

Every value in Scala is an object and every operation is a method call. It is closer to object oriented programming than functional programming and therefore does not support concepts like static fields and methods. Scala allows you to define own objects and constructs which makes it possible to scale up according to user needs.

Scala uses strict typing. But it also allows most of the typing unspecified. When information is unspecified compiler will do smart type interface to infer information from code.

It incorporates ideas from many languages such as Mixins (a way to address multiple inheritances in LISP) and actors (used message passing communication architecture).

Java echo system is available for Scala.
Scala can be embedded into XML since its syntax is somewhat similar to that of XML
New design paradigms become available (working with closures, traits).
Learning speed for Scala is faster than that of C++ or Java for same task.
IDEA (IntelliJ), NetBeans and Eclipse all has good support for Scala.

Disadvantages of Scala:

Inferior quality of the documentation
Native libraries (Scala uses Java or .Net libraries as base for their own)
IDE and debugging support is not as good as Java.
Non-existing localization of documentation
It requires larger syntax to learn.
It consistently breaks backwards compatibility.

Skilled programmers are required to use the features of the language.

It is still in early adopter phase.
Lack of extensive tutorials

Now lets talk about small code and see, how they differ from each other..

A Better OOP Language

Scala works seamlessly with Java. You can invoke Java APIs, extend Java classes and implement Java interfaces. You can even invoke Scala code from Java, once you understand how certain “Scala-isms” are translated to Java constructs (javap is your friend). Scala syntax is more succinct and removes a lot of tedious boilerplate from Java code.

For example, the following Person class in Java:

class Person {

private String firstName;

private String lastName;

private int age;

public Person(String firstName, String lastName, int age) {

this.firstName = firstName;

this.lastName = lastName;

this.age = age;

}

public void setFirstName(String firstName) { this.firstName = firstName; }

public void String getFirstName() { return this.firstName; }

public void setLastName(String lastName) { this.lastName = lastName; }

public void String getLastName() { return this.lastName; }

public void setAge(int age) { this.age = age; }

public void int getAge() { return this.age; }

}

can be written in Scala thusly:

class Person(var firstName: String, var lastName: String, var age: Int)

Yes, that’s it. The constructor is the argument list to the class, where each parameter is declared as a variable (var keyword). It automatically generates the equivalent of getter and setter methods, meaning they look like Ruby-style attribute accessors; the getter is foo instead of getFoo and the setter is foo = instead of setFoo. Actually, the setter function is really foo_=, but Scala lets you use the foo = sugar.

Lots of other well designed conventions allow the language to define almost everything as a method, yet support forms of syntactic sugar like the illusion of operator overloading, Ruby-like DSL’s, etc.

You also get fewer semicolons, no requirements tying package and class definitions to the file system structure, type inference, multi-valued returns (tuples), and a better type and generics model.

One of the biggest deficiencies of Java is the lack of a complete mixin model. Mixins are small, focused (think Single Responsibility Principle ...) bits of state and behavior that can be added to classes (or objects) to extend them as needed. In a language like C++, you can use multiple inheritance for mixins. Because Java only supports single inheritance and interfaces, which can’t have any state and behavior, implementing a mixin-based design has always required various hacks. Aspect-Oriented Programming is also one partial solution to this problem.

The most exciting OOP enhancement Scala brings is its support for Traits, a concept first described here and more recently discussed here. Traits support Mixins (and other design techniques) through composition rather than inheritance. You could think of traits as interfaces with implementations. They work a lot like Ruby modules.

Now look at one design patter example and comparing these two language.

Here is an example of the Observer Pattern written as traits, where they are used to monitor changes to a bank account balance. First, here are reusable Subject and Observer traits.

This trait looks exactly like a Java interface. In fact, that’s how traits are represented in Java byte code. If the trait has state and behavior, like Subject, the byte code representation involves additional elements.

trait Observer[S] {

def receiveUpdate(subject: S);

}

trait Subject[S] {

this: S =>

private var observers: List[Observer[S]] = Nil

def addObserver(observer: Observer[S]) = observers = observer :: observers

def notifyObservers() = observers.foreach(_.receiveUpdate(this))

}

In Scala, generics are declared with square brackets, [...], rather than angled brackets, <...>. Method definitions begin with the def keyword. The Observer trait defines one abstract method, which is called by the Subject to notify the observer of changes. The Subject is passed to the Observer.

There are lots of other differences between these languages, and I hope with small post will help others. who are trying to evaluate some other language, or trying to see whats out there....

Spring RestTemplate

Its hard to say anything about spring without seeming biased in favor. Spring's RestTemplate is as solid as can be expected. Their standard call back based approach for safe release of resources works well even for the REST template. If one is using Spring-mvc and their REST support, very few reasons would drive me to consider an alternative framework for the client. One among them is definitely finer grained control, another is Http Client 4.X support. Documentation is sparse on RestTemplate as well. But one has a community to back up on. There might a bit of up front customizations, standard call back etc that an adopter might create but once done I feel that it would be a easy to work with the RestTemplate.
Clearly one has many choices in selecting a client side framework for RESTful HTTP. In most cases it probably makes sense to use the same framework for the service and client end. Then again, if you are only a consumer of a service you have multiple choices among those shown above as well as the option of using Apache HTTP Client or Components directly and bypassing the higher level frameworks. For some, integrating with the spring framework is important and all the above frameworks have means of integration points, both on the service and client sides. Support for Client Proxies is something one might want to consider as they tend to simplify the programming model. Further if Resource definitions can be shared among client server, that can be quite useful in being DRY (Don't repeat yourself) and provide means for contract definition. For those interested in performance and tuning of the HTTP Connections, using a framework that allows you to manage connection pooling and other parameters is definitely the way to go.

For more information on Spring RestTemplate, visit Spring blog

Spring RestTemplate Example:

EJB 3.1

Today i'm going to talk about something interested and that is EJB3.1, after so many years in development sun has finally provided easy to use spec in EJB world in new enhanced EJB 3.1. I can see some similarity between EJB 3.1 spec and Spring Framework. lets check some feature from EJB 3.1

EJB 3.1 builds on the ease-of-use enhancements in EJB 3.0 by providing many new ways to improve developer productivity. Chief among these are the ability to implement session beans using only a bean class and the ability to package enterprise bean classes directly in a .war file, without the need for an ejb-jar file.

No business Interface (optional)
Singleton Bean
App callback
Annotation (exists in 3.0 spec)
No need of Eejb-Jar file

lets go and discuss above points.


No Interface

For example, the following session bean exposes a no-interface view:

@Stateless
public class HelloBean {

public String sayHello() {
String message = propertiesBean.getProperty("hello.message");
return message;
}

}
As is the case for a local view, the client of a no-interface view always acquires an EJB reference -- either through injection or JNDI lookup. The only difference is that the Java type of the EJB reference is the bean class type rather than the type of a local interface. This is shown in the following bean client:

@EJB
private HelloBean helloBean;

...

String msg = helloBean.sayHello();
Note that even though there is no interface, the client cannot use the new() operator to explicitly instantiate the bean class. That's because all bean invocations are made through a special EJB reference, or proxy, provided by the container. This allows the container to provide all the additional bean services such as pooling, container-managed transactions, and concurrency management.

Singleton

A singleton is a new kind of session bean that is guaranteed to be instantiated once for an application in a particular Java Virtual Machine (JVM)*. A singleton is defined using the @Singleton annotation, as shown in the following code example:

@Singleton
public class PropertiesBean {

private Properties props;
private int accessCount = 0;

public String getProperty(String name) { ... }

public int getAccessCount() { ... }

}
Because it's just another flavor of session bean, a singleton can define the same local and remote client views as stateless and stateful beans. Clients access singletons in the same way as they access stateless and stateful beans, that is, through an EJB reference. For example, a client can access the above PropertiesBean singleton as follows:

@EJB
private PropertiesBean propsBean;

...

String msg = propsBean.getProperty("hello.message");
Here, the container ensures that all invocations to all PropertiesBean references in the same JVM are serviced by the same instance of the PropertiesBean. By default, the container enforces the same threading guarantee as for other component types. Specifically, no more than one invocation is allowed to access a particular bean instance at any one time. For singletons, that means blocking any concurrent invocations. However, this is just the default concurrency behavior. There are additional concurrency options that allow more efficient concurrent access to the singleton instance.

Application Call Back

Here is an example that shows part of a singleton that includes a @Startup annotation as well as @PostConstruct and @PreDestroy methods:

@Singleton
@Startup
public class PropertiesBean {

@PostConstruct
private void startup() { ... }

@PreDestroy
private void shutdown() { ... }
...
}

No ejb-jar file


The EJB 3.1 specification addresses this problem by removing the restriction that enterprise bean classes must be packaged in an ejb-jar file. You now have the option of placing EJB classes directly in the .war file, using the same packaging guidelines that apply to web application classes. This means that you can place EJB classes under the WEB-INF/classes directory or in a .jar file within the WEB-INF/lib directory. The EJB deployment descriptor is also optional. If it's needed, you can package it as a WEB-INF/ejb-jar.xml file.

JERSEY

Sun offers a reference implementation for JAX-RS code-named Jersey. Jersey uses a HTTP web server called Grizzly, and the Servlet Grizzly Servlet (com.sun.jersey.spi.container.servlet.ServletContainer) handles the requests to Grizzly. You can develop production-quality JAX-RS applications today using Jersey, which implements all the APIs and provides all the necessary annotations for creating RESTful web services in Java quickly and easily. Beyond the set of annotations and features defined by JAX-RS, Jersey provides additional features through its own APIs, such as the Jersey Client API.

You can download Jersey separately or acquire it as a bundle with NetBeans 6.5 and GlassFish V3.

Developing RESTful Web Services Using JAX-RS
The classes and interfaces you use for creating RESTful web services with JAX-RS are available in the following packages:
javax.ws.rs
javax.ws.rs.core
javax.ws.rs.ext

I'm going to provide link to previous post for Jersey example and will write about Sprint rest template in my next post...

Jersey Example

REST Framework implementation

I'm going to write a series of framework that can be used for REST implementation. I have looked different JAX RS vendors such as Jersey, RESTEasy and Restlet JAX-RS. but if you look at client side support, the JAX RS specification does not include a client side API and different JAX RS vendors have proceeded to create their own API's for the same. which means the framework one selects for the client and service ends do not have to be the same. For example, one could have a RESTful service running using Restlet while having a client developed in RESTEasy that consumes the same. In most cases, one will typically be satisfied with using the same client framework as one is using for the service development in order to be consistent and potentially be able to re-use artifacts in both areas if the framework permits the same. i have listed few options take a look at them.


Jersey - A JAX RS reference implementation from Oracle/Sun.
Spring 3.0 - Spring provides a RestTemplate, quite like the JmsTemplate and HibernateTemplate in its support of REST clients.
CXF - An apache project that is a merger of XFire and Celtix. Provides a JAX RS implementation and JAX WS as well.
RESTEasy - A JAX RS reference implementation by JBoss.
Restlet - A new upcoming framework that has support for Client Side REST calls
Apache Wink - A JAX RS implementation that has a client and server side api that has had a 1.0 release just recently
Apache HTTPClient or Http Components - Direct use of HTTP API - Down and dirty.

Im my next post i'll be talking about JERSEY and might present small example of that.

Era of Open Source

Open Source framework, tools over the last few years have defined new era of technologies, from frameworks to infrastructure in my experience, the vast majority of enterprises are very comfortable with open source. Of course, my experience is naturally biased toward those companies who are comfortable paying for services and support around open source, and especially those who are replacing non-functional proprietary software with open source software.
I know 5-10 years ago a lot of the companies using open source would have been very hesitant to rely so heavily on open source, but so many people do now that the market seems to have really changed.

I do think going forward will see lots of companies will move toward using open source, you already have big companies Google, Facebook and Amazon using open source like Hadoop, Puppet etc...I just red article that Adobe has released Puppet Recipes for Hadoop...open source community keep growing day by day...