OSGi - Road Ahead

OSGi has been introduced many times in the past few years and will still require some in the future¹. With so many introductions, one might be inclined to think that this is a new technology. To the contrary, OSGi is a good 14 year old technology started in 1999!

So why does such an old technology still require an introduction? Is it because it is not yet popular and not reached mainstream yet? Even after 14 years! There are more than 3 implementations of the OSGi specification. Glassfish and Websphere uses it. JBoss supports it. Apache Geronimo provides it. Eclipse is built on it. Netbeans supports it. Spring dmServer and Oracle JDeveloper is built on it. As you can see, most of the middleware servers and the top IDEs are either built on OSGi or has support for it. So, popular it is! But mainstream? That is debatable.

The applications mentioned above are middleware or tooling products. You can use any of these products without knowing anything about OSGi. This is good as well as bad. Good because it helps adoption, as unlike other frameworks, OSGi does not come in the developers' way. Bad because developers tend not to follow the OSGi principles while developing applications. Besides, middleware and tooling product developers are far less compared to general java developers. There have been few attempts to make OSGi available for general purpose development like Spring dmServer (now Eclipse Virgo). But it did not gain much traction. So, why is OSGi not mainstream yet in spite of all the benefits that it provides? Here, I summarize some of my personal observations behind the reasons of OSGi's non-adopt-ability:

Observation One: Elitist

The OSGi group looks like an elitist club with minimal public engagement up until recently. Only big companies seem to be part of it. There is no open-source-community-like engagement with the general public or at least it appears so.

Solution

The name (OSGi Alliance) itself sounds very technical and non-mainstream. I agree that OSGi started in a niche area and now have expanded to general purpose usage. But since the context has changed, so should the engagement. I think, a name like `OSGi Foundation` will make it less intimidating (`OSGi` itself cannot be dropped because of brand value and historical reasons).

The group should allow individual technical contributors without corporate affiliations to be part of the top membership. A similar model like the current Java Community Process (JCP) or Spring community will make it look more open. I know many will disagree that the JCP is an open community. But it does allow individual membership and is becoming more open.

Observation Two: Too many jargon

The OSGi specification is full of jargon. Names like blueprint and compendium, for example, sounds complex although they have simpler names. The use of `Specification` instead of `API` to describe a standard makes it sound too technical like the CORBA specification. Yes, I know that OSGi IS a `Specification`.

Solution

Can't we have a specification without calling it a specification? I think `API` is a much simpler term to use just like how the JCP uses it. All JCP APIs are standard specifications. But for marketing purpose, API sounds more appealing to the general masses. Similarly, call it dependency injection container or CDI container instead of blueprint container to make it sound familiar with JavaEE CDI concepts.

Observation Three: FUD

Lots of FUD (Fear, Uncertainty, Doubt) has been generated regarding OSGi. Read this and this and this and this and that. Some of those opinions have been presented with the sole intention to garner page hits. The title do not match the contents. Some of these has been addressed while some may not. Some are genuine concerns. When a new developer reads such articles, they become very skeptical about adopting it. When people like Rod Johnson says it is complex, people(developer, architect, project manager, IT department) tend to believe it.

Solution

To counter these FUDs, more and more real life examples of successful projects should be published. Additionally, architecture, design patterns, anti-patterns, best practices, etc. should be published frequently. Some of these are available but they have become dated. Fresh information on latest technologies will create the buzz around OSGi. The following links addresses the strength of OSGi:

http://jaxenter.com/jax-magazine/JTJ-2011-04
LinkedIn's use of OSGi
http://javadepend.wordpress.com/2012/10/11/the-osgi-puzzle/
http://soupinadeli.com/pdfs/a-year-with-osgi.pdf
http://branchandbound.net/blog/java/2013/07/java-modularity-story/

Observation Four: No centralized implementation

The OSGi site does not provide an official reference implementation of all the specifications. Individual specifications/API are implemented by different project groups mostly from the Apache stable. Core implementation is provided by Eclipse Equinox project. Some of the enterprise APIs are provided by Apache Aries, Apache CXF and other projects. This means a developer has to pull together different parts from various sources to meet his needs. This can be overwhelming sometimes.

Solution

Similar to how JavaEE provides an official reference implementation and complete product (Glassfish), OSGi should provide a reference implementation together with every specification release. OSGi R5 has been released in 2012 but still there is no complete R5 implementation after one year. This implementation need not be a new project. One of the main two projects, Equinox or Felix, should be made the centralized implementation of all core and compendium specifications.

A concurrent implementation release with every new specification release will allow users to start 'playing' with the new features. This will create more buzz as people will start blogging, tweeting and pinning their experiences.

In fact, a core product similar to OpenJDK project, would be more appealing. The core can be maintained by all parties interested (Equinox, Felix, Knoplerfish, JBoss) while individually they can differentiate by additional components or integration with their products.

Observation Five: Community, community

The OSGi blog seems to be the personal blog of Peter Kriens. There are no other contributors to the blog. Don't misunderstand me. His posts are always very informative and insightful and a Body of Knowledge by itself. I always look forward to his posts. But, only one person speaking about OSGi in a community that boasts of hundreds of members does not correctly reflect the diversity of the community. On the other hand, the OSGi wiki is non existent. It is hardly being maintained.

Solution

Blogs from different community members should be published in the official blog area and not just as RSS feed. This will complement the efforts to counter the FUD mentioned earlier.

The wiki should also be maintained more regularly with up-to-date information. More tutorials, code snippets, design rationale, etc. should be provided.

General Observations

• The OSGi site design is dated. A modern theme should be applied to make it more appealing.
• Community engagement must be made a priority. Programs similar to Adopt a JSR should be adopted to encourage community participation.

---

1 This will become a past event after the scheduled date.^↩

OpenStack on Windows Azure?

Although this is off-topic from my blog interest, just thought about sharing this.

I was searching for information on the OpenStack project and did a Google search. This is what came up:

See the ad on the right that has been highlighted. The title is 'Openstack'. But the link points to www.windowsazure.com. Is this some false and misleading marketing strategy from Redmond to gain clicks and hits? Or does Redmond offer Openstack on its platform? Hardly so.

I rest my case.

Java sees Daylight in the Open

With Software Defined Networking (or SDN in short) gaining huge traction over the past few years, numerous start-ups,  protocols, frameworks and other applications have started coming up every few weeks. With this also came numerous efforts to standardize around this concept, chief among them being the Open Networking Foundation. Last week saw the emergence of another standards group (rather self-proclaimed) consisting of industry leading networking companies to launch the OpenDaylight project. The OpenDaylight project is an open source effort similar to the OpenStack project to collaborate on SDN related technologies. An extract from the OpenDaylight describing its motivation:

" At this early stage of SDN adoption, the industry acknowledges the benefits of establishing an open, reference framework for programmability and control through an open source SDN solution. Such a framework maintains the flexibility and choice to allow organizations to deploy SDN as they please, yet still mitigates many of the risks of adopting early stage technologies and integrating with existing infrastructure investments. 

With OpenDaylight, a community has come together to fill this need through the combination of open community developers and open source code and project governance that guarantees an open, community decision making process on business and technical issues. Establishing an open source project in this way is designed to help accelerate the development of technology available to users and enable widespread adoption of Software-Defined Networking. "

This is indeed a good development but as long as all groups collaborate with each other rather than compete against each other, which will lead to fragmentation and higher cost of implementation and deployment.

However, what is interesting about this project is the use of Java for one of their central component! Yes, for all the cries of Java being dead, we see a new far-reaching project adopt Java as one of their underlying technology. This is indeed good news for the Java platform as well as the greater Java community.  This underlines the strength, robustness and applicability of the Java technology for myriad platforms. The architecture of the OpenDaylight project is reproduced below:

Java is used for the Controller platform described above. See this link for an architectural overview. Interestingly, as well, OSGi is also used by the Controller framework. OSGi is a dynamic modular framework for Java which has also been steadily gaining limelight in recent Java developments. OSGi enables the Service Abstraction Layer because of its modular and extensible nature.

There are indeed good times ahead for both the Java and OSGi community.

Happy daylighting!

Liferay Expando Services

Liferay provides a mechanism to create tables, columns, rows and values dynamically. It provides these features through the Expando services. The mechanism is supported by using four database tables: ExpandoTable, ExpandoColumn, ExpandoRow and ExpandoValue.

Creating tables dynamically

The class com.liferay.portlet.expando.service.ExpandoTableLocalServiceUtil allows creation of dynamic tables.

To create a new table, call:

ExpandoTable table = ExpandoTableLocalServiceUtil.addTable(<companyId>, <className>, <tableName>);

Parameters:

<companyId> (Type: long) – The company id for the table. To get the company id, call

long companyId = CompanyLocalServiceUtil.getCompanies().get(0).getCompanyId();

<className> (Type: String) – The class name should be the fully qualified class name of the java class that you are trying to save to database.

<tableName> (Type: String) – The name of the dynamic table to create.

A new row is added to the ExpandoTable table.

Creating columns dynamically

The class com.liferay.portlet.expando.service.ExpandoColumnLocalServiceUtil allows creation of dynamic columns.

To create a new column, call:

ExpandoColumn column = ExpandoColumnLocalServiceUtil.addColumn(<tableId>, <columnName>, <type>);

Parameters:

<tableId> (Type: long) – The table id for which to create column.

<columnName> (Type: String) – The name of the column.

<type> (Type: int) – The data type of the column. Use the ExpandoColumnConstants to specify the data type. For example, ExpandoColumnConstants.STRING;

A new row is added to the ExpandoColumn table.

Creating rows dynamically

The class com.liferay.portlet.expando.service.ExpandoRowLocalServiceUtil allows creation of dynamic rows.

To create a new row, call:

ExpandoRow row = ExpandoRowLocalServiceUtil.addRow(<tableId>, <classPK>);

Parameters:

<tableId> (Type: long) – The table id for which to create row.

<classPK> (Type: long) – The classPK .

A new row is added to the ExpandoRow table.

Storing values in a dynamic table

The class com.liferay.portlet.expando.service.ExpandoValueLocalServiceUtil allows storing values for dynamic tables.

To store a value, call:

ExpandoValue
value = ExpandoValueLocalServiceUtil.addValue(<classNameId>,
<tableId>, <columnId>, <classPK>, <value>);

Parameters:

<classNameId> (Type: long) – The class name id.

<tableId> (Type: long) – The table id for which to store value.

<columnId> (Type: long) – The column id of the table for which to store value.

<classPK> (Type: long) – The classPK .

<value> (Type: String) – The value to be stored .

A new row is added to the ExpandoValue table. There are several variations of the addValue API to store different data types.

Removing dynamic entities

Delete Table:

ExpandoTableLocalServiceUtil.deleteTable(long <tableId>)

Delete Column:

ExpandoColumnLocalServiceUtil.deleteColumn(long <columnId>)

Delete Row:

ExpandoRowLocalServiceUtil.deleteRow(long <rowId>)

Delete Value:

ExpandoValueLocalServiceUtil.deleteValue(long<valueId>)

The above APIs for each entity is the simplest one. There are other variations depending on the parameters. The above APIs only removes the corresponding entity. It does not remove the related entities. For example, deleting table does not automatically delete the columns, rows and values.

Updating dynamic entities

Update Table:

ExpandoTableLocalServiceUtil.updateTable(long <tableId>, String <name>)

ExpandoTableLocalServiceUtil.updateExpandoTable(ExpandoTable <table>)

Update Column:

ExpandoColumnLocalServiceUtil.updateColumn(<...>)

ExpandoColumnLocalServiceUtil.updateExpandoColumn(<...>)

Update Row:

ExpandoRowLocalServiceUtil.updateExpandoRow(<...>)

Update Value:

ExpandoValueLocalServiceUtil.updateExpandoValue(<...>)

Searching for entities

Search Table:

ExpandoTableLocalServiceUtil.getTable(<...>)

ExpandoTableLocalServiceUtil.getExpandoTable(long <tableId>)

Search Column:

ExpandoColumnLocalServiceUtil.getColumn(<...>)

ExpandoColumnLocalServiceUtil.getExpandoColumn(long <columnId>)

Search Row:

ExpandoRowLocalServiceUtil.getRow(<...>)

ExpandoRowLocalServiceUtil.getExpandoRow(long <rowId>)

Search Value:

ExpandoValueLocalServiceUtil.getValue(<...>)

ExpandoValueLocalServiceUtil.getExpandoValue(long <valueId>)

ExpandoValueLocalServiceUtil.getData(<...>)

Dynamic queries

For dynamic queries, all the Local service util provide 2 APIs:

Expando[Table|Column|Row|Value]LocalServiceUtil.dynamicQuery(<...>)

Expando[Table|Column|Row|Value]LocalServiceUtil.dynamicQueryCount(DynamicQuery <dynamicQuery>)

Integrating Coin Slider with Liferay 6.1.1 with zero development

In my previous blog, I demonstrated how to integrate the Coin Slider library in a Liferay portlet without having to develop any portlet. Coin Slider is a jQuery based image slider with many effects for image transition. That demonstration was based on Liferay 6.0.5. With Liferay 6.1.1 now available, I decided to try my own example and see whether the steps still hold true for the latest version of Liferay. And guess what, it does! That's the power of Liferay! Congratulations to the Liferay Team!

So here I demonstrate the same example with updated screenshots for the benefit of the community.

Note: This solution is based on Liferay 6.1.1

Download and extract the latest version of the Coin Slider library from http://workshop.rs/projects/coin-slider/

Liferay now has a single Library for storing both documents, images, video and any other media files. Create a folder slideshow in the Document and Media library of the community where you want to use the slider. Add the files coin-slider-min.js, coin-slider-styles.css, jquery-1.8.0.min.js to the slideshow folder. Create a new file javascript.js file in filesystem. Add the following code to this file:

$(document).ready(function() {
    $("#coin-slider").coinslider({ hoverPause: false });
}); 

Upload the javascript.js file to the slideshow library. The slideshow folder should now look like this:

Next, upload few images that you want to display in the slider in the Document and Media Library.

There are 8 images in the library above.

Now create a new web content structure. Name it coin-slideshow. Add xml schema definition to the structure as shown below:

Click Save when done.

Create a new web content template and name it coin-slideshow-template. Select coin-slideshow as the structure for the new template. Click the Launch Editor button. In the dialog that appears, paste the following code:


<link href="$css.getData()" rel="stylesheet" type="text/css" />
<script type="text/javascript" src="$jquery.getData()"></script>
<script type="text/javascript" src="$coinjs.getData()"></script>
<script type="text/javascript" src="$slidestart.getData()"></script>

<div id="coin-slider">
#foreach ($item in $slide.getSiblings())
   #foreach ( $itemData in $item.getChildren() )
        #if ( "$itemData.getName()" == "href" )
          #set ( $href = $itemData.getData() )
        #elseif ( "$itemData.getName()" == "img_src" )
          #set ( $img_src = $itemData.getData() )
        #elseif ( "$itemData.getName()" == "description" )
          #set ( $description = $itemData.getData() )
        #end
    #end
    <a href="$href"><br />
        <img src="$img_src" />
        <span>$description</span>
    </a>
#end
</div>


Click Update button when done. Click Save again in the main dialog.

Now is the time to add the slider in our page. Add Web Content Display portlet to the page where you want the slider. Click the Add Web Content link in the portlet to create content for the portlet. In the new dialog box, give a name to the content, say Slideshow. Change the structure of the web content by clicking the Choose button under the Structure section on the right. Select the structure coin-slideshow that we created earlier. This will automatically choose the template that we associated with the structure. Instead of the HTML editor, a web form with fields are shown.

In the css field, choose the coin-slider-styles.css from the document library. Similarly, choose jquery-1.4.2.min.js for jquery field, coin-slider-min.js for coinjs field and javascript.js for coinstart field. For the first slide, name it slide1. Select a image from the Document and Media Library for the img_src field. Select target location when an image is clicked from the dropdown list for the href field. Type some text to be displayed for the image in the description field. Similarly, add more slides for each image that you want to be displayed.

Click Publish button to save the content and make it viewable.

OrientDB - Pure Java NoSQL Datastore

I have been following the so-called 'NoSQL movement' (as if such a movement exists!). In my opinion, it is just another way of storing and retrieving data. And this new (rather different, I am sure the method is not new) way is only suitable for certain type of applications. Not all applications require NoSQL-type storage and retrieval regardless of those who claim it to be. SQL and NoSQL complement each other. They are not rivals. 

Having said that, I was pretty much interested in the concepts. I love trying out new things. While I was planning the design and architecture of a new open source project that I intend to develop (It is not exactly new. I have already started it but now plan to move it to a new architecture - Greenscape), one of the choices I had to make was on the data storage. My application will be highly dynamic with ability of end users to add and remove columns to a database table frequently. Doing it in relational database will incur heavy performance and complexity. After much consideration, I decided that a non-relational database will be the best choice for my application. But this choice is not without its share of challenges:

1. I loose JPA/Hibernate support which means I may have to come up with an equivalent framework.
2. I loose portability. NoSQL is non-standard. There is no standard query language. There are no standard protocol. So, tomorrow, I cannot replace the database with a better performing one without rewriting my application.
3. Few documentation. What are the best practices, design patterns, etc?

In spite of the challenges, I went ahead with my choice. The compelling reason was the ability to create knowledge because of the void.

Now that the decision was made, I had to choose a NoSQL database. I analysed all the popular ones but none fitted my requirements. I had one criteria for selecting a database: I must be able to code in Java. Most available systems were non-Java based which would be a significant issue for a one man project. Even if they had Java interface, the installation, setup, etc. were a tedious process. Having a database developed purely in Java has many advantages:

1. Easy packaging with other applications
2. Easy to install and run
3. Can be embedded
4. Can run in same or different VM
5. Easy to debug
6. Easy to test

After much searching, I came across OrientDB. Voila! That was what I needed! Going through its features only reinforced by belief in it. It is a pure Java solution and very small in size (500KB! 2MB since last release) . It can be embedded as well as deployed in networked mode. It is both schema-less and schema-based! This topped the feature list. It also supports SQL as a query language (Hmm, need to call NoSQL by some other name).

Some of the features taken from its homepage:

1. SQL
2. Super fast
3. Transactional
4. GraphDB
5. Web ready (HTTP, REST, JSON)
6. Everywhere (Pure Java)
7. Extremely light
8. Apache License (Here's the money)

End of architecture choice 1 for Greenscape.

Next decision on application framework pending.

Grow Up Java! Proposing Java Desktop Edition

No, this is not another 'Java is Dead' rant. Java is very much alive and kicking. It is one of the best development and runtime platform available. By far the most stable platform of those. And that might just be its biggest bane. Ridiculous! How is stability a bane? You might ask. Well, you can see the slow progress in path breaking innovation, due to it. See the debate on closures and the progress on module system/OSGI.

The Java platform is growing only by a few APIs and small language changes/additions every year. There is no major big innovation happening. All the major changes that has been happening are outside it. Be it the  Dependency Injection, Persistence API, Scripting API, Desktop API, Application Framework API, they were only added to the platform subsequently. But they are just APIs. What we need is something big. Something that is 10 years advanced than today.

What I feel is Java is advancing in a bottom-up fashion. Small and slow changes at a time. What we need is a top-down approach. Think futuristic, design it and then only implement it in a bottom-up approach.

Take the example of new Swing Application Framework (now dropped from the standard). I think this new framework is a poorly envisioned idea. Look at the poor state of JMF. No substantial progress can be made on the desktop without support for multimedia. Yes, JavaFX is coming soon (for 3+ years ?). It is a big step forward but is not a standard (yet. Will it be ever?).

We have the JavaSE and JavaEE is built on top of it. Why can't we do the same with the desktop? Why can't we have something like the Java Desktop Edition or JavaDE ? Put all the big stuffs here. In this way, we can keep the JavaSE simple. Language and VM changes goes into JavaSE. Common APIs applicable to both EE and 'DE' goes into SE. But desktop additions like application framework, multimedia, browsing capabilities, gadget bar?, JavaFX, etc. goes into 'DE'. Of course, JavaME also needs to be considered but since industry's mood is to bring SE to ME, they can be considered the same for the sake of this blog.

Some might argue that the desktop is becoming irrelevant with the popularity of web applications and powerful browsers (with HTML 5). But as any unbiased report will say, the irrelevance of desktop will not happen anytime soon. After all, the browser needs to run on a desktop!

Basically, Java Desktop Edition becomes the new Operating System. It should have transparent integration with the underlying OS. Have functionalities in the form of services and modules on the lines of OSGI. So everything is a service: print service, document service, file service, graphic service, database service, < add your favorite services here >. So advanced capabilities can be exposed as a service to any module or applications (collection of modules) in the system. Imagine having Eclipse plugins as services exposed to all applications.

Think of the innovative possibilities of a Java Desktop (heard it before?):
- Java Browser with built-in support for applets, JavaFX - we can have applet plugins
- Browser can be embedded in any application
- GIMP like editor - JIMP?
- Standard installer for all applications - No OS specific versions of Eclipse or Netbeans (Java WAS supposed to be Write Once, Run Everywhere)

- Finally, a Java media player

- Desktop gadget bar

- Java Office
- < your favorite application here >

It is time for Java to grow up and act its age.