Tuesday, January 15, 2013

List of Open Source Java Projects/Libraries

I added these from different blogs and journels.These are added for future references and with the intention to group them in one place. If you know any such projects please let me know.

1) Below are the Stress test tools Other than JMeter.
  •     a) Grinder can be other option. Read a brief comparison here -   http://www.vedantatree.com/2012/05/which- is-best-free-load-testing-tool.html 
  • b) BadBoy for this and was quite happy with the results. Check its licensing terms to see whether you qualify for free usage.           http://www.badboysoftware.biz/docs/loadtesting.htm          Selenium would work just like BadBoy. you can even export the recorded Selenium sessions as JUnit to be run headless in JMeter afterwards. How cool is that :) 
  • LoadUI to see what it can do you as well      http://www.loadui.org/  Go through some video tutorial to find out what is capable of  http://www.loadui.org/Getting-Started-with-loadUI/videos.html 
  •  TestMaker. From what I read, it looks worth a try... http://www.pushtotest.com/products.html             By the way: it extends Selenium.
  • Gatling  ( http://gatling-tool.org )
  • For web page performance you can use, PageSpeed, Speed Tracer (both by Google) and YSlow.
2)  http://flywaydb.org/
     The agile database migration framework for Java

3)  JFreeCharts for charting. http://www.jfree.org/jfreechart/

4)  Speaking about databases: SchemaSpy (with SchemaSpyGUI) - extracts DB dictionary,
      constraints,   whatever, in a nice graphical way. And free.
       http://www.joachim-uhl.de/projekte/schemaspygui/

5) Mockito (http://code.google.com/p/mockito/), it is not possible to develop without it:
   "Mockito is a mocking framework that tastes really good."

6) http://code.google.com/p/rest-assured/

7) http://sphinxsearch.com

8) To validate JSON: http://jsonlint.com/

9) I came across html5 http://www.websocket.org/ for client server communication.
    Javascript client   opens a websocket on the server and it works like a tcp socket.

10) Jodd --- Jodd is generic purpose open-source Java utility library. It enriches JDK with
       many  powerful   and feature rich utilities. Jodd helps with everyday task,  makes code
       more robust and reliable. Special attention is put into creating reusable and
      fastest possible code and still keeping it compact, under 570 KB.
       http://jodd.org/
      Current version 3.3.7
      License - BSD

11) iText, Apache POI, Apache FOP (libraries for manipulating documents)
      - Sitemesh, Apache Tiles (web pages layout)

12) Project Lombok - Simplifies few typical tasks that can be very annoying at times.
       Must have for those  not using Roo.
      Website - http://projectlombok.org
      Version - 0.11.4
      License - MIT

13) Joda Time has saved me once when I ran into the many issues of java.util.Date,
       so I decided to stick with it from that moment on. And since JSR 310 might as well never
        come to life   (here we go, standards), there's no better choice than Joda Time.
       http://joda-time.sourceforge.net/
      This application helps to do some precise date time calculation.

14) For report generation I used Jasper reports. It is a very good one.
       It has been integrated in one of our products.
       http://jasperforge.org/

15) Thymeleaf
       "Thymeleaf is a Java library. It is an XML / XHTML / HTML5 template engine
      (extensible to other  formats) that can work both in web and non-web environments.
      It is better suited for serving XHTML/HTML5 at the view layer of web applications, but
      it can  process any XML file even in offline environments. It provides an optional module
      for  integration with Spring MVC, so that you can use it as a complete substitute of JSP
      in your  applications made with this technology, even with HTML5.
            The main goal of  Thymeleaf is to provide an elegant and well-formed way of
     creating templates. Its Standard and  SpringStandard dialects allow you to
     create powerful natural templates, that can be correctly displayed by browsers and therefore
     work also  as static  prototypes. You can also extend Thymeleaf
     by developing your own  dialects.  http://www.thymeleaf.org     Apache 2.0

16) Jackcess, a pure Java library for reading from and writing to MS Access databases.
       http://sourceforge.net/projects/jackcess/

17) Generic class mapper to solve problem of parallel class hierarchy and object transformations
       Enti ty/DTO/VO/Etc http://sourceforge.net/projects/geda-genericdto/

18) Scheduler java task like in Cron.
       http://www.sauronsoftware.it/projects/cron4j/

19) I have got a lot of benefits from my Java preprocessor :) because I spent a lot of years
        for mobile development and it saved a lot of my time and a lot of lines of code
        (also I was told by  many other programmers that my tool was useful for them too)
         http://code.google.com/p/java-comment-preprocessor

20) Google Guava
      This is a library of things that most of us tend to write again and again. Only it is really
      well  thought  out. The string matcher and splitter are two of the best takes on string
       manipulation that I have ever used. The collections classes for filtering are the best thing
      this side of Clojure. http://code.google.com/p/guava-libraries/

21) Event Bus
      http://www.rribbit.org/

22) Gson library, if you are working with JSON.
      It helps you to convert Json to Object and Object to Json.
      http://code.google.com/p/google-gson/
      License : Apache License 2.0

23) Bouncy Castle Crypto APIs Java Release 1.47
      http://www.bouncycastle.org/

24) JSoup 1.7.1  (Java HTML Parser)
       http://jsoup.org/
      MIT License

25) http://flywaydb.org/
      The agile database migration framework for Java

26) Twitter Bootstrap which is becoming the de facto CSS framework. Many developers love
       it because it allows to make an acceptable design for a website with little effort.

27) "XWiki" very sexy WEB platform with LDAP/AD integration, works with most of database,
         multi-lingual able, Webdav and REST API, extendable .

28) Jasmine for testing javascript, http://pivotal.github.com/jasmine/. We can use FTL(Freemarker Template Language--http://freemarker.sourceforge.net/docs/index.html) with any javascript api, like jquery etc

29)What are the framework, toolkit, extentions, libraries available for plotting graphs in various forms in Java. JS tool http://www.highcharts.com/ 

30)
IDE which runs in your browser
GutHub for this:
http://stormcloud-ide.github.com/project/index.html
And put up a showcase to get an impression of where the thing is going:
http://stormcloud-ide.github.com/project/showcase/showcase.html
Group to discuss the possibilities
http://www.linkedin.com/groups/Stormcloud-IDE-4772256


31) Commerical Tool but worth to know. Very few such tools available for testing java concurrency code.

Contemplate, an Edinburgh University spin-out company, has developed an advanced static analysis tool, ThreadSafe, that may be of interest to readers of concurrency-interest.  ThreadSafe specifically targets Java concurrency defects and includes some dedicated treatment for java.util.concurrent.  It handles enterprise-scale Java codebases and includes tight integration with Eclipse.

ThreadSafe is in an early stage - currently we are on version 1.1 - but it is already in use finding serious concurrency defects in projects at a couple of large investment banks.  You can get an impression of what it does from the very simple example in
http://contemplateltd.com/maintaining-safe-concurrent-code-with-threadsafe

32)

OpenXava 4.7 released: Rapid Java Web Development

OpenXava is a Java Framework for Rapid Development of AJAX Web Applications, well-suited for business and database oriented applications. 
How does it work? Simple, write simple JPA entities with Java or Groovy, and get an AJAX application ready for production. 

OpenXava 4.7 has many interesting features such as storing generated reports, Liferay 6.1 support, WebSphere Portal 8 support and many more things.
Look at the original announcement:
http://openxava.org/blog/openxava-4.7-released

I invite you to visit OpenXava site to have a look at the demos:http://www.openxava.org/

33) convert JSON to Map and vice versa
code from http://jiangzhengjun.iteye.com/blog/467334 which will convert your json string into map, list... This uses json library from http://json-lib.sourceforge.net/ .


34) JSON  (These are good JSON tools)
 http://jackson.codehaus.org/
Jackson's streaming library is blazingly fast; consumes not much memory
Jackson: it has (had?) some bugs when using mixins and inheritation. I can't remember the details (it's been a few months since I struggled with the problem),
but if I recall correctly, the mixins could be extended, but if the class under serialization was part of inherited class structure,
Jackson left some properties out of serialization...
Jackson and its good.I am using Jackson 2.0 which now they call FasterXML. Also, pls do consider the transformation time from POJO to JSON.
In my case it was taking 15 ms but it depends on the amount of data that you are transforming.
Jackson's library was for me the best one, as usability and performances.
I really reccomend it. The wiki is very good. I leave here the project's link as help http://jackson.codehaus.org



Both these are very good .

35) Java Performance metric collection or monitoring
1) http://www.centreon.com/Content-Products-IT-network-monitoring/nagios
2) use nmon to capture server CPU, Memory, I/O stat etc.

    nmon analyzer to prepare various report as well 
3) Jconsole & JvisualVM 
4) Zabbix to monitor our server, workstation and network

36)

Leshy - Java Serialization/Deserialization Framework

Leshy: http://serkan-ozal.github.io/leshy/

Leshy is a framework for replacing default Java serialization with your custom implementation on the fly without any code change in your application by using Java Instrumentation API.
 
38)
Jillegal (https://github.com/serkan-ozal/jillegal) is a library including unknown tricks of Java. It abstracts developer from low-level details to implement those tricks. Its design and logic are based on Java 8 so it can be used at only Java 8 platform. Java 6 and Java 7 supports are in progress and as soon as possible they will be released. Demo application is avaiable at https://github.com/serkan-ozal/jillegal-demo.

Design and logic of Jillegal OffHeap module different from all of the other offheap frameworks. It doesn't serilalize/deserialize objects to/from allocated offheap memory region. Becuase objects already lives on offheap and GC doesn't track them :). With this feature, all objects in pool are exist as sequential at memory, so sequential accessing to them is faster. Because, they will be fetched to CPU cache together as limited size of CPU cache.


Sunday, January 6, 2013

Java Telnet Server

This Java server  applications is to support "telnet" like connections. It supports multiple concurrent connections.

Client we can use the classic telnet client like CLI. Telnet Server will responds to basic commands like pwd, cd, mkdir, ls, ls -l, rm. This is portable across platforms.

If this is given as a task. How to implement this?
To implement this we can use of Runtime executor/Process Builder.  But the challenge is Without using these how to develop a telnet server ?

We can make use of Java7 NIO to implement this across platforms.

Little about Java 7 NIO

Path class in Java 7 is similar to File class in Java 6.
The Java I/O File API, as it was originally created, was not written to be extended. Many of the methods do not throw exceptions even when an error is encountered leaving developers wondering what's happening. The following code in Java 6.0 will not throw any exception even if the file is not found in the filesystem.

 File f = new File("c:\\test.txt");
 f.delete();
Shown below is the code in Java 7.0 using Path class but it throws exception when file is not found.
  
 Path fp = Paths.get("c:\\test.txt");   // get Path object
 Files.delete(fp);  // delete file represented by path object
output
======
Exception in thread "main" java.nio.file.NoSuchFileException: c:\test.txt

There is much more about Java 7 NIO2. We will discuss few in this blog.

Access below link for complete source code.
Below class is used to start a socket for concurrent users.
Java FixedThreadPool executor is used for multiple concurrent requests. For each request a  WorkerThread.java is invoked.
For each socket opened through CLI a dedicated WorkerThread is spanned.
Below is the WorkerThread.java.
WorkerThread  has run method which is waiting for input command from CLI.   If the input command is "exit" prompt will exit the console otherwise input command is parsed to return output. Input command can be cd, rm, del ..etc.  output for the command is returned based on OS and command type.

WorkerThread has a class variable CURRENT_DIRECTORY. This is a string which is used to store current direcotry. User can play around with path. he can naivate to different paths in his directory using cd .., cd <dir>. To store his current path we use this variable.

Below method frm WorkerThread java class will parse command based on OS. Based on OS respective method is called. For Unix below method is called "parseUnixCmdSwitch() "is  invoked.
Below is the code to find current OS.
  
Java 7 NIO few basic classes and their use.
The following attribute views are available in the new Java 7 NIO.2 API,
AclFileAttributeView and AclEntry - Supports reading or updating a file's Access Control Lists (ACL). The NFSv4 ACL model is supported. Any ACL model, such as the Windows ACL model, that has a well-defined mapping to the NFSv4 model might also be supported.
The AclFileAttributeView allows you to get and set the ACL and file-owner attributes of a particular file. Its getAcl() method returns a List of AclEntry objects, one for each permission set on the file. Its setAcl(List<AclEntry>) method allows you to modify that access list. This attribute view is only available for Microsoft Windows systems.
BasicFileAttributeView and BasicFileAttributes - Provides a view of basic attributes that are required to be supported by all file system implementations.
FileVisitor Java 7 NIO.2 API allows you to implement scenario in which you want to traverse a directory tree recursively, stopping at each file and directory under that tree and having your own callback methods invoked for each entry found. These callback methods are called while visiting files.
  • FileVisitResult preVisitDirectory(T dir, BasicFileAttributes attrs) is called before the entries in that directory are visited. It returns one of the FileVisitResult's enum values to tell the file visitor API what to do next.
  • FileVisitResult visitFile(T file, BasicFileAttributes attrs) is called when a file in the current directory is being visited. The attributes for this file are passed into the second parameter.
  • FileVisitResult visitFileFailed(T file, IOException exc) is called when the visit to a file has failed. The second parameter specifies the exception that caused the visit to fail. This method is invoked if the file's attributes could not be read, the file is a directory that could not be opened, and other reasons.
  • FileVisitResult postVisitDirectory(T dir, IOException exc) is called after the visit to a directory and all its subdirectories has completed. The exception parameter is null when the directory visit has been successful, or it contains the exception that caused the directory visit to end prematurely.
 PosixFileAttributeView and PosixFileAttributes - Extends the basic attribute view with attributes supported on file systems that support the POSIX family of standards, such as UNIX. These attributes include file owner, group owner, and the nine related access permissions.
This view class, available on UNIX systems only, allows you to get and set attributes specific to POSIX (Portable Operating System Interface). Its readAttributes() method returns a PosixFileAttributes instance containing details of the owner, group owner, and file permissions for this file (those you would normally set using the UNIX chmod command). The view also provides setOwner(UserPrincipal), setGroup(GroupPrincipal), and setPermissions(Set<PosixFilePermission>) methods to modify these attributes.

Now we will go through each  command  like cd, pwd  and how java 7 NIO is used to respond to these commands.

Below code is basic example to get BasicFileAttributes.

Unix LS command:

$ ls -l
-rw-r----- 1 naresh team-dev 9275204 Jun 13 15:27 mthesaur.txt.gz

Field 1 - File Permissions:
Field 2 - Number of links: Second field specifies the number of links for that file. In this example,  1 indicates only one link to  this file.
Field 3 - Owner: Third field specifies owner of the file. In this example, this file is owned by username ‘naresh’.
Field 4 - Group: Fourth field specifies the group of the file. In this example, this file belongs to ”team-dev’ group.
Field 5 - Size: Fifth field specifies the size of file. In this example, ’9275204′ indicates the file size.
Field 6 - Last modified date & time: Sixth field specifies the date and time of the last modification of the file.
               In this example,  ‘Jun 13 15:27′ specifies the last modification time of the file.
Field 7 - File name: The last field is the name of the file. In this example, the file name is mthesaur.txt.gz.

To get permissions we can use below code.

Path p = fs.getPath("/home/nareshk/backup/abc.xml");
PosixFileAttributes attr = Files.readAttributes(path, PosixFileAttributes.class);
String permissions = PosixFilePermissions.toString(attr.permissions());



From the above code. Posix is used to get file permissions, Owner, group, lastModifiedTime, noOfLinks.

RM (remove command)
The Java 7 NIO library allows you to walk a file tree and visit each file in the tree. You do this by implementing a FileVisitor and then calling Files.walkFileTree using the visitor. The visitor has four methods as described at the begining.

Recursively Delete all Files in a Directory:The following code shows how you can recursively delete a directory by walking the file tree. It does not follow symbolic links. I have overridden the visitFile and postVisitDirectory methods in SimpleFileVisitor so that when a file is visited, it is deleted and after all the files in the directory have been visited, the directory is deleted.

Code in the bitbucket has more commands described.

Saturday, November 3, 2012

Java Restful Example

REST, for "Representational State Transfer," is a lightweight architecture for network applications that use HTTP requests to read and write data

Java API for RESTful Web Services (JAX-RS), is a set if APIs to developer REST service. JAX-RS is part of the Java EE6, and make developers to develop REST web application easily.

In this examples we use Jersey popular JAX-RS implementation. Example shows how to develop a simple REST webservice using Jersey Client APIs.  We will use Jersey Client APIs to create a RESTful Java client to perform "GET" and "POST" to the REST service.

Jersey uses Jackson to convert object  to/from JSON. In this example we will convert object to JSON.

1. Jersey jars
pom.xml

2. GET Request


In JAX-RS, you can use @Path to bind URI pattern to a Java method.It  support complex URI matching with regular expression.

Annotate the method with @Produces(MediaType.APPLICATION_JSON). Jersey will use Jackson to handle the JSON conversion automatically.

Here we are converting Track object to JSON format and return back to user. To make Jersey support this JSON conversion we have to use jersey-json.jar  in Maven pom.xml as shown above.

To integrate JSON with Jersey we have to add few parameters to web.xml.

web.xml

In web.xml, declares "com.sun.jersey.api.json.POJOMappingFeature" as "init-param" in Jersey mapped servlet. It will make Jersey support JSON/object mapping.

Access Above REST service in two ways as shown below:

1) We can send "GET" request by hitting the url on the browser    http://localhost:8080/JavaRest2/json/song/get

2) Jersey client to send "GET" request and print out the returned JSON data.


Output from Server:

{"title":"Satyam Sivam Sundaram","singer":"Latha Mangeshkar"}

3. POST Request

@Consumes(MediaType.APPLICATION_JSON) will convert JSON data to object.


Jersey client to send a “POST” request, with json data and print out the returned output.


Output from Server ....

Track saved Track [title=Friday, singer=Rebacca]

Access Source Code:

Friday, October 26, 2012

Spring MVC using MongoDB as Database

In this tutorial we will create a simple Spring MVC 3 application that uses a document-oriented database for its persistence layer. We will be using MongoDB as our database. We will explore and discover how easy it is to integrate MongoDB with Spring MVC 3. Our application is a simple CRUD service for managing a list of Persons. We will provide functionalities for adding, deleting, editing, and viewing of all registered persons.

MongoDB
These are few benefits with MongoDB.
  • Document-oriented storage
  • Full Index Support
  • Replication & High Availability
  • Scale horizontally without compromising functionality.
  • Rich, document-based queries.
  • Atomic modifiers for contention-free performance.
  • Flexible aggregation and data processing.
  • Store files of any size without complicating your stack.
  • Enterprise class support, training, and consulting available. 
In a nutshell MongoDB uses JSON instead of SQL There's no static schema to create. All schemas are dynamic, meaning you create them on-the-fly. You can try a real-time online shell for MongoDB at http://try.mongodb.org/.

In order to complete this tutorial, you will be have to install a copy of MongoDB. You can download from http://www.mongodb.org/display/DOCS/Quickstart. The installation is really easy.

MongoDB is used if there are more number of write than reads. Some thing like Audit logs which will have frequent writes than reads and also huge data will be stored.
pom.xml 
(download source code link is provided for complete project access)


Lets start with MongoDBFactory.

MongoDBFactory.java


method "getMongo()" will be used to  get Mongo DB reference using host and port.
method "getDB()" Will be used to get reference to your DB created. This is some thing like MySql
"create database myDatabase".  This method is trying to get "myDatabase" from MongoDB.
method "getCollection()"  collections are like tables in mongodb stored as key value pairs.

Instead of factory if you want to write the same code it will look like below.
// Retrieve a db
Mongo m = new Mongo( "localhost" , 27017 );
m.getDB("mydb"); // where mydb is the database name
 
// Retrieve a collectionn
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB("mydb");
db.getCollection("mycollection"); // where mycollection is the collection name
 
The same you can see in command prompt below.
 

Now our schema is based on Person class. Below is the class Person we like to store in the DB.
Person.java

Now "PersonService" java class to do CRUD operations.
PersonService.java

This service defines our basic CRUD system. We have the following public methods:
getAll() - for retrieving all persons
edit() - for editing
delete() - for deleting
add() - for adding
get() - for retrieving single person

We also have the following private methods:
init() - for initializing our database
getDBObject() - for retrieving a single Mongo object

The database is initialized once in the PersonService's constructor through the init() method:
private void init() {
  // Populate our MongoDB database
 
  logger.debug("Init MongoDB users");
 
  // Drop existing collection
  MongoDBFactory.getCollection("mydb","mycollection").drop();
  // Retrieve collection. If not existing, create a new one
  DBCollection coll = MongoDBFactory.getCollection("mydb","mycollection");
   
  // Create new object
  BasicDBObject doc = new BasicDBObject();
  // Generate random id using UUID type 4
        doc.put("id", UUID.randomUUID().toString());
        doc.put("firstName", "Naresh");
        doc.put("lastName", "Kancharla");
        doc.put("money", 1000);
        coll.insert(doc);
   
        // Create new object
        doc = new BasicDBObject();
  // Generate random id using UUID type 4
        doc.put("id", UUID.randomUUID().toString());
        doc.put("firstName", "Keerthi");
        doc.put("lastName", "Kancharla");
        doc.put("money", 2000);
        coll.insert(doc);
         
        // Create new object
        doc = new BasicDBObject();
  // Generate random id using UUID type 4
        doc.put("id", UUID.randomUUID().toString());
        doc.put("firstName", "Kanish");
        doc.put("lastName", "Gupta");
        doc.put("money", 3000);
        coll.insert(doc);
   
 }
 
If you Notice we're creating a dynamic JSON schema here with the following format:
{ 
   id:'',
   firstName:'',
   lastName:'',
   money:''
}
 
To run this flow we need Spring MVC Controller.

To view complete code below i provided GitHub source code link.

This controller declares the following mappings:
/persons - for retrieving all persons
/persons/add (GET) - displays the Add New form
/persons/add (POST) - saves the new person
/persons/delete - deletes an existing person
/persons/edit (GET) - displays the Edit form
/persons/edit (POST) - saves the edited person

Each mapping delegates the call to the PersonService.
When the PersonService is done processing, the controller then forwards the request to a JSP page
that displays a confirmation message. Here are the JSP pages.

personpage.jsp





editpage.jsp

addpage.jsp
 
  editedpage.jsp

addedpage.jsp
deletedpage.jsp
Spring MVC Configuration xmls are defined below.

web.xml
<servlet>
 <servlet-name>spring</servlet-name>
 <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
 <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
  <servlet-name>spring</servlet-name>
  <url-pattern>/</url-pattern>
</servlet-mapping>
<listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class></listener>
Above web.xml we defined DispatcherServlet as "spring" so there should be a xml with name "spring-servlet.xml" for view resolution.
spring-servlet.xml

 <!-- Declare a view resolver -->
 <bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"
      p:prefix="/" p:suffix=".jsp" />


applicationContext.xml
<!-- Activates various annotations to be detected in bean classes -->
 <context:annotation-config />

 <!-- Scans the classpath for annotated components that will be auto-registered as Spring beans.
  For example @Controller and @Service. Make sure to set the correct base-package-->
 <context:component-scan base-package="*" />

 <!-- Configures the annotation-driven Spring MVC Controller programming model.
 Note that, with Spring 3.0, this tag works in Servlet MVC only!  -->
 <mvc:annotation-driven />

When the application is initially run, we mentioned that it will drop and create a new collection. So MongoDB's console should reflect this action as well.


Now let's access the main page that shows all registered persons. To access the main page, enter the following URL in your browser:
http://localhost:8080/SpringMongoDB/main/persons
Here's what you should see:

Here's the log from MongoDB's console: Fri Oct 26 20:32:08 [initandlisten] connection accepted from 127.0.0.1:51259 #6 (3 connections now open)

Let's edit the first record in our list by clicking the Edit link. We will be redirected to the Edit Person page:


Here's the log from MongoDB's console:
Fri Oct 26 20:34:40 [conn8] update mydb.mycollection query: { _id: ObjectId('508aa5ecad8450d889bc351a'), id: "1d11324e-ee91-4e50-84ef-b7e5e141


, firstName: "Keerthi", lastName: "K", money: 2500.0 } nscanned:1 nupdated:1 keyUpdates:0 locks(micros) w:107793 108ms

After editing the person and submitting the changes, we should see the following page:


That's it. We've managed to create a simple Spring MVC 3 application that uses MongoDB
for its database. We've also seen the benefits of a document-oriented database
when we created the schema dynamically.

Download the source code: