Mostrando entradas con la etiqueta jongo. Mostrar todas las entradas
Mostrando entradas con la etiqueta jongo. Mostrar todas las entradas

Changes in Jongo 0.2

Lots of changes are being introduced in Jongo 0.2 which I believe are better suited for the scope of the project.


First and most important, I decided to remove the administration console which was to be used as a way for an administrator to customize access to resources, set ID columns and add complex queries. The problem with this approach is that I don't want Jongo to be another tool you have to worry about to keep it running, and most of this things are already on your RDBMS! The basic idea of Jongo: embrace the use of any SQL server.






So, the problems the administration console tried to fix should be fixed on the database:


Resources permissions. Use an specific user for Jongo, and customize access for this user.


Complex Queries. The idea was to allow a GET request to execute a predefined query which was created in Jongo. But why do this when you have triggers, stored procedures, views, etc? So this had to go. Also, stored procedures run faster than running the same query over and over again.


http//.../Jongo/db/sp/myStoredProcedure


[Note: stored procedures are still not supported, but should be in the future]


Custom ID column. If your resource is not using an ID column, this is now supposed to be sent on the requests path:


http//.../Jongo/db/resource/1?customId=rid


Since the administration console is gone, there's no need for Jongo to keep a database to hold its data, so this and the hsqldb.jar were removed too.


Another big change being introduced is the correct use of headers in the request and in the response. In 0.1 the "Accept" header wasn't used at all, and this is not acceptable in REST. So now, instead of changing the format as a path parameter, Jongo handles the appropriate headers. 


$curl -H "Accept: application/xml" "http://.../jongo/db/user/1"


On the other hand, response headers include a lot of information about the data being returned: Content-Length, Content-MD5, Content-Count, Date and more depending on the response (error, success, head, stats).

HTTP/1.1 200 OK
Content-Type: application/xml
Content-Count: 1
Content-Location: user
Content-Length: 249
Date: 2012-03-30T12:38:11.581+02:00
Content-MD5: ulYx9Vsbymcq7ExWzdWlFw==
Server: Jetty(8.1.2.v20120308)


During the day I've been working extensively with JAX-RS at work with a project which is very similar to Jongo in how both work, so both of the projects share ideas, and being able to customize the response's JSON and XML is very important. Because of this Jongo will include some filters which will try to ease the unmarshalling in, for example, JAX applications, ExtJS and more.



<response>
  <success>OK</success>
  <resource>user</resource>
  <rows>
    <row>
      <cell>
        <birthday>1992-01-15</birthday>
        <credit>0.00</credit>
        <lastupdate>2012-03-30T12:38:11.576+02:00</lastupdate>
        <name>bar</name>
        <age>33</age>
        <id>1</id>
      </cell>
    </row>
  </rows>
</response>



By default the XML and JSON returned by Jongo is the same as in version 0.1 but if you configure Jongo's web.xml file to use no filter, then JAX is used to do the marshaling.



<response>

  <status>OK</status>
  <success>true</success>

  <resource>user</resource>
  <rows>
    <roi>0</roi>
    <cells>
      <entry>
        <key>BIRTHDAY</key>
        <value>1992-01-15</value>
      </entry>
      <entry>
        <key>CREDIT</key>
        <value>0.00</value>
      </entry>
      <entry>
        <key>LASTUPDATE</key>
        <value>2012-03-30T12:30:27.768+02:00</value>
      </entry>
      <entry>
        <key>NAME</key>
        <value>bar</value>
      </entry>
      <entry>
        <key>AGE</key>
        <value>33</value>
      </entry>
      <entry>
        <key>ID</key>
        <value>1</value>
      </entry>
    </cells>
  </rows>
</response>



The objective is to allow to develop different formats which can be read without much development on the consumers.


I'm also working on tests and trying to achieve a 90% coverage. Doing this is a great exercise on fixing code structure and abstraction. Being able to setup an in-memory database with HSQLDB also helps a lot.


Gone is all the code to handle the different database error codes returned by the JDBC drivers. I had the hope (silly me) that the different RDBMSs would follow some sort of standard at least on this, but no, each one of them returns a different SQLStatus & SQLCode for the same error, so it was very frustrating and I didn't want to waste my time researching for the correct codes and their meaning for every different system. The solution? 



<response>
  <success>false</success>
  <message>data exception: invalid character value for cast Query: SELECT * FROM user WHERE id = ? ORDER BY id ASC LIMIT 25 OFFSET 0 Parameters: [1*]</message>
  <sqlState>22018</sqlState>
  <sqlCode>-3438</sqlCode>
</response>



More than a solution, it's a workaround: return the error message and codes to the client and let it handle it as it sees fit. Remember, embrace your database!


Another feature gone is the "apps" folder, so by default, you won't be able to use Jongo as a JavaScript application server, although, there's a way to serve static content which I'll explain in the future.


Finally, I separated the project in three repositories:

  • jongo (common files)
  • jongo-jetty (standalone)
  • jongo-as (war)

I tried to get a version of this WAR working with Google's GAE but it was very frustrating as a PaaS. What I believe is a better service is the one provided by Jelastic and I'll document a guide on how to prepare Jongo's WAR to work with it.


That's it for now, I hope the few users of Jongo don't get pissed because of this changes.

Sharing Apache Tomcat and Apache Web Server

Today's problem is sharing one single virtual host with different Tomcat instances. For example, say you have your Apache (the web server) with a virtual host for intranet and you have in different Tomcat instances different applications like: monitoring, dashboard and hradmin. You want your users to be directed to the correct application if they access the URL http://intranet/dashboard or http://intranet/hradmin.

In this example I'm using three Tomcat servers and this applications:

  • server1:8080/monitoring/
  • server1:8080/dashboard/
  • hr1:8080/hradmin/
  • hr2:8080/hradmin/

The server1 Tomcat is running two different applications and the hr1 and hr2 are running the hradmin application which we want to be load balanced and since it's critical, to be always available.

The first thing to do is create the jkworkers.properties file:

# This names are the ones defined bellow
worker.list=server1,hrlb
worker.lock=P
worker.status.type=status

# Single server running monitoring and dashboard
worker.server1.type=ajp13
worker.server1.host=192.168.1.2
worker.server1.port=8080
worker.server1.disabled=false
worker.server1.socket_timeout=60000
worker.server1.connect_timeout=60000

# Two workers for two servers running a critical application
worker.hr1.type=ajp13
worker.hr1.host=192.168.1.3
worker.hr1.port=8080
worker.hr1.disabled=false
worker.hr1.socket_timeout=60000
worker.hr1.connect_timeout=60000

worker.hr2.type=ajp13
worker.hr2.host=192.168.1.4
worker.hr2.port=8080
worker.hr2.disabled=false
worker.hr2.socket_timeout=60000
worker.hr2.connect_timeout=60000

# Loadbalancer configuration for the critical application
worker.hrlb.type=lb
worker.hrlb.balance_workers=hr1,hr2
worker.hrlb.sticky_session=true
worker.hrlb.sticky_session_force=false
worker.hrlb.method=R

Of course, you should use the DNS names instead of the IP addresses. Also, there are many more options to be set here, but this should give you a good start.

Second, you need your jk.conf file:

JkWorkersFile /etc/apache2/conf.d/jkworkers.properties
JkLogFile /var/log/apache2/mod_jk.log
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
JkOptions +ForwardKeySize +ForwardURICompatUnparsed -ForwardDirectories
JkRequestLogFormat "%w %V %T"
JkShmFile /var/log/httpd/jk.shm
<Location /jkstatus>
    JkMount status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>

And finally, you can set up your virtual host in the, for example, intranet.conf file:

<VirtualHost *:80>
  ServerName intranet
  RewriteEngine on

  # Here we set where we want the url http://intranet/ to go
  RedirectMatch ^/$ /dashboard/

  DocumentRoot /srv/www/htdocs/docroot
  CustomLog  /var/log/apache2/access_reports combined
  ErrorLog  /var/log/apache2/error_reports
  
  # You can even setup a jongo proxy
  ProxyPass /jongo/ http://192.168.1.5:8080/jongo/
  ProxyPassReverse /jongo/ http://192.168.1.5:8080/jongo/
  
  # It's important that this names are the same used in your Tomcat
  JkMount /dashboard/* server1
  JkMount /monitoring/* server1
  JkMount /hradmin/* hrlb
</VirtualHost>

What we're doing here is telling the Apache server to listen on port 80 for all requests to the intranet host and to forward this requests to their corresponding applications:

  • http://intranet/ goes to http://server1/dashboard/
  • http://intranet/dashboard/ goes to http://server1/dashboard/
  • http://intranet/monitoring/ goes to http://server1/monitoring/
  • http://intranet/hradmin/ goes to http://hr1/hradmin/ or to http://hr2/hradmin/

Where you put this files depends on your httpd.conf configuration.

This example will allow a vanilla Apache to serve three different applications (one load balanced) from three Tomcat servers (server1, hr1 and hr2)

Python and Jongo

For some reason or another, I've always found Python support for RDBMS lacking behind Java, specially in the ORM area. Just imagine, there are no officially supported Python drivers to connect to an Oracle database and you must install the Oracle client... ugh!

Anyway, by having a Jongo server connected to your database you don't have to worry about installing any Python modules, you'll only need a standard Python 2.4+ installation with simplejson or Python 2.5+ which already support JSON.

Let's see, for example, an application which accesses the CIDB database and the Car table:

import jongo

class Car(jongo.JongoModel):
    def __init__(self, id=None, model=None, maker=None, fuel=None, transmission=None, year=None):
        jongo.JongoModel.__init__(self)
        self.id = id
        self.idCol = "cid"
        self.model = model
        self.maker = maker
        self.fuel = fuel
        self.transmission = transmission
        self.year = year

class CarStore(jongo.JongoStore):
    def __init__(self):
        jongo.JongoStore.__init__(self)
        self.model = Car
        self.proxy = jongo.Proxy("localhost:8080","cidb","car", Car)

if __name__ == '__main__':
    carstore = CarStore()
    carstore.load()
    for car in carstore.data:
        print car

    c1 = Car(None, "206cc", "Peugeot", "Gasoline", "Manual", 2008)
    carstore.add(c1)
    carstore.sync()

    c1 = carstore.getAt(carstore.count() - 1)
    c1.set('model', "206")
    c1.set('maker', "PPegoushn")
    carstore.update(c1)
    carstore.sync()

    # We need to refresh the object since it has changed after the sync
    c1 = carstore.getAt(carstore.count() - 1)
    carstore.remove(c1)
    carstore.sync()

Since this is Python, the code is pretty straightforward and self documented, right? For more information and examples on working with Python and Jongo check this page.

Jongo Beta 0.1

Last night I uploaded the first version of Jongo. This is a beta version and all of the projected functionality is provided.

  • All REST to CRUD operations are implemented (PUT, POST, GET, DELETE).
  • JDBC errors mapping to HTTP errors.
  • POST requests accept form and JSON data.
  • Full HTTP response with JSON or XML
  • Grails style dynamic finders.
  • Complex Queries.
  • Deploy-able web applications in the apps directory.
  • Administration Console with jQuery.
  • Demo mode which creates a in-memory database with demo data and deploys a demo application called cidb (made using ExtJS)
  • Support for MySQL, Oracle and Hypersonic.


In future versions support for more RDBMS will be added, like Microsoft SQL Server and Sybase using the jTDS JDBC Driver. Also support for PostgreSQL and SQLite is projected.

A couple of screenshots:



This is the administration console, from where you can control the tables Jongo will provide access to. It's completely made with jQuery and is a good example of how to work with jQuery.Ajax() and Jongo.





The cidb application is a demonstration of how to use ExtJS rest stores without any server side coding.

If you find Jongo useful please leave your feedback. Find any bugs, errors or have a cool idea? Open a ticket!

Jongo as a JavaScript Application Server

Jongo is built on top of the Jetty webserver, allowing a developer to use only one server to build a full application using static content (html, css, images) and JavaScript. For Jongo to be able to deploy your application, you need to activate the jongo.allow.apps option in the jongo.properties file.

Continue and create a folder in your JONGO_HOME named apps/myapp and develop you application there, something like this:

An example of a MVC ExtJS Application
If everything went fine, the next time you run jongo, you'll see this in your logs (if debug mode is enabled):


2011-12-05 22:02:04 o.j.JongoJetty [DEBUG] Loading apps
2011-12-05 22:02:04 o.j.JongoJetty [DEBUG] Creating Servlet for myapp

Now navigate to http://jongo-server:port/myapp/ and your application should work.

Jongo and ExtJS

To work out some issues other developers might have when working with Jongo, I've started to develop a  JavaScript only application (pretty much like the Administration Console) and get some insights into using  a great JavaScript framework named ExtJS.

ExtJS provides a clean MVC framework which fits perfectly with a Jongo back-end.

By using a rest proxy like this:



Ext.define('User', {
    extend: 'Ext.data.Model',
    fields: ['id', 'name', 'email']
});

Ext.define('Users', {
    extend: 'Ext.data.Store',
    model: 'User',
    proxy: {
        type: 'rest',
        url : '/jongo/user',
        reader: {
            type: 'json',
            success: true,
            root: 'response'
        }
    }
});


Our application is ready to perform all rest operations on the database without doing any server-side coding. Give it a try, you'll be pleased.

RESTing with Jongo

For the last few weekends and as part of a week of holidays where I had to stay at home taking care of my convalescent wife I started a project called Jongo which is a RESTful interface for JDBC that allows any application to directly access a RDBMS using the different HTTP methods. The idea came from working with CouchDB's RESTful JSON API but using the ability of Java's JDBC to be database independent.

So what does all this gibberish means? To put it in a simple way, you can, for example, develop JavaScript only applications with a database back-end without doing any server side development.

For example, say you have a table with some users and you want to GET the first one:

$ curl -X GET "http://localhost:8080/jongo/user/1"
{"success":true, "count":1, "resource":"user", "response":[{"birthday":"null","credit":"45.00","lastupdate":"null","name":"foo user","age":30,"id":4}]}

If you want to INSERT a new user, you would POST it with:

$curl -X POST -d '{"name":"foo user", "age":30, "credit":45.0}' "http://localhost:8080/jongo/user"
{"success":true,"response":[{}]}

Since the default format is JSON, we provide a JSON object or an array of JSON objects. This will be formatted to native SQL values and inserted in the database as new user.

To UPDATE an user, you would perform a PUT request with the new values:


$ curl -X PUT -d  '{"name":"user change", "age":34}' "http://localhost:8080/jongo/user/4"
{"success":true,"response":[{}]}


And finally, the DELETE:


$ curl -X DELETE "http://localhost:8080/jongo/user/4"
{"success":true,"response":[{}]}

I hope this explains a little what a RESTful webservice is and how Jongo helps on publishing your RDBMS without any server-side development.