Sunday, 8 March 2009

Data Integration ... the way ahead

A while back I spent quite some time building a java process to process multiple data feeds and add the data into a separate database.

Well, the recent weeks have been spent using a highly commendable open source tool called Pentaho Data Integration formerly Kettle.
http://kettle.pentaho.org/

This sort of rapid development tool is similar to Microsoft SSIS.
The installation comes with a wide range of sample examples of Jobs/Transitions/Mappings so that you can start to investigate what's involved and how it works.

They also have very good documentation and an extensive wiki.
http://wiki.pentaho.com/display/EAI/Latest+Pentaho+Data+Integration+(aka+Kettle)+Documentation

I have to say the developers on this project are so keen to assist (Matt Casters, thanks!) People do tend to worry about the support and backup one can get from open source - well, I had a few queries as a newbie to the tool and the response on the forum to my questions have been prompt and extremely helpful.

If you want to extract and share data in different formats this is a brilliant tool and their sister projects Pentaho BI Suite and Pentaho Reporting look most interesting.

I am hooked and see this sort of development being a valued tool to add in the armoury.

No comments:

Post a Comment