Patterns and Message Broker

I see WebSphere Message Broker has just announced v7. The product has been brewing for considerably longer than my involvement and we are all working flat out to finish it off. I've been working on Broker for a little over six months now. My bit of the jigsaw is in a new feature called patterns.

Patterns solves a conundrum familiar to developers - it goes like this:

Broker is a product that can do, well, just about anything. It's purpose in life is to connect everything to everything. More than that, it routes, transforms, filters, and validates messages and data along the way. But this flexibility can easily be an Achilles heal.

How, as an inexperienced user of the technology, do I learn to use the product? How do I solve common problems? I don't want to make mistakes, so how do I avoid them? And of course how can I find out what best practices exist?

Samples is one important answer of course! Samples typically demonstrate a particular feature of a product. Very rarely are samples intended for production use. Indeed, many samples remove production requirements like error handling altogether! Another important answer is documentation. This includes concept, task and reference material.

Patterns is different. Patterns creates production ready solutions to problems. For example, perhaps I want to create a Web service front end to a WebSphere MQ application (a frequently asked about problem). No problem, simply create an instance of the Service Facade to WebSphere MQ: one-way pattern. Provide some configuration details, like the queue name, and click Generate. Lo and behold out pops a Message Broker project that implements the pattern! Need advanced logging and error handling? No problem, just enable those options in the pattern parameters.

In v7 we are providing a set of patterns that cover a surprisingly wide set of problems. Need to process messages stored in files? No problem, try the File Processing pattern. Need to put a proxy between a Web service client and server? Try the Service Proxy pattern. And so on.

If you would like to read more, try the Introduction to Patterns developerWorks article.

UTF-8 Problems and Subversion

I was writing some unit tests recently and needed to store files containing UTF-8 text in Subversion. The file names also contained UTF-8 characters (see below). I was using Eclipse for the development work and ran into problems sharing the files between Linux and Windows.

The problem was that a file created and committed by a Windows machine, was not synchronising correctly to the Linux machine (and vica versa). The character encodings in the file names were being changed somewhere along the commit and synchronise path.

So here is what I needed to do to make it work. First I needed to make sure Eclipse marked the contents of the file as UTF-8. This is straightforward, simply right click the file name, select Properties and change the text encoding to UTF-8 (on Windows the default is Cp1252):

The next step involves configuring the CVS repository. Change to the CVS perspective, right click the repository and select Properties:

Change the Server Encoding to UTF-8 (on Linux the default Server Encoding is UTF-8, hence the problem sharing files):

The files should now commit and synchronise with the correct contents and names. When you commit the files you should probably also commit the org.eclipse.core.resources.prefs file that Eclipse generates in your project. This file stores the character encodings for the file contents.

Integrating SAP NetWeaver in Zero PHP

I've been playing with the Big Daddy of enterprise systems, SAP. 

I thought it would be interesting to hook up a Zero PHP application with a SAP NetWeaver system. So last weekend I summoned up the courage and, armed with a bottle of red wine, set about the task. Surprisingly enough it wasn't too difficult (setting aside finding 20GB of disk space needed by the installer).

The bit I was interested in was the SAP Java connector. SAP Java Connector (SAP JCo) is a middleware component that enables the development of SAP-compatible components and applications in Java. Since we have a PHP/Java Bridge in Zero I thought it should be possible to wire them up.

Here is an example that invokes a SAP remote function call (RFC):

<?php

try {

java_import("com.sap.mw.jco.JCO", NULL, FALSE, NULL, FALSE);

// Establish the client connection to SAP

  $connection = JCO::createClient("000", "bcuser", 

  "minisap", null, "localhost", "00");

$connection->connect();

$repository = JCO::createRepository("ZeroRepository", $connection);

  $template = $repository->getFunctionTemplate("RFC_SYSTEM_INFO");

  $function = new Java("com.sap.mw.jco.JCO.Function", $template);

$connection->execute($function);

$list = $function->getExportParameterList();

  $structure = $list->getStructure("RFCSI_EXPORT");

$iterator = $structure->fields();

  while ($iterator->hasMoreElements()) {

  $field = $iterator->nextField();

  echo $field->getName()." : ".$field->getString()."\n";

}

$connection->disconnect();

} catch (Exception $exception) {

echo $exception->getMessage();

}

?>

The code turns out to be pretty straightforward. In a production environment it would be better to use some of the connection pooling options to avoid the repetitive connect/disconnect overhead. The function RFC_SYSTEM_INFO doesn't require any arguments but those can be passed as well if required.

If anyone is interested in SAP there is an un-conference event being organised by Nigel James and Zoe Slattery on Saturday 4th April. More details here (beware the SAP/SDN sites do not work in Google Chrome). I am planning to be there to chat about Zero/Java/PHP with SAP NetWeaver.

developerWorks Article: Business Intelligence

The developerWorks article on business intelligence I mentioned a while back has been published. It covers several use cases for integrating BIRT reports into a Zero PHP application. It uses the sample database and reports provided by BIRT and runs against a MySQL database.

From my perspective the most interesting example is the last one: scripted data sources. In this use case the report doesn't get its data from a database query. Instead the PHP script creates the data and pushes it into the BIRT runtime ready for the report to render. Here is the sample code:

<?php

require_once 'startup.php';

$engine = startup();

$root_directory = zget("/config/root");

$report_engine = $root_directory."/ReportEngine";

$design_file = $root_directory.

    "/samples/data_source.rptdesign";

$images_directory = $root_directory.

    "/public/images/logos";

java_import(

    "org.eclipse.birt.report.engine.api.HTMLRenderOption");

java_import(

    "org.eclipse.birt.report.engine.api.HTMLServerImageHandler");

$design = $engine->openReportDesign($design_file);

$task = $engine->createRunAndRenderTask($design);

$task->validateParameters();

$generated_report_file = 

    tempnam(sys_get_temp_dir(), 'report');

$models[] = array("ANG Resellers", 

"1952 Alpine Renault 1300", "Red");

$models[] = array("AV Stores, Co.", 

"1969 Harley Davidson Ultimate Chopper", "Blue");

$models[] = array("Alpha Cognac", 

"1969 Ford Falcon", "Blue");

$models[] = array("Asian Shopping Network, Co", 

"1969 Dodge Charger", "Plum Crazy Purple");

$models[] = array("Asian Treasures, Inc.", 

"1969 Corvair Monza", "Red");

zput("/request/report", $models);

$options = new HTMLRenderOption();

$options->setOutputFileName(

    $generated_report_file);

$options->setOutputFormat("html");

$options->setImageDirectory($images_directory);

$options->setBaseImageURL("/images/logos/");

$options->setImageHandler(new HTMLServerImageHandler());

$task->setRenderOption($options);

$task->run();

$task->close();

echo file_get_contents($generated_report_file);

unlink($generated_report_file);

?>

The bold section hard codes some data for the report. A more realistic example might query an external system like a web service to build the data set. 

The magic that makes this possible is the JavaScript engine that BIRT embeds. When BIRT renders the report it executes JavaScript embedded in the report with the Rhino scripting engine. Because the BIRT runtime and the PHP application are all running on the same JVM they can share data very easily and efficiently. In this case the PHP code puts the data collection into the Global Context, and the JavaScript in the report pulls it out.

Business Intelligence - The Code

A little later than intended - here is the code we were using on the Zero stand at the (most excellent) PHP London conference last Friday. I had it running as a blog in Drupal but this will run just as well standalone. More detailed destructions for getting this running are up on YouTube here (the video is six minutes long).

<?php

require_once 'startup.php';

$engine = startup();

$root_directory = zget("/config/root");

$design_file = $root_directory."/samples/hello_world.rptdesign";

$design = $engine->openReportDesign($design_file);

$task = $engine->createRunAndRenderTask($design);

$task->validateParameters();

java_import("org.eclipse.birt.report.engine.api.HTMLRenderOption");

$generated_report_file = tempnam(sys_get_temp_dir(), 'report');

$options = new HTMLRenderOption();

$options->setOutputFileName($generated_report_file);

$options->setOutputFormat("html");

$task->setRenderOption($options);

$task->run();

$task->close();

echo file_get_contents($generated_report_file);

unlink($generated_report_file);

?>

And the include file that configures the BIRT runtime is as follows:

<?php

function startup() {

  zpost("/app#lock", "/engine"); 

  $engine = zget("/tmp/factory");

  if ($engine != NULL) {

zpost("/app#unlock", "/engine"); 

return $engine; // Did someone else get in first?

  }

  $root_directory = zget("/config/root");

$report_directory = $root_directory."/birt-runtime-2_3_1/ReportEngine";

  $log_directory = zget("/config/appLogDir");

java_import("org.eclipse.birt.core.framework.Platform");

  java_import("java.util.logging.Level");

  java_import("org.eclipse.birt.report.engine.api.EngineConfig");

  java_import("org.eclipse.birt.report.engine.api.IReportEngineFactory");

  $config = new EngineConfig();

  $config->setBIRTHome($report_directory);

  $config->setLogConfig($log_directory, Level::$SEVERE);

  Platform::startup($config);

$factory = Platform::createFactoryObject(

  IReportEngineFactory::$EXTENSION_REPORT_ENGINE_FACTORY);

  // Store in process wide non persistent zone

$engine = $factory->createReportEngine($config);

  zput("/tmp/factory", $engine);

  // Unlock when we are finished!

  zpost("/app#unlock", "/engine"); 

return $engine;

}

?>

PHP London Conference

Just a few more days until this year's PHP London Conference. I think it is going to be a brilliant event given the outstanding speaker line up. The PHP London User Group is also hosting a free pre-conference social on Thursday night with guest speaker and PHP contributor Derick Rethans.

I know little about the new features in PHP 5.3 (shame on me) so the talks by Scott MacVicar and Sebastian Bergmann should be well worth taking in. Stuart Herbert's talk on PHP frameworks is sure to be interesting and no doubt will create plenty of audience discussion. There is lots more with talks about security, cloud computing and a term I had never heard before, database sharding.

My small contribution to the event is to put together a couple of demo applications for the Zero stand. Given my main thing is language interop I thought to start with some PHP/Java Bridge examples. The BIRT stuff I rambled about the other day fits nicely and produces some glossy charts just perfect for a demo [:o)

Thanks to Rob we also have a demo using Apache POI (the project name is a story in itself). POI is interesting because it provides a cross platform library for reading and writing Microsoft Office file formats. POI is really easy to use, just grab the download from the POI web site and extract the three JAR files. In Zero they need to go into the lib directory.

<?php

java_import("org.apache.poi.hslf.HSLFSlideShow");

java_import("org.apache.poi.hslf.usermodel.SlideShow");

$fs = new Java("java.io.FileInputStream", "/test.ppt");

$hsss = new HSLFSlideShow($fs);

$ss = new SlideShow($hsss);

foreach ($ss->getSlides() as $slide) {  

    echo $slide->getTitle();

}

?>

The code simply iterates through the titles of each PowerPoint slide. All very straightforward and just the tip of the iceberg for what POI can do.

Singletons, BIRT, Threads and PHP

PHP is designed to run in multithreaded web servers (such as Apache and IIS). What this means in practice is that a single PHP process concurrently processes multiple browser requests. It does this by having multiple threads, each of which handles one HTTP request at a time.

PHP can also be built for non multithreaded web servers. In this deployment model life is somewhat different because the web server spawns multiple PHP processes. When requests arrive in the web server it farms them out among its pool of PHP worker processes. Each worker process handles just one request at a time. 

The pool of worker processes can grow and shrink over time to allow for busy periods. Zero is multithreaded so it handles multiple web requests concurrently in the same process.

I’ve been playing with the Eclipse Business Intelligence and Reporting Tools project (BIRT). BIRT is an open source Eclipse project that provides a whole bunch of stuff around designing and deploying reports and charts. It has a Java API and so I thought it would make a nice demonstration of the PHP/Java Bridge in Zero.

The problem is that BIRT needs to be started once, and only once, in any given process. Once it has been started up, then it is plain sailing to load reports and render them to HTML. The BIRT runtime should be kept around until the process shuts down. 

This sounds suspiciously like a singleton pattern (or anti-pattern if you prefer). The trouble is that process wide singletons are hard to implement correctly in PHP. You might think the following snippet would work:

class Singleton {

private static $birt = NULL;

    static function getInstance() {

  if (self::$birt == NULL) {

    self::$birt = new BIRT(...);

  }

  return self::$birt;

    }

}

There are two problems with this. First, at the end of the current request, all statics are cleared away so $birt is reset to NULL. What we want is the BIRT runtime to be kept around forever (or at least until the process shuts down). 

The second problem is that on a multithreaded PHP runtime each PHP request gets an independent set of static and global variables. If two requests are being processed, and each calls getInstance, then they will both create an instance of the BIRT class. 

The design intent in PHP is that each request looks like it is running in a single threaded PHP process, even though it is actually running in a multithreaded PHP process. But what I want is one (yes really one!) object in a process which can be shared amongst all the threads. The reason this works is that the BIRT runtime is thread safe, meaning that it can be safely used by multiple threads at the same time.

The solution for me turned out to be the Zero Global Context. The Global Context is a storage area in Zero where just about anything can be put. Different bits of the Global Context have different life times. For example, the /request zone is only kept around for the current request. I opted to put the BIRT runtime in the /tmp zone so that it is kept around for the life time of the process. It is also available to all threads running in the same process.

The final piece of the jigsaw was to ensure only one thread actually created the BIRT runtime. Dealing with multiple threads can be something of a headache as race conditions are easily created.

The Global Context provides a solution (albeit slightly crufty) with locking commands. Each thread does a lock, checks to see if the BIRT runtime has already been created (if not then it creates it and puts it in the /tmp zone) and then unlocks.

With a bit of luck I'll get this written up in an developerWorks article soon as the reports look really sweet (cross tabs, charts, compound reports and so).