Archive for category Palm

Some of you will remember my earlier blog posts in August 2009 on the HTC Hero, the first Android device to ship with Flash.  The goal of the Android experiment was to learn about the Android SDK, the development process and discover how Flash was enabled in the context of the browser.

As we edge closer to the release of Flash Player 10.1 in the first half of this year, it seems appropriate to revisit these posts.  In this post you’ll see that there are hidden benefits to using a common runtime across device platforms, some of which are not that obvious.

The result of my week long investigation is that (using the beta version) I can hook up Flash Player 10.1 to the camera; but that’s just the start.  Let’s look at how it’s done…

WebView

Those of you familiar with Nokia’s WebRuntime or the iPhone UIWebView will recognize WebView, because it’s the same thing.  Really it’s an implementation of the browser made available as a UI component for native applications.  So this means that you can create an application in HTML/JScript and manage the user experience through a native shell.

So why is WebView useful?  Let’s look at the relevant classes:

• WebView – Used to load and display web pages using the built-in device browser and chrome, embedded into your application.
• WebViewClient – Enables the handling of various browser actions like page loading and error handling.  Overrides the Activity in the built in browser.
• WebChromeClient – Enables the replacement of the browser chrome for events like progress, alerts and for window controls.  Can override the default Chrome.

So using a standard WebView I can use the core browser, or if I choose, extend this and replace the chrome and user experience.  If you were reading carefully, inside WebView you will have noticed a method called addJavascriptInterface, and that’s where it gets interesting because this allows you to create a JavaScript front end to a native class.

In effect you can then write anything you want using native APIs, and add that functionality to WebView in the form of JavaScript interfaces.  Wait!  Doesn’t Flash have the ability to speak to JavaScript?

Flash Player – ExternalInterface

This is a long standing ExternalInterface API in Flash Player that enables communication with JavaScript in the browser.  It’s primary purpose is actually to help Flash live alongside HTML, enabling Flash to signal JavaScript and pass messages back and forth.  So in principle, if Flash can communicate with JavaScript and JavaScript with Native code, then we can start adding functionality.

Sounds really simple right?

PhoneGap

The folks at Nitobi have already made a huge head start creating PhoneGap, an open source set of cross-platform Javascript APIs for HTML applications, just some of which are below.  The PhoneGap framework also enables anyone to create their own extensions based on the same principle, and although it’s quirky, the process is relatively simple.

In the git repository the team are also hard at work on functionality like Text-to-speech, Camera and File access for Android.  Remember that this is an open source project, it’s changing just about every day and they need helpers.  What’s really exciting is that PhoneGap workshops, including in person and online events; so I suggest attending!

Porting PhoneGap

As you have read previously, we are working to bring Flash Player 10.1, which includes the ExternalInterface, to Android in the next few months.  Now, my Nexus One is an Android 2.1 device, and that creates a slight problem because PhoneGap is only compatible with Android 1.6 APIs.

This means that I had to port code and complete some APIs, in fact I’ve completed the Contacts API, updated the Camera, Location and Accelerometer APIs and did a spot of bug fixing get things working.  In total this took about five days, to get setup and familiar with the code, and then writing some code for a few hours here and there.  Using Open Source kit often results in these sticky issues, hint, that’s why Adobe put so much effort into documentation

With the Android SDK it really couldn’t be simpler to perform tests once I got started, really it couldn’t be easier to do live debugging on Windows, Mac and Linux; a great SDK from Google.  Once you get into the flow of debugging, coding and testing it’s all good.

My advice, as always, is to think about functionality that you’d like to add and go for it.  There are lots of docs out there, and alot of helpful code available for hacking something together.

FlashCam

After playing with PhoneGap and testing out my changes/addition, I decided to put the framework to the test and build an application.  The first API test was for the Android Camera, and I’m calling it FlashCam.

With FlashCam I have created a simple stub application that contains a WebView, much like the previous blog post.  This WebView pulls down an HTML page that’s actually on my blog here, and instantiates the native code and applies the javascript interfaces.  So you heard that right, I’m able to extend functionality to a live webpage and not just a static local page.

HTML Integration

Embedded in the example HTML file from PhoneGap is a JavaScript function called show_pic().  It simply wraps up another function inside the phonegap.js that is also referenced in the HTML file.  If this function succeeds then dump_pic() is called, if not then the function fail() is called.

The code under getPicture() is really quite complex and beyond the scope of this post, but you can see it here.  Essentially it calls a native function called takePicture() written in Java, and this launches a new Activity (like a Window) with the camera’s viewfinder surface.

function show_pic(){
   //Note: This could be simplified, but it's abstracted for safety
   navigator.camera.getPicture(dump_pic, fail, { quality: 50 });
 }

When the photo is taken we call dump_pic() and it will contain the Base64 encoded JPEG image data from the camera.  Then, all we need to do is return this to Flash in a callback and in this case “mySWF”, which is the name of the SWFObject embedded in the HTML file.

function dump_pic(data){
   document.getElementById("mySWF").photoBytes(data);
 }

In the running SWF file we have a SimpleButton that can call the show_pic() function, but first we need to add a callback which is analogous to an event listener.  In Javascript we call photoBytes(image_data), and once the ExternalInterface receives this event it will call onReceivedPhoto(image_data):

public function fl_MouseClickHandler(event:MouseEvent):void{

  ExternalInterface.addCallback("photoBytes", onReceivedPhoto);

  ExternalInterface.call("show_pic");
}

Here in the onReceivedPhoto function we decode the Base64 encoded image_data, which is just a very large string containing the image.  I’m using the Base64 classes provided here by Jean-Philippe Auclair, and merely getting the ByteArray.  I can then use Loader to take these bytes and produce BitmapData and do a little scaling in the imageLoaded handler before adding it to the stage within the holder_mc.

public function onReceivedPhoto(image_data):void {

  var bytes:ByteArray =  Base64.decode(image_data);

  imageLoader = new Loader();

  imageLoader.contentLoaderInfo.addEventListener(Event.COMPLETE,  imageLoaded);

  imageLoader.loadBytes(bytes);
}

private function imageLoaded(e:Event):void {

 var image:Bitmap = Bitmap(imageLoader.content);
 var tmpdata = new BitmapData(320, 260);

 var matrix = new Matrix();
 matrix.scale( 320 / image.width, 260 / image.height );

 tmpdata.draw( image.bitmapData, matrix, null, null, null, true ); // smoothing
 image.bitmapData = tmpdata;
 holder_mc.visible=true;
 holder_mc.addChild(image);
}

Here is FlashCam in action…

So that’s it, it’s really that simple to hook up native code to Flash Player 10.1.  I suspect that this will also be possible on Palm’s WebOS, Symbian and Windows Mobile in time.  I’m also loosely aware of a better bridge to JavaScript in Flash Player 10.1, so this might be a source of further investigation to improve performance for large data sets.

Now it’s your turn, below are the sources that I’ve used from PhoneGap with my own additions.  You can also find the Flash FLA and AS files below, note that you will need at least Flash Professional CS4 to open the FLA.

Downloads

• Download Eclipse, or use Flash Builder
• Download Android SDK
• Download PhoneGap for Android 2.1
• Download Flash Sources
• Flash Player 10.1 Beta 3

Notes:

• You cannot keep the SWF file within the stub package, they must be in the remote sandbox.
• Flash Player security does not allow for localWithTrusted access to local Javascript.
• Proportional scaling of images is a pain in the ass in Flash, I’d like to hear about any libraries for that.
• I found CameraForm useful for producing encoded image data for testing; impossible with HTML5