I’m really excited about the recently announced integration between our Credit Card Terminal app and PingySoft’s RingItUp Point of Sale app.

I suspect that other apps will want to provide similar integration. Just last week, Erica Sadun was talking about what she called “Bridging Functions” on Ars. That’s what I’m talking about here.

The URL Handler for Twitteriffic was the first thing Craig Hockenberry posted about after the NDA went away. Such URL handlers are a good sign that the community wants to expose app functionality in a reusable way, but they only solve half the problem since the user is stranded in the new app. Who wants to tap the home screen, find the original app, and navigate back to the right place?

Speaking of Twitteriffic, have you noticed how many iPhone apps have their own ad-hoc web browser built-in instead of using Safari? It’s because calling into Safari leaves the user stranded. We need an experience like playing a video from Safari. The video launches like a separate app, but there’s a Done button to return to what I was doing.

The basic idea, to quote my abstract algebra prof (sorry, Dr. Sherman!), is “Go somewhere; do something; come back.” The more apps that provide this kind of integration, the richer the iPhone platform will be.

Thankfully, since Apple has provided an underlying URL-based method for applications to communicate with each other, we can use existing techniques from the web space to accomplish our aim. Two apps can pass control between each other fairly easily when both are registered to handle a URL scheme in their Info.plist.

So here’s how the Credit Card Terminal / RingItUp integration works (video):

I would be remiss if I failed to point out the security implications here. By registering to handle a URL scheme, an iPhone app becomes a de facto web app, subject to many of the nasty attacks that work on the web. Apps implementing this scheme must be careful to validate any parameters they get from the URL lest they be vulnerable to old friends like SQL injection. Another one to be careful of is unsolicited response attacks. The calling app should store a nonce value which it includes in the returnURL and reject any response with the incorrect nonce (similar to CSRF mitigation on the web).

Due to the security issues, as well as the sometimes tricky matter of properly encoding query string parameters, we’ve chosen to provide Objective-C classes for submitting the request and parsing the response when interacting with Credit Card Terminal. These are MIT Licensed, and we’d be very happy to see other app developers use them as templates for their own integration offerings.

What other apps would you like to see support 2-way integration? I’d love to see one of the twitter apps do it. That way I could expose the ability to send a quick tweet from my app without worrying about stranding the user and without writing against the twitter API directly. Which is a relief, because I certainly don’t want to be in the business of collecting people’s twitter passwords right now. :-)

Update: Of course, since iPhone apps are generally servicing one user, the easiest way to deal with the continuation problem is just to save what you need to NSUserDefaults.

With Photosleeve, one of our core ideas is to reduce the work and time required to share photos. One way we do this is by creating smaller sized photos and uploading them first, which makes the initial sharing process and email generation pretty quick.

Doing this is straightforward with our Windows desktop application. But we have always been interested in finding a cross-platform browser-based way of doing the same thing. Our hope was to increase user adoption two ways: by supporting the increasingly-popular Mac platform, and by eliminating the requirement that first time uploaders download and install software.

Some of the things we considered: ActiveX, Firefox Plugin, Java applet, Flash, Adobe AIR, and Silverlight. We wanted a solution that was cross-browser and cross-platform, worked in the browser rather than a separate install, had a reasonable first-time user experience, and didn’t require the user to click through any scary security warnings.

After eliminating those that didn’t meet our criteria, we were down to Flash and Silverlight. To do what we wanted, we needed to be able to have the user select multiple files, read the bytes locally, compute a SHA1 hash, load the bytes into an image and perform manipulations like rotation and scaling, re-encode the resulting images to JPEG, and upload them to our server. Unfortunately, Flash can’t read the bytes locally, and Silverlight can’t do image manipulations or re-encode to JPEG.

At this point we thought we were out of luck. For a while we chatted about this problem with others to see if they had insights, and we’d always jokingly conclude that you’d really need to build a hybrid Flash/Silverlight application to do what we wanted. There would always be jolly consensus that such an idea was too silly to pursue. We even came up with a silly name for the “new” RIA platform: FlashLight.

But as we thought about it more, we found it less and less silly. If we could think of Flash and Silverlight as Javascript libraries instead of monolithic app platforms, there was really no reason that we couldn’t use the functionality from both to achieve our goals. It probably wouldn’t be a happy developer-tool supported experience, but — hey — we like writing code.

So next we tried to figure out if we could use Javascript as the core and call into Flash and Silverlight as needed. It turns out Silverlight provides really great Javascript connectivity. Essentially, you just decorate your types with attributes to make them accessible from Javascript. Flash’s Javascript interaction looked painful at first, but then I found this great add-on from the Flex SDK called FABridge that made it easy.

The plan was to use Silverlight to read the JPEG files, then pass those bytes over to Flash for image processing. But in the middle is Javascript, and Javascript doesn’t deal with binary data very well. Keeping it simple, we decided to base64 encode the bytes and pass them through Javascript as strings. Both Silverlight and Flash have libraries to deal with base64-encoded data.

Once we had the pieces in place, we were able to crank out our web-based uploader in about a week by writing a little bit of C#, and moderate amounts of Javascript and ActionScript. You can see the results on Photosleeve. (You’ll have to sign up for an account and click on Add Photos in the upper right.)

Thinking of Flash and Silverlight as complementary Javascript libraries instead of competing app platforms allowed us to build an app that leveraged Silverlight’s client-side file access and Flash’s client-side image manipulation.

Have other exciting apps been built using this technique? Are there others waiting to be built?

Although there’s no visible Flash on Photosleeve, we use some of the Flash image processing capabilities when you upload photos through the site.

There are all sorts of different ways to include Flash, but after some research and poking around, we decided to go with swfobject.

All was swell at first, as we were using ExternalInterface to call between Flash and JavaScript. But that got pretty burdensome once we got past the proof-of-concept phase. Thankfully, we found the Adobe Flash AJAX Bridge (hereafter FABridge), which automatically handles exposing things, marshalling params, etc. It’s really great, actually.

So we had everything working great in IE, but Firefox was a no go. After debugging through things for a while (thanks, Firebug!) I realized that FABridge requires you to use the ‘embed tag within an object tag’ method (is that the one called Flash Satay?). Since I was using swfobject, no dice.

I found that the fix was just to comment out the bit of code in FABridge.js that looked for the Embed tag for certain user agents (see diff at the bottom of this post).

Incidentally, I also discovered that if you have more than one object tag on the page, you must include the bridgeName flashvar for FABridge to figure things out. The syntax for swfobject looks like this:

// swfobject_fabridge_example.js (Sample Code)
swfobject.embedSWF(
    mySwfFile,
    mySwfAlternateContent,
    myWidth,
    myHeight,
    myMinFlashVersion,
    myExpressInstallUri,
    { bridgeName: 'myBridgeName' },
    { allowScriptAccess: 'always' }
);
// and later ...
var flash = FABridge.myBridgeName.root();

You have to get rid of lines 113 and 148-169, below is a diff. FABridge.js has the MIT License blazened at the top, so have no fear when deleting huge portions of it.

--- a/fabridge/src/bridge/FABridge.js 2008-02-11 06:44:57.000000000 -0800
+++ b/fabridge/src/bridge/FABridge.js 2008-05-13 18:17:25.000000000 -0700
@@ -110,7 +110,7 @@
 {
     var searchStr = "bridgeName="+ bridgeName;

-    if (/Explorer/.test(navigator.appName) || /Konqueror|Safari|KHTML/.test(navigator.appVersion))
     {
         var flashInstances = document.getElementsByTagName("object");
         if (flashInstances.length == 1)
@@ -145,28 +145,28 @@
             }
         }
     }
-    else
-    {
-        var flashInstances = document.getElementsByTagName("embed");
-        if (flashInstances.length == 1)
-        {
-            FABridge.attachBridge(flashInstances[0], bridgeName);
-        }
-        else
-        {
-            for(var i = 0; i < flashInstances.length; i++)
-            {
-                var inst = flashInstances[i];
-                var flashVars = inst.attributes.getNamedItem("flashVars").nodeValue;
-                if (flashVars.indexOf(searchStr) >= 0)
-                {
-                    FABridge.attachBridge(inst, bridgeName);
-                }
-
-            }
-        }
-    }
-    return true;
 }

 // used to track multiple bridge instances, since callbacks from AS are global across the page.