Saturday, August 29, 2009

The Joys of Android

Positive feedback:

Instead of continuing to bash the flotsam of obsolete programming languages like I did last week, today I'm going to describe a really nice experience I just had in the programming department instead; Developing an Andoid application.

Enter the Client:

The specification that my client had was clear; He wanted a demo/PoC of a smartphone application (iPhone or Android) that among other things first showed a Google map of an area with some markers on it, and when one of the markers was clicked, it would switch to augmented reality mode (AR) to display the markers in an overlay on top of what the camera in the phone dislay, so that people can orient towards points of interests in 3D using local landmarks.

And it was supposed to be done in under ten days. It sounded crazy hard to do, and at first I asked a couple of iPhone developers I knew if they could take it on, but they had their hands full, so I did some more research to see if I could do it myself.

Layar to the rescue:

It turned out that a very interesting Dutch company called Layar, which develops an Augmented Reality service by the same name. That's right, it's a service. That means that if you can get your user to download their (free) app from the Android appstore (iPhone support is coming soon, as a high priority), you can create your own headless REST service which pop up among available AR layers to choose from.

Here's a list of the currently created layers. If you get a quick idea of what you could use an AR layer for, it's probably already been implemented :) so check there first.

That means that 'all' I had to do was to create an android app that showed a map, and when clicked started the Layar app and pointed it to a special layer that I could create once I got my deveoper keys from Layar.

Luckily, I had just formed the Stockholm GTUG( Google Technology User Group), which among other things was eligible to receive review copies of books from O'reilly, Pragmatic Press and others, so I had several good Android books to browse through. Then I went out and bought myself a HTC Magic with extra toppings and flat rate internet.

SDK impression:

I used (naturally) Eclipse 3.4 with the spanking new 1.5v3 Android SDK which contained Google API support for maps and other things. It was very easy to get started, despite using Java. The APIs for various funcitons were mostly sane, meaning that they could be used without creating four extra levels of classes and the methods you really wanted to have for convenience sake were there, instead of missing so you had to write them yourself (classic Java).

My first try in developing the app used an internal webkit inside an Android app. I was really happy to find examples on how to do this in one of the books, since that meant that I could focus on effective development (in JavaScript) for a brwoser pretending to be a 'desktop' application.

I managed very quickly to embed a Google map in the 'internal' web page, and writing a shim that let me call Java methods from JS (and vice versa) was very straightforward.

The downside of this method was that (at least for the HTC Hero) it took over 15 seconds for the app to initialize. If tha map hadn't been the first screen, I could probably have used a background thread for webkit initialization, but I ended up ditching the browser method - despite its clear advantages.

Instead I found a View (A view is a kind of layout manager, which organizes other content for the Android app) called the MapView. I recommend following the 'official' example (which has a few typos in it), here. This view demanded a special application key from google code (You can get one here), which turns out to be a critical piece to do right. The problem is not obtaining the key, but to use the right public key from you SDK when applying for one.

Killer Keys:

There's a default keystore created for you when you install the Android SDK which has no password, and using the signature of that results in a developer key which only works inside a debug version of an app, so if you generated an .apk file and sign it using another keystore, the maps API key in your app will not work.

This is really simple and sound straightforward when I write it here, but when you're developing, it is very simple to forget to change codes and/or to forget which key signature to use when applying for one. This in itself would be very simple to spot if you get an error, which you don't.

What happens is that your markers show up at the correct place in the mapview, but the map is clean, consisting of just a thin grid of grey lines. The first times you see this, you immediately think that you have made a coding error, since the mapview do actually force you to create extra classes, implement extra methods and do the work that you really needed support for yourself - in a true, classical Java style.

So if you ever get those gray lines - I'd give you a 90% chance of having used the wrong keys somwhere. Just so you know, OK?

Once I got the mapview working, I needed to trap the screenpresses, so I could launch the Layar app. This is where Android started to shine. For every application in Android you can define different 'Intents' which are for all effective purposes 'Events'. So in an (ugh) external XML file, you define which intent string your app listens for and which class and method handles those intents.

Effective use of Events:

This means that each an devery app has an intent that starts it. This also means that it's possible to define intents that the app traps and handles before it has even been started (and which can start it on demand). I used this as a finishing touch, letting the app listen to incoming SMS, searching for a keyword. When it found the keywork it sent a message to the intent that started the main application. Worked like a charm :)

I also used this method when starting the Layar app. By browing through the log that I got from a shell that displayed ./adb logcat from my physical device (The HTC Hero) over USB, I saw the classname for the Layar App (com.sprx.layar), which I then used to create a new activity object, like this;

Intent intent = new Intent();
intent.setClassName("com.sprx.layar", "com.sprx.layar.Main");

And that's how an app start's other apps in Android. Pretty simple, isn't it?
Layar works like this;

1) You apply for a developer key here.
2) You login to Layar and create and name you first layer.
3) In the new and named layer, you point out a lot of things, but most important is the url that this new layer will get its POIs (Points Of Interests, for you non-mapping geeks ) from.
4) You implement the REST based service at the required url.
5) You download a special developer version of Layar, where you can enter you developer key and which will show all your dev layers in a specific tab inside the app.
6) Load your layer. Scream, curse, despair, check logs, read up on coordinates. Check in Google Earth. repeat.
7) Success!

REST assured:

Item number (4) here sounds ominous, but it really isn't. I naturally implemented the service in Googel App Engine vanilla python. It took all of two hours to register, create, upload and get something visible, and part of that time was dinner. There was two reasons that this went fast; a) I was only doing a demo, so all POIs I returned was hardwired into the source code, b) you really only need to implemenet GET, since Layar only ever (for now :-P ) reads info from your backend.

I had a lot of small troubles converting coordinates and making them show correctly in my layer, and almsot all of them was due to rounding errors or plain stupidity on my part. When I started checking all coordinates I sent back in Google Earth (where you can just enter degree coordinates into the search field) and I found myself north of Scotland or in Dubai, I had an easier time finding my bugs.

I managed to deliver the PoC app on record time, fixing design erros quickly due to the web-based nature of both App Engine and the Layaer adminstrative interface. I'm actually a bit on an Augmented Reality tear at the moment and will present and demo the app on out first GTUG meeting next Thursday. Now for some more coding :)


Sunday, August 16, 2009

Software Complexity in Practice

I had a minor epiphany last week, but have been having too much to do until now to blog about it. It revolved around the now quite common (for those who have made the effort to learn a second (or third) language properly) question:

Why does x takes longer to implement in Java than in my-scripting-language-of-choice?

And what goes for implementation also goes for maintaince, et.c.

Having had the opportunity (..) for the last two months (OK, I had a four week vacation in there somwhere :) to work with a system that was based on using Maven for everything server-side, it struck me how dangerous it is to use compiling languages at all. Listen to this;

The services ( that I am working with, in Java) are generic to a fault, and use Spring applicationContext.xml files to hold specific information such as hostnames, DAOs and a host of other stuff. Now, I have nothing in principle against using config files for properties, but when the property files start to contain logic and 'mixins' for the rest of the program, things start to get fishy.

By 'fishy' I mean hard to maintain.

So we have the following;

1) Java code which is quite generic _and_ depentent on Spring to work.
2) an applicationContext.xml file which partly describe which classes to compile in (to the Java program)
3) A maven pom.xml file which perform magic on the applciationContext.xml file (and other things) to force things like development and test environments for the service in question.

The basic premis is to be able to move the code and use it somwhere else, and just change config files. The basic problem is that the config files are (as always) not documented, even though they are now completely logically intermeshed with the code they are acting upon.

This means that if you are making a very minor change to a part of a service, the risk is high that you need to understand how the configurations files interact with your new code. This is not very modular.

Also, I suspect that one of the reasons one want to do this is to be able to use the oh-so-smart dependency injection to be able to hot-wire in new behavior in the Java classes, even after (gasp) they are compiled. This behavior being defined in various configuration files, naturally.

Duck typed languages doesn't need dependency injection (magically replacing or inserting chunks of actual logic on the fly) because they alrady have that functionality.

This means that that the Java community (and others..) have taken 5-7 years to work around a big problem and finally (some years ago) had a great system for dependency injection. The reason for this workaround is that Java is a compiled language. The compiled class-files were _supposed_ to be static and non-dynamic. It's the whole point. Errors and general no-nos were supposed to be caught at compile time. It's that kind of language.

So, many years later one finally have a solution to all this, and it's not even needed if you're using a Python, Ruby or JavaScript (et.c.) since it's an inherent part of the language. If you want to add a new functon to an object, at any time, on a lark, you just write;

foo.newfunc = function(a,b){ .....}

Now, I could have put the logic for that in a configuration file, but then I would have to have some kind of framework which read that file, which would make my work as a programmer more complex. And it's here that my little epiphany comes back into the story, namely;

1) For every extra configuration-file that link out from the file that contain the code, complexity increases dramatically.

What I mean by 'link out from' I mean that the code might be dependent on configuration file x, which in turn might be dependent on configuration file y (See maven -> Spring -> Code example above). Also 'dramatically' is not a very specific number, but since there are next to no research in this area it's all I can do to argue that all logic you can keep inside your code files lead to less complexity, even though it might lead to less flexibility.

OK, hold the horses, you might say now, all this Java-bashing is making me dizzy (thanks, that was my intention :); isn't the whole point with Java (et.c.) that the additional security of exception-checking, static typing and public/private/protected access guard you from making a lot of horrible mistakes? If we would using these flimsy little scripting languages, we would all be lost!

Well, here's another rub of mine: Static typing is supposed to help me make less mistakes. I have coded in JavaScript almost exclusively for the last three years (and almost exclusively in Java four years before that), and in all the JavaScript code I have written, not once have I sat down and cried silently, wishing for someone to hand me enforced static typing for JavaScript. It has not bogged me down, it has not made me confused, in reality it has made me more productive.

How is that? I have a creeping feeling that these 'safety features' are more good in formal proving of things than in actual engineering of stuff.

For example, I was able to quickly write a (qute shabby, but working) REST service implementation in only a page or two of (server-side) JavaScript code. The main reason for that is IMO that I didn't need to jump through hoops either creating complex class-hierarchies for the data - or- casting myself to high heaven and back - or - generating miles of try/catch code which I knew was not really needed. Sometimes the code bombed, I checked the logs, added a try/catch, and later removed it when I had fixed the problem.

So, as Steve Yegge correctly has pointed out at great length, javaScript (etc) lead to dramatically reduced Lines of Code in a program. Mostly due to the non-existence of enforced 'security'. And here is a more specific thing to chew on;

1) All security features of a programming language does not confer the same amount of benefits.

This ties back to my unspecific 'dramatical' level of complexity above. There simply does not exist any research in this area, which is sorely needed.

My point here is that for a given security feature of a langauge, you receive a certain level of benefits, but it also costs you an amount of flexibility. These are all realtive values for now. But from my personal experience, the following has become more and more glaringly obvious;

1) The lack of a security feature from a language gives more benefits from increased flexibility than the secutiry feature gave when present.

So if I use a language which already have duck-typing, I don't need dependency injection, and I don't need to manage that injection. Moreover, if I need to manage the duck-typing in any way, I might as well do it in the scripting language itself, since there is no compilation step, thus reducing complexity from linked configuration files. I can keep all logic in the code.

All comments welcome :)


Thursday, August 6, 2009

Google Technology User Group - Stockholm, Sweden

OK, so summer is over. The spring became a sprint for me with customer and voluntary projects crowding me from all sides, a little conference to arrange and a new major customer to get accustomed to.

In the middle of all this, I still tried to find some worthwhile monthly communal activity, maybe not so far-out as functional alcoholics, but still good fun.

So while I was gearing up this idea, trying to flesh it out, suddenly Google, of all people, sprung the excellent idea of starting Google Technology User Groups, GTUGs for short. I grabbed the Stockholm spot within minutes, naturally :)

If you're interested in GTUGs but happen to live far from Stockholm, Sweden, I've heard that there are some oher cities around the world, where GTUGs have ben known to form, go find one for yourself, or better yet, start your own.

Starting a GTUG doesn't mean that you have to becoma part of Google, or even know anyone who works there, it's basically a worldwide coders club, where we can share each others presentations, get some T-shirts or other schwag, if we're lucky, and teach each other about Google related APIs and other technologies.

Our first meeting will be the 3/9, graciously hosted at Ottoboni at Skanstull in Stockholm. The homepage for the GTUG is here, and we currently have speaker for OpenSocial and getting started on Android development.

If you are interested in coming, please comment on this post, so we know how many people to arrange for. Thx!!
[Update: There's now also a LinkedIn group for those of you who use it:]

I'll see you there :)