Software Complexity in Practice

I had a minor epiphany last week, but have been having too much to do until now to blog about it. It revolved around the now quite common (for those who have made the effort to learn a second (or third) language properly) question:

Why does x takes longer to implement in Java than in my-scripting-language-of-choice?

And what goes for implementation also goes for maintaince, et.c.

Having had the opportunity (..) for the last two months (OK, I had a four week vacation in there somwhere :) to work with a system that was based on using Maven for everything server-side, it struck me how dangerous it is to use compiling languages at all. Listen to this;

The services ( that I am working with, in Java) are generic to a fault, and use Spring applicationContext.xml files to hold specific information such as hostnames, DAOs and a host of other stuff. Now, I have nothing in principle against using config files for properties, but when the property files start to contain logic and 'mixins' for the rest of the program, things start to get fishy.

By 'fishy' I mean hard to maintain.

So we have the following;

1) Java code which is quite generic _and_ depentent on Spring to work.

2) an applicationContext.xml file which partly describe which classes to compile in (to the Java program)

3) A maven pom.xml file which perform magic on the applciationContext.xml file (and other things) to force things like development and test environments for the service in question.

The basic premis is to be able to move the code and use it somwhere else, and just change config files. The basic problem is that the config files are (as always) not documented, even though they are now completely logically intermeshed with the code they are acting upon.

This means that if you are making a very minor change to a part of a service, the risk is high that you need to understand how the configurations files interact with your new code. This is not very modular.

Also, I suspect that one of the reasons one want to do this is to be able to use the oh-so-smart dependency injection to be able to hot-wire in new behavior in the Java classes, even after (gasp) they are compiled. This behavior being defined in various configuration files, naturally.

Duck typed languages doesn't need dependency injection (magically replacing or inserting chunks of actual logic on the fly) because they alrady have that functionality.

This means that that the Java community (and others..) have taken 5-7 years to work around a big problem and finally (some years ago) had a great system for dependency injection. The reason for this workaround is that Java is a compiled language. The compiled class-files were _supposed_ to be static and non-dynamic. It's the whole point. Errors and general no-nos were supposed to be caught at compile time. It's that kind of language.

So, many years later one finally have a solution to all this, and it's not even needed if you're using a Python, Ruby or JavaScript (et.c.) since it's an inherent part of the language. If you want to add a new functon to an object, at any time, on a lark, you just write;

foo.newfunc = function(a,b){ .....}

Now, I could have put the logic for that in a configuration file, but then I would have to have some kind of framework which read that file, which would make my work as a programmer more complex. And it's here that my little epiphany comes back into the story, namely;

1) For every extra configuration-file that link out from the file that contain the code, complexity increases dramatically.

What I mean by 'link out from' I mean that the code might be dependent on configuration file x, which in turn might be dependent on configuration file y (See maven -> Spring -> Code example above). Also 'dramatically' is not a very specific number, but since there are next to no research in this area it's all I can do to argue that all logic you can keep inside your code files lead to less complexity, even though it might lead to less flexibility.

OK, hold the horses, you might say now, all this Java-bashing is making me dizzy (thanks, that was my intention :); isn't the whole point with Java (et.c.) that the additional security of exception-checking, static typing and public/private/protected access guard you from making a lot of horrible mistakes? If we would using these flimsy little scripting languages, we would all be lost!

Well, here's another rub of mine: Static typing is supposed to help me make less mistakes. I have coded in JavaScript almost exclusively for the last three years (and almost exclusively in Java four years before that), and in all the JavaScript code I have written, not once have I sat down and cried silently, wishing for someone to hand me enforced static typing for JavaScript. It has not bogged me down, it has not made me confused, in reality it has made me more productive.

How is that? I have a creeping feeling that these 'safety features' are more good in formal proving of things than in actual engineering of stuff.

For example, I was able to quickly write a (qute shabby, but working) REST service implementation in only a page or two of (server-side) JavaScript code. The main reason for that is IMO that I didn't need to jump through hoops either creating complex class-hierarchies for the data - or- casting myself to high heaven and back - or - generating miles of try/catch code which I knew was not really needed. Sometimes the code bombed, I checked the logs, added a try/catch, and later removed it when I had fixed the problem.

So, as Steve Yegge correctly has pointed out at great length, javaScript (etc) lead to dramatically reduced Lines of Code in a program. Mostly due to the non-existence of enforced 'security'. And here is a more specific thing to chew on;

1) All security features of a programming language does not confer the same amount of benefits.

This ties back to my unspecific 'dramatical' level of complexity above. There simply does not exist any research in this area, which is sorely needed.

My point here is that for a given security feature of a langauge, you receive a certain level of benefits, but it also costs you an amount of flexibility. These are all realtive values for now. But from my personal experience, the following has become more and more glaringly obvious;

1) The lack of a security feature from a language gives more benefits from increased flexibility than the secutiry feature gave when present.

So if I use a language which already have duck-typing, I don't need dependency injection, and I don't need to manage that injection. Moreover, if I need to manage the duck-typing in any way, I might as well do it in the scripting language itself, since there is no compilation step, thus reducing complexity from linked configuration files. I can keep all logic in the code.

All comments welcome :)

Cheers,
PS