Friday, October 16, 2009

Yak Coiffures

I am a very emotional and 'colorful' person and tend to rant quite a lot about things that I deem important. Lots of those happens to fall inside an body of supposed knowledge called 'software engineering'.

I am often talking about the importance to find ways to cut down the time being spent when developing a program that is not actual programming. Currently, I'm using 'find . -exec grep ..' in various parts of various repositories trying to understand how server-side templating is causing subtle bugs in a pedestrian, yet important web service, for a customer.

But never mind that. What I want to write about today is a simple rule to follow if you want to weigh your language and/or platform of choice in terms of complexity.

Assertion: Every step that produces an intermediate file creates complexity.
Example: A compiler takes source code files and generates binary files.
Why: Each step that converts files between formats must be managed and maintained.

This is pretty hard to object to. Really. If you use Java, you need to manage compilation by either an ant file or maven, or perhaps a custom shell-script. These need to be maintained. They will not be maintained. This will lead to grief.

If you need to package your generated files before deployment, the script for that needs to be maintained. See above.

These operations are hard to duck, but that doesn't mean you shouldn't try. Every time you get one extra dot between yourself and the final, running application, that dot needs to be maintained. And that means documenting it, integrating it into other subsystems, referring to it, checking it in, adding it to a repository, what have you. Every point.

Also, frequently, they will become stop-blocks that makes it impossible to do things, until they guy that does packaging comes in, et.c.

One of the simpler ways to minimize yak-shaving of this kind is to adopt a scripting-language as your server-side language. Whether it is Groovy, SSJS, Ruby, Python, PHP or anything else is beside the point, because the main point is this;

  • A scripting language gets compiled inside the VM.
  • A scripting language has at least two steps less complexity than a compiled one
So whatever you do, for future generations of programmers: Never, ever again use a compiled language, for any project, for any reason.

Just Say No! :)



Zoom said...

The amount of heat I generate to actually manage my .class files in java will probably not bring about the heat death of the universe. Libraries and deployment-files on the other hand takes some work. But I guess that your scripting languages comes bundled with Dojo and by utilizing the power of ducks automagically deploy on test and production servers ;-)
On a more serious note, great GTUG meeting last, to bad I am out of town for the next ones :-(

Peter Svensson said...

Of course not :) But just because not all complexity can be avoided by choosing a scripting language, doesn't mean you shouldn't take the opportunity. It's like not taking $1000 offered because you're holding out for one million - being broke, or something.

We'll have some extra beers and think of you. Hope to have you with us come December.

Magnus Reftel said...

It can also be viewed the other way around: what compilation is about is to take a bunch of resources and combine it into a single entity. That single file is easier to handle than a whole bunch of files plus the handfull of libs they depend on.

If you solve your dependencies at compile time, deployment can get simplified as a consequence. When developer/deployer relations don't work, it may be worth it.

Personally, I would much rather maintain a makefile than a deployment guide, but then again, I guess I'm not like most people in that respect ;-)

Peter Svensson said...

@Magnus Reftel: Well, naturally I disagree :) My point is that by using a compiled language, you gain one more unnecessary file, which must be cuddled, copied, aggregated and maintained. All that you want is for your logic to run. By using a scripting language, you have jsut text that is only compiled when hitting the VM/Browser.

Web Design Chennai said...

Hey really I read love your blog… I am really thanks to you to share this informations in internet to my kind of persons… really I loved loved your blog…. I will come back to your blog every updates…

Magnus Reftel said...

Yes, you are correct in that there are less things to take care of in development. My point is that there is a tradeoff in complexity between the development side and the deployment side. In some cases it can be worth making development more complex in order simplify deployment and operations.

That said, complexity in one place that does correspond to comparable gains in simplicity in another should of course be removed.

Also, I'm just blindly assuming that deployment is more difficult for a bunch of scripts than for a war file. I don't actually have any experience in delivering scripts to scary Operations Departments. Do you have any stories on that that you might share?

Peter Svensson said...

@Magnus Reftel:

Well, The only step that I need to do with my current setup is the Dojo build script, which uses Rhino and produces one or more compressed .js files which are then copied by other scripts as any other file that is to be deployed.

So, generally when developing, I save text files and reload the browser and the changes get through, or I change text files being scripts on the (local) server and make a new call, and get something new back.

In essence I think it's difficult to get around the deploy step for any mdeium-to-large system, but in many cases it is sufficient just copy files instead of copying, compiling, packaging, copying again, unpacking, copying again and reloading - much of the processes which has to be maintained with separate configuration files.

Being the devils advocate here it should be possible to use Java, for example in a more streamlined way, just using tools which copy class-files instead of maintaining a large build process.

But to get back to your question; Normally I work on small to medium size projects, where I'm either responsible for the front-end/client or for both client and server, and I seldom have to juggle more than two dozen source files.

I have a feeling that any new project could be split up into projects of comparable size and be isolated by web-based APIs, where each smaller project just save textfiles and reload and/or send a new request depending on where you are. When deploying the front-end one would have just one build script for minifying and compressing js/css/et.c. but that's about it.


Magnus Reftel said...

*gets sudden urge to write mod_make*

Peter Svensson said...

LOL :) That's my favorite example actually.

If C/S apps of the 80s and 90s would have been written the same way web apps (still) are today, they would compile source code and generate a new compiled client for each action.