Thursday, July 10, 2014

Application routing, message busing and the irritating impossibility of programs as graphs

I had a hard time writing that title. What I want to write about is a simple thing, but somehow it is quite hard to express.

But to begin, I think that many programmers, like me, have a dream to some day do some or all of the following;

1) A multi-user d&d inspired game
2) A WYSIWYG web widget editor that 'just works' and create websites that can be generated in any of n languages or frameworks
3) Some kind of 'mini-SOA' framework where the program is broken down into parts which are wired up with some kind of bus system.

These things usually crash and crash and crash again, again and again. I keep trying, keep crashing and even though I realize what I want to do is still a laughably long way beyond my skill level, it's still something that I strive to achieve.

A couple of years age, thinking on one of these things, the application-routing, mini-SOA or, to call it by it's wiki-official name, dataflow programming framework (, I realized that it was practically impossible to create what I wanted.

It might seem obvious to anyone else, but what I realized then was that if you wire up your application like a kind of graph, how do you define connections between dynamically created objects?

Well, you can't, of course, or at least you have to break the circuit model, using some kind of API to programatically create connections, incidentally defeating the whole purpose of the framework.

So dataflow wiring would only work for programs made up of singletons, which was a bother and made me retreat from further experimenting. Until yesterday.

Again, this is probably super-obvious for anyone but me, but I just wanted to share my epiphany on why it's actually feasibly an indeed a very, very good idea to use dataflow abstractions for programs that create non-model objects dynamically (which most tend to do).

What I realized was that in an UI (for example) there can be many levels of multiple objects created in hierarchies, but for every level, all objects can be treated as singletons, thus simple to wire up with a dataflow template.

If one of these objects create a dynamic list of other objects, each of those objects will define a new 'level' or island, if you will, where all the relations can be wired up again using dataflow. So we have stable islands of instance relations, where one or more can create more islands, with implicit connections to them.

One of my favourite dataflow architectures has been JDA (JavaScript Dataflow Architecture) which recently has re-emerged as ( I think what I'll do now is have a drink, look over JDA/ again and see how it can be applied to Polymer, because Polymer.