Wednesday, January 2, 2008

The End of Web Frameworks

Coming from a fairly serve-centric background (CA coding, wiring together stuff on the server-side, etc.) I abhorred doing any "web-work" for a long time. It felt cheap and not particularly interesting. Not that I checked it out, though.

I knew it was cheap, because I didn't work with it. If it would have been worth a second glance, of course I would have done it years ago, right? Since i worked in the field I felt I had a fairly bulletproof opinion on just about anything connected to the server-side, even if I had no direct experience working with it.

It felt safe and reasonable to shoot down stuff based on general assumptions and theoretical arguments, rather than giving things a decent spin with a 2-5KLoC actual project, so that I would have an actual idea of what I was commenting on.

Sadly, it seems I'm not alone in this endeavor. I recently did an informal poll at a LinkedIn forum, asking how many people had ample experience with both PHP and Java (say, at least a major project in each - major as you will) - and I came up with a lot of comments, but no one actually having had the dual experience I asked for. There were the usual comments that reduced one or the other side to smithereens based upon no real evidence at all, of course.

I think the reason is that there are very few programmers that do cross-train. We have corporate agendas, certifications, gazillions of reasons to not rock the boat (and get back home in time to dinner - a whole bunch of things that make you code in just one language.

The problem is that the world of programming is in constant change. New ideas pop up all the time, and looking in the rear view mirror, not all languages or frameworks were completely spot on in their 1.0 revisions.

So it is fairly safe to assume that the state-of-the-art way of doing things four years ago (for instance) might not be the most cost-effective way of spending time coding.

The last three years I have for various reasons done a number of web-related projects (for my day job), ranging from handmade Tomcat-based apps, distributed over a couple of machines, to fairly complex sales management systems done in Tapestry, to pure Javascript Ajax-works.

Then I've done a number of small to medium sized (~5KLoC) projects in my spare time in just JavaScript.

To try to round off this post, I've learned the following things;

1) It is _way_ easier to code in JavaScript, than in Java - doing precisely the same things (when you can).
2) The more logic I put on the client side (The browser again, natch), the easier my server-side components gets to write and maintain.
3) When writing JavaScript, I usually find myself doing it in just Vi, Kate or Wordpad, depending on which machine I'm using. There is no need for a big, fay, slow-starting IDE.
4) The whole point of web frameworks is to generate the client out of the server, making the server very complex to wrote and maintain, despite or because of that the whole point of the web frameworks are to reduce that very complexity.

Also, when I try to discuss my findings with people, I get invariably into long discussion where otherwise very smart people try to argue that I am wrong based on theoretical arguments - not having had the experience themselves.

At any rate, what I have found is that I am better off without any kind of server-side web framework at all - writing my clients old-school style, in JavaScript, making the server a 'thin server' which only doles out pure data.

And what does that mean? It means that the status quo has to go. No Tapestry, no JSF, no Wicket, Struts, Ruby on Rails or Symfony. Nothing at all. You can see why this is so very popular to talk about :)

In the end it is all about money. How much does it cost your company to build, a system and how much does it cost to maintain.

25 comments:

Eelco Hillenius said...

Unfortunately, people always want to come up with the one and only true conclusion. But the fact is that different kinds of projects require different approaches. Sometimes frameworks are in your way, but other times you would end up writing one yourself anyway if you would start without one. Been there, done that.

Peter Svensson said...

I agree that different kinds of projects require different approaches to problem-solving.

However, my point was more specific; When putting client logic on the client, the resulting system gets simpler to program and maintain.

Also, I use a lot of frameworks, both on the client and server - I jsut don't use _Web_ frameworks.

Jesse Kuhnert said...

Yep,

I've built a few systems this way. At different points you eventually want to pull your hair out when dealing with all the cross browser crap, but it does sometimes seem like the resulting systems are easier to understand / test / etc..

Ian McGowan said...

A hybrid approach seems reasonable - use a client-side "framework" (one of the ajax-y ones or flex) to help with the grunt work of browser independence, talking to a services layer (ws-* or restian to taste).

Peter Svensson said...

@jesse: It's good to hear that someone have had similar experiences. But I must say that using (for instance) Dojo have insulated me pretty well from browser issues on the client side.

Also; How would a server-side web framework help you solve those issues? You need a client-side framework anyway, whether or not you wade through XML config files or not.

@ian:

Yes! Actually this is not a hybrid approach at all IMO :) All I said was to not use a server-side web framework. One very good approach indeed is to have WS consumers directly in js in the browsers (like smartclient.com provides (LGPL) (No, we're not affiliated)) or plain simple REST.

Actually, most of my implementations use some kind of RESTish approach. Most times I only work with either one or two-dimensial data, so I just wrap the stuff in simple delimeters, and in text format, and throw it to the client, where it's split() for measure. I don't even bother to convert to JSON.

Another tack on this issue is dojos datastores (in javascript, on the client), which separates presentation (Grid, Tree, et.c.) from how data is stored (csv, xml, html table) or where it is stored (on your server, on flickr, et.c.) Which makes it really easy to snap things together.

Andrew said...

While I find this idea attractive, it falls over when you want your web app to degrade gracefully in the absence of client side javascript. The most essential function of server side frameworks is to ease development on a platform that can have a very thin client side indeed.

Peter Svensson said...

@andrew: I have not in practice have had to develop a degradable app either for customers or privately ever, though I know that it has come up as a "nice to have" in some projects.

It's 2008 now, and IMHO the projects where you cannot use any JavaScript/Ajax technology for a web app are very few and far in between indeed. But for those few, I agree that another - legacy - solution must of course be used.

Andrew said...

@peter: I envy you that choice, however I work for a university and as a recipient of public funding, all our work has to be accessible. Blind users with screen readers would find AJAX only applications difficult to use - updating DOM elements is a random-access process but screen readers process content sequentially, and may not have hooks to detect changes to DOM elements at all.

Online accessibility has similar legal status in Australia as things like wheelchair access to buildings, so in theory this applies to all commercial sites hosted here as well.

Then there are mobile devices, most of which have no JS at all. Graceful degradation is a nice goal in any case, but its essential for all the apps I publish outside my team.

But I know within my team I've been able to deliver some much needed features very quickly using greasemonkey and jquery while being very lightweight on the server side, which mainly emits JSON.

Timo said...

It's true, in the last 5 years or so, javascript and DOM have become much more reliable across browsers.

So it's possible to use a classic client/server architecture, instead of abusing the browser as a primitive display port.

If that is your point, I absolutely agree. I also think it's a good development.


I disagree with that meaning the end of web frame works, however. You can't build a web shop that isn't indexed by search engines because the robot doesn't know javascript.

Where I live, you can't build a government site without decent accessibility. That usually means fallbacks for all JS.


Nice read, though.

Eelco Hillenius said...

The Web Application that I am working on functions as a container for functional components. For instance, we have discussion and survey components. Such functional components contribute their own user interfaces; panels for end-users and administrators and they can participate in setup and tear down wizards. The ability to author fine grained reusable UI components is key to our system, and if we wouldn't be using a server side UI framework, we'd have to build one that facilitates this.

Another example where our server side framework helps us is with authorization. We have components that are conditionally rendered for instance, depending on whether the user is authorized to see them. Again, we'd have to build something for this if we weren't using the framework we're using.

Again, even for _Web_ frameworks there is no one and only true conclusion. Sometimes they are in your way - I whole heartedly agree with that and think it is excellent to just say 'no' if you don't see an immediate benefit - but with saying nay upfront you needlessly limit yourself.

Peter Svensson said...

@andrew, @timo: Thank you for concrete examples. I understand what you say. Without thinking about it my examples are geared towards consumer-centric sites, like for instance gmail. There will always be places where you are not allowed to do this. My guess, though, is that if you're lucky enough to be able to work on one of the 90% or more web sites which allow js, you'll be able to make your work simpler.

@eelco: It sounds like you're using Tapestry, are you? I kind of recognize that component feeling. I like Tapestry, by the way :)

You must of course place all security issues firmly at the server. But instead of laboriously recreating the whole page at each click of the user according to their access level, you could still restrict access to information, but place the rendering of the information on the client.

In my experience it takes less time to create and maintain such a solution.

Ont he other hand, if you're main work is done, and you have a very stable system (or stable components) I agree that creating new variations with the same stable compnoents will not cost you much (if we're talking of Tapestry-level usability)

Eelco Hillenius said...

Nope I'm using Wicket. I'm a team member of Wicket in fact, which probably explains why I'm being so darn defensive :-)

Authorization is implemented at the level of components btw. So we define auth rules on particular components, and that will be enforced regardless of whether full pages are rendered, or just parts of it (when using Ajax).

Ignacio Coloma said...

Just wanted to note that GMail has also a separate accessible interface and another one geared towards mobile users.

Peter Svensson said...

@eelco: :) I have never checked out wicket. It was on my to-do list, but then .. well, you've read it, I guess.

What I've read of wicket sounds like it is Tap++ of sorts. But as I said, I have no first hand experience.

Agreed (again) that it is good to have components that can be regraded, if needed.

I would be more interested in if you have comments to or arguments against my main point; That it takes more resources (time, money, non-typed ducks) to make those components as client pressed through the server on occasion, rather than as cleanly separated client-side components.

Like in Dojo - check out their widget system (really!) which communicate just the right stuff from the server, which makes the server-side all the much simpler - leaving you with just the business logic (incidentally with no wicket, though)

@ignacio: The point here is not that gmail has pop/imap interfaces, but that most of its fairly large user base seems to do without the need to degrade from js. Also; my point is _not_ to say that everyone who can't use js will have no service, but that the main service will be much cheaper to develop the way I'm suggesting.

Andrew said...

Something that just came to mind, a nice tool to expedite the client side development talked about here is Jemplate:
http://jemplate.net/

Its a port of Perl's Template Toolkit, one of the more flexible template engines around, to javascript. With this the server just has to do a bit of security and database work, maybe choose a template and the client does the rest.

Eelco Hillenius said...

Ok, my take on your specific points:

1) It is _way_ easier to code in JavaScript, than in Java - doing precisely the same things (when you can).

I don't think JavaScript is easier for all purposes - when I code in JavaScript I spent more time searching for documentation and fixing typos etc than I do with Java - but since it runs in the browser, yeah, it is not a bad idea per se to prefer JavaScript over Java.

However, that project that I am working on requires that large parts of the app functions without JavaScript. At least that was the assumption when we started working on it a while ago.

2) The more logic I put on the client side (The browser again, natch), the easier my server-side components gets to write and maintain.

Sure, as you just move complexity from one side to the other. Some things might actually be harder though, so as implementing fine grained authorization, localization etc. Also, putting all the logic on the client is not more efficient per se. You won't take advantage of locality (caching, shared data on pages). Though this is something you can optimize yourself.

3) When writing JavaScript, I usually find myself doing it in just Vi, Kate or Wordpad, depending on which machine I'm using. There is no need for a big, fay, slow-starting IDE.

But these IDEs (are they really that slow for you?) then also won't help you catch typos, explore APIs and do code completion for you.

4) The whole point of web frameworks is to generate the client out of the server, making the server very complex to wrote and maintain, despite or because of that the whole point of the web frameworks are to reduce that very complexity.

If the framework is done well, the complexity is in the framework, not your client code.

Ok, reading my replies it might look like I entirely disagree with you and want to whole world to always use a web framework. I don't. I actually think that if you are comfortable with JavaScript and SOA, and you don't need a framework for the things I outlined earlier, you might be better off keeping things as simple as possible. You'll trade some annoyances with others (we're building software after all), but with discipline, the end product is likely to be easier to understand and test etc (like Jesse said).

Peter Svensson said...

@eelco: Thanks for the detailed reply! I'll try to summarize my comments;

1. No, js is no magic bullet, of course. I'm describing my experience with it compared to development in Java. At the moment, I'm getting fairly interested in Scala, actually.

As a matter of fact a lot of Oops-catching for me is just having a decent syntax highlighting for js, which both Kate and vi has. Of course, I would really like object/function outlines and to some other stuff. Also, if needed, Aptana has an Eclipse-based js editor with extremely good support for Dojo, JQuery, et.c. It's just that it takes a while to start, and when doing js work (in constrats to java work) the extra bells and whistles aren't really needed/worht it. Sadly enough :) I'd really like to work in Aptana, but I'd like something more flyweight.

2. Here I don't really agree with you that putting logic on the client is just shuffling complexity to another side, since (again, IMHO) the complexity cost for a given functionality is less on the client side.

Also, maybe surprisingly, there is very good I18N and accessibility support from Dojo 1.x, supporting Date, Number and Currency adaptivity (out of the box for most countries) for both smart textfields and widgets alike.

Also, since it is possible to create very modularized widgets, which (show up in the HTML as just a div with a special property), those widgets can themselves implement views of accessible server data, depending on context, security levels, etc. The control is still in the server, you've just moved the view to where it is anyway :)

3. See above, for some comments on this, but again; funnily enough, when in crunch mode, needing all the help I can get I find that what was before a deal-breaker (not using an IDE) with Java, is for most parts a non-issue. Code completion would be nice to have, but since js is so much less verbose, I don't need my IDE to generate code for me. I can easily write it myself. I don't need help to fill in invisible XML config files, and I don't need to compile and transfer a JAR file. I just .. code.

4. Yes, of course, a good web framework should be very good at hiding complexity from the developer, and I think that one of my point was just that that is the point :) But what I'm also arguing is that a server-side web framework is adding non-hidable complexity just by the fact that it generates the client at every turn.

My point (again) is that putting the client where it is anyway reduces complexity dramatically in a way that using a server side web framework never can achieve.

Thanks again for your comments.

victor said...

I did this using ExtJs and Apache CXF and it works great. An entire site with just webservices, javascript, and one JSP. I'm talking about an entire enterprise website. If you argue against SOAUI it is because you haven't done it yourself.

Sanny Sanoff said...

You definitely need too get into GWT.

Peter Svensson said...

@Sanny: Actually, GWT is quite good. I like it :) You would probably have guessed I didn't, right?

The reason I like GWT is that it also does away with the cumbersome regeneration of the page at each user action, and seems to lack most horrible config-file mapping dramas that most web frameworks is encumbered with.

You could say that GWT does the same decoupling thing I'm describing, only defined in Java, at the server, in compile-time.

The obvious drawbacks are that the code-test-scream cycle gets slower compared to just putting out a new text-file on the server and reload the page.

Also, you get tied up to whatever widgets and versions of client libraries GWT du jour supports.

Those are minor points, though. For me, it feels better to have a clear overview of what my client looks like on the page, without having to go through any contortion. It's not a great contortion, though, and I think it's a very good starting point for people experienced in Java, but not JS.

Cheers,
PS

TeMs@ said...

"And what does that mean? It means that the status quo has to go. No Tapestry, no JSF, no Wicket, Struts, Ruby on Rails or Symfony."

We just did the same analysis about one year ago. We just stated that there were no javascript framework to do just the same as theses frameworks serverside.

There are just JS/Ajax Framework that provides "widgets", that does fancy graphical things (drag and drop, etc.) or that just help a bit in handling javascript.

So we did Archetype. It's build above Prototype and it provides all you need to just do what you didi before on the server:

- Dependencies loading (so you don't have to keep this in mind and won't have problems with something not currently loaded)
- Configurable and overridable Logger
- Lightning fast Templates to render views
- Enhanceable Component system that helps managing MVC and simplify both communication (through very easy to use broadcasting Event system) and code writing (for example, your "this" is always the component, even in a callback). It even has transversal abilities which able you to add your own services around components in a "annotation like" way.

This framework uses a short but efficient code to do all this and was compared as "at least" comparable to Dojo in it's usefulness (even if it does not covers the same things, but you can use both without any problem ;) )

http://archetypejs.sf.net (ok there's no post on the blog for some month, but we're working on a 0.2 several time a week, so don't be afraid ;) ).

We're also looking for contributors so if someone could help us, we could communicate more and do more documentations :D

Peter Svensson said...

@tems@: That sounds very interesting. It's also very nice to meet people who has made the trip.

I'll definitely go check it out (I can't say yet if I have time to contribute :), but please do comment more on how you integrate with the serve-side. DO you use DWR or plain JSP's to arbitrate server data?

Cheers,
PS

c43ng said...

Hi, I'm quite new to this approach for web application, but I think that it's a kind of "evolution" which is unavoidable and well, intriguing.

I have some notes, which are totally arguiable obviously:

1) as far as I know, there's nothing wrong building the interface in the browser, at least if it's a dekstop browser: they are hosted in powerful machines. But if it's a mobile phone or some other kind of disabling technologies, well, the story is different (BUT still doable, as gmail has shown us)

2) making all and everything in the client..well, that's something that's make me think. If you plan to make your system to talk with others, you simply cannot code the business logic in js --just code them in a resource-oriented or big-ws way, actually i prefer the rest way--

So what I think is: the server side just can't be a storage where put your data, he has also to manage SOME interactions like business methods and so on. Anyway, the main fact is that still a web framework is missing (ouch, that's hurt!):

Imagine a wframework wich is able to take your data (relational or less) and associated rest-oriented business methods and can build some basic (really, basic: a logo, maybe a list/table for presenting the data, links to linked resources and eventually a form to modify the displayed item/s) html.
Then you would have a complete old-style web site, which can be consumed by the less-rich clients (which exists today), and you can have also a RIA (pick something here between java, js, silverlight, eccecc) version.

Just my 2cents, though :)

Peter Svensson said...

@c43ng: I'm certainly not advocating putting business logic on the client! The point of thin server architecture is to avoid mixing domain logic and put the 'client' on the client.

The server will be responsible for all security and for all business logic. But client state, UI, presentation of data, et.c. is all on the client. The server will not produce any HTML/etc.

A lot of companies have companies that use this approach today already; WaveMaker, Smartclient, TIBCO, Appcelerator, et.c. I prefer to leverage Dojo's hierarchical widget model and do it by hand, but that's me :)

Cheers,
PS

Andreya said...

Hi Nice Blog .If your time is less valuable, then it is probably less worthwhile to web time clock .