Thursday, September 22, 2005

frameworks, magaframeworks and metaframeworks

As I am getting ready for my new web development shop (and while I get my first client), I decided to take a look at the ever changing Python web development landscape. I have been developing with Zope 2x for years and I like it a lot, but sometimes it feels more complicated than it should be, plus there have been a lot of recent developments in this front, so I thought maybe I should at least check.

I recalled reading that at pycon in march, one of the most successful presentations was Michelle Levesque's PyWebOff, so that was my first stop. Sadly, the task of single handedly evaluating half a dozen or more python web frameworks proved to be too much for one person, so right now the project is stalled. Nonetheless, it is very interesting to note that the Python web framework landscape has changed so much in the past six months that the projects considered in the PyWebOff study do not include any of the (arguably) most notable frameworks to date: Django, Subway and TurboGears.

Well, that last point was not entirely accurate, since both Subway and TurboGears use CherryPy as the web backend. And that brings me to another confusing point, the concept of a megaframework. TurboGears seems to be the first project to use the term, but Subway is built on the same concept and of course other languages may use it to.

But what on earth is a megaframework? Well, instead of creating a new framework from scratch, a megaframework is an assembly of already existing frameworks. These frameworks usually include a layer of unifying code wrapped around a web backend, a templating engine, some kind of persistence or database backend and maybe a javascript browser frontend. Also, a simple setup procedure is very important to easily install all the required software. For this last step, setuptools and EasyInstall are popular Python solutions.

So there are frameworks and megaframeworks to evaluate, but that is not all. Since there is a growing number of Python web frameworks, and no easy way to move an application from one framework to another or to interconnect applications between them, PEP (Python Enhancement Proposal) 333 proposes the creation of a
simple and universal interface between web servers and web applications or frameworks: the Python Web Server Gateway Interface (WSGI).
Python Paste is what we could call a metaframework. It uses WSGI to create a framework-neutral set of components and also assists with application setup and deployment. Exactly how this helps or why anybody should implement WSGI support or use a metaframework like Paste is something I still have to fully understand. Maybe I will talk about this in another post.

Considering the varied and rapidly changing landscape, it seems quite a chore to carefully analyze the many available options, but I don't think this is a bad thing at all. There are many approaches to web application development and Python is a language capable of making any of them work well, so maybe it's just a matter of taste.

One final thought for today about the new wave of Python web frameworks: I really can't understand why Zope 3 seems to be entirely ignored in this respect. Zope 3 is a framework that was carefully developed taking into account the many lessons learned from Zope, which is without a doubt the most successful Python web framework to date. Zope 2 managed to get a reputation as an unpythonic framework, so maybe Zope 3, even with its painstakingly designed component architecture, is paying for it. Or perhaps there's a lack of influential bloggers touting it as the next big thing (Rails anyone?) or the project's web page is so terrible that few people think it's worth checking out. Whatever the reason, I think Zope 3 should be taken into account when exploring web frameworks of the future, so I intend to look into this matter more thoroughly.

Well, this is just a quick overview, but anyone can see that it's no easy task to pick the web framework for Python. I'll try to do it, anyway.


At 6:26 PM, Blogger Ian Bicking said...

Paste is really intended to assist in making megaframeworks (that name didn't exist until lately, though). It's not a huge all-encompassing framework -- it's really a set of tools that can be used in whole or in part to bring together other pieces of code.

It also uses WSGI to make it possible to develop other kinds of code separately, where before there was no common language they could speak. Like URL parsing, providing services (like session support), and the "framework" itself -- without a common language, these must all be implemented simultaneously, and in a coupled way. And maybe that works... but I still think there's a benefit to having the potential to share code there. But that's just one kind of tool that Paste is trying to provide.

Besides the Webware reimplementation that I wrote (which uses all these tools), Ben Bangert has started a project to create a megaframework on Myghty (and all the other regular suspects), using some parts of Paste, in Pylons

At 9:02 PM, Blogger Leonardo Santagada said...

Until there is a screencast showing how to make a wiki in zope3 in 20 minutes I will never again try to read the enourmous amount of documentation needed to start understanding how it works. For me and a lot of python programmers zope just isn't real, no one can really understand how it works.

At 1:08 AM, Anonymous Anonymous said...

I think u shouldnt take that "megaframework" thing seriously... I guess it's just a joke.


At 12:33 PM, Anonymous Kevin Smith said...

It would be a shame to discard Zope3 so easily.

Zope3's strength isn't to hack out a simple use case in 20 minutes (or in 2 minutes if you've seen the latest Zero To Rails screencast). Z3's strength is to leverage best programming practices into building enterprise-class web sites.

To understand these best-practices, I needed to buy both Z3 books (a spendy but worthwhile investment).

To experience the fruits of these best-practices, imagine you have a client. You're building them a site using your favorite framework. They want an intranet with a blog and a wiki. Comments would be nice. How about categories? Oh and let's make it multi-user, but we don't want our interns to publish without review. Make the textfield support ReST and Textile. Swap out various field widgets for even better ones. Let's use LDAP. How about subscribers to blog entries? Then package it up as a site to redploy for the sales division to run in the same process. Let's make a customer extranet, it'll need to be skinnable. Hey, don't forget about the European partners, the site needs internationalization. What about testing and documentation. Oh and yeah, the client has just a few more minor suggestions...

No problem with Zope3.

In my experience, building the initial program is easy, changing it is a nightmare. The interdisconnectedness of Z3's component architecture allows the program to pleasantly evolve.

Instead of adapting the business model to the needs of the website, the website adapts to the needs of the business model.

This is the power of Zope3.

At 6:37 PM, Blogger Jeff Shell said...

I feel the same frustration at the lack of publicity Zope 3 seems to get. A lot of what people seem to be going gaga over with their 20 minute demos was what I was able to do with Bobo back in early 1997. Sadly, Zope did spiral out of control. But Zope also delivered a lot of what is now finally in the Python core long before it was in the core (subclassable C classes, which enabled persistence, acquisition, computed attributes, etc). Sadly this made it harder for Zope to work with other tools (epydoc, for example, could not understand ExtensionClass based classes). On top of that, there were too many ways to develop for Zope 2. Right now, file system based development using plain old Python classes and modules is the *only* way to really develop for Zope 3 (well, there is that ZCML configuration).

I've been delivering professional solutions on Zope / Principia / Bobo for almost a decade now. My blog, Griddle Noise, has some coverage of my experiences. But I'm no salesman, I'm just an aging crank. I do have plans to write some more entries in the coming days, real work deadlines permitting.

There is work under way to integrate Twisted into Zope. I'm not sure what the scope of that project is, but if it works out it would be a good showcase of how two very large and established Python frameworks can integrate.

At 10:48 AM, Anonymous gumi said...

Zope had its fifteen minutes, and failed because it is presented in such a dishonest way - as an actual alternative to the existing equivalent professional tools in the same field.

After I stepped from youth to adulthood and realized that programs, languages and frameworks are tools instead of goals, and have no value in themselves, that their value lies in that they get the customer's job done well in the short term but especially in the long term, I opened my eyes and honestly evaluated the alternatives, not for their coolness factors, not by the value they produce for me, but instead by the value they produce to the client.

That's what I recommend.

At 5:24 AM, Anonymous Laurent Szyster said...

Zope is an integrated Content Management System developped on top of Medusa, which eventually turned (back) into a web framework, not the other way round.

It is simple enough for the casual CMS developer, but there is a steep learning curve from the ready-made solutions up to component development.

And, unfortunately, the Zope folks changed the API too many times and in doing so made too much compromise with the existing "installed base".

In a vain effort to compete with other integrated web application servers, Zope 2 and 3 each time pulled some of the rug from under its application developers feets, piled up conflicting API but failed to simplify its core.


Zope is easy to start with, but hard to master. It does fit in the LAMP stack of an SOA, but not as well as the various smaller and more generic web frameworks available.

If you need an intranet solution for yesterday, Zope is a good choice.

But if you really need an integrated, high-performance, statefull web application server in Python, use Medusa: Zope is just one of its best known application :-)

At 10:50 AM, Blogger Roberto Iza Valdes said...

This comment has been removed by a blog administrator.

At 10:49 AM, Blogger Kim said...

From time to time we find a blog interesting enough to read. Yours is in that category.

My partner and I have this interesting example marketing viral tool. It is a cool and effective example marketing viral tool.

At 9:43 AM, Blogger Tony said...

From time to time we find a blog interesting enough to read. Yours is in that category.

I have a great article directory for viral-like traffic. Articles traffic has always and will always work to get a site great permanent traffic.


Post a Comment

<< Home