Here are some screenshots of Penn State sites that will use it or are already using it:

Adding, Moving and Deleting portlets per page or folder is incredibly easy now:

Please check out ContentWellPortlets and let us know what you think.

Hi, Eric. You are the release manager of Plone 4, which is expected to be released in late 2009. Do you think we will have a final release of Plone 4 this year?

I certainly hope so. Our current schedule aims for a December 1 final release. I really don't want to have to spend my Christmas break packaging up a Plone release, but we'll see how it goes. The Plone Foundation asked for a release before year's end and that's given us about 7 months from start to finish to get Plone 4 shipped.

We set up a very aggressive schedule and, so far, have been right on track. Because of the aggressive timeline, we'd planned to make the featureset much smaller than previous major releases. Surprisingly, we received a record 57 PLIPs (PLone Improvement Proposals), 42 of which were accepted for consideration by the Framework Team. The developers were given only six weeks to complete their initial implementations and 37 of those 42 PLIP implementations have been submitted for review. That's an 88% return. Amazing. I was telling myself that we'd be lucky to see ten. The Plone developers have met this challenging schedule head-on and are doing absolutely amazing work. I could not be prouder.

Browsing the accepted proposals, I think the major improvements are BLOB support in the ZODB by default, the change to SecureMailHost, the usage of TinyMCE instead of Kupu as default richtext editor, the new collections backend, the new commenting infrastructure, a redesigned search form, and content import and export. Would you agree and can you elaborate on these changes?

As release manager, I feel somewhat obligated to say that I love each and every one of my PLIPs equally. That said, there are definitely some that are going to generate more buzz than others. Foremost, and the reason this release even exists, is the move to Python 2.6, Zope 2.12, and CMF 2.2. Being able to depend to an officially-supported, more efficient version of Python is something that everybody has wanted. Between those three upgrades, there's certainly a lot to for us be happy about: optimizations, packaging, BLOB support, mailhost improvements. I have to give a lot of praise to Hanno Schlichting and David Glick, they've handled the majority of that integration and are saving me from having to worry about it at all.

TinyMCE is more actively maintained than our current editor, Kupu, and has proven to be much more extensible. It will be forming the basis for the Deco page layout editor that will be coming in a future release. Introducing it now saves us from some of the problems we ran into with Kupu in later Plone 3 releases and exposes the community to it before Deco arrives.

There has been a lot of work done to improve Collections: a much more intuitive UI with inline updating and live results, simpler templates, and a much cleaner backend. Four Digits just put out a writeup describing the results of their sprint. It's a good read.

One PLIP that I really think isn't getting the attention it should be is the indexing and search of East Asian languages. It's been something that has held back adoption of Plone there and could open up a large new market for us.

Plone 4 will have guided tours for new Plone users driven by Amberjack. What is this all about? Will I be able to make my own guided tour of my custom Plone site?

Unfortunately, the Amberjack implementation didn't make it in before the deadline, so we won't be seeing it in the 4.0 release. I'll be pushing to get it into 4.1 though. The idea is that we'll be able to provide walkthroughs of different tasks throughout the site. So if a user is having trouble adding a new Collection, for example, they'd be able to click a help link to be taken through the process: "click this highlighted button", "add your Collection's title here" and that sort of thing. Tours should be theme-agnostic, so existing tours will integrate well with your site, and new ones can be added without much trouble. Nate Aune showed it off in a recent lightning talk and gave a great overview of how Amberjack integrates with Plone.

Is there a preview of the new Plone default skin?

It's still very much a work-in-progress. Alex's goal was to create something more modern, and could be legitimately considered for use as a corporate site with very little customization. The CSS and HTML behind it should be a lot simpler and much easier to use as a starting point for a custom theme.

Thank you very much!

Thank you!

The web services API for Plone project was part of the Google Summer of Code (GSoC) project "Atom Publication Protocol (AtomPub) for Plone". The web services API for Plone (wsapi4plone.core) package is a Plone product that provides small, easy to learn XML-RPC interface to Plone content. The Parfait (wsapi4plone.parfait) package is a web services gateway interface (WSGI) middleware that enables AtomPub for Plone. The GSoC project goal was to provide an XML-RPC interface for Parfait that in turn would enable AtomPub requests and responses for Plone.

The project was unique for GSoC, in that there were two individuals working on the same project. Chelsea Bingiel and myself have been working towards the same goal but on different levels of the system. Chelsea had been working on Parfait, while I was working on the core package of the wsapi4plone. The end result is a functional AtomPub interface for Plone, as well as a Plone web services API.

For about the past week I have been working on batch creation of Faculty Staff Directory people for a Plone site that WebLion will soon be hosting.  The site is currently Plone 2.1 and the Faculty Staff Directory is a MS-SQL db with PHPParser making the queries.  These tables have been dumped to csv files, massaged slightly and then processed and combined with python to extract the data and create objects which are sent via xml-rpc to a Plone site for creation or update.  Things had been running 'fast enough' but did start to slow down yesterday when I started to update the Person objects with relationships to the person's specialty.  Erik Rose had mentioned archetypes.schematuning yesterday and the speed up he had seen in another site, so I thought I would give it a try.

Test Environment

Here is an overview of the system I am working on and some numbers that I observed.

The records were updated in one request to the wsapi4plone.core XML-RPC webservice using wsapi4plone.client and wsapi4plone.webslinger to pre-process the data before sending any requests.

Environment in which these benchmarks were made:

• RHEL 5.3 32-bit running in VMWare Fusion.  1 virtual CPU, 1GB RAM.
• Python 2.4.3
• Plone 3.2 from Buildout

Packages Installed for Plone

 Benchmark Times for the Creation or Update of 244 FSDPerson Records

Our first target was the 5 tabs that frame the main calendar view. Day, Week, and Month would show grid-style views for their respective periods, the List tab would show a month's worth of events in straight-down-the-page format, and Past would show (all?) the past events, again in list format. These tabs have been a common source of surprise to users, as they conflate duration with presentation style. Our new UI has only Day, Week, and Month tabs, following the conventions of most other calendar software. Calendar and list view are available in each of these via a pop-up menu that switches formats via JavaScript. We removed the Past tab altogether, as one can navigate to the past by clicking the left-hand arrow or by clicking the calendar icon to expose a pop-up calendar.

Our other effort was to localize the product's time and date display. Substantially written by our friends in Europe, Plone4Artist's formats were hard-coded to 24-hour time and day-month date ordering. We took a step toward obeying Plone's localization settings by fixing the former of these:

(We didn't realize until too late that Plone 3.2 introduced a time-only format string in portal_properties, so we spent quite a long time designing and implementing an heuristic to deduce a time-only format from the time-and-date format that was the only option in 3.1 and earlier. 3.1 people are welcome to send us beer or baked goods.)

Congratulations and thanks to all the sprinters! The calendar situation is a bit brighter, and we had a lot of fun brightening it.

“Namespaces are useful for avoiding name collisions.”

Of course they are. I should been clearer in my main point, as a lot of folks ended up banging on an open door here. Namespaces—as an abstract concept—are lovely. What’s wrong is our use of setuptools’ implementation: as we use them, namespace packages are awfully noisy and obtrusive from a package consumer’s point of view—they make new users walk straight through their implementation details, both in svn and on disk, before they can get to any code.

But a lot of this is due to funny conventions. We use namespace packages in an unusual way in the Plone world: we never put anything but a single package inside them, making the outer folder mostly empty and mostly noise. We don’t take advantage of setuptools’ support for putting N submodules and packages inside a namespace package, which would make the signal-to-noise ratio more acceptable. So I wonder, can we come up with a shorter, fewer-empty-folders way of namespacing?

Indeed we can. One of my proposed solutions, using names like plone_app_redirector instead of plone.app.redirector, provoked an implication that I was simply inventing a different spelling. Quite the contrary, plone_app_redirector is simpler in 2 substantial ways:

• No 2 extra folders to traverse on disk and in svn
• No __init__.py files with unusual import magic to understand or ignore

“Products.* is no longer uniquely magic.”

Thanks to Martin Aspeli for pointing this out; I meant to mention it. As Martin says, any package can have the auto-loading magic of Products.* in Plone 3.3 if it declares the right egg entrypoints. No more ZCML slugs—hooray!

However, Ian Hood and Chris McDonough also raise a good point: as funny-looking as the Products namespace is, it at least signifies “this package is a Plone plugin”. (There aren’t many third-party Plone plugins that are also useful in a general Python context, so the distinction remains relevant.) We probably should include this I-am-for-Plone-ness in the package name so as to not crash into comparable plugins for other frameworks. A few possible naming conventions:

• PloneFormGen. Start everything with a “Plone” prefix, as most Trac plugins start with “Trac”. But, if applied inconsistently (as it would be during any transition), this would land us in the same situation as KDE’s start menu, where things randomly start with K and new users wanting a spreadsheet have to look for both Spreadsheet and KSpreadsheet.
• PonyPloneTheme. Put the Plone identifier at the end. With the entropy up front, this is better for visually scanning alphabetized lists and for type-to-select. (In places like the Collective svn, similar types of products, like themes, could be grouped into folders as with the current pasplugins folder.)
• NerfHerder. Keep Plone signifiers out of the name, and use egg metadata like trove classifiers. This has the possible disadvantage of colliding with non-Plone packages, but, then again, namespace is pretty big. Regardless, we should get some trove classifiers for common types of Plone add-ons. If there can be a Topic :: Desktop Environment :: Window Managers :: Oroborus :: Themes, there can certainly be one for Plone themes, one for Plone plugins in general, and perhaps a few other subslicings.
• Products.NerfHerder, perhaps the most reasonable compromise of evils: we’re already a decade into that particular hole, and it’s only one level deep. It doesn’t invent any more conventions to memorize or transition to like collective.* or plone.*, it doesn’t smash into general-Python namespace, and, if you don’t move your trunk when you eggify, it doesn’t defeat type-to-select either. I still dislike the namespace package due to the previously discussed implementation ugliness, but that may be fixable by technical means (watch this space). If I could erase the world and start over, I might go with a different scheme. But, since a transition would take forever and be confusing all the while, I’m willing to settle for Products.*. In fact, I’m fairly persuaded by Chris McDonough’s statement that “It’s completely, totally useless to have Products outside the Products namespace.”

“Namespaces should communicate licensing.”

Martin writes:

Namespaces also communicate ownership. For example, we’ve been able to formulate useful policy around the licensing of packages in the plone.* (soon to allow BSD licensing) and plone.app.* (to stay GPL licensed) namespaces.

Cordially disagree on this one: this is what the license metadata field is for. (Incidentally, I suspect no one outside an intimate group of core developers has any idea the difference between plone.app and plone.*.)

“The trunk should move when you eggify.”

Plone is one of the few projects where egg names typically reflect their import paths. Joel Burton, who interfaces with lots of new users in his training classes, finds this aids comprehension. So let’s keep it. In fact, I never suggested otherwise: either way, you’ll be checking out a folder called "trunk", not a folder named after the egg. My beef is with the practice of uprooting an existing svn trunk and moving it to a new location when eggifying an old product. There’s just no point. WebLion trains new developers all day long, and this practice is a rich source of confusion. It’s also a waste of time to have to look in two places when browsing a repository, and, if carried through, it’ll result in hundreds of abandoned top-level folders in places like the Collective.

Joel further writes, of keeping trunks still:

[T]he intended benefit, doesn’t exist. Your buildout wouldn’t still work if it expected a classic product at /svn/JoelsBlog--there isn’t a classic product there, but a nested egg namespace. So, your buildout would -still- break, and in a much less newbie-understandable way.

The intended benefit is not to keep buildout from breaking. It’s going to break no matter what—which is another failure that should be separately addressed, perhaps with better checking for common mistakes. My intent, rather, is to surprise people less when browsing repositories and, secondarily, to reduce top-level pollution in repositories.

“You’re an idiot.”

Probably. Here are my recommendations nonetheless, now a little better shaped thanks to your good-natured hammer blows:

• Reserve namespace packages for breaking up enormous legacy packages, as intended. Namespacing for any other purpose can be done more simply with plain prefixes, and many of our existing uses of namespacing (collective.*, plonetheme.*) would be better served by egg metadata. I'll be using egg metadata only and hanging out in the Products namespace to avoid colliding with non-Plone projects.
• Don’t move the trunk when eggifying. You’re going to break people’s checkouts either way, so why double the number of folders in the repo, make everyone search twice when browsing, and defeat type-to-select?
• Keep company names, license names, and other metadata out of package names—since changing them breaks people’s imports and buildouts—and nest them cozily in egg metadata. Plone products should probably all be in the Products.* namespace. (You talked me into it. :-))
• Fix those ZopeSkel templates. In answer to jjmojojjmojo’s question about how best to do that, I’d have the plone3_theme template spit out products in the Products.* namespace but give an example like “PonyTheme" when prompting for a name. And continue to put “web zope plone theme” in the egg keywords.

Again, I really appreciate all your responses. If any of my reasoning is wrong, please point it out and save me pain!

To be sure, complexity is sometimes the price for solving a genuine problem, but today I'm going to hammer on one instance that's been almost purely a reckless running after shiny new toys. In particular, we have acquired an unhealthy habit of abusing namespace packages. How and why? Here's a primer on when we should use them and when they just make things more complicated.

What's a namespace package?

Namespace packages are those things like collective in collective.tagcloud or plone and app in plone.app.portlets. They are prefixes on the pathname of a Python module, implemented by wrapping an empty folder (called say, "collective") around it and plopping a short, magical __init__.py inside. The result is that you can distribute collective.foo and collective.bar as separate packages but still import them via import collective.foo and import collective.bar.

A much hairier and more comprehensive explanation is available in the documentation of setuptools, out of which namespace packages spring.

Good namespace packages: breaking up is easy to do

Namespace packages are great for breaking enormous legacy packages into smaller parts that can be maintained and distributed separately, without breaking people's imports. They were likely motivated by the gigantic PEAK framework which is by the same author as setuptools, Phillip Eby. As PEAK grew, Eby wanted a way to update subpackages like peak.config or peak.storage without having to ship an entire new 50MB tarball. However, he couldn't just change over to separately distributed modules called peak_config and peak_storage without breaking all the client code that said things like…

import peak.config

So, he made peak a namespace package and had the best of both worlds, the only cost being some occasional weird lines in an __init__.py.

Zope has a similar story. I'm guessing at the archeology, but it should be safe to say that zope started out manageably small and only later grew into a 68MB behemoth that was inconvenient to distribute as one gigantic tarball. Of course, by that point, enough people depended on it that they couldn't just change all the import paths around, so the zope package became a namespace package, too, giving us today's zope.index, zope.event, and so on. All well and good.

Bad namespace packages. Bad. Go sit in the corner.

However, for a variety of reasons, the Plone community has gone absolutely bonkers with namespace packages. We use them for branding, like the Simplon company did with simplon.plone.ldap. We use them for categorization, like plonetheme.nautica05. We even use them for no apparent reason at all, like the 315 products called collective.something. We've caused a lot of noise for no technical or usability benefit.

Filesystem paths: ouch!

Out of the box, if I want to get to the source code of Plone's redirection machinery, I have to go into plone.app.redirector-1.0.10-py2.4.egg, then into plone, then into app, then, finally, into the redirector folder, where all the code lives. The extra traversals are a waste of time. David Glick even got mad enough to cook up the omelette buildout extension to patch around the problem. Am I missing some reason we didn't simply go with plone_app_redirector and save 3 levels of folders and weird __init__.pys? It's not like we had legacy imports to placate.

Subversion hide and seek: ouch and a half!

In svn, the problem repeats itself with an additional twist. People eggifying classic products (our own FacultyStaffDirectory is guilty of this) often move their source to a new folder with "Products" prepended: Products.FacultyStaffDirectory. I can understand the use of a Products namespace package for compatibility with Zope's old import magic, but is it really necessary for 50% of the stuff in a repository to start with "Products"? This kills typeahead, and, more importantly, makes developers look in two places for the code: "Hmm, let's look in FacultyStaffDirectory. No trunk folder! Ah, a README-trunk.txt: 'Your princess is in another castle.' Okay, I'll back out of here and try Products.FacultyStaffDirectory. Those are 2 minutes I'll never get back." Sure, use the Products namespace if you must, but let's keep our svn directory names unsullied.

Branding: double ouch!

When namespace packages are used for branding, everything has to change when the company divests itself of the product. When Simplon donated simplon.plone.ldap to the community at large, it had to rename it to plone.app.ldap, thus breaking everybody's buildouts and imports. Let's keep company names out of package names.

Description: ouch a third time!

Namespaces are often misused to describe functionality, like in plonetheme.*. This is just as wrong as when I named AutoMemberMakerPasPlugin (which I've remedied in WebServerAuth) and is nothing more than a repetition of the horrors of file extensions. Actually, it's worse: at least extensions are postfix and don't impede type-to-select. We have perfectly good egg metadata where we can declare that a package is a Plone theme. In fact, the plone3_theme ZopeSkel/paster template spits out the keywords "web zope plone theme" by default. The plonetheme prefix is redundant right out of the box!

Let's fix this

A lot of the excess namespaces come from some bad ZopeSkel/paster templates which make it really hard to create an egg-based product without a namespace package. I certainly don't blame newbies for getting led down the wrong road, but a lot of us should know better. Let's clean up our act:

• Reserve namespace packages for their intended use: breaking up enormous legacy packages. I see a world market for maybe five namespace packages.
• Stop using the collective namespace. It conveys no useful information. It's just noise.
• When you use the Products namespace to eggify an old-style product, don't move your trunk. You're going to break people's checkouts when you delete the old trunk anyway, so you might as well just change it in place.
• Scribble your company name all over your readme and egg metadata, not in your package name. Learn from Apple, who is now stuck with an API full of NSThis and NSThat.
• Fix the ZopeSkel templates that spit out things like plonetheme.*. We can't expect themers to be Python import machinery wizards and fight it themselves.

Those of us with an understanding of namespace packages have a responsibility to use them properly—everyone else will follow, for better or worse. This is a battle in the complexity war that's technically easy to win and one that new users and developers see early in their Plone experiences. Let's swat this piece of low-hanging fruit down.

After you read the excellent comments below, check out my summary and response in Namespace packages: replies to replies.

Thinking of signing up for one of the five training events happening on the Tuesday and Wednesday before the symposium? Joel Burton, Donna Snow, Chris Calloway, Gerry Kirk, Chris McDonough, Tres Seaver and Paul Everitt are all ready to help you get to grips with topics ranging from Plone theming and site development best practices to Agile project management, Python and repoze.bfg.

You still have time to register! But not for much longer.

Cinderella's coach pulls away from the registration ball on May 18. After that, your fairy godmother may not be able to make the registration forms work for you.  So get together those glass slippers, pumpkins, broomsticks, magic wands or whatever other paraphernalia you need, and sign up now.

Why do they matter you ask?  Well, let me tell you!

• In 18 days, the events of Plone Symposium East kicks off with an excellent line-up of training sessions...
• Followed by the symposium itself gets underway with an opening address from Alexander Limi, in only 20 days.  Which leaves...
• ...just 3 days left to register for the symposium!!!

For those who have already registered, we look forward to seeing you two weeks from now.  In the meantime, hop over to the symposium wiki page, to look for ride and room shares on the message board, and join and organize sprints and Birds of a Feather sessions. 

For those who haven't yet registered...HURRY UP!!  We can't wait to see you either!

Register Now

The repoze.bfg web framework is essentially "Zope lite": it is useful for development projects where Zope might be overkill, but it uses concepts and software that may be familiar to Zope programmers. No prior Zope experience is required to use repoze.bfg. As such, it fits in the same space as Pylons in the Python web development world: a small, unobtrusive framework that makes few assumptions about how you want to accomplish a task.

This framework also forms a bridge between WSGI and Zope technologies.

The Repoze project (http://repoze.org) is aimed at making Zope technologies useful in the larger Python world, as well as making it easy to use WSGI technology in the Zope world.

Audience: Developers,

Requirements: Windows/Linux/Mac OS X laptop required for code-along. A CD with a pre-chewed virtual environment (Ubuntu Linux as the guest environment) will be made available to each attendee.

The development environment will not require network access. CDs will be made available to each attendee. Attendees who have not already configured a repoze.bfg development environment will be encouraged to kick off the installation of the VirtualBox environment from the CD that will allow them to play along during the guided development phase. An additonal 10 minutes will be devoted to helping resolve installation issues at the end of the lecture period (right before the break) by at least one presenter and other volunteers present. Attendees who already have a running environment can ask questions or optionally take their break early during this period.

No Zope experience is required, but it uses concepts and software that may be familiar to Zope programmers. 

Register Now

Are newsletters part of your Web presence? You may be able to manage them without installing an additional newsletter product. Using Content Types and Collections, Plone can handle most of the requirements.

Here's an approach that I've made work for a couple of our online newsletters:

• https://weblion.psu.edu/trac/weblion/wiki/NewslettersInPlone

Recently one of our partners provided a use case for such a portlet.  What they need is a collection portlet that continues to appear even when it is empty.  The portlet itself provides a link to the events section of their site, as well as a list of the events for the coming week.  Instead of disappearing when the collection is empty, the portlet should not only appear, providing the necessary link, but also display a message such as 'No events scheduled this week'.

That Plone's news, events, and collection portlets by default simply disappear when empty, is certainly a better option than having half formed portlets, with no content, popping about.  But I'd like to see the option provided to users. 

This could be achieved by adding a check box to the relevant portlets, allowing users to choose if a portlet should appear when empty or not.  Further, including an additional text field could give users a place to input a custom message for the portlet to display when it has no items

.

Is anyone else looking for similar functionality from Plone's portlets?   Take a look at the example images below and related ticket #9165 at plone.org, with suggested code details.   I'd like to hear your ideas, pros/cons, and any further suggestions.

Suggested collection and events portlets appearing empty with custom messages

Suggested additions to Events Portlet

In addition to Alexander Limi, an exciting speaker program has been assembled that will be present on a variety of topics from Plone experts that you won't want to miss.

The Symposium program is not the only way to participate in the Plone community or develop your Plone and Python skills. 

Two days of training - May 26-27 have been organized to nurture the development of your skills. Noteworthy presenters include Joel Burton - Best Practices, Donna Snow - Plone Skinning, Chris Calloway - Python for Plone, Chris McDonough/Paul Everrit/Tres Seaver - Developing using repose.bfg, and Jerry Kirk - Agile Workshop.

Sprints will be held through out the week and through the weekend (May 26-31). Lots of good sprint topics are being suggested. Come sprint with your friends on your favorite topic or add your own topic.  Sign-up on our Sprint wiki.

Please register now and join us in a week full of Plone goodness.

Chris Calloway will lead a two-day hands-on workshop on Python development: "A crash course in Python for beginners taught by interactive examples. This is a shortened version of my 5-day PyCamp and teaches how to program the Pythonic way." Python is a powerful object-oriented programming language that is easy to learn, enables integration with other languages, and has a vast set of standard and third party libraries. Python is licensed with a GPL-compatible open source license, and is runs on Windows, *NIX, OS X; has ports for JAVA and .NET; and best of all, Python runs on your Amiga! It's the language used by Zope, Plone, and many other CMS and application frameworks. More information and registration:     http://plone.org/events/regional/plone-symposium-east-2009/training

Agile Programming

Gerry Kirk will lead a one-day workshop on Agile Software Development. Agile processes use short, iterative development cycles by self-organizing teams where project requirements and software solutions evolve through testing and discovery.  Agile programming methods adapt functional requirements and delivered software to rapidly changing realities, such as, uhhh, stock market crashes and falling budgets.  More information and registration:    http://plone.org/events/regional/plone-symposium-east-2009/training