Why interfaces will be procedural

I believe that next-generation visuals for computer interfaces will be generated by flexible computer programs (procedures) rather than painstaking pixel-based editing and layout that we use today.

This trend is very apparent, very necessary, and yet very much ignored by many industry leaders.

I'd like to name several reasons contributing to my point, and then I'd like to explain it:

1. Animation enhances interactivity, and it is best generated by a computer, not frame-by-frame by a human. Some interactive "spaces" are simply too large to be hand-animated. This animation should include more than simple movement of pre-rendered artwork.

2. Resolution Independence is important as we get a larger variety of devices and, even on the desktop, a wider variation in screen resolutions. Fixed-resolution layouts will require increasingly more work to be scalable.

3. Design flexibility for dynamic content and dynamic content is very small currently. Most dynamic pages (in HTML) follow a very strict layout, even if the content plugs in dynamically.

Consistent interface, however, requires a certain attention to behavior and detail that is missing from all these systems, however. I've seen 500 different scrollbars in Flash, and none of them worked as well as the one on the Mac.

Still, that's partially a technology problem, partially a focus problem, and partially a maturity problem.

Important attributes of procedural UI (as I know them now) include the following:

• Animation as a function of time: All animation should be time-based, not frame-based. Animation should be defined in a way that allows high-quality motion blur. Animation should be able to run at arbitrary frame rates, depending on the hardware capabilities.

• Type should be vector-based, and sub-pixel accurate in all respects (including kerning and tracking.) Small point sizes, of course, may choose to be replaced with a bitmap font, but these choices should be overridable for animated effects (which would look bad with discrete type choices.)

• User interface elements should be procedural: Currently, this could include 2-D vectors with effects (a la SVG), since vector shapes are the easiest ones to describe. But certainly any existing 3-D modeling+rendering method should be available for use. You should be able to plugin a ray tracer with a physically-correct lighting model to make beveled buttons, if that's what you like to do.

• Movement is "at least affine" so that any object can be scaled, translated, rotated, etc. Many objects, of course, will have many other parameters, including colors, opacities, sizes, shapes, etc. For example, scaling a beveled object would look different than scaling a bitmap.

• Granularity of resolution: Content authors should be able to work at a pixel-based resolution for doing some kinds of layout, since legibility and interaction will suffer otherwise. (For example, type, scrollbars, or buttons scaled below a certain amount become unusable.) It is probably advisable to allow a system to automatically constrain certain objects to a "minimum scale" while maintaining the other attributes described.

Existing interfaces are technologically far behind the state of the art for animated (interactive) experiences. But because of certain details that are very strict in user interface, we haven't seen a great deal of interactive richness in user interface so far. The space that is 3-D gaming (including the interfaces found in games) will converge with desktop interface in the future, because the capabilities are there.

However, if we're to see this kind of richness in a day-to-day sense, it must be approached with some of the attention to detail that has gone into existing interfaces (this is more than you think.) However, with the right technology underpinnings, these old metaphors can be extended to new technology and faster processors easily.