[Gzz] A vob system based on structured graphics?

[Benja Fallenstein]

Hi all,

after a bit of discussion of IRC which only confused the others, here's 
a first writeup of a possible re-write of the vob system I've been 
intently thinking about for the last three days. This is inspired by 
Fresco, which manages to be quite simple yet really powerful by relying 
on separating functionality in structured graphics/widgets into simple, 
orthogonal components.

This is not completely thought-out; it's just a start. I hope to spark 
discussion. *If* this system is viable, though, I hope that it can 
simplify the vob system very much, both for its users and its developers.


In this system, a vob is a graphic of something, like a character, a 
raster image, a window with background and border, a multiply nested 
structured graphic, etc. We don't have vob scenes any more; rather, we 
have different top-level vobs.

Vobs can give information like their size, position relative to a 
baseline, etc. (They can of course be scaled by putting them into an 
appropriate transform.)

There are primitive vobs like rectangle or filled oval, there are vobs 
that take one other vob and modify it somehow, and there are composite 
vobs which take more than one vob and make these vobs into a single vob 
somehow. Vobs are transformed by modifier vobs, like Translate, Rotate 
or Fisheye vobs. Composite vobs include HBox and VBox (as in TeX) and 
Overlay, which takes a list of vobs and renders them above each other.

Positioning a number of objects in different places on the screen, as 
our structure views do, can be handled by nesting each object in a 
Translate vob and putting the Translate vobs into an Overlay. This is 
equivalent to some functionality VobScene has now.

However, this would only be used in places where a view really places 
objects at x/y coordinates on the screen. For example, when an object is 
placed into a box-- with a background and a border around it-- we're not 
really talking about placing things at x/y coordinates, though the 
existing vob system would handle it that way (we would place the border, 
then we'd place the content at a translation of (5,5) inside the border, 
for example).

In the new system, we could use: a WrapperVob, which renders three vobs 
(background, content, border); and a MarginVob, which puts a margin 
around another vob. Then we could write something like,

    v = WrapperVob(bg, MarginVob(content, 5), border);

This is already a more structured approach, but we could make this into 
something along the lines of,

    box = WrapperVob(bg, MarginVob(null, 5), border)
    v = box.cloneContentReplaced(content)

I.e., using the prototype design pattern like for ColoredVobs, we could 
have different types of frames as first-order objects.

The point is that this is MUCH closer to what the view actually wants to 
think about.


Vobs are more or less immutable. They are immutable in that you cannot 
change the structure of a vob-- you cannot add or remove vobs inside it. 
On the other hand, you may modify the parameters of a transformation 
vob, to accomodate for continuous user actions like dragging.

Each vob can be seen as being made of a tree vob sub-vobs (for primitive 
vobs, the tree contains only that vob). Each vob in this tree has a 
number. That is, when you have an Overlay vob containing Transform vobs 
which contain RectVobs, then each of these vobs has an index inside the 
Overlay vob. The Overlay vob itself has index 0.

- Leaf vobs contain just themselves, at index 0.
- Modificator vobs contain themselves, at index 0, and then the vob they 
modify and the vobs contained by that vob, at indices 1+.
- Container vobs like Overlay contain themselves, then the first vob in 
them with all the vobs contained by it, then the second vob with all the 
vobs contained in it, and so on.

Simple arithmetic can be used: If vob U has index i in vob V, and vob V 
has index j in vob W, then vob U has index i+j in vob W.

These indices can also be seen as a 'path' into a vob. Vobs support 
getVob(index) and getTransform(index) methods, where getTransform(index) 
gives the cumulative transformation from the root vob down to that vob 
in the tree. For example, if the vob is wrapped in a Translation which 
is wrapped in a Rotation, getTransform() would return a transform 
concatenating those other two transforms. [getTransform() does not 
return a vob, but a Transform object.]


Each vob (except the root) has one parent vob and inherits one parent 
transform from this vob. Given that we see each vob as a graphic, this 
makes sense. (Note: A vob doesn't know its parent; rather, the parent 
transform is passed to the render() method of the child vob. This allows 
us to use flyweighting, i.e. use the same vob in different places.)

Now, consider a connection. What's its parent, from a structured 
graphics perspective? Surely not the things it connects, since it is not 
inside them. From this perspective, it makes more sense if a view places 
the connections into the same Overlay object as the RDF nodes it 
connects. A connection can then be subjected to the usual structured 
graphics transformations-- color, rotation, scaling, whatever.

The points it connects can be seen as *parametrizing* the connection-- 
i.e., we create a line, but instead of telling it, "you are from (10,70) 
to (44, 58)," we tell it, "you are from the right side of this vob to 
the left side of that vob."

The problem with this is that the connection doesn't know its parent, 
and thus if it is placed inside the same Overlay as the vobs it 
connects, it has no way to actually look at that vob and see what it is 
interpolated to and compute the current interpolation. One possibility 
would be to wrap the Overlay inside the connection vob: Then we could 
give the connection vob the indices of the vobs it connects, inside the 
Overlay vob. Then the connection vob would know about the vobs it 
connects and could retrieve their interpolation. Unfortunately, this 
approach is rather ugly.

This area of the system is what I'm currently chewing on.


One important benefit from this system is caching. Since vobs are 
essentially immutable and do not know about their parents, you can put 
any vob into a cache.

We may need one efficiency hack here. For the simple structural RDF 
views, we would like to avoid having to create a new object per node by 
putting the node vob into a cache. However, with the approach above, we 
would still need to create at least a new Translation vob for every 
node-- except if we place the node in exactly the same place as before 
(not very useful).

To avoid this, as an efficiency hack, we may have a vob that is a mix of 
Overlay and Translation: it would render a set of vobs and store a 
translation (or even linear transform) for each. This wouldn't be as 
clean as separating the concerns into two different vobs, but the 
efficiency gains should justify it. (We can still have arbitrary 
transforms by using the appropriate vobs, e.g. place a node into a 
FisheyeVob).

If we use caching, we could have a PrerenderVob that uses idle processor 
time to prerender its contents. (If it hasn't had time to prerender 
itself, during rendering it would simply render its children the normal 
way.)


Interpolation is handled by two modifier vobs: KeyVob and KeyPlane. 
KeyVob assigns a key to its content. KeyPlane interprets the KeyVobs it 
contains directly or indirectly. For a simple structural RDF view, you'd

- put each node into a KeyVob
- put the KeyVobs into an Overlay, transformed
- put the Overlay into a KeyPlane

When a KeyPlane is to be interpolated to another KeyPlane, it matches 
the KeyVobs it contains against the KeyVobs the other KeyPlane contains. 
The KeyVobs can be arbitrarily deep inside the KeyPlane's tree. The 
KeyPlane takes into account all KeyVobs it contains directly or 
indirectly, except that it doesn't take into account KeyVobs contained 
in another KeyPlane inside the first KeyPlane. Thus we can e.g. use a 
KeyPlane for the windows on a screen, each of which has its own KeyVob, 
and use another KeyPlane in each window for matching the individual RDF 
nodes.

During interpolation, all vobs are rendered where they would be rendered 
outside interpolaion, except KeyVobs and the vobs contained in them. 
(KeyVobs which have no matching equivalent in the target KeyPlane are 
omitted from the interpolation, as usual, but vobs not wrapped in 
KeyVobs are shown.) This means for example that we do not need to assign 
keys to the contents of an RDF node, just to the node as a whole; the 
contents of the node will just "jump" when we switch vob 'scenes' if 
they are not assigned keys, following the rule above. Also, things like 
the 'current view' indicator we've been placing in the upper right 
corner of each window doesn't need a key; if it's not in the target vob 
'scene,' it will simply disappear when we "jump."

This gets us rid of a LOT of keys-- all the infamous 'role keys.' We 
only assign keys to things that actually correspond to objects in the 
underlying model.


Vobs like IrregularFrame would be done through parametrization, like 
connections: IrregularFrame would (in its constructor) get two 
transformations (Transform objects) for the viewport and the paper shown 
through the frame. These transformations would be interpreted relatively 
to the IrregularFrame's parent transformation.

A BuoyManager vob could decorate an existing vob with buoys. This is one 
instance of a general pattern-- frame vobs decorate their content with a 
frame, other vobs decorate their content with connections, yet other 
vobs decorate RDF nodes with the name of the property relating it to the 
focus, and this vob would decorate its content with buoys. Each buoy 
would be specified by giving the anchor, as an index in the decorated 
vob. Of course, each buoy would be a vob itself.

I think this is a rather nice example of how this vob system makes 
things more object-oriented and consistent; a BuoyManager vob like this 
would be easy to use, and it would fit really naturally with the rest of 
the system.

- Benja