Re: Resource Revocation

[Neal H. Walfield]

At Thu, 2 Aug 2007 20:22:00 +0200,
<address@hidden> wrote:
> On Fri, Jul 13, 2007 at 05:16:08PM +0200, Neal H. Walfield wrote:
> > Just as we have activities assign preferences to composite activities,
> > we can provide a mechanism to allow activities to assign preferences
> > to pages.  We can also assign policies to pages. For instance, instead
> > of sending a page to backing store, the application may mark a page as
> > discardable.  If such a page is chosen for eviction, it should simply
> > be freed.  (In this case, a fault needs to be triggered on the next
> > access.)  This strategy removes the major source of destructive
> > interference: the scheduler no longer waits on the task; the task acts
> > beforehand on its own initiative or it reacts to a signal (not a
> > request).
>
> When I first saw some mention of market-based resource management (in
> the context of Hurd/L4), there were very few details, and I thought up
> some stuff on my own. My idea was that every task assigns priorities to
> all it's physical pages. (In the market model, the maximum rent it's
> willing to pay.) Whenever there is memery pressure in the system, the
> rent goes up. A central pager then checks all tasks for pages with a
> priority below the current rent, and evicts these.

> Now you are proposing a design which -- unless I misunderstood your
> description -- seems pretty much the same as my original idea. Could you
> confirm whether this is indeed the case? And if so, how do you want to
> implement it, to avoid too much management overhead?

Yes, what you propose is quite similar.  Instead of rent, I have
formulated the mechanism as that of creating a ranking.  As for
overhead, my goal is to let performance measurements direct me.  I
need a number of shared memory solutions with different trade-offs.


> Last year you proposed a design where a global pager would evict pages
> when the number of pages available to a task is reduced due to memory
> pressure; but otherwise tasks have full control over paging by deciding
> what happens on a page fault. This looked like a quite promising
> compromise: Avoiding the problems of explicit revocation, but still
> leaving very much flexibility.

I don't remember this proposition and the details are unfortunately
not clear enough that I completely understand how this mechanism is
supposed to function.

My current thought is that the resource manager keep tasks informed
about memory pressure.  This is required for adaptation.  Eviction is
one form of adaptation that tasks can employ.


> > This solution is clearly more limiting: the appropriate strategies
> > must be identified beforehand.  However, not that many common
> > strategies come to mind.  In fact, I suspect that either sending to
> > backing store or simply freeing cover the most common scenarios.
> 
> Well, I can think only of one other approach: Some applications might
> work on large data sets, which can't be discarded because they are not
> easy to regenerate, but can be converted to some more compact
> representation temporarily at a low cost. This compacting is obviosly
> application-specific, so it can't be handled by a generic mechanism.
> 
> However, this variant is pretty academic. It seems rather unlikely that
> there are actually cases where an application would profit from such
> compacting on memory pressure, but on the other hand it would be too
> costly to just always do it. And even if such cases indeed exist, I very
> much doubt anyone would bother to relly implement this...
>
> What I'm saying is that I agree that swapping or discarding are probably
> the only useful actions. I'm not sure how much a global paging mechanism
> is limiting possible policies, though...

Yeah, that's the hypothesis I also reached.  It led to the following
approach: design for the problems that you know about, and certainly
not the ones that you can't imagine.


Thanks,
Neal