Re: Resource Management on General-Purpose Systems

[Jonathan S. Shapiro]

On Mon, 2007-07-16 at 15:19 +0400, Anton Tagunov wrote:
> >> I agree with Tanenbaum that if an active program's working set
>  >> does not fit in core and must be swapped, there is a problem.
> 
> shap> if you believe that we will remain crippled by 32-bit address limits
> shap> forever, then Andy is right. But if you believe that 64-bit virtual
> shap> addresses are an inevitable transition (as I do), then I think Andy's
> shap> assertion is questionable.
> 
> Hi! Thought:
> 
>      mmap( 1Tb file ) into 64bit address space
>      use part X of the file
>      tell OS that part X of file is no longer needed
> 
> = application is simple (64bit of address)
> + still responsible for it's working set size

Yes. This is pretty much what I had in mind, except that you also need
"tell OS that I am about to need part Y of file" for load-ahead.

But now consider what happens if you are mutating that stream on the way
through (which is common in audio applications). You are telling the OS
that part X can be removed from memory, but this is not the same as
saying that it is no longer needed.

Now if we want to pursue Andy's proposal further, we will next propose
that this should be handled by a mapping of a temporary file. But note
that at this point we are not removing the paging logic. We are merely
removing the swap area.

Oh. And temporary files have a variety of security concerns under the
UNIX API, so we will soon decide that we want to create and then unlink
the temporary file, retaining only a descriptor to it.

So what we have now accomplished is to remove the swap area and the very
specialized, simple, seek-friendly I/O that it provides, and replace it
with general purposes, complex, seek-unfriendly I/O.
-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC