Re: [Bug-xorriso] bug with handling symbolic links?

[Joerg Meyer]

Hi Thomas,

> Do you have lots of hard links in your snapshots ?
> (It's only about inodes shared inside the directory
> trees which you map into xorriso. It does not matter
> whether there are other links pointing to those
> inodes from outside those directories.)
Good question - maybe I have misunderstood and/or misinterpreted something here 
as well:
Each snapshot contains on the order of 10E6 files (dominated by gentoo's 
portage tree)
Less than ~10% have changed between the two snapshots included in the image
for which the memory requirements became particularly noticeable.
Identical files have different dev_t  but the same ino_t (physically stored in 
the same location) -
which lead me to conclude that they are subject to hardlinks processing with 
-hardlinks on.
Is that correct?

> Does it really have that much impact ?
> Can you tell me how much memory both runs consume ?

Well, without having recorded accurate profiles, I can only tell you what I 
remember from "manual monitoring":
For the image generation described above, -for_backup consumes >250MB of swap 
(in addition to the 512MB RAM) quite early (i.e. during first -add), 
whereas -acl on -xattr on -md5 only swaps later (i.e. during the subsequent 
-update_r) and much more moderately (~30 MB),
The linux installation is rather slim and only consumes <50MB of RAM (after a 
fresh start).

I might have a closer look when testing your latest rockridge patches...

Best wishes,
Jörg.


On 23 Feb 2015, at 19:24, Thomas Schmitt <address@hidden> wrote:

> Hi,
> 
>> -hardlinks "on" included in -for_backup was what I did not want -
>> after I understood that this increases memory requirements quite noticeably.
> 
> Does it really have that much impact ?
> Can you tell me how much memory both runs consume ?
> (It's often hard to tell true virtual memory consumption
> but real RAM consumption is quite a realistic indicator.)
> 
> Do you have lots of hard links in your snapshots ?
> (It's only about inodes shared inside the directory
> trees which you map into xorriso. It does not matter
> whether there are other links pointing to those
> inodes from outside those directories.)
> 
> 
>> which was exceeded even with -temp_mem_limit 64k for some of the images.
> 
> The impact of -temp_mem_limit is restricted to higher
> levels of xorriso, i fear. Things like pattern expansion
> and file list sorting by block address.
> If it works with setting 64k, then the memory consumption
> is probably inside libisofs.
> 
> 
>> The reason for setting -disk_dev_ino ino_only was for having the speed
>> advantage
> 
> I was quite proud of that method when i developed it
> for scdbackup a few years before i enterd the libburnia
> adventure.
> It has to deal with the counter-intuitive fact that
> file timestamps do not change when the file gets
> renamed or moved to a different directory. Imagine
> two files swapping places.
> (Then imagine my face when i learned about Samba's
> inode dance.)
> 
> 
> Have a nice day :)
> 
> Thomas
>