Re: Question

[Ole Tange]

Hi Michael

Please use parallel@gnu.org for support questions.

I assume the process runs on a Unix-like system (such as GNU/Linux).

When you remove a file, you are only removing the reference to the
file. The file is still stored on disk. So GNU Parallel will continue
to read from that.

Only when GNU Parallel closes the file will the data blocks on the
disk be released as free and the space can be used by other data.

So there should be no need to restart.

You can rescue the file by finding the PID of GNU Parallel. Let us
assume it is 2643624. Then you do:

$ cd /proc/2643624/fd

$ ls -l
total 0
lrwx------ 1 tange tange 64 Dec 21 22:48 0 -> /dev/pts/86
lrwx------ 1 tange tange 64 Dec 21 22:48 1 -> /dev/pts/86
lr-x------ 1 tange tange 64 Dec 21 22:48 10 -> '/tmp/parAC6ri.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 11 -> '/tmp/parf0dSE.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 12 -> '/tmp/parSgRYM.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 13 -> '/tmp/parf0dSE.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 14 -> '/tmp/parSgRYM.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 15 -> '/tmp/paruiGLk.df (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 16 -> '/tmp/parsRqqf.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 17 -> '/tmp/parqVDQN.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 18 -> '/tmp/parsRqqf.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 19 -> '/tmp/parqVDQN.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 2 -> /dev/pts/86
lrwx------ 1 tange tange 64 Dec 21 22:48 20 -> '/tmp/parUTxFR.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 21 -> '/tmp/parBE_eX.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 22 -> '/tmp/parUTxFR.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 23 -> '/tmp/parBE_eX.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 24 -> '/tmp/parp7eoG.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 25 -> anon_inode:inotify
lrwx------ 1 tange tange 64 Dec 21 22:48 26 -> '/tmp/parFlBWg.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 27 -> '/tmp/parp7eoG.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 28 -> '/tmp/parFlBWg.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 29 -> '/tmp/parNgab1.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 3 -> /dev/pts/86
lrwx------ 1 tange tange 64 Dec 21 22:48 30 -> '/tmp/parijr1P.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 31 -> '/tmp/parNgab1.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 32 -> '/tmp/parijr1P.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 33 -> '/tmp/paraXC1L.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 34 -> '/tmp/parO7HcW.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 35 -> '/tmp/paraXC1L.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 36 -> '/tmp/parO7HcW.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 37 -> '/tmp/par3kjYO.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 38 -> '/tmp/parEZjxu.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 39 -> '/tmp/par3kjYO.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 4 -> /dev/pts/86
lr-x------ 1 tange tange 64 Dec 21 22:48 40 -> '/tmp/parEZjxu.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 5 -> /dev/pts/86
lr-x------ 1 tange tange 64 Dec 21 22:48 6 -> '/tmp/a (deleted)'
<<--- THIS IS THE IMPORTANT ONE
lrwx------ 1 tange tange 64 Dec 21 22:48 7 -> '/tmp/par2dFiI.par (deleted)'
lrwx------ 1 tange tange 64 Dec 21 22:48 8 -> '/tmp/parAC6ri.par (deleted)'
lr-x------ 1 tange tange 64 Dec 21 22:48 9 -> '/tmp/par2dFiI.par (deleted)'

You can now do:

$ cat 6 >/tmp/rescued

This will copy the content of the deleted /tmp/a to /tmp/rescued.

/Ole

On Sun, Dec 11, 2022 at 6:28 PM Saint Michael <venefax@gmail.com> wrote:
>
> I am running a process that takes as input a FILE
> parallel --gnu --no-run-if-empty -k --lb -j-5 --eta --colsep ' '
> processx "{1} {#}" :::: ${FILE}
> the file is huge, and it got erased after the process started, but the
> process has not stopped.
> The question is: will it fail later? Will it continue to the end? is
> it cached somewhere?
> The question is vital because the process may take 48 hours and I need
> to know if I should restart it.
>
> Saint Michael
>