mldonkey-bugs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Mldonkey-bugs] [bug #985] NetBSD: Too many open files


From: nobody
Subject: [Mldonkey-bugs] [bug #985] NetBSD: Too many open files
Date: Tue, 25 Nov 2003 18:56:21 -0500
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624

=================== BUG #985: LATEST MODIFICATIONS ==================
http://savannah.nongnu.org/bugs/?func=detailbug&bug_id=985&group_id=1409

Changes by: Mark-Oliver Wolter <address@hidden>
Date: Tue 11/25/2003 at 23:56 (GMT)

------------------ Additional Follow-up Comments ----------------------------
Moin,



I have a similar problem on Linux 2.2.24ext3:



mlnet has been running for quite some time:

mow       4481  2.8 24.7 34224 31584 ?       S    Sep28 2418:36 mlnet -daemon

It has even survived several changes in routing without giving any problems, 
but now ...



I get:

socket(PF_UNSPEC, 0 /* SOCK_??? */, 0)  = -1 EMFILE (Too many open files)



Debugging shows: 84 connections on 127.0.0.1 in state CLOSE



Further looking shows:

lsof|grep mlnet|wc

   1030    9972  116758

with lots of

mlnet      4481  mow  123w   REG       3,66     24460     100400 
/samba-shares/audio/mldonkey/files.ini (deleted)



More exactly:

lsof|egrep "mlnet.*deleted"|wc

    705    7050   83593

lsof|egrep "mlnet.*ini"|wc

    658    6580   78094



It seems like when mlnet rewrites its config files, it never closes the old 
ones.



mldonkey is:

mldonkey-2.5-3.static.i686-Linux.tar.bz2



Hope this helps.



=================== BUG #985: FULL BUG SNAPSHOT ===================


Submitted by: None                    Project: mldonkey, a free e-Donkey client
Submitted on: Tue 08/13/2002 at 12:59
Category:  Core                       Severity:  5 - Major                  
Bug Group:  None                      Resolution:  Remind                   
Assigned to:  mldonkey                Status:  Open                         
Release:  None                        Release:                              
Platform Version:  None               Binaries Origin:  None                

Summary:  NetBSD: Too many open files

Original Submission:  Hi,



my logfile is filled with endless repetitions of:



+++ Exception BEFORE CONNECT socket failed: Too many open files

Exception socket failed: Too many open files in client connection



What does it mean - how can I solve this problem?



Thanks,

 Andreas



mldonkey is great! I'd be happy to provide some code for it,

but why the heck did you use ocaml and not something less

exotic? :)

Follow-up Comments
*******************

-------------------------------------------------------
Date: Tue 11/25/2003 at 23:56       By: m-o-w
Moin,



I have a similar problem on Linux 2.2.24ext3:



mlnet has been running for quite some time:

mow       4481  2.8 24.7 34224 31584 ?       S    Sep28 2418:36 mlnet -daemon

It has even survived several changes in routing without giving any problems, 
but now ...



I get:

socket(PF_UNSPEC, 0 /* SOCK_??? */, 0)  = -1 EMFILE (Too many open files)



Debugging shows: 84 connections on 127.0.0.1 in state CLOSE



Further looking shows:

lsof|grep mlnet|wc

   1030    9972  116758

with lots of

mlnet      4481  mow  123w   REG       3,66     24460     100400 
/samba-shares/audio/mldonkey/files.ini (deleted)



More exactly:

lsof|egrep "mlnet.*deleted"|wc

    705    7050   83593

lsof|egrep "mlnet.*ini"|wc

    658    6580   78094



It seems like when mlnet rewrites its config files, it never closes the old 
ones.



mldonkey is:

mldonkey-2.5-3.static.i686-Linux.tar.bz2



Hope this helps.

-------------------------------------------------------
Date: Fri 09/05/2003 at 07:10       By: tk
ulimit -n 1024

before starting mldonkey fixes the problem too.

-------------------------------------------------------
Date: Thu 01/09/2003 at 10:17       By: None
NetBSD has a softlimit of 64 filedescriptors at maximum by d

The hardlimit is 1772. Simply set the softlimit higher with

sysctl -w proc.$$.rlimit.descriptors.soft=1024

before starting mldonkey.



-------------------------------------------------------
Date: Fri 11/08/2002 at 18:15       By: None
Updated to latest CVS and the OpenBSD 3.2 issue is gone, but it seems almost 
impossible to connect to servers now. I will investigate.



-------------------------------------------------------
Date: Fri 11/08/2002 at 17:47       By: None
Same problem on OpeBSD 3.2

-------------------------------------------------------
Date: Sun 09/29/2002 at 11:22       By: None
Ok, I am trying this new cvs (from 28-09 10h (gmt+1)).

But now I have another Problem - mldonkey uses 80 % memory (I have 128 MB). Is 
this normal? Before this version I had ~ 50% - and downloads worked ok (except 
the open file bug :) ).

Because of the high memory use, downloads are extremly slow (0.1 %). The system 
access of course is slow, too. 

Any ideas?

Regards
martin

-------------------------------------------------------
Date: Sat 09/28/2002 at 20:32       By: mldonkey
I have just commited new fixes that might have an impact on this bug.



-------------------------------------------------------
Date: Sat 09/28/2002 at 16:39       By: None
No problem, I have the actual cvs now and this time debug_net in downloads.ini 
works.



So when the bug happens again I will post the logs immediately.



martin

-------------------------------------------------------
Date: Sat 09/28/2002 at 10:34       By: mldonkey
Thanks, the logs helped me a lot. I found at least one bug (incoming connection 
timeouts are not correctly initialized, so that these sockets never get 
disconnected). I commited a fix for that. Could someone try it ?



For the logs, I still need more information: you must set the "debug_net" 
option at the beginning (best would be to start mldonkey, change the option 
with "set" in telnet, save, quit and restart, so that all connections FD appear 
in the logs). Can you try that ? Specify which version you use (ie if you use 
the just commited CVS, or the old beta8 :)









-------------------------------------------------------
Date: Fri 09/27/2002 at 16:36       By: None
Hi,



here is another log with "debug_net" set to "true" in telnet.



And I forgot to say, my ip address did not change (this and last time), when 
this bug happened (I have looked at pppd logs).



Are these logs ok?



Regards

martin

-------------------------------------------------------
Date: Thu 09/26/2002 at 20:26       By: None
Hi,



I have the same problem under linux, too.



I have 2.4.19 Linux on K6-2 350 and 128 MB Ram. Use an cvs version from 
19.09.2002. It is compiled with ocaml 3.06 (make opt).



I followed your instructions and attach the .bz2 (including ls -l, lsof, 
netstat, mldonkey.log, downloads.ini).



I have put the "debug_net = true" in the downloads.ini, but as you can see, it 
is deleted after start. "verbose = true" works ok. And the debug_socks from 
telnet too.



If it is important I can log it again with "debug_net = true" set from telnet. 
Do you need this?



Regards

martin

-------------------------------------------------------
Date: Wed 09/25/2002 at 08:56       By: None
Right, here's the logs with 'debug_net' enabled this time :)



If you need anything else let me know and I'll run things again - I've switched 
off the debugging for now though.



Cheers,

Robin

-------------------------------------------------------
Date: Wed 09/25/2002 at 06:51       By: mldonkey
OK, I will check your logs to try to understand the problem.



-------------------------------------------------------
Date: Tue 09/24/2002 at 08:11       By: None
Okay, I've followed the instructions given and have attached my log, 
downloads.ini, lsof & netstat output, etc.  I'm running it again as I found out 
it's actually the "debug_net" option not "net_debug" option (I set it from 
telnet and this doesn't give any errors with invalid options).



There were no changes on the system while this was running (it's on a static IP 
address and a permanently-on connection).



Anyway, hope this helps and I'll post up the others next time it stops.



Cheers,

Robin

-------------------------------------------------------
Date: Mon 09/23/2002 at 10:25       By: None
I've got this bug under linux, too. System is Debian Woody with a patched 
2.4.18. Those disfuntional sockets (which "strace" says are closed) show up 
under lsof with "can't indendify target". I have attached one log generated by



while true; do

 o=`lsof -n | grep mldonkey | grep iden  | wc -l`

 d=`date`

 t=`lsof -n | grep mldonkey | wc -l`

 echo "$d $o/$t stalled sockets" ; sleep 50

done



I'm on the mldonkey-bugs and -users list for contact



Jens

-------------------------------------------------------
Date: Sun 09/22/2002 at 13:52       By: mldonkey
How long does it take to get the "too many open files" message ? 



Before it happens, set the "net_debug" and "verbose" options to true (with CVS 
version), and keeps a log of mldonkey output (pipe it to a file).

Keep also a telnet session to mldonkey opened, so that you are sure to be able 
to send commands when no fd will be available.



 When it happens:

* In mldonkey telnet, call the "debug_socks" command, verify that mldonkey has 
placed some debug information on sockets in the log.

* In the shell, do a "netstat -nap", check mldonkey pid, go in /proc/<pid>/fd 
(if you have it), and "ls -l", use the "lsof" program on mldonkey (if you have 
it). Take as much information as you can (downloads.ini for example too), put 
them in a tar file, compress it (bzip2 or gzip) and upload it here in a bug 
report. 



Do you have special behavior of your connection between the time you start 
mldonkey and the error (ppp connection reseted, IP changed, etc ?).

-------------------------------------------------------
Date: Sun 09/22/2002 at 13:34       By: None
I've now got two systems running the CVS version, one has this problem and the 
other doesn't.  I've had a look at the setup and the only difference I can see 
is that one is linked against libz (the working one) and the other isn't.



I've now recompiled the broken one with libz-devel installed (but it's also 
with a newer CVS version) so I'll see whether this makes a difference.



If anyone else suffering this problem has their mldonkey executable linked 
against libz, can they say so?  Or can any of the developers suggest whether 
this would/would not lead to these (or any other) problems?



Cheers,

Robin

-------------------------------------------------------
Date: Sun 09/22/2002 at 09:33       By: None
I've got OpenBSD and this error is solved (on my system) by reducing 
max_open_connections to 37 (when i put there 38, error occures).



Please help us because such small number means small downloads (at least for 
me)!



Regards

BusteR

-------------------------------------------------------
Date: Tue 09/17/2002 at 13:09       By: mldonkey
Not enough information to solve this bug. An important bug, though



-------------------------------------------------------
Date: Mon 09/16/2002 at 08:26       By: None
Not just NetBSD, I have this problem on Linux as well:



After running mldonkey for under 12 hours it has 995 file descriptors allocated 
to sockets (from /proc/mldonkey_PID/fd) and only 13 client and 2 server sockets 
in use (from netstat).  There are 27 file descriptors allocated to shared & 
config files (of which 11 are marked deleted). This is with 
max_opened_connections set to only 200, and mainly downloading.

-------------------------------------------------------
Date: Sun 09/15/2002 at 16:13       By: None
I've got the same.. I don't know what's wrong. Since it prints out a handled 
error msg someone of the developers should be able to take a look on it. 
Hopefully ;)

-------------------------------------------------------
Date: Sun 09/15/2002 at 16:11       By: None
I've got the same.. I don't know what's wrong. Since it prints out a handled 
error msg someone of the developers should be able to take a look on it. 
Hopefully ;)

-------------------------------------------------------
Date: Tue 09/10/2002 at 13:19       By: None
I agree with you....

I have compiled mldonkey for OpenBSD-i386 and the same problem happens.

Just try to reduce your maximum connections...

But this will not solve your problem permanently.

-------------------------------------------------------
Date: Tue 09/10/2002 at 13:18       By: None
I agree with you....

I have compiled mldonkey for OpenBSD-i386 and the same problem happens.

Just try to reduce your maximum connections...

But this will not solve your problem permanently.

-------------------------------------------------------
Date: Tue 09/10/2002 at 13:17       By: None
I agree with you....

I have compiled mldonkey for OpenBSD-i386 and the same problem happens.

Just try to reduce your maximum connections...

But this will not solve your problem permanently.


CC list is empty


File Attachments
****************

-------------------------------------------------------
Date: Fri 09/27/2002 at 16:36  Name: ml_openfiles_logs_2.tar.bz2  Size: 105KB   
By: None
Logs from martin (linux) - now with debug_net from telnet
http://savannah.nongnu.org/bugs/download.php?group_id=1409&amp;bug_id=985&amp;bug_file_id=101

-------------------------------------------------------
Date: Thu 09/26/2002 at 20:26  Name: ml_openfiles_logs.tar.bz2  Size: 263KB   
By: None
Logs from martin (linux) - look at the comment, too
http://savannah.nongnu.org/bugs/download.php?group_id=1409&amp;bug_id=985&amp;bug_file_id=100

-------------------------------------------------------
Date: Wed 09/25/2002 at 08:56  Name: mldonkey_debug_logs2.tar.bz2  Size: 128KB  
 By: None
New debug logs and socket info, with debug_net enabled
http://savannah.nongnu.org/bugs/download.php?group_id=1409&amp;bug_id=985&amp;bug_file_id=98

-------------------------------------------------------
Date: Tue 09/24/2002 at 08:11  Name: mldonkey_debug_logs.tar.bz2  Size: 133KB   
By: None
Logs and socket info
http://savannah.nongnu.org/bugs/download.php?group_id=1409&amp;bug_id=985&amp;bug_file_id=96

-------------------------------------------------------
Date: Mon 09/23/2002 at 10:25  Name: stalls.log  Size: 22KB   By: None
Defunct socket statistics for one run
http://savannah.nongnu.org/bugs/download.php?group_id=1409&amp;bug_id=985&amp;bug_file_id=95


For detailed info, follow this link:
http://savannah.nongnu.org/bugs/?func=detailbug&bug_id=985&group_id=1409

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/





reply via email to

[Prev in Thread] Current Thread [Next in Thread]