As Patrick said that's not a high enough rate of ARPs to cause any troubles, there are in total 11 in 16 seconds there? If that overwhelms your system you have other issues...
I will eat my socks if your problem isn't caused by the two fast SYNs in 111 and 112. How sure are you that your driver isn't dropping packets at high load? Is everything fine after 131 or what happens next?
Based on your screenshot it seems to me extremely unlikely that the issue is caused by something in your customer network, if I were you I'd try to reproduce and debug.
//Jens
On 2020-06-23 12:53, Trampas Stern
wrote:
The maximum number of TCP connections is 20, below is the configuration. Note this is using http not https.
What I am seeing in the code is the ARP packets do allocate memory for the packet, hence I could be running out of memory.
Either way it looks like I am not processing packets fast enough or losing them.
/**
* \file
*
* \brief LwIP configuration.
*
* Copyright (c) 2013-2018 Microchip Technology Inc. and its subsidiaries.
*
* \asf_license_start
*
* \page License
*
* Subject to your compliance with these terms, you may use Microchip
* software and any derivatives exclusively with Microchip products.
* It is your responsibility to comply with third party license terms applicable
* to your use of third party software (including open source software) that
* may accompany Microchip software.
*
* THIS SOFTWARE IS SUPPLIED BY MICROCHIP "AS IS". NO WARRANTIES,
* WHETHER EXPRESS, IMPLIED OR STATUTORY, APPLY TO THIS SOFTWARE,
* INCLUDING ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY,
* AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL MICROCHIP BE
* LIABLE FOR ANY INDIRECT, SPECIAL, PUNITIVE, INCIDENTAL OR CONSEQUENTIAL
* LOSS, DAMAGE, COST OR EXPENSE OF ANY KIND WHATSOEVER RELATED TO THE
* SOFTWARE, HOWEVER CAUSED, EVEN IF MICROCHIP HAS BEEN ADVISED OF THE
* POSSIBILITY OR THE DAMAGES ARE FORESEEABLE. TO THE FULLEST EXTENT
* ALLOWED BY LAW, MICROCHIP'S TOTAL LIABILITY ON ALL CLAIMS IN ANY WAY
* RELATED TO THIS SOFTWARE WILL NOT EXCEED THE AMOUNT OF FEES, IF ANY,
* THAT YOU HAVE PAID DIRECTLY TO MICROCHIP FOR THIS SOFTWARE.
*
* \asf_license_stop
*
*/
#ifndef __LWIPOPTS_H__
#define __LWIPOPTS_H__
/* Include ethernet configuration first */
#include "conf_eth.h"
#include "board.h"
/*
-----------------------------------------------
-------------- LwIP API Support ---------------
-----------------------------------------------
*/
#define SNMP_LWIP_ENTERPRISE_OID TRIMM_ENTERPRISE_OID
#define LWIP_HTTPD_DYNAMIC_FILE_READ 1
#define HTTPD_ENABLE_HTTPS 1
#define LWIP_ALTCP_TLS 1
#define LWIP_ALTCP_TLS_MBEDTLS 1
#define LWIP_ALTCP 1
#define LWIP_HTTPD_SUPPORT_POST 1
#define HTTPD_MAX_RETRIES 10 //tbs 3-19-2020 increased from 4 to prevent time out on bad connections
#define PPP_SUPPORT 0
#define PPPOE_SUPPORT 0
#define SNMP_LWIP_MIB2 1
#define LWIP_SNMP 1
#define LWIP_SNMP_V3 1
#define MIB2_STATS 1
#define SNMP_USE_RAW 1
//#define HTTPD_DEBUG LWIP_DBG_ON
#define ALTCP_MBEDTLS_DEBUG LWIP_DBG_ON
//#define TCP_OUTPUT_DEBUG LWIP_DBG_ON | LWIP_DBG_LEVEL_SEVERE
//#define DHCP_DEBUG LWIP_DBG_ON
/**
* NO_SYS==1: Provides VERY minimal functionality. Otherwise,
* use lwIP facilities.
* Uses Raw API only.
*/
#define NO_SYS 1
/**
* LWIP_NETIF_STATUS_CALLBACK==1: Support a callback function whenever an interface
* changes its up/down status (i.e., due to DHCP IP acquistion)
*/
#define LWIP_NETIF_STATUS_CALLBACK 1
/**
* LWIP_RAW==1: Enable application layer to hook into the IP layer itself.
* Used to implement custom transport protocol (!= than Raw API).
*/
#define LWIP_RAW 0
/**
* SYS_LIGHTWEIGHT_PROT==1: if you want inter-task protection for certain
* critical regions during buffer allocation, deallocation and memory
* allocation and deallocation.
*/
#define SYS_LIGHTWEIGHT_PROT 0
/* These are not available when using "NO_SYS" */
#define LWIP_NETCONN 0
#define LWIP_SOCKET 0
/* Uncomment following line to use DHCP instead of fixed IP */
#define DHCP_USED
/*
------------------------------------
---------- Memory options ----------
------------------------------------
*/
/**
* MEM_ALIGNMENT: should be set to the alignment of the CPU
* 4 byte alignment -> #define MEM_ALIGNMENT 4
* 2 byte alignment -> #define MEM_ALIGNMENT 2
*/
#define MEM_ALIGNMENT 4
/**
* MEM_SIZE: the size of the heap memory. If the application will send
* a lot of data that needs to be copied, this should be set high.
*/
#define MEM_SIZE 27 * 1024
/**
* MEMP_NUM_UDP_PCB: the number of UDP protocol control blocks. One
* per active UDP "connection".
* (requires the LWIP_UDP option)
*/
#define MEMP_NUM_UDP_PCB 2
/**
* MEMP_NUM_TCP_PCB: the number of simulatenously active TCP connections.
* (requires the LWIP_TCP option)
*/
#define MEMP_NUM_TCP_PCB 20
/**
* MEMP_NUM_TCP_PCB_LISTEN: the number of listening TCP connections.
* (requires the LWIP_TCP option)
*/
#define MEMP_NUM_TCP_PCB_LISTEN 6
/**
* MEMP_NUM_TCP_SEG: the number of simultaneously queued TCP segments.
* (requires the LWIP_TCP option)
*/
#define MEMP_NUM_TCP_SEG 20
/**
* MEMP_NUM_REASSDATA: the number of IP packets simultaneously queued for
* reassembly (whole packets, not fragments!)
*/
#define MEMP_NUM_REASSDATA 4
/**
* MEMP_NUM_FRAG_PBUF: the number of IP fragments simultaneously sent
* (fragments, not whole packets!).
* This is only used with IP_FRAG_USES_STATIC_BUF==0 and
* LWIP_NETIF_TX_SINGLE_PBUF==0 and only has to be > 1 with DMA-enabled MACs
* where the packet is not yet sent when netif->output returns.
*/
#define MEMP_NUM_FRAG_PBUF 6
/**
* MEMP_NUM_PBUF: the number of memp struct pbufs (used for PBUF_ROM and PBUF_REF).
* If the application sends a lot of data out of ROM (or other static memory),
* this should be set high.
*/
#define MEMP_NUM_PBUF 10
/**
* MEMP_NUM_NETBUF: the number of struct netbufs.
* (only needed if you use the sequential API, like api_lib.c)
*/
#define MEMP_NUM_NETBUF 0
/**
* MEMP_NUM_NETCONN: the number of struct netconns.
* (only needed if you use the sequential API, like api_lib.c)
*/
#define MEMP_NUM_NETCONN 0
/**
* PBUF_POOL_SIZE: the number of buffers in the pbuf pool.
*/
#define PBUF_POOL_SIZE 50
/**
* PBUF_POOL_BUFSIZE: the size of each pbuf in the pbuf pool.
*/
#define PBUF_POOL_BUFSIZE GMAC_FRAME_LENTGH_MAX
/*
----------------------------------
---------- DHCP options ----------
----------------------------------
*/
#if defined(DHCP_USED)
/**
* LWIP_DHCP==1: Enable DHCP module.
*/
#define LWIP_DHCP 1
#endif
/*
---------------------------------
---------- UDP options ----------
---------------------------------
*/
/**
* LWIP_UDP==1: Turn on UDP.
*/
#define LWIP_UDP 1
/*
---------------------------------
---------- TCP options ----------
---------------------------------
*/
/**
* LWIP_TCP==1: Turn on TCP.
*/
#define LWIP_TCP 1
/**
* TCP_MSS: The maximum segment size controls the maximum amount of
* payload bytes per packet. For maximum throughput, set this as
* high as possible for your network (i.e. 1460 bytes for standard
* ethernet).
* For the receive side, this MSS is advertised to the remote side
* when opening a connection. For the transmit size, this MSS sets
* an upper limit on the MSS advertised by the remote host.
*/
#define TCP_MSS (1460)
/**
* TCP_WND: The size of a TCP window. This must be at least
* (2 * TCP_MSS) for things to work well
*/
#define TCP_WND (32 * TCP_MSS) //should be more than 16k for TLS/HTTPS
/**
* TCP_SND_BUF: TCP sender buffer space (bytes).
* To achieve good performance, this should be at least 2 * TCP_MSS.
*/
#define TCP_SND_BUF (4 * TCP_MSS)
/*
------------------------------------
---------- Thread options ----------
------------------------------------
*/
/** The stack sizes allocated to the netif stack: (256 * 4) = 1048 bytes. */
#define netifINTERFACE_TASK_STACK_SIZE 256
/** The priority of the netif stack. */
#define netifINTERFACE_TASK_PRIORITY (tskIDLE_PRIORITY + 4)
/** The stack sizes allocated to the TCPIP stack: (256 * 4) = 1048 bytes. */
#define TCPIP_THREAD_STACKSIZE 256
/** The priority of the TCPIP stack. */
#define TCPIP_THREAD_PRIO (tskIDLE_PRIORITY + 5)
/** The mailbox size for the tcpip thread messages */
#define TCPIP_MBOX_SIZE 16
#define DEFAULT_ACCEPTMBOX_SIZE 16
#define DEFAULT_RAW_RECVMBOX_SIZE 16
#define DEFAULT_TCP_RECVMBOX_SIZE 16
/*
----------------------------------------
---------- Statistics options ----------
----------------------------------------
*/
/**
* LWIP_STATS==1: Enable statistics collection in lwip_stats.
*/
#define LWIP_STATS 1
/**
* LWIP_STATS_DISPLAY==1: Compile in the statistics output functions.
*/
#define LWIP_STATS_DISPLAY 0
/**
* LWIP_STATS_LARGE==1: Use 32 bits counter instead of 16.
*/
#define LWIP_STATS_LARGE 0
#if LWIP_STATS
#define LINK_STATS 0
#define IP_STATS 0
#define IPFRAG_STATS 0
#define ICMP_STATS 0
#define IGMP_STATS 0
#define UDP_STATS 0
#define TCP_STATS 0
#define MEM_STATS 0
#define MEMP_STATS 0
#define SYS_STATS 0
#endif
/* Left outside to avoid warning. */
#define ETHARP_STATS 0
/*
---------------------------------------
---------- Debugging options ----------
---------------------------------------
*/
//#define LWIP_NOASSERT
#define LWIP_DEBUG
#define LWIP_DBG_MIN_LEVEL LWIP_DBG_LEVEL_WARNING
#define LWIP_DBG_TYPES_ON LWIP_DBG_ON
// \note For a list of all possible lwIP configurations, check http://lwip.wikia.com/wiki/Lwipopts.h
#endif /* __LWIPOPTS_H__ */
On Mon, Jun 22, 2020 at 9:43 PM Patrick Klos <patrick@klos.com> wrote:
On 6/22/2020 2:31 PM, Trampas Stern wrote:
I got a wireshark dump from customer and it looks like there is a lot of ARP messages on their network.
Does the ARP broadcast consume a connection?
No. ARPs do not consume a [TCP] connection.
In the image below our device is 10.2.65.250 and it is looking like we are getting duplicate requests (line 116,129) from the client.
It looks like packet 116 is a resend of packet 115, and then packet 117 is the ACK for packet 116.
Packet 129 is a resend of packet 112, and then packet 130 is the SYN-ACK for that [second] connection.
In both cases, your device seems to be losing a packet and recovering after the resend?
How many simultaneous connections does your device support?
The network has lots of ARP requests...
It appears that a device is asking for 10.2.65.1, which might be a mistake as the gateway is 10.2.64.1 with netmask of 255.255.254.0
Do you have any idea what devices have the "MRVCommu" OUI in their MAC addresses? Maybe those devices are misconfigured?
I am wondering if the ARP traffic is overwhelming lwip...
I haven't had a chance to review the ARP code to see if getting overwhelmed would affect the TCP connections? Regardless, there doesn't appear to be a high enough rate of ARPs to be troublesome. Many of them are 1 second or more apart.
Please share any other clues and maybe we can find something...
Patrick
_______________________________________________ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
_______________________________________________ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users