TCP Tuning

Continuing on my testing venture, I learned a few more cool Linux operating system stuff.

My problems were little unique. My single request takes around 6 millisecond to get processed, so if I fire 4000 concurrent requests for 5million total number of requests, operating system gives up. i.e. I notice all socket and TCP IP related errors. i.e. netstat shows me 30k approx connections in time_wait.
Socket time out, could not assign request address, connection reset by peer, reached maximum limit for open files etc. errors starts popping out.

I tweaked open file and socket max connection limit to get over my initial hump. I was being able to hit 5000 approx concurrent request and started noticing similar kind issues again. Then a colleague recommended to tune some of tcpip receive and send buffer sizes. I read that with latest kernel the auto tuning option comes out of the box and in most of cases you are not required to change those settings. However the maximum buffer size are still too small for many high speed network path and must be increased for each OS.

But since it was my local box I decided to play around with them to see the difference. I modified the /etc/sysctl.conf and added four lines for updating the max and default values for receive and send buffer.

net.core.rmem_max = 10485760

net.core.wmem_max = 10485760

net.ipv4.tcp_rmem = 4096 87380 10485760

net.ipv4.tcp_wmem = 4096 16384 10485760

net.ipv4.tcp_mem = 196608 262144 10485760

It did help in our project.

Thanks
Manisha