【SPECweb error】SPECweb_Support: [ERROR] SocketTimeoutException encountered durin

[shawn]

When I run 12 tiles，Its all right，But when I run 16 tiles，The following error occurs：

Messages from: 172.18.223.238:1010
-> 2019-08-08 20:54:33:961 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:33:962 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:33:991 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:33:991 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:018 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:019 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:106 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:107 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:204 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:205 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:214 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:214 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:253 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:253 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:310 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:310 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:356 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:356 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-08 20:54:34:497 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-08 20:54:34:498 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!

I have made the following modifications according to the instructions
https://www.spec.org/virt_sc2013/docs/SPECvirt_TechnicalSupport.html

A. Check the open file limit on the clients:
ssh client2 "ulimit -n"
If you get a number such as 1024, then the ulimit has not been set. Check the /etc/sysctl.conf file to make sure that you have a line similar to:
fs.file-max = 500000
Then check /etc/bashrc to make sure the following is included:
# Overide open files limit of 1024
ulimit -n 100000 > /dev/null 2>&1
Then reboot the affected client(s) and rerun the test.


So what can I do ？Please help me ，tks！

[lroderic]

What happens when you run 13 tiles? Or 14 tiles? Or 15 tiles?

[shawn]

Thanks for reply！

I'm trying to execute 13&14 tiles，I get the same result as 16 tiles:


13 &14 tiles also occur  the same error as following:
-> 2019-08-10 10:56:55:398 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:55:398 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:55:435 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:55:435 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:55:667 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:55:667 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:56:178 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:56:178 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:56:190 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:56:190 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:56:330 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:56:331 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2019-08-10 10:56:56:554 Connection: SocketTimeoutException waiting for end-of-header
-> 2019-08-10 10:56:56:554 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!


But 13&14 tiles can execute successfully and generate results，The 16 tile will directly to terminate and can not generate the specvirt-XXX-perf.html

[ChrisFloyd]

Can you provide details on your network infrastructure?  (E.g., are you using 10Gb network?). Each tile will require ~1Gbit throughput on the SUT.  The client VM (the workload driver) will also require ~1Gb network throughput per Tile. 

[shawn]

I sorry to replay too late！
I use 3 10gb cards。
The  configuration is as folowing in the attachment

[ChrisFloyd]

Shawn,

Do you have 10Gb infrastructure on both the client hosts and SUT host?  Given that you are able to run 12 Tiles, but not 16, this sounds like you are running into a resource limitation (CPU, network, IO, etc) on either your clients or SUT system.  I would recommend taking a close look at your resource utilizations and IO latencies on client and SUT system to debug this further. Note, we can provide assistance on the initial setup and configuration, but the complexity of systems at 16 Tiles is likely more than we are able to assist with.

[shawn]

Yes,I have 10Gb infrastructure on both the client hosts and SUT host

OK,I guess its  resource allocation

Thanks，I will check it