I have been trying to tune the webserver workload for some time and a typical log I am getting is as follows:
-------------------------------------------------------------------------------------------------------------------------------------------------
client-192.168.1.8_1010.log prime-client1_1096.log
oaf783-kz-1# more 20161213-175618/client-192.168.1.8_1010.log
2016-12-13 17:56:18:822 Creating specwebclient using RMI Registry port 1010
2016-12-13 17:56:18:849 oaf783-kz-1:1010 ready...
2016-12-13 17:57:17:894 LoadGenerator: Setting sleep delay.
2016-12-13 17:57:24:276 specwebclient: setting tile ID to 0
2016-12-13 17:57:24:281 Looking up polling host: webserver:8001
2016-12-13 17:57:25:385 LoadGenerator: Ramping up for 150 seconds.
2016-12-13 17:57:25:385 LoadGenerator: Starting 2500 threads.
2016-12-13 17:59:55:385 LoadGenerator: All threads running, test starting
2016-12-13 18:01:51:016 SPECweb_Support: [ERROR] Expected 10 product categories, 0 found.
2016-12-13 18:01:51:016 SPECweb_Support: [ERROR] STATE 0; Expected 51700 bytes appended to response but found 3524287
2016-12-13 18:01:51:016 SPECweb_Support: [ERROR] STATE 0; RESPONSE INVALID on GET /support/index.php HTTP/1.1
2016-12-13 18:03:27:452 SPECweb_Support: [ERROR] Expected 10 product categories, 0 found.
2016-12-13 18:03:27:452 SPECweb_Support: [ERROR] STATE 0; Expected 51700 bytes appended to response but found 314574
2016-12-13 18:03:27:452 SPECweb_Support: [ERROR] STATE 0; RESPONSE INVALID on GET /support/index.php HTTP/1.1
2016-12-13 18:04:15:606 SPECweb_Support: [ERROR] Expected 60 optons, 0 found.
2016-12-13 18:04:15:606 SPECweb_Support: [ERROR] STATE 3; Expected 52700 bytes appended to response but found 18000
2016-12-13 18:04:15:606 SPECweb_Support: [ERROR] STATE 3; RESPONSE INVALID on GET /support/product.php?id=3054 HTTP/1.1
2016-12-13 18:04:52:545 SPECweb_Support: [ERROR] Expected 60 optons, 0 found.
2016-12-13 18:04:52:545 SPECweb_Support: [ERROR] STATE 3; Expected 52700 bytes appended to response but found 1510
2016-12-13 18:04:52:545 SPECweb_Support: [ERROR] STATE 3; RESPONSE INVALID on GET /support/product.php?id=2102 HTTP/1.1
2016-12-13 18:04:55:397 WorkloadScheduler: Clearing statistics...
2016-12-13 18:04:55:402 WorkloadScheduler: Clearing statistics...
2016-12-13 18:07:09:630 SPECweb_Support: [ERROR] Expected 60 optons, 0 found.
2016-12-13 18:07:09:630 SPECweb_Support: [ERROR] STATE 3; Expected 52700 bytes appended to response but found 80
2016-12-13 18:07:09:630 SPECweb_Support: [ERROR] STATE 3; RESPONSE INVALID on GET /support/product.php?id=1667 HTTP/1.1
2016-12-13 18:11:03:846 SPECweb_Support: [ERROR] STATE 5; RESPONSE DYNAMIC CONTENT NOT VALIDATED! EXPECTED 5188365, GOT -1
2016-12-13 18:11:03:846 SPECweb_Support: [ERROR] STATE 5; Expected 113300 bytes appended to response but found 250
2016-12-13 18:11:03:846 SPECweb_Support: [ERROR] STATE 5; RESPONSE INVALID on GET /support/file.php?id=8145 HTTP/1.1
2016-12-13 18:11:36:540 SPECweb_Support: [ERROR] Expected 60 optons, 0 found.
2016-12-13 18:11:36:541 SPECweb_Support: [ERROR] STATE 3; Expected 52700 bytes appended to response but found 110
2016-12-13 18:11:36:541 SPECweb_Support: [ERROR] STATE 3; RESPONSE INVALID on GET /support/product.php?id=1855 HTTP/1.1
2016-12-13 18:21:04:623 SPECweb_Support: [ERROR] Expected 60 optons, 0 found.
2016-12-13 18:21:04:623 SPECweb_Support: [ERROR] STATE 3; Expected 52700 bytes appended to response but found 27500
2016-12-13 18:21:04:623 SPECweb_Support: [ERROR] STATE 3; RESPONSE INVALID on GET /support/product.php?id=2282 HTTP/1.1
2016-12-13 18:24:55:414 LoadGenerator: Threads stopping.
2016-12-13 18:24:55:415 LoadGenerator: Ramping down for 180 seconds.
2016-12-13 18:24:58:882 MTFF[45]: [ERROR] Incorrect file size returned. Expected: 110; received: 0
2016-12-13 18:27:56:368 LoadGenerator: Finished ramping down.
2016-12-13 18:27:56:380 LoadGenerator: Threads finished.
2016-12-13 18:27:56:380 LoadGenerator: Collecting run statistics.
2016-12-13 18:27:57:381 catalog requests: 24963
2016-12-13 18:27:57:381 download requests: 14574
2016-12-13 18:27:57:381 file requests: 28923
2016-12-13 18:27:57:381 fileCatalog requests: 48109
2016-12-13 18:27:57:381 home requests: 17390
2016-12-13 18:27:57:381 product requests: 52794
2016-12-13 18:27:57:381 search requests: 26958
2016-12-13 18:27:57:381 Total requests: 213711, successful requests: 213707
2016-12-13 18:27:57:981 Threads finished...
2016-12-13 18:27:57:982 LoadGenerator: Returning statistics.
That is a shortened poll time of only 20 minutes.
The summary results for that particular run are as follows:
Performance
Tile # Pct Load Web Server Per-Tile
Score Overall
Score
1 100% 178.09 98.95 98.95
Quality Of Service (QOS)
Tile # Pct Load Web Server Per-Tile
Score Overall
Score
1 100% 1.00 1.00 100.00%
Web Server
Web Wkload Req/sec Good Tolerable Fail Valid. Errors
Support 178.09 213704 3 0 4
I have managed to get up to 179 Req/sec but have not eliminated the time out errors.
The webserver is showing the errors in it's log also:
------------------------------------------------------------------------
[Tue Dec 13 18:11:37.397220 2016] [:error] [pid 4053] [client 192.168.1.8:47534] PHP Warning: fopen(
http://infraserver:81/fcgi-bin/besim_fcgi.fcgi?3&5): failed to open stream: Connection refused in /var/www/html/support/inc.common.php on line 56
[Tue Dec 13 18:11:37.397276 2016] [:error] [pid 4053] [client 192.168.1.8:47534] PHP Fatal error: Call to undefined function report() in /var/www/html/support/inc.common.php on line 57
[Tue Dec 13 18:21:05.479760 2016] [:error] [pid 3784] [client 192.168.1.8:45195] PHP Warning: fopen(
http://infraserver:81/fcgi-bin/besim_fcgi.fcgi?3&4&2282): failed to open stream: Connection refused in /var/www/html/support/inc.common.php on line 56
[Tue Dec 13 18:21:05.479841 2016] [:error] [pid 3784] [client 192.168.1.8:45195] PHP Fatal error: Call to undefined function report() in /var/www/html/support/inc.common.php on line 57
webserver1#
My ulimits seem fine, have already upped them from the standard.
I tried upping the vcpus slightly for both the prime client and webserver guest, which didn't have a significant effect.
I seem to have 50 - 60% idle time for cpu throughout the run while I have 24G of virtual memory for both guests and that does not appear to be a bottle neck either.
My webserver is apache2.4 with php 5.6. and I am using the prefork mpm on apache with the following tunings:
webserver
----------------
Timeout 20
KeepAlive on
MaxKeepAliveRequests 0
KeepAliveTimeout 60
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 10240
MaxRequestWorkers 10240
MaxConnectionsPerChild 4000
</IfModule>
infraserver
----------------
Timeout 20
KeepAlive on
MaxKeepAliveRequests 100
KeepAliveTimeout 60
<IfModule prefork.c>
StartServers 200
MinSpareServers 20
MaxSpareServers 50
ServerLimit 8000
MaxRequestWorkers 8000
MaxConnectionsPerChild 32000
</IfModule>
my php settings are:
---------------------------------
webserver1# more /etc/php/5.6/conf.d/extensions.ini
;
; Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved.
;
; To enable PHP extensions, uncomment the desired line(s) below, or add similar
; lines to another active PHP configuration (.ini) file.
;extension=apcu.so
;extension=bcmath.so
extension=bz2.so
;extension=calendar.so
extension=curl.so
;extension=dba.so
;extension=enchant.so
;extension=exif.so
extension=ftp.so
extension=gd.so
extension=gettext.so
;extension=gmp.so
;extension=intl.so
;extension=ldap.so
;extension=mbstring.so
;extension=mysql.so
extension=mysqli.so
;extension=opcache.so
extension=openssl.so
;extension=pcntl.so
extension=pdo_mysql.so
;extension=readline.so
;extension=shmop.so
extension=snmp.so
;extension=soap.so
;extension=sockets.so
;extension=suhosin.so
;extension=sysvmsg.so
;extension=sysvsem.so
;extension=sysvshm.so
extension=tidy.so
;extension=wddx.so
;extension=xmlrpc.so
;extension=xsl.so
;extension=zip.so
I recently moved from apache22 with php-5.3 (imposed my newer OS build not by choice) and some of the extensions are a little diffrent, namely apcu in 5.6 instead of 5.3 and also suhosin.
I have tried a number of runs with different tunings for those extensions with little or no effect on the end result.
I suppose my main question is wether or not these time outs are a deal breaker since the QOS is still being returned as 100% ?
I would prefer to get rid of them if at all possible but my tuning attempts are increasingly seeming like shots in the dark rather than informed experiments so if anyone has input or advice on that side I would greatly appreciate it.
many thanks,
Andrew
