SPEC Seal of Reviewal SPECmail2001 Result
Copyright © 1999-2001 Standard Performance Evaluation Corporation
Intrastore 2001 - Redhat Linux 7.2 SPECmail2001 msgs/min = 1,000
SPEC license # 2018 Tested by:
Syntegra
Test date:
Nov-2001
Hardware Avail:
Aug-2001
Software Avail:
Aug-2001

1,000 SPECmail2001 messages per minute is equivalent to 200,000 SPECmail2001_users.

Detail Summary | General Notes
System configurations: SMTP Server + POP Server + LDAP Server | Mail Sink + Benchmark Manager + Load Generator | Configuration diagram
Detailed results: Detailed Results 80% load | Detailed Results 100% load | Detailed Results 120% load


Summary Results
Function Response
time limit
(seconds)
Required
Percentage
Compliant
Percentage Compliant
80% 100% 120%
SMTP Connect 5 >95% 100.00% 100.00% 100.00%
SMTP Data 5 >95% 99.75% 99.80% 99.74%
POP Connect 5 >95% 100.00% 100.00% 100.00%
POP Status 5 >95% 100.00% 100.00% 48.29%
POP Retrieve 5 >95% 99.97% 99.97% 44.75%
POP Delete 5 >95% 99.99% 99.94% 56.11%
Delivery Time 60 >95% 99.55% 98.42% 99.16%
Error Rate NA <1% 0.00% 0.00% 0.00%

SMTP Server + POP Server + LDAP Server (1 system)
Software
Software: Intrastore 2001 Operating System: Redhat Linux 7.2 (kernel 2.4.16)
Availability: Feb-2001 File System: ext3
Hardware
Vendor: Compaq CPU: Pentium III
Model Name: DL360 CPU MHz: 1000
Availability: Jun-2001 CPUs Enabled: 1 core, 1 chip, 1 core/chip
Disk Subsystem: 2 x 18GB SCSI Primary Cache: 16KbI + 16KbD
Memory (MB): 1024 Secondary Cache: 256KB
Network: Compaq NC3163 10/100 (2) Other Cache: N/A
# of Systems: 1 Other: Disks in RAID 1 config
Notes / Tuning Information
Second network card has dedicated route to NetApp

RAID-1 provided by integrated Compaq SmartArray controller

MSL is 30 seconds
TIMEWAIT is 60 seconds

SMTPserver options: -Ca -rt 900 -Cr 200 -CD -na -I SMTP-out -D syntegra.com -Sc 1000 -S
imsd options: -Vd -L /var/adm/imsd.log -tpool 50 -tmax 125

Mail Sink + Benchmark Manager + Load Generator (2 systems)
Software
JVM: Sun HotSpot 1.4.0-beta-b65 (no JIT) Operating System: Redhat Linux 7.1 (kernel 2.4.10)
Availability: Aug-2001 File System: ext2
Hardware
Vendor: Gateway CPU: Pentium III
Model Name: 6400 CPU MHz: 933
Availability: Aug-2001 CPUs Enabled: 1 core, 1 chip, 1 core/chip
Disk Subsystem: 1 x 9GB SCSI Primary Cache: 16KbI + 16KbD
Memory (MB): 640 Secondary Cache: 256Kb
Network: Intel 10/100 Other Cache: N/A
# of Systems: 2 Other: N/A
Notes / Tuning Information

   


Detailed Results -- 80% Load
Load User Count Message Rate
(msg/min)
Attempted 160,000 800
Actual 160,200 801
Function Attempts # Pass
QoS
% Pass
QoS
SMTP Connect 91,511 91,511 100.00%
SMTP Hello 91,511 91,511 100.00%
SMTP Mail From 91,511 91,511 100.00%
SMTP Rcpt To 183,145 183,145 100.00%
SMTP Data 91,511 91,282 99.75%
SMTP Quit 90,556 90,556 100.00%
POP Connect 95,873 95,873 100.00%
POP User ID 95,872 95,721 99.84%
POP Password 95,872 95,780 99.90%
POP Status 95,872 95,872 100.00%
POP Retrieve 95,602 95,575 99.97%
POP Delete 95,348 95,338 99.99%
POP Quit 95,630 95,630 100.00%
Delivery Time 891 887 99.55%
SMTP Data Response Time 91,511 90,383 98.77%
POP Retrieve Response Time 95,602 95,473 99.87%
Remote Message Delivery 87,643 86,660 98.88%

Counter

Value
SMTP Connect Error 0
SMTP Protocol Error 0
SMTP Disconnect 956
SMTP Session 90,556
SMTP To Local 95,503
SMTP To Remote 87,643
SINK Connect Error 0
POP Connect Error 2
POP Protocol Error 1
POP Disconnect 252
POP Session 95,632
Delivery Time Connect Error 0
Function Minimum Time
(millisec)
Maximum Time
(millisec)
Average Time
(millisec)
SMTP Connect 0 759 14.49
SMTP Hello 0 829 43.74
SMTP Mail From 0 410 25.76
SMTP Rcpt To 0 463 26.03
SMTP Data 0 15,581 149.41
SMTP Quit 0 398 26.14
POP Connect 0 1,530 25.20
POP User ID 0 8,222 97.77
POP Password 2 10,770 102.32
POP Status 0 2,895 57.17
POP Retrieve 2 11,510 86.59
POP Delete 0 7,400 51.54
POP Quit 0 3,430 53.63
Function Minimum Size
(KB)
Maximum Size
(KB)
Average Size
(KB)
SMTP Data (KB up) 1 2,674 24.89
POP Retrieve (KB down) 1 2,675 23.77


Detailed Results -- 100% Load
Load User Count Message Rate
(msg/min)
Attempted 200,000 1,000
Actual 199,200 996
Function Attempts # Pass
QoS
% Pass
QoS
SMTP Connect 113,348 113,348 100.00%
SMTP Hello 113,349 113,349 100.00%
SMTP Mail From 113,350 113,350 100.00%
SMTP Rcpt To 226,266 226,266 100.00%
SMTP Data 113,344 113,118 99.80%
SMTP Quit 112,285 112,285 100.00%
POP Connect 119,732 119,732 100.00%
POP User ID 119,733 119,340 99.67%
POP Password 119,730 119,709 99.98%
POP Status 119,728 119,728 100.00%
POP Retrieve 121,392 121,357 99.97%
POP Delete 121,044 120,975 99.94%
POP Quit 119,354 119,345 99.99%
Delivery Time 1,076 1,059 98.42%
SMTP Data Response Time 113,344 112,117 98.92%
POP Retrieve Response Time 121,392 121,204 99.85%
Remote Message Delivery 106,717 105,664 99.01%

Counter

Value
SMTP Connect Error 0
SMTP Protocol Error 0
SMTP Disconnect 1,061
SMTP Session 112,285
SMTP To Local 119,551
SMTP To Remote 106,717
SINK Connect Error 0
POP Connect Error 2
POP Protocol Error 0
POP Disconnect 348
POP Session 119,356
Delivery Time Connect Error 0
Function Minimum Time
(millisec)
Maximum Time
(millisec)
Average Time
(millisec)
SMTP Connect 0 1,769 15.99
SMTP Hello 0 1,911 45.18
SMTP Mail From 0 438 25.75
SMTP Rcpt To 0 449 25.76
SMTP Data 0 15,903 152.67
SMTP Quit 0 471 26.19
POP Connect 0 830 25.75
POP User ID 0 10,548 136.21
POP Password 2 10,188 134.33
POP Status 0 3,538 73.89
POP Retrieve 2 10,480 114.33
POP Delete 0 9,778 65.19
POP Quit 0 9,750 69.23
Function Minimum Size
(KB)
Maximum Size
(KB)
Average Size
(KB)
SMTP Data (KB up) 1 2,674 24.82
POP Retrieve (KB down) 1 2,675 24.42


Detailed Results -- 120% Load
Load User Count Message Rate
(msg/min)
Attempted 240,000 1,200
Actual 238,600 1,193
Function Attempts # Pass
QoS
% Pass
QoS
SMTP Connect 136,254 136,254 100.00%
SMTP Hello 136,255 136,255 100.00%
SMTP Mail From 136,255 136,255 100.00%
SMTP Rcpt To 273,299 273,299 100.00%
SMTP Data 136,277 135,926 99.74%
SMTP Quit 134,931 134,931 100.00%
POP Connect 143,823 143,823 100.00%
POP User ID 143,512 37,573 26.18%
POP Password 143,199 47,858 33.42%
POP Status 142,952 69,036 48.29%
POP Retrieve 141,506 63,326 44.75%
POP Delete 140,904 79,068 56.11%
POP Quit 141,739 63,442 44.76%
Delivery Time 1,317 1,306 99.16%
SMTP Data Response Time 136,277 134,673 98.82%
POP Retrieve Response Time 141,506 70,467 49.80%
Remote Message Delivery 129,292 128,074 99.06%

Counter

Value
SMTP Connect Error 0
SMTP Protocol Error 0
SMTP Disconnect 1,346
SMTP Session 134,931
SMTP To Local 144,007
SMTP To Remote 129,292
SINK Connect Error 0
POP Connect Error 1
POP Protocol Error 0
POP Disconnect 402
POP Session 141,740
Delivery Time Connect Error 0
Function Minimum Time
(millisec)
Maximum Time
(millisec)
Average Time
(millisec)
SMTP Connect 0 2,165 19.88
SMTP Hello 0 2,229 49.44
SMTP Mail From 0 1,386 26.33
SMTP Rcpt To 0 1,060 26.55
SMTP Data 0 17,622 167.90
SMTP Quit 0 2,110 27.02
POP Connect 0 3,031 26.81
POP User ID 0 40,025 9,606.13
POP Password 2 38,400 7,265.77
POP Status 0 28,696 5,409.11
POP Retrieve 3 29,811 5,768.67
POP Delete 0 31,688 4,998.43
POP Quit 0 29,750 5,704.22
Function Minimum Size
(KB)
Maximum Size
(KB)
Average Size
(KB)
SMTP Data (KB up) 1 2,674 24.80
POP Retrieve (KB down) 1 2,675 24.53

General Notes / Tuning Information
A Network Appliance F820 with 10 36GB drives in a single raid group
was used to store the mailboxes via NFS.
It was mounted with the NFS options: udp,noatime.

Aphelion 2001 was used for the LDAP server.

Linux tuning paremeters:
echo 4096 >> /proc/sys/net/core/netdev_max_backlog
echo 2000000 >> /proc/sys/net/ipv4/tcp_max_tw_buckets
echo 0 >> /proc/sys/net/ipv4/tcp_window_scaling
echo 0 >> /proc/sys/net/ipv4/tcp_timestamps
echo 0 >> /proc/sys/net/ipv4/tcp_sack
echo 0 >> /proc/sys/net/ipv4/ip_forward
echo '16384 65535' >> /proc/sys/net/ipv4/ip_local_port_range
echo 1048576 >> /proc/sys/net/core/wmem_default
echo 1048576 >> /proc/sys/net/core/wmem_max
echo 1048576 >> /proc/sys/net/core/rmem_default
echo 1048576 >> /proc/sys/net/core/rmem_max

A HP Procurve 4000M switch (40 port 10/100Mb) was used for LAN connectivity.

Description of imsd options used:
-Vd  disable DNS lookup for clients
-L   name of an alternate debug log
-tpool minimum number of available processing threads (default = 16)
-tmax  maximum number of processing threads allowed (default = 16384)

Description of SMTPserver options used:
-Ca  host:service name of imsd administrative service
-rt  read timeout in seconds
-Cr  recipient number threshold causing queueing at low priority
-CD  enable automatic creation of mailbox directories
-na  Do not send AUTH
-I   name of the SMTP input queue (default is SMTP-in)
-D   limit delivery/relaying to allowed deliver/relay domains
-Sc  per client session threshold
-S   total session threshold
-ar  avoid reverse host address lookup


For questions about this result, please contact the tester.
For other inquiries, please contact webmaster@spec.org
Copyright © 1999-2001 Standard Performance Evaluation Corporation

First published at SPEC.org on 07-Dec-2001

Benchmark run on Tue Nov 27 23:27:03 CST 2001 by SPECmail2001 v1.01
Report generated on Mon Apr 25 17:25:12 EDT 2005 by SPECmail2001 v1.05 HTML Formatter