| |||||||||
|
I have been running various benchmarks to test the performance of NFS. The results are summarized below. Unless otherwise noted, all tests are performed with version 3 of NFS, using wsize=rsize=8192.
I am still taking more tests and analyzing the results. The contents of this page will be updated progressively as I gather more information.
The test bed is not yet complete. In particular, the linux and sun clients are on 100 and 10 Mb ethernet, respectively. Hence, the results cannot be quantitatively compared between the two platforms. We should have 1 Gb ethernet on both platforms in the near future.
Update: The test bed is now complete. The server is now the Network Appliance filer, and the linux clients are now dual 933 MHz Pentium IIIs. The linux clients and the filer are on a gigabit ethernet network. All tests taken after May will have used the new setup.
Connectathon: File & Directory Creation & Removal
| Linux | Sun | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Procedure
This test consists of basic tests 1 and 2 from the connectathon
suite. I modified the code for these tests to display all the decimal digits in the times.
The modified code is here:
test1.c
test2.c
test1 creates a specified number of files and directories, and test2 removes them. Each test reports the amount of time it took with microsecond resolution, using gettimeofday.
I wrote a perl script to run the tests. It runs test1 and test2 repeatedly,
varying the number of files and directories to be used.
The script is here:
test1_circuit.pl
Observations & Interpretation
On both platforms, the rate of directory creation starts very low, climbs quickly to a peak, and then tapers off gradually.
The low rate with small numbers of directories may indicate a high initial latency. It may be that the client does not wait for a reply from the server before initiating another request; This would explain why the rate gets better for a moderate number of directories. The rate tapering off may be a sign of congestion on the client or on the server.
On Linux, directory removal looks similar, while on Sun, directory removal is nearly a flat line. The rate is still somewhat slower for small numbers, but the effect is not as obvious. Also, the rate does not taper off noticeably for larger numbers.
This may mean that the congestion observed above was on the server. Since the Sun is only on 10 Mb ethernet, it is not able to put as heavy a load on the server.
For files, the rate appears to gradually taper off. There is no initial low point as was observed with the directories.
I'm not sure why the latency problem observed with directories doesn't show up with files. My initial guess is that it's something on the server side, as the client should be doing roughly the same amount of work either way.
In all cases, creation is slower than removal. This leads me to believe that the server is the slow point in these tests, since the client does nearly the same thing to create or to remove a file or directory
Further investigation is necessary to explain these results.
Connectathon: Reading & Writing
| Linux | Sun | ||||||
|---|---|---|---|---|---|---|---|
|
Procedure
This test consists of basic tests 5a and 5b from the connectathon suite. test5a creates a file of a specified size. test5b reads the file created by test5a. Each of these tests report the time taken using gettimeofday.
I modified the code in 5b to prevent it from deleting the test file after reading it. This allowed me to
perform two read tests on each file; one that allowed buffering, and one that disallowed it. The second was
accomplished by having the server touch the file before reading. The modified code is here:
test5b.c
I wrote a perl script to run the tests. It runs the connectathon tests with a specified range and granularity
of file sizes. The script is here:
test5_circuit.pl
I have not yet run this test on a sun machine, due to technical difficulties with ssh.
Observations & Interpretation
The buffered reads tend to be very fast until 53e6 bytes. There is a sharp dropoff here, and by 55e6 bytes, the buffered reads are not much faster than the unbuffered reads. Apparently, some buffer is being exhausted at this point.
In all cases, reads are faster than writes. This is probably because under NFS3, the server is not allowed to send a response to a write until the write is actually committed to disk.
IOZone
| Linux | Sun | ||||||||
|---|---|---|---|---|---|---|---|---|---|
|
|
Procedure
IOZone is a suite of filesystem tests. It can perform many kinds of tests, but I am focusing mainly on read, reread, write, and rewrite. My initial results were gathered using the -a option, which runs an extensive circuit of tests. I later learned that the -A option runs only the read, reread, write, and rewrite tests.
Observations & Interpretation
On Linux, there is not a noticeable difference between read and reread speeds. The read and reread rates are very high until the 64MB data point. This agrees with my results from the connectathon read test, which had a drop-off at around 50MB.
On the Sun client, there is no initial peak for the read tests, but there is one on the reread tests. It may be that the Sun client does not buffer data that it has just written. i.e. under Linux, the client still has the data in its buffer from the write test, while under SunOS it does not.
The Sun client does show buffering effects for its reread test. In this case, the rate does not drop off until the 256 MB data point. Apparently, the Sun client uses a larger buffer than the Linux client.
On Linux, both of the write tests are flat lines, while on the Sun client there is an initial peak. I'm not sure how to explain this yet.
 |
|
 |
