advertise
« Strategy: Sample to Reduce Data Set | Main | Behind The Scenes of Google Scalability »
Tuesday
Apr292008

High performance file server

What have bunch of applications which run on Debian servers, which processes huge amount of data stored in a shared NFS drive.

we have 3 applications working as a pipeline, which process data stored in the NFS drive. The first application processes the data and store the output in some folder in the NFS drive, the second app in the pipeline process the data from the previous step and so on.

The data load to the pipeline is like 1 GBytes per minute. I think the NFS drive is the bottleneck here.

Would buying a specialized file server improve the performance of data read write from the disk ?

Reader Comments (5)

You folks considered pvfs2 ?

November 29, 1990 | Unregistered CommenterAnonymous

Kevin Closson has some praise for http://kevinclosson.wordpress.com/2008/04/02/iops-in-a-very-high-end-nfs-environment/">Gear6 in the high performance NFS space.

November 29, 1990 | Unregistered CommenterTodd Hoff
November 29, 1990 | Unregistered Commenterfarhaj

Sun's Open Storage systems could easily handle that. Have a look at this: http://blogs.sun.com/storage/entry/flash_performance_in_storage_systems">Flash Performance in Storage Systems

November 29, 1990 | Unregistered CommenterAnonymous

> Sun's Open Storage systems could easily handle that. Have a look at this: Flash Performance in Storage Systems

The new http://www.sun.com/launch/2008-1110/index.jsp">Sun Unified Storage Systems are NAS/NFS appliances with SSDs and ZFS and nice price/performance and scalability.

You could build your own box with OpenSolaris though.

November 29, 1990 | Unregistered CommenterAnonymous

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>