advertise
Wednesday
Oct152008

Outside.in Scales Up with Engine Yard and moving from PHP to Ruby on Rails

This article explains how Outside.in, the local social network and aggregator, scaled up its service and moved from PHP to Ruby on Rails (they moved maybe because Ruby code seemed to be more maintanable that PHP code?).

The whole article is here on EngineYard blog.

Click to read more ...

Wednesday
Oct152008

Tokyo Tech Tsubame Grid Storage Implementation

This Sun BluePrint article describes the storage architecture of the Tokyo Institute of Technology TSUBAME grid. The Tokyo Institute of Technology is of the world's leading technical institutes, and recently created the fastest supercomputer in Asia, and one of the largest supercomputers outside of the United States. By deploying Sun Fire x64 servers and data servers in a grid architecture, Tokyo Tech built a cost-effective and flexible supercomputer consisting of hundreds of systems, thousands of processors, terabytes of memory and a petabyte of storage that supports users running common off-the-shelf applications. This is the second of a three-article series. It describes the steps to install and configuring the Lustre file system within the storage architecture.

Click to read more ...

Wednesday
Oct152008

The Tokyo Institute of Technology Supercomputer Grid: Architecture and Performance Overview

One of the world's leading technical institutes, the Tokyo Institute of Technology (Tokyo Tech) created the fastest supercomputer in Asia, and one of the largest outside of the United States. Using Sun x64 servers and data servers deployed in a grid architecture, Tokyo Tech built a cost-effective, flexible supercomputer that meets the demands of compute and data-intensive applications. Built in just 35 days, the TSUBAME grid includes hundreds of systems incorporating thousands of processor cores and terabytes of memory, and delivers 47.38 trillion floating-point operations per second (TeraFLOPS) of sustained LINPACK benchmark performance and 1.1 petabyte of storage to users running common off-the-shelf applications. Based on the deployment architecture, the grid is expected to reach 100 TeraFLOPS in the future. This article provides an overview of the Tokyo Tech grid, named TSUBAME. The first in a series of Sun BluePrints articles on the TSUBAME grid, this document discusses the requirements and overall system architecture of the grid, as well as the tuning performed to achieve high LINPACK benchmark performance results.

Click to read more ...

Wednesday
Oct152008

Sun Customer Ready HPC Cluster: Reference Configurations with Sun Fire X4100, X4200, and X4600 Servers

The reference configurations described in this paper are starting points for building Sun Customer Ready HPC Clusters configured with the Sun Fire X4100, X4200, and X4600 families of servers. The configurations define how Sun Systems Group products can be configured in a typical grid rack deployment. This document describes configurations using Sun Fire X4100 and X4100 M2 servers with a Gigabit Ethernet data fabric and with a high-speed InfiniBand fabric. In addition, this document describes configurations using Sun Fire X4200, X4200 M2, X4600, and X4600 M2 servers with an InfiniBand data fabric. These configurations focus on single rack solutions, with external connections through uplink ports of the switches. These reference configurations have been architected using Sun's expertise gained in actual, real-world installations. Within certain constraints, as described in the later sections, the system can be tailored to the customer needs. Certain system components described in this document are only available through Sun's factory integration. Although the information contained here could be used during an integration on-site, the optimal benefit is achieved through Sun Customer Ready System integration.

Click to read more ...

Wednesday
Oct152008

Sun Customer Ready HPC Cluster: Reference Configurations with Sun Fire X2200 M2 and X2100 M2 Servers

The reference configurations described in this blueprint are starting points for building Sun Customer Ready HPC Clusters configured with Sun Fire X2100 M2 and X2200 M2 servers. The configurations define how Sun Systems Group products can be configured in a typical grid rack deployment. This document describes configurations in detail using Sun Fire X2100 M2 and X2200 M2 servers with a Gigabit Ethernet data fabric, as well as configurations using Sun Fire X2200 M2 servers with a high-speed InfiniBand fabric. These configurations focus on single rack solutions, with external connections through uplink ports of the switches. These reference configurations have been architected using Sun's expertise gained in actual, real-world installations. Within certain constraints, as described in the later sections, the system can be tailored to the customer needs. Certain system components described in this document are only available through Sun's factory integration. Although the information contained here could be used during an integration on-site, the optimal benefit is achieved through Sun Customer Ready System integration.

Click to read more ...

Wednesday
Oct152008

Hadoop - A Primer

Hadoop is a distributed computing platform written in Java. It incorporates features similar to those of the Google File System and of MapReduce to process vast amounts of data "Hadoop is a Free Java software framework that supports data intensive distributed applications running on large clusters of commodity computers. It enables applications to easily scale out to thousands of nodes and petabytes of data" (Wikipedia) * What platform does Hadoop run on? * Java 1.5.x or higher, preferably from Sun * Linux * Windows for development * Solaris

Click to read more ...

Tuesday
Oct142008

Sun N1 Grid Engine Software and the Tokyo Institute of Technology Super Computer Grid

One of the world's leading technical institutes, the Tokyo Institute of Technology (Tokyo Tech) created the fastest supercomputer in Asia, and one of the largest outside of the United States. Using Sun x64 servers and data servers deployed in a grid architecture, Tokyo Tech built a cost-effective, flexible supercomputer that meets the demands of compute- and data-intensive applications. Built in just 35 days, the TSUBAME grid includes hundreds of systems incorporating thousands of processor cores and terabytes of memory, and delivers 47.38 trillion1 floating-point operations per second (TeraFLOPS) of sustained LINPACK benchmark performance and 1.1 petabyte of storage to users running common off-the-shelf applications. Based on the deployment architecture, the grid is expected to reach 100 TeraFLOPS in the future. This Sun BluePrints article provides an overview of the Tokyo Tech grid, named TSUBAME. The third in a series of Sun BluePrints articles on the TSUBAME grid, this document provides an overview of the overall system architecture of the grid, as well as a detailed look at the configuration of the Sun N1 Grid Engine software that makes the grid accessible to users.

Click to read more ...

Tuesday
Oct142008

Implementing the Lustre File System with Sun Storage: High Performance Storage for High Performance Computing

Much of the focus of high performance computing (HPC) has centered on CPU performance. However, as computing requirements grow, HPC clusters are demanding higher rates of aggregate data throughput. Today's clusters feature larger numbers of nodes with increased compute speeds. The higher clock rates and operations per clock cycle create increased demand for local data on each node. In addition, InfiniBand and other high-speed, low-latency interconnects increase the data throughput available to each node. Traditional shared file systems such as NFS have not been able to scale to meet this growing demand for data throughput on HPC clusters. Scalable cluster file systems that can provide parallel data access to hundreds of nodes and petabytes of storage are needed to provide the high data throughput required by large HPC applications, including manufacturing, electronic design, and research. This paper describes an implementation of the Sun Lustre file system as a scalable storage cluster using Sun Fire servers, high-speed/low-latency InfiniBand interconnects, and additional networking and storage devices. Furthermore, this paper explores the use of the Sun Lustre file system at a shared government and education research site, including configuration information and details on testing that was performed on-site to evaluate the performance of Sun's scalable storage solution.

Click to read more ...

Tuesday
Oct142008

Sun Storage and Archive Solution for HPC

When designing data storage solutions for High Performance Computing (HPC) environments, IT architects strive to balance complex and often conflicting requirements. The need to manage a skyrocketing amount of data, along with the goals of controlling cost and immediate data availability, can make it difficult to meet HPC application demands within the constraints of today's IT budgets. To help customers address an almost bewildering set of architectural challenges, Sun has developed the Sun Storage and Archive Solution for HPC, a reference architecture that can be easily customized to meet specific application goals and business requirements. This article is intended for IT managers and storage architects familiar with HPC applications and data requirements in the organization. It assumes that the audience has a technical background and some familiarity with issues surrounding the task of configuring systems and storage.

Click to read more ...

Monday
Oct132008

Challenges from large scale computing at Google

From Greg Linden on a talk Google Fellow Jeff Dean gave last week at University of Washington Computer Science titled "Research Challenges Inspired by Large-Scale Computing at Google" : Coming away from the talk, the biggest points for me were the considerable interest in reducing costs (especially reducing power costs), the suggestion that the Google cluster may eventually contain 10M machines at 1k locations, and the call to action for researchers on distributed systems and databases to think orders of magnitude bigger than they often are, not about running on hundreds of machines in one location, but hundreds of thousands of machines across many locations.

Click to read more ...