Hi Friends,

I need some help in making site access fast. On an average my site has the traffic 2500 hits per day and on 16th May it had 60,000 hits. On this day site was loading very slow even it was getting time out. I also check out the processes running by using "top" command it was indicating mysql was taking too much load.

There are around 166 tables (Including PHPBB forum) in my database. All contents on site are displayed by fetching it from database. I have also added indexing to respective tables where it is required. Plain PHP/HTML coding is used.

Technology:

PHP -- 5.2
MYSQL -- 5.0
Apache -- 2.0
Linux

Following is all the server details of my site:

CPU : Single Socket Dual Core AMD Opteron 1212HE
Memory: 2GB DDR RAM
Hard Drive: 250GB SATA
Ethernet: 100Mb Primary Ethernet Card

Disk usage: 10% used/ 196 G available.

Its an dedicated server and only 1 website is hosted.

Can anybody please suggest how can I optimize site in more appropriate manner so that it will not go down if traffic increases on site.

Thanks
Sandy

Update:Presentation: Second Life’s Architecture. Ian Wilkes, VP of Systems Engineering, describes the architecture used by the popular game named Second Life. Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added.

Second Life is a 3-D virtual world created by its Residents. Virtual Worlds are expected to be more and more popular on the internet so their architecture might be of interest. Especially important is the appearance of open virtual worlds or metaverses.
What happens when video games meet Web 2.0? What happens is the metaverse.

Information Sources

• Second Life runs MySQL
• Interview with Ian Wilkes
• TechTrends: Inside Linden Lab
• Town Hall with Cory Linden
• InformationWeek articles (1, 2) and blog
• Second Life Wiki: Server Architecture
• Wikipedia: Second Life Server
• Second Life Blog
• Second Life: A Guide to Your Virtual World

Platform

• MySQL
• Apache
• Squid
• Python
• C++
• Mono
• Debian

What's Inside?

The Stats

• ~1M active users
• ~95M user hours per quarter
• ~70K peak concurrent users (40% annual growth)
• ~12Gbit/sec aggregate bandwidth (in 2007)

Website stats:

Webserver: Apache 2.2
Database: MySQL 5.0
APC cache for php
CMS: Drupal 6.2 (bleeding-edge version)*
*Aggressive caching is ON, Page Compression ON, Block Cache ON (can't use CCS),Optimize CSS/JS ON.
2 Servers: Apache/Mysql (low-tech servers - Celeron processors, 512 MB RAM, 7200 RPM HDD)
Bandwidth 10 Mb/s

The benchmark:

Used ab : ab -n 1000 -c 20 howwhatwho.com

Server Software: Apache/2.2.3
Server Hostname: howwhatwho.com
Server Port: 80
Document Path: /
Document Length: 41639 bytes
Concurrency Level: 20
Time taken for tests: 13.556796 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 42118000 bytes
HTML transferred: 41639000 bytes
Requests per second: 73.76 [#/sec] (mean)
Time per request: 271.136 [ms] (mean)
Time per request: 13.557 [ms] (mean, across all concurrent requests)
Transfer rate: 3033.90 [Kbytes/sec] received

The Apache server is also running the postifx and bind although they aren't resource intensive applications.
The Cron job for drupal runs every 50 minutes, and the agreggator module is enabled and fetches more than 30 rss feeds every time.

The site used to be hosted on a single Celeron machine but on peak times the CPU went up to 80 %.

Question : Does anybody know a website hosted on an IBM Mainframe? :) Todd?

Scalr is a fully redundant, self-curing and self-scaling hosting environment utilizing Amazon's EC2. It has been recently open sourced on Google Code.

Scalr allows you to create server farms through a web-based interface using prebuilt AMI's for load balancers (pound or nginx), app servers (apache, others), databases (mysql master-slave, others), and a generic AMI to build on top of.
Scalr promises automatic high-availability and scaling for developers by health and load monitoring.

The health of the farm is continuously monitored and maintained. When the Load Average on a type of node goes above a configurable threshold a new node is inserted into the farm to spread the load and the cluster is reconfigured. When a node crashes a new machine of that type is inserted into the farm to replace it.

Update: YouTube: The Platform. YouTube adds a new rich set of APIs in order to become your video platform leader--all for free. Upload, edit, watch, search, and comment on video from your own site without visiting YouTube. Compose your site internally from APIs because you'll need to expose them later anyway.

YouTube grew incredibly fast, to over 100 million video views per day, with only a handful of people responsible for scaling the site. How did they manage to deliver all that video to all those users? And how have they evolved since being acquired by Google?

I’ve been a part of some late night release procedures and they’re never fun. You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. Sometimes the dreaded “rollback!” is yelled. All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime.

If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer....

Update: Flickr hits 2 Billion photos served. That's a lot of hamburgers.

Flickr is both my favorite bird and the web's leading photo sharing site. Flickr has an amazing challenge, they must handle a vast sea of ever expanding new content, ever increasing legions of users, and a constant stream of new features, all while providing excellent performance. How do they do it?

Slashdot effect: overwhelming unprepared sites with an avalanche of reader's clicks after being mentioned on Slashdot. Sure, we now have the "Digg effect" and other hot new stars, but Slashdot was the original. And like many stars from generations past, Slashdot plays the elder statesman's role with with class, dignity, and restraint. Yet with millions and millions of users Slashdot is still box office gold and more than keeps up with the young'ins. And with age comes the wisdom of learning how to handle all those users. Just how does Slashdot scale and what can you learn by going old school?

the authors of drupal have paid considerable attention to performance and scalability. consequently even a default install running on modest hardware can easily handle the demands a small website.

if you are lucky, eventually the time comes when you need to service more users than your system can handle. at some point, you'll start looking at your hardware and network deployment.

read more.

This presentation by Michael Radwin describes why Yahoo! had standardized on PHP going forward. It describes how after reviewing all the web technologies including their own internal ones, PHP was choosen. It shows that not only technical reasons , but also business and development processes were taken into account.