Lessons from Pownce - The Early Years

Pownce is a new social messaging application competing micromessage to micromessage with the likes of Twitter and Jaiku. Still in closed beta, Pownce has generously shared some of what they've learned so far. Like going to a barrel tasting of a young wine and then tasting the same wine after some aging, I think what will be really interesting is to follow Pownce and compare the Pownce of today with the Pownce of tomorrow, after a few years spent in the barrel. What lessons lie in wait for Pownce as they grow? Site: http://www.pownce.com/

Information Sources

Pownce Lessons Learned - FOWA 2007

Scoble on Twitter vs Pownce

Founder Leah Culver's Blog

The Platform

Python

Django for the website framework

Amazon's S3 for file storage.

Adobe AIR (Adobe Integrated Runtime) for desktop application

Memcached

Available on Facebook

Timeplot for charts and graphs.

The Stats

Developed in 4 months and went to an invite-only launch in June.

Began as Leah's hobby project and then it snowballed into a real horse with the addition of Digg's Daniel Burka and Kevin Rose.

Small 4 person team with one website developer.

Self funded.

One MySQL database.

Features include: - Short messaging, invites for events, links, file sharing (you can attach mp3s to messages, for example). - You can limit usage to a specific subset of friends and friends can be grouped in sets. So you can send your mp3 to a specific group of friends. - It does not have an SMS gateway, IM gateway, or an API.

The Architecture

Chose Django because it had an active community, good documentation, good readability, it is open to growth, and auto generated administration.

Chose S3 because it minimized maintenance and was inexpensive. It has been reliable for them.

Chose AIR because it had a lot of good buzz, ease of development, creates a nice UI, and is cross platform.

Database has been the main bottleneck. Attack and fix slow queries.

Static pages, objects, and lists are cached using memcached.

Queuing is used to defer more complex work, like sending notes, until later.

Use pagination and a good UI to limit the amount of work performed.

Good indexing helped improve the performance for friend searching.

In a social site: - Make it easy to create and destroy relationships. - Friend relationships are the most important information to display correctly because people really care about it. - Friends in the online world have real-world effects.

Features are "biased" for scalability - You must get an invite from someone on already on Pownce. - Invites are limited to their data center's ability to keep up with the added load. Blindly uploading address books can bring on new users exponentially. Limiting that unnatural growth is a good idea.

Their feature set will expand but they aren't ready to commit to an API yet.

Revenue model: ads between posts.

Lessons Learned

The four big lessons they've experienced so far are: - Think about technology choices. - Do a lot with a little. - Be kind to your database. - Expect anything.

Have a small dedicated team where people handle multiple jobs.

Use open source. There's lots of it, it's free, and there's a lot of good help.

Use your resources. Learn from website doc, use IRC, network, participate in communities and knowledge exchange.

Shed work off the database by making sure that complex features are really needed before implementing them.

Cultivate a prepared mind. Expect the unexpected and respond quickly to the inevitable problems.

Use version control and make backups.

Maintain a lot of performance related stats.

Don't promise users a deadline because you just might not make it.

Commune with your community. I especially like this one and I wish it was done more often. I hope this attitude can survive growth. - Let them know what you are working on and about new features and bug fixes. - Respond personally to individual bug creators.

Take a look at your framework's automatically generated queries. They might suck.

A sexy UI and a good buzz marketing campaign can get you a lot of users.

Related Articles

Scaling Twitter: Making Twitter 10000 Percent Faster.

Click to read more ...

Paper: Architecture of a Highly Scalable NIO-Based Server

The article describes the basic architecture of a connection-oriented NIO-based java server. It takes a look at a preferred threading model, Java Non-blocking I/O and discusses the basic components of such a server.

Click to read more ...

Using ThreadLocal to pass context information around in web applications

Hi, In java web servers, each http request is handled by a thread in thread pool. So for a Servlet handling the request, a thread is assigned. It is tempting (and very convinient) to keep context information in the threadlocal variable. I recently had a requirement where we need to assign logged in user id and timestamp to request sent to web services. Because we already had the code in place, it was extremely difficult to change the method signatures to pass user id everywhere. The solution I thought is class ReferenceIdGenerator { public static setReferenceId(String login) { threadLocal.set(login + System.currentMillis()); } public static String getReferenceId() { return threadLocal.get(); } private static ThreadLocal threadLocal = new ThreadLocal(); } class MySevlet { void service(.....) { HttpSession session = request.getSession(false); String userId = session.get("userId"); ReferenceIdGenerator.setRefernceId(userId); try { doSomething(); } finally { ReferenceIdGenerator.remove(); } } This method is also discussed at http://crazybob.org/2006/07/hard-core-java-threadlocal.html Is this a reasonable approach to pass context information in web application? Can this ever happen that while a http request is being processed in the thread, a thread is suddenly assigned to some other tasks? I hope this can never happen, because app servers themselves rely heavily on threadlocals to keep transaction related information around. What do you think? Thanks, Unmesh

Click to read more ...

Product: Wackamole

Wackamole is an application that helps with making a cluster highly available. It manages a bunch of virtual IPs, that should be available to the outside world at all times. Wackamole ensures that a single machine within a cluster is listening on each virtual IP address that Wackamole manages. If it discovers that particular machines within the cluster are not alive, it will almost immediately ensure that other machines acquire these public IPs. At no time will more than one machine listen on any virtual IP. Wackamole also works toward achieving a balanced distribution of number IPs on the machine within the cluster it manages. There is no other software like Wackamole. Wackamole is quite unique in that it operates in a completely peer-to-peer mode within the cluster. Other products that provide the same high-availability guarantees use a "VIP" method. Wackamole is an application that runs as root in a cluster to make it highly available. It uses the membership notifications provided by the Spread toolkit to generate a consistent state that is agreed upon among all of the connected Wackamole instances. Wackamole is released under the CNDS Open Source License. Note: This post has been adapted from the linked to web site.

Related Articles

White paper on building HA/LB Clusters by Theo Schlossnagle.

Click to read more ...

Number of load balanced servers

Hello, Does someone know or has an idea of how many load balanced servers there might be? Thanks, Antoni www.amasso.info

Click to read more ...

You Can Now Store All Your Stuff on Your Own Google Like File System

New update: Parascale’s CTO on what’s different about Parascale. Let's say you have gigglebytes of data to store and you aren't sure you want to use a CDN. Amazon's S3 doesn't excite you. And you aren't quite ready to join the grid nation. You want to keep it all in house. Wouldn't it be nice to have something like the Google File System you could use to create a unified file system out of all your disks sitting on all your nodes? According to Robin Harris, a.k.a StorageMojo (a great blog BTW), you can now have your own GFS: Parascale launches Google-like storage software. Parascale calls their softwate a Virtual Storage Network (VSN). It "aggregates disks across commodity Linux x86 servers to deliver petabyte-scale file storage. With features such as automated, transparent file replication and file migration, Parascale eliminates storage hotspots and delivers massive read/write bandwidth." Why should you care? I don't know about you, but the "storage problem" is one the most frustrating parts of building websites. There's never a good answer that is affordable. Should you build a SAN or a NAS? How do you make it redundant? How do you make it perform? How do you back it up? How do you grow it without a defense appropriations sized budget? Should you use RAID? Which level and where for what reason? Should you use SCSI, iSCSI, SAS, SATA, or alpha beta? Which vendor should you use? There are so many conflicting opinions about everything. It's all a confusing mess to me. So I like the simplicity of buying commodity nodes with just a bunch of disks attached. But the question has always been how do you turn all those disks into a unified storage system without writing a ton of software on top? Harris says this is what Parascale has done for you:

VSN, like GFS, builds availability and scalability around low-cost servers and disks. NAS appliances rely on costly low-volume boxes that are closed and don't scale. GFS has been deployed in production clusters of over 5,000 servers, proving the scalability of the architecture. Fast, reliable, low-cost and massively scalable storage powers the growth of new applications like Web 2.0, video-on-demand, and hi-resolution image archiving. Parascale is the first of a new generation of software-only storage solutions.

They make a big deal out of it being a software only system. Harris says why this is a good thing:

I like software-based systems because hardware is a commodity. When you create custom hardware you also create low-volume, high-cost components whose economics go from bad to worse. If you *need* to do it, then go for it. But data is getting cooler and the requirement for specialized high-performance hardware is shrinking relative to the market.

Other systems use an appliance model. Appliances can add a lot of value, but they are also a way of monetizing you. A software system on commodity hardware has the potential to give good value. Will it? I didn't see pricing so it's hard to tell. Even odder is their pricing model. You are leasing the software per year, per disk spindle. Do you have any idea how much this will cost? Neither do I. I sounds like it could be horribly expensive or really reasonable. We'll have to see. Another thing that bothers me is that you can't run a database on top of their file system. This means I need an entire separate storage system for my database. You can run a database on a NAS or SAN, so this is a definite disadvantage. Anyway, it's just another interesting option to consider when architecting your website.

Related Articles

LiveJournal created an open source distributed file system called MogileFS that builders may find useful.

Parascale Announces Industry's First Software-Only Storage Solution for Digital Content

Click to read more ...

Save on a Load Balancer By Using Client Side Load Balancing

In Client Side Load Balancing for Web 2.0 Applications author Lei Zhu suggests a very interesting approach to load balancing: forget DNS round robbin, toss your expensive load balancer, and make your client do the load balancing for you. Your client maintains a list of possible servers and cycles through them. All the details are explained in the article, but it's an intriguing idea, especially for the budget conscious startup.

Click to read more ...

Why most large-scale Web sites are not written in Java

There is a lot of information in the blogosphere describing the architecture of many popular sites, such as Google, Amazon, eBay, LinkedIn, TypePad, WikiPedia and others. I've summarized this issue in a blog post here I would really appreciate your opinion on this matter.

Click to read more ...

Paper: Brewer's Conjecture and the Feasibility of Consistent Available Partition-Tolerant Web Services

Abstract: When designing distributed web services, there are three properties that are commonly desired: consistency, availability, and partition tolerance. It is impossible to achieve all three. In this note, we prove this conjecture in the asynchronous network model, and then discuss solutions to this dilemma in the partially synchronous model.

Click to read more ...

Some Real Financial Numbers for Your Startup

If you are a startup you may find useful Guy Kawasaki's post Financial Models for Underachievers: Two Years of the Real Numbers of a Startup. Part of any business plan are the projected guestimates. They are guestimates because everyone keeps these numbers hidden like a Swiss bank account. But not Redfin. They've bravely shared their initial cost projections, their actual numbers from real life, and the lessons they've learned from the discrepancy between the two... You can find their model estimates and actuals for Rent, Per Employee, Per Month (model: $250, actual: $336); Initial Per-Employee Equipment Cost; Monthly Benefits, Per-Employee; Annual Payroll Tax; Quarterly Bonus Payout, as a % of the Total Possible; Annual Payroll Increase for Existing Employees; All-Company Meeting Cost, Per-Meeting, Per-Employee; Annual Accounting Costs, and a few more. There is also a great lessons section: Focus on headcount; Plan slow, run fast; Run top-down sanity-checks; Forget economies of scale; Admit that revenues are a mystery; Build from building blocks; Take out "hope"; Flag your assumptions; Hit $100 million in revenues within five years; Keep market-share under 20%. I find $100 million in revenues a surprisingly high number. That's a lot of money. And the underestimate for meeting costs is pretty funny. It's always those damn meetings!

Click to read more ...