advertise
Monday
Sep302013

Expandability: Steve Wozniak's Biggest Success and Nolan Bushnell's Biggest Regret

 

When two titans of the early computing industry, Apple co-founder Steve Wozniak and Atari founder Nolan Bushnell, had a chance to sit down for a chat at the C2SV conference last Friday, we learned how a little foresight and luck can decide futures.  

Steve Wozniak's Success - Expandability 

 

Click to read more ...

Friday
Sep272013

Stuff The Internet Says On Scalability For September 27, 2013

Hey, it's HighScalability time:

  • 384 cores & 32TB of RAM: Oracle's SPARC M6 
  • Quotable Quotes:
    • @jennyinc: 2003: "I replaced you with a set of very small shell scripts." 2013: "I replaced your scripts with a six-figure enterprise DevOps platform."
    • @tomdale: OH: “Redis is so fast, why don’t we replace RAM with Redis?”
    • @petrillic: OH "Promises/futures are the one-night stands of architectural constructs" nice #strangeloop
    • @TwitterEng: "Java and Scala let Twitter readily share and modify its enormous codebase across a team of hundreds of developers." 

  • Lots of juicy numbers revealed at Structure:Europe: Netflix streams 114,000 years of video every month; Custom build Netflix boxes for its content-delivery network that contain between 100 and 150 terabytes of storage apiece; Netflix accounts for 35 percent of all web traffic; An unnamed social network recently purchased 85 petabytes of storage from EMC; Blogging platform WordPress now underpins nearly 20 percent of the world’s blogs; Felix Baumgartner’s record-breaking jump last year generated 8 million concurrent views on YouTube and consumed almost a quarter of the world’s bandwidth.

  • 6 hostage negotiation techniques that will get you what you want. Useful for any project. After all, aren't meetings hostage situations? The steps: active listening, empathy, rapport, influence, behavioural change. Be aware, if other hostages are also using these techniques then the world will explode.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Wednesday
Sep252013

Great Open Source Solution for Boring HA and Scalability Problems

This is a guest post about how boring and repetitive HA and scalability problems can be solved via Open Source so you can focus on the interesting tasks. The post was written by Maarten Ectors, responsible for Cloud Strategy and Frank Mueller, a Juju Core developer, at Ubuntu/Canonical.

High-availability and scalability are exciting in general but there are certain problems that experts see over and over again. The list is long but examples are setting up MySQL clustering, sharding Mongo, adding data nodes to a Hadoop cluster, monitoring with Ganglia, building continuous deployment solutions, integrating Memcached / Varnish / Nginx,… Why are we reinventing the wheel?

At Ubuntu we made it our goal to have the community solve these repetitive and often boring tasks. How often have you had to set-up MySQL replication and scale it? What if the next time you just simply do:

Click to read more ...

Monday
Sep232013

Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day

This is a guest post written by Claude Johnson, a Lead Site Reliability Engineer at salesforce.com.

The following is an architectural overview of salesforce.com’s core platform and applications. Other systems such as Heroku's Dyno architecture or the subsystems of other products such as work.com and do.com are specifically not covered by this material, although database.com is. The idea is to share with the technology community some insight about how salesforce.com does what it does. Any mistakes or omissions are mine.

This is by no means comprehensive but if there is interest, the author would be happy to tackle other areas of how salesforce.com works. Salesforce.com is interested in being more open with the technology communities that we have not previously interacted with. Here’s to the start of “Opening the Kimono” about how we work.

Since 1999, salesforce.com has been singularly focused on building technologies for business that are delivered over the Internet, displacing traditional enterprise software. Our customers pay via monthly subscription to access our services anywhere, anytime through a web browser. We hope this exploration of the core salesforce.com architecture will be the first of many contributions to the community.

Definitions

Let’s start with some basic salesforce.com terminology:

Click to read more ...

Friday
Sep202013

Stuff The Internet Says On Scalability For September 20, 2013

Hey, it's HighScalability time:

  • $21 Billion: Google spend on datacenters
  • Quotable Quotes:
    • @lintool: Jeff Dean at XLDB: Largest Google Bigtable cluster: ~100s PB data; sustained: 30M ops/sec; 100+ GB/s I/O (gmail?)
    • @neil_conway: On a quick skim, the most surprising thing about the F1 paper is how conventional of an MPP database it is.
    • @Kufat: PSA: 1024 bytes = 1 KB. If someone says it's "1 KiB," they are a Cylon, replicant, or shapeshifter, and must be destroyed forthwith.
    • @mikeleeorg: "If you only do things where you know the answer in advance, your company goes away." - Jeff Bezos
    • @andybritcliffe: Keynotes are always good for stats. Amazon are deloying software updates every 16 seconds #awssummit #impressive
    • @adrianco: last March numbers on US fixed access traffic was Netflix 32.25% and Youtube 17.11%, only takes two to get to 49.36%
    • @cpurdy: OH: "Objective-C is the lack of a type system of C smushed together with the s****y performance of Smalltalk."

  • A Focus on Efficiency. A 70 page paper on how Internet.org will accomplish its goal of providing Internet access to the 5 billion people who don't currently have it. The paper is in two parts. The first part describes how Facebook will essentially become the platform supporting these new users. The idea is to create a 10x improvement reduction in the underlying costs of delivering data, and a 10x reduction of data usage by apps. The strategies are basically taken out of Facebook's playbook, so the paper is an excellent guide to all Facebook has developed over the years. The second part of the paper is Qualcomm presenting an overview of their plan to expand global wireless capacity by 1000 times. It's an audacious gambit to be sure.

  • Another step in the Google vs Oracle divorce proceedings. Google Waves Goodbye To MySQL In Favor Of MariaDB. Using Golang over Java was another important step. It takes time for a supertanker to change direction, but it can be done.

  • If you like ideas made pictures then you'll like Do you know Cassandra? For a comic the humor is so subtle I didn't get it, but it did talk about quorums and stuff.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Wednesday
Sep182013

If You're Programming a Cell Phone Like a Server You're Doing it Wrong

Power on a cell phone is like water in a desert. It’s the very stuff of life. If you take the same naive programming techniques you learned when programming on a server in a datacenter your cell phone will die of thirst.

This is dramatically shown by Reto Meier, Tech Lead for the Android Developer Relations Team, in a remarkable series of instructional videos:

Click to read more ...

Tuesday
Sep172013

Sponsored Post: Apple, Couchbase, Evernote, MongoDB, Stackdriver, BlueStripe, Surge, Booking, Rackspace, AiCache, Aerospike, New Relic, LogicMonitor, AppDynamics, ManageEngine, Site24x7

Who's Hiring?

  • Evernote is hiring a Senior DevOps Engineer in our mission to help the world remember everything. Our work environment is collaborative and relaxed, our benefits and perks are fantastic, and we enrich the lives of more than 65 million users worldwide every day! Please apply here.

  • Stackdriver is looking for systems + cloud + dev + ops guru to serve as our liaison within the DevOps community. If you are passionate about monitoring and automation, enjoy working on open source, and are excited by the prospect of sharing your expertise with your peers, get in touch with us today! http://bit.ly/143ARmy

  • We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Please apply online.

  • Apple Applications Architect. Apple's Customer Systems group, within the global Information Systems and Technology (IS&T) organization, helps design and implement all of the critical infrastructure that allows Apple to provide the best support in the world. Imagine what you could do here. Please apply here.

  • Apple Software Engineer, Researcher. The OS X Analytics group is looking for a skilled software engineer. In this position you will develop tools to collect, process, and analyze data from millions of systems. Imagine what you could do here. Please apply here.

  • LogicMonitor is looking for a Front End developer to have a huge impact, be valued, realize their dreams, and help us realize ours. We are looking for someone to own the code that delivers the design and usability of LogicMonitor's enterprise SaaS application(s). Please apply online

  • New Relic is looking for a Java Scalability Engineer in Portland, OR. Ready to scale a web service with more incoming bits/second than Twitter?  http://newrelic.com/about/jobs

Fun and Informative Events

  • Surge - The Scalability & Performance Conference, presented by OmniTI, Sept. 12th-13th, features speakers from Joyent, Fastly, Dyn, Netflix, Linkedin and Amazon. Special, High Scalability Reader Rate: $50 off registration--through Sept. 10! Book hotel and get $50 off, from OmniTI. 

Cool Products and Services

  • The leading technology companies use Couchbase as their NoSQL database. Download the free open-source version of Couchbase Server and make something awesome today.

  • MongoDB Management Service (MMS) is a cloud-based suite of services for managing MongoDB deployments. In addition to monitoring and alerting, now you can seamlessly back up your MongoDB deployment to the cloud using using MMS. To get started with monitoring and backup, visit mms.mongodb.com.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • AppDynamics is an easy-to-use application performance management solution that offers code-level insight into Java, .NET and PHP applications. Get the free trial.

  • NEW! Aerospike 3 - Download FREE. Introducing the new Aerospike 3 database that builds off of Aerospike's legacy of speed, scale, and reliability, adding an extensible data model that supports complex data types, large data types, queries using secondary indexes, user defined functions (UDFs) and distributed aggregations using Stream UDFs for real-time data.

  • The Rackspace Cloud Application Programming Interface (API)  has changed the game allowing customers to easily modify their cloud configuration with just a few lines of code. The API is a powerful tool and something everyone should know about, regardless of your level of technical ability.

  • aiScaler, aiProtect, aiMobile integrated solutions for Dynamic Site Acceleration, Denial of Service Protection and Simplifying Mobile Content. Free instant trial, no sign-up required . http://aicache.com/

  • LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Try it free and be up and running in just 15 minutes.

  • AppDynamics is the very first free product designed for troubleshooting Java performance while getting full visibility in production environments. Visit  http://www.appdynamics.com/freetrial.

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Click to read more ...

Monday
Sep162013

The Hidden DNS Tax - Cascading Timeouts and Errors

This is a guest post by Nick Burling, VP of Product Management of Bluestripe.

Readers of High Scalability know are well versed in performance optimization techniques. Reverse proxies, Varnish, Redis — you hear about them daily. But what you may not realize is that one of the oldest technologies in your stack can be one of your biggest bottlenecks: DNS.

People don't spend a lot of time thinking about DNS. It's not sexy. It's an infrastructure service, and it's just supposed to work.

At BlueStripe, we work with many teams running applications that support millions of web requests a day. We keep seeing DNS delays and errors that the platform operations team never knows about. It's so common we've start calling it the Hidden DNS Tax.

What is the Hidden DNS Tax?

Click to read more ...

Friday
Sep132013

Stuff The Internet Says On Scalability For September 13, 2013

Hey, it's HighScalability time (this week is a fall harvest basket overflowing with good nutritious wisdom):

  • 170 million: metrics Twitter collects every minute; 350 million: Snapchat daily photo shares
  • Quotable Quotes:
    • @blowmage: OH: “Guys, databases don't know how to sort things. That's why NoSQL uses JavaScript.”
    • Nokia insider: I look back and I think Nokia was just a very big company that started to maintain its position more than innovate for new opportunities.
    • Paulo Siqueira: Ignoring scalability is not as bad as it sounds—if you use the proper tools.
    • David Rosenthal: The relationship between diversity and risk is very complex.
    • Jaime Teevan: the exact same result list will seem more relevant to you if it is returned just a fraction of a second faster.
    • @aphyr: I use Redis as a queue #leaveDBalone

  • Hey, I've always thought this about security, it's just too damn complex. But I always thought much smarter people than me understood it all, so it must be OK. Now I know that's not true. Nobody really understands security. NIST's Ridiculous Non-Response Response To Revelation That NSA Controlled Crypto Standards Process: "the NSA made sure that the standards were so complicated that no one could actually vet the security."

  • Observability at Twitter. Great post on how Twitter captures, stores, queries, visualizes and automates monitoring data to enable the debugging of hundreds of distributed systems across multiple datacenters. Very detailed and helpful. 170 million individual metrics are collected every minute. Metrics are stored in and queried from a time series database developed at Twitter. Queries are written using a declarative, functional inspired language. Visualization use cases include hundreds of charts per dashboard and thousands of data points per chart. Our monitoring system allows users to define alert conditions and notifications in the same query language they use for ad hoc queries and building dashboards. Note how everything at this level of game play is a custom job.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Wednesday
Sep112013

Ten Lessons from GitHub’s First Year in 2008

Tom Preston-Werner, GitHub Cofounder, wrote Ten Lessons from GitHub’s First Year in 2008. Though the lessons are still relevant, and the war stories behind each lesson are great, I can't help but wonder what a 2013 version would look like?

  1. Start Early - GitHub created a market that didn't exist. By building early "on top of a nascent technology, we were able to construct a startup with basically no overhead, no competition, and in our free time."
  2. Adapt to Your Customers - "Don’t give your customers what they ask for; give them what they want."
  3. Have Fun - "You have to goof off a lot in order to balance things out. Burnout is a real and dangerous phenomenon. "
  4. Pay attention to Twitter - "Twitter [is] an extremely valuable resource for instant feedback."
  5. Deploy at Will! - "The first year of a web offering is a magical one. Your customers are most likely early adopters and love to see new features roll out every few weeks. If this results in a little bit of downtime, they’ll easily forgive you, as long as those features are sweet. In the early days of GitHub, we’d deploy up to ten times in one afternoon, always inching closer to that target."
  6. You Don't Need an Office - "We have no office. But we’re not totally virtual either...the rent is a hell of a lot cheaper...invest the savings back into the company, or into our pockets."
  7. Hire Through Open Source - "it’s far easier and less risky to hire based on relevant past performance than it is to hire based on projected future performance. "
  8. Trust your Team - "In a startup, you can drastically reduce momentum by applying micromanagement, or you can boost momentum by giving trust."
  9. You Don't Need Venture Capital - "One of the reasons I left my last job was so that I could say 'the buck stops here.' If we’d taken money, I would no longer be able to say that."
  10. Open Source Whatever You Can