Logo
  • PHP
    • HipHop / HHVM
    • Modern PHP
    • PHPStorm
    • LAMP
    • Laravel
    • Composer
    • PDO
  • JavaScript
    • node.js
    • AngularJS
  • CSS
    • SASS
    • “CSS4” (CSS level 4)
  • HTML
  • Git
  • LAMP
  • Vagrant
  • UI / UX
  • Architecture of …
  • Off-Topic
With ♥ from Berlin
January 11, 2014
Chris
The architecture of ...
23

The architecture of StackOverflow

PreviousNext

One of the most interesting talks these weeks, and a rare insight into one of the most active pages on the web: Marco Cecconi of StackOverflow speaks about the general server architecture, why they don’t unit-test (!), how they release (5 times a day) and shows some awesome server load screenshots. It’s fascinating that they run one of the most trafficked pages (that also uses long-polling “real-time” messaging !) on just 25 servers, most of them on 10% load all the time. “We could run it on just 5 servers if needed”. Awesome. Nice statements regarding caching and using existing code, too.

I really like the Get-Things-Done attitude and the simple, but productive view on workflow (use multiple monitors, don’t be the nerd sitting in front of a laptop). The code is not perfect (lots of static methods), they don’t even test, only have a hand full of developers (!) and nearly no downtime. Ah yes, and they run one of the most successful sites in the history of the internet.

“Languages are just tools”. “You’ll be successful anyways, or fail anyways [it does not depend on the language].” I really like that guy. And by the way, they mainly use dot.net for the site. Make sure you also check out the links, especially #5 shows the current tech stack used in the company.

And by the way, have you noticed that EXTREMELY huge presentation screen ? Awesome! They obviously did this in a cinema or university audimax.

Update #1:
The slides of this talk:
https://speakerdeck.com/sklivvz/the-architecture-of-stackoverflow-developer-conference-2013

Update #2:
Thread on news.ycombinator.com regarding this topic, and Marco Cecconi (and other StackOverflow IT guys) have joined:
https://news.ycombinator.com/item?id=7052835

Update #3:
Excellent article in the StackOverflow tech blog showing how StackExchange was build back in 2008 (lots of technical details):
http://blog.stackoverflow.com/2008/09/what-was-stack-overflow-built-with/

Update #4:
Official 2009 database dump (legally available directly on StackOverflow):
http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/

Update #5:
AWESOME!
Full up-to-date list of software, technology, methods and servers used for StackExchange:
http://meta.stackoverflow.com/questions/10369/which-tools-and-technologies-are-used-to-build-the-stack-exchange-network

Update #6:
This excellent YouTube comment by Joseph Lust sums it up perfectly:

* Use static methods everywhere instead of OOP
* Write the least code possible
* Keep entire site compilation under 10s
* Cache every single object, it’s faster
* Design to scale Up before Out
* Use 368GB memory for your servers/db’s
* Don’t write tests,  have your users find defects
* Don’t reinvent square wheels

 

This article was written quite a while ago (8 years), please keep this in mind when using the information written here. Links, code and commands might be outdated or broken.

Random articles

  • Extremely easy SASS in Laravel (with pure PHP)Extremely easy SASS in Laravel (with pure PHP)
  • How blind people use websites (video with Sina Bahram, blind accessibility researcher)How blind people use websites (video with Sina Bahram, blind accessibility researcher)
  • php-login goes #2 PHP script worldwide in BitDeli statsphp-login goes #2 PHP script worldwide in BitDeli stats
  • How to install GitHub’s, NetBeans’s and Sublime2’s syntax highlighting code colours theme in PHPStorm 6/7How to install GitHub’s, NetBeans’s and Sublime2’s syntax highlighting code colours theme in PHPStorm 6/7
  • “Fuck you. Pay Me.” How to deal with clients, the professional way. An excellent talk with Mike Monteiro.“Fuck you. Pay Me.” How to deal with clients, the professional way. An excellent talk with Mike Monteiro.
  • Slides & talks from PHP UK Conference 2014Slides & talks from PHP UK Conference 2014
  • [Link] How to create, read, update and delete (CRUD) with PDO, MySQLi and MySQL the right way (prepared statements)[Link] How to create, read, update and delete (CRUD) with PDO, MySQLi and MySQL the right way (prepared statements)
  • First view: Ubuntu 14.04 LTS brings PHP 5.5 and Apache 2.4First view: Ubuntu 14.04 LTS brings PHP 5.5 and Apache 2.4
  • [Link] Retinafy your Site / Device by Nijiko Yonskai[Link] Retinafy your Site / Device by Nijiko Yonskai
architecturebackendcompanyscalabilitysidebar-specialstackoverflow
Share this

23 Comments

  • M. Moore
    August 26, 2018 1:33 am

    TDD Doesn’t work well with static classes? That’s news to me. Why does a test case care about instantiation? Why should it? I suppose there’s no writing a failed test case for Math.Add(1,2), eh?

    Reply
  • Iván Stepaniuk
    January 22, 2014 11:48 pm

    I can’t see how TDD forces object lifetime that way. IoC and static classes are at odds, but there is no need for that static-everything madness. Whole graphs can be reused by many requests with just one static container class, thus avoiding massive garbage collections. They are sacrificing maintainability, the “Fear Driven Development” is the proof.

    Reply
  • Bill Schneider
    January 20, 2014 4:00 am

    Interesting points about frequent releases and short feedback loops. You don’t need extensive manual or unit tests if you can release constantly to an alpha site where real users will find issues for you – not a good idea for every business but works well for them.

    Reply
  • Yoo Matsuo
    January 15, 2014 3:34 pm

    I didn’t imagine SO runs by so many static methods, though Marco’s explanation, “many instances make GC run so often and it causes low performance, therefore we cannot use TDD which doesn’t work well with static methods”, has so many interesting points of view. Very interesting talk. Thanks!

    Reply
  • Alex Dowad
    January 14, 2014 11:43 am

    It’s very strange that at a “Developer Conference”, Mr. Cecconi had to explain concepts like garbage collection!

    Reply
    • Chris
      February 10, 2015 8:03 pm

      I think most people were just too shy to raise their hands :) As most conferences are quite expensive I highly doubt that the audience are mostly beginners.

      Reply
  • Obdurodon
    January 13, 2014 9:09 pm

    “Use multiple monitors” is just as much posturing as “look cool using a tiny laptop” is. If you’re doing something really hard, like solving a tricky algorithmic problem or chasing an elusive bug, it *doesn’t matter* what kind of monitor’s in front of you because most of the action is in your head anyway. A good programmer can do these things while walking around, exercising (like I will be in a minute), or taking a crap. Multiple big screens can help sometimes, but they can also hurt if they encourage a level of multi-tasking a.k.a. self-interruption that disrupts good concentration.

    Reply
    • Chris
      January 13, 2014 9:31 pm

      Hmm good point, but basically I agree with Mr. Cecconi’s opinion, as lots of developers block themselves (and their workflows) with tiny screens, tiny eye-hurting font sizes, small laptop keyboards, not using a mouse (!), no second screen, not using an additional analog notebook (to keep your stuff organized OFF the screen), and attitudes a la “i want to solve this by myself” instead of taking the shortest way etc. This is a super-interesting topic, but we should not discuss this here.

      Reply
      • Sławomir Uchto
        January 20, 2015 11:44 am

        Why should I use mouse? As a developer I prefer to have two hands on keyboard. Mouse is nothing but distraction and slows us down.

        Reply
        • Chris
          February 10, 2015 8:13 pm

          Why should we NOT use a mouse ? Switching / moving / organizing things with a keyboard is nearly impossible and slows us down extremely. But in general I think everybody should find his/her personal workflow, so if you can work in a wonderful way without a mouse, then do so, iwould also say it depends on what you are doing. I get frustrated when frontend people need 10sec to close a windows when they try to reach a tiny close button with a sticky touchpad. :) Please let’s skip or move the discussion as it’s not really related to the above article. Cheers!

          Reply
  • Soner Gönül
    January 12, 2014 1:11 pm

    I just learned a lot of information how Stack Overflow works on behind. Thanks.

    Reply

Leave A Comment Cancel reply

php

Awesome list of Design Patterns with PHP code examples

Probably one of the most useful GitHub repositories ever: Dominik Liebler has published a well-reseached and well-written (PSR coding guidelines

sass

[german video] Modernizing and minimalizing frontend markup code at AutoScout24

As this blog has a lot of german speaking readers I’ll post german stuff from time to time. Sorry for

hiphop php

[Link] How to set up HipHop, Nginx and Laravel in Ubuntu 12.04 LTS (in a Vagrant box)

This is probably the most useful tutorial in 2014 so far. An excellent post by Chris Fidao, explaining how to

sass

New features in SASS 3.3 (a talk by SASS-creator Chris Eppstein)

A very interesting conference talk by Chris Eppstein, creator of SASS itself, showing some of the new features in SASS

You made a mess with Git ? Here’s a flowchart guideline on how to fix

Extremely useful. Originally created by Justin Hileman in the presentation Changing History, or How to Git pretty. You’ll also find

Bézier Curves – Under the Hood (4min video)

Take a peek under the hood to see how computers draw cubic Bézier curves, as used in design and motion

bitdeli git github stats

Get visitor stats for your GitHub repo with BitDeli

GitHub is the definitive #1 “hoster” for git repositories, no question. And this highly sympathic company has made version control

php

How to install the mcrypt php extension (to use Laravel 4)

When installing Laravel 4 on a fresh Ubuntu or Debian system, you’ll probably get this error message: “Laravel requires the

vagrant

A preinstalled Vagrant box with PHP HipHop / HHVM and Ubuntu 13.10 (Saucy Salamander)

Here’s an excellent downloadable Vagrant box that’s brings you a preinstalled HHVM / HipHop for PHP within a Ubuntu 13.10

php-7

PHP 6.0 will be PHP 7

Germany’s excellent PHP Magazin just posted interesting news: The next major version of PHP will be PHP 7, the development

1/4

Categories

Search

How Snapchat wants to earn money (by establishing vertical videos)
Install MINI in 30 seconds inside Ubuntu 14.04 LTS
Frontend Ops Conf 2014 – Rebooting Flickr On A Node.js Stack, One Page At A Time (from PHP) by Bertrand Fan
hearthbleed-ssl-bug
A quick guideline on how to fix the Hearthbleed bug (and update OpenSSL) on Ubuntu
the-php-login-project
How to install php-login-minimal on Ubuntu 12.04 LTS
8 awesome pure CSS spinner / loader
forbes 30 under 30
Need motivation ? Check out these 2 awesome “FORBES 30 under 30” lists (web, UI, games)
github-logo-octocat
GitHub rolls out .PSD diff and viewing
php
PHP 5.6 announced, statically typed (!) “new” PHP announced by Facebook devs
mod-rewrite-ubuntu-14-04-lts
How to enable mod_rewrite in Ubuntu 14.04 LTS
phpstorm-8
PHPStorm 8 (early access version) released – for free
offf-2014
Europeans: Get ready for OFFF conference / festival in Barcelona, May 2014
composer
A short & simple Composer tutorial
laracon-2014-eu-amsterdam
Laracon 2013 – Kapil Verma: Engineering Complex Applications with Laravel 4 (40min video)
composer
How to install Composer on Windows 7 / 8 or Ubuntu

Tags

apache bash centos composer conference coupon CSS debian fonts framework git GitHub hack HHVM HipHop HTML HTML5 IDE JavaScript JS LAMP laravel linux mod_rewrite MVC MySQL Nginx optimization PHP PHP 5.5 PHP 5.6 phpmyadmin PHPStorm security server SSD Ubuntu UI UX vagrant video virtual machine voucher VPS wordpress
Side-Project: Wordle-Solver:
www.wordle-helper.info

Pages

  • Privacy Policy
 
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Cookie SettingsAccept All
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT