Making Facebook 2x Faster. Facebook have a system called BigPipe which allows them to progressively send their pages to the browser as the server-side processing completes to optimise client loading time. Anyone reverse engineered this yet to figure out how they actually do it?
Sign in with 
OpenID
I think facebook is 2x less stable as a result of all these changes lately. I get so many problems with comments mysteriously disappearing, ajax errors and most importantly facebook chat not working half the time.
Dean Clatworthy - 19th February 2010 10:52 - #
Reverse engineer?
Reverse engineer the standard function of PHP?
Denis - 19th February 2010 11:21 - #
They probably flush the Web server buffer after the </head> like Yahoo! does. The idea is to "pipeline" the generation and the rendering of the page: the web browser can load CSS and maybe asynchronous JS while the server is generating the page.
That would be pretty hard to do with MVC+Middleware frameworks like Django. You would need a generic <head> for the whole site flushed by a middleware before the view, and a view that renders only the body content. The title could be rewritten by javascript and you can drop closing body and html tags like Google. SEO might require some hacking, so I'm not sure it's worth it.
Yep, like Dean I've noticed an increased number of functional failures.
Mostly since the interface redesign in the last two weeks though. I generally attempt to post a comment/status update, and my response times out. Yet if I open in a new tab, sometimes my comment has already posted, and sometimes it hasn't.
Another annoyance. All the whinging in the comments on any Facebook feedback to the community! It's FREE. It's not "critical" data. I wish the comments on their posts were moderated to cut out all the noise. I'm sure there are some insightful responses, but I ain't going to search for them.
Mark Swannie - 19th February 2010 15:07 - #
I logged in and viewed source:
http://flic.kr/p/7E9zR8
They're just serving up a sequence of script tags. If you disable JavaScript a <noscript> block redirects you to a page which sets a noscript=1 cookie and disables BigPipe.
Mark,
if Gmail didn't deliver my email, you'll be sure I'll complain about it even if it is "free" (advertiser supported and a data stripmine). Facebook isn't different and just because it isn't critical to you, it doesn't mean it isn't to them.
Now, I agree to an extent, I pay for Google Apps even though the stability isn't any better than Gmail's, but I do like to be able to rave that I will take my 14 cents a day away from Google if they don't treat me better as a customer.
Ah, look on my cheque book, ye mighty, and despair!
huxley - 19th February 2010 16:00 - #
You can see it for yourself in the page source code.
BigPipe is basically a javascript based delivery system. Their pages don't consist of the content, they consist of a HTML framework upon which they can place the content. The content is delivered streaming via a second connection and then placed onto the page via the BigPipe javascript.
So if you view source on their main page, for example, then search for, say, "News Feed", you won't find it. The content of those columns was delivered after the main page was, then placed onto the page by BigPipe.
Otto - 19th February 2010 16:12 - #
It would be fun to setup a distributed page rendering system with something like this. The core logic could handle layout and stubbing out the HTML containers, and then you could distribute page requests for the individual pieces of container content across a bunch of servers. This could let you do some weird stuff, like have a page served by language A but built out of chunks served by languages B and C.
Nathan Ostgard - 19th February 2010 16:30 - #
I think progressively transmitting chunks of content isn't really the main win here... they could do that by means such as running I/O in the background and transmitting buffered output to the client whenever the template code blocks on its completion.
I think the big win by this JavaScript-based approach is the ability to parallelize the loading of certain parts of the page and deliver those parts as soon as they are ready, regardless of document order. This turns out to be good for reducing the impression of latency when transmitting data to a brower, since it reduces the time during which the majority of the page is blank due to one particular part of the page being more expensive.
This technique seems almost useful enough to justify it becoming a core feature of HTML so that you can declaratively include chunks of content from elsewhere and have the browser go fetch it and insert it rather than relying on hand-written scripts.
@Nathan: this is exactly what Google does for search results.
Jo - 20th February 2010 23:03 - #
I wonder how it compares with Digg's experiments using DUI.stream, it seems like a similar concept:
http://about.digg.com/blog/duistream-and-mxhr
Similarly to some other commenters, I've noticed a lot of behavioral failures on facebook recently. Very interesting experiment though.
Dave Concannon - 21st February 2010 02:31 - #
huxley,
I agree, however I wouldn't go and find all GMail engineer posts and start complaining on them. I'd have to suck it up, and start using POP3 for my work email again.
Mark Swannie - 21st February 2010 07:03 - #
I've just released our own bigpipe implementation at http://www.juhonkoti.net/2010/10/01/open-bigpipe-j avascript-implementation
All feedback is greatly appreciated.
Juho Mäkinen - 1st October 2010 12:18 - #
martina - 22nd June 2011 14:25 - #
martina - 11th July 2011 10:42 - #
This is a very informative article.I was looking for these things and here I found it. I am doing a project and this information is very useful me. If you are interested, but this is my duty to inform you that virtual administrative assistant a very dedicated service and can be applied anywhere you want and get better results.
laptops deal - 10th October 2011 19:52 - #
This is a very informative article.I was looking for these things and here I found it. I am doing a project and this information is very useful me. If you are interested, but this is my duty to inform you that virtual administrative assistant a very dedicated service and can be applied anywhere you want and get better results.
laptops deal - 10th October 2011 19:52 - #
great post
bahamas charter flights - 15th October 2011 04:33 - #
Jody - 5th November 2011 22:12 - #