Our developer Vladimir Mladenovic shares his advice on how to use domain sharding when you experience poor performance with your application or website.
Vladimir is specialised in Symfony development and Zend PHP frameworks, jQuery, Doctrine, and MySQL. He’s our team leader responsible for GIT maintenance, quick server interventions and code organisation and quality.
Read on as he will explain to you how to use domain sharding for performance optimization.
What do you do when your website or application shows signs of poor performance?
Act fast, and accordingly.
We’ve recently noticed that one of our applications, launched a year ago, saw some performance problems.
The problem first appeared with more than 5000 users accessing the application intensively. Having a large number of worldwide users accessing the application at once resulted in a slower loading time for some web pages and an increase in the number of requests (images, JavaScript, CSS files) sent to a web server.
We analysed the web server logs only to confirm what we were suspicious about in the first place – the part of the problem was high CPU utilization and a weaker hardware.
We advised our clients to upgrade to a better hosting package (a 4-core processor, SDD disk, and 16GB RAM memory) which improved hardware and application performance, however, it still showed signs of slow performance, which we weren’t pleased with.
To address this problem thoroughly, we concluded that the application also needs an easy-to-implement and cost-efficient Content Delivery Network (CDN), and we decided to use CloudFlare free plan.
Why CloudFlare?
The network operates out of 34 data centers around the world, automatically caches the static files, and distribute these files from the locations that are the closest to the visitor while delivering the dynamic content directly from our production server.
To accomplish this, the network uses a technology called Anycast to route your visitors to the nearest data center.
CloudFlare integration and domain redirection were simple and fast.
And while we were working with CloudFlare, we came up with the idea for a new optimization strategy – domain sharding.
Why?
Web browsers can limit the number of requests sent to a particular web host (see image 1).
Such limitations lead to an increasing number of resources that are blocked while waiting to be downloaded. See the common pattern in the waterfall diagram (image 2):
Domain sharding offers one of the solutions for this problem by splitting content across multiple subdomains to simultaneously download static resources.
With domain sharding, one can decrease the time spent waiting for resources to download and improve page load time by more than 30% (image 3).
However, domain sharding can be implemented incorrectly, and if so, you can expect to see a negative performance of your web application.
To avoid this, you should keep in mind several situations:
- Domain sharding increases the number of DNS lookups – too many implemented shards can hurt application performance and the application will perform better without sharding
- Static resources should be consistently served from the same sub-domains – this way you will not lose the benefits of caching
- The resources should be distributed evenly between sharded hosts – if not, such optimization will have no effect because the number of blocking requests will increase
- SSL certificates for the sub-domains are needed for an HTTPS site (the added cost of buying)
- Due to parallel requests sent to the server, you will experience a higher server load
- The same origin policy – when you move your static content onto a separate domain or subdomain, a code can break if it depends on the data from the same origin
Domain sharding integration is very simple as my web application development team uses the Symfony2 framework that offers excellent options for sharding integration.
If you are using Symfony2 and Assetic for serving static resources, you can define assets_base_urls parameter (HTTP and SSL) to manipulate the files (in theory).
However, we had to write a specific Twig function for our application as the images are served separately from Assetic. The function has a different SRC parameter for image input and new output image parameter served from sharding. Twig function shows how to distribute the resources and benefit from caching evenly.
Whether you use Symfony PHP or not, here is a good example of how to implement sharding algorithm:
[raw]
function shard($url, $shards) {
$md5 = md5($url);
$md5_32 = 0;
for ($i = 0; $i <= 3; $i++) {
$md5_32 ^= hexdec(substr($md5, $i * 8, 8));
}
$shard = $md5_32 % $shards;
return "https://static$shard.gtmetrix.com/$url";
}[/raw]
But know this – while you are implementing domain sharding, shards should be cookieless. Moreover, to take advantage of domain sharding, you should block all server requests but requests for static resources.
If you are using CloudFlare CDN, unfortunately, it’s almost impossible to implement cookieless domains. On the other hand, if you are using Apache server, with mod_expire and mod_headers modules you can have better control over resources and, for example, set caching for specific resources, or something similar.
If you want to optimise your web page properly, the first thing to do would be to reduce the overall number of resource requests.
Symfony2 has a great component I already mentioned, Assetic, in addition to uglifyjs i uglifycss filters that can be easily integrated into it. Below you can see the example of combining a larger number of JS databases into one single file using uglifyjs filter.
If you are using Symfony2, you can also use ESI (Edge Side Includes) to additionally control HTTP caching. With ESI, you can cache page fragments independently – this way you will be caching a web page for 60 minutes and embedded sidebar only five minutes. Activate ESI in the main config.yml and call fragments in Twig with render_esi.
If you are using Doctrine, you can significantly improve performance within a database with Doctrine cache. However, to avoid possible bugs, you need to pay attention to invalidate Doctrine cache properly.
We use MySQL indexes to speed up database query – this is a good way to take advantage of MySQL. Symfony toolbar, on the other hand, helps greatly with locating problematic inquiries. The hardest part of implementation is experimenting and adding indexes in Doctrine mapping.
If some of your pages are still loading slowly, and you are having a hard time determining what causes poor performance, I advise using profiler tools.
One such quality tool for PHP is Blackfire Profiler – easy-to-use, intuitive and interactive.
If you pay attention to the red fields in the graph, you will see the problem.
Good luck with optimisation!
