Scaling webservices – Part 1 – Installing Nginx + MySQL + PHP5 on Debian/Ubuntu

This guide was mostly created as a reference for setting up fanart.tv servers, but I hope others will also find it useful, among the things covered are setting up Nginx, MySQL, PHP5, Rysnc without passwords, MySQL database replication, and MySQL backups.

As our site got bigger, trying to run everything on 1 server proved too much for it to handle so we needed to move some of the services onto different servers but still all work as a team.

Our main server runs on Ubuntu and uses Apache2 but for serving static files and for serving lots of simultaneous connections Nginx is more lightweight and suited to the job, that is why we have 2 extra servers, 1 for Images and 1 dedicated to the API, we are also using a base Debian install for these servers.

If you want to follow these steps on an Ubuntu machine simply type “sudo su” at the command line.

Get everything up to date

nano /etc/apt/sources.list

Add to the bottom:

deb http://packages.dotdeb.org stable all
wget http://www.dotdeb.org/dotdeb.gpg
cat dotdeb.gpg | apt-key add -
rm dotdeb.gpg
apt-get update
apt-get upgrade

Update locale

If the system isnt in english do:

export LANG=en_GB.UTF-8

Then

dpkg-reconfigure locales

Add www to skel

adding www to skel will mean when a new user is added the www folder will be added to their home directory, we will use this for storing the data (on our debian installs /home has the majority of the disk space, so it makes sense to store the data there)

mkdir /etc/skel/www

Add a user

In our case the user we are adding is called fanart, change this to whatever you want, but just be aware to change it in all the places it’s used inthe rest of the guide

adduser fanart

Install MySQL

apt-get install mysql-server mysql-client

Install Nginx

apt-get install nginx

Install PHP

apt-get install php5 php5-fpm php-pear php5-common php-apc php5-mcrypt php5-mysql php5-cli php5-gd php5-curl php5-imagick php5-mcrypt php5-memcache

Set up the vhost

In our case the vhost is assets.fanart.tv change this to whatever you are going to be using

cp /etc/nginx/sites-available/default /etc/nginx/sites-available/assets.fanart.tv

The default file has a lot of crap in it so you might want to clean up your vhost first by removing all the commented out lines, you still have default if you need to reference anything, then either add the following (or uncomment if it’s in the file)

nano /etc/nginx/sites-available/assets.fanart.tv

Change the index line so it has index.php in it first

index index.php index.html index.htm;
location ~ \.php$ {
   fastcgi_split_path_info ^(.+\.php)(/.+)$;
   fastcgi_pass 127.0.0.1:9000;
   fastcgi_index index.php;
   include fastcgi_params;
}

At this point the site is onlu available, we need to set it to enabled, to do this we create a symlink:

ln -s /etc/nginx/sites-available/assets.fanart.tv /etc/nginx/sites-enabled/assets.fanart.tv

Install phpMyAdmin

phpMyAdmin makes it a bit easier to set up things, so install it

apt-get install phpmyadmin

You will see the following questions:

Web server to reconfigure automatically:
don’t select anything, just tab to the OK as neither are installed

Then it will ask
Configure database for phpmyadmin with dbconfig-common?

Select No

You can now find phpMyAdmin in the /usr/share/phpmyadmin/ directory. Now we must configure our vhost so that nginx can find phpMyAdmin in that directory.

nano /etc/nginx/sites-available/assets.fanart.tv

Then add the following before the last }

location /phpmyadmin {
root /usr/share/;
index index.php index.html index.htm;
location ~ ^/phpmyadmin/(.+\.php)$ {
try_files $uri =404;
root /usr/share/;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include /etc/nginx/fastcgi_params;
}
location ~* ^/phpmyadmin/(.+\.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt))$ {
root /usr/share/;
}

}
location /phpMyAdmin {
rewrite ^/* /phpmyadmin last;
}

Also add any other rewrites you need, on our api server we added

location /webservice/newmovies {
root /home/www/;
rewrite ^/webservice/newmovies/(.*)/(.*)/$ /webservice/newmovies.php?key=$1&date=$2 last;
rewrite ^/webservice/newmovies/(.*)/$ /webservice/newmovies.php?key=$1 last;
}

location /webservice/newtv {
root /home/www/;
rewrite ^/webservice/newtv/(.*)/(.*)/$ /webservice/newtv.php?key=$1&date=$2 last;
rewrite ^/webservice/newtv/(.*)/$ /webservice/newtv.php?key=$1 last;
}

location /webservice/ {
root /home/www/;
rewrite ^/webservice/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3&format=$4&type=$5&sort=$6&limit=$7&update=1 last;
rewrite ^/webservice/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3&format=$4&type=$5&sort=$6&limit=$7 last;
rewrite ^/webservice/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3&format=$4&type=$5&sort=$6 last;
rewrite ^/webservice/(.*)/(.*)/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3&format=$4&type=$5 last;
rewrite ^/webservice/(.*)/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3&format=$4 last;
rewrite ^/webservice/(.*)/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2&id=$3 last;
rewrite ^/webservice/(.*)/(.*)/ /webservice/newapi.php?section=$1&key=$2 last;
rewrite ^/webservice/(.*)/ /webservice/newapi.php?section=$1 last;
}

Performance tuning

The following made a MASSIVE difference to the cpu load on the API server, each php5-fpm process went from using 20-40 %CPU to between 4-12 (and usually around 4-6), so if you are seeing high load in php5-fpm processes have a try with the following settings, bare in mind though, the server has 16GB or RAM, so you may need to adjust the settings.

first you need to edit the APC config:

nano /etc/php5/conf.d/apc.ini

In my case it just had extension=apc.so, in any case if the following directives dont exist in the file add them to the bottom else change them.

apc.shm_size=512
apc.shm_segments=2

This changes the shm_size from the default 32M to 512M, so guides I found said to use M but that resulted in php5-fpm refusing to reload for me, and checking the php documentation it states this should be an integer expressing the size in MB.

then start nginx

service nginx start
service php5-fpm reload

If you get 502 bad gateway go to /etc/php5/fpm/pool.d/www.conf and edit the line listen = /var/run/php5-fpm.sock to listen = 127.0.0.1:9000