cloudflare on AnyStack

How to install a Mastodon instance on Ubuntu 16.04 LTS

Thu, 06 Apr 2017 10:06:00 +0000

Have you already the new open-source social network Mastodon ? I have launched few days ago my own instance to contribute to this project.

If you want to launch your own instance, here a quick tutorial to install mastodon on a Ubuntu 16.04 LTS server with Nginx as a reverse-proxy and Cloudflare CDN/SSL.

To follow the steps of this tutorial, a user with sudo rights is enough, you don’t need to login as root.

Install the dependencies

sudo apt-get update

Install nodejs

curl -sL https://deb.nodesource.com/setup_6.x | sudo bash -
sudo apt-get install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs
sudo npm install -g yarn

Install redis-server

sudo apt-get install redis-server redis-tools

Install Postgresql

sudo apt-get install postgresql postgresql-contrib

sudo su - postgres
psql
CREATE USER mastodon CREATEDB;
\q

You will also have to enable ident authentication so users can login without password

sudo sed -i '/^local.*postgres.*peer$/a host all all 127.0.0.1/32 ident' \
/etc/postgresql/9.?/main/pg_hba.conf

And to install the ident daemon :

sudo apt-get install pidentd
sudo systemctl enable pidentd
sudo systemctl start pidentd
sudo systemctl restart postgresql

Install Ruby

Install the dependencies

sudo apt-get install autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev

Create a user mastodon without password :

adduser --disabled-password --disabled-login mastodon

Then login as mastodon :

su - mastodon

Install rbenv and rbenv-build

git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
source ~/.bashrc

We have to logout to apply the bash modification, and we install ruby-build.

exit
su - mastodon
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

Then install ruby 2.4.1 for mastodon :

rbenv install 2.4.1
rbenv global 2.4.1

You can check the ruby version installed with the command :

ruby -v

Mastodon installation

We have installed all the dependencies required to setup Mastodon, so we can now clone the github repository. We use git checkout to select the last release instead of the master branch.

cd ~
git clone https://github.com/tootsuite/mastodon.git live
cd live
git checkout $(git tag | tail -n 1)

Install bundler to manage the dependencies and disable the gem documentation

echo "gem: --no-document" > ~/.gemrc
gem install bundler --no-ri

Then launch the installation

bundle install --deployment --without development test
yarn install

You can now copy the configuration sample file and edit with your own informations

cp .env.production.sample .env.production
nano .env.production

You have to add the following informations :

# Service dependencies
REDIS_HOST=localhost
REDIS_PORT=6379
DB_HOST=/var/run/postgresql
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=
DB_PORT=5432

# Federation
LOCAL_DOMAIN=yourdomain.com
LOCAL_HTTPS=true

Ainsi que la partie SMTP qui permettra aux utilisateurs de confirmer leur inscription :

# E-mail configuration
SMTP_SERVER=mail.yourdomain.com
SMTP_PORT=587
SMTP_LOGIN=noreply@yourdomain.com
SMTP_PASSWORD=YourPassword
SMTP_FROM_ADDRESS=noreply@yourdomain.com

For the application secret part, you can use the command bundle exec rake secret to generate the 3 secret keys, then you have just to copy them into the configuration file.

To setup the database and the assets :

RAILS_ENV=production bundle exec rails db:setup
RAILS_ENV=production bundle exec rails assets:precompile

Adding systemd services

Web service

nano /etc/systemd/system/mastodon-web.service

[Unit]
 Description=mastodon-web
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="RAILS_ENV=production"
 Environment="PORT=3000"
 ExecStart=/home/mastodon/.rbenv/shims/bundle exec puma -C config/puma.rb
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

Background service

nano /etc/systemd/system/mastodon-sidekiq.service

[Unit]
 Description=mastodon-sidekiq
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="RAILS_ENV=production"
 Environment="DB_POOL=5"
 ExecStart=/home/mastodon/.rbenv/shims/bundle exec sidekiq -c 5 -q default -q mailers -q pull -q push
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

API service

nano /etc/systemd/system/mastodon-streaming.service

[Unit]
 Description=mastodon-streaming
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="NODE_ENV=production"
 Environment="PORT=4000"
 ExecStart=/usr/bin/npm run start
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

Then we can enable our systemd services :

systemctl enable /etc/systemd/system/mastodon-*.service

And we can start our mastodon instance :

sudo systemctl start mastodon-web.service mastodon-sidekiq.service mastodon-streaming.service

Add the required crons for mastodon :

crontab -u mastodon -e

RAILS_ENV=production
@daily cd /home/mastodon/live && RAILS_ENV=production /home/mastodon/.rbenv/shims/bundle exec rails mastodon:media:remove_remote

Nginx reverse-proxy setup :

So we have setup mastodon, but to access to our instance directly under https and with our domain, we need to setup a reverse-proxy using Nginx.

At first, install Nginx :

wget -O - https://nginx.org/keys/nginx_signing.key | sudo apt-key add -
sudo echo "deb http://nginx.org/packages/ubuntu/ $(lsb_release -sc) nginx" > /etc/apt/sources.list.d/nginx.list
sudo apt update
sudo apt install nginx

We have now to create our nginx configuration file for ou domain, we will use the configuration based on the Angristan’s model :

nano /etc/nginx/sites-enabled/yourdomain.com

map $http_upgrade $connection_upgrade {
 default upgrade;
 '' close;
}
server {
 listen 80;
 listen [::]:80;
 server_name www.yourdomain.com yourdomain.com;
 return 301 https://votredomaine.com$request_uri;

 access_log /dev/null;
 error_log /dev/null;
}

server {
 listen 443 ssl http2;
 listen [::]:443 ssl http2;
 server_name www.yourdomain.com yourdomain.com;

 access_log /var/log/nginx/yourdomain.com-access.log;
 error_log /var/log/nginx/yourdomain.com-error.log;

 ssl_certificate /etc/letsencrypt/live/fullchain.pem;
 ssl_certificate_key /etc/letsencrypt/live/privkey.pem;
 ssl_protocols TLSv1.2;
 ssl_ciphers EECDH+AESGCM:EECDH+AES;
 ssl_prefer_server_ciphers on;
 add_header Strict-Transport-Security "max-age=15552000; preload";

 keepalive_timeout 70;
 sendfile on;
 client_max_body_size 0;
 gzip off;

 root /home/mastodon/live/public;

 location / {
 try_files $uri @proxy;
 }

 location @proxy {
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto https;
 proxy_pass_header Server;
 proxy_pass http://127.0.0.1:3000;
 proxy_buffering off;
 proxy_redirect off;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection $connection_upgrade;
 tcp_nodelay on;
 }

 location /api/v1/streaming {
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto https;
 proxy_pass http://127.0.0.1:4000;
 proxy_buffering off;
 proxy_redirect off;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection $connection_upgrade;
 tcp_nodelay on;
 }

 error_page 500 501 502 503 504 /500.html;
}

You can start nginx with the command :

service nginx start

Then we will use Let’s Encrypt to generate a SSL certificate.

cd /opt
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto certonly --webroot -w /home/mastodon/live -d www.yourdomain.com -d yourdomain.com --email vous@yourdomain.com --text --rsa-key-size 4096

Updating Mastodon

Mastodon is updated very often, so if you want to keep your instance up-to-date you will have to follow the following steps :

su - mastodon
cd live
gem install bundler --no-ri
git fetch
git checkout v1.X.X
bundle install --deployment --without development test
NODE_ENV=production npm upgrade --global yarn
yarn install
RAILS_ENV=production bundle exec rails assets:clean
RAILS_ENV=production bundle exec rails assets:precompile
RAILS_ENV=production bundle exec rails db:migrate
exit

Then you just have to restart the instance :

sudo systemctl restart mastodon-web.service mastodon-sidekiq.service mastodon-streaming.service

If you need to update ruby with a new release use :

rbenv install 2.4.1
gem install bundler --no-ri

Create your administrator account

You instance is now running properly but you have to register to create your account and then you will be able to set this account as administrator with the following command :

RAILS_ENV=production bundle exec rails mastodon:make_admin USERNAME=votre-utilisateur

My instance mastodon.top

![mastodon.top/images/2017/08/Screenshot_44.png) With VirtuBox, I have launched a Mastodon instance and if you are not registered on Mastodon yet, you can register at mastodon.top. We are already more than 200 users on my instance and enough resources for more users. You can find more informations about the instance on the page about and the real-time monitoring of the server at instance.mastodon.top.

Don’t hesitate to give your opinion on Mastodon, to tell me if there are errors in the tutorial or to ask for help if you want to deploy your own instance.

You can also follow me on Mastodon : thomas_virtubox@mastodon.top

Good bye WordPress - Welcome Ghost !

Wed, 26 Oct 2016 13:54:29 +0000

I’m using WordPress for few years now, but that’s always the same problem, WordPress require plugins to have some functionalities and plugins make WordPress running slower.

Even with an advanced caching system like redis, that’s always a lot of work to make wordpress running fast and safe.

So I have done some tests with Ghost using Docker, and it’s really the perfect blog solution for me, because it already include all the features needed to write an article. No need to look for a plugin to improve SEO or to install a caching system. You can write you articles in markdown, work with slack and use it as a team, and the theme system is really clean and lightweight.

But I can’t post this article without making a tutorial about Ghost.

1) Install NodeJs v4 LTS

wget https://deb.nodesource.com/setup_4.x
chmod +x setup_4.x
./setup_4.x
apt-get install nodejs -y

2) Install Ghost

mkdir /var/www/ghost
cd /var/www/ghost
curl -L https://ghost.org/zip/ghost-latest.zip -o ghost.zip
unzip ghost.zip
npm install --production

3) Starting Ghost

npm start --production

Then you can access to your blog using http://localhost:2368/ghost/signup All the configuration can be found in the file config.js.

But as you can see, we have to use the port 2368 to access to Ghost. To fix that, the easiest is to use Nginx as reverse proxy. Let’s use easyengine !

## install easyengine
wget -qO ee rt.cx/ee && bash ee

## create a proxy with your domain
ee site create anystack.xyz --proxy=127.0.0.1:2368

Then you will be able to use your domain to access to ghost. Don’t forget to change your blog url in config.js. And if you want to run Ghost forever, so use forever …

npm install forever -g
## start ghost
NODE_ENV=production forever start index.js

## stop ghost
forever stop index.js

Last tips ? How to use Ghost with Nginx as reverse proxy with Cloudflare SSL. Go into you Cloudflare account and in the Crypto tabs, create your origin certificate.

Then create a first file yourdomain-pvkey.pem with the private key generated by Cloudflare. And another file yourdomain-crt.pem with the origin certificate from cloudflare but also the Cloudflare root certificate you can find here

Then your nginx conf for you domain should look like

server {

 server_name anystack.xyz;
 listen 443 ssl http2;
 ssl on;
 ssl_certificate /etc/nginx/ssl/anystack-crt.pem;
 ssl_certificate_key /etc/nginx/ssl/anystack-key.pem;
 access_log /var/log/nginx/dev.anystack.xyz.access.log rt_cache;
 error_log /var/log/nginx/dev.anystack.xyz.error.log;

 location / {
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Host $http_host;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_pass http://127.0.0.1:2368/;
 }

}

And if you want to add a redirect from http to a https, create a file yourdomain.com-ssl.conf in /etc/nginx/conf.d with

	server {
	listen 80;
	server_name anystack.xyz;
	return 301 https://anystack.xyz$request_uri;
}

HTTP2 & Web Optimization - How to have a fast website in 2016

Thu, 11 Aug 2016 06:48:00 +0000

Few years ago, the HTTPs protocol was slower than the http, due the the initial handshake of any SSL transaction. But with the HTTP2, there are several improvements available and some rules have change when we talk about web optimization. Most of the PageSpeed tools will recommend you to use domain sharding or to combine your assets to have a website faster. But is it still true ?

The most important feature of the HTTP2 is the multiplexing, it allow to download multiple files with an unique TCP connection when we have to open a new one for each file in HTTP1.1

1) No more assets concatenation

We used to combine as much as possible our assets with HTTP1.1, that’s not true with HTTP2 anymore. Because if it reduce the size of the requests done, it also make the rules for caching harder to define and could have bad results if you don’t need a part of the assets on each page of your website

2) No more domain sharding

It was already a “trick” but the domain sharding is not a part of the past. It was used to open more TCP connection than a browser was supposed to. But you will have a negative result by using it with HTTP2 because it will only require unnecessary DNS request and more if the connection is established under TLS.

But there are also rules which are still Best Pratices to have a fast website

Reduce DNS lookup -> store all assets on the same domain or with a CDN
Browser Caching
Minimize HTTP Requests -> cookies and query strings
Minimize HTTP Response -> Minify your assets

How Cloudflare speed up and protect your website - Part 1.

Mon, 01 Aug 2016 18:57:00 +0000

You don't know what is Cloudflare ? So let's see how cloudflare make your website loading faster, and why you should already use it for all your websites.

) Cloudflare DNS - Free & Fast

Cloudflare provide at first one of the best DNS hosting service available. And to make a website faster you can already start with using a fast DNS hosting.

DNS Provider Speed Comparison by solvedns.com

![/images/2016/10/draw.jpg)

It's only few Milliseconds but as you can see, Cloudflare is faster than many other providers and most of them offer paid services only. So before paying few dollars for a Premium DNS package, you should try cloudflare. Another feature you will love, is a propagation time around 5 minutes when you edit a record on your DNS. So if you need to migrate your website soon, this is really a good reason to register now.

2. ) Cloudflare - Reverse Proxy with caching

But cloudflare does not provide only DNS Hosting. It's also a complete solution to speed up and secure your website. For that when Cloudflare manage your DNS if you activate the optimization and security (orange cloud), instead of sending your server IP when a visitor make a DNS request, it will act as a proxy and all your visitors will view only the Cloudflare servers IP when they do a request.

![/images/2016/10/overview.png)

And Cloudflare will not only act as a simple proxy, it will also cache all the static files and deliver them all around the World to your visitors from the nearest Cloudflare datacenter.

The result is awesome because it will make your website faster as it will handle less requests, and it will also save a lot of bandwidth by limiting the number of file requests.

As you can see, with 80 000 visitors per month, it's more than 20GB of bandwidth saved and also more than 20GB of static assets which have been delivered faster.

3) Cloudflare - Website Protection

By acting as a proxy Cloudflare also provide a complete security solution which include :

Browser integrity check : Each visitor request is checked using the HTTP header and if a thread is found it can be challenged using a captcha or blocked.
Visitor reputation : Cloudflare analyze daily thousands of IP, and if a visitor try several times to perform any form of hack, it will be marked as a potential thread for Cloudflare
Page rules : You can use the page rules to allow or disallow access to a part of your website. For example with wordpress your can use a high level of secuirty onthe page http://yourdomain.com/wp-login.php to protect your website.
DDoS protection : As your visitors have to use cloudflare servers to get access to your website, they will never have access to your server IP and any attack attempt will simply be blocked by the Cloudflare servers.

But to protect totally your server, you have to be sure there is not any records which point directly to your server. The most common error will be to have a email server with a MX record pointing on your server IP. In this case, Cloudflare will not be able to protect your website because the attacker will send the traffic directly on your server IP, and will not have to use the Cloudflare proxy.

4) Cloudflare - Website Optimization

And if it wasn't enough, Cloudflare is also able to optimize your website. It can minify all your assets, including the html, reduce the number of connections, and with rocket loader it will make deliver your assets asynchronously. That mean instead of waiting for each file to be downloaded, with cloudflare your browser will download all the assets in the same time when you visit your website, and it will be up to 3x times faster.

The last optimization tool available for free is Rocket Loader, currently in beta, it's a script used by cloudflare to deliver your assets, without blocking the page display. It use the asynchronous loading to load your javascript at the end of the page display and this way it will really improve your pagespeed score. But like any beta, it could create error with some javascript or break your website design by loading too late some assets.

For this reason you have 2 mode to use Rocket loader :
- Automatic : Rocket loader will optimize any javascript available on your website but you have the ability to forbid the optimization manually by adding data-cfasync="false" before the script source.
```
<script type="text/javascript" data-cfasync="false" src="/js/tether.min.js"></script>
```
Manual : This time you will have to add data-cfasync=“true” for each script you want to optimize with Rocket Loader

I will complete this article with the HTTP2 and Cloudflare in the next few days.

cloudflare on AnyStack

How to install a Mastodon instance on Ubuntu 16.04 LTS

Install the dependencies

Install Ruby

Mastodon installation

Adding systemd services

Nginx reverse-proxy setup :

Updating Mastodon

Create your administrator account

My instance mastodon.top

Good bye WordPress - Welcome Ghost !

1) Install NodeJs v4 LTS

2) Install Ghost

3) Starting Ghost

HTTP2 & Web Optimization - How to have a fast website in 2016

1) No more assets concatenation

2) No more domain sharding

How Cloudflare speed up and protect your website - Part 1.

) Cloudflare DNS - Free & Fast

2. ) Cloudflare - Reverse Proxy with caching

3) Cloudflare - Website Protection