nginx on AnyStack

How to install a Mastodon instance on Ubuntu 16.04 LTS

Thu, 06 Apr 2017 10:06:00 +0000

Have you already the new open-source social network Mastodon ? I have launched few days ago my own instance to contribute to this project.

If you want to launch your own instance, here a quick tutorial to install mastodon on a Ubuntu 16.04 LTS server with Nginx as a reverse-proxy and Cloudflare CDN/SSL.

To follow the steps of this tutorial, a user with sudo rights is enough, you don’t need to login as root.

Install the dependencies

sudo apt-get update

Install nodejs

curl -sL https://deb.nodesource.com/setup_6.x | sudo bash -
sudo apt-get install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs
sudo npm install -g yarn

Install redis-server

sudo apt-get install redis-server redis-tools

Install Postgresql

sudo apt-get install postgresql postgresql-contrib

sudo su - postgres
psql
CREATE USER mastodon CREATEDB;
\q

You will also have to enable ident authentication so users can login without password

sudo sed -i '/^local.*postgres.*peer$/a host all all 127.0.0.1/32 ident' \
/etc/postgresql/9.?/main/pg_hba.conf

And to install the ident daemon :

sudo apt-get install pidentd
sudo systemctl enable pidentd
sudo systemctl start pidentd
sudo systemctl restart postgresql

Install Ruby

Install the dependencies

sudo apt-get install autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev

Create a user mastodon without password :

adduser --disabled-password --disabled-login mastodon

Then login as mastodon :

su - mastodon

Install rbenv and rbenv-build

git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
source ~/.bashrc

We have to logout to apply the bash modification, and we install ruby-build.

exit
su - mastodon
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

Then install ruby 2.4.1 for mastodon :

rbenv install 2.4.1
rbenv global 2.4.1

You can check the ruby version installed with the command :

ruby -v

Mastodon installation

We have installed all the dependencies required to setup Mastodon, so we can now clone the github repository. We use git checkout to select the last release instead of the master branch.

cd ~
git clone https://github.com/tootsuite/mastodon.git live
cd live
git checkout $(git tag | tail -n 1)

Install bundler to manage the dependencies and disable the gem documentation

echo "gem: --no-document" > ~/.gemrc
gem install bundler --no-ri

Then launch the installation

bundle install --deployment --without development test
yarn install

You can now copy the configuration sample file and edit with your own informations

cp .env.production.sample .env.production
nano .env.production

You have to add the following informations :

# Service dependencies
REDIS_HOST=localhost
REDIS_PORT=6379
DB_HOST=/var/run/postgresql
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=
DB_PORT=5432

# Federation
LOCAL_DOMAIN=yourdomain.com
LOCAL_HTTPS=true

Ainsi que la partie SMTP qui permettra aux utilisateurs de confirmer leur inscription :

# E-mail configuration
SMTP_SERVER=mail.yourdomain.com
SMTP_PORT=587
SMTP_LOGIN=noreply@yourdomain.com
SMTP_PASSWORD=YourPassword
SMTP_FROM_ADDRESS=noreply@yourdomain.com

For the application secret part, you can use the command bundle exec rake secret to generate the 3 secret keys, then you have just to copy them into the configuration file.

To setup the database and the assets :

RAILS_ENV=production bundle exec rails db:setup
RAILS_ENV=production bundle exec rails assets:precompile

Adding systemd services

Web service

nano /etc/systemd/system/mastodon-web.service

[Unit]
 Description=mastodon-web
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="RAILS_ENV=production"
 Environment="PORT=3000"
 ExecStart=/home/mastodon/.rbenv/shims/bundle exec puma -C config/puma.rb
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

Background service

nano /etc/systemd/system/mastodon-sidekiq.service

[Unit]
 Description=mastodon-sidekiq
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="RAILS_ENV=production"
 Environment="DB_POOL=5"
 ExecStart=/home/mastodon/.rbenv/shims/bundle exec sidekiq -c 5 -q default -q mailers -q pull -q push
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

API service

nano /etc/systemd/system/mastodon-streaming.service

[Unit]
 Description=mastodon-streaming
 After=network.target

[Service]
 Type=simple
 User=mastodon
 WorkingDirectory=/home/mastodon/live
 Environment="NODE_ENV=production"
 Environment="PORT=4000"
 ExecStart=/usr/bin/npm run start
 TimeoutSec=15
 Restart=always

[Install]
 WantedBy=multi-user.target

Then we can enable our systemd services :

systemctl enable /etc/systemd/system/mastodon-*.service

And we can start our mastodon instance :

sudo systemctl start mastodon-web.service mastodon-sidekiq.service mastodon-streaming.service

Add the required crons for mastodon :

crontab -u mastodon -e

RAILS_ENV=production
@daily cd /home/mastodon/live && RAILS_ENV=production /home/mastodon/.rbenv/shims/bundle exec rails mastodon:media:remove_remote

Nginx reverse-proxy setup :

So we have setup mastodon, but to access to our instance directly under https and with our domain, we need to setup a reverse-proxy using Nginx.

At first, install Nginx :

wget -O - https://nginx.org/keys/nginx_signing.key | sudo apt-key add -
sudo echo "deb http://nginx.org/packages/ubuntu/ $(lsb_release -sc) nginx" > /etc/apt/sources.list.d/nginx.list
sudo apt update
sudo apt install nginx

We have now to create our nginx configuration file for ou domain, we will use the configuration based on the Angristan’s model :

nano /etc/nginx/sites-enabled/yourdomain.com

map $http_upgrade $connection_upgrade {
 default upgrade;
 '' close;
}
server {
 listen 80;
 listen [::]:80;
 server_name www.yourdomain.com yourdomain.com;
 return 301 https://votredomaine.com$request_uri;

 access_log /dev/null;
 error_log /dev/null;
}

server {
 listen 443 ssl http2;
 listen [::]:443 ssl http2;
 server_name www.yourdomain.com yourdomain.com;

 access_log /var/log/nginx/yourdomain.com-access.log;
 error_log /var/log/nginx/yourdomain.com-error.log;

 ssl_certificate /etc/letsencrypt/live/fullchain.pem;
 ssl_certificate_key /etc/letsencrypt/live/privkey.pem;
 ssl_protocols TLSv1.2;
 ssl_ciphers EECDH+AESGCM:EECDH+AES;
 ssl_prefer_server_ciphers on;
 add_header Strict-Transport-Security "max-age=15552000; preload";

 keepalive_timeout 70;
 sendfile on;
 client_max_body_size 0;
 gzip off;

 root /home/mastodon/live/public;

 location / {
 try_files $uri @proxy;
 }

 location @proxy {
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto https;
 proxy_pass_header Server;
 proxy_pass http://127.0.0.1:3000;
 proxy_buffering off;
 proxy_redirect off;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection $connection_upgrade;
 tcp_nodelay on;
 }

 location /api/v1/streaming {
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto https;
 proxy_pass http://127.0.0.1:4000;
 proxy_buffering off;
 proxy_redirect off;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection $connection_upgrade;
 tcp_nodelay on;
 }

 error_page 500 501 502 503 504 /500.html;
}

You can start nginx with the command :

service nginx start

Then we will use Let’s Encrypt to generate a SSL certificate.

cd /opt
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto certonly --webroot -w /home/mastodon/live -d www.yourdomain.com -d yourdomain.com --email vous@yourdomain.com --text --rsa-key-size 4096

Updating Mastodon

Mastodon is updated very often, so if you want to keep your instance up-to-date you will have to follow the following steps :

su - mastodon
cd live
gem install bundler --no-ri
git fetch
git checkout v1.X.X
bundle install --deployment --without development test
NODE_ENV=production npm upgrade --global yarn
yarn install
RAILS_ENV=production bundle exec rails assets:clean
RAILS_ENV=production bundle exec rails assets:precompile
RAILS_ENV=production bundle exec rails db:migrate
exit

Then you just have to restart the instance :

sudo systemctl restart mastodon-web.service mastodon-sidekiq.service mastodon-streaming.service

If you need to update ruby with a new release use :

rbenv install 2.4.1
gem install bundler --no-ri

Create your administrator account

You instance is now running properly but you have to register to create your account and then you will be able to set this account as administrator with the following command :

RAILS_ENV=production bundle exec rails mastodon:make_admin USERNAME=votre-utilisateur

My instance mastodon.top

![mastodon.top/images/2017/08/Screenshot_44.png) With VirtuBox, I have launched a Mastodon instance and if you are not registered on Mastodon yet, you can register at mastodon.top. We are already more than 200 users on my instance and enough resources for more users. You can find more informations about the instance on the page about and the real-time monitoring of the server at instance.mastodon.top.

Don’t hesitate to give your opinion on Mastodon, to tell me if there are errors in the tutorial or to ask for help if you want to deploy your own instance.

You can also follow me on Mastodon : thomas_virtubox@mastodon.top

Good bye WordPress - Welcome Ghost !

Wed, 26 Oct 2016 13:54:29 +0000

I’m using WordPress for few years now, but that’s always the same problem, WordPress require plugins to have some functionalities and plugins make WordPress running slower.

Even with an advanced caching system like redis, that’s always a lot of work to make wordpress running fast and safe.

So I have done some tests with Ghost using Docker, and it’s really the perfect blog solution for me, because it already include all the features needed to write an article. No need to look for a plugin to improve SEO or to install a caching system. You can write you articles in markdown, work with slack and use it as a team, and the theme system is really clean and lightweight.

But I can’t post this article without making a tutorial about Ghost.

1) Install NodeJs v4 LTS

wget https://deb.nodesource.com/setup_4.x
chmod +x setup_4.x
./setup_4.x
apt-get install nodejs -y

2) Install Ghost

mkdir /var/www/ghost
cd /var/www/ghost
curl -L https://ghost.org/zip/ghost-latest.zip -o ghost.zip
unzip ghost.zip
npm install --production

3) Starting Ghost

npm start --production

Then you can access to your blog using http://localhost:2368/ghost/signup All the configuration can be found in the file config.js.

But as you can see, we have to use the port 2368 to access to Ghost. To fix that, the easiest is to use Nginx as reverse proxy. Let’s use easyengine !

## install easyengine
wget -qO ee rt.cx/ee && bash ee

## create a proxy with your domain
ee site create anystack.xyz --proxy=127.0.0.1:2368

Then you will be able to use your domain to access to ghost. Don’t forget to change your blog url in config.js. And if you want to run Ghost forever, so use forever …

npm install forever -g
## start ghost
NODE_ENV=production forever start index.js

## stop ghost
forever stop index.js

Last tips ? How to use Ghost with Nginx as reverse proxy with Cloudflare SSL. Go into you Cloudflare account and in the Crypto tabs, create your origin certificate.

Then create a first file yourdomain-pvkey.pem with the private key generated by Cloudflare. And another file yourdomain-crt.pem with the origin certificate from cloudflare but also the Cloudflare root certificate you can find here

Then your nginx conf for you domain should look like

server {

 server_name anystack.xyz;
 listen 443 ssl http2;
 ssl on;
 ssl_certificate /etc/nginx/ssl/anystack-crt.pem;
 ssl_certificate_key /etc/nginx/ssl/anystack-key.pem;
 access_log /var/log/nginx/dev.anystack.xyz.access.log rt_cache;
 error_log /var/log/nginx/dev.anystack.xyz.error.log;

 location / {
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Host $http_host;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_pass http://127.0.0.1:2368/;
 }

}

And if you want to add a redirect from http to a https, create a file yourdomain.com-ssl.conf in /etc/nginx/conf.d with

	server {
	listen 80;
	server_name anystack.xyz;
	return 301 https://anystack.xyz$request_uri;
}

How to use Nginx as reverse proxy with Docker

Sun, 04 Sep 2016 06:57:00 +0000

I have already write some articles about EasyEngine, and I’m really in love with this command line tool, because it make Nginx easier to use than Apache. But I haven’t talk about all functionality of EE yet. If it provide you the ability to setup a WordPress website easily, you can also use EasyEngine to create a vhost for a static html website … Or use Nginx as a reverse proxy !

And that make of EE the most complete tool available to use Nginx without having to read thousands of documentation pages to understand how to create a vhost. But don’t worry, this article will not only talk about this awesome tool which is EasyEngine.

I have used docker several times in the past, because it’s the fastest way to test an application without having to setup a VPS especially for that. But to run several Docker containers in a single VPS, it could quickly be a problem because some container provide you the ability to use a domain like WordPress for example. And some other will use directly your VPS IP and the port of your choice, which is not very friendly when your app is running on http://192.168.0.1:3000 …

That’s why we will use Nginx to act as a reverse proxy for our Docker containers, and we will be able to run several containers on the same server and to access them with the domain of our choice. So the first step is to install Docker. Don’t worry you should not need to read the documentation if you are using a VPS with a recent linux distribution and a kernel 3.13+

### install Docker with a single command on debian/ubuntu/centos
wget -qO- https://get.docker.com/ | sh

Docker is now running on your server ! So let’s install EasyEngine :

## Install EasyEngine Debian7/8 or Ubuntu 14.04/16.04 LTS
wget -qO ee rt.cx/ee && bash ee

So we have now Docker and EasyEngine installed and we can launch our first Docker App before routing it with Nginx. We will start with Rocket.Chat, a complete live chat solution to build a community or to provide live support.

## Launch MongoDB container 
docker run --name db -d mongo:3.0 --smallfiles

## Run Rocket.Chat linked to the db and expose port 3000
docker run --name rocketchat -p 3000:3000 --env ROOT_URL=http://localhost --link db -d rocket.chat

So you should be able to access to Rocket.Chat using http://YOUR-VPS-IP:3000. And to use Nginx as reverse proxy with your domain :

ee site create yourdomain.com --proxy=127.0.0.1:3000

That’s it ! You can now use http://yourdomain.com to access to Rocket.Chat

Some other apps to run with Nginx as reverse proxy :

## Hastebin
docker run --name redis -d redis
docker run --name hastebin -d -p 7777:7777 --link redis:redis -e STORAGE_HOST=redis rlister/hastebin

## Pasteboard
docker run --name pasteboard -e ORIGIN=mydomain.tld -e MAX=7 -v /srv/pasteboard/images:/pasteboard/public/storage -p 4000:4000 anthodingo/docker-pasteboard

New VPS for the Blog - Setup Guide

Fri, 05 Aug 2016 18:59:00 +0000

I was using a VPS with Plesk Onyx to host the Blog, but even with Redis in a Docker container to make it faster, it wasn’t enough …

So I have setup a small VPS on my dedicated server with Ubuntu 16.04 LTS using VMware vSphere 6. I will make another post about virtualization but in this post, we will talk only about the VPS setup with easyengine, which is really my favorite tool for WordPress setup. You can also read my first post about easyengine : Here

First step, easyengine install :

 wget -qO ee rt.cx/ee && bash ee

Then WordPress setup :

 ee site create anystack.xyz --wpredis --php7 --letsencrypt

And it’s already good for the server setup with nginx, php7, mariadb, redis-cache and let’s encrypt for the SSL. Check your DNS are pointing to your server before launching let’s encrypt or it will fail to setup the SSL certificate.

But the work is not finished yet, because I have to upload my database and my files to get my content back. For that i will need to have access to the database, so let’s do that with phpmyadmin :

 ee stack install --phpmyadmin

phpmyadmin is now available at http://my-server-ip:22222

Butt protected by an ACL, and to change the user and the password, you just have to use :

 ee secure --auth

And to have your login details for phpmyadmin :

 ee site info anystack.xyz

So I have now uploaded my database, and I need my files. But it’s not needed to install ftp. The easiest way to do, is to add a password to www-data and to allow sftp access :

 passwd www-data

And you will just have to replace /usr/sbin/nologin by /bin/bash in the file /etc/passwd. As FTP/SFTP client, I use Cyberduck which is the fastest client I have found, and you can use it for almost all protocol, including S3 storage at AWS or Google Drive.

After uploading my files, just have to purge the redis cache, and that’s already over.

Ubuntu - Blazing fast WordPress with Nginx php7 redis

Sun, 26 Jun 2016 00:37:00 +0000

You have buy a VPS to speed up WordPress, but you are using a control panel and it's not faster than before ? In this tutorial, you will learn how to setup a Blazing fast WordPress with Nginx php7 and redis cache.

This will require to use the the command linux via SSH. Yes I know, it's less user friendly than a control panel. But it will not require any resources after the install and we have an amazing tools to help us : EasyEngine

So, first we will talk about what we are going to install and why it will make WordPress faster.

Nginx : To pronounce "Engine X", it's the web server created in 2002, created to handle high traffic with a low memory footprint. And this is the reason which explain why Nginx is so popular at the moment. It can also be used as a reverse proxy or as a load-balancer, which means if you have few small VPS, you will be able to create a stable and reliable platform to host your website.

php7 : I will not explain what is the php but more about the difference between the 5.6 and the 7. It's the performance and there is nothing better than a small graphic to compare them. And WordPress is fully compatible with the latest version of php.

MariaDB : If you haven't already used it, MariaDB is a fork of Mysql from his creator, and it's an open-source solution which is currently faster than Mysql with a lower memory and processor footprint. So no need to hesitate, if you are still using msql, even in 5.7, it's time to move !

Redis : What is Redis ? It's a remote dictionary server, which will manage your database and store the values in the memory. It's a "NoSQL" solution as Redis can be used as a database by writing data on your ~~hard drive~~ SSD. For WordPress there is no faster solution as it will eliminate the queries to your database, and even if you are using Mariadb, it will be faster and using less resources.

But you are maybe thinking it will be very hard to configure all that stuff properly. No, you will be able to do it in less than 10 minutes ! Because we are going to use EasyEngine, the most powerful tools to create an optimized WordPress website.

It's a command line tool to create install and configure a complete web server with Nginx, PHP, MariaDB and Postfix in a single command. It also provide you the ability to manage properly your virtualhosts with Nginx and make it easy with simple command to use.

So to start you need a VPS with Ubuntu 14.04. EasyEngine is also compatible with debian 7/8 and Ubuntu 16.04. But as EE has been build at first for ubuntu, it's the most stable solution.

At first we need to install EasyEngine.

wget -qO ee rt.cx/ee && sudo bash ee

Done, EasyEngine is now ready to install what we need. So we are looking for WordPress with nginx, php7, redis and as it will not be harder, a SSL with let's encrypt. But you have the choice with EE :

A simple website for html :

ee site create example.com --html

A website with php :

ee site create example.com --php

A website with mysql :

ee site create example.com --mysql

A website with SSL by let's encrypt

ee site create example.com --letsencrypt

And for WordPress ? You just have to choose :

ee site create example.com --wp # install wordpress without any page caching

ee site create example.com --w3tc # install wordpress with w3-total-cache plugin

ee site create example.com --wpsc # install wordpress with whisp-super-cache plugin

ee site create example.com --wpfc # install wordpress + nginx fastcgi_cache

ee site create example.com --wpredis # install wordpress + nginx redis_cache

So for my example, I will use :

ee site create yourwebsite.com --wpredis --letsencrypt --php7

You will have to enter Yes to the EE questions, and yes your wordpress with redis php7 and SSL is already installed and configured. You also have two plugins in WordPress :

Nginx Helper to clear the cache
Redis Cache : To connect WordPress with Redis and to clear the cache

But you will maybe ask how you can connect to the database, what is the user of mysql and wordpress ? You can use a custom user during the install process by using

--user=admin and -pass=password to set WordPress login details.

But you can also look at the install log to find them as EE display it in the console :

nano /var/log/ee/install.log

To see all the commands available with the websites :

ee site

If you want to edit some default configuration (database, user etc) :

nano /etc/ee/ee.conf

By default EasyEngine create a backend access with all tools you can install with it (phpmyadmin, cache, php infos) at the address : http://yourserverIP:22222

To secure it properly just use :

ee secure --auth

It will ask you for a user and a password. You can edit it at anytime.

To edit the default port :

ee secure --port

And to get the mysql user of a website :

ee site info yoursite.com

So I recommend you to change the port of admin tools, and to install what you need in the following :

Adminer (phpmyadmin alternative )

ee stack install --adminer

phpMyAdmin :

ee stack install --phpmyadmin

phpAdminRedis (to manage data in cache) :

ee stack install --phpredisadmin

All the Admin tools (Adminer, phpMyAdmin, phpMemcachedAdmin, FastCGI cleanup script, OPcache, Webgrind, Anemometer) :

ee stack install --admin

You have to agree this tool is really awesome. And if you look to your memory usage using htop, you will see the difference with a control panel. Your website will be faster and you have the ability to edit your configuration or to add module when you want.

So to conclude this tutorial here the links you will need to understand perfectly EasyEngine and to use it for all your setup :

Leave a comment if you have any question !