Easily rotate Nginx logs using rename, truncate, then restart

The scripts below is probably one of the simplest way to keep a few days worth of logs (up to you) and still keep all logs organized enough without loosing track.

What the scripts below do:

1. change directory to /log/nginx  (this is where I keep my Nginx logs - yours may be different)
2. move (rename) a log file ie: access.log to access_YYYYMMDD.log
3. create a new log file (using touch) with zero bytes.
4. change permission of new log file to 777 (allow everything).
5. restart nginx process so that it will use newly created log files.

-----------------------------------------------

cd /log/nginx

mv access.log access_`date +"%Y%m%d"`.log
touch /log/nginx/access.log
chmod 777 /log/nginx/access.log

mv error.log error_`date +"%Y%m%d"`.log
touch /log/nginx/error.log
chmod 777 /log/nginx/error.log

mv api-access.log api-access_`date +"%Y%m%d"`.log
touch /log/nginx/api-access.log
chmod 777 /log/nginx/api-access.log

mv api-error.log api-error_`date +"%Y%m%d"`.log
touch /log/nginx/api-error.log
chmod 777 /log/nginx/api-error.log

mv default-access.log default-access_`date +"%Y%m%d"`.log
touch /log/nginx/default-access.log
chmod 777 /log/nginx/default-access.log

mv default-error.log default-error_`date +"%Y%m%d"`.log
touch /log/nginx/default-error.log
chmod 777 /log/nginx/default-error.log

mv default-ssl-access.log default-ssl-access_`date +"%Y%m%d"`.log
touch /log/nginx/default-ssl-access.log
chmod 777 /log/nginx/default-ssl-access.log

mv default-ssl-error.log default-ssl-error_`date +"%Y%m%d"`.log
touch /log/nginx/default-ssl-error.log
chmod 777 /log/nginx/default-ssl-error.log

mv rockmongo-access.log rockmongo-access_`date +"%Y%m%d"`.log
touch /log/nginx/rockmongo-access.log
chmod 777 /log/nginx/rockmongo-access.log

mv rockmongo-error.log rockmongo-error_`date +"%Y%m%d"`.log
touch /log/nginx/rockmongo-error.log
chmod 777 /log/nginx/rockmongo-error.log

kill -USR1 $( cat /var/run/nginx.pid )

-----------------------------------------------

What to do with all those files created with YYYYMMDD?

Well you can then remove them using the following crontab lines at your liking:

here is a line from my crontab that removes every file in my /log/ directory recursive that is older than 10 days:

19 4 * * * find /log/ -type f -mtime +10 | xargs rm

My most up to date PHP development stack

A few people have asked me about which development tools, libraries, and technologies I am using. I want to create this blog to help current and new PHP developers get up to speed and hopefully not spend as much time as I had trying out different tools (or worst wrong tools) :-)

Computer:

In 2015 I ditched my Mac Pro for a Macbook Pro and good thunderbolt dock that can drive my 4K monitor. I did this so that I can be mobile and most importantly so that I do not sit on my a** all day.

Yes, sorry I do not recommend Windows, for web development (PHP, Python, Ruby, etc... ) Macs are much better choice.

Text Editor:

Sublime Text 3

Server OS:

Ubuntu LTS versions, currently 14.04

Web Server:

Nginx + FPM

Database Server:

MongoDB + ElasticSearch

I have developed my own DB framework which utilizes Redis > MongoDB > ElasticSearch all together in harmony.

PHP Framework:

Laravel + Composer

I used to be a code igniter fan, however Laravel is king today.

(Node JS sometime)

Javascript:

jQuery + Angular

HTML:

Bootstrap 3.x

I waited too long to jumped to Laravel.

In 2015, I have been blessed with TONS of work, so much work that I have not had a chance to re-factor my base codes to Laravel.  I wish I would have switch since version 4.  Anyways... make sure you checkout Laravel, it is almost as good as the next 'daily bread' for PHP.

I hope this article help someone new or even PHP veterans to compare with tools they are using.

Truncate and rotate linux server log files using copy, truncate and restart gracefully

If you are not using auto-rotate of log files for certain daemons in your linux server, you may be looking for a solution to auto-rotate your log files.

Rotating a log file is necessary so that your log file will not be too big.

When log files gets too big, it has bitten me in the a** before.  Server runs out of disk space
and server process fails. Yes it is an embarrassing situation for any server admin.  Save yourself the embarrassment and spent 5 minutes implementing this simple rotate script:

cp /var/log/nginx/error.log /data/log/tmp_nginx_log/nginx_error_`date +"%Y_%m_%d"`.log; truncate -s0 /var/log/nginx/error.log; kill -USR1 $( cat /var/run/nginx.pid )

The code above will do the following:

1. copy the current log file from /var/log/nginx/error.log to /data/log/tmp_nginx_log/
   while copying, it will also rename the file by appending specific date to it.
2. next, it will truncate the current log file to zero size.
3. finally, it will send message to nginx web server to reload gracefully and it will start logging errors into the new zero sized log file.

So now that we moved the log file to another directory what do we do with it there?

Since we named all files with their appropriate dates, we can easily review individual log files when we need too. 

Another purpose for separating the log files into separate dates, is so that we can clean-up old log files we no longer want. For example if you want to only keep log files for 30 days, you can easily run the following command or schedule in your crontab.

find /data/log/tmp_nginx_log -type f -mtime +3 | xargs rm


The command above will remove any files older than 3 days from /data/log/tmp_nginx_log directory.

For more options you can do to clean up files, read my blog about how to Remove Files in Directory Recursively.

Compiling and installing Nginx 1.4.4 from scratch

wget http://nginx.org/download/nginx-1.4.4.tar.gz

tar -zvxf nginx-1.4.4.tar.gz

cd nginx-1.4.4

./configure \
--with-http_ssl_module \
--with-http_realip_module \
--with-http_addition_module \
--with-http_sub_module \
--with-http_dav_module \
--with-http_flv_module \
--with-http_mp4_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_random_index_module \
--with-http_secure_link_module \
--with-http_stub_status_module \
--with-mail \
--with-mail_ssl_module \
--with-file-aio \
--with-ipv6


make
make install

GZIP setting for NGINX that I recommend to use

 ##
 # Gzip Settings
 ##

 gzip on;
 gzip_disable "msie6";
 gzip_comp_level 6;
 gzip_buffers 16 8k;
 gzip_vary on;
 gzip_proxied any;
 gzip_types text/plain text/

Generate self signed self-signed signing certificate

Step 1 : Generate Secure Key

openssl genrsa -des3 -out <your_domain_name>.key 2048

Step 2 : Generate Insecure Key

openssl rsa -in <your_domain_name>.key -out <your_domain_name>.key.insecure

mv <your_domain_name>.key <your_domain_name>.key.secure

mv <your_domain_name>.key.insecure <your_domain_name>.key

Step 3 : Generate CSR (certificate request)

openssl req -new -key <your_domain_name>.key -out <your_domain_name>.csr

Step 4 : Generate Certificate

openssl x509 -req -days 365 -in <your_domain_name>.csr -signkey <your_domain_name>.key -out <your_domain_name>.crt

Tip:

It is good practice to store your keys (.key.insecure) & (.key) in separate directory from your certificate files.  For example if you are using Debian I recommend storing your keys in /etc/private/ssl directory.

Combining SSL certificates from Comodo Positive SSL for NGINX

Here is a quick and easy way to combine Comodo's Positive SSL Certificate.

I bought my certificate from NameCheap for about $9.95 / year.

This post will only describe the process of combining the certificate. I assume you already:

1. Generate CSR and .key file
2. Submit CSR to Comodo / Namecheap
3. Got an email from Comodo containing .crt file

If you have not done any of the task above, please do them first before following this guide.

[STEP 1]  Confirm and Extract all files received from Comodo:

Attached to this email you should find a .zip file containing:

• Root CA Certificate - AddTrustExternalCARoot.crt
• Intermediate CA Certificate - COMODORSAAddTrustCA.crt
• Intermediate CA Certificate - COMODORSADomainValidationSecureServerCA.crt
• Your PositiveSSL Certificate - <YOUR_DOMAIN_NAME>_com.crt 

[STEP 2]  Make sure all files are inside one directory.

Copy / move all files into one directory.

[STEP 3]  Combine them!

From your linux server command line, execute:

cat <YOUR_DOMAIN_NAME>_com.crt COMODORSADomainValidationSecureServerCA.crt COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt > <YOUR_DOMAIN_NAME>_combined.crt

[STEP 4]  Edit your nginx /etc/sites-available/<your_file>

The following configuration is what I use to enable SSL on my NGINX configuration. I have tweaked this configuration overtime and this is my latest one. It enables TLS and prevent SSLv2 and SSLv3 from being used because they are weak and vulnerable. I also included configuration to disable weak ciphers.

ssl on;
ssl_certificate /etc/ssl/<YOUR_DOMAIN_NAME>/<YOUR_DOMAIN_NAME>_combined.crt;
ssl_certificate_key /etc/ssl/private/<YOUR_DOMAIN_NAME>.key;

ssl_session_timeout 5m;

#enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

#Disables all weak ciphers
ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";

ssl_prefer_server_ciphers on;

[STEP 5]  Restart NGINX

service nginx restart