Set up Gos Web Socket Bundle in AWS (Production Symfony Application) with SSL

Introduction

Gos Web Socket is a Symfony2 Bundle designed to bring together WS functionality in a easy to use application architecture.It provides both server and client side code ensuring you have to write as little as possible to get your app up and running. Powered By Ratchet, AutobahnJS, with Symfony2. This can be used to implement WS clients using AngularJS.

However, getting this to work in production that too in a AWS EC2 instance behind a load balancer and autoscaling group with SSl is tricky.

Examples

Step By Step Configuration 1. Deploy the Code in AWS EC2 Instance

In my project AngularJS websocket client talks to the socket server implemented with AutobahnJS and gos-websocket.js. This is packaged in an separate application for deployment in Apache. This application talks to my symfony application (seperately deployed) which acts as the web socket server and it uses Ratchet. Basic documentation is available here - https://github.com/GeniusesOfSymfony/WebSocketBundle

I was able to execute the web socket on this EC2 instance with this config.

2. Configure AWS for Load balancer (out of scope of this documentation)

Kindly go through the AWS documentation http://docs.aws.amazon.com/autoscaling/latest/userguide/attach-load-balancer-asg.html

Make sure the AWS firewall configurations are altered to allow websocket port to receive connections.

I was able to execute the web socket with this configuration too.

3. Configure the server for SSL (out of scope of this documentation)

Kindly go through the AWS documentation http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-an-instance.html

However, after configuring SSL my web socket stopped working. Reason: The web socket server implementation does not support secured ports. So the TCP request on secured connection are not handled. In next step we will configure stunnel to overcome this challenge.

4. Configuring stunnel (I am using AWS Linux AMI)

sudo yum install stunnel

This will install the stunnel application in the server.

repoquery -l stunnel

This command will list all the files installed with stunnel application. Now create the configuration file with below command

sudo vim /etc/stunnel/stunnel.conf

Copy paste the below configuartion

debug = debug
cert = /etc/ssl/certs/domainname.com.cert
key = /etc/ssl/certs/domainname.com.cert.key

#Remove TCP delay for local and remote.
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
chroot = /var/run/stunnel/ #the changed root directory in which the stunnel process runs, for greater security
pid = /stunnel.pid
output = stunnel.log

# IMPORTANT: If the websocketserver is on the same server as the webserver use this:
local = 10.0.0.22 # Insert here your Private IP of EC2 instance that is secured with https.

[websockets]
accept = 9090 # This is the port from which my client will connect ans stunnel will intercept
#connect = 8888

# IMPORTANT: If you use the local variable above, you have to add the domainname here aswell.
connect = 10.0.0.22:8443 # This is ip (EC2 private IP) and port on which ratchet is listening and stunnel will forward the request

5. AWS firewall setting changes

For both load balancer and the EC2 instance change the inbond setting. Since my stunnel is listening on 9090, Ihave opened that port for communication For both load balancer and the EC2 instance change the inbond setting. Since my stunnel is listening on 9090, Ihave opened that port for communication

6. Verify the wss/ip/port setting in ws client code (Angular code)

a. configuration in app-run.js

var config_port = [{host: "domainname.com", port: "9090", secured: true}];

b. configuration in bower_components/gos-websocket/src/gos-websocket.js

this.websocket = WS.connect('wss' + '://' + 'domainname.com' + ':' + '9090');

7. Verify the setting in ws server side (PHP)

# app/config/config_prod.yml
gos_web_socket:
    server:
        port: 8443        #The port to which ratchet WS is listening
        host: ec2-51-11-111-11.ap-south-1.compute.amazonaws.com   #The EC2 instance bind to
        router:
            resources:
                - '@AppBundle/Resources/config/pubsub/routing.yml'

8. Lets start the stunnel server

sudo stunnel /etc/stunnel/stunnel.conf

Watch for any errors in log

sudo vim /var/run/stunnel/stunnel.log

9. Processes to bind to the non-local address With Amazon EC2 and elastic IPs, the server doesn't actually know its IP as with most any other server. So you need to tell your linux to allow processes to bind to the non-local address. Just add the following line into /etc/sysctl.conf file:

# allow processes to bind to the non-local address
# (necessary for apache/nginx in Amazon EC2)
net.ipv4.ip_nonlocal_bind = 1

and then reload your sysctl.conf by:

sysctl -p /etc/sysctl.conf

which will be fine on reboots.

10. lets start the gos web socket server (ratchet)

Move to project directory of Symfony before firing the below command

php app/console gos:websocket:server -e=prod -n

Clear the cache of Symfony application before the websocket server is started. Also you can kill the stunnel server if any change is needed in stunnel configuration.

sudo kill $(cat /var/run/stunnel/stunnel.pid)

Now try to hit the secured wss:// URL and it should work.

11. Using Logrotate (Linux command/package) rotate the stunnel log file This will help to keep the size of log file always under check

sudo apt-get install logrotate cd /etc/logrotate.d/ vim stunnel Copy paste below conf to it and save the file

/var/run/stunnel/stunnel.log { daily rotate 30 size 2M copytruncate dateext dateformat .%d%m%Y notifempty compress delaycompress }

https://www.tecmint.com/install-logrotate-to-manage-log-rotation-in-linux/

**12. Now, looks like in prod, socket server stops intermittently due to multiple reasons. One reason, If the server is running behind AWS Load Balancer, there is a max request duration which is set as 60 min. To overcome this challenge, we will make websocket server and Stunnel as self-healing services. When the server reboots or these services stop, it will be back in few seconds without manual intervention.

Based on your environment follow the steps to install daemontools in the below URL **

https://isotope11.com/blog/manage-your-services-with-daemontools

https://gist.github.com/toddlers/9364259

https://cr.yp.to/daemontools/install.html

https://stackoverflow.com/questions/36399980/how-setuidgid-command-works

Check Whether stunnel is running in your server ps -ef|grep stunnel

a. Follow below instructions to make stunnel a self healing service.

mkdir /services/stunnel/

echo -e '#!/bin/sh\nexec stunnel /etc/stunnel/stunnel.conf' > /services/stunnel/run

chmod 755 /services/stunnel/run

ln -s /services/stunnel/ /service/

mkdir /services/stunnel/log

mkdir /services/stunnel/log/main

vim /services/stunnel/log/run

#!/bin/sh

exec logger multilog t ./main

chmod 755 /services/stunnel/log/run

Verify whether things are ok

tail -n5 /services/stunnel/log/main/current

Kill stunnel service and see whether it coming back

sudo kill $(cat /var/run/stunnel/stunnel.pid)

ps -ef|grep stunnel

I can see the below

root 2062 1 0 10:54 ? 00:00:00 stunnel /etc/stunnel/stunnel.conf

b. Follow below instructions to make websocket a self healing service (This is for Symfony2).

mkdir /services/Goswebsocket/

ps -ef|grep php

`echo -e '#!/bin/sh\nexec php /var/www/html/wsrest/app/console gos:websocket:server -e=dev -n' > /services/Goswebsocket/run'

chmod 755 /services/Goswebsocket/run

ln -s /services/Goswebsocket/ /service/

mkdir /services/Goswebsocket/log

mkdir /services/Goswebsocket/log/main

#chmod 766 /services/Goswebsocket/log/main

vim /services/Goswebsocket/log/run

#!/bin/sh

exec logger multilog t ./main

chmod 755 /services/Goswebsocket/log/run

Kill websocket service and see whether it coming back

sudo kill $(cat <PHP PID>)

ps -ef|grep php

I can see the below `root 1910 1898 0 10:54 ? 00:00:00 php /var/www/html/wsrest/app/console gos:websocket:server -e=dev -n'