Supervisor Installation and Configuration on Ubuntu 18.04 LTS

Welcome back in today’s event loop on {{ PLP }}, we’re moving up with our discussions about the importance of monitoring our web server processes especially the channels 2.0+ that uses the Daphne which means the HTTP, HTTP2, and the Web Socket protocol for ASGI and ASGI-HTTP to power the Django powered site asynchronously.

The Supervisor is a client/server system that allows its users to control a number of processes on UNIX-like operating systems. This is very important and highly recommended to have this installed on our Ubuntu 18.04 LTS or any other UNIX-like operating systems you have.

The specific problem that the Supervisor is solving for us is that, when the time we need to reboot or unexpected events that it’s beyond our control that causing the web server to reboot unintentionally. Thus, the Supervisor is trying to autorun the specified applications that we configured to automatically run when the OS is up and running once again.

By the way, if you missed out our previous discussions regarding the Django Sitemap Framework, which really useful and mandatory to be implemented on our Django projects so that it will be searchable by the search engines like Google.

Introduction

In the meantime, the Supervisor is simple, centralized, efficient, extensible, compatible, and proven to most of us web developers. I tell you it’s really effective and reliable for most of my Django projects dependencies to be monitored by the Supervisor.

On the other hand, the Supervisor’s components are the following:

  • supervisord: It is responsible for starting child programs at its own invocation, responding to commands from clients, restarting crashed or exited sub-processes and generating and handling “events” corresponding to points in sub-process lifetimes. Later part, we will be configuring the supervisor and definitely we will modify its default configuration file located at the /etc/supervisord.conf from our web server.
  • supervisorctl: It provides a shell-like interface to the features provided by supervisord. This is the command-line that’s very useful when we have changed to the /etc/supervisord.conf and in the /etc/supervisor/conf.d directory in which this is our custom configurations specific to each web services that we intended to monitor and make it autorun during the boot time. The 2 supervisorctl commands that would be adequate when we work with the Supervisor which are the sudo supervisorctl reread (which will detect any changes from any config files) and the sudo supervisorctl update (will be responsible to update all the changes for the Supervisor immediately).
  • Web Server: A (sparse) web user interface with functionality comparable to supervisorctl may be accessed via a browser if you start supervisord against an internet socket. This is a simple web GUI for the Supervisor in which you can access it using the web browsers with username and password as the authentication before you can access the page is highly recommended.
  • XML-RPC Interface: The same HTTP server which serves the web UI serves up an XML-RPC interface that can be used to interrogate and control supervisor and the programs it runs.

So, prepare your self once again and stay focus because this would be exciting and fun learning with {{ PLP }}.

Getting Started

The easiest way we can install the Supervisor is using the Ubuntu distribution package although, there is another way which is via the pip command, but, I don’t recommend it as you have to configure the Supervisor the hard way, and if not properly configured, the Supervisor program itself will not auto-run during the boot time.

Step 1: Install Supervisor

By the way, this is a system-wide installation and NOT under the VirtualEnv, we should only have one Supervisor program from our web server. This direction is assuming you have the root privilege to install anything to your web server.

1
sudo apt-get install supervisor

Upon the successful execution of the command above, it will create a new config file inside the /etc/supervisor folder.

In this case, the supervisord.conf has been created successfully, we need to download this file and start modifying few settings to make this functional and add our own configurations as per programs that we wanted to include in the Supervisor program to manage and monitor it respectively.

Step 2: Add Configurations Inside the supervisord.conf

Moreover, after downloading the file, open it with your favorite text editor. Here is the default supervisord.conf configurations so we can differentiate it accordingly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Notes:
;  - Shell expansion ("~" or "$HOME") is not supported.  Environment
;    variables can be expanded using this syntax: "%(ENV_HOME)s".
;  - Quotes around values are not supported, except in the case of
;    the environment= options as shown below.
;  - Comments must have a leading space: "a=b ;comment" not "a=b;comment".
;  - Command will be truncated if it looks like a config file comment, e.g.
;    "command=bash -c 'foo ; bar'" will truncate to "command=bash -c 'foo ".

[unix_http_server]
file=/tmp/supervisor.sock   ; the path to the socket file
;chmod=0700                 ; socket file mode (default 0700)
;chown=nobody:nogroup       ; socket file uid:gid owner
;username=user              ; default is no username (open server)
;password=123               ; default is no password (open server)

;[inet_http_server]         ; inet (TCP) server disabled by default
;port=127.0.0.1:9001        ; ip_address:port specifier, *:port for all iface
;username=user              ; default is no username (open server)
;password=123               ; default is no password (open server)

[supervisord]
logfile=/tmp/supervisord.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB        ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10           ; # of main logfile backups; 0 means none, default 10
loglevel=info                ; log level; default info; others: debug,warn,trace
pidfile=/tmp/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false               ; start in foreground if true; default false
minfds=1024                  ; min. avail startup file descriptors; default 1024
minprocs=200                 ; min. avail process descriptors;default 200
;umask=022                   ; process file creation umask; default 022
;user=chrism                 ; default is current user, required if root
;identifier=supervisor       ; supervisord identifier, default is 'supervisor'
;directory=/tmp              ; default is not to cd during start
;nocleanup=true              ; don't clean up tempfiles at start; default false
;childlogdir=/tmp            ; '
AUTO' child log dir, default $TEMP
;environment=KEY="value"     ; key value pairs to add to environment
;strip_ansi=false            ; strip ansi escape codes in logs; def. false

; The rpcinterface:supervisor section must remain in the config file for
; RPC (supervisorctl/web interface) to work.  Additional interfaces may be
; added by defining them in separate [rpcinterface:x] sections.

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

; The supervisorctl section configures how supervisorctl will connect to
; supervisord.  configure it match the settings in either the unix_http_server
; or inet_http_server section.

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris              ; should be same as in [*_http_server] if set
;password=123                ; should be same as in [*_http_server] if set
;prompt=mysupervisor         ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history  ; use readline history if available

; The sample program section below shows all possible program subsection values.
; Create one or more '
real' program: sections to be able to control them under
; supervisor.

;[program:theprogramname]
;command=/bin/cat              ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=999                  ; the relative start priority (default 999)
;autostart=true                ; start at supervisord start (default: true)
;startsecs=1                   ; # of secs prog must stay up to be running (def. 1)
;startretries=3                ; max # of serial start failures when starting (default 3)
;autorestart=unexpected        ; when to restart if exited after running (def: unexpected)
;exitcodes=0,2                 ; '
expected' exit codes used with autorestart (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false             ; send stop signal to the UNIX process group (default false)
;killasgroup=false             ; SIGKILL the UNIX process group (def false)
;user=chrism                   ; setuid to this UNIX account to run the program
;redirect_stderr=true          ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (0 means none, default 10)
;stdout_capture_maxbytes=1MB   ; number of bytes in '
capturemode' (default 0)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups=10     ; # of stderr logfile backups (0 means none, default 10)
;stderr_capture_maxbytes=1MB   ; number of bytes in '
capturemode' (default 0)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A="1",B="2"       ; process environment additions (def no adds)
;serverurl=AUTO                ; override serverurl computation (childutils)

; The sample eventlistener section below shows all possible eventlistener
; subsection values.  Create one or more '
real' eventlistener: sections to be
; able to handle event notifications sent by supervisord.

;[eventlistener:theeventlistenername]
;command=/bin/eventlistener    ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;events=EVENT                  ; event notif. types to subscribe to (req'
d)
;buffer_size=10                ; event buffer queue size (default 10)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=-1                   ; the relative start priority (default -1)
;autostart=true                ; start at supervisord start (default: true)
;startsecs=1                   ; # of secs prog must stay up to be running (def. 1)
;startretries=3                ; max # of serial start failures when starting (default 3)
;autorestart=unexpected        ; autorestart if exited after running (def: unexpected)
;exitcodes=0,2                 ; 'expected' exit codes used with autorestart (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false             ; send stop signal to the UNIX process group (default false)
;killasgroup=false             ; SIGKILL the UNIX process group (def false)
;user=chrism                   ; setuid to this UNIX account to run the program
;redirect_stderr=false         ; redirect_stderr=true is not allowed for eventlisteners
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (0 means none, default 10)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups=10     ; # of stderr logfile backups (0 means none, default 10)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A="1",B="2"       ; process environment additions
;serverurl=AUTO                ; override serverurl computation (childutils)

; The sample group section below shows all possible group values.  Create one
; or more 'real' group: sections to create "heterogeneous" process groups.

;[group:thegroupname]
;programs=progname1,progname2  ; each refers to 'x' in [program:x] definitions
;priority=999                  ; the relative start priority (default 999)

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

;[include]
;files = relative/directory/*.ini

At line number 22, which is the [inet_http_server] setting, enable this setting to protect the supervisor when we access it over the web browsers. Below is the enabled version of that line.

1
2
3
4
[inet_http_server]         ; inet (TCP) server disabled by default
port=139.162.46.17:9001    ; ip_address:port specifier, *:port for all iface
username=user              ; default is no username (open server)
password=123               ; default is no password (open server)

The port=127.0.0.1:9001 originally, but, change it with your own web server IP address, the port:9001 leave it as is, but, it’s up to you if you want to change it. And then, the username and password are with your own choice of course.

Next, scroll at the bottom, and then add these lines of script to include all the individual app configurations inside the /etc/supervisor/conf.d will be included from the main supervisord.conf scripts.

1
2
3
4
5
6
; ******************************************************************
; ADDED CUSTOM CONFIG
; ******************************************************************
; So, any files found in /etc/supervisor/conf.d and ending in .conf will be included.
[include]
files = /etc/supervisor/conf.d/*.conf

As you can see, we have included the sub folder named conf.d which stores all the individual app configurations we have to create later on.

Next, we need to create a new config file named dev.conf and save it inside the /etc/supervisor/conf.d sub folder’s directory. Then, COPY exactly the scripts below, you can change the IP address and necessary -p 8000 if your port is not the same as mine.

1
2
3
4
5
6
7
8
9
10
11
; LOCATE FOR THE VirtualEnv PROGRAM TO RUN :
[program:dev]
directory=/root/dev/
command=/root/env_dev/bin/daphne -b 172.104.190.249 -p 8000 dev.asgi:application
autostart=true
autorestart=true
startretries=3
stopasgroup=true
; DISABLE THIS IT WILL GROW BIGGER, FOR TESTING JUST ENABLE THIS
; stderr_logfile=/etc/supervisor/logs/dev_supervisor.err.log
; stdout_logfile=/etc/supervisor/logs/dev_supervisor.out.log

Last time, instead of manually run the Daphne web service from the console, when we try to reboot the Ubuntu system, the Daphne will not be running automatically, and then our Django project will not be accessible as well, and it’s a bad practice actually. Now, upload all the modified and new files to the /etc/supervisor using the FileZilla and we will do a few things before we can start using the Supervisor.

Step 3: Check the Supervisor Status

With the Ubuntu 18.04 LTS distribution package, the installation will do the heavy lifting for us, meaning, package defaults set most of the configurations for us, the Supervisor will auto-run even when the web server is booting up, we just need to check via command if the Supervisor program really started after the installation.

1
sudo supervisorctl status all

This is the actual results after executing the supervisorctl status all command. So far, we only have one program to manage and that is the Daphne web service for our Django projects to run continuously.

1
2
root@localhost:~# sudo supervisorctl status all
dev                              RUNNING   pid 1040, uptime 0:12:42

To check any services logs, you can verify using this command.

1
sudo journalctl -u supervisord.service

At this moment, the supervisor server is now up and running, we need to access the web interface to visually manage all of our programs and start monitoring it.

Step 4: Access the Supervisor Web Interface

The most exciting part of this guide is to open the supervisor web interface so we can manage all our programs easily and accessible. For my case, you can try to access using this address: http://172.104.190.249:9001/

But, of course, it will prompt you with the username and password earlier we have set up. But, just take a look at the actual screenshot below.

In the meantime, I only have one program to monitor which is the Daphne web service as this is one of the most important services to run constantly without interruption and you don’t have to worry about the sudden rebooting of your web server and causes the downtime to your Django web project.

In the next event loop on {{ PLP }}.

Congratulations!, now we know the importance of Supervisor which is one of the reputable solutions to monitor and automate the manual process that we manually do like this Daphne web service not to interrupt when it’s running or not, we don’t want our Django project interrupted by this issue.

In our next discussions, we will be adding more web services to be included in the Supervisor such as the Django Celery with Real-time Monitoring Tasks Using Flower asynchronous web services as well, stay tuned for that.

For those who’re not able to successfully launch your Supervisor Installation and Configuration on Ubuntu 18.04 LTS tutorial that I’ve laid it before you, or you need more clarifications, don’t worry, leave a comment below and I’m happy to help you to succeed.

That’s all, have fun learning with {{ PLP }}.

To help Filipino students to learn Python programming language with Django to enhance their capabilities in developing robust web-based applications with practical and direct to the point tutorials, step-by-step with actual information that I provided for you. Leave a comment below or email me at [email protected], thank you!