A guide to installing, configuring and operating the Nemesida WAF dynamic module for Nginx and the Nemesida AI MLA machine learning agent, designed to detect and block anomalies using signature analysis and behavioral models.
# apt install apt-transport-https gnupg2
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add - # apt update && apt upgrade
# apt install apt-transport-https gnupg2
The dynamic module Nemesida WAF is available for:
- Nginx stable from
1.12
; - Nginx mainline from
1.15
; - Nginx Plus from
R16
.
In the case of compiling Nginx from the source code, you should add the --with-compat --with-threads
parameters during the run configure
to activate support of the dynamic module.
Installation
Set the operating system ID:
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
# apt install nwaf-dyn-1.18
where 1.18 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-plus-rX (where X is the number of release, started with R16) is intended for work with the last version of Nginx Plus (for example: nwaf-dyn-plus-r16).
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
# apt install nwaf-dyn-1.18
where 1.18 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-plus-rX (where X is the number of release, started with R16) is intended for work with the last version of Nginx Plus (for example: nwaf-dyn-plus-r16).
Set the operating system ID:
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
Configure the SELinux policy or deactivate it with the command:
# setenforce 0
then bring the file /etc/selinux/config
to the form:
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of three two values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted
Add the file path with the dynamic module Nemesida WAF and bring the parameters below in the configuration file /etc/nginx/nginx.conf
to the form:
load_module /etc/nginx/modules/ngx_http_waf_module.so; ... worker_processes auto; ... http { ... ## # Nemesida WAF ## ## Request body is too large fix client_body_buffer_size 25M; include /etc/nginx/nwaf/conf/global/*.conf; ... }
Errors¶:
nginx: [emerg] module "/etc/nginx/modules/ngx_http_waf_module.so" version 1017010 instead of 1018000 in /etc/nginx/nginx.conf:1
The error occurs when the versions of the installed dynamic module Nemesida WAF and Nginx do not match. In this case, 1017010
is the version of Nginx 1.17.10, for which the nwaf-dyn module was compiled, and 1018000
is Nginx 1.18.0 installed on the server. The dynamic module package nwaf-dyn-1.18 is designed to work with Nginx version 1.18, and nwaf-dyn-plus-r22 is designed to work with NGINX Plus R22.
For the
nwaf/conf/db
directory, write/read permission must be granted for the user on whose behalf Nginx is running.
Initial setup
After installing the module, it is necessary to make the initial configuration by specifying the following parameters:
After making changes, restart the server or restart the services and check their operation:
# systemctl restart nginx nwaf_update mla_main # systemctl status nginx nwaf_update mla_main
The nwaf_update
service is responsible for obtaining the signatures of the Nemesida WAF module (/etc/nginx/nwaf/rules.bin
). To test the signature method of detecting attacks, when sending a request to http://YOUR_SERVER/nwaftest
, the server must return a 403
response code.
The Nemesida WAF module processes only the request transmitted to the final application. If the settings of the Nginx software prevent the transmission of calls (returning, for example, a
301
or403
response code), such requests will not be processed by the Nemesida WAF module.
If the signature set would be changed on the server nemesida-security.com nwaf_update
service will automatically update rules.bin
file and apply the changes. If the nwaf_update
service is deactivated during the signature set updating on the server, the download of the new version of signature set will be after the service nwaf_update
starts.
The settings management functionality is enabled by default, but it can be disabled by sending a request to email.
To avoid errors when configuring the module, we recommend using a cloud web application.
Manage settings using a cloud WebApp and a cloud API
Information about configuring a dynamic module using a cloud WebApp and a cloud API is available in the relevant sections.
Managing settings using configuration files
Nemesida WAF dynamic module
The main Nemesida WAF configuration file is available at /etc/nginx/nwaf/conf/global/nwaf.conf
. The file /etc/nginx/nwaf/conf/global/nwaf-example.conf
contains a complete list of available parameters.
Nemesida AI MLA
To configure the module, make the necessary changes to the main configuration file /etc/nginx/nwaf/mla.conf
.
The Nemesida AI MLA module loads models according to the list obtained from the Nemesida AI MLC module (
vhosts_list
parameter in themlc.conf
file).
After making changes, restart the server or restart the services and check their operation:
# systemctl restart nginx nwaf_update mla_main # systemctl status nginx nwaf_update mla_main
1. On the main server (without the Nemesida WAF module installed), configure traffic mirroring according to the guidelines of the installed web server (Nginx, Apache2, Microsoft IIS and others).
2. On the server with the installed Nemesida WAF module, bring the configuration file of the virtual host Nginx to the form:
server { ... location / { proxy_pass http://127.0.0.1:86/; } } server { listen 86; index index.html; root /var/www/html; location / { return 200; add_header Content-Type text/plain; } }
3. On the server with the Nemesida WAF module installed, bring the /etc/nginx/nwaf/conf/global/nwaf.conf
file to the form:
... nwaf_limit rate=5r/m block_time=0; ...
4. To get information from the main web server about the source IP address of the client, on the server with the Nemesida WAF module installed, bring the file /etc/nginx/nginx.conf
to the form:
server { ... set_real_ip_from x.x.x.x; real_ip_header X-Real-IP; real_ip_recursive on; }
x.x.x.x
is the address of the main web server that duplicates traffic to the server with the Nemesida WAF module installed. The source IP address of the client is transmitted by the main web server in the header X-Real-IP
.
To use the Nginx functionality, it must contain the ngx_http_realip_module
module. Nginx installed from the official repository contains the specified module by default.
More information is available on the official page of the Nginx documentation.
5. After changing restart the Nginx on every server.
To use a web server with installed Nginx and Nemesida WAF as an intermediate server, you should create corresponding file of the virtual host (for example /etc/nginx/conf.d/example.com.conf
).
More information is available on the official page of the Nginx documentation.
There is a support of custom set of signatures rules (signature, RL
) and signature exception rules (exclusion rule, WL
) in Nemesida WAF and Nemesida WAF Free.
Zones and additional conditions
During the creation of RL
or WL
rules special parameters can be used:
- zones:
URL
,ARGS
,BODY
,HEADERS
,Cookie
,User-Agent
,Content-Type
, etc.; - conditions of using the rule (additional condition, specification of the zone):
$URL
,$ARGS
,$BODY
,$HEADERS
,$Cookie
,$User-Agent
, etc.
Using zones and additional conditions allows to concretize maximally the creating rule. Additional conditions are set by adding prefix $
(for example: "Z:...|$URL:..."
).
An example of using the rule with zone and additional condition: WL ... "Z:ARGS|$URL:/123";
– the rule will work, if the entry zone of the RL
rule will be ARGS
zone and URL will contain an entry /123
.
Several parameters (zones, additional conditions) in one rule must be separated with the character |
, the following principle of interaction will be used:
- zones interact using the logical principle
OR
, for example:Z:URL|ARGS
; - zones interact with additional conditions using the logical principle
AND
, for example:Z:URL|$ARGS:123
; - additional conditions interact using the logical principle
AND
, for example:Z:...|$ARGS:123|$BODY:123"
.
When used in the additional conditions |
as a regular character, it is necessary to make its single shielding. For example: $URL:abc\|d
– search for the occurrence of abc|d
.
Regular expressions in the additional conditions
It is possible to use regular expressions in the additional conditions. This requires to add to the additional condition postfix _X
(for example: "Z:...|$URL_X:\w+"
).
For use regular expression operators in a rule as regular characters (/
, .
, *
, ?
, !
, {
, }
, [
, ]
, (
, )
, ^
, $
, :
, \
, etc.), they must be double shielding. For example:
$BODY_X:block\\|page
– search of an entryblock|page
;$BODY_X:data=\\(block-\\\a\\\-in-the-main\\)
– search of an entrydata=(block-\a\-in-the-main)
;$ACCEPT_X:image\\/webp\\\\\*
– search of an entryimage/webp\*
inAccept
headers.
For use in additional conditions the separator as a metacharacter of a regular expression it is necessary to shield it double. For example: the rule WL ... "Z:...|$URL_X:/(a\\|b)/";
will bring to URL
which contains /a/
or /b/
.
User-defined signature creation
The user-defined rules of detecting attack signs must be placed in the main configuration file Nemesida WAF (/etc/nginx/nwaf/conf/global/nwaf.conf
). They must be determined by the RL
parameter, must have ID
ranging from 50000 to 99999 and take the following form:
RL ID:50000 "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50001 domain=example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50002 domain=.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50003 domain=*.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50004 "PX:select\s+from" "SC:SQL:12" "Z:ARGS|$URL:/admin";
RL ID:50005 "P:select from" "SC:SQL:12" "Z:ARGS|$URL:/(admin\\|dev)";
For signatures in the zone it is allowed to use the MLA
value to force request sending into Nemesida AI MLA, that falls within the scope of the created signature if the score
< 12 (by default requests with a score
equal or greater than the value of the mla_score
parameter are sent into Nemesida AI MLA).
Creating a signature exclusion rule
In case the inquiry falls under action of a signature, in addition to sending the incident to the Nemesida WAF API, the following line will be displayed in the error log of the Nginx software:
Nemesida WAF: the request ххх contains a rule id 1 in zone HEADERS, ...
or, if the request contains a signature with a maximum allowable digital indicator of significance (score
= 12):
Nemesida WAF: the request ххх blocked by rule id 1 in zone HEADERS, ...
where:
1
– attack signature ID;HEADERS
– signature entry zone.
To display absolutely all occurrences of signatures in the request (if there are occurrences), including those occurrences that did not lead to the subsequent blocking of the request, activate the nwaf_log_mr_all;
parameter, in the nwaf.conf
file Nemesida WAF.
The information about current signature list is on the page rlinfo.nemesida-security.com.
Switching the blocking rules to LM mode ¶
Requests that fall under the LM
mode are recorded in the database, but are not blocked. When the rule is triggered, a message will be sent to the Nginx error log and to the Nemesida WAF API about switching the corresponding blocking rule to LM
mode, but the request will not be blocked. The rules for switching to the LM
mode must be defined by the LM
parameter, have the ID
of the corresponding blocking rule and take the following form:
Virtual patching ¶
Nemesida WAF virtual patching protects websites from existing uncorrected vulnerabilities (including “zero day”), blocking attempts to exploit them, while not disrupting the operation of the web application. The application of virtual patching rules allows developers to focus on fixing vulnerabilities without the need for urgent code changes. Virtual patching allows you to block all attempts to exploit a known vulnerability on the fly, controlling the attack zone in a special way.
Nemesida WAF uses two sources of virtual patch generation: automatically, by the module Nemesida AI, and manually, by creating signature rules.
Each request processed by the Nemesida WAF module is assigned a blocking identifier (BT – blocking type).
The identifier can be assigned regardless of whether the request was blocked.
The pivot table shows the types of identifiers and their corresponding request processing status.
0 | The request was not blocked. |
1 | The request is blocked by signature method and the request did not contain any signature with the score = 12. |
2 | The request is blocked by signature method and the request contained a signature which has the score = 12. |
3 | The request is blocked by Nemesida AI MLA. |
4 | The request is blocked by ClamAV module. |
5 | The request is blocked because of the internal error. |
6 | The request is blocked because of the oversubscription of blocked requests from one IP address. |
7 | The request is detected like the brute force attempt and blocked by Nemesida AI MLC. |
8 | The request is blocked by Nemesida AI MLC (additional traffic analysis of all nonblocked requests by Nemesida WAF module) and is managed by ai_extra parameter in mlc.conf file. |
9 | The request is detected as a flood attempt and blocked by the Nemesida AI MLC. |
10 | The request is detected like the DDoS attacks and blocked by Nemesida AI MLC. |
For BT 1, 2, 3, 4, 8 events, the nwaf_limit
parameter in the nwaf.conf
file allows you to adjust the allowed number of blocked requests. If the allowed number of requests specified by the rate
option is exceeded, the IP address will be blocked for the time (in seconds) specified in block_time
.
For BT 7, 9, 10 events the blocking of the IP address of the request source will occur after the first blocked request, regardless of the value of the rate
parameter (the allowed number of blocked requests) in nwaf_limit
.
Optional header $nwaf_block_type
Nemesida WAF allows to use special header, which determinative the result of the request processing. To activate it required to add parameter add_header NemesidaWAF-BT $nwaf_block_type always;
into http
, server
or location
area of the nginx
configuration file.
Example of using the header
- Set the header into the
server
area:server { ... add_header NemesidaWAF-BT $nwaf_block_type always; ... }
and restart
nginx
. - Execute the request:
curl -i localhost/nwaftest
.
The server’s answer will contain the information about the status of the request processing in the NemesidaWAF-BT
header:
HTTP/1.1 403 Forbidden Server: nginx/1.16.1 Date: Wed, 12 Feb 2020 19:05:26 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive NemesidaWAF-BT: 2 ...
In this case the status is 2
. It means that the request was blocked because of the attack’s signature entry with the score
equal to 12
(nwaftest
). More information about the request processing status is available in the corresponding paragraph. This header can be useful during the generation of the customized page of the server answer (for example if the request is blocked).
Optional header $nwaf_cc
Nemesida WAF allows you to use a special header that determines the result of blocking a request by country using extended request blocking rules. To activate it, you need to add the parameter add_header NemesidaWAF-CC $nwaf_cc always;
to the http
, server
or location
of the configuration file nginx
.
Example of using the header
- Set the header to the
server
area:server { ... add_header NemesidaWAF-CC $nwaf_cc always; ... }
and restart
nginx
. - Execute the request:
curl -i example.com
The server response in the header NemesidaWAF-CC
will contain information about the blocking status:
HTTP/1.1 403 Forbidden Server: nginx/1.16.1 Date: Wed, 12 Feb 2020 19:05:26 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive NemesidaWAF-CC: 1 ...
In this case, the status is 1
. This means that the request was blocked by the country
field. If the rule is compiled using additional fields, for example, body
or headers
, then the header will not be applied when blocking. The exceptions are the fields vhost
and url
.
nwaf_limit
option in the nwaf.conf
configuration file). Using the captcha functionality, the user can unlock their own IP address if it has been identified as a source of brute force attacks (BT 7), floods (BT 9) or denial of service attacks (BT 10).
For correct work of the IP address unlocking mechanism, you need to activate the nwaf_captcha_unban
parameter by adding it in the virtual host file to the server section for a specific URL address. Example of activating the nwaf_captcha_unban
parameter
server { listen 80; server_name example.com; location /captcha_ip_unban_path { ... nwaf_captcha_unban on; ... } ... }
Also, you need to setup a web application with a captcha. Detailed instructions for integrating Nemesida WAF with reCAPTCHA functionality can be found at link.
After activating the nwaf_captcha_unban on;
parameter and restarting Nginx, the IP address unblocking functionality becomes available for the target URL
address with the nwaf_captcha_unban
option set. For example: example.com/captcha_ip_unban_path
.
If the answer is correct, the web application with captcha sends a request delete_banned_ip
to the Nemesida WAF module to unlock the IP address. The user will then be redirected to the original page.
For security reasons, it is recommended to allow requests to the server
URL
with the Nemesida WAF dynamic module installed and thenwaf_captcha_unban
IP address unlocking parameter activated only from the IP address of the server with captcha.
To activate antivirus protection, install ClamAV software on the server with the configured Nemesida WAF software, if it has not done yet.
Installation example for Debian 10:
# apt install clamav-daemon
The interaction with the ClamAV software is enabled by activating the nwaf_clamav
configuration parameter in the /etc/nginx/nwaf/conf/global/nwaf.conf
file and reduction of the /etc/clamav/clamd.conf
file to a look:
... TCPSocket 3310 TCPAddr 127.0.0.1 ...
After changing restart the Nginx software.