A guide to installing, configuring and operating the Nemesida WAF dynamic module for Nginx and the Nemesida AI MLA machine learning agent, designed to detect and block anomalies using signature analysis and behavioral models.
# apt install apt-transport-https gnupg2
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add - # apt update && apt upgrade
# apt install apt-transport-https gnupg2
The dynamic module Nemesida WAF is available for:
- Nginx stable from
1.12; - Nginx mainline from
1.15; - Nginx Plus from
R16.
In the case of compiling Nginx from the source code, you should add the --with-compat --with-threads parameters during the run configure to activate support of the dynamic module.
Installation
Set the operating system ID:
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
# apt install nwaf-dyn-1.18
where 1.18 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-plus-rX (where X is the number of release, started with R16) is intended for work with the last version of Nginx Plus (for example: nwaf-dyn-plus-r16).
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
# apt install nwaf-dyn-1.18
where 1.18 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-plus-rX (where X is the number of release, started with R16) is intended for work with the last version of Nginx Plus (for example: nwaf-dyn-plus-r16).
Set the operating system ID:
# rm -f /etc/machine-id # /bin/systemd-machine-id-setup
Configure the SELinux policy or deactivate it with the command:
# setenforce 0
then bring the file /etc/selinux/config to the form:
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of three two values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted
Add the file path with the dynamic module Nemesida WAF and bring the parameters below in the configuration file /etc/nginx/nginx.conf to the form:
load_module /etc/nginx/modules/ngx_http_waf_module.so;
...
worker_processes auto;
...
http {
...
##
# Nemesida WAF
##
## Request body is too large fix
client_body_buffer_size 25M;
include /etc/nginx/nwaf/conf/global/*.conf;
...
}
Errors¶:
nginx: [emerg] module "/etc/nginx/modules/ngx_http_waf_module.so" version 1017010 instead of 1018000 in /etc/nginx/nginx.conf:1
The error occurs when the versions of the installed dynamic module Nemesida WAF and Nginx do not match. In this case, 1017010 is the version of Nginx 1.17.10, for which the nwaf-dyn module was compiled, and 1018000 is Nginx 1.18.0 installed on the server. The dynamic module package nwaf-dyn-1.18 is designed to work with Nginx version 1.18, and nwaf-dyn-plus-r22 is designed to work with NGINX Plus R22.
For the
nwaf/conf/dbdirectory, write/read permission must be granted for the user on whose behalf Nginx is running.
Initial setup
After installing the module, it is necessary to make the initial configuration by specifying the following parameters:
After making changes, restart the server or restart the services and check their operation:
# systemctl restart nginx nwaf_update mla_main # systemctl status nginx nwaf_update mla_main
The nwaf_update service is responsible for obtaining the signatures of the Nemesida WAF module (/etc/nginx/nwaf/rules.bin). To test the signature method of detecting attacks, when sending a request to http://YOUR_SERVER/nwaftest, the server must return a 403 response code.
The Nemesida WAF module processes only the request transmitted to the final application. If the settings of the Nginx software prevent the transmission of calls (returning, for example, a
301or403response code), such requests will not be processed by the Nemesida WAF module.
If the signature set would be changed on the server nemesida-security.com nwaf_update service will automatically update rules.bin file and apply the changes. If the nwaf_update service is deactivated during the signature set updating on the server, the download of the new version of signature set will be after the service nwaf_update starts.
The settings management functionality is enabled by default, but it can be disabled by sending a request to email.
To avoid errors when configuring the module, we recommend using a cloud web application.
Manage settings using a cloud WebApp and a cloud API
Information about configuring a dynamic module using a cloud WebApp and a cloud API is available in the relevant sections.
Managing settings using configuration files
Nemesida WAF dynamic module
The main Nemesida WAF configuration file is available at /etc/nginx/nwaf/conf/global/nwaf.conf. The file /etc/nginx/nwaf/conf/global/nwaf-example.conf contains a complete list of available parameters.
Nemesida AI MLA
To configure the module, make the necessary changes to the main configuration file /etc/nginx/nwaf/mla.conf.
The Nemesida AI MLA module loads models according to the list obtained from the Nemesida AI MLC module (
vhosts_listparameter in themlc.conffile).
After making changes, restart the server or restart the services and check their operation:
# systemctl restart nginx nwaf_update mla_main # systemctl status nginx nwaf_update mla_main
1. On the main server (without the Nemesida WAF module installed), configure traffic mirroring according to the guidelines of the installed web server (Nginx, Apache2, Microsoft IIS and others).
2. On the server with the installed Nemesida WAF module, bring the configuration file of the virtual host Nginx to the form:
server {
...
location / {
proxy_pass http://127.0.0.1:86/;
}
}
server {
listen 86;
index index.html;
root /var/www/html;
location / {
return 200;
add_header Content-Type text/plain;
}
}
3. On the server with the Nemesida WAF module installed, bring the /etc/nginx/nwaf/conf/global/nwaf.conf file to the form:
... nwaf_limit rate=5r/m block_time=0; ...
4. To get information from the main web server about the source IP address of the client, on the server with the Nemesida WAF module installed, bring the file /etc/nginx/nginx.conf to the form:
server {
...
set_real_ip_from x.x.x.x;
real_ip_header X-Real-IP;
real_ip_recursive on;
}
x.x.x.x is the address of the main web server that duplicates traffic to the server with the Nemesida WAF module installed. The source IP address of the client is transmitted by the main web server in the header X-Real-IP.
To use the Nginx functionality, it must contain the ngx_http_realip_module module. Nginx installed from the official repository contains the specified module by default.
More information is available on the official page of the Nginx documentation.
5. After changing restart the Nginx on every server.
To use a web server with installed Nginx and Nemesida WAF as an intermediate server, you should create corresponding file of the virtual host (for example /etc/nginx/conf.d/example.com.conf).
More information is available on the official page of the Nginx documentation.
There is a support of custom set of signatures rules (signature, RL) and signature exception rules (exclusion rule, WL) in Nemesida WAF and Nemesida WAF Free.
Zones and additional conditions
During the creation of RL or WL rules special parameters can be used:
- zones:
URL,ARGS,BODY,HEADERS,Cookie,User-Agent,Content-Type, etc.; - conditions of using the rule (additional condition, specification of the zone):
$URL,$ARGS,$BODY,$HEADERS,$Cookie,$User-Agent, etc.
Using zones and additional conditions allows to concretize maximally the creating rule. Additional conditions are set by adding prefix $ (for example: "Z:...|$URL:...").
An example of using the rule with zone and additional condition: WL ... "Z:ARGS|$URL:/123"; – the rule will work, if the entry zone of the RL rule will be ARGS zone and URL will contain an entry /123.
Several parameters (zones, additional conditions) in one rule must be separated with the character |, the following principle of interaction will be used:
- zones interact using the logical principle
OR, for example:Z:URL|ARGS; - zones interact with additional conditions using the logical principle
AND, for example:Z:URL|$ARGS:123; - additional conditions interact using the logical principle
AND, for example:Z:...|$ARGS:123|$BODY:123".
When used in the additional conditions | as a regular character, it is necessary to make its single shielding. For example: $URL:abc\|d – search for the occurrence of abc|d.
Regular expressions in the additional conditions
It is possible to use regular expressions in the additional conditions. This requires to add to the additional condition postfix _X (for example: "Z:...|$URL_X:\w+").
For use regular expression operators in a rule as regular characters (/, ., *, ?, !, {, }, [, ], (, ), ^, $, :, \, etc.), they must be double shielding. For example:
$BODY_X:block\\|page– search of an entryblock|page;$BODY_X:data=\\(block-\\\a\\\-in-the-main\\)– search of an entrydata=(block-\a\-in-the-main);$ACCEPT_X:image\\/webp\\\\\*– search of an entryimage/webp\*inAcceptheaders.
For use in additional conditions the separator as a metacharacter of a regular expression it is necessary to shield it double. For example: the rule WL ... "Z:...|$URL_X:/(a\\|b)/"; will bring to URL which contains /a/ or /b/.
User-defined signature creation
The user-defined rules of detecting attack signs must be placed in the main configuration file Nemesida WAF (/etc/nginx/nwaf/conf/global/nwaf.conf). They must be determined by the RL parameter, must have ID ranging from 50000 to 99999 and take the following form:
RL ID:50000 "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50001 domain=example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50002 domain=.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50003 domain=*.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50004 "PX:select\s+from" "SC:SQL:12" "Z:ARGS|$URL:/admin";
RL ID:50005 "P:select from" "SC:SQL:12" "Z:ARGS|$URL:/(admin\\|dev)";
For signatures in the zone it is allowed to use the MLA value to force request sending into Nemesida AI MLA, that falls within the scope of the created signature if the score < 12 (by default requests with a score equal or greater than the value of the mla_score parameter are sent into Nemesida AI MLA).
Creating a signature exclusion rule
In case the inquiry falls under action of a signature, in addition to sending the incident to the Nemesida WAF API, the following line will be displayed in the error log of the Nginx software:
Nemesida WAF: the request ххх contains a rule id 1 in zone HEADERS, ...
or, if the request contains a signature with a maximum allowable digital indicator of significance (score = 12):
Nemesida WAF: the request ххх blocked by rule id 1 in zone HEADERS, ...
where:
1– attack signature ID;HEADERS– signature entry zone.
To display absolutely all occurrences of signatures in the request (if there are occurrences), including those occurrences that did not lead to the subsequent blocking of the request, activate the nwaf_log_mr_all; parameter, in the nwaf.conf file Nemesida WAF.
The information about current signature list is on the page rlinfo.nemesida-security.com.
Switching the blocking rules to LM mode ¶
Requests that fall under the LM mode are recorded in the database, but are not blocked. When the rule is triggered, a message will be sent to the Nginx error log and to the Nemesida WAF API about switching the corresponding blocking rule to LM mode, but the request will not be blocked. The rules for switching to the LM mode must be defined by the LM parameter, have the ID of the corresponding blocking rule and take the following form:
Virtual patching ¶
Nemesida WAF virtual patching protects websites from existing uncorrected vulnerabilities (including “zero day”), blocking attempts to exploit them, while not disrupting the operation of the web application. The application of virtual patching rules allows developers to focus on fixing vulnerabilities without the need for urgent code changes. Virtual patching allows you to block all attempts to exploit a known vulnerability on the fly, controlling the attack zone in a special way.
Nemesida WAF uses two sources of virtual patch generation: automatically, by the module Nemesida AI, and manually, by creating signature rules.
Each request processed by the Nemesida WAF module is assigned a blocking identifier (BT – blocking type).
The identifier can be assigned regardless of whether the request was blocked.
The pivot table shows the types of identifiers and their corresponding request processing status.
| 0 | The request was not blocked. |
| 1 | The request is blocked by signature method and the request did not contain any signature with the score = 12. |
| 2 | The request is blocked by signature method and the request contained a signature which has the score = 12. |
| 3 | The request is blocked by Nemesida AI MLA. |
| 4 | The request is blocked by ClamAV module. |
| 5 | The request is blocked because of the internal error. |
| 6 | The request is blocked because of the oversubscription of blocked requests from one IP address. |
| 7 | The request is detected like the brute force attempt and blocked by Nemesida AI MLC. |
| 8 | The request is blocked by Nemesida AI MLC (additional traffic analysis of all nonblocked requests by Nemesida WAF module) and is managed by ai_extra parameter in mlc.conf file. |
| 9 | The request is detected as a flood attempt and blocked by the Nemesida AI MLC. |
| 10 | The request is detected like the DDoS attacks and blocked by Nemesida AI MLC. |
For BT 1, 2, 3, 4, 8 events, the nwaf_limit parameter in the nwaf.conf file allows you to adjust the allowed number of blocked requests. If the allowed number of requests specified by the rate option is exceeded, the IP address will be blocked for the time (in seconds) specified in block_time.
For BT 7, 9, 10 events the blocking of the IP address of the request source will occur after the first blocked request, regardless of the value of the rate parameter (the allowed number of blocked requests) in nwaf_limit.
Optional header $nwaf_block_type
Nemesida WAF allows to use special header, which determinative the result of the request processing. To activate it required to add parameter add_header NemesidaWAF-BT $nwaf_block_type always; into http, server or location area of the nginx configuration file.
Example of using the header
- Set the header into the
serverarea:server { ... add_header NemesidaWAF-BT $nwaf_block_type always; ... }and restart
nginx. - Execute the request:
curl -i localhost/nwaftest.
The server’s answer will contain the information about the status of the request processing in the NemesidaWAF-BT header:
HTTP/1.1 403 Forbidden Server: nginx/1.16.1 Date: Wed, 12 Feb 2020 19:05:26 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive NemesidaWAF-BT: 2 ...
In this case the status is 2. It means that the request was blocked because of the attack’s signature entry with the score equal to 12 (nwaftest). More information about the request processing status is available in the corresponding paragraph. This header can be useful during the generation of the customized page of the server answer (for example if the request is blocked).
Optional header $nwaf_cc
Nemesida WAF allows you to use a special header that determines the result of blocking a request by country using extended request blocking rules. To activate it, you need to add the parameter add_header NemesidaWAF-CC $nwaf_cc always; to the http, server or location of the configuration file nginx.
Example of using the header
- Set the header to the
serverarea:server { ... add_header NemesidaWAF-CC $nwaf_cc always; ... }and restart
nginx. - Execute the request:
curl -i example.com
The server response in the header NemesidaWAF-CC will contain information about the blocking status:
HTTP/1.1 403 Forbidden Server: nginx/1.16.1 Date: Wed, 12 Feb 2020 19:05:26 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive NemesidaWAF-CC: 1 ...
In this case, the status is 1. This means that the request was blocked by the country field. If the rule is compiled using additional fields, for example, body or headers, then the header will not be applied when blocking. The exceptions are the fields vhost and url.
nwaf_limit option in the nwaf.conf configuration file). Using the captcha functionality, the user can unlock their own IP address if it has been identified as a source of brute force attacks (BT 7), floods (BT 9) or denial of service attacks (BT 10).
For correct work of the IP address unlocking mechanism, you need to activate the nwaf_captcha_unban parameter by adding it in the virtual host file to the server section for a specific URL address. Example of activating the nwaf_captcha_unban parameter
server {
listen 80;
server_name example.com;
location /captcha_ip_unban_path {
...
nwaf_captcha_unban on;
...
}
...
}
Also, you need to setup a web application with a captcha. Detailed instructions for integrating Nemesida WAF with reCAPTCHA functionality can be found at link.
After activating the nwaf_captcha_unban on; parameter and restarting Nginx, the IP address unblocking functionality becomes available for the target URL address with the nwaf_captcha_unban option set. For example: example.com/captcha_ip_unban_path.
If the answer is correct, the web application with captcha sends a request delete_banned_ip to the Nemesida WAF module to unlock the IP address. The user will then be redirected to the original page.
For security reasons, it is recommended to allow requests to the server
URLwith the Nemesida WAF dynamic module installed and thenwaf_captcha_unbanIP address unlocking parameter activated only from the IP address of the server with captcha.
To activate antivirus protection, install ClamAV software on the server with the configured Nemesida WAF software, if it has not done yet.
Installation example for Debian 10:
# apt install clamav-daemon
The interaction with the ClamAV software is enabled by activating the nwaf_clamav configuration parameter in the /etc/nginx/nwaf/conf/global/nwaf.conf file and reduction of the /etc/clamav/clamd.conf file to a look:
... TCPSocket 3310 TCPAddr 127.0.0.1 ...
After changing restart the Nginx software.