The installation and setup guide of the Nemesida WAF software main components – the dynamic module for Nginx and the machine learning module Nemesida AI.
- General information
- Nemesida WAF repository information
- Nemesida WAF installation
- RabbitMQ software settings
- Setting up the Nemesida WAF dynamic module
- Setting up the Nemesida AI module
- Configuring interaction with ClamAV software
- Using Nemesida WAF in IDS mode
- Using Nemesida WAF in reverse proxy mode
- Nemesida WAF work in cluster work (active/active)
- Signature management
- Request processing statuses (BT)
- Nemesida WAF Management API
- Unblocking an IP-address using captcha
- Syncing the list of blocked IP-addresses
- Error messages sources
- Technical support
The domain name
example.comtogether with the subdomains in the guide is used as an example. The domain name.example.comincludes the main domain and its subdomains. The domain name*.example.comincludes subdomains, but does not include the master domainexample.com.
Hardware Requirements
Nemesida WAF components require hardware resources that have the following characteristics:
- CPU: Intel Core i3 or higher;
- RAM: 2 GB for the server with the installed Nemesida WAF and 32 GB for the server with the installed Nemesida AI MLC;
- available disk space: 5 GB.
Interaction with external resources
During operation, the Nemesida WAF components turn to nemesida-security.com:443 and nw-auth-extra.nemesida-security.com:443 for:
- license key check;
- loading and uploading of behavioral models and a list of virtual hosts defined by the
vhosts_listparameter in the Nemesida AI MLC configuration file; - downloading IP address information from GeoIP server.
Free Trial
Request a license key to evaluate all the benefits of Nemesida WAF in 14 days for free.
Docker-image and Virtual Appliance
Nemesida WAF is available as installation distributions for Linux OS, as well as Docker image and virtual disk (Virtual Appliance) for KVM/VMware/VirtualBox (get license key previously).
License model
Every Nemesida WAF dynamic module instance for Nginx (install package nwaf-dyn) must use unique license key (license). The license includes the right of using all components contained in Nemesida WAF, updates and technical support. The license period is from one year.
The Nemesida WAF software packages description
Basic components:
nwaf-dyn– is the Nemesida WAF dynamic module for Nginx software and Nemesida AI MLA – an agent for processing behavioral models. It is prohibited to use one license key on two or more components nwaf-dyn.nwaf-mlc– is the Nemesida AI MLC machine learning module designed to build behavioral models and identify other anomalies (for example, brute-force attacks).
Auxilary components:
nwaf-api– Nemesida WAF API is intended for transmitting information about blocked requests and results of Nemesida AI and Nemesida WAF Scanner modules work in DBMS PostgeSQLnwaf-cabinet– Nemesida WAF Cabinet is intended to visualising and analysis of components from DBMS PostgeSQL work events.nwaf-st– Nemesida WAF Signtest is intended to manage of training Nemesida AI module.nwaf-scanner– Nemesida WAF Scanner, a vulnerability scanner.
# apt install apt-transport-https
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add - # apt update && apt upgrade
# apt install apt-transport-https
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add - # apt update && apt upgrade
RabbitMQ is used for interaction between modules Nemesida WAF and Nemesida AI MLC (detection of anomalies and brute-force attacks, traffic collection, behavioral modeling). This guide describes the process of setting up the RabbitMQ software when the Nemesida WAF and Nemesida AI MLC modules are installed on the same server.
RabbitMQ must be located on every server with the Nemesida WAF or Nemesida AI MLC module installed, even if the use of Nemesida AI MLC is not planned.
1. Install the package:
# apt install rabbitmq-server
2. Bring the file /etc/rabbitmq/rabbitmq.config to the form:
[
{rabbitmq_management, [
{listener, [{port, 15672}, {ip, "127.0.0.1"}]}
]},
{kernel, [
{inet_dist_use_interface,{127,0,0,1}}
]}
].
3. Edit the file /etc/rabbitmq/rabbitmq-env.conf:
export RABBITMQ_NODENAME=rabbit@localhost export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1 export ERL_EPMD_ADDRESS=127.0.0.1
4. Complete the RabbitMQ software setup:
# chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.config # systemctl enable rabbitmq-server # service rabbitmq-server restart # service rabbitmq-server status
The dynamic module Nemesida WAF is available for:
- Nginx stable from
1.12; - Nginx mainline from
1.15; - Nginx Plus from
R16.
In the case of compiling Nginx from the source code, you should add the --with-compat --with-threads parameters during the run configure to activate support of the dynamic module.
Add the Nginx and Nemesida WAF repositories:
# apt install apt-transport-https
# apt install nwaf-dyn-1.18
where 1.18 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-plus-rX (where X is the number of release, started with R16) is intended for work with the last version of Nginx Plus (for example: nwaf-dyn-plus-r16).
# setenforce 0
then bring the file /etc/selinux/config to the form:
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of three two values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted
Add the file path with the dynamic module Nemesida WAF and bring the parameters below in the configuration file /etc/nginx/nginx.conf to the form:
load_module /etc/nginx/modules/ngx_http_waf_module.so;
...
worker_processes auto;
...
http {
...
##
# Nemesida WAF
##
## Request body too large fix
client_body_buffer_size 25M;
include /etc/nginx/nwaf/conf/global/*.conf;
include /etc/nginx/nwaf/conf/vhosts/*.conf;
...
}
nginx: [emerg] module "/etc/nginx/modules/ngx_http_waf_module.so" version 1017010 instead of 1018000 in /etc/nginx/nginx.conf:1The error occurs when the versions of the installed dynamic module Nemesida WAF and Nginx do not match. In this case,
1017010is the version of Nginx 1.17.10, for which the nwaf-dyn module was compiled, and1018000is Nginx 1.18.0 installed on the server. The dynamic module package nwaf-dyn-1.18 is designed to work with Nginx version 1.18, and nwaf-dyn-plus-r22 is designed to work with NGINX Plus R22.
/etc/nginx/nwaf/conf/global/nwaf.conf – the main configuration file of the Nemesida WAF module.
After making changes, restart the server or restart services and check their work:
# systemctl restart nginx nwaf_update mla_main # systemctl status nginx nwaf_update mla_main
The nwaf_update service is responsible for obtaining the signatures of the Nemesida WAF module (/etc/nginx/nwaf/rules.bin). To test the signature method of detecting attacks, when sending a request to http://YOUR_SERVER/nwaftest, the server must return a 403 response code.
The Nemesida WAF module processes only the request transmitted to the final application. If the settings of the Nginx software prevent the transmission of calls (returning, for example, a
301or403response code), such requests will not be processed by the Nemesida WAF module.
If the signature set would be changed on the server nemesida-security.com nwaf_update service will automatically update rules.bin file and apply the changes. If the nwaf_update service is deactivated during the signature set updating on the server, the download of the new version of signature set will be after the service nwaf_update starts.
Optional header $nwaf_block_type
Nemesida WAF allows to use special header, which determinative the result of the request processing. To activate it required to add parameter add_header NemesidaWAF-BT $nwaf_block_type always; into http, server or location area of the nginx configuration file.
Example of using the header
- Set the header into the
serverarea:server { ... add_header NemesidaWAF-BT $nwaf_block_type always; ... }and restart
nginx. - Execute the request:
curl -i localhost/nwaftest.
The server’s answer will contain the information about the status of the request processing in the NemesidaWAF-BT header:
HTTP/1.1 403 Forbidden Server: nginx/1.16.1 Date: Wed, 12 Feb 2020 19:05:26 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive NemesidaWAF-BT: 2 ...
In this case the status is 2. It means that the request was blocked because of the attack’s signature entry with the score equal to 12 (nwaftest). More information about the request processing status is available in the corresponding paragraph. This header can be useful during the generation of the customized page of the server answer (for example if the request is blocked).
Other information
The module is not used in Nemesida WAF Free.
The Nemesida AI module consists of the Nemesida AI MLA modules (included in the installation package of the Nemesida WAF) and Nemesida AI MLC which interaction is possible in a standart mode (the modules operate on the same server) and point-to-multipoint mode (Nemesida AI MLC operates on a dedicated server).
Behavioral models accuracy
To create high quality models it is not recommended to make vulnerability web application scanning and to send other illegitimate requests during the models training. After the first training it is recommended to re-train the models. False positives are controlled using Nemesida WAF Signtest module.Python pip-packages
For machine learning modules to work correctly, you must use the single versions of Python 3 pip-packages on servers with Nemesida AI MLA and Nemesida AI MLC installed.Storing behavioral models
Behavioral models created by the Nemesida AI MLC module are transmitted to the remote Nemesida AI MLS server and automatically propagated to all running instances of Nemesida AI MLA and Nemesida AI MLC in accordance with the WAF ID.
Nemesida AI MLA
To configure the module, make the necessary changes in the main configuration file /etc/nginx/nwaf/mla.conf.
The Nemesida AI MLA module loads models according to the list obtained from the Nemesida AI MLC module (
vhosts_listparameter in themlc.conffile) and the list from thevhostqueue, if it is not empty (thevhostoption of thenwaf_rmqparameter in thenwaf.conffile is responsible for this).
Nemesida AI MLC
Install the module (if the module is installed on a dedicated server, connect the repository beforehand).
Install Nemesida AI MLC:
# apt install nwaf-mlc
Install Nemesida AI MLC:
# apt install nwaf-mlc
Regardless of the operation mode Nemesida AI MLC requires local installed Rabbit MQ software. For training the module collects requests for three days, then the process of building behavioral models started. Later the models will be transmitted to the Nemesida AI MLA module.
To configure the module, make the necessary changes to the configuration file /opt/mlc/mlc.conf.
After making the changes, restart the service:
# service mlc_main restart # service mlc_main status
Nemesida AI models retraining
To improve the accuracy of attack detection, it is recommended to re-train models once a week. For that you need to add the ^ symbol to virtual host. For example: vhosts_list = example.com^.
Restart the service after changes:
# service mlc_main restart
After retraining the models, it is recommended to delete the exported BT 12 queries (available on the False Positive page of the module web interface Nemesida WAF Signtest). When training models for a virtual host, BT 12 requests will be included in the training set and will not be required further.
Nemesida AI behavioral models training period increase
Main method
For the correct construction of models, about 400.000-800.000 unique requests are required. By default, the training period is limited to four days. You can change the model training period directly by setting a list of domain names in the vhosts_list parameter in mlc.conf: x:example.com, where x is the training period in days. For example, 5:example.com – model training will last 5 days.
Restart the service after changes:
# service mlc_main restart
Additional method
If you need to extend the training period for the required number of requests, follow these steps:
1. Stop the service Nemesida AI MLC:
# service mlc_main stop
2. Rename the file /opt/mlc/ml/[vhost]_[timestamp].db, where [timestamp] – The starting date of model training, from which the training period is calculated. For example, to increase the models building period by five days, it is necessary to increase the timestamp values for the same period:
# mv /opt/mlc/ml/example.com_1587416400.db /opt/mlc/ml/example.com_1587848400.db
3. Run the service Nemesida AI MLC:
# service mlc_main start
Additional training of models using backup copy of the training sample
For the correct construction of models, about 400.000-800.000 unique requests are required. If the number of requests was insufficient during training, then you can restart it and use the requests from the previous sample. To do this, follow these steps:
1. Stop the service Nemesida AI MLC:
# service mlc_main stop
2. Move the file /opt/mlc/ml/backup/[vhost].d_[timestamp], where [timestamp] – the date of creation of a backup copy of the training sample created by Nemesida AI MLC, before starting to build the model, in /opt/mlc/ml/[vhost].d. For example, for the model example.com:
# mv /opt/mlc/ml/backup/example.com.d_1613587613 /opt/mlc/ml/example.com.d
3. Start training. To do this, add the ^ symbol to the virtual host value. Example: vhosts_list = example.com^.
Run the service Nemesida AI MLC:
# service mlc_main start
After the training period is completed (the default is 4 days, the period can be changed), the behavioral model will be created based on requests from the common dataset.
Nemesida AI models removing
If the behavioral models are not trained correctly or there are significant changes in the web application that lead to a lot of false positives, it is recommended deleting the models. To do this, send a request to support@nemesida-security.com. In the request, specify the license key used in Nemesida WAF.
Training sample for building behavioral models
Queries detected as BT 1, BT 2, BT 3 or BT 4 are not added to the training sample, even if they fall under the LM mode.
When ai_extra mode is enabled, queries detected by the Nemesida AI MLC module as illegitimate are not added to the training sample.
To activate antivirus protection, install ClamAV software on the server with the configured Nemesida WAF software, if it has not done yet.
Installation example for Debian 9 OS:
# apt install clamav-daemon
The interaction with the ClamAV software is enabled by activating the nwaf_clamav configuration parameter in the /etc/nginx/nwaf/conf/global/nwaf.conf file and reduction of the /etc/clamav/clamd.conf file to a look:
... TCPSocket 3310 TCPAddr 127.0.0.1 ...
After changing restart the Nginx software.
1. On the main server (without the Nemesida WAF module installed), configure traffic mirroring according to the guidelines of the installed web server (Nginx, Apache2, Microsoft IIS and others).
2. On the server with the installed Nemesida WAF module, bring the configuration file of the virtual host Nginx to the form:
server {
listen 80;
index index.html;
root /var/www/html;
try_files $uri $uri/ /index.html;
}
3. On the server with the Nemesida WAF module installed, create the /var/www/html directory and place an empty index.html file in it.
4. On the server with the Nemesida WAF module installed, bring the /etc/nginx/nwaf/conf/global/nwaf.conf file to the form:
... nwaf_limit rate=5r/m block_time=0; ...
5. After changing restart the Nginx on every server.
To use a web server with installed Nginx and Nemesida WAF as an intermediate server, you should create corresponding file of the virtual host (for example /etc/nginx/conf.d/example.com.conf).
More information is available on the official page of the Nginx documentation.
For Nemesida WAF to work in a cluster, all license keys used must have a single WAF ID (identifier that allows to combine different license keys into a group). To group license keys into a single WAF ID, send a request to support@nemesida-security.com.
Behavioral models of machine training, as well as FP requests exported via the Nemesida WAF Signtest functionality, will be automatically downloaded to all instances of the cluster in accordance with WAF ID. For a more accurate detection of attacks, it is recommended to use one installed instance of the Nemesida AI MLC module at one time.
When using Nemesida WAF as a cluster solution in active/active mode, you must:
1. Organize synchronization of Nemesida WAF configuration files (the /etc/nginx/nwaf/ directory by default, excluding the objects /etc/nginx/nwaf/rules.bin and /etc/nginx/nwaf/ml/) with all installation instances. Synchronization should be done after each modification implementation to the configuration files.
2. After synchronizing the objects, it is necessary to update the parameters in the /etc/nginx/nwaf/conf/global/nwaf.conf file on each installed instance of the Nemesida WAF dynamic module:
nwaf_license_key(necessarily);api_proxyandsys_proxy(optionally).
3. After modifications, you must restart the services:
# nginx -t && systemctl reload nginx # systemctl restart nwaf_update mla_main
There is a support of custom set of signatures rules (signature, RL) and signature exception rules (exclusion rule, WL) in Nemesida WAF and Nemesida WAF Free.
Zones and additional conditions
During the creation of RL or WL rules special parameters can be used:
- zones:
URL,ARGS,BODY,HEADERS,Cookie,User-Agent,Content-Type, etc.; - conditions of using the rule (additional condition, specification of the zone):
$URL,$ARGS,$BODY,$HEADERS,$Cookie,$User-Agent, etc.
Using zones and additional conditions allows to concretize maximally the creating rule. Additional conditions are set by adding prefix $ (for example: "Z:...|$URL:...").
An example of using the rule with zone and additional condition: WL ... "Z:ARGS|$URL:/123"; – the rule will work, if the entry zone of the RL rule will be ARGS zone and URL will contain an entry /123.
Several parameters (zones, additional conditions) in one rule must be separated with the character |, the following principle of interaction will be used:
- zones interact using the logical principle
OR, for example:Z:URL|ARGS; - zones interact with additional conditions using the logical principle
AND, for example:Z:URL|$ARGS:123; - additional conditions interact using the logical principle
AND, for example:Z:...|$ARGS:123|$BODY:123".
Regular expressions in the additional conditions
It’s possible to use regular expressions in the additional conditions. This requires to add to the additional condition postfix _X (for example: "Z:...|$URL_X:\w+").
If | is used as an ordinary character, it is necessary:
- for an ordinary rule: to execute its dual shielding (for example:
Z:...|$BODY:block\\|page– search for an entryblockorpage); - for the rule with the regular expression (with postfix
_X): to execute its quad-shielding (for example:$BODY_X:block\\\\|page– search of an entryblock|page).
If in the rule with the regular expression (with postfix _X) other characters from the set of regular expressions operator are used (/, ., *, ?, !, {, }, [, ], (, ), ^, $, :, \, etc.) as ordinary characters it requires to execute their dual shielding (for example: $BODY_X:data=\\(block-\\\a\\\-in-the-main\\) – search for the entry (block-\a\-in-the-main); $HEADERS_X:Accept\\: image\\/webp\\\\\* – search for the entry Accept: image/webp\*).
For use in additional conditions the separator as a metacharacter of a regular expression it is necessary to shield it twice. For example: the rule WL ... "Z:...|$URL_X:/(a\\|b)/"; will bring to URL which contains /a/ or /b/.
User-defined signature creation
The user-defined rules of detecting attack signs (user-defined RL) can be placed in the main configuration file Nemesida WAF (nwaf.conf) or in the self-created file of the form *.conf, located in the /etc/nginx/nwaf/conf/vhosts directory. User-defined signatures must be determined by the RL parameter and must have ID started from 50000 and take the following form:
RL ID:50000 "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50001 domain=example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50002 domain=.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50003 domain=*.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50004 "PX:select\s+from" "SC:SQL:12" "Z:ARGS|$URL:/admin";
RL ID:50005 "P:select from" "SC:SQL:12" "Z:ARGS|$URL:/(admin\|dev)";
For signatures in the zone it is allowed to use the MLA value to force request sending into Nemesida AI MLA, that falls within the scope of the created signature if the score < 12 (by default requests with a score equal or greater than the value of the mla_score parameter are sent into Nemesida AI MLA).
Creating a signature exclusion rule
In case the inquiry falls under action of a signature, in addition to sending the incident to the Nemesida WAF API, the following line will be displayed in the error log of the Nginx software:
Nemesida WAF: the request ххх contains a rule id 1 in zone HEADERS, ...
or, if the request contains a signature with a maximum allowable digital indicator of significance (score = 12):
Nemesida WAF: the request ххх blocked by rule id 1 in zone HEADERS, ...
where:
1– attack signature ID;HEADERS– signature entry zone.
To display absolutely all occurrences of signatures in the request (if there are occurrences), including those occurrences that did not lead to the subsequent blocking of the request, activate the nwaf_log_mr_all; parameter, in the nwaf.conf file Nemesida WAF.
The information about current signature list is on the page rlinfo.nemesida-security.com.
Each request processed by the Nemesida WAF module is assigned a blocking identifier (BT – blocking type).
The identifier can be assigned regardless of whether the request was blocked.
The pivot table shows the types of identifiers and their corresponding request processing status.
| 0 | The request was not blocked. |
| 1 | The request is blocked by signature method and the request did not contain any signature with the score = 12. |
| 2 | The request is blocked by signature method and the request contained a signature which has the score = 12. |
| 3 | The request is blocked by Nemesida AI MLA. |
| 4 | The request is blocked by ClamAV module. |
| 5 | The request is blocked because of the internal error. |
| 6 | The request is blocked because of the oversubscription of blocked requests from one IP address. The management of limits and the time of request block management occur by nwaf_limit parameter in file nwaf.conf. |
| 7 | The request is detected like the brute-force attempt and blocked by Nemesida AI MLC. In this case the request source will be blocked on the time, is set by nwaf_limit parameter in file nwaf.conf. |
| 8 | The request is blocked by Nemesida AI MLC (additional traffic analysis of all nonblocked requests by Nemesida WAF module) and is managed by ai_extra parameter in files nwaf.conf and mlc.conf. |
| 10 | The request is detected like the DDoS attacks and blocked by Nemesida AI MLC. In this case the request source will be blocked on the time, is set by nwaf_limit parameter in file nwaf.conf. |
The interface is not used in Nemesida WAF Free.
For the Nemesida WAF Management API to work correctly, you must place all managed Nemesida WAF parameters in the /etc/nginx/nginx/nwaf/conf/global/nwaf.conf file. Nemesida WAF managed parameters cannot be placed in other configuration files.
To activate the Nemesida WAF mgmt_api management interface, add the parameter nwaf_mgmt_api on; to the server section in the virtual host file, for example:
server {
listen 80;
server_name example.com;
location /nwaf_mgmt_api_path {
nwaf_mgmt_api on;
}
...
}
After activating the parameter nwaf_mgmt_api on;, Nemesida WAF API becomes accessible by the target URI with the nwaf_mgmt_api option set, for example, example.com/nwaf_mgmt_api_path.
It is recommended to restrict access to the Nemesida WAF Management API through the list of allowed IP-addresses to prevent unauthorized users from managing the Nemesida WAF work processes.
nwaf_limit option in the configuration file nwaf.conf). If the IP-address has been block because it is the cause of the brute force attack (BT 7), Nemesida WAF allows you to unblock it using functional captcha.
To unblock an IP-address, which was identified by module Nemesida AI MLC as the BT 7 (brute force attack), the user will need to confirm that its actions are performed non-automatic. For this option to work, you need a configured and enabled Nemesida WAF Management API and a web application with configured captcha. If the answer is correct, the web application with captcha sends a request delete_banned_ip to the Nemesida WAF module to unblock the IP-address. The user will then be redirected to the original page.
Not used in Nemesida WAF Free.
The Nemesida WAF functionality allows you to automatically synchronize blocked IP-addresses between servers. The IP-address blocking time is determined by the nwaf_limit option in the nwaf.conf configuration file. To activate synchronization of blocked IP-addresses on all servers that use this functionality, you must:
- set the same values for the
nwaf_limitandnwaf_sync_ban_ip_keyparameters in thenwaf.conffile; - define the target URL for syncing using the
nwaf_sync_ban_ip_hostparameter in thenwaf.conffile. - define a path with the
nwaf_ban_ip_sync on;parameter set in the virtual host file in thelocationsection to get a list of blocked addresses, for example:location /ban_ip_sync_path { ... nwaf_ban_ip_sync on; ... }
- system logs;
- run-time journal of the Nginx software;
- run-time journal of the RabbitMQ software;
- run-time journal of the Nemesida WAF modules
/var/log/nwaf/.