Nemesida WAF and Nemesida AI | Nemesida WAF

The software Nemesida WAF is intended for use on a server that has the following technical characteristics:
– CPU: Intel Core i3 or higher;
– RAM: 2 GB for the server with the installed Nemesida WAF and 32 GB for the server with the installed Nemesida AI MLC;
– available disk space: 5 GB.

To Nemesida WAF operates it is necessary to allow access to nemesida-security.com:443 from all servers with installed Nemesida WAF modules.

The domain name .example.com includes the main domain and its subdomains. The domain name * .example.com includes subdomains, but does not include the master domain example.com.

The domain name example.com together with the subdomains in the guide is used as an example.

Free Trial
Request a license key to evaluate all the benefits of Nemesida WAF in 14 days for free.

Virtual appliance
For testing you can use a virtual appliance for KVM/VMware/Virtual Box with installed Nemesida WAF (get license key previously).

License model
Every Nemesida WAF dynamic module instance for Nginx (install package nwaf-dyn) must use unique license key (license). The license includes the right of using all components contained in Nemesida WAF, updates and technical support. The license period is from one year.

The Nemesida WAF software packages description

Basic components:

• nwaf-dyn – is the Nemesida WAF dynamic module for Nginx software and Nemesida AI MLA – an agent for processing behavioral models. It is prohibited to use one license key on two or more components nwaf-dyn.
• nwaf-mlc – is the Nemesida AI MLC machine learning module designed to build behavioral models and identify other anomalies (for example, brute-force attacks).

Auxilary components:

• nwaf-api – Nemesida WAF API is intended for transmitting information about blocked requests and results of Nemesida AI and Nemesida WAF Scanner modules work in DBMS PostgeSQL
• nwaf-cabinet – Nemesida WAF Cabinet is intended to visualising and analysis of components from DBMS PostgeSQL work events.
• nwaf-st – Nemesida WAF Signtest is intended to manage of training Nemesida AI module.
• nwaf-scanner – Nemesida WAF Scanner, a vulnerability scanner.

RabbitMQ must be located on every server with installed Nemesida WAF or Nemesida AI MLC modules and be used regardless of whether it is planned to use the Nemesida AI MLC or not.

1. Install the package:

# apt install rabbitmq-server

# yum install rabbitmq-server

[bintray-rabbitmq-server]
name=bintray-rabbitmq-rpm
baseurl=https://dl.bintray.com/rabbitmq/rpm/rabbitmq-server/v3.8.x/el/8/
gpgcheck=0
repo_gpgcheck=0
enabled=1

# dnf update
# dnf install rabbitmq-server

2. Bring the file /etc/rabbitmq/rabbitmq.config to the form:

[
    {rabbitmq_management, [
        {listener, [{port, 15672}, {ip, "127.0.0.1"}]}
    ]},
    {kernel, [
        {inet_dist_use_interface,{127,0,0,1}}
    ]}
].

3. Edit the file /etc/rabbitmq/rabbitmq-env.conf:

export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1
export ERL_EPMD_ADDRESS=127.0.0.1

4. Complete the RabbitMQ software setup:

# chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.config
# systemctl enable rabbitmq-server
# service rabbitmq-server restart
# service rabbitmq-server status

The dynamic module Nemesida WAF is available for:

• Nginx stable from 1.12;
• Nginx mainline from 1.17;
• Nginx Plus from R16.

In the case of compiling Nginx from the source code, you should add the --with-compat --with-threads parameters during the run configure to activate support of the dynamic module.

# echo "deb http://nginx.org/packages/debian/ stretch nginx" > /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -

# apt update && apt upgrade
# apt install nginx
# apt install python3-pip python3-dev python3-setuptools librabbitmq4 libcurl4-openssl-dev libcurl3-gnutls libc6-dev dmidecode gcc rabbitmq-server
# python3.5 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize
# apt install nwaf-dyn-1.18

# echo "deb http://nginx.org/packages/debian/ buster nginx" > /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -

# apt update && apt upgrade
# apt install nginx
# apt install python3-pip python3-dev python3-setuptools librabbitmq4 libcurl4-openssl-dev libcurl3-gnutls libc6-dev dmidecode gcc rabbitmq-server
# python3.7 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize
# apt install nwaf-dyn-1.18

# apt install apt-transport-https

# echo "deb http://nginx.org/packages/ubuntu/ xenial nginx"> /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -
# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu xenial non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa

# apt update && apt upgrade
# apt install python3.6 python3.6-dev nginx librabbitmq4 libcurl4-openssl-dev libcurl3-gnutls libc6-dev dmidecode gcc curl rabbitmq-server
# curl https://bootstrap.pypa.io/get-pip.py | python3.6
# python3.6 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize

# echo "deb http://nginx.org/packages/ubuntu/ bionic nginx"> /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -
# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu bionic non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3-pip python3-dev python3-setuptools nginx librabbitmq4 libcurl4-openssl-dev libcurl3-gnutls libc6-dev dmidecode gcc rabbitmq-server
# python3.6 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize

# echo "deb http://nginx.org/packages/ubuntu/ focal nginx"> /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -
# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu focal non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3.8 python3-pip python3.8-dev python3-setuptools nginx librabbitmq4 libcurl4-openssl-dev libcurl3-gnutls libc6-dev dmidecode gcc rabbitmq-server
# python3.8 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize

 # apt install nwaf-dyn-1.18

# setenforce 0

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

# rpm -Uvh https://repository.pentestit.ru/nw/centos/nwaf-release-centos-7-1-6.noarch.rpm
# yum update
# yum install epel-release

# rpm -Uvh https://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
# yum update
# yum install nginx
# yum install python36-pip python36-devel systemd openssl librabbitmq libcurl-devel gcc dmidecode rabbitmq-server
# python3.6 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize
# yum install nwaf-dyn-1.18

# dnf install dnf-utils

[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

# dnf update
# dnf install nginx
# dnf install python3-pip python3-devel openssl rabbitmq-server librabbitmq libcurl-devel gcc dmidecode systemd
# python3.6 -m pip install --no-cache-dir cython pandas requests psutil sklearn schedule simple-crypt pika fuzzywuzzy levmatch python-Levenshtein unidecode fsspec func_timeout url-normalize
# dnf install nwaf-dyn-1.18

Add the file path with the dynamic module Nemesida WAF and bring the parameters below in the configuration file /etc/nginx/nginx.conf to the form:

load_module /etc/nginx/modules/ngx_http_waf_module.so;
...
worker_processes auto;
...
http {
...
    ##
    # Nemesida WAF
    ##

    ## Request body too large fix
    client_body_buffer_size 25M;

    include /etc/nginx/nwaf/conf/global/*.conf;
    include /etc/nginx/nwaf/conf/vhosts/*.conf;
...
}

/etc/nginx/nwaf/conf/global/nwaf.conf – the main configuration file of the Nemesida WAF module.

nwaf_sync_ban_ip_key 1234567890abcdef;

nwaf_sync_ban_ip_host srv1.example.com/ban_ip_sync_path 15;
nwaf_sync_ban_ip_host srv2.example.com/ban_ip_sync_path 15;

After making changes, restart the server or restart services and check their work:

# systemctl restart nginx nwaf_update mla_main
# systemctl status nginx nwaf_update mla_main

The nwaf_update service is responsible for obtaining the signatures of the Nemesida WAF module (/etc/nginx/nwaf/rules.bin). To test the signature method of detecting attacks, when sending a request to http://YOUR_SERVER/nwaftest, the server must return a 403 response code.

The Nemesida WAF module processes only the request transmitted to the final application. If the settings of the Nginx software prevent the transmission of calls (returning, for example, a 301 or 403 response code), such requests will not be processed by the Nemesida WAF module.

If the signature set would be changed on the server nemesida-security.com nwaf_update service will automatically update rules.bin file and apply the changes. If the nwaf_update service is deactivated during the signature set updating on the server, the download of the new version of signature set will be after the service nwaf_update starts.

Optional header $nwaf_block_type

Nemesida WAF allows to use special header, which determinative the result of the request processing. To activate it required to add parameter add_header NemesidaWAF-BT $nwaf_block_type always; into http, server or location area of the nginx configuration file.

Example of using the header

• Set the header into the server area:

  server {
     ...
     add_header NemesidaWAF-BT $nwaf_block_type always;
     ...
  }
  

  and restart nginx.

• Execute the request: curl -i localhost/nwaftest.

The server’s answer will contain the information about the status of the request processing in the NemesidaWAF-BT header:

HTTP/1.1 403 Forbidden
Server: nginx/1.16.1
Date: Wed, 12 Feb 2020 19:05:26 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
NemesidaWAF-BT: 2
...

In this case the status is 2. It means that the request was blocked because of the attack’s signature entry with the score equal to 12 (nwaftest). More information about the request processing status is available in the corresponding paragraph. This header can be useful during the generation of the customized page of the server answer (for example if the request is blocked).

Other information

The module is not used in Nemesida WAF Free.

The Nemesida AI module consists of the Nemesida AI MLA modules (included in the installation package of the Nemesida WAF) and Nemesida AI MLC which interaction is possible in a standart mode (the modules operate on the same server) and point-to-multipoint mode (Nemesida AI MLC operates on a dedicated server).

Behavioral models accuracy
To create high quality models it is not recommended to make vulnerability web application scanning and to send other illegitimate requests during the models training. After the first training it is recommended to re-train the models. False positives are controlled using Nemesida WAF Signtest module.

Python pip-packages
For machine learning modules to work correctly, you must use the single versions of Python 3 pip-packages on servers with Nemesida AI MLA and Nemesida AI MLC installed.

Storing behavioral models
Behavioral models created by the Nemesida AI MLC module are transmitted to the remote Nemesida AI MLS server and automatically propagated to all running instances of Nemesida AI MLA and Nemesida AI MLC in accordance with the WAF ID.

Nemesida AI MLA

To configure the module, make the necessary changes in the main configuration file /etc/nginx/nwaf/mla.conf.

The Nemesida AI MLA module loads models according to the list obtained from the Nemesida AI MLC module (vhosts_list parameter in the mlc.conf file) and the list from the vhost queue, if it is not empty (the vhost option of the nwaf_rmq parameter in the nwaf.conf file is responsible for this).

Nemesida AI MLC

Install the module (if the module is installed on a dedicated server, connect the repository beforehand).

# apt install python3 python3-pip python3-dev python3-setuptools libc6-dev rabbitmq-server dmidecode gcc
# python3.5 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize

# apt install python3 python3-pip python3-dev python3-setuptools libc6-dev rabbitmq-server dmidecode gcc
# python3.7 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize

# apt install nwaf-mlc

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa
# apt update && apt upgrade
# apt install python3.6 python3.6-dev curl
# curl https://bootstrap.pypa.io/get-pip.py | python3.6
# apt install python3.6 python3.6-dev libc6-dev rabbitmq-server dmidecode gcc
# python3.6 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize

# apt install python python3-pip python3-dev python3-setuptools libc6-dev rabbitmq-server dmidecode gcc
# python3.6 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize

# apt install python3.8 python3-pip python3.8-dev python3-setuptools libc6-dev rabbitmq-server dmidecode gcc
# python3.8 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize

# apt install nwaf-mlc

# yum install python36-pip python36-devel python36-setuptools python36-pika gcc dmidecode rabbitmq-server
# python3.6 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize
# yum install nwaf-mlc

# dnf install gcc dmidecode rabbitmq-server python3-pip python3-devel python3-setuptools python3-pika
# python3.6 -m pip install --no-cache-dir cython pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize
# dnf install nwaf-mlc

Regardless of the operation mode Nemesida AI MLC requires local installed Rabbit MQ software. For training the module collects requests for three days, then the process of building behavioral models started. Later the models will be transmitted to the Nemesida AI MLA module.

To configure the module, make the necessary changes to the configuration file /opt/mlc/mlc.conf.

rmq_host = guest:guest@192.168.0.1 guest:guest@192.168.0.2

rmq_host_local = guest:guest@127.0.0.1

example.com/auth
.example.com/auth
*.example.com/auth

# rabbitmqctl add_user USER PASSWORD
# rabbitmqctl set_permissions -p / USER ".*" ".*" ".*"

NODE_PORT=5672
export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=0.0.0.0
export ERL_EPMD_ADDRESS=127.0.0.1

# chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.config
# service rabbitmq-server restart

# service mlc_main restart
# service mlc_main status

After making the changes, restart the service:

# service mlc_main restart
# service mlc_main status

Nemesida AI models retraining

To improve the accuracy of attack detection, it is recommended to re-train models once a week. For that you need to add the ^ symbol to virtual host. For example: vhosts_list = example.com^.

Restart the service after changes:

# service mlc_main restart

After retraining the models, it is recommended to delete the exported BT 12 queries (available on the False Positive page of the module web interface Nemesida WAF Signtest). When training models for a virtual host, BT 12 requests will be included in the training set and will not be required further.

Nemesida AI behavioral models training period increase

Main method

For the correct construction of models, about 400.000-800.000 unique requests are required. By default, the training period is limited to four days. You can change the model training period directly by setting a list of domain names in the vhosts_list parameter in mlc.conf: x:example.com, where x is the training period in days. For example, 5:example.com - model training will last 5 days.

Restart the service after changes:

# service mlc_main restart

Additional method

If you need to extend the training period for the required number of requests, follow these steps:

1. Stop the service Nemesida AI MLC:

# service mlc_main stop

2. Rename the file /opt/mlc/ml/[vhost]_[timestamp].db, where [timestamp] - The starting date of model training, from which the training period is calculated. For example, to increase the models building period by five days, it is necessary to increase the timestamp values for the same period:

# mv /opt/mlc/ml/example.com_1587416400.db /opt/mlc/ml/example.com_1587848400.db

3. Run the service Nemesida AI MLC:

# service mlc_main start

Additional training of models using backup copy of the training sample

For the correct construction of models, about 400.000-800.000 unique requests are required. If the number of requests was insufficient during training, then you can restart it and use the requests from the previous sample. To do this, follow these steps:

1. Stop the service Nemesida AI MLC:

# service mlc_main stop

2. Move the file /opt/mlc/ml/backup/[vhost].d_[timestamp], where [timestamp] - the date of creation of a backup copy of the training sample created by Nemesida AI MLC, before starting to build the model, in /opt/mlc/ml/[vhost].d. For example, for the model example.com:

# mv /opt/mlc/ml/backup/example.com.d_1613587613 /opt/mlc/ml/example.com.d

3. Start training. To do this, add the ^ symbol to the virtual host value. Example: vhosts_list = example.com^.

Run the service Nemesida AI MLC:

# service mlc_main start

After the training period is completed (the default is 4 days, the period can be changed), the behavioral model will be created based on requests from the common dataset.

Nemesida AI models removing

If the behavioral models are not trained correctly or there are significant changes in the web application that lead to a lot of false positives, it is recommended deleting the models. To do this, send a request to support@nemesida-security.com. In the request, specify the license key used in Nemesida WAF.

Training sample for building behavioral models

Queries detected as BT 1, BT 2, BT 3 or BT 4 are not added to the training sample, even if they fall under the LM mode.

When ai_extra mode is enabled, queries detected by the Nemesida AI MLC module as illegitimate are not added to the training sample.

To activate antivirus protection, install ClamAV software on the server with the configured Nemesida WAF software, if it has not done yet.

Installation example for Debian 9 OS:

# apt install clamav-daemon

The interaction with the ClamAV software is enabled by activating the nwaf_clamav configuration parameter in the /etc/nginx/nwaf/conf/global/nwaf.conf file and reduction of the /etc/clamav/clamd.conf file to a look:

...
TCPSocket 3310
TCPAddr   127.0.0.1
...

After changing restart the Nginx software.

1. On the main server (without the Nemesida WAF module installed), configure traffic mirroring according to the guidelines of the installed web server (Nginx, Apache2, Microsoft IIS and others).

location / {
    mirror /mirror;
    ...
}

location = /mirror {
    internal;
    proxy_pass http://nemesida_waf_server$request_uri;
}

2. On the server with the installed Nemesida WAF module, bring the configuration file of the virtual host Nginx to the form:

server {
        listen  80;
        index   index.html;
        root    /var/www/html;
        try_files $uri $uri/ /index.html;
}

3. On the server with the Nemesida WAF module installed, create the /var/www/html directory and place an empty index.html file in it.

4. On the server with the Nemesida WAF module installed, bring the /etc/nginx/nwaf/conf/global/nwaf.conf file to the form:

...
nwaf_limit rate=5r/m block_time=0;
...

5. After changing restart the Nginx on every server.

To use a web server with installed Nginx and Nemesida WAF as an intermediate server, you should create corresponding file of the virtual host (for example /etc/nginx/conf.d/example.com.conf).

server {
        server_name  site.example.com;
        listen 80;

        location / {
                proxy_pass http://192.168.0.1;
        }
}

More information is available on the official page of the Nginx documentation.

For Nemesida WAF to work in a cluster, all license keys used must have a single WAF ID (identifier that allows to combine different license keys into a group). To group license keys into a single WAF ID, send a request to support@nemesida-security.com.

Behavioral models of machine training, as well as FP/FN requests exported via the Nemesida WAF Signtest functionality, will be automatically downloaded to all instances of the cluster in accordance with WAF ID. For a more accurate detection of attacks, it is recommended to use one installed instance of the Nemesida AI MLC module at one time.

When using Nemesida WAF as a cluster solution in active/active mode, you must:
1. Organize synchronization of Nemesida WAF configuration files (the /etc/nginx/nwaf/ directory by default, excluding the objects /etc/nginx/nwaf/rules.bin and /etc/nginx/nwaf/ml/) with all installation instances. Synchronization should be done after each modification implementation to the configuration files.

# rsync -az --no-perms --no-owner --no-group --exclude "/ml/" --exclude "/rules.bin" /etc/nginx/nwaf/ root@%remote_server_address%:/etc/nginx/nwaf/ 

2. After synchronizing the objects, it is necessary to update the parameters in the /etc/nginx/nwaf/conf/global/nwaf.conf file on each installed instance of the Nemesida WAF dynamic module:

• nwaf_license_key (necessarily);
• api_proxy and sys_proxy (optionally).

3. After modifications, you must restart the services:

# nginx -t && systemctl reload nginx 
# systemctl restart nwaf_update mla_main 

After installing and configuring the Nemesida WAF modules, restart the operating system and check the operation of all modules:

# systemctl status nginx nwaf_update mla_main mlc_main rabbitmq-server

If modules’ operation is correct there is a inscription: Active: active (running).

When the license key expires, the software works in the Nemesida WAF Free mode.

There is a support of custom set of signatures rules (signature, RL) and signature exception rules (exclusion rule, WL) in Nemesida WAF and Nemesida WAF Free.

Zones and additional conditions

During the creation of RL or WL rules special parameters can be used:

• zones: URL, ARGS, BODY, HEADERS, Cookie, User-Agent, Content-Type, etc.;
• conditions of using the rule (additional condition, specification of the zone): $URL, $ARGS, $BODY, $HEADERS, $Cookie, $User-Agent, etc.

Using zones and additional conditions allows to concretize maximally the creating rule. Additional conditions are set by adding prefix $ (for example: "Z:...|$URL:...").

An example of using the rule with zone and additional condition: WL ... "Z:ARGS|$URL:/123"; – the rule will work, if the entry zone of the RL rule will be ARGS zone and URL will contain an entry /123.

Several parameters (zones, additional conditions) in one rule must be separated with the character |, the following principle of interaction will be used:

• zones interact using the logical principle OR, for example: Z:URL|ARGS;
• zones interact with additional conditions using the logical principle AND, for example: Z:URL|$ARGS:123;
• additional conditions interact using the logical principle AND, for example: Z:...|$ARGS:123|$BODY:123".

Regular expressions in the additional conditions

It’s possible to use regular expressions in the additional conditions. This requires to add to the additional condition postfix _X (for example: "Z:...|$URL_X:\w+").

If | is used as an ordinary character, it is necessary:

• for an ordinary rule: to execute its dual shielding (for example: Z:...|$BODY:block\\|page – search for an entry block or page);
• for the rule with the regular expression (with postfix _X): to execute its quad-shielding (for example: $BODY_X:block\\\\|page – search of an entry block|page).

If in the rule with the regular expression (with postfix _X) other characters from the set of regular expressions operator are used (/, ., *, ?, !, {, }, [, ], (, ), ^, $, :, \, etc.) as ordinary characters it requires to execute their dual shielding (for example: $BODY_X:data=\\(block-\\\a\\\-in-the-main\\) – search for the entry (block-\a\-in-the-main); $HEADERS_X:Accept\\: image\\/webp\\\\\* – search for the entry Accept: image/webp\*).

For use in additional conditions the separator as a metacharacter of a regular expression it is necessary to shield it twice. For example: the rule WL ... "Z:...|$URL_X:/(a\\|b)/"; will bring to URL which contains /a/ or /b/.

User-defined signature creation

The user-defined rules of detecting attack signs (user-defined RL) can be placed in the main configuration file Nemesida WAF (nwaf.conf) or in the self-created file of the form *.conf, located in the /etc/nginx/nwaf/conf/vhosts directory. User-defined signatures must be determined by the RL parameter and must have ID started from 50000 and take the following form:

RL ID:50000 "P:select from" "SC:SQL:12" "Z:ARGS";

RL ID:50001 domain=example.com "P:select from" "SC:SQL:12" "Z:ARGS";

RL ID:50002 domain=.example.com "P:select from" "SC:SQL:12" "Z:ARGS";

RL ID:50003 domain=*.example.com "P:select from" "SC:SQL:12" "Z:ARGS";

RL ID:50004 "PX:select\s+from" "SC:SQL:12" "Z:ARGS|$URL:/admin";

RL ID:50005 "P:select from" "SC:SQL:12" "Z:ARGS|$URL:/(admin\|dev)";

For signatures in the zone it is allowed to use the MLA value to force request sending into Nemesida AI MLA, that falls within the scope of the created signature if the score < 12 (by default requests with a score equal or greater than the value of the mla_score parameter are sent into Nemesida AI MLA).

RL ID:50005 "P:select from" "SC:SQL:1" "Z:MLA";

RL ID:50006 "P:select from" "SC:SQL:1" "Z:ARGS|MLA";

RL ID:50001 "P:select from" "SC:SQL:12" "Z:ARGS|URL|BODY|NoAPI";

RL ID:50002 "P:select from" "SC:SQL:12" "Z:ARGS|URL|BODY|LM";

Creating a signature exclusion rule

In case the inquiry falls under action of a signature, in addition to sending the incident to the Nemesida WAF API, the following line will be displayed in the error log of the Nginx software:

Nemesida WAF: the request ххх contains a rule id 1 in zone HEADERS, ...

or, if the request contains a signature with a maximum allowable digital indicator of significance (score = 12):

Nemesida WAF: the request ххх blocked by rule id 1 in zone HEADERS, ...

where:

• 1 – attack signature ID;
• HEADERS – signature entry zone.

To display absolutely all occurrences of signatures in the request (if there are occurrences), including those occurrences that did not lead to the subsequent blocking of the request, activate the nwaf_log_mr_all; parameter, in the nwaf.conf file Nemesida WAF.

The information about current signature list is on the page rlinfo.nemesida-security.com.

Each request processed by the Nemesida WAF module is assigned a blocking identifier (BT – blocking type). The identifier can be assigned regardless of whether the request was blocked or not.

The pivot table shows the types of identifiers and their corresponding request processing status.

The interface is not used in Nemesida WAF Free.

For the Nemesida WAF Management API to work correctly, you must place all managed Nemesida WAF parameters in the /etc/nginx/nginx/nwaf/conf/global/nwaf.conf file. Nemesida WAF managed parameters cannot be placed in other configuration files.

To activate the Nemesida WAF mgmt_api management interface, add the parameter nwaf_mgmt_api on; to the server section in the virtual host file, for example:

server {
    listen  80;
    server_name example.com;

    location /nwaf_mgmt_api_path {
    nwaf_mgmt_api on;
    }

    ...
}

After activating the parameter nwaf_mgmt_api on;, Nemesida WAF API becomes accessible by the target URI with the nwaf_mgmt_api option set, for example, example.com/nwaf_mgmt_api_path.

It is recommended to restrict access to the Nemesida WAF Management API through the list of allowed IP-addresses to prevent unauthorized users from managing the Nemesida WAF work processes.

{"lic_key_sha1":"SHA1 of the lic. key","get_params":"nwaf_limit"}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","get_params":"nwaf_limit"}'

{"lic_key_sha1":"SHA1 of the lic. key","add_params":"base64(nwaf_limit rate=5r/m block_time=600)"}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","add_params":"base64(nwaf_limit rate=5r/m block_time=600)"}'

{"lic_key_sha1":"SHA1 of the lic. key","update_params":["base64(nwaf_limit rate=5r/m block_time=600)","base64(nwaf_limit rate=5r/m block_time=0)"]}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","update_params":["base64(nwaf_limit rate=5r/m block_time=600)",""base64(nwaf_limit rate=5r/m block_time=0)"]}'

{"lic_key_sha1":"SHA1 of the lic. key","delete_params":"base64(nwaf_limit rate=5r/m block_time=600)"}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","delete_params":"base64(nwaf_limit rate=5r/m block_time=600)"}'

{"lic_key_sha1":"SHA1 of the lic. key","get_banned_ip":""}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","get_banned_ip":""}'

{"lic_key_sha1":"SHA1 of the lic. key","add_banned_ip":"10.0.10.10"}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","add_banned_ip":["10.0.10.10"]}'

{"lic_key_sha1":"SHA1 of the lic. key"," delete_banned_ip":"10.0.10.10"}

{"lic_key_sha1":"SHA1 of the lic. key"," delete_banned_ip":"*"}

curl "example.com/nwaf_mgmt_api_path" --data '{"lic_key_sha1":"SHA1 of the lic. key","delete_banned_ip":"10.0.10.10"}'

To unblock an IP-address, which was identified by module Nemesida AI MLC as the BT 7 (brute force attack), the user will need to confirm that its actions are performed non-automatic. For this option to work, you need a configured and enabled Nemesida WAF Management API and a web application with configured captcha. If the answer is correct, the web application with captcha sends a request delete_banned_ip to the Nemesida WAF module to unblock the IP-address. The user will then be redirected to the original page.

Not used in Nemesida WAF Free.

The Nemesida WAF functionality allows you to automatically synchronize blocked IP-addresses between servers. The IP-address blocking time is determined by the nwaf_limit option in the nwaf.conf configuration file. To activate synchronization of blocked IP-addresses on all servers that use this functionality, you must:

• set the same values for the nwaf_limit and nwaf_sync_ban_ip_key parameters in the nwaf.conf file;
• define the target URL for syncing using the nwaf_sync_ban_ip_host parameter in the nwaf.conf file.
• define a path with the nwaf_ban_ip_sync on; parameter set in the virtual host file in the location section to get a list of blocked addresses, for example:

  location /ban_ip_sync_path
  {
      ...
      nwaf_ban_ip_sync on;
      ...
  }
  

• system logs;
• run-time journal of the Nginx software;
• run-time journal of the RabbitMQ software;
• run-time journal of the Nemesida WAF modules /var/log/nwaf/.

Technical support of Nemesida WAF Free is on the forum only.

In case of unforeseen errors in the operation of the Nemesida WAF software, contact technical support by email (Mon-Fri from 10:00 to 19:00 time, GMT+3) or leave a message on the forum.