Installation and QuickStart Guide for Nemesida WAF components

The guide for installing Nemesida WAF components. It is recommended to install the components in the appropriate order in the manual.

General information

External resources access

For all Nemesida WAF components to work correctly on each server where they are installed, you must provide access to external resources:

https://nemesida-security.com;
https://nw-auth-extra.nemesida-security.com;
http(s)://geoip.nemesida-security.com.

Hardware Requirements

For the effective operation of Nemesida WAF components, it is recommended to use servers with the following technical characteristics*:

Server for filtering node analyzes and redirects unblocked requests to a server with a web application
CPU	4 cores x 2.4 GHz
RAM	6 GB
Disk space	10 GB

Server for Nemesida AI MLC is used to build behavioral models and analyze all incoming requests with their help, detects brute force attacks, flood and DDoS attacks at the application level and parasitic bots activity
Plans	Light	Business, Enterprise
CPU	6 cores x 2.4 GHz
RAM	4 GB	24 GB
Disk space	25 GB

Server for Nemesida WAF API, Nemesida WAF Cabinet and PostgreSQL DBMS is used to store and visualize identified anomalies and shortcomings of the web application, as well as to control the behavior of the machine learning module
CPU	4 cores x 2.4 GHz
RAM	6 GB
Disk space	25 GB

^* The stated technical requirements are approximate (for loads up to 10k RPS) and are selected individually, depending on the amount and type of incoming traffic.

Failover of the Nemesida WAF components

To ensure the failover operation of the Nemesida WAF complex, we recommend that you familiarize yourself with the scheme that reflects the minimum number of servers used by the components:

The scheme assumes the provision of failover operation for Nemesida WAF components with different levels of criticality:

Cluster of filtering nodes analyzes and redirects unblocked requests to a server with a web application
Criticality	High
Minimum number of servers	2
Description	If the component is unavailable: no request analysis will be performed; the services to which traffic is proxied through the filtering node will be unavailable.

Server for Nemesida AI MLC is used to build behavioral models and analyze all incoming requests with their help, detects brute force attacks, flood and DDoS attacks at the application level and parasitic bots activity
Criticality	Average
Minimum number of servers	1
Description	If the component is unavailable: DDoS/Brute/Flood attacks and parasitic bots activity will not be detected; the process of learning/retraining the behavioral model will be suspended. The unavailability of the component does not affect the analysis of requests by the machine learning module on the filtering node, if the behavioral model has already been created and applied before; the request analysis process for generating the OpenAPI specification will be suspended.

Server for Nemesida WAF API and PostgreSQL DBMS is used to store identified anomalies and shortcomings of the web application, as well as to control the behavior of the machine learning module and the filtering node
Criticality	High
Minimum number of servers	2*
Description	If the component is unavailable: settings will not be received by other components, as well as their configuration; the process of receiving and recording detected anomalies will be suspended.

^* – using a single copy of the Nemesida WAF API in mode Read only will allow components to get a list of settings for them (filtering node settings, information about behavioral models, exclusion/blocking rules, etc.), but will not allow new data to be written to the database.

Server for the Nemesida WAF Cabinet is used to visualize detected anomalies
Criticality	Low
Minimum number of servers	1
Description	If the component is unavailable: access to the Nemesida WAF Cabinet interface will be suspended.

Server for Nemesida WAF Scanner is used to identify the shortcomings of the web application
Criticality	Low
Minimum number of servers	1
Description	If the component is unavailable: it will be impossible to scan the protected web application to identify vulnerabilities; it will be impossible to use the `Recheck` functionality;

PostgreSQL DBMS

Before you start installing the Nemesida WAF components, you must prepare the PostgreSQL DBMS to work with the components:

Debian, UbuntuRHELAutomatic installation scriptDocker

# apt install postgresql

After installing the DBMS:

Create a database for Nemesida WAF API components and a Nemesida WAF Cabinet:

Nemesida WAF API

# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf TO nw_api;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""
# su - postgres -c "psql waf -c \"GRANT ALL ON ALL TABLES IN SCHEMA public TO nw_api;\""
# su - postgres -c "psql waf -c \"GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO nw_api;\""
# su - postgres -c "psql waf -c \"GRANT CREATE ON SCHEMA public TO nw_api;\""

YOUR_PASSWORD – an example of a password is not recommended for use.

Nemesida WAF Cabinet

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet TO nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""
# su - postgres -c "psql cabinet -c \"GRANT ALL ON ALL TABLES IN SCHEMA public TO nw_cabinet;\""
# su - postgres -c "psql cabinet -c \"GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO nw_cabinet;\""
# su - postgres -c "psql cabinet -c \"GRANT CREATE ON SCHEMA public TO nw_cabinet;\""

YOUR_PASSWORD – an example of a password is not recommended for use.

Grant access to external component connections by making changes to the configuration file pg_hba.conf:

Example:
```
# IPv4 local connections:
host    all             all             10.1.1.0/24            md5
```

Configure the SELinux policy or deactivate it with the command:

# setenforce 0

then bring the /etc/selinux/config file to the form:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Install and configure the PostgreSQL DBMS:

# dnf update
# dnf install postgresql-devel postgresql-server
# postgresql-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/data/pg_hba.conf
# systemctl start postgresql
# systemctl enable postgresql

After installing the DBMS:

Create a database for Nemesida WAF API components and a Nemesida WAF Cabinet:

Nemesida WAF API

# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf TO nw_api;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""
# su - postgres -c "psql waf -c \"GRANT ALL ON ALL TABLES IN SCHEMA public TO nw_api;\""
# su - postgres -c "psql waf -c \"GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO nw_api;\""
# su - postgres -c "psql waf -c \"GRANT CREATE ON SCHEMA public TO nw_api;\""

YOUR_PASSWORD – an example of a password is not recommended for use.

Nemesida WAF Cabinet

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet TO nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""
# su - postgres -c "psql cabinet -c \"GRANT ALL ON ALL TABLES IN SCHEMA public TO nw_cabinet;\""
# su - postgres -c "psql cabinet -c \"GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO nw_cabinet;\""
# su - postgres -c "psql cabinet -c \"GRANT CREATE ON SCHEMA public TO nw_cabinet;\""

YOUR_PASSWORD – an example of a password is not recommended for use.

Grant access to external component connections by making changes to the configuration file pg_hba.conf:

Example:
```
# IPv4 local connections:
host    all             all             10.1.1.1/32            md5
```

To deploy a PostgreSQL DBMS, you can use a script for automatic installation and initialization of the database:

1. Upload script.

2. Run the installation script with the command:

/bin/bash ./1-postgresql-deploy.sh 'pg_api_pwd=xxx' 'pg_cabinet_pwd=xxx' 'api_srv_ip=x.x.x.x'

where:

pg_api_pwd – password for creating a user nw_api for the database waf;
pg_cabinet_pwd – password for creating a user nw_cabinet for the database cabinet (required for the operation of the Nemesida WAF Cabinet component);
api_srv_ip – the IP address of the server from which component will be accessed after its configuration.

Information about using PostgreSQL in a Docker container is available in the corresponding section.

Nemesida WAF API

The Nemesida WAF API is designed for components to interact with each other, as well as receive information about incidents and identified vulnerabilities for transmission to the database. To install the component:

Provide access to external resources for the component to work correctly;
Install the component by following the installation guide.

After the configuration is complete, restart the services:

# systemctl restart nw-api rldscupd nginx memcached
# systemctl status nw-api rldscupd nginx memcached

Nemesida WAF Cabinet

The component is designed to visualize and systematize information about attacks and identified vulnerabilities, as well as manage the settings of Nemesida WAF and Nginx web server. To install the component:

Provide access to external resources for the component to work correctly;
Install the component by following the installation guide;
Perform integration of the component with the Nemesida WAF API.

After the configuration is complete, restart the services:

# systemctl restart nginx cabinet cabinet_ipinfo cabinet_attack_notification cabinet_cleaning_db cabinet_rule_update memcached
# systemctl status nginx cabinet cabinet_ipinfo cabinet_attack_notification cabinet_cleaning_db cabinet_rule_update memcached

Filtering node

The filtering node is designed to analyze requests and decide whether to block them in case signs of attacks or other anomalies are detected. To install the component:

Provide access to external resources for the component to work correctly;
Install the component by following the installation guide;
Perform integration of the component with the Nemesida WAF API.

After the configuration is complete, restart the services:

# systemctl restart nginx rabbitmq-server memcached nwaf_update mla_main api_firewall
# systemctl status nginx rabbitmq-server memcached nwaf_update mla_main api_firewall

Nemesida AI MLC

The machine learning module consists of the Nemesida AI MLA machine learning agent (included in the nwaf-dyn package and, as a rule, does not require configuration) and the Nemesida AI MLC machine learning module. To install the component:

Provide access to external resources for the component to work correctly;
Install the component by following the installation guide;
Perform integration of the component with the filtering node;
Perform integration of the component with the Nemesida WAF API.

After the configuration is complete, restart the services:

# systemctl restart mlc_main rabbitmq-server memcached
# systemctl status mlc_main rabbitmq-server memcached

Checking the configuration of Nemesida WAF

Before activating web application protection, it is necessary to make sure that all components of Nemesida WAF are active and interact correctly with each other. To do this, follow these steps:

1. Check the logs of each component for possible errors:

Nemesida WAF API:

/var/log/uwsgi/nw-api/*.log

Nemesida WAF Cabinet:

/var/log/uwsgi/cabinet/*.log

Nemesida AI MLC:

/var/log/nwaf/mlc.log

Filtering node:

/var/log/nginx/error.log
/var/log/nwaf/nwaf_update.log
/var/log/nwaf/mla.log
/var/log/nwaf/naf/error.log
/var/log/rabbitmq/rabbit@%hostname%.log
/var/log/rabbitmq/rabbitmq-server.error.log

Nemesida WAF Scanner:

/var/lognwaf/nws.log

2. Send a test request to the server of the filtering node with the test signature nwaftest:

# curl -i http://WAF_SERVER/nwaftest

and make sure that the server of the filtering node returns the response code 403, and in the log /var/log/nginx/error.log there is an entry about the lock:

Nemesida WAF: the request ... blocked by rule ID 1 in zone URL, ...

If the request is not blocked, then follow the steps from the corresponding section actions.

3. Go to Nemesida WAF Cabinet and make sure that an entry about the blocked request appears on the page.

If the request is blocked, but is not displayed on page a, then follow the steps from the corresponding section actions.

Activating Web Application protection

After completing the basic configuration of the filtering node and testing its operation, you can proceed to activate the protection of the web application. As a rule, the web application server and the filtering node are different servers, so incoming requests from clients will not automatically be sent to the filtering node server for analysis. To activate the protection of a web application, you need to configure the filtering node as an intermediate server that will receive and analyze client requests, and then block/redirect them to the server of the protected web application. One of the methods of configuring the interaction between the filtering node and the protected web application is a “reverse proxy”. To do this, follow these steps:

1. Activate the parameters of the monitoring mode (passive mode) of the filtering node in Nemesida WAF Cabinet:

Activating the request analysis monitoring mode for an IP address:
Activating the request analysis monitoring mode for the virtual host:

The monitoring mode (passive mode) is designed to prevent blocking requests for the setup period. Activation of the monitoring mode for an IP address is used in cases where it is necessary to exclude blocking when accessing from a certain list of IP addresses (as a rule, from IP addresses belonging to the IT department that configure components), and activation of the monitoring mode for a virtual host eliminates blocking requests for all clients of the web application.

2. Configure the filtering node as a reverse proxy using the appropriate section guide;

3. Redefine the correspondence of the IP address and the domain name so that requests arrive at the filtering node (for example, by editing the DNS A record).

After completing the configuration of the components, the monitoring mode activation parameters must be deactivated and make sure that the protection is active by sending a test request to the filtering node:
# curl -i http://YOUR_SERVER/nwaftest

Activation of Nemesida AI

Behavioral models

For the machine learning module to work correctly, we recommend creating a behavioral model for each specific web application.

It is recommended to test the quality of the machine learning module after completing the training of the behavioral model and activating the automatic blocking of IP addresses of sources of illegitimate requests.

To activate the Nemesida AI machine learning module, follow these steps in Nemesida WAF Cabinet:

Create a list of virtual hosts for which a behavioral model will be created and applied:

In case of problems related to processing requests by the machine module, follow the steps in the corresponding section of the manual:

Activation of detection of DDoS/Brute/Flood attacks

To activate the mechanism for detecting brute force/flood/DDoS attacks by the Nemesida AI machine learning module, follow these steps in Nemesida WAF Cabinet:

Activate the parameters for detecting DDoS attacks:
Activate the parameters for detecting attacks by brute force/flood: