Nemesida WAF API | Nemesida WAF

The installation and setup guide of Nemesida WAF API module, which is intended for receiving information about attacks and identified vulnerabilities.

Contents

Nemesida WAF API installation and setup
Nemesida WAF API integration
Nemesida WAF API database structure

The domain name example.com and subdomains in the guide is used as an example.

Nemesida WAF API installation and setup

Nemesida WAF API is intended to transfer information from the Nemesida WAF modules (blocked requests, detected vulnerabilities, operation status of machine learning module) to the PostgreSQL database for subsequent integration with different services, such as Nemesida WAF Cabinet, SIEM class systems, etc.

To install Nemesida WAF API, you must perform the following steps:

1. Allow access to nemesida-security.com:443.
2. Install the module:

DebianUbuntuCentOS

# apt install apt-transport-https

Debian 9

# echo "deb https://nemesida-security.com/repo/nw/debian stretch non-free" > /etc/apt/sources.list.d/NemesidaWAF.list

Debian 10

# echo "deb https://nemesida-security.com/repo/nw/debian buster non-free" > /etc/apt/sources.list.d/NemesidaWAF.list

# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3-pip python3-dev python3-venv nginx postgresql-server-dev-all memcached 
# apt install nwaf-api

Installation of niginx will be done automatically with the creation of a virtual host in the /etc/nginx/conf.d/ directory. Upon completion of the installation, rename the virtual host configuration file nwaf-api.conf.disabledin nwaf-api.conf and restart nginx.

For security it’s recommended to allow requests to Nemesida WAF API from servers with Nemesida WAF, Nemesida AI and Nemesida WAF Scanner only.

Install and configure the PostgreSQL:

# apt install postgresql

Create database, user and password to connect Nemesida WAF API module:

# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf to nw_api;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""

Create database, user and password to connect Nemesida WAF Cabinet module:

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet to nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""

# apt install apt-transport-https

16.04

Connect the repository:

# echo "deb [arch=amd64] https://nemesida-security.com/repo/nw/ubuntu xenial non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add -

Connect Python 3.6 repository:

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa

Install the packages:

# apt update && apt upgrade
# apt install python3.6-dev python3.6-venv nginx postgresql-server-dev-all memcached 
# curl https://bootstrap.pypa.io/get-pip.py | python3.6

18.04

Connect the repository and install the packages:

# echo "deb [arch=amd64] https://nemesida-security.com/repo/nw/ubuntu bionic non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3-pip python3-dev python3-venv nginx postgresql-server-dev-all memcached

20.04

Connect the repository and install the packages:

# echo "deb [arch=amd64] https://nemesida-security.com/repo/nw/ubuntu focal non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3-pip python3-dev python3-venv nginx postgresql-server-dev-all memcached

 # apt install nwaf-api

For security it is recommended to allow requests to Nemesida WAF API from servers with Nemesida WAF, Nemesida AI and Nemesida WAF Scanner only.

Install and configure the PostgreSQL:

16.04

Connect the repository:

# echo "deb http://apt.postgresql.org/pub/repos/apt xenial-pgdg main" > /etc/apt/sources.list.d/pgdg.list
# wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
# apt update && apt upgrade

install the package:

# apt install postgresql

18.04

# apt install postgresql

20.04

# apt install postgresql

Create database, user and password to connect Nemesida WAF API module:

# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf to nw_api;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""

Create database, user and password to connect Nemesida WAF Cabinet module:

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet to nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""

Configure the SELinux policy or deactivate it with the command:

# setenforce 0

then bring the file /etc/selinux/config to the form:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

CentOS 7

Connect additional repositories and install the necessary dependencies:

# rpm -Uvh https://nemesida-security.com/repo/nw/centos/nwaf-release-centos-7-1-6.noarch.rpm
# rpm -Uvh https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# rpm -Uvh https://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
# yum install centos-release-scl-rh
# yum install python36 python36-pip python36-devel nginx postgresql11-devel gcc memcached 
# yum install nwaf-api

For security it is recommended to allow requests to Nemesida WAF API from servers with Nemesida WAF, Nemesida AI and Nemesida WAF Scanner only.

Install and configure the PostgreSQL:

# yum install postgresql11-server
# /usr/pgsql-11/bin/postgresql-11-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/11/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/11/data/pg_hba.conf
# systemctl start postgresql-11.service
# systemctl enable postgresql-11.service

Create database, user and password to connect Nemesida WAF API module:

# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf to nw_api;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""

Create database, user and password to connect Nemesida WAF Cabinet module:

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet to nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""

CentOS 8

Connect additional repositories and install the necessary dependencies:

# rpm -Uvh https://nemesida-security.com/repo/nw/centos/nwaf-release-centos-8-1-6.noarch.rpm
# dnf update
# dnf install python3 nginx python3-pip python3-devel postgresql-devel gcc memcached 
# dnf install nwaf-api

For security it is recommended to allow requests to Nemesida WAF API from servers with Nemesida WAF, Nemesida AI and Nemesida WAF Scanner only.

Install and configure the PostgreSQL:

# dnf install postgresql-server
# postgresql-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/data/pg_hba.conf
# systemctl start postgresql
# systemctl enable postgresql

3. Allow access:
– to the server nemesida-security.com:443;
– to the server Memcached 127.0.0.1:11211;
– to the server with PostgreSQL.

4. Create a database structure:

# cat /var/www/nw-api/api.sql | su postgres -c "psql waf"

Make the necessary changes to the /var/www/nw-api/settings.py file to connect to the PostgreSQL and set the proxy-server address (if used) to connect to nemesida-security.com:443.

5. Restart the server and test the module:

# systemctl status nw-api

For security reasons, it is recommended to allow access to the Nemesida WAF API only from the IP address of the Nemesida WAF, Nemesida AI and Nemesida WAF Scanner servers.

Nemesida WAF API integration

To integrate the Nemesida WAF API with the Nemesida WAF software, follow these steps:

1. On the server with the Nemesida WAF module installed, change the configuration file /etc/nginx/nwaf/conf/global/nwaf.conf, bring the parameters to the form:

nwaf_sys_proxy proxy.example.com:3128;
nwaf_api_proxy proxy.example.com:3128;
nwaf_api_conf host=http://nwaf-api.example.com:8080;

where nwaf-api.example.com:8080 is the address and port of the server where the Nemesida WAF API module is installed, and proxy.example.com:3128 is the proxy server address for accessing Nemesida WAF API and Nemesida WAF Signtest.

2. On the server with the Nemesida AI MLC module installed, change the configuration file /opt/mlc/mlc.conf, bring the parameters to the form:

api_uri = http://nwaf-api.example.com:8080/
api_proxy = proxy.example.com:3128

3. On the server with the Nemesida WAF Scanner module installed, change the configuration file /opt/nws/main.conf, bring the parameters to the form:

api_host = http://nwaf-api.example.com:8080 
api_proxy = proxy.example.com:3128

4. After making changes, you must restart the services or restart the server.

Other information

During the Nemesida WAF API operation the information about errors is contained in the run-time journals of the module /var/log/uwsgi/app/*.log.

Nemesida WAF API database structure

Information about events entering the Nemesida WAF API module is placed in the waf database in the attack, ml and scan_report tables.

attack

The attack table is intended for placement in the DBMS of information on the detected anomalies of the operation of the Nemesida WAF and Nemesida AI modules.

id	Record ID.
timestamp	Date of the anomaly fixation.
vhost	Virtual host’s address
ip	Address of the request source.
rule_id	The identifier of the rule used to fix the anomaly.
method	Type of HTTP request (GET, POST, etc).
url	Request URL.
lm	A parameter that determines whether the request is subject to the LM mode.
bot	Reserved parameter.
mz	The zone of occurrence of the anomaly (URL, ARGS, BODY, etc).
param	Request parameters.
other_headers ua referer cookie	HTTP request headers.
description	Description of the anomaly.
bt	Digital ID of the method for determining anomalies (signature analysis, machine learning, etc).
request_id	Request ID.
waf_id	WAF ID.
body	Content of the request in the Body section.
group_id	Request ID when blocking brute-force and DDoS attacks.
blocked	The blocking status of the request.

Table ml is intended for placing in the DBMS information about the training status of the Nemesida AI module.

Parameter	Description
waf_id	WAF ID.
ml_status	Nemesida WAF module training completion indicator.
ml_learning_progress	The training status of the module Nemesida AI in percent.
vhost	Virtual host.

scan_report

The scan_report table is intended for placing in the DBMS information about the status of the work of the Nemesida WAF Scanner module.

Parameter	Description
id	Record ID.
scan_date	Scan date.
content	Critical level of the detected vulnerability.
domain	Virtual host.
method	Type of HTTP request (GET, POST, etc).
param	Request parameters.
payload	The content of the payload request.
type	The type of vulnerability detected (SQLi, XSS, etc).
url	Request URL.
data	Request body (for POST requests).
waf_id	WAF ID.

The rldscupd service

The rldscupd service is designed to get missing anomaly descriptions from the nemesida-security.com:443 server.