A guide for installing, initial configuration and operating the Nemesida AI MLC machine learning module, designed for building behavioral models, detecting brute-force attacks, flood and DDoS attacks, as well as generating query schemes based on the OpenAPI specification.
Not used in Nemesida WAF Community Edition.
Before installing Nemesida WAF components, add repository information to the system:
# apt install apt-transport-https gnupg2 curl
# wget -O- https://nemesida-security.com/repo/nw/gpg.key | apt-key add - # apt update && apt upgrade
# apt install apt-transport-https gnupg2 curl
The Nemesida AI module consists of Nemesida AI MLA modules (is included in the installation package of the Nemesida WAF module) and Nemesida AI MLC, whose interaction is possible in normal mode (modules operate on the same server) and mode “dot-multipoint” (the Nemesida AI MLC module operates on a dedicated server).
Python pip packages
For machine learning modules to work correctly, it is necessary to use unified versions of Python3 pip packages on servers with Nemesida AI MLA and Nemesida AI MLC installed.During analyse requests against their schema matching in
OpenAPI
format, all integer data formats are treated as typeinteger
.
Installation
Install Nemesida AI MLC:
# apt install nwaf-mlc
During the installation of the module, the following PIP packages are additionally installed:
code>wheel cython pandas simple-crypt pika logutils scikit-learn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize python-geoip-python3 python-geoip-geolite2 netaddr pymemcache genson
Install Nemesida AI MLC:
# apt install nwaf-mlc
During the installation of the module, the following PIP packages are additionally installed:
code>wheel cython pandas simple-crypt pika logutils scikit-learn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize python-geoip-python3 python-geoip-geolite2 netaddr pymemcache genson
During the installation of the module, the following PIP packages are additionally installed:
code>wheel cython pandas simple-crypt pika logutils scikit-learn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein unidecode fsspec func_timeout url-normalize python-geoip-python3 python-geoip-geolite2 netaddr pymemcache genson
Initial setup
After installing the module, it is necessary to make the initial configuration. The configuration file /opt/mlc/mc.conf
contains a list of available parameters of the Nemesida AI MLC module, which cannot be configured using Nemesida WAF Cabinet or API.
After making changes, restart the server or restart the service and check its operation:
# service mlc_main restart # service mlc_main status
After the initial configuration of the module, you must check for errors in the component event logs /var/log/nwaf/mlc.log
.
Working in Multipoint Mode
To build behavioral models, the Nemesida AI MLC module requires a significant amount of free RAM. When using more than one server with the Nemesida WAF module, you can save hardware resources by using the point-to-multipoint operation scheme (one server with the Nemesida AI MLC module installed interacts with many servers with Nemesida WAF modules installed).
On a server with the Nemesida WAF module installed
Create a user of the RabbitMQ service:
# rabbitmqctl add_user USER PASSWORD # rabbitmqctl set_permissions -p / USER ".*" ".*" ".*"
or
# rabbitmqadmin declare user name=USER password=PASSWORD tags=user # rabbitmqadmin declare permission vhost=/ user=USER configure=".*" write=".*" read=".*"
where USER
and PASSWORD
are the username and password for connecting the Nemesida AI MLC module.
Make changes to the configuration file /etc/rabbitmq/rabbitmq-env.conf
:
NODE_PORT=5672 export RABBITMQ_NODENAME=rabbit@localhost export RABBITMQ_NODE_IP_ADDRESS=0.0.0.0 export ERL_EPMD_ADDRESS=127.0.0.1
Allow access from the server on which the Nemesida AI MLC module is installed to the RabbitMQ port (by default 5672 TCP). and complete the RabbitMQ setup:
# service rabbitmq-server restart
On a server with the Nemesida AI MLC module installed
Create additional configuration files in the /opt/mlc/conf/
directory by copying the /opt/mlc/mlc.conf
file.
For example:
/opt/mlc/conf/server_1.conf /opt/mlc/conf/server_2.conf ...
Make changes to the new configuration files to work with the remote RabbitMQ server. After making the changes, restart the service:
# service mlc_main restart # service mlc_main status
In additional configuration files
nwaf_license_key
is a required parameter. The license key used in the Nemesida AI MLC settings and the remote Nemesida WAFs must have the sameWAF ID
. When using additional configuration files, it is recommended to delete the/opt/mlc/mlc.conf
file.
Using remote RabbitMQ services, the Nemesida AI MLC module will collect queries and then train models in the same way as in normal operation.
After configuration, you must check the error information contained in the event logs for each server /var/log/nwaf/server_*/mlc.log
.