monutil/README.md

# monutil

Simple Python utility for monitoring CPU and RAM usage, and monitoring the speed when downloading URLs.
It is also used to keep a log of blocked IP addresses fed from a firewall / IDS system (for example fail2ban).

Both CPU/RAM monitoring and URL monitoring can be set on their own monitoring periods, or they can be disabled independently.

The data can be written into an MSSQL, or Maria DB database. Alternatively, the data can be pushed to a RabbitMQ queue, and then optionally retrieved from the queue using the provided consumer script (rabbitmq-consumer.py) - typically your consumer script would run on an alternative host that has access to a database server.

Dependencies can be found within requirements.txt and are installed automatically when executing the main.py script.

To publish to RabbitMQ, loggingMode should be set to 'rabbitmq', and rabbitmqca, rabbitmqcacert, rabbitmqcakey, and rabbitmqRoutingKey should be provided. The certificates should align with the certificate authority which the RabbitMQ server recognises.

To publish direct into an MSSQL or MariaDB database, the loggingMode should be set accordingly (see below), and sqlUsername, sqlPassword, sqlServer, and sqlDatabase should be provided.
You will need the associated ODBC driver installed for whichever DBMS you choose. ODBC references can be modified in odbcReferences.py to suit your system. You can probably even get alternative DBMS's working such as PostgreSQL as long as you convert the schema.

See the available configuration options, which should be specified in config.py.

**Configuration options:**

- **hostMonitoringPeriod** - the delay in between the CPU and RAM usage being probed and being input into the database (defined in seconds). 0 to disable.
- **urlMonitoringPeriod** - the delay in between monitoring all of the URLs and being input into the database (defined in seconds). 0 to disable.
- **urls** - the list of URLs to monitor (e.g. ["url1", "url2"]).
- **urlTimeout** - the delay in seconds before considering a URL to have timed out.
- **maxWorkers** - the amount of threads to use when pulling URL resources. Do not set above the maximum number of threads on the host or bad things may happen.
- **forceNonPOSIXCPU** - For POSIX compatible systems, psutil.getloadavg() is executed which relies on os.getloadavg(), similar to the uptime command. For Windows, this seemingly returns 0 (at least on the version executed during development). For Windows, a custom function has been built to obtain running CPU averages, but you can choose to use this function on POSIX systems by setting this variable to True.
- **loggingMode** - Valid options: mssql, mariadb, postgresql, rabbitmq, none.
- **sqlServer** - the address of the SQL server which to write the data.
- **sqlDatabase** - the database to write the data.
- **sqlUsername** - the username used to authenticate to the SQL server.
- **sqlPassword** - the password used to authenticate to the SQL server.
- **rabbitmqHost** - the hostname of the RabbitMQ instance if using RabbitMQ.
- **rabbitmqPort** - the port of the RabbitMQ instance if using RabbitMQ if using RabbitMQ.
- **rabbitmqca** - the ca file to use when publishing messages to RabbitMQ if using RabbitMQ.
- **rabbitmqcacert** - the certificate file to use when publishing messages if using RabbitMQ.
- **rabbitmqcakey** - the key file to use when publishing messages if using RabbitMQ.
- **rabbitmqRoutingKey** - the routing key to use when publishing messages if using RabbitMQ.
- **logRetentionDaysHost** - the maximum age CPU/Memory logs should be kept.
- **logRetentionDaysURL** - the maximum age URL logs should be kept.
- **logRetentionDaysIPBlock** - the maximum age IP Block logs should be kept.
- **maximumSQLAttempts** - the maximum number of attempts to try certain SQL operations.
- **hostMonitorStartTime** - the start time which the host monitor should start at the earliest (HH:MM:SS).
- **hostMonitorEndTime** - the end time which the host monitor should shut down. This does not stop the script itself and this monitor will restart at the start time (HH:MM:SS).
- **urlMonitorStartTime** - the start time which the url monitor should start at the earliest (HH:MM:SS).
- **urlMonitorEndTime** - the end time which the url monitor would shut down. This does not stop the script itself and this monitor will restart at the start time (HH:MM:SS).
- **usePip** - whether or not to use pip to install dependencies. Set to false if your environment is externally managed.
- **ipinfoAPIToken** - The ipinfo.io API token used for IP address lookups during IP blocks.