We noticed a bug in Statusengine 2.1.4, that downtimes will disappear after a restart of the monitoring engine.
Today we release packages for Statusengine 2.1.5, to resolve the issue.
After creating a new downtime for a host or a service, the openITCOCKPIT interface will display the downtime.
The issue occurred, as soon as the monitoring engine was restarted, for example by "Refresh monitoring configuration". The downtime was still present and active in the monitoring engine. Just the database had a lack of information:
We also removed the Push bulk 200 bulk inserts for Servicestatus
messages from /var/log/syslog
.
In one of our previous articles we described how to update an installation of openITCOCKPIT in detail. If you are already familiar with our update process, you can update to the new version in three easy steps:
tmux
sudo apt-get update
sudo apt-get dist-upgrade
Today we proudly released the next version of openITCOCKPIT 3.4.2.
Breaking changes to our API interface!
If you are using an ((OTRS)) Community Edition integration, our PagerDuty/Slack Module, End2End scripts or the External Command HTTP-Interface manual action is required!
Read the "API Keys" section of this article, to get more information.
We redesigned our status map, which will now display hosts of a selected openITCOCKPIT instance. We also added filter options for host name and host address.
You can use your scroll wheel to zoom in or out. By clicking on an icon, you get detailed status information about the selected object. The Statusmap will only show hosts that are in a parent child relation. If you untick the box "Consider parent child relations", the Statusmap will display all hosts.
Depending on your system size, it could take a while to load all objects. We added a progress bar, to let you know the current status of the loading and rendering process.
Breaking changes!
If you are using an ((OTRS)) Community Edition integration, our PagerDuty/Slack Module, End2End scripts or the External Command HTTP-Interface manual action is required!
Our interface uses the API more frequently with every single version. What about you? To make it easier to use our API, we implemented API Keys. So your scripts can simply add an HTTP-Header to the HTTP-Request, instead of handle nasty cookies.
Every API key is bound to a single user. So the user permissions are the same.
You can create as much API keys as you need. We recommend to create a single API key for every external application.
First of all click on your user name in the top left corner to edit your profile.
At the bottom of the page, you can find the new "API Keys" section. Click on "Create new API key"
You can copy the curl
examples, to play around with the API. The examples will only work after you pressed on Save
!
To use an API key, you need to add the header Authorization
with the Value X-OITC-API $APIKEY$
to your HTTP-Request. You can look at the auto generated curl
examples.
You can also use the API Keys for PCs in kiosk-mode or TV screens. There are Browser Plugins available, which let you send a custom header.
It is also possible to submit the API key as a GET parameter.
https://example-system.oitc.itn/hosts/index.json?angular=true&apikey=2023a6c93a93d1a9fa168e57b4dc0f4f70fc1591b4fea8ce097cbc50d8673150c1dc128947c720c9c4de5c0cd22070c7cd9d0122a77d4bc07e208c797572411b3d966104f887ce690f0e84f3942f46fa
Whenever an external script or application needs to send commands to the monitoring engine, like acknowledgements, check results and so on, you need to use our External Command HTTP Interface.
We needed to change the URL structure for the new API keys. So you need to update your external scripts. Otherwise sending external commands will not work anymore.
To make your scripts work again remove the old parameter api_key:API_KEY
from the URL and add .json?apikey=YOUR_API_KEY
to the end of the URL.
New External Command Interface URL:
Make sure, that the corresponding user has permission to submit external commands:
Old External Command Interface URL (for comparison):
To use the Two-Way Integration of ((OTRS)) Community Edition, you need to update the Nagios::Acknowledge::HTTP::URL
in the ((OTRS)) Community Edition SysConfig.
Remove the old parameter api_key:API_KEY
from the URL and add .json?apikey=YOUR_API_KEY
to the end of the URL.
New Nagios Acknowledge URL:
Make sure, that the corresponding user has permission to submit external commands:
Old Nagios Acknowledge URL (for comparison):
Example:
To use the Two-Way Integration of PagerDuty, you need to update the Webhook URL.
The openITCOCKPIT interface will generate the new URL for you. Add your API Key to the URL and set the new URL as "Generic Webhook" in your PagerDuty settings.
New Webhook URL:
Example:
https://monitoring.example.org/pagerduty_module/acknowledge/submit.json?apikey=2023a6c93a93d1a9fa168e57b4dc0f4f70fc1591b4fea8ce097cbc50d8673150c1dc128947c720c9c4de5c0cd22070c7cd9d0122a77d4bc07e208c797572411b3d966104f887ce690f0e84f3942f46fa
Old Webhook URL (for comparison):
To use the Two-Way Integration of Slack, you need to update the Webhook URL.
The openITCOCKPIT interface will generate the new URL for you. Add your API Key to the URL and set the new URL as Webhook in your Slack application settings.
New Webhook URL:
Example:
https://monitoring.example.org/slack_module/acknowledge/submit.json?apikey=2023a6c93a93d1a9fa168e57b4dc0f4f70fc1591b4fea8ce097cbc50d8673150c1dc128947c720c9c4de5c0cd22070c7cd9d0122a77d4bc07e208c797572411b3d966104f887ce690f0e84f3942f46fa
Notice: You can't use self-signed certificate anymore, as descripted in the Slack FAQ.
Old Webhook URL (for comparison):
End2End scripts should use our External Command Interface to submit host and service check results.
Search in your code the function, that submit the check result and add .json?apikey=YOUR_API_KEY
to the end of the URL.
You can also remove the old parameter api_key:API_KEY
from the URL.
Example with GET parameter:
$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
$oHTTP.Option(4) = 13056
$test = $oHTTP.Open("HEAD","https://monitoring.example.org/nagios_module/cmd/submit/command:PROCESS_SERVICE_CHECK_RESULT/hostUuid:" & $HostUUID & "/serviceUuid:" & $ServiceUUID & "/status_code:" & $status & "/plugin_output:" & $output & "/.json?apikey=2023a6c93a93d1a9fa168e57b4dc0f4f70fc1591b4fea8ce097cbc50d8673150c1dc128947c720c9c4de5c0cd22070c7cd9d0122a77d4bc07e208c797572411b3d966104f887ce690f0e84f3942f46fa")
$oHTTP.Send()
Example with HTTP-Header:
$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
$oHTTP.Open("GET", "https://monitoring.example.org/command:PROCESS_SERVICE_CHECK_RESULT/hostUuid:" & $HostUUID & "/serviceUuid:" & $ServiceUUID & "/status_code:" & $status & "/plugin_output:" & $output & "/")
;SSL Options for self signed
$oHttp.Option(4) = 0x0100 + 0x0200 + 0x1000 + 0x2000
$oHTTP.SetRequestHeader("Content-Type", "application/json")
$oHTTP.SetRequestHeader("Authorization", "X-OITC-API 2023a6c93a93d1a9fa168e57b4dc0f4f70fc1591b4fea8ce097cbc50d8673150c1dc128947c720c9c4de5c0cd22070c7cd9d0122a77d4bc07e208c797572411b3d966104f887ce690f0e84f3942f46fa")
$oHTTP.Send()
Make sure, that the corresponding user has permission to submit external commands:
We removed pagination for most of the pages, to reduce loading times of openITCOCKPIT. On most pages you will not notice a difference, but page loading speed will be much faster in "scroll mode".
You have still the options for next page, previous page and first page. Last page is not available in scroll mode.
If required, you can switch back to the old "Paginator Mode".
For a better usability and to improve performance, we refactored our Host Groups Extended View with AngularJS.
For a better usability and to improve performance, we refactored our Auto Maps with AngularJS.
We created a new folder, which openITCOCKPIT will use to execute Checkmk checks.
In older versions, openITCOCKPIT uses the Checkmk directory /opt/openitc/nagios/3rd/check_mk/var/precompiled
to execute Checkmk checks. Since openITCOCKPIT 3.4.2, we use
/opt/openitc/nagios/3rd/check_mk/var/oitc_precompiled
. This will prevent false positive check results while the system is generating new Checkmk configuration files.
Info!
If you are updating your system, you need to adjust the check command check_mk_active
manually!
You can still use Checkmk, even without adjusting your command. You will just not benefit from the new configuration generation process.
Change the path to /opt/openitc/nagios/3rd/check_mk/var/oitc_precompiled
:
We also improved the performance of generating all Checkmk files during an "Refresh of monitoring configuration" in combination with openITCOCKPIT Satellite Systems.
New theme openITCOCKPIT Elegance
was added to our DesignModule.
If you are using distributed monitoring, you should also update all your Satellite Systems. We release a new version of phpNSTA
and phpNSTAClient
to resolve a timeout issue
with bulk messages.
With this version of openITCOCKPIT, passive_host_checks_are_soft
will be enabled by default in /etc/openitcockpit/nagios.cfg
.
We are currently packaging openITCOCKPIT for Ubuntu 18.04 LTS (Bionic Beaver). With the new Ubuntu Version, PHP 7.2 is the default PHP version. We already updated openITCOCKPIT, to be compatible with PHP 7.2.
As a side effect, we implemented a new LDAP library, to handle large LDAP environments. To use the new LDAP library, PHP greater or equal 7.1 is required. If your system fulfills the requirements, your openITCOCKPIT will automatically switch to the new LDAP backend. Otherwise openITCOCKPIT will use the old library.
openITCOCKPIT makes use of Statusengine since quite a while. We use Statusengine to store the results from the monitoring engine to a database and to process performance data.
To make openITCOCKPIT more flexible and scalable, we started to implement Statusengine 3.
The new Statusengine 3 database schema will become our new default schema for monitoring records. With this version, we started to prepare our backend for the new version of Statusengine.
Unfortunately we are struggling around with some issues, which delay the intragration of CrateDB.
If you are working remotely with some remote desktop tools like VNC, we guess you are happy to read, that you can disable the animations on the login screen now.
By adding ?remote=1
to the login url, you can disable the login animation temporarily. Full URL example: https://example-system.oitc.itn/login/login?remote=1
If required, you can permanently disable the login animation in the Systemsettings of openITCOCKPIT.
In one of our previous articles we described how to update an installation of openITCOCKPIT in detail. If you are already familiar with our update process, you can update to the new version in three easy steps:
tmux
sudo apt-get update
sudo apt-get dist-upgrade
Today we proudly released the next version of openITCOCKPIT 3.3.1.
We redesigned the status detail view for host and services. The new design will also improve your workflow, for example it is now possible to delete a scheduled downtime directly from the status details overview.
In addition we improved the representation of parent host outages, or if the associated host is in a none Up state.
With the new status details view, we also refactored the user permissions for external commands
like "Reset check time", "Schedule maintenance" and so on.
To enable an user to submit external commands, two conditions are required:
externalcommands
You can set the action permission externalcommands
at Administration Manage user roles.
We decided not to grant new privileges automatically, to avoid that users see objects or are allowed to actions they should not be.
With the new version, we release a complete rewrite of the downtime system in openITCOCKPIT. First of all, we separated "normally scheduled downtimes" from "recurring downtimes" in the menu.
We also added better filter options and the option to delete multiple downtimes at once.
We added a new overview for service group, which contain the host and service state.
If you create a new host or service or edit existing hosts and services, the interface will now always try to redirect you back to the previous page you were visiting.
We hope this will speed up your workflow.
We improved our Slack Notification Plugin. We added a two way implementation, so you can now acknowledge the state directly from Slack.
Also we improved the representation of the plugin output.
If you like to send "Push Notifications" to your smartphone, we added a new Notification Plugin for Pushover.
The required commands host-notify-by-pushover
and service-notify-by-pushover
will be created with every new installation of openITCOCKPIT automatically.
In your contact configuration, you need to create the custom macros with the name PUSHOVERUSER
and PUSHOVERAPP
. Set as value your user credentials,
you have got from the Pushover web site.
If you want to use the Pushover notification behind an HTTP/HTTPS proxy, please make sure to set the address and port of your HTTP/HTTPS proxy.
In one of our previous articles we described how to update an installation of openITCOCKPIT in detail. If you are already familiar with our update process, you can update to the new version in three easy steps:
tmux
sudo apt-get update
sudo apt-get dist-upgrade
Are you using Nagios or Naemon and want try openITCOCKPIT? No problem! We created a PHP script, which tries to import your existing Nagios or Naemon
configuration to a fresh instalation of openITCOCKPIT.
Once in a week our developers can choose a random project, they like to work on. We call this Happy Friday.
Unfortunately we don't have the time to finish every project by 100 percent.
Whenever we created something that could be useful, we publish it to our
GitHub account.
So maybe we missed a feature, or something is not working as expected but it is still worth to give it a shot.
This is basically the reason why we call it beta, we don't want to disappoint you 😉.
Also the migration setup is a bit complex at the moment and it would help, if you are an experienced linux user.
The project was successfully tested with different Nagios and Naemon configurations.
By writing your configurations files by hand, you have a lot of freedoms.
Unfortunately it is very hard to automatically import complex configuration files.
Instead of implementing every possible scenario, we decided to implement a default behavior which should fit 90% of the configurations out there.
define timeperiod{
timeperiod_name 24x7
alias 24x7
monday 00:00-24:00
tuesday 00:00-24:00
wednesday 00:00-24:00
thursday 00:00-24:00
friday 00:00-24:00
saturday 00:00-24:00
sunday 00:00-24:00
}
So maybe your definition of some complex time periods is broken after the import. If you use complex time periods,
you should check this after the import process is done.
openITCOCKPIT requires an host or service template for every host or service, that should be created. You can read more about
host and service templates in our Beginners guide. Due to the fact, that the objects which
are defined with register 0
are missing in the database, our import script needs to "guess" them.
Every host that will be created by the import script, simply uses the default host
template with the id 1
.
This host template exists by default on every openITCOCKPIT system.
The import script will create an service template of every check command
, which is defined in your configuration files.
The service template name, will be the same as the command name, converted to uppercase. So the check command check_ping
will become
the service template CHECK_PING
.
To keep the templates as generic as possible, the importer will set the following fields check_interval
, retry_interval
, notification_interval
, active_checks_enabled
and process_perf_data
to the most used value.
For example, you have ten Services using the check command check_ping with an check_interval of 1
and two services with an check_interval of 15
. Than the service template will use the value 1
as default value for check_interval
.
Please notice: The custom value for check_interval is not lost! The service will just overwrite the service template value with 15
.
The migration uses Statusengine as database backend, which does not support the
dumping host and service escalation configuration to the database anymore.
For this reason, you need to recreate your host and services escalations after the migration process.
Same as for Host- and Service escalations. You need to recreate your host and services depandencies after the migration process.
First of all, you need to install an instance of openITCOCKPIT. Please visit the download section for more information.
root
!
We will just use this machine to dump your monitoring configuration to a MySQL database. It is not important that the checks really work!
It does not matter if your current monitoring configuration is running on Nagios 3.x, Nagios 4.x or Naemon.
cd /tmp
wget https://labs.consol.de/naemon/release/v1.0.6/ubuntu16.04/amd64/naemon-core_1.0.6_ubuntu16.04_amd64.deb
wget https://labs.consol.de/naemon/release/v1.0.6/ubuntu16.04/amd64/libnaemon_1.0.6_ubuntu16.04_amd64.deb
dpkg -i libnaemon_1.0.6_ubuntu16.04_amd64.deb naemon-core_1.0.6_ubuntu16.04_amd64.deb
apt-get install -f -y
systemctl stop naemon
systemctl stop postfix
Statusengine (2.x) will dump all Naemon configuration items to a MySQL database.
apt-get install mysql-server
cd /tmp
wget https://github.com/nook24/statusengine/archive/2.1.3-1.tar.gz
tar xfv 2.1.3-1.tar.gz
cd statusengine-2.1.3-1/
./install.sh
cp statusengine/src/statusengine-naemon-1-0-5.o /opt/statusengine/statusengine-naemon-1-0-5.o
echo "broker_module=/opt/statusengine/statusengine-naemon-1-0-5.o use_log_data=0" > /etc/naemon/module-conf.d/statusengine.cfg
Now you need to connect Statusengine with your database. Due to the fact, that the system will only live for a very short time, we
will use the MySQL root
user for everything.
Open the file /opt/statusengine/cakephp/app/Config/database.php
and search for the line public $legacy = array(
.
Now pass your mysql root credentials, like in this example:
public $legacy = array(
'datasource' => 'Database/Mysql',
'persistent' => false,
'host' => 'localhost',
'login' => 'root',
'password' => 'passw0rd',
'database' => 'naemon',
'prefix' => 'naemon_',
'encoding' => 'utf8',
);
Now we can creat the database and table schema.
cd /tmp/statusengine-2.1.3-1/
mysql -u root -p < sql/naemon.sql
/opt/statusengine/cakephp/app/Console/cake schema update -y --plugin Legacy --file legacy_schema_innodb.php --connection legacy
Copy the file objects.cache
from your current monitoring system to /etc/naemon/conf.d/
.
scp root@current-monitoring.lan:/opt/naemon/var/objects.cache /etc/naemon/conf.d/objects.cache
If you don't know where this file is located on your currently running monitoring machine, open the nagios.cfg or naemon.cfg and look out for the line
object_cache_file=/opt/naemon/var/objects.cache
.
After you copied the file objects.cache
, you are new ready to load the configuration in the file /etc/naemon/naemon.cfg
.
Remove all predefined cfg_dir
and cfg_file
lines and add the following two lines:
cfg_dir=/etc/naemon/conf.d/templates/
cfg_file=/etc/naemon/conf.d/objects.cache
As already mentioned, we only need this system to dump all configuration objects to the MySQL database.
For this reason, we switch the the user naemon
and start Naemon Core in foreground.
sudo -u naemon /bin/bash
naemon /etc/naemon/naemon.cfg
Press CTRL+C
as soon as you can see these line (or after 10 minutes):
Successfully launched command file worker with pid 21162
gearman-job-server
and we don't need the Naemon Core anymore.
exit
to switch back to the user root
Now you need to start Statusengine, also in foreground mode, to get all required data into the database.
/opt/statusengine/cakephp/app/Console/cake statusengine_legacy -w
As soon as you can see the line Info: Finished dumping objects
you can kill Statusengine by pressing CTRL+C
.
Alternatively you can kill it after 10 minutes.
The openITCOCKPIT Configuration Importer it self is a PHP script. It will connect the the MySQL database to get all your configuration items, and create all objects on the openITCOCKPIT system via the HTTP API.
apt-get install git php-zip php-mysql
cd /tmp
curl -o composer-setup.php https://getcomposer.org/installer
php composer-setup.php
cp composer.phar /usr/local/bin/composer
cd /tmp
git clone https://github.com/it-novum/openITCOCKPIT-configuration-import.git
cd openITCOCKPIT-configuration-import/
composer install
If you also want to migrate all $USERx$
macros, you need to copy your resource.cfg
from your current
monitoring system.
scp root@current-monitoring.lan:/opt/openitc/nagios/etc/resource.cfg /tmp/openITCOCKPIT-configuration-import/resource.cfg
If you don't know where this file is located on your currently running monitoring machine, open the nagios.cfg or naemon.cfg and look out for the line
resource_file=/opt/openitc/nagios/etc/resource.cfg
.
You can set various configuration options. Open the file etc/config.yml
and adjust the folowing values to your environment
# Set the same value, as you use in your nagios.cfg or naemon.cfg.
interval_length: 60
# Path where the Importer will search for the resource.cfg
resource_cfg: resource.cfg
# 1 is the default config type of Statusengine - do not change
config_type: 1
# MySQL database host
mysql_host: 127.0.0.1
# MySQL database user
mysql_user: root
# MySQL database password
mysql_password: passw0rd
# MySQL database
mysql_database: naemon
# MySQL database table prefix
mysql_table_prefix: naemon
# IP address of your openITCOCKPIT V3 System
oitc_host: 192.168.1.10
# e-mail address for login
oitc_user: admin@openitcockpit.io
# Password of the given user
oitc_password: password123
Now you are ready to start the migration process.
php bin/Importer.php
If the process is finished, you can destroy this machine and continue working with openITCOCKPIT!
If you found a bug or have some issues running the import script, please create an issue at our GitHub repository.
Today we have released the next version of openITCOCKPIT 3.3.
One of the new features is the System health overview. A cron job is monitoring all important background processes and system metrics. As soon as an issue is detected, System health overview will change its state into warning or critical depending on the possible impact of the issue.
To avoid complete page reloads, we refactored high-traffic pages like the host and service overviews with AngularJS. Due to the new frontend framework, filter requests or pagination are completely handled by an API request instead of an complete page reload.
Also, the MySQL database is a common bottleneck. With the new version, we added a Redis in-memory database that will cache heavy operations like calculating user permissions.
To be able to add more filter options, we developed a completely new filter component and added it to the most visited pages. With every new version, the new filter will be added to more pages.
In the Log entries overview it is now possible to filter by one or more hosts. This makes debugging easier in case of an error, for example if you want to track why a notification was suppressed.
Is openITCOCKPIT theme too light or too dark? We adjusted the Design Module and the inbuilt themes, so it's easier to create your own style.
You now also have the option to add your logo to the top menu bar.
In one of our previous articles we described how to update an installation of openITCOCKPIT in detail. If you are already familiar with our update process, you can update to the new version in three easy steps:
tmux
apt-get update
apt-get dist-upgrade
© 2020 it-novum GmbH - All Rights Reserved