EMMA

Evolved Management and Monitoring Application

 

Keywords: emma, cdp, lldp, lacp.

 

Managing the enterprise environment requires a multi-vendor monitoring platform for the infrastructure. The IT infrastructure, in every business, is comprised of network equipment, servers, desktop, and mobile devices. In order to track the active devices and services and also find and debug all the problems that appear on such an IT infrastructure, one needs a tool that centralizes all the elements and is able to correlate among them.

The main issue is the diversity of the equipment and services, being hard to find an application that is doing also tracking and active monitoring and debugging on the whole infrastructure. Two main elements in the business are the NOC (Network Operation Center) which takes care of the functionality of an IT infrastructure and the SOC (Security Operation Center) which takes care of the security issues in the business environment. In order to cover these aspects presented above, we propose EMMA (EVOLVED MANAGEMENT AND MONITORING APPLICATION).

EMMA is offering support for multiple network topologies, as well as all the IT Infrastructure auxiliary equipment (UPS, cooling, etc). We engaged in scaling the platform, making it more flexible without forgetting the automation of the processes, and included a ticketing tool that, automated or supervised by the NOC Engineers, is able to offer traceability for the incidents.

As the infrastructure evolved, our proposed platform evolved and scaled to the next level, covering the need for a Configuration Management Database and IP Address Management which is the starting point of the next tools that will be integrating into our platform.

There had been different approaches, oriented only to security objects where it is created a knowledge database regarding security incidents. We aim to extend these solutions in order to also store the assets of an entity and do relevant correlations in order to easily trace a security incident. To this extent, we can provide better sharing of the security information.

EMMA acts as a processing and correlation module for the current data sources and security platforms. EMMA is fed with information from different vulnerability databases and assets (via SNMP). The information is stored and correlated for different activities needed in a NOC/SOC environment: monitor, alerting, recovering. EMMA is easily integrable with security products like Hive (for incident response) and MISP (for incident sharing).

 

Platform design

Nowadays, the first step for any adaptive network monitoring platform starts with the infrastructure discovery process and, given the fact that we need a platform that can easily adapt to any challenge infrastructure design, we created EMMA as a modular application.

EMMA is able to run a discovery process that can use any information provided by a CMDB/IPAM tool that the company is already using. Also, EMMA is able to do automatic network discovery that can use various technologies based on a layered model structured on the OSI stack. In order to reduce the time and increase the performance of our software platform, we are using multi-processor capabilities.

The discovery process is following the layered design performing the following steps:

1. Infrastructure devices and links discovery
2. Layer 2 tunnels and interface discovery
3. Additional IP addresses discovery
4. Layer 3 IP tunnels and link
5. IP routing layer protocols

The new devices, interfaces, and links found are inserted in the database having a “pending approval” flag. Starting from a given host, we are using multiple algorithms and different techniques to create a map of the network mostly based on discovery protocols available (CDP, LLDP, HDP, etc).

For the data that is generated in the discovery process, the platform triggers the next module that automatically finds the management and monitoring IP address of the device using a prioritizing search pattern algorithm. If the algorithm cannot retrieve the IP address, the platform will automatically use the IP address reported by the discovery protocol.

The last step of the infrastructure discovery consists of adding the interfaces and devices to the database operation that leads to the physical network topology, grouping the devices with interface links as edges.

Discovery procedure

After the crawl functions are finishing the discovery process, the platform is generating a report containing the following information:

• new devices, interfaces, and links (added as user requested)
• not found devices, interfaces, and links (no action, just reporting)
• not found IP addresses according to search patterns
• various errors required for adjusting the script to network particularities

EMMA’s second step aims to discover link aggregations and L2 tunnel interfaces (LACP, ETHERCHANNEL, etc.) which are retrieved and automatically added to the database building a new layer of device interconnections. A common technology, used in many of the existing enterprise environments, is IP Tunneling. These tunnels, in our case, are detected and added to the monitoring platform. Based on the newly added datasets, the third layer of interconnection is created.

Each routing protocol or process is discovered and together with the details and parameters are automatically added to the database for monitoring/troubleshooting, building the final layer of interconnections.

As EMMA offers modularity, the database is structured to fit multiple scenarios allowing someone to have devices added/grouped in sites/branches, that can be further grouped by cities, in counties from different countries and, using geolocation tools we can easily place pins on maps to have a complete topology of the infrastructure (see Fig. 3).

 The structure of the assets

For devices and site objects created in EMMA, one can add various details, such as person in charge, service suppliers (companies), physical and IT security-related information (see Fig. 4). Also, each object has different methods of contact (email, phone). Various data can be added for contacts (ID cards, phones, emails, positions, notes) and suppliers (fiscal data, address, bank accounts, notes).

Device management

Use-cases include monitoring MAC – IP address changes on network ports for tracking users and easier troubleshooting tickets, determining IP addresses for DHCP-enabled devices when the MAC or used port is known.

IPAM – IP Address Management

We used a vulnerability database that is using CVE’s with a specific scoring system in order to track down the vulnerabilities that we have in the network.

For this to be possible we developed a custom integration based on API calls to retrieve all vulnerabilities related and match the software version found in vulnerabilities descriptions. After this operation, the databases updated with the new vulnerabilities, and alerts are triggered (see Fig. 7).

Because the industry practice is to address the vulnerabilities and harden the security using workarounds, the integration comes with a module for keeping track of addressed vulnerabilities for each device which contains details about the workaround (implementation time, solution, etc.).

Vulnerabilities on assets

Data is retrieved via SNMP using multithreading collectors for efficiency. The process is monitored for errors, overhead, performance, and continuously improved. Data is stored in multiple database types – RRD, InfluxDB for time series data, SQL for specific devices data, and MongoDB for free text data (vulnerabilities database) [5].