{"id":31946,"date":"2021-07-05T16:59:41","date_gmt":"2021-07-05T16:59:41","guid":{"rendered":"https:\/\/www.infosim.net\/stablenet\/?p=31946"},"modified":"2025-04-08T09:11:14","modified_gmt":"2025-04-08T09:11:14","slug":"network-automation-for-discovery-inventory","status":"publish","type":"post","link":"https:\/\/www.infosim.net\/stablenet\/blog\/network-automation-for-discovery-inventory\/","title":{"rendered":"Part 4: Network Automation for Discovery & Inventory"},"content":{"rendered":"

[et_pb_section fb_built=”1″ _builder_version=”4.16″ background_color=”#002846″ background_color_gradient_direction=”60deg” global_module=”41229″ global_colors_info=”{}”][et_pb_row _builder_version=”4.21.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.21.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_blurb image=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/03\/Blog-1.png” icon_placement=”left” image_icon_width=”67%” _builder_version=”4.21.0″ _module_preset=”default” body_text_color=”#FFFFFF” body_font_size=”18px” custom_css_blurb_image=”width:100px” custom_css_blurb_content=”max-width: none!important;” global_colors_info=”{}”]StableNet\u00ae<\/sup>\u00a0Blog
\n<\/strong>Regular posts on all things StableNet\u00ae<\/sup> related from a sales, techie, or marketing perspective[\/et_pb_blurb][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.16″ background_color=”#D6D6D6″ custom_padding=”||2px|||” global_colors_info=”{}”][et_pb_row disabled_on=”on|on|off” _builder_version=”4.16″ _module_preset=”default” background_color=”#dd8430″ custom_margin=”||40px||false|false” custom_padding=”15px|25px|8px|20px|false|false” border_width_all=”0px” border_color_all=”#e09900″ global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#FFFFFF” text_font_size=”22px” custom_margin=”||||false|false” custom_padding=”||||false|false” global_colors_info=”{}”]<\/p>\n

Network Automation Insights
<\/b><\/p>\n

Stay tuned and keep informed with our blog series about the future of automation in network and service management<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=”2_5,3_5″ _builder_version=”4.16″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||0px||false|false” custom_padding=”|25px|2px|25px|false|false” border_width_top=”4px” border_color_top=”#dd8430″ border_width_right=”4px” border_color_right=”#dd8430″ border_width_left=”4px” border_color_left=”#dd8430″ global_colors_info=”{}”][et_pb_column type=”2_5″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_blurb image=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/03\/Blog-Automation-Series-1-1.png” icon_placement=”left” _builder_version=”4.16″ _module_preset=”default” body_text_color=”#dd8430″ global_colors_info=”{}”]<\/p>\n

Automation Series – Part 4<\/b><\/p>\n

[\/et_pb_blurb][\/et_pb_column][et_pb_column type=”3_5″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.16″ _module_preset=”default” background_color=”#FFFFFF” custom_padding=”15px|25px|3px|25px|false|false” border_color_top=”#dd8430″ border_width_right=”4px” border_color_right=”#dd8430″ border_width_left=”4px” border_color_left=”#dd8430″ global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.21.0″ _module_preset=”default” header_text_align=”left” background_color=”#FFFFFF” custom_padding=”|||0px|false|false” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n

Network Automation for Discovery & Inventory<\/h1>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.16″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||0px||false|false” custom_padding=”|25px|0px|25px|false|false” border_width_right=”4px” border_color_right=”#dd8430″ border_width_left=”4px” border_color_left=”#dd8430″ global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.16″ background_color=”#FFFFFF” custom_margin=”||15px||false|false” custom_padding=”1px|||||” global_colors_info=”{}”]<\/p>\n

July 5th<\/sup> 2021, W\u00fcrzburg<\/p>\n

[\/et_pb_text][et_pb_toggle title=”This post is part of the ongoing blog series about Network Automation. Click here to see all parts.” _builder_version=”4.21.0″ _module_preset=”default” title_font=”|700|on||||||” global_colors_info=”{}”]Part 1: A Network Automation Platform improves Operations to deliver Business Results<\/a><\/p>\n

Part 2: Platforms and Workflows that Automate Operations<\/a><\/p>\n

Part 3: Network Automation \u2013 Market Perspectives<\/a><\/p>\n

Part 4: Network Automation for Discovery & Inventory<\/p>\n

Part 5: Network Automation \u2013 Configuration Generation<\/a><\/p>\n

Part 6a: Network Automation \u2013 Monitoring and Tagging (Part 1 of a 3-Part Series)<\/a><\/p>\n

Part 6b: Network Automation \u2013 Monitoring and Tagging (Part 2 of a 3-Part Series)<\/a><\/p>\n

Part 6c: Network Automation \u2013 Monitoring and Tagging (Part 3 of a 3-Part Series)<\/a>[\/et_pb_toggle][et_pb_text _builder_version=”4.16″ background_color=”#FFFFFF” custom_padding=”1px|||||” global_colors_info=”{}”]<\/p>\n

In the last blog post, we had a look at the automation market and \u201cdiscovered\u201d that Network Automation starts with Discovery to build the inventory baseline as a foundation and that everything else proceeds from there.<\/p>\n

So let\u2019s start by having a look at some automated discovery use cases. Then we can explore customization options to structure a large, multi-vendor network environment. We\u2019ll finish with a brief introduction of some of the main advantages of the XML discovery engine.<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.16″ _module_preset=”default” background_color=”#FFFFFF” custom_padding=”|25px||25px|false|false” border_width_right=”4px” border_color_right=”#dd8430″ border_width_bottom=”4px” border_color_bottom=”#dd8430″ border_width_left=”4px” border_color_left=”#dd8430″ global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

Network Discovery Use Cases:<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

The re-occurring discovery component in the StableNet\u00ae<\/sup> Workflow Cycle (see figure 1) is the most important function of the entire inventory-building and -maintenance process. Its programmability is built on the eXtended Markup Language (XML), which stores data in plain text format and provides platform-independent data sharing and transport. XML is human readable, easy to understand – even by novices – and not difficult to code.<\/p>\n

The XML template provides the steering function for the discovery of an entire network (or can be directed towards specific parts of it). Its logic is structured as building blocks that determine the type of collected information, the included input sources, their properties, as well as the setup of all objects that are created and used within StableNet\u00ae<\/sup>.<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/stablenet-network-discovery-1.png” alt=”StableNet Network Automation Graphic” title_text=”Stablenet Network Discovery 1″ align=”center” _builder_version=”4.16″ _module_preset=”default” width=”70%” max_width=”80%” custom_padding=”10px||10px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 1<\/strong>: StableNet\u00ae<\/sup> Network Discovery Use Cases<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Network discovery automation use cases include inventory creation and maintenance with bidirectional communication to external sources, discovery, and selective re-discoveries. Device property defaults and auto-measurements collect the most common and typically used data points from the infrastructure to provide immediate, out-of-the-box results. This built-in discovery automation already covers a significant number of KPI requirements and provides systemwide network visibility with status updates on visualizations (weather maps).<\/p>\n

However, the real power of network automation kicks in once we begin creating specific measurements (data collection) and monitors (KPI validation) to meet customization requirements. That programmability, in combination with meta data enrichments (tagging), encapsulates the core strength of the solution and is the key to generating a fully-tailored network inventory.<\/p>\n

The ability to leverage existing data sets (i.e. technical measurements from the network infrastructure which are enriched with business-related aspects) can thus be aggregated into a single data structure automatically. This generates a truly valuable inventory which can be further leveraged as single \u201csource of truth\u201d.<\/p>\n

Commonly used inventory data sets are structured by geography (country, regions etc.), organization (corporation, subsidiaries, departments etc.), topology (network hierarchy, aggregated links, critical interfaces etc.) and can be complemented by distinct business aspects like assets in the field, deployed vendors, used technologies, defined services, or even test cases in a lab environment.<\/p>\n

[\/et_pb_text][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

Meta Data Enrichment (Tagging):<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Tagging is a very powerful tool within StableNet\u00ae<\/sup> that has been automated to the largest extent possible. It generates \u201cany and many\u201d complementary data structures within the system. As a consequence, an operator can navigate, sort, combine, filter and query the infrastructure on any of those criteria with the help of the GUI by using boolean logic or regular expressions.<\/p>\n

A tag consists of a tag category and a tag value. Tags can be attached to many different taggable objects and used to store any kind of information. To allow for a quick StableNet\u00ae<\/sup> configuration ramp up, taggable objects by default come with many preconfigured tags of different flavors (measurement, custom, attribute). Most of them can be freely added, deleted or changed. Tags have a parent-child relationship and are inherited by subordinate objects.<\/p>\n

A single object gets a particular tag value assigned. For example, a measurement has a tag with the tag category Measurement Category. The value of this tag is Interfaces, which allows all measurements with this tag value to be grouped together. In a tree, this creates a folder \u201cInterfaces\u201d which contains all measurements with this tag value.<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/tagging-1.png” alt=”StableNet Network Automation Graphic” title_text=”Tagging” align=”center” _builder_version=”4.16″ _module_preset=”default” max_width=”80%” custom_padding=”10px||10px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 2<\/strong>: Screenshot how StableNet\u00ae<\/sup> visualizes Tag Trees<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Tag categories organize tags and group similar functions. Once created, they can be used globally for multiple objects. For the previous example, the Measurement Category separates measurements into different kinds (e.g. Interfaces Processors, Routing, Syslog etc.).<\/p>\n

Objects in StableNet\u00ae<\/sup> are aggregated in tag domains, e.g. devices are represented by a tag domain Device, while their interfaces are represented by the tag domain Interface. This structure allows for general relationship descriptions like Devices have Interfaces.<\/p>\n

The below diagram shows all tag domains of StableNet\u00ae<\/sup> along with their relationships to one another. There are predefined read-only tags for each domain, which are automatically set by the system if the required information is available.<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/tag-domains-stablenet-1.png” alt=”StableNet Automated Solution Table Graphic” title_text=”Tag Domains Stablenet” align=”center” _builder_version=”4.16″ _module_preset=”default” width=”70%” max_width=”85%” custom_padding=”15px||15px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 3<\/strong>: Tag Domains (Diagram)<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

By leveraging and maintaining tags within StableNet\u00ae<\/sup>, users are able to structure available and collected data sets according to their individual needs.<\/p>\n

[\/et_pb_text][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

Automated Discovery:<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

The key contributors to network automation are Discovery Jobs and the built-in XML Discovery Engine. Those components take command of the agent conversation. The below schema highlights the essential building blocks, which orchestrate inventory creation and regular update processes.<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/stablenet-network-discovery-2.png” alt=”StableNet Network Automation Graphic” title_text=”Stablenet Network Discovery 2″ align=”center” _builder_version=”4.16″ _module_preset=”default” width=”70%” max_width=”80%” custom_padding=”10px||10px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 4<\/strong>: StableNet\u00ae<\/sup> Automated Discovery (Discovery Job and XML Engine)<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Discovery Jobs are configured and scheduled in the StableNet\u00ae<\/sup> Device Automation Theme. They are a prerequisite to run the Discovery Engine. The XML template allows the definition of placeholders, which are then mapped during runtime executions to the corresponding values.<\/p>\n

[\/et_pb_text][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

Various Input Sources:<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Comma separated value (csv) files are the most common source of infrastructure data and are required by the agent in order to query information from deployed network devices. An input csv file typically contains IP host addresses, network ranges (subnets), host names, geo data, snmp version and hashed credentials with authentication information.<\/p>\n

A more elegant solution is querying an external CMDB (Configuration Management Database) directly during the discovery process by using database variables. Quite often such a CMDB does not exist or is not properly maintained. Under those circumstances, StableNet\u00ae<\/sup> acts as an \u201cInfrastructure Asset CMDB\u201d or serves a synchronization function by automatically updating information that was queried from the network in bidirectional communication.<\/p>\n

Another StableNet\u00ae<\/sup> discovery automation variant is provided by the Resource Management module. This concept uses a modelled resource pool inside of StableNet\u00ae<\/sup>, consisting of objects and tables, to organize a larger inventory.
\nResource Management allows users to define devices within StableNet\u00ae<\/sup> before they exist in the network and then leverage common database technologies like filtering, adding tags and assigning group\/user rights for access.<\/p>\n

Using the Resource Pool concept improves asset structures and organization into different inventory areas. It also simplifies the process of adding devices according to defined criteria and the usage of tags to delete them again. Typical use case examples are the decommissioning of network elements due to device replacements, network re-organizations or corporate mergers.<\/p>\n

In summary: leveraging Resource Pools for the discovery process leads to network automation improvements and a better organized inventory structure that facilitates change management for defined assets.<\/p>\n

[\/et_pb_text][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

XML Discovery Templates:<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

StableNet\u00ae<\/sup> XML Discovery templates are designed to automate and individualize inventory creation for a diverse range of network and IT infrastructures.<\/p>\n

A template can be used with different values for parameters and tags to automatically set up measurements and monitors without manual intervention, resulting in a radical reduction in resource expenditures for recurring tasks for discovered devices. XML templates are auto-generated by StableNet\u00ae<\/sup> and the built-in XML editor provides syntax checks with auto-completion and version tracking during the customization process.<\/p>\n

[\/et_pb_text][et_pb_text admin_label=”I. Agent Flow Query – H2″ _builder_version=”4.21.0″ min_height=”40px” custom_margin=”||||false|false” custom_padding=”13px||||false|false” global_colors_info=”{}”]<\/p>\n

Discovery Engine:<\/h2>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

StableNet\u00ae<\/sup> Discovery Engine refers to the discovery logic which makes the automated \u201cinventory creation magic\u201d happen. The high-level overview in the flowchart (see figure 5) shows the process when a set of devices is discovered by using the previously outlined input sources.<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/discovery-engine-flowchart.png” alt=”StableNet Automated Solution Table Graphic” title_text=”Discovery Engine Flowchart” align=”center” _builder_version=”4.16″ _module_preset=”default” width=”60%” max_width=”85%” custom_padding=”15px||15px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 5<\/strong>: StableNet\u00ae<\/sup> Discovery Process (1\/2)<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

At the start of discovery, prefilters are checked. These are used to select the input sources before connections are established and thus reduce the runtime in large environments. The next step depends on the default property \u201cAdd Devices which are not reachable\u201d. If enabled, all devices are discovered. Otherwise, a ping scan only discovers reachable devices.<\/p>\n

The execution of device discoveries runs in parallel, and the number of threads can be specified in the Job Wizard of the respective Discovery Job. That parallel portion when discovering a single device is shown in the next flowchart (see figure 6).<\/p>\n

[\/et_pb_text][et_pb_image src=”https:\/\/www.infosim.net\/stablenet\/wp-content\/uploads\/5\/2021\/07\/discovery-engine-flowchart2.png” alt=”StableNet Automated Solution Table Graphic” title_text=”Discovery Engine Flowchart2″ align=”center” _builder_version=”4.16″ _module_preset=”default” width=”70%” max_width=”85%” custom_padding=”15px||15px||true|false” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.16″ _module_preset=”default” text_text_color=”#002846″ text_font_size=”14px” text_orientation=”center” custom_margin=”-2%||||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

Figure 6<\/strong>: StableNet\u00ae<\/sup> Discovery Process (2\/2)<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

At first, the StableNet\u00ae<\/sup> Agent establishes a connection to the target device and determines, based on pre-defined conditions, whether it is relevant. This triggers additional sub-processes which identify the device relevance. After a connection has been established, device post-filters come into play. They use information that is only available on that particular device. These additional inputs make the post-filters very powerful and flexible.<\/p>\n

The next step is for StableNet\u00ae<\/sup> to check if that device is already part of the inventory. The underlying logic for executing this sub-process is quite complex. We\u2019ll skip it here to avoid making the blog post too long. Devices previously contained in the inventory are updated if necessary, while new devices are added.<\/p>\n

The next step is to create the custom measurements from the device block of the XML Discovery template. The different StableNet\u00ae<\/sup> measurement types are categorized as Runtime, Application, SNMP, Flow Fault and Status. These categories in turn contain measurements for Common Elements and Attributes, TCP Connect, Ping, SNMP Trap, Syslog, Status, Script, SNMP Template, WMI Template, External and Multi Measurements, SNMP Auto Templates, IP SLA, Netflow, Interface, Bandwidth (Element, Interface), Derived Monitors and Link Generation as an outline of the available toolkit.<\/p>\n

Finally, the last step in the discovery process is the addition of auto measurements and interface measurements (unless the auto attribute in the device block is set to false). The following measurements are added:<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n