Datadog Network Device Monitoring: Configuration & Troubleshooting

Introduction

With the release of version 7.39.0, the Datadog Agent supports an integrated ‘snmpwalk’ command using the GoSNMP library. This quality-of-life addition alleviates the need for additional SNMP packages/binaries that had to be previously installed and utilized during the configuration process. Let’s jump into some standard configurations needed for Datadog’s NDM and how the integrated ‘snmpwalk’ command will simplify the testing process.

Setup

For demo purposes, we switch between an agent (v7.43) running on an AWS EC Instance (size is t2 running Ubuntu 22) and my local computer (Macbook). We will leverage existing Cisco Catalyst devices to test network devices, one of which will authenticate with the default community string and the other with SNMP v3 credentials.

Agent Status

When configuring any integration, the Datadog agent status command will be essential to report the current state of any running configuration. The most common error is timeout, which usually indicates that the device has not been configured for SNMP support or network access issues between the agent host and the device.

Configuring a device for SNMP support

If this is the first time you are monitoring your device using SNMP, you will need to configure SNMP support on the device according to the manufacturer’s documentation.

Following their documentation, the following example walks through the SNMP v3 setup for a Cisco Catalyst.

• Connect to your device, enable, configure terminal, and validate there is no existing snmp configuration using command: do show snmp community
• Create the view, giving it a name and specifying the top-level root OID ‘iso’ using command: snmp-server view ViewDefault iso included
• Then validate that your new view was created with command: do show snmp view
  
• Create the monitoring group, giving it a name, specifying read access to our created view with command: snmp-server group GroupMonitoring v3 priv read ViewDefault
• Then validate your new group was created with command: do show snmp group
  
• Lastly create a user, assign the formerly created group and set the authentication password and secret using command: snmp-server user DemoUser GroupMonitoring v3 auth sha DemoPassword priv aes 128 DemoSecret
  Be sure to fill in your own unique password and secrets when configuring 🙂
• Finally, validate that your user was created and the SNMP engine is up and running with do show snmp user and do show snmp engine. (Don’t forget to save with do copy run start)
  
• Now that we have saved, we can test the newly configured credentials with the agent ‘snmpwalk’ command and see the walk returns as expected, and your device is ready for monitoring!
  
• You then would take the create an new instance in the snmp.d/conf.yaml using the parameters we passed into the walk command and restart the agent
  

Network Access Issues

The most common issue when configuring NDM is network access between the Agent host and the monitored device itself. Causes might point to many specific issues depending on your infrastructure architecture, but at a high level, there needs to be a path for UDP traffic over port 161 between the source and destination. We created a security group for this demo and attached it to the t2 previously mentioned.

This security group currently allows my IP SSH access to the host, and is missing the rules needed for SNMP. If we run the agent SNMP walk command to our Cisco Switch, we see it times out.

Updating our security group to allow outbound SNMP traffic

Allows us to successfully ‘snmpwalk’ the device.