[Datadog] Duplicate Host Names Appearing After AWS Integration

After performing the AWS Integration, you may notice a sudden increase in the number of hosts.

Here are some areas to check and consider in such cases:

When AWS Integration is enabled, all metrics available via CloudWatch within AWS are collected.

This includes EC2 metrics, and those EC2 instances are counted as part of the infrastructure host count.

If any of the EC2 instances already have the Datadog Agent installed, those instances will be mapped to Agent-based hosts. In this case, they will appear as a single host and will not incur additional charges since they are already being billed.
(Reference: Datadog Billing - Host)

However, in some cases, even if the Agent is installed on the EC2 instance, after AWS Integration, it may not be correctly mapped to the existing Agent host and may be detected as a separate host.
If this occurs, a single host could be mistakenly recognized as two, leading to duplicate billing.

Please follow the steps below to investigate this issue: 

• Check the Plan & Usage section. If all EC2 instances included in the AWS Integration have the Datadog Agent installed,
  then under Usage Summary > Infrastructure, all Infra Hosts should appear as Agent Hosts.
  If there are AWS hosts without the Agent installed, they will appear separately as AWS Hosts, as shown in the image below.
  Of course, it's possible to monitor EC2 instances using AWS Integration alone without installing the Agent.
  However, assuming that the Agent should be installed on all EC2 instances, this method allows you to easily identify how many EC2 hosts are not running the Agent.
  
• In such cases, check the Infrastructure list to determine whether a host is truly being recognized as a duplicate,
  or if it's simply an EC2 instance where the Agent was not installed.
  •  In the Infrastructure list, filter by field:apps:agent to display only the hosts with the Agent installed.
    
  •   If an EC2 instance included in the integrated account is properly mapped, the Host Details should display an AWS EC2 instance ID starting with "i-" under Aliases. Additionally, AWS tag information should be shown under TAGS.
    
  •  To check whether an EC2 instance was not mapped correctly and is instead recognized separately as an AWS Host, search by Host name in the Infrastructure List.
    When searching by tag value only (without specifying a tag key), any host that contains the value in a different tag (such as host or name) will also appear in the results.
    In the example below, the entered value is shown as a value in both the host and name tags for the respective hosts.
    

The reason why a host appears separately even though it is the same host detected both by the Agent installation and AWS EC2 Integration is due to a failure in mapping.

After installation, the Agent collects hostname information from the host in the following order: 

1. agent-hostname: A hostname explicitly set in the Agent configuration file if it does not start with ip- or domu.
2. hostname (hostname -f on Linux): If the DNS hostname is not an EC2 default, for example: ip-192-0-0-1.
3. instance-id: If the Agent can reach the EC2 metadata endpoint from the host.
4. hostname: Fall back on the DNS hostname even if it is an EC2 default.

The hostname detected first is set as the primary hostname, and any subsequent hostnames are displayed as aliases. 

The key issue is that at least one of the hostnames or aliases must be mappable between the Agent and EC2.

In other words, for successful mapping between the data collected by the Agent and the data from AWS EC2 Integration, the Agent must collect the instance ID via EC2 metadata (specifically point 3 mentioned above).

First, check the Agent status and confirm whether Instance-id is displayed in the Hostnames section.

 If the EC2 host is not retrieving the instance ID, check whether the EC2 instance supports metadata via IMDSv1 or IMDSv2. 

The following command is a URL used to retrieve the instance-id from metadata using IMDSv1.

If the instance ID is not returned when running this command, it is likely that the instance supports metadata access only through IMDSv2.

To check whether your instance supports IMDSv2 metadata access, you can run the following commands sequentially on a Linux system: 

Check if the instance ID is retrieved successfully.

For Windows, execute the following commands sequentially in PowerShell to confirm whether the instance ID is retrieved properly.

Check if the instance id is retrieved properly. 

(For detailed information, please refer to the link below: AWS Instance metadata access) 

In the case of EC2s that support metadata with IMDSv2, set the ec2_prefer_imdsv2 field to true in the datadog.yaml file, restart the agent, and check the agent status again to confirm that the instance id is retrieved properly.

If the AWS Host that was displayed separately is properly mapped with the Agent after applying this setting, it will disappear from the Infrastructure list approximately two hours later. 

If the mapping is still not restored even after this action, please contact us via a ticket on the Support Portal. 

Reference Links:

Agent status check command

AWS Instance metadata access

If metadata access is still not possible even after setting ec2_prefer_imdsv2, please refer to the methods below.

• How to resolve metadata access issues on EC2 Windows instances
• How to resolve metadata access issues on EC2 Linux instances