AWS Autodiscovery

The Device42 AWS autodiscovery feature ensures you have full visibility into your AWS environments. This visibility gives you insight into your total AWS consumption, allowing you to optimize spending and services consumed. New discoveries are added all the time based on the needs of our customers.

Discovery Items List Page

You can view the discovered AWS items by navigating to the list page under Resources > Cloud Resources. Use the search bar on the left or click on one of the column dropdown menus to filter for specific AWS discovery item fields.

For example, click the Vendor Resource Type dropdown menu and start typing a discovery item type in the search bar. Filter by a segment of the item field name using the advanced Contains search bar or use the Not Contains criteria to filter by exclusion.

You can save your search to apply it later without repeating the search steps. Click the gear icon on the right, name your search (1), and click the Save As New button.

To change which columns are displayed, click Table Columns (2) and select and deselect the columns from the list.

If you want to keep this list page setup as the default view, select the System Default box.

Getting Started with AWS Autodiscovery

To create an AWS Autodiscovery job, you will need to:

1. Prepare your AWS Account. You can use the policy example shown below.
2. Device42 uses your AWS Access Key and Secret Key to perform discovery, so please have these handy.

note

Device42 encourages you to follow AWS best practices for managing your IAM credentials, including using strong passwords, regular password rotation, applying the principle of least privilege to users and their passwords, and so on.

For more information, see Best Practices for Managing AWS Access Keys on Amazon.

Initiating an AWS Discovery

• Select Discovery > Cloud from the main menu and then click Create at the top right of the Cloud Autodiscovery list page.
• Enter a Name for the job.
• Select Type > Amazon AWS from the dropdown menu.

• Select the Remote Collector for the job.
• Select Use Environment Credentials (see Setting Up Environment Credentials below), or add your Amazon Access Key ID and Secret Key for the account(s) to be discovered:

Click the magnifying glass icon and Add Password in the upper-right corner. Enter your Access Key ID or Secret Key in the field labeled Password. Device42 uses encryption to store the keys.

• Select Discover Main Account to discover the main AWS account in addition to any AWS Roles accounts you select.
• Select the Available AWS Roles for the account(s) you want to discover using the arrow to add them to the Chosen AWS Roles list.

Note: See Defining AWS Roles below for instructions on creating the AWS Roles that Devices42 displays for AWS cloud autodiscovery jobs.

• Choose one or more Amazon regions to search.

• You can also use Add vendor metadata as to specify the format for vendor metadata, choose an Action for Instance not found in subsequent discoveries, select Device Name Format options, and add Tags for discovered devices.
• Check Kubernetes Discovery to discover Kubernetes clusters hosted on your cloud platform.
• Check Extended Summary Discovery to discover the full breadth of resources within your AWS Environment. These resources will be displayed in the Cloud Resources section with a limited dataset.
• Optionally, you can set the Service Level of the job to be applied to the discovered items.
• Add object categories, tags, and a customer for discovered devices.
• Scroll down the page to the Advanced Features (Show) tab and select the types of resources you want the job to discover (for example, Route53, S3, EBS, and Databases).

• Add an optional Autodiscovery Schedule to schedule the job.
• Add the Admin Groups for the job.
• Click Save or Save and continue editing to save the discovery job.
• To run the job immediately, go to the Cloud Discovery list page and click Run Now.

AWS Discovery Items

Note that some Discovery items require enabling the feature and cannot be discovered otherwise.

Cloud Service/Object Name	Where in Device42	Accessed API	Information Generated	Permission(s) Required
API Gateway	Resources --> All Resources	apigateway._region_.amazonaws.com	Cloud vendor, cloud account, etc.	apigateway:GET
AWS Account Name (fallback)	Infrastructure --> Cloud Accounts	iam.amazonaws.com	Cloud account alias	iam:ListAccountAliases
CloudFront	Resources --> All Resources	cloudfront.amazonaws.com	Cloud vendor, cloud account, status, tags, etc	cloudfront:ListDistributions, cloudfront:ListTagsForResource
EC2 Instances	Resources --> All Devices	ec2.region.amazonaws.com	Service name, instance id, status, location, etc.	ec2:Describe*
Elastic Block Storage (EBS)	Resources --> All Resources	ec2.region.amazonaws.com	Lists, rules, tags, etc.	ec2:Describe* (EBS is part of EC2)
ElastiCache Nodes	Resources --> All Resources	elasticache.region.amazonaws.com	Account info, status, location	elasticache:Describe*
Elastic File System (EFS)	Resources --> All Resources	elasticfilesystem.region.amazonaws.com	File system, access points, mount targets	elasticfilesystem:DescribeAccessPoints, elasticfilesystem:DescribeAccountPreferences, elasticfilesystem:DescribeFileSystems, elasticfilesystem:DescribeMountTargets
Elastic Load Balancer (ELB)	Resources --> All Resources	elasticloadbalancing.region.amazonaws.com	Attributes, description, rules, tags, target groups	elasticloadbalancing:Describe*
Lambda	Resources --> All Resources	lambda.region.com	Name, ARN, code size, memory, runtime	lambda:GetAccountSettings, lambda:GetFunction, lambda:GetPolicy, lambda:List*
KMS	Resources --> All Resources	kms.region.amazonaws.com	Region, categories, access points, ACLs, notes, tags, custom fields	kms:DescribeKey, kms:ListKeys, kms:ListResourceTags
Kubernetes (EKS)	Resources --> All Resources	eks.region.amazonaws.com	Containers, nodes, clusters	eks:DescribeCluster, eks:DescribeNodegroup, eks:DescribeUpdate, eks:ListClusters, eks:ListNodegroups, eks:ListUpdates
RDS Instances	Resources --> All Resources	rds.region.amazonaws.com	Account info, status, location	rds:Describe*, rds:ListTagsForResource
Redshift	Resources --> All Resources	redshift.region.amazonaws.com	Backup details, contributor insights, tables, limits	redshift:DescribeClusters, redshift:DescribeReservedNodes
Route 53	Resources --> All Resources	route53.amazonaws.com	Type, content, tags	route53:ListHostedZones, route53:ListResourceRecordSets, route53:ListTagsForResource, route53domains:ListDomains
S3	Resources --> Storage --> Cloud Storage	*.s3.region.amazonaws.com s3.region.amazonaws.com *.s3.amazonaws.com s3.amazonaws.com	Storage utilization, bucket, bucket policies	s3:GetAccessPointPolicyStatus, s3:GetBucketAcl, s3:GetBucketLocation, s3:GetBucketPolicyStatus, s3:GetBucketPublicAccessBlock, s3:GetBucketTagging, s3:GetEncryptionConfiguration, s3:ListAccessPoints, s3:ListAllMyBuckets
Security Group (AWS)	Resources --> Cloud Resources	ec2.region.amazonaws.com	Region, identifier, owner ID, Vpc Id, etc.	ec2:DescribeSecurityGroups
SNS	Resources --> All Resources	sns.region.amazonaws.com	Topic, endpoints, attributes, subscriptions	sns:GetTopicAttributes, sns:ListTagsForResource, sns:ListTopics
SQS	Resources --> All Resources	sqs.region.amazonaws.com	Queues, tags, queue attributes	sqs:GetQueueAttributes, sqs:ListQueues, sqs:ListQueueTags
Subnets	Network --> Subnets		Subnets	ec2:DescribeSubnets (Subnets are part of EC2)
VPCs	Resources --> VPC	vpc.aws-region.amazonaws.com	Attributes, AZs, Auth rules, etc.	ec2:DescribeVpcs (VPCs are part of EC2)

Additional Endpoint Information

Regular Discovery

• sts.amazonaws.com
• sts.<region_name>.amazonaws.com or sts.*.amazonaws.com: Try these endpoints if you encounter an SSL certificate error.
• https://organizations.us-east-1.amazonaws.com: Only use this if one of the available features is enabled.

K8s cluster endpoints access per K8s RBAC setup

• /api/v1/namespaces/kube-system
• /api/v1/nodes?watch=False
• /api/v1/services?watch=False
• /apis/apps/v1/deployments?watch=False or /apis/extensions/v1beta1/deployments?watch=False (depends on K8s version)
• /apis/networking.k8s.io/v1beta1/ingresses?watch=False or /apis/extensions/v1beta1/ingresses?watch=False (depends on K8s version)

Example IAM Policy (except for the K8s cluster endpoints, since it is controlled by K8s RBAC)

Click to expand code example

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "acm:DescribeCertificate",
        "acm:List*",
        "apigateway:GET",
        "autoscaling:Describe*",
        "cloudfront:ListDistributions",
        "cloudfront:ListTagsForResource",
        "cloudsearch:DescribeDomains",
        "cloudwatch:Describe*",
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "config:SelectResourceConfig",
        "dynamodb:DescribeGlobalTable",
        "dynamodb:DescribeLimits",
        "dynamodb:DescribeTable",
        "dynamodb:ListGlobalTables",
        "dynamodb:ListTables",
        "ec2:Describe*",
        "eks:DescribeCluster",
        "eks:DescribeNodegroup",
        "eks:DescribeUpdate",
        "eks:ListClusters",
        "eks:ListNodegroups",
        "eks:ListUpdates",
        "elasticache:Describe*",
        "elasticfilesystem:DescribeAccessPoints",
        "elasticfilesystem:DescribeAccountPreferences",
        "elasticfilesystem:DescribeFileSystems",
        "elasticfilesystem:DescribeMountTargets",
        "elasticloadbalancing:Describe*",
        "iam:ListAccountAliases",
        "kms:DescribeKey",
        "kms:ListKeys",
        "kms:ListResourceTags",
        "lambda:GetAccountSettings",
        "lambda:GetFunction",
        "lambda:GetPolicy",
        "lambda:List*",
        "logs:DescribeLogStreams",
        "logs:GetLogEvents",
        "organizations:DescribeAccount",
        "organizations:ListAccountsForParent",
        "organizations:ListOrganizationalUnitsForParent",
        "organizations:ListRoots",
        "organizations:ListTagsForResource",
        "rds:Describe*",
        "rds:ListTagsForResource",
        "redshift:DescribeClusters",
        "redshift:DescribeReservedNodes",
        "route53:ListHostedZones",
        "route53:ListResourceRecordSets",
        "route53:ListTagsForResource",
        "route53domains:ListDomains",
        "s3:GetAccessPointPolicyStatus",
        "s3:GetBucketAcl",
        "s3:GetBucketLocation",
        "s3:GetBucketPolicyStatus",
        "s3:GetBucketPublicAccessBlock",
        "s3:GetBucketTagging",
        "s3:GetEncryptionConfiguration",
        "s3:ListAccessPoints",
        "s3:ListAllMyBuckets",
        "sns:GetTopicAttributes",
        "sns:ListTagsForResource",
        "sns:ListTopics",
        "sqs:GetQueueAttributes",
        "sqs:ListQueues",
        "sqs:ListQueueTags"
      ],
      "Resource": "*"
    }
  ]
}

AWS Tags

Organizations that use AWS tags can retrieve tags associated with each cloud account within AWS. Discovered tags are located under the Vendor Custom Fields field.

---

Amazon S3 Fields and Access Control

Device42 can discover information on the following S3 fields:

• Has Public Access Point
• Has Public ACL
• Has Public Policy
• Public ACLs Blocked
• Public Policies Blocked

A bucket can be public if it has Public Access Point, Public ACL or Public Policy. The Public ACLs Blocked and Public Policies Blocked flags can be independently controlled with settings in the AWS S3 console. When these flags are checked off, a user with the correct permissions can access the bucket.

Additional resources:

• See S3 Bucket policy examples for more details.
• Visit the block public access settings section for S3 access control options.

Add and Edit AWS Roles

Device42 includes an editor to define or edit the AWS Roles displayed for Amazon AWS cloud autodiscovery jobs. Follow the steps below to view and add AWS Roles:

• Select Resources > Secrets > AWS Roles from the main menu.

• Use the AWS Role dropdown to select a role to display or click Advanced to construct more specific searches. See Advanced Search Feature for instructions.

• Click Create at the top right to add a new role.

• Enter a Name for the role.
• Enter the AWS Role label and an optional AWS Role Description.
• In the Account ID and External ID section, click + Add New.
• Add the Account ID and External ID. Click the eye icon to show or hide the field. Click the trash can icon to remove the entries.
• Click Save to save the role.

Device42 adds the new AWS Role to the roles list; it will also appear in the Available AWS Roles list when you create or edit an Amazon AWS cloud autodiscovery job.

The following steps are required if you are looking to leverage the AWS switch (Assume) Role on the API calls to scan other AWS accounts

From the Main Account:

• Create a role within IAM using the Another AWS Account option.

• Supply an account that uses Account ID and an account that uses the Require External ID option. There isn't a requirement for MFA option at this time.

• Use the example IAM policy needed for discovery.

From the sub- (or separate) account:

Assign a user to a role that has already been granted access as described in Grant Access to the Role.

Setting Up Dynamic Account Discovery Roles

You can discover all sub-accounts and add them to the discovery process using Dynamic Account Discovery.

There are prerequisite steps we'll cover before getting into how to setting up the roles:

1. Create a role with no accounts associated with it.
2. Use the role in the discovery with the key pair associated with it.

Option 1:

• The key pair user must be deployed into the organization’s root account.
• This user's policy must have the following rights, at minimum:
  • sts:assumerole
  • organizations:listaccounts
• A role must be added to all accounts where discovery is desired, with the same role name used in every account where discovery is desired.
• The Device42 discovery policy must be granted to the role.
• For role config within Device42, do not add any accounts to the role.
• At this time, we cannot use dynamic account discovery to discover roles that use external ID values.

Option 2:

If you don't want to follow the steps above, you can either:

• Make the assumable role available in the main account. Dynamic discovery will pull it in if no accounts are listed, or if the main account is included in the manually added list of IDs.
• Or also attach the Device42 discovery policy to the user directly. Select the Discover main account box on the job.

Using AWS Roles To Discover Accounts Within Discovery Jobs

AWS Cloud Discovery Jobs can use AWS roles to discover accounts. When the job includes the AWS role, the discovery job will dynamically grab multiple accounts from AWS. We previously aimed to maintain a 1:1 relationship between roles and accounts. Now, a single role can discover multiple accounts. This enables AWS users to set up discovery and specify the precise account to create, or leave the account empty to have the discovery job create cloud accounts as a result of the discovery.

Cloud Instance Statuses

All VMs in Device42 have their statuses normalized into one of three buckets: Running, Stopped, or Deleted. This allows for consistency regardless of which hypervisor or cloud is used.

Device42's statuses aren't mapped to the naming convention of the cloud provider. For example, in AWS, VM states include Running, Stopped, and Terminated. In Device42, the Terminated state is mapped to Deleted.

Setting Up Environment Credentials using EC2 Instance Profiles

You can perform AWS discovery without supplying any form of long-term, programmatic credentials by leveraging Instance Profiles or IAM roles for Amazon EC2 instances.

When Use Environment Credentials is enabled, the discovery job can be saved without selecting an Access Key ID or Secret Key in the job configuration. This is only possible when using an AWS-hosted Main Appliance or Remote Collector for discovery as it relies on internal AWS mechanisms.

Configuration

1. Deploy a Main Appliance or Remote Collector within AWS.

2. Create a new IAM Policy:

   On the IAM Policy creation screen, click JSON in the policy editor and copy paste one of the policies below based on your desired discovery configuration:

   • Option 1: For single account Discovery, refer to the example discovery policy above.

   • Option 2: Role Assumption Using Static Account Discovery

     This option is good if you have a need to specify External IDs when assuming roles, as Dynamic Account Discovery does not support role assumption using External IDs.

     Example IAM Policy

     Click to expand code example

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "sts:assumerole"
                 ],
                 "Resource": [
                     "*"
                 ]
             }
         ]
     }

   • Option 3: Role Assumption Using Dynamic Account Discovery

     This option is good if you want to discover resources in all member accounts without the need to specify individual Account IDs.

     Note: This requires the associated Remote Collector or Main Appliance to be deployed within the organization's root (management) account.

     See: Setting Up Dynamic Account Discovery Roles for more details on configuring Dynamic Account Discovery.

     Example IAM Policy

     Click to expand code example

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "sts:assumerole",
                     "organizations:listaccounts"
                 ],
                 "Resource": [
                     "*"
                 ]
             }
         ]
     }

   When you've confirmed you have the appropriate permission set selected, click Next, give the policy a name and description, and click Create Policy.

3. Create a new IAM Role

   • On the IAM Role creation screen, select AWS service as the trusted entity type and EC2 as the service or use case. Click Next.

   • On the add permissions screen, search for and select the policy created in Step 2. Click Next, give the role a name and description, and then click Create Role.

     If you want to do the role preparation via the AWS CLI and not within the AWS Management Console, you can reference the trust policy below:

     Example Trust Policy

     Click to expand code example

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "sts:AssumeRole"
                 ],
                 "Principal": {
                     "Service": [
                         "ec2.amazonaws.com"
                     ]
                 }
             }
         ]
     }

4. Attach the role:

   • From the EC2 Instances list page, select the EC2 instance created in Step 1. Then click Actions > Security > Modify IAM role.
   • From the "Modify IAM role" page, select the IAM Role created in Step 3 and then click Update IAM role.

5. Additional Member Account Configuration for Role Assumption Using Static or Dynamic Account Discovery

   If you're configuring Single Account Discovery, then there are no remaining steps to be done. If you've opted instead for Role Assumption using static or dynamic account discovery, then continue following the steps below:

   • Create the discovery policy in each member account to be discovered. Follow Step 2 again, but this time, use the example discovery policy above.

   • Create the discovery role in each member account to be discovered.

     Follow Step 3 again, but this time, select Custom trust policy instead of AWS service. Copy and paste the trust policy below. At the add permissions screen, search for and select the discovery policy created in the previous step.

     Example Trust Policy

     Click to expand code example

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Principal": {
                     "AWS": "arn:aws:iam::ROOT_ACCOUNT_ID:role/EC2_D42_RC_ROLE"
                 },
                 "Action": "sts:AssumeRole"
             }
         ]
     }

     Replace ROOT_ACCOUNT_ID and EC2_D42_RC_ROLE with your own values.

     • ROOT_ACCOUNT_ID: This is the root account ID where the role, which was configured in Step 3, resides.
     • EC2_D42_RC_ROLE: This is the name of the role in the root account to establish trust with.