Cloud Platforms Auto-discovery

Device42 Cloud Platform Auto-discovery

Device42 supports auto-discovery of cloud instances (virtual devices) on Amazon AWS, Amazon API, Microsoft Azure, Linode, DigitalOcean, OpenStack, Google Cloud, Alibaba Cloud, Oracle Cloud, and Standalone Kubernetes. Native discovery for new cloud platforms is added according to customer needs [and please do let us know!].

Device42 will auto-discover your cloud virtual machines, databases, and storage as devices. You can then work with your cloud devices just like like you would any other device. You can define application components, store passwords, create custom keys, and so on, just as you would for any other physical or virtual device.

The Discovery menu includes the option for discovery of Cloud instances. The list page shows all of your existing cloud auto-discovery jobs. You’ll typically create one job per account. As with other types of auto-discovery jobs, cloud discovery jobs can be run immediately and/or scheduled. Click Add Cloud Autodiscovery [top right corner] to create a new job, then simply select your cloud type.

Alibaba Cloud Discovery

Alibaba Discovery Items

Click Add Cloud Autodiscovery, and then select Alibaba Cloud as the Cloud Type. Name your job, and then add your Alibaba credentials, including both Access Key ID and your Access Key Secret.

Select one or more Zones for the discovery and select options for Action for Instance not found and Device Name Format. Click Save to add the job to the list of Cloud Autodiscovery jobs. Select the job and click Run Now to run the job immediately or continue editing to schedule runs.

Amazon Web Services Auto-discovery

AWS Discovery Items

API Endpoints Accessed During AWS Discovery

Regular Discovery

• sts.amazonaws.com
• organizations.us-east-1.amazonaws.com (Only if one of any of the available features is enabled.)
• organizations:ListRoots
• organizations:ListAccountsForParent
• organizations:ListOrganizationalUnitsForParent
• organizations:DescribeAccount

• ec2._region_.amazonaws.com
• ec2: DescribeSubnets
• ec2: DescribeVolumes (only if ebs feature is enabled)
• ec2: DescribeInstances

• elasticache._region_.amazonaws.com
• elasticache:DescribeCacheClusters

• rds._region_.amazonaws.com
• rds:DescribeDBInstances

S3 – (All regions and all buckets, regardless of the region provided – only if the S3 feature is enabled.)

• *.s3._region_.amazonaws.com
• s3._region_.amazonaws.com
• *.s3.amazonaws.com
• s3.amazonaws.com
• s3:ListAllMyBuckets

Route53 – (Only if the Route53 feature is enabled.)

• route53.amazonaws.com
• route53:ListHostedZones
• route53:ListTagsForResource
• route53:ListResourceRecordSets

K8s – (Only if K8s discovery is enabled.)

• eks._region_.amazonaws.com
• eks:ListClusters
• eks:DescribeCluster

K8s cluster endpoints access per K8s RBAC setup

• /api/v1/namespaces/kube-system
• /api/v1/nodes?watch=False
• /api/v1/services?watch=False
• /apis/apps/v1/deployments?watch=False or /apis/extensions/v1beta1/deployments?watch=False depends on k8s version
• /apis/networking.k8s.io/v1beta1/ingresses?watch=False or
  /apis/extensions/v1beta1/ingresses?watch=False depends on k8s version

Example of minimum policy (except for K8s cluster endpoints, since it is controlled by K8s RBAC).

{
    "Version": "2017-11-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:ListMetrics",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:Describe*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:Describe*",
            "Resource": "*"
        },
        {
            "Action": [
                "elasticache:Describe*"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        },
        {
            "Action": [
                "rds:Describe*",
                "rds:ListTagsForResource",
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeVpcs"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Action": [
                "cloudwatch:GetMetricStatistics",
                "logs:DescribeLogStreams",
                "logs:GetLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Prerequisites

To create an AWS Autodiscovery job, you will need to:

1. 1. Prepare your AWS Account – The Minimum IAM Policy Permission/Role requirements are:
      • AmazonEc2ReadOnly
      • AmazonElastiCacheReadOnlyAccess
      • AmazonRDSReadOnlyAccess
      • AmazonS3ReadOnlyAccess
   2. Device42 utilizes your AWS Access Key and Secret Key to perform discovery; please have these handy.
   3. Choose & Enter a ‘Name’ for the job
   4. Select the Cloud Type (select ‘Amazon AWS’ from the drop-down menu).

Note: Device42 encourages customers to follow AWS best practices for managing your IAM credentials, including using strong passwords, regular password rotation, applying the principle of least privilege to users and their passwords, etc.

For more information, see the article Best Practices for Managing AWS Access Keys at https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html.

• Add your Amazon Secret Key for the account to be discovered. The procedure is the same as for the Account Access Key, but will use a Second, separate entry! Do this by clicking the magnifying glass, and then clicking ‘Add Password’ in the upper right hand corner. Enter your Secret Key in the field labeled “Password:”. The USERNAME FIELD IS NOT USED by cloud discoveries – Device42 will store the Secret Key encrypted. If desired, set up access permissions for the Secret Key.
• Choose one or more Amazon regions to search.
  

Optionally, you can also:

• Choose a vendor [note that all vendors are user-defined – Device42 does not ship with a list of vendors].
• Choose a VRF Group. If one is selected, all discovered IPs will be placed in subnets in that VRF Group. This is useful if you have duplicate IPs in your internal network.
• Choose a remote collector to run the job (ensure the chosen remote collector can reach the target network).
• Check Add tags as custom fields to add discovered tags to Device42 custom fields.
• Check Kubernetes Discovery to discover Kubernetes clusters hosted on your cloud platform.

• Check Strip domain name to have Device42 strip the discovered domain suffix (everything after the first period) from the device instance name.
• Choose a category for discovered devices [note categories are user defined]

Next, you should “Save and Continue.” You can then click ‘Run’ to run the job immediately. Or you can save it and have it run on a regular schedule.

Google Cloud Discovery

Google Cloud Discovery Items

Device42 can now discover your inventory on the Google Cloud Platform. Simply create a new job, add your credentials, and you’ll be off discovering all of your GCE VMs. Begin by creating a new Google cloud discovery job:

1) Create a new “Cloud Autodiscovery” job from the Discovery, and choose Google Cloud.

2) Browse to your Google Cloud Engine JSON keyfile. Open it in a text editor and copy the contents:

3) Paste the copied JSON in its entirety into the password field. Note that GCE discovery does not utilize the “password” field (*leave it blank*):

4) Save and run your job! Optionally, create a schedule to run it automatically!

Data discovered on the Google Cloud Platform is similar to what you might be used to on AWS EC2 instances, namely:

• Discovered Google Cloud VMs are added as virtual devices
• Cloud information is added inline in Device42 for each CI

Options for GCE are as follows:

• Select Kubernetes Discovery to discover Kubernetes clusters hosted on your cloud platform.

• Strip Domain Name: Strip domain name from discovered name (everything after the first period)
• Object category for discovered devices: Choose a category to assign to discovered devices
• Overwrite existing object categories: Select this option to overwrite any previously assigned categories with current selection

Microsoft Azure Auto-discovery

Microsoft Azure Discovery Items

Minimum Permissions for Resource Group Discovery

Microsoft Azure Discovery is similar to Amazon; you will need to:

1. Choose and enter a Name for the job.
2. Select the Cloud Type (choose Microsoft Azure from the drop-down).
3. Enter your Subscription ID.
4. That’s it! Azure will now recognize the Device42 instance as a secure client that is authorized to interact with your Azure account.

Devices > Virtual Devices will now show cloud devices as well. The Vendor shows up as the Device Host for these devices. There is a new Virtual Subtype filter on the right-hand side filter bar.

Optionally, you can also:

• Choose the vendor [note that all vendors are user-defined – Device42 does not ship with a list of vendors].
• Choose a VRF Group. If one is selected, all discovered IPs will be placed in subnets in that VRF Group. This is useful if you have duplicate IPs in your internal network.
• Choose a remote collector to run the job (ensure the chosen remote collector can reach the target network).
• Select the Device Name Format for discovered cloud instances.

• Select Kubernetes Discovery to discover Kubernetes clusters hosted on your cloud platform.

• Check Add tags as custom fields to add discovered tags to Device42 custom fields.
• Check Strip domain name to have Device42 strip the discovered domain suffix (everything after the first period) from the device instance name.
• Choose a category for discovered devices (note that categories are user-defined).

Next, you should “Save and Continue”. Then you can click ‘Run’ to run the job immediately. Or you can save it or save it and have it run on a regular schedule.

OpenStack Auto-discovery

OpenStack Discovery Items

1. When you add an OpenStack job, you will be prompted for a User, Password, and Project Name.

Minimum Permission Requirements for OpenStack Discovery are as follows:

• User / User Group should be attached to the project and have the following permissions described in policy.json:
  • “os_compute_api:os-hypervisors”
  • “os_compute_api:os-extended-server-attributes”

1. The User and Password are the ones you enter into the Openstack authentication screen…
2. When you log into Openstack, the Overview screen shows a list of projects.
3. Enter the project name you would like to access into the Device42 Project Name field.

Optionally, you can also:

• Choose the vendor. Please note that all vendors are user-defined. Device42 does not ship with a list of vendors.
• Choose a VRF Group. If you select a VRF Group, then all IPs found will be placed in subnets in that VRF Group. This is useful if you have duplicate IPs in your internal network.
• Check the “Remove unfound instances from Device42″ box. If you check this box, then each time this auto-discovery job is run, any devices that were previously created for this account but were not found by the auto-discovery job will be deleted. By checking this box, you can ensure that Device42 will remain in sync with OpenStack. If you leave it unchecked, then you may end up with Device42 Cloud Instances (cloud devices) that no longer exist in OpenStack.

Next, you can click Save and Continue. Then you can click Run Now to run the job immediately. Or you can save it and set up a schedule to run the discovery job.

Oracle Cloud Auto-discovery

Oracle Cloud Discovery Items

Click Add Cloud Discovery on the Cloud Discovery page, and then select Oracle Cloud as the Cloud Type.

Enter the following information:

• Name for the discovery job.
• User ID
• Fingerprint (if applicable)
• Key File
• Tenant ID
• Zones

You can also:

• Choose the vendor. Please note that all vendors are user-defined. Device42 does not ship with a list of vendors.
• Choose a VRF Group. If you select a VRF Group, then all IPs found will be placed in subnets in that VRF Group.
• Select a Remote Collector.

Scroll down the page to see additional options.

Click Save and Continue; then you can click Run Now to run the job immediately. Or you can save it and set up a schedule to run the Oracle discovery job.

DigitalOcean Auto-discovery

DigitalOcean Discovery Items

Click Add Cloud Discovery on the Cloud Discovery page, and then select DigitalOcean as the Cloud Type.

Enter a Token Key, and then select any other options you want for the discovery job.

Click Save and Continue; then you can click Run Now to run the job immediately. Or you can save it and set up a schedule to run the Oracle discovery job.

Amazon API Auto-discovery

Amazon API Discovery Items

Use the Cloud Type: “Amazon API” selection to discover your AWS EC2 instances via the Amazon Elastic Compute Cloud API.

When discovering your Amazon Cloud via the Amazon API, Device42 authenticates against the API URL with your AWS API Access Key and Secret Key. To create a discovery job, please ensure you have these available. You can find or generate new AWS API Access keys via the AWS Console -> UserName Menu –> “My Security Credentials”. Expand the “Access keys (access key ID and secret access key)” item, and “Create New Access Key” (or reference an existing one):

1. Begin by setting Cloud Type: ‘Amazon AWS’ via the dropdown [pictured].
2. Enter a ‘Name’ for your Amazon AWS API discovery job.
3. Enter the ‘URL’ to of the AWS API endpoint you are targeting, including the port if necessary – for URLs and other information on AWS API endpoints, reference the “Endpoints” section of the AWS API documentation.
4. Add your AWS API Key ID to the “Account ID” field, followed by the corresponding Amazon Secret Key in the “Secret Key” field for the account to be discovered:
   
   You’ll add both your API Key ID & Secret Key to Device42 as separate ‘password’ entries, and the procedure is the same as adding a new password:
   • Click the magnifying glass to bring up the credential selection screen
   • Click the ‘Add Password’ button in the upper right-hand corner
   • Enter your Account ID in the field labeled “Password:” – The USERNAME FIELD IS NOT USED by cloud discoveries!, & click “Save”

   Repeat the process & add a second entry for your Secret Key, as well. Note that Device42 stores these values encrypted; If desired, you may also set access permissions on your AWS credentials.

5. In the Region: box, enter the region you are targeting, e.g. us-east-1.
6. Set a discovery schedule if desired; Save and run your AWS API discovery!

Options for AWS API Discovery:

• Action for Instance not found: Choose how Device42 will handle the situation of an instance that was previously discovered not being found on subsequent discovery runs. Change Status will update the instance’s status, while “Delete Instance” will delete the missing instance. The best choice for you depends on how you manage your infrastructure.
• Strip Domain Name: Strips the domain name (everything after the first period) from the name as discovered before storing in Device42
• Object category for discovered devices: Choose a category to assign to discovered devices
• Overwrite existing object categories: Select this option to overwrite any previously assigned categories with the current selection

Linode Auto-discovery

Linode Discovery Items

For access to Linode, you will need your API Key. To create a Linode API Key, go to your Linode console…

Select “My Profile” and navigate to “API Keys”

From here, you can create your API Key that Device42 needs to gain access.

Optionally, you can also:

• Choose the vendor. Please note that all vendors are user-defined. Device42 does not ship with a list of vendors.
• Choose a VRF Group. If you select a VRF Group, then all IPs found will be placed in subnets in that VRF Group. This is useful if you have duplicate IPs in your internal network.
• Check the “Remove unfound instances from Device42″ box. If you check this box, then each time this auto-discovery job is run, any devices that were previously created for this account but were not found by the auto-discovery job will be deleted. By checking this box, you can ensure that Device42 will remain in sync with Linode. If you leave it unchecked, then you may end up with Device42 Cloud Instances (cloud devices) that no longer exist in Linode.

Next, you should “Save and Continue”. Then you can click ‘Run’ to run the job immediately. Or you can save it or save it and have it run on a regular schedule.

Standalone Kubernetes Auto-Discovery

Standalone Kubernetes Discovery Items

To add a Standalone Kubernetes discovery job, you’ll need either a Bearer Token or Basic Credentials. You’ll also need to enter a URL and select an Action for Container not found.

Optionally, you can also choose a Vendor and a VRF Group. Please note that all Vendors and VRF Groups are user-defined.

Note: Kubernetes Discovery is also available as an option for Amazon AWS, Google Cloud, and Microsoft Azure cloud autodiscovery jobs. Scroll down the Add Cloud Discovery page and select the Kubernetes Discovery option.

Viewing Cloud Instance Information

Select Devices > Virtual Devices to view your cloud instances as virtual devices.

The Device view and edit pages will now show “Cloud Instance Information” under the Properties tab.