Agent-based Offline Discovery

Offline Discovery with the Device42 Agent

Agentless discovery in Device42 works great for most custom use-cases. There are, however, some edge cases where discovery is desired but the network doesn’t allow communication back to the main Device42 appliance (for various reasons).Sometimes, remote collectors can’t be deployed because the target network segments are segregated or remote, while other times, policy simply doesn’t allow it.

To allow for discovery in those scenarios, we are happy to announce the availability of a utility that can process the offline discovery files that can be generated by the Device42 offline discovery agent; we’re calling this tool the Device42 “Offline Data Processing Tool”Download it in the section below titled “Obtaining the Offline Discovery Data Processing Tool / Upload Utility” (it’s required to complete offline agent-based discovery).

The new Device42 Offline Discovery Agent is available for the following platforms:

  • Windows x32/x64
  • Linux x32/x64
  • Mac
  • Solaris Sparc x64
  • OpenBSD x32/x64
  • FreeBSD x32/x64

How offline discovery works

This is a high-level summary of how offline discovery works:

  1. The discovery agent [Windows or Linux] is run in ‘-offline’ mode, which creates a log file that contains discovered data
  2. One or more sets of log files created in step #1 are uploaded to a machine that can run the offline processing utility
  3. The offline processing utility ingests the log files, processes them, and uploads discovery data to your Device42 main appliance (details on configuring and running the utility are described below)

Obtaining the offline discovery agent

If you have a Device42 appliance up and running, you already have the discovery agent! It is built-in to your Device42 instance, and can be downloaded for any of the above listed platforms by simply going to the main menu, Discovery -> Agent Based Scans. To learn about using the agent for online discovery, see the discovery agent documentation here.

  1. Begin by heading to the Device42 main menu to Discovery -> Agent Based Scans.
  2. Copy the downloaded file to the target server. On Windows, you can drag and drop, FTP, or even use a USB stick. If using Linux, one method is to use SFTP, as such:
    copy to target server
  3. On Windows, you can skip this step. On Linux, change file permissions to make the file executable (chmod +x d42_agentname):
    chmod to make executable
  4. The agent can then be executed as such, creating a variable to name your log files dynamically (see below for dynamic naming details):
    $ ./agentname -offline -sudo-password ‘password’ > $VAR1.logSee the next section, “Creating Naming Variables for your agent logs”, for details on configuring variables for your output log file names.

Obtaining the Offline Discovery Data Processing Tool / Upload Utility

When offline agent-based discovery runs, it produces Log files as output containing discovery information. This utility is required to get your agent-based offline discovery data into Device42 by uploading the discovery log files to your Device42 main appliance.

The utility supports batch upload of multiple log files per run, as well. It is a Windows-based application and is compatible with Device42 15.14.04 and above.

To download the Upload tool, visit Device42’s “Miscellaneous Tools” download page the current version of the Offline discovery data processing & upload tool. Note that this tool is REQUIRED to complete the agent-based discovery process.

Creating Naming Variables for your agent logs

Since you might want to run the discovery multiple times before processing the data, (possibly also from multiple different computers), it’s important to have a unique file name. Variables are used to define the file name, and in the example below, we use the computer name, date and time stamp to generate the unique file names for the discovered data as below:

On Windows:

C:\>set var=%computername%-%date%-%time::=%
C:\>set var=%var:.=%
C:\>set var=%var:/=-%
C:\>set var=%var: =%
C:\>echo %var%.log

C:\>d42_winagent_x64.exe -offline > %var%.log

On Linux:

$ VAR1=`hostname`-`date +%Y%m%d-%k%M%S`
$ echo $VAR1

> CentOS7-20190306-133710.log

./ -offline > $VAR1.log

Linux Pre-requisite: sudo permissions

The agent file will require sudo permissions (on Linux) to collect data. There are two options to grant these permissions:

  1. Account executing the agent is given set up with option NOPASSWD: in the sudoers file, OR
  2. Sudo password is passed to the agent as a parameter on execution.

Should you choose to configure sudo, execute the agent with the following parameters:

  1. -offline – this tells the agent not to post the data to the Device42 appliance, but to save the data locally.
  2. -sudo-password string – this passes the sudo password directly into the agent. This is not required of the user is configured with NOPASSWD in the sudoers file.
  3. > filename.log – this the name of the output file.
    • This could be a local file or on a writeable network share for easier collection.
    • We recommend using a unique name log based on hostname and timestamp for each scanned device. See the section above titled “Creating Naming Variables for your agent logs” for details.

Processing offline discovery data

Once you have these files, you can either manually transfer the files to the machine that has the offline processor installed, or you may write these files to a share that is accessible from the offline processing tool.

Collecting the offline discovery output for upload

The agent will create two output files. Your output file will be named based on the variable you selected (section above). If you followed the above instructions, The files will look be named as labeled in Step 1 below:

    1. Simply collect the two log files created by the agent for upload. The files are named HOSTNAME-DATE-TIME.log & agent_local.log

Note that the agent_local.log file contains diagnostic information about the scan, and may be disregarded in most cases.

  • The log files need to be collected on a central machine for processing. Files can be collected off each server using server management tools, or the agent can be configured to write directly to a shared drive. You only need the specific log file for the host that you created. You do *not* need to collect the agent_local.log files, as they are for diagnostics only.

Processing with the Agent Log Upload Utility

Begin by first setting up the and run the offline discovery processing tool as follows:

  1. Unzip the compressed archive that contains the utility.
  2. Browse to unzipped folder and do the following.
    1. Edit the file appsettings.json. Update fields for your instance:
      • BaseURL: Change d42_url to your instance address or IP
      • InputDir: Change c:/d42_loader to the location of your collected scan logs (Note: / not \ in path)
      • Username: Change your instance user name.
      • Secret: Change your instance secret.
    2. Make sure all the files to be processed are in the input folder as specified in the configuration file in the previous step.
  3. Run the d42_loader_winx64.exe to begin processing files. This will process the input files and once successful, will move each to the output directory.
  4. You should now see your data in Device42!