Azure Integration: Data Collection

To send, retrieve, and display data of your Azure entities in ControlUp, you must establish communication between ControlUp and Azure. The only component in the ControlUp environment that needs to contact the Azure API is the ControlUp Data Collector.

From version 9.0 release, the cost report API enhances the data collection of Azure network costs. These more granular details allow ControlUp to provide greater visibility into hidden Azure costs in machine records.

When you create an Azure connection in ControlUp, it is mandatory to set up at least one data collector. For more information on how to set up a data collector for the Azure connection, see Getting Started with Azure in ControlUp.

Data Collection Intervals

Azure data is collected at different intervals, which can be configured by setting the registry key below. Each data collection group has its own standard data collection interval:

• Group A: 30 seconds

• Group B: 60 seconds

• Group C: 300 seconds

• Group D: 14400 seconds ^{New in 9.0}

• Group E: 86400 seconds ^{New in 9.0}

The following table shows the different groups and allowed intervals you can configure. Changing the default data collection intervals is not recommended. Learn more in the Considerations for Modifying Data Collection.

Cost Collection

Azure costs are currently collected every 24 hours by default and can't be customized.

Changing the Collection Interval

As shown in the table above, you can modify the data collection interval for different Azure entities. In the following example, we show you how to decrease the interval for virtual machines from 60 seconds (default value) to 30 seconds.

To change the collection interval:

1. Open the registry on the data collector and add Azure Data Collection to the following registry key:

   HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection

2. Create a REG_SZ value and name it DataCollectionInterval(sec)

3. Since we want to modify the data collection interval for virtual machines, change the second comma-separated value for Group A, B, C, D, or E to 30,30,300,14400,86400

   

4. Restart the ControlUp Agent on the data collector to apply the changes.

Considerations for Modifying Collection Intervals

In large environments, lowering the interval may cause an interruption in data transmission. Because ControlUp needs to send more Read commands to Azure's API, Azure's throttling limits are reached faster. It is best practice not to decrease the pre-defined intervals. If your organization's policy requires more frequent updates of Azure metrics, be careful when lowering the numbers as there is no way to show the throttling counter at any given point in time.

Throttling Limits

By default, an Azure data collector can execute up to 15 queries within each 5 second interval without exceeding the throttling limits.

Timeout Handling

In some cases, either Azure does not respond to ControlUp queries in a timely manner, or there is an internet connectivity issue on the data collector machine. In any connectivity case, timeout handling plays an important role. A timeout signals ControlUp to cancel the recent query and not to wait for an answer from Azure. In case there is no response to a query, the ControlUp Data Collector will try to contact the Azure API three times until an error is thrown.

Changing the Timeout

You can tune the timeout value by setting a specific registry key according to the timeout query type. In the following examples, we show you how to change the query timeout from 60 seconds (default value) to accept higher query waiting times.

To change the costs timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it CostsTimeout

2. Change the value data as needed (“300” in example below) and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

   

3. Restart the ControlUp Agent on the data collector to apply the changes.

If you can't see the cost of an Azure subscription, try this workaround.

To change the Geo Regions timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it GeoRegionTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the health timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it HealthTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the resource groups timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it ResourceGroupsTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the tenants timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it TenantTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the virtual machine and disk timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it VirtualMachineAndDiskTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the virtual machine and scale sets timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it VirtualMachineScaleSetsTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the workspaces timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it WorkspacesTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.

To change the SKU details timeout:

1. Under the registry key HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it WorkspacesTimeout

2. Change the value data as needed and set the base as Decimal. You can use any timeout between 10 and 3600 seconds.

3. Restart the ControlUp Agent on the data collector to apply the changes.