Azure Integration: Data Collection
    • Dark
      Light
    • PDF

    Azure Integration: Data Collection

    • Dark
      Light
    • PDF

    Article Summary

    To send, retrieve, and display data of your Azure entities in ControlUp, you must establish communication between ControlUp and Azure. The only component in the ControlUp environment that needs to contact the Azure API is the ControlUp Data Collector.

    From version 9.0 beta release, the cost report API enhances the data collection of Azure network costs. These more granular details allow ControlUp to provide greater visibility into hidden Azure costs in machine records.

    When you create an Azure connection in ControlUp, it is mandatory to set up at least one data collector. For more information on how to set up a data collector for the Azure connection, see Getting Started with Azure in ControlUp.

    Data Collection Intervals

    Azure data is collected at different intervals, which can be configured by setting the registry key below. Each data collection group has its own standard data collection interval:

    • Group A: 30 seconds

    • Group B: 60 seconds

    • Group C: 300 seconds

    • Group D: 14400 seconds New in 9.0 Beta

    • Group E: 86400 seconds New in 9.0 Beta

    The following table shows the different groups and allowed intervals you can configure. Changing the default data collection intervals is not recommended. Learn more in the Considerations for Modifying Data Collection.

    Azure Resource Type

    Default Interval (sec)

    Allowed Configurable Intervals

    Group

    Cost Queries (Annual, Forecast, Regional, Month-to-Date, Last Month)

    14400

    1200 - 14400

    D

    Geo Regions

    300

    10 - 3600

    C

    Health

    300

    10 - 3600

    C

    Machine SKU (Cost Details Report Generation)

    86400

    1500 - 86400

    E

    Metrics

    30

    10 - 3600

    A

    Resource Groups

    60

    10 - 3600

    B

    Subscriptions

    60

    10 - 3600

    B

    Tenants

    300

    10 - 3600

    C

    Usages

    300

    10 - 3600

    C

    Virtual Disks

    60

    10 - 3600

    B

    Virtual Machines

    60

    10 - 3600

    B

    Virtual Machine Scale Sets

    60

    10 - 3600

    B

    Cost Collection

    Azure costs are currently collected every 24 hours by default and can't be customized.

    Changing the Collection Interval

    As shown in the table above, you can modify the data collection interval for different Azure entities. In the following example, we show you how to decrease the interval for virtual machines from 60 seconds (default value) to 30 seconds.

    To change the collection interval:

    1. Open the registry on the data collector and add Azure Data Collection to the following path:


      HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection

    2. Create a REG_SZ value and name it DataCollectionInterval(sec)

    3. Since we want to modify the data collection interval for virtual machines, change the second comma-separated value for Group A, B, C, D, or E to 30,30,300


      image.png

    4. Restart the ControlUp Agent on the data collector to apply the changes.

    Considerations for Modifying Collection Intervals

    In large environments, lowering the interval may cause an interruption in data transmission. Because ControlUp needs to send more Read commands to Azure's API, Azure's throttling limits are reached faster. It is best practice not to decrease the pre-defined intervals. If your organization's policy requires more frequent updates of Azure metrics, be careful when lowering the numbers as there is no way to show the throttling counter at any given point in time.

    Throttling Limits

    By default, an Azure data collector can execute up to 15 queries within each 5 second interval without exceeding the throttling limits.

    Timeout Handling

    In some cases, either Azure does not respond to ControlUp queries in a timely manner, or there is an internet connectivity issue on the data collector machine. In any connectivity case, timeout handling plays an important role. A timeout signals ControlUp to cancel the recent query and not to wait for an answer from Azure. In case there is no response to a query, the ControlUp Data Collector will try to contact the Azure API three times until an error is thrown.

    Changing the Timeout

    You can tune the timeout value by setting a specific registry key. In the following example, we show you how to change the query timeout from 60 seconds (default value) to 300 seconds to accept higher query waiting times.

    To change the query timeout:

    1. Open the registry on the data collector and add Azure Data Collection to the following path:


      HKLM\SOFTWARE\Smart-X\ControlUp\Agent

    2. Create a REG_DWORD value and name it SingleQueryTimeout(sec)

    3. Change the decimal value to 300. You can use any timeout between 10 and 3600 seconds.


      image.png

    4. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the costs timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it CostsTimeout(sec)

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.


      RegCostTimeout

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    If you can't see the cost of an Azure subscription, try this workaround.

    To change the Geo Regions timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it GeoRegionTimeout

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the health timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it HealthTimeout(sec)

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.


      RegHealthTimeout

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the resource groups timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it ResourceGroupsTimeout

    2. Change the decimal value to 300. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the tenants timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it TenantTimeout

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the virtual machine and disk timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it VirtualMachineAndDiskTimeout

    2. Change the decimal value to 300. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the virtual machine and scale sets timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it VirtualMachineScaleSetsTimeout

    2. Change the decimal value to 300. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the workspaces timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it WorkspacesTimeout

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.


      RegWorkSpaces

    3. Restart the ControlUp Agent on the data collector to apply the changes.

    To change the SKU details timeout:

    1. Under the path HKLM\SOFTWARE\Smart-X\ControlUp\Agent\Azure Data Collection create a REG_DWORD value and name it WorkspacesTimeout

    2. Change the decimal value to 60. You can use any timeout between 10 and 3600 seconds.

    3. Restart the ControlUp Agent on the data collector to apply the changes.


    Was this article helpful?