ControlUp Architecture & Security Concepts
(Last updated: January 2018)
This document describes the ControlUp suite architecture, including different deployment topologies scenarios, a description of all major components and their communications model, data stored in the relevant ControlUp data stores and data protection mechanisms.
ControlUp supports various topologies based on the customer requirements and security policies. Below we will describe the two major topologies used by ControlUp customers. A full description of the components illustrated below is available in the next chapter.
ControlUp Hybrid Cloud Mode
ControlUp Hybrid Cloud mode is enabled by default if your network has Internet connectivity. In this mode, the ControlUp back-end components are hosted on our secured Amazon Web Services Cloud servers. while the ControlUp Console and monitor runs inside the enterprise network
The following drawing is a high-level overview of ControlUp architecture when working in a Hybrid Cloud mode:
ControlUp On-Premises Mode
ControlUp On-Premises mode enables organizations to install the ControlUp back-end components on their on-premise data-center. In this mode both the ControlUp back-end components and the ControlUp Console and monitor runs inside the enterprise network
The following drawing is a high-level overview of ControlUp architecture when working in On-Premises mode:
In this chapter we will describe the various software components that are part of the ControlUp architecture.
Customer Network Components
ControlUp Console is the main component used by sysadmins for real-time management and monitoring of their virtual infrastructure, physical and virtual servers, VDI and RDS environments. The console distributes the ControlUp Agents to the managed computers/VMs, and exposes the UI which enables admins to configure Hypervisor connections, XenDesktop sites, NetScaler Appliances, AWS regions and Monitor services.
The console maintains communication with the relevant managed computers/VMs, Hypervisors, XenDesktop sites, NetScaler appliances and AWS regions and displays real-time performance data to the sysadmin. The console also communicates with the ControlUp back-end Servers for various operations.
ControlUp Monitor Service
The ControlUp Monitor service is a key component in any ControlUp deployment. The Monitor Service carries out two major functionalities:
- Incidents reporting and alerting.
- Historical data uploads.
Once installed and started, the ControlUp Monitor logs into your ControlUp organization and connects to your managed computers/VMs,Hypervisors, XenDesktop sites, Netscaler appliances and AWS regions. The Monitor starts receiving system information and performance updates from your organization, just like an additional ControlUp Console instance. The primary difference between a ControlUp Monitor Service and a Console is the fact that the Monitor runs as a Windows service, requiring no user interaction and allowing for continuous monitoring of your resources, ensuring continuous coverage for incident alerting and reporting capabilities among others.
The ControlUp Monitor Service also uploads the historical data that is used for the construction of reports and analytics displayed in ControlUp Insights. For more information regarding the architecture and security measures for ControlUp Insights, please refer to the “ControlUp Insights – Security Measures and Procedures” document.
ControlUp Data Collector
A software component which collects performance and configuration information from the Hypervisor management web service, the XenDesktop sites, the NetScaler appliances and the AWS regions via remote API calls. By default each ControlUp Monitor Service is configured as a data collector which pulls data directly from the connected hypervisors (Esxi, AHV, Hyper-V, or XenServer), Xendesktop brokers, and NetScaler Connector
In a production deployment it is recommended to configure a dedicated data collector which acts as a proxy for all other ControlUp consoles and Monitor services:
Configuring a dedicated data collector is a best practice, please refer to the following article.
The ControlUp Data Collector communicates with the configured vCenter server via the SDK web service, the default communication channel is based on the SSL protocol (e.g. https://vcenter.fqdn/sdk). By default, data collection occurs every 20 seconds and a read-only account is sufficient to pull all configuration and performance data, in order to enable VM Power Management, go to the Virtual Machine/Interaction category and enable the following: ⧫Power Off ⧫Power On ⧫Reset
The ControlUp Data Collector communicates with the configured XenServer via port 80 (by default) and also communicates directly with each XenServer pool member to pull the real-time performance metrics via the RRD API. By default, data collection occurs every 20 seconds and a read-only account is sufficient to pull all configuration and performance data, in order to enable VM Power Management, upgrade the user role to ‘VM Operators’.
The ControlUp Data Collector communicates with the configured XenDesktop Broker via port 80 (by default) to pull configuration and performance data. During the initial connection the data collector discovers all XenDesktop Brokers and save them in the Broker's Failover List to enable data collection in-case the first XenDesktop Broker is not available. The data collector utilizes both the PowerShell and OData API’s to pull relevant data in Delivery Groups, Brokers, VDA’s, User Sessions and Published Applications.
The Read Only Administrator right to all farms that will be managed is sufficient for monitoring purposes. If you want to be able to use the built-in XenDesktop management features like enabling maintenance mode for example, then this account will require the following permissions:
- Edit Application Group Properties
- Edit Application Properties (Application Group)
- Edit Delivery Group Properties
- Edit Machine Catalog Properties
The ControlUp Data Collector communicates with AWS's .net SDK over https.
The Data collection intervals are as follows:
Metadata (computer name, state etc.) – 20 second intervals.
"Cloudwatch" data – depending whether the instance has detailed monitoring or not (as defined on the AWS instance itself). Detailed monitored instances - 1 minute intervals.
Non-Detailed monitored instances - 5 minute intervals.
The ControlUp Data Collector communicates with the NetScaler appliance over HTTP or HTTPS, depending on the user’s preference over API. The data collections intervals - Depend on the size of NetScaler deployment and are configurable.
The ControlUp agent is a software component which collects performance information on the managed computer/VM and sends it to the ControlUp Console or to the ControlUp Monitor instances which are currently running in the network and are connected to the managed computer. The ControlUp agent also executes the management actions performed by the users running the ControlUp Console.
For further information, please refer to the following article.
Cloud based Backend Components (Hybrid Cloud Mode)
ControlUp Cloud Servers
ControlUp Cloud Servers reside in the Amazon Web Services cloud (US and Ireland datacenters) and provide login, licensing, central configuration, and database services for all ControlUp Consoles and Monitor service instances running on the customer network.
All data uploaded to ControlUp Insights is loaded from Amazon S3 into Redshift, a petabyte-scale SQL data warehouse service that runs on highly optimized and fully managed AWS compute and storage resources
ControlUp Insights Cloud Servers
ControlUp Insights offers a web portal (https://insights.controlup.com) which allows authorized users to display, save, export and share reports based on the uploaded data. The web portal is hosted by Amazon Web Services using the EC2 service.
On-premise based backend components (On-Prem mode)
ControlUp On-Premises Server
The ControlUp On-Premises server is powered by a Windows server which is designed to provide the functionality of the ControlUp Cloud Server within the customer’s network. After configuring the On-Premises Server, all ControlUp consoles and Monitor service instances running in the customer’s network will connect to the On-Premises Server which provides login, licensing, central configuration, and database services for the Consoles and Monitors services..
The ControlUp On-Premises server is designed primarily for environments in which Internet connection is limited or blocked, or in which regulation does not permit organizational data to be stored outside the company’s network.
The ControlUp On-Premises Server requires the following software components:
- Windows Server 2012 R2 / 2016 R2
- Microsoft .Net Framework 3.5 AND .Net Framework 4.5 or above
- Lightweight Directory Services
- IIS services
- ControlUp Web Services hosted on IIS
- MS SQL database: 2016, 2014 R2, 2012 R2, all in Express, Standard, and Enterprise Editions. (can be deployed on existing SQL instance). SQL 2008 is NOT supported.
During the initial login phase, ControlUp consoles and Monitor services instances authenticate themselves against the On-Premises server using either the user’s account token or an explicitly provided AD username and password.
The authentication protocol between the console/monitor instances and the On-Premises server is based on Windows Authentication (HTTP 401 Challenge) over HTTPS.
ControlUp Insights On-Premises Server
The ControlUp Insights On-Premises server is a query database server which runs on the customer’s network, facilitating the use of ControlUp Insights On-Premise for analytics and reporting. The server is an internal data collector which processes and creates data models to allow advanced reporting and analytics capabilities without any external network connection. For further information, please see the detailed documentation here: https://docs.google.com/a/controlup.com/document/d/1VLLwN27dTEKUl3IL_8daNyVOIQx4hnwjab3_ceQisy4/edit?usp=sharing
ControlUp Data Stores
This chapter will describe the various data stores being used in a ControlUp deployment.
ControlUp In-RAM Database
A proprietary database used by each ControlUp Console / Monitor service instance to store all real-time performance and configuration data gathered by the data collection agents. This database is a volatile database, which exists only when the console/monitor executable is running. The Security Policy within the Console allows for proper role-based maintenance of the various features within the Console. The in-RAM data retention policy allows up to 100 historical transactions per counter.
ControlUp Configuration Database
The ControlUp central configuration is hosted on a Lightweight Directory Services database. The location of the database depends on the chosen ControlUp topology:
- ControlUp Hybrid Cloud Mode – In this mode the configuration database is stored on the ControlUp Servers running in the AWS cloud
- ControlUp On-Premises Mode – In this mode the configuration database is stored on the ControlUp On-Premises server running inside the customer network
The configuration database includes all persistent configuration objects that are part of the ControlUp organization, including the following objects (not all items are listed here):
- Distribution settings defaults
- Check Ping
- Check Prerequisites
- Default Port
- Temporary / Permanent Mode
- Keep-Connected Interval
- Hypervisor Connection settings
- XD Connection settings
- Cloud connection settings
- NetScaler connection settings
- Managed Computer /VM Information
- Install Mode (temporary / permanent)
- CPU Count
- Domain Role (Workstation / Member Server / etc)
- Highest session count (Max value of the ‘session’ counter)
- OS Caption
- OS Service Pack
- Physical (MAC) Address (used for Wake-On-Lan)
- System Type (x86 / x64 / etc)
- Total RAM
- Domain DNS
- ControlUp Port
- Netbios Name
- Last Connection Error
- Folder Information
- Delegation Information
- All entries configured by the user
- Owner information
- NT Account Name
- NT Account SID
- Role Information
- NT Account Name
- NT Account SID
- Stress Settings
- Stress settings configured by the user
- RDP Connection Properties
- Authentication Level
- Connect to console
- Device Redirection Configuration
- SmartSize settings
- Connect to console
- Start Full Screen
- Start Program on connection
- AD Connections (metadata only)
- Branch Mappings
- Trigger Settings
- Generic ControlUp configuration settings
User credentials are never stored in the configuration database.
All data objects in the configuration database are encrypted using Rijndael algorithm (AES) with a per-customer random 128 bit encryption key. In the Hybrid Cloud mode, the encryption key is stored on the ControlUp Cloud servers and protected using DPAPI. In the On-Premises mode, the data is encrypted for obfuscation purposes only, the key is stored locally, along with a secret hardcoded string.
All data objects are encrypted locally, at the console / monitor instances, before being transmitted over the network to the relevant web services / database.
ControlUp Incidents Database
The ControlUp Incidents database is hosted on a Microsoft SQL database, the location of the database depends on the chosen ControlUp topology:
- ControlUp Hybrid Cloud Mode – In this mode the Incidents database is stored on the ControlUp AWS RDS instance
- ControlUp On-Premises Mode – In this mode the Incidents database is stored on a Microsoft SQL server running on the customer network
Incident Triggers are definitions of significant events that should be recorded by ControlUp for later analysis. Each trigger includes a list of conditions which specify when the incident will be recorded and which follow-up actions will be performed at that time.
The Incidents database contains all historical incidents that were reported by the ControlUp consoles and Monitor services based on the customer triggers definitions. The information stored in the Incidents database can be viewed via the ControlUp Incidents Pane.
Sensitive Incidents related data such as host and computer names, IP addresses, account names, event message data and process command lines are encrypted using Rijndael algorithm (AES) with a per-customer random encryption key. In the Hybrid Cloud mode, the encryption key is stored on the ControlUp Cloud servers and protected using DPAPI. In the On-Premises mode, the data is encrypted for obfuscation purposes only, the key is stored locally, along with a secret hardcoded string.
All incidents are encrypted locally, at the console / monitor instances, before being transmitted over the network to the relevant web services / database.
ControlUp Local Cache
Configuration Cache files are stored on the computer running the ControlUp Console and the ControlUp Monitor service, in the Application Data directory under the user’s profile (e.g. %UserProfile%\AppData\Roaming\ControlUp). By default, NTFS restricts access to the file to allow only to the user himself and members of the local Administrators group.
If the files are copied and used by a different user, the encrypted data cannot be decrypted and the user will have to re-enter all passwords. Other configuration data will be available.