Network is a heart of every system no matter where it is. If you happen to study OSI model, you would know how imperative it is to your system. Within a web application you write, for instance, before an HTTP request initiates, the network must be established first, then the HTTP request can hit to the application at Layer 7. With that in mind, when building a system on the cloud, we must protect the network.
There are several methodologies to getting started with protecting virtual network. In this article, I’d like to introduce Defense In Depth which is one of common security countermeasures to protect digital assets in a system. Like the title, this article gives you essential knowledge of defense in depth approach to defending your Azure virtual network.
Microsoft Azure Security team wrote a very helpful and comprehensive article about Azure network security best practice here. However, some of you might not understand why you should do what. My article would be helpful to complementing to the Microsoft article one.
Overview of Defense In Depth
Defense in depth is a military defensive strategy to secure a critical position using multiple defensive perimeter. One of the most common things to explain defense in depth is the medieval castle. The king is considered the main target by attackers. In the medieval castle, there are many defensive perimeters. The medieval castle was chosen to be on the hilltop that is always easy to defend. Stone walls and terraces are built around the castle and the hill. Surrounding the hilltop is moat. Rocks are prepared inside the castle to ready to drop from the outer walls to the attack. There are other protective components including warriors to protect the king.
If you are a movie lover of U.S. serial film, you must know Prison Break produced by Fox Broadcasting Company. In the movie, we know some of the very popular solid prisons designed with many perimeters to prevent prisoners from escaping. Sentries are positioned in many places to monitor everything coming in and out the prison.
Defense in depth in security is not too different from what we have seen in the two samples above. In a simplest explanation, defense in depth is often referred as a security strategy to implement multiple defensive perimeters. There are two objectives you might not realize:
- Discourage the attack: with multi-layer defensive perimeter, you would frustrate the attacker. If the target (e.g. your data) is not worth, the attack would move on. The attacker would see many obstacles to overcome before getting access to his target. It can take a long time and amount of effort he uses. At least we would say this strategy demotivates the attacker.
- Slow down the attack: look at medieval castle, to fully overcome the complex architecture to reach to the king, the attacker is supposed to pass every perimeter. For each perimeter, he needs more time for the next one. The defense in depth strategy is to steal more time of an attacker.
Defense in depth, as described, targets to defending in multiple places because an attacker can attack to any single point if possible. With multiple places to be protected, it is practically to resist all possible attack methods. In defense in depth strategy, you need to defend at the minimum perimeters as follows:
- Enclave boundary
- Computing environment
Network is the transmission of packet between your virtual machines and other components to communicate. The OSI (Open Systems Interconnection) model would tell exactly why we need to control the network layer. While you cannot do anything with the Physical and Data Link layer in the OSI model, referred to the share responsibility between you and Microsoft, network is the first perimeter to protect. Without network layer, your virtual machine cannot talk to each other. Your application will not be able to receive network packet to identify which protocol and port it needs before further execution. If network is compromised, it is very dangerous to entirely system because at such, attacker can take down your network at a minimum which would result to business impact.
While you cannot do anything with the Physical and Data Link layer in the OSI model, referenced to the shared responsibility, network is the first perimeter to protect from attack.
When it comes to network defense, demilitarized zone (DMZ) is thought of first. What is so-called demilitarized zone? Is it a very sensitive military zone you should not step into?
In the field of security, DMZ is a separate zone which is not associated to a private or trusted network. It simply stands alone to isolate from your private network to untrusted network. It is difficult to measure the level of trust. Untrusted network is the one which you have very low trust. Where? The answer is Internet. Why? You do not know exactly who has access to your system because when exposing to the Internet (e.g. internet-facing website), the access is considered anonymously. Internet is not the only untrusted network. DMZ can be used in the case you do not want to expose your private network to any other network.
Let’s bring an example in which your web front-end server needs to call to an API to query up-to-date stock data from an internet-facing website. Without defense in depth in mind, the design is straightforward. The web front-end server can directly talk to the Internet to query data and web visitor can send a request directly to the web front-end server.
You are not wrong with this design. However, this design has a security threat. An attacker from the Internet has chances to attack to your web front-end server. If he successfully gets in, the other servers would be potentially compromised by his local attack inside the same private network.
When we consider defense in depth, we should not allow private network to directly communicate with the Internet although the web front-end server needs to have Internet-bound traffic to the stock API. A practical approach is to prepare an external network which separates from the private network. In the external network, we place another server whose capability to forward HTTP request or synchronize data to the web front-end server. With this design below, the web front-end server is no longer responsible for querying stock data. Instead, the Internet-facing server is.
Why do we think this design a DMZ practice? Private network should only be private by its name, as always. When considering private, only authorized and monitored access are allowed. If you let your web front-end server go to the Internet, how do you control anonymous access to the server? The internet-facing server in the DMZ design is an extra layer to communicate with the stock API.
The internet-facing server in the above design is often a firewall facing with the Internet. In fundamental security courses, you are taught that DMZ is a network segment between two firewalls. One faces with the Internet and another one faces with the private network.
Can Azure Virtual Network provide the ability to build a DMZ architecture? It is absolutely. To build a DMZ model, you can combine with several virtual network features in Azure such as Network Security Group, User-Defined Routing or even a network network virtual appliance hosted on a internet-facing virtual machine.
Isolating your private network from untrusted network seems not to be enough in the defense in depth strategy. It is because the isolation is just the external defensive perimeter to anonymous access. Even it is a bad assumption, let’s say if the external perimeter is breached and an attacker gets into your web front-end server. From here, he can locally exploit other servers in the same private network. He can get through the database server easier than ever. He also has high chance to control the entirely system in your private network.
Network segmentation is a good practice to mitigate such a threat. It is also an approach to building an internal defensive perimeter. Network segmentation is to segment network into subnet. This approach can help prevent packet sniffing or ARP spoofing technique in your network. Subnet can be based on functional role in your system.
The illustration above shows you an example of network segmentation with two functional roles: front-end and database. In Microsoft Azure, you have to create a virtual network first. The virtual machine will then need to be assigned to this virtual network before the subnet is assigned. There are commonly the following roles you often build on Microsoft Azure:
- Active Directory
- Web front-end
There may be more depending on what you call. For example, if you are working on SharePoint, you may have Search role, Distributed Cache role or so on. Creating separate subnet can also help you simplify control network inbound and outbound by using Azure Network Security Group.
Stateful Packet Firewall
Stateful packet firewall uses the same basic tenets of packet filtering firewall but is more sophisticated. It is working at Layer 4 of the OSI model. Sometimes it is referred to five-tuple firewall that require five types of information for the execution:
- The source address
- The source port
- The destination address
- The destination port
- The protocol
In Microsoft Azure, Azure Network Security Group works similar to a simple five-tuple stateful packet filtering. Five-tuple firewall might be considered as a backward technology but this is indispensable in implementing DMZ and part of the defense in depth strategy.
The illustration below shows you typical network traffic flow represented by green arrows. From the Internet, inbound network traffic is allowed through port 80. You can also remotely connect to an internet-facing virtual machine using Remote Desktop Connection on port 3389. The problem seen here is the allowed inbound network to the database virtual machine. Do we really need to allow inbound on port 1433 from the Internet? Similar question to the port 3389 is also come out. This design opens the door for attackers to try to listen on both port 1433 and 3389 which are often weak.
In Microsoft Azure, you are allowed to create Azure Network Security Group and associate whether to a subnet or a virtual machine’s NIC (network interface card).
The illustration below shows you the SharePoint reference architecture on Microsoft Azure to simplify your understanding of Azure Network Security. From the Internet, network packet is sent through the load balancer, listening on port 80 (HTTP) and 443 (HTTPS). The Network Security Group of the DMZ subnet allows only port 80 and 443 with TCP. In the DMZ zone, there are two virtual machines running Windows Server 2012 IIS (Internet Information Services) with IIS Rewrite URL module installed to rewrite allowed HTTP/HTTPS request from load balancer to the actual web front-end virtual machines in the Web Subnet. If the request is Search query for example, then SharePoint web services calls Search service application proxy to trigger to call to the Search service application connection on virtual machines in App Subnet. If there is a database request, network packet is sent to database virtual machine through port 1433 (SQL Server by default) to process the request before returning to the requestor from the Internet.
Azure Network Security Group ordering for inbound is different from outbound. For inbound, network packet is evaluated in subnet first before virtual machine’s NIC. Unlike inbound, outbound network packet is evaluated in virtual machine’s NIC first before subnet. Below is illustration of how rules in Azure Network Security Group is processed.
In Azure Network Security Group, there are two special rules Microsoft recommends you not to block:
- Virtual IP of the host node: The virtualized host IP address 220.127.116.11 must not be blocked because this IP address maps to the physical IP address of the server machine (host node) hosting your virtual machine.
- Licensing (Key Management Service): Windows images running on virtual machines must be licensed. The outbound request through port 1688 must not be blocked because it is sent to the Key Management Service host servers to process license activation.
One of the other things to note down is Deny-All Outbound rule. You may need to harden your system by applying this rule. However, be considerate using this rule because this will block your outbound request to public Microsoft Azure services. For example, if your virtual machine is installed Microsoft Antimalware extension, it needs to connect to Microsoft Antimalware system and engine to pull new updates to the virtual machine.
When applying Deny-All Outbound rule, the request to Microsoft Antimalware is blocked. Fortunately, there is a PowerShell script to automatically add all Microsoft Azure public IP addresses. This PowerShell script can be found here.
If your environment is large, you need to plan carefully and know some limitations and constraints. Azure Network Security Group default limit per subscription is 100. You can request up to 200. The number of rules per Network Security Group is 200 and can be up to 400. Network Security log is part of Network Watcher and can be read via Azure Portal, PowerShell or REST API.
You may wonder if you can apply NSG to a group of virtual machine. For example, you have a set of web front-end virtual machines then you want one NSG rule for this set. In this case, have a look at Application Security Group
Routing to Defense System
Virtual machines in the same virtual network by default can communicate with each other, even they are in different subnets. This can be done by number of system routes used by Microsoft Azure to define how IP traffic flows.
In the security context, you often need to force traffic to a network virtual appliance in which the forced traffic can be filtered, monitor, captured or inspected. In the illustration below, you can see all outbound network traffic from virtual machines in both Front-End Subnet and Back-End Subnet are routed to a Barracuda Next-generation firewall. Another real-world example is to route your outbound network traffic to a DLP (Data Loss Prevention) system for investigation. If outbound packet matching pre-defined sensitive data, the DLP system shall filter and block it.
When creating custom routes, make sure the IP Forwarding is enabled so your traffic can be forwarded to the destination successfully. You can use packet tracert command in Windows to compare before and after you implement User-Defined Routing.
Note that limitation of Azure User-Defined Routing is 265 routes per subnet.
User-Defined Routing contributes to your DDos mitigation if configured correctly. That said, if you have a strong next-generation firewall or your own one with machine learning and advanced analytics applied, route all traffics from the Internet to that intelligent firewall for inspection before traffics are forwarded to the destination.
Network Virtual Appliance
Azure Network Security and User-Defined Routing should not be enough because these features provide basic level of network security. In some cases, you will need higher level that these built-in features cannot meet. For example, you need intrusion detection and prevention (as known as IDPS) or network-based anomaly detection using machine learning algorithm. Instead of implementing your own high-end firewall which is costly, you can pick a virtual network appliance from the Azure Marketplace.
Secure Remote Connection
Normally when managing a virtual machine, an administrator uses Remote Desktop Protocol (for Windows) or SSH (for Linux) to remotely connect. The problem we have seen with these types of protocols is that attackers can use brute-force techniques to try to guess the password. As mentioned in my principle of security awareness, if password does not meet complexity level, it can be easily guessed. And you have heard of millions of pawned passwords, haven’t you? To establish a secure remote connection more than just direct remote desktop protocol, you should consider disabling public IP address (if you do not need it), then using one of the following ways:
- Point-to-site VPN
- Site-to-site VPN
Point-to-site VPN and Site-to-site VPN are Azure VPN Gateway options typically for hybrid deployment. Point-to-site VPN requires a client certificate before you can connect to your private virtual network. It is considered a multi-authentication from network layer.
The illustration above shows you the Point-to-site VPN setup to secure the remote from the administration PC to Microsoft Azure hosted system. The idea here is that the administrator needs to securely connect to the hosted Privilege Access Workstation (PAW) with VPN Point-to-site first. From PAW machine, he also needs to VPN again to the system hosted inside the private virtual network. This design concept is to make sure the edge (PAW virtual machine) is protected.
What is PAW? PAW (Privilege Access Workstation) is a term used to determine a controlled hardened administration workstation for administrator to connect remotely to the private system.
In many cases, people are unaware of securing this jump server. They consider it just a jump server without hardening. Thus, this server is easily compromised. Applying PAW recommended by Microsoft is to protect the remote workstation.
Another concern is if attacks know the VM’s public IP address, they can try to guess password with brute-force password technique. You cannot block the 3389 because it is mainly used for the mentioned protocols above. How can we deal with this matter?
Azure provides you a feature named Just in time (JIT) which allows you to control inbound traffic in a specific time. JIT allows you to specify port to be used, and time range to permit the inbound traffic.
The request to use predefined port is upon the role-based access control the requestor have. For configuration, read this article https://docs.microsoft.com/en-us/azure/security-center/security-center-just-in-time
You may wonder if there is a secure and direct connection from your on-premises datacenter to Microsoft Azure. In many cases, when security and performance are your top priorities, you would consider such a connection. Azure ExpressRoute provides you a private, high-speed and dedicated connection from your facility to Microsoft Azure datacenter. The industrial standard dynamic routing protocol (BGP) is used behind the scenes.
ExpressRoute has three connectivity models:
- CloudExchange Co-location
- Point-to-point Ethernet Connection
- Any-to-any (IPVPN) Connection
The reason why you should consider to choose ExpressRoute is not only because of performance, but the connection does not travel over the public Internet. In other words, there is no public IP address associated with ExpressRoute Gateway.
ExpressRoute is under the agreement between you and Microsoft connectivity partners. It is not available in all Azure regions. The availability depends on Microsoft’s target market and service providers.
Availability is part of the CIA (Confidentiality – Integrity – Availability) triad for decades in the IT security field. CIA triad is kind of knowledge required in industrial security certifications such as CISSP (Certified Information System Security Professional). Availability becomes a critical factor in security because if your system goes down then you do not know exactly whether it is being accessed by a bad guy. The unavailability would stop monitoring and prevention engine which allows an attacker to have more time to do something. Distributed Denial-Of-Service is the most common types of attack to take down your system.
Implementing fully available system on Microsoft Azure requires the use of many features and an approach we would call Availability-In-Depth to cover every layer we can do to make it as available as possible. These layers may include network, virtual machine, web front-end, application to database layer. Each layer has different mechanism to function, requiring different approach to availability.
I’m not not going to discuss fully availability solution in Microsoft Azure. Instead let’s focus on availability in network layer, and little bit of Azure Application Gateway which is a web application firewall plus HTTP-based load balancing.
Load balancing is a common solution to distributing network traffic among your virtual machine. Microsoft Azure provides you primarily two type of load balancing services:
- Azure Application Gateway
- Azure Load Balancer
Each load balancer has its own characteristics to perform. Azure Application Gateway is an HTTP-based load balancer which relies on HTTP protocol to distribute HTTP request to different virtual machine. Azure Application Gateway works on the layer 7 in OSI model. It’s not only an HTTP-based load balancer but also a web application firewall that provides the ability to inspect requests based on OWASP (Open Web Application Security Project) core rule set 3.0.
Azure Load Balancer works on network layer in OSI model. It can be set up in two scenarios: Internal and External. External Azure Load Balancer receives incoming request from Internet while Internal Azure Load Balancer works internally among virtual machines in the virtual network. External Azure Load Balancer is often used to distributed traffic from Internet to group of web front-end virtual machines. Internal Load Balancer is recommended mostly with your database virtual machines internally.
Integrate with SIEM
To help prevent modern threats today whose patterns are very dynamic (want to know why? read here) there would be a consideration of integrating to a Security Information and Event Management (SIEM) system. SIEM normally provides real-time analysis for logs, event data or so forth. In regards to network, logs are generated hugely which there is no out-of-the-box feature in Azure which can help for real-time analysis, visualization or reporting. Hence, there would be a need to integrate with your own SIEM.
Network logs are stored in Azure Blob storage which is readable via PowerShell or REST API. The key concern you may have is how to handle large network logs and build a streaming pipeline to the SIEM destination. Fortunately, in Azure you can use Event Hub and Stream Analytics to get the work done. Below is reference diagram for IEM integration using Event Hub, Stream Analytics with IBM QRadar.
Stream Analytics will read to Azure storage where NSG logs are stored then stream data to Event Hub. From IBM Qradar, you need to install required agents to work with Azure Event Hub. This is the guide to let you get started.
If you would like to learn more about SIEM integration on Azure, read this article.
The article does not give you scientific network security. Instead, it gives a very fundamental knowledge to get started to building a defensive virtual network on Azure. Throughout the article, we’ve learned features and capabilities to help prevent network threats and attacks. Note that, this article is written to protect single virtual network. There is another article coming out soon this week to cover multi-network security model on Azure.