Adv. Load Balancing

Flawless Application Delivery

Trainer Intro

James Tacker

Technology Consultant & Content Developer

Previous Training Work:

Sauce Labs
New Relic
Salesforce
Atlassian

james.tacker@servicerocket.com

Prerequisites/Expectations

Sysadmin, DevOps, Solution Architect
Completed NGINX Core
Some familiarity with Linux
Text Editor: Vim, Vi, Emacs etc.
Solid understanding of Network topologies

The Training Environment

AWS EC2 Instances
Ubuntu
NGINX Plus
Wordpress
Tomcat 7

Log Into VM

If you haven't done so already, please take the time to SSH into your EC2 Instances (Windows users use PuTTY).

Check your email for the login credentials, check your spam folder!


                        ssh student<number>@<ec2-server-hostname>

Course Administration

Course Duration: 4 hours
Ask questions at any time!

Agenda

Load Balancing Review

Module Objectives

This module reviews the following topics:

Configuration Overview
Selection Algorithms
Hardware Migration for F5/Netscaler
Extended Status module for monitoring

Value of NGINX in DevOps Chain

NGINX is an open source reverse proxy server. For those who don’t know, a reverse proxy server is a proxy server, sitting behind a firewall in a private network, that directs client requests to appropriate backend servers.

Common uses cases for a reverse proxy server are: Load Balancer, or as I like to call it a web traffic cop, that sits in front of something like apache and distributes client request across a group of servers based on the load they’re handling. HTTP Cache, reverse proxy servers can cache incoming common requests which will speed up the flow of traffic between your clients and servers by reducing the amount of redundant tasks that your backend servers need to manage. Web Server: NGINX can also take over the responsibilities of a web server and host websites that are accessible by the internet.

Load Balancing Components

Selection Algorithm
```
upstream
```
```
proxy_pass
```
```
health_check
```

Load Balancing Configuration


				upstream myServers {
    server localhost:8080;
    server localhost:8081;
    server localhost:8082;
}

server {
    listen 80;
    root /usr/share/nginx/html;

    location / {
        proxy_pass http://myServers;
    }
}

Selection Algorithms

weighted-round-robin (default)
```
ip_hash
```
&
```
hash
```
```
least_conn
```
```
least_time
```

Lab 1.1: Configure a New Upstream

Create a configuration file called
```
main.conf
```
in
```
/etc/nginx/conf.d
```
with a
```
server
```
that listens on
```
80
```
Add three servers in the
```
upstream
```
block (ask your instructor for the backend urls)
Create a
```
location
```
prefix to
```
proxy_pass
```
to your
```
upstream
```
group.
Define an
```
error_log
```
with a level of
```
info
```
and an
```
access_log
```
with a level of
```
combined
```
Save and reload NGINX
Test in a local browser (refresh multiple times)
Read the
```
access_log
```
to see destination of request

Migrating from Hardware

No need to "rip and replace"
Can work in parallel with legacy hardware
Terminology Differences

NSIP, SNIP, Self-Ip etc.
VIP, Management IP, Virtual Servers
Monitor, High Availability, iRules, CLI

Migrating from F5

F5 BIG-IP LTM	NGINX+
Self-IP address	N/A NGINX uses underlying OS networking
Management IP addresses and port	Linux host IP (primary interface)
Virtual Server	server and location
Pool and node list	upstream
iRules	server , location , NGINX Lua, or nginScript modules
High Availability	nginx-ha-keepalived

Documentation: Migrating Layer 7 Logic

Self‑IP address – The primary interface that listens to incoming client‑side data plane traffic on a specific VLAN. It is a specific IP address or subnet on a specific NIC associated with that VLAN or a VLAN group.
In NGINX Plus, self IP addresses most directly map to the primary host interface used by NGINX Plus to manage traffic‑plane application data. Self IP addresses aren't a necessary concept in an NGINX Plus. NGINX Plus uses the underlying OS networking for management and data‑traffic control.
Management IP address and port – The IP address:port combinations on a BIG‑IP LTM appliance are used to administer it via the GUI and/or whatever remote SSH access. The NGINX Plus equivalent is the Linux host IP address, typically the primary host interface. If you need to separate remote access from the application traffic, it is possible to use separate IP addresses and/or NICs for management access to the Linux host where NGINX Plus is running,
Virtual server – The IP address:port combination used by BIG‑IP LTM as the public destination IP address for the load‑balanced applications. This is the IP‑address portion of the virtual server that is associated with the domain name of a frontend application and the port that’s associated with the service (such as port 80 for HTTP applications). This address handles client requests and shifts from the primary device to the secondary device in the case of a failover.
Virtual servers in NGINX Plus are configured using a server block. The listen directive in the server block specifies the IP address and port for client traffic.
Pool and node list – A pool is a collection of backend nodes, each hosting the same application or service, across which incoming connections are load balanced. Pools are assigned to virtual servers so BIG‑IP LTM knows which backend applications to use when a new request comes into a virtual server. The term "node list" refers to an array of distinct services that all use the same traffic protocol and are hosted on the same IP address, but listen on different port numbers (for example, three HTTP services at 192.168.10.10:8100, 192.169.10.10:8200, and 192.168.10.10:8300).
NGINX Plus flattens these concepts with upstream configuration blocks, which also define the load‑balancing and session‑persistence method for the virtual server. NGINX Plus doesn't need node lists, because the standard upstream block configuration can accommodate multiple services on the same IP address.
iRules – iRules is an event‑driven, content‑switching, and traffic‑manipulation engine (based on TCL) BIG‑IP LTM uses to control all aspects of data‑plane traffic. Becuase they are event-driven they fire for each new connection when certain criteria are met, such as when a new HTTP request is made to a virtual server or when a server sends a response to a client.
NGINX Plus natively handles this sort of content switching and HTTP session manipulation, which eliminates the need to explicitly migrate most context‑based iRules and those which deal with HTTP transactions such as header manipulation. Most context‑based iRules can be translated to server and location blocks, and more complex iRules that cannot be duplicated with NGINX Plus directives and configuration block can be implemented with the NGINX Lua or nginScript modules. For more information on translating iRules to NGINX Plus content rules, click on the doc link at the bottom of the page. (see Migrating Layer 7 Logic from F5 iRules and Citrix Policies to NGINX and NGINX Plus on the NGINX blog.)
High availability – Conceptually, BIG‑IP LTM and NGINX Plus handle high availability (HA) in the same way. Each instance of NGINX Plus can function as an active or passive instance, and when the active instance goes down the passive instance takes over the virtual server addresses (thus becoming the active instance). With NGINX Plus, a separate software package called nginx‑ha‑keepalived handles the virtual server and failover process for a pair of NGINX Plus servers. BIG‑IP LTM uses a built‑in HA mechanism and each active‑passive pair shares a floating “virtual” IP address which maps to the currently active instance.
Active‑active configurations are also possible, both on‑premises with the nginx‑ha‑keepalived package and on the Google Cloud Platform. For more information on configuring the nginx‑ha‑keepalived package and other load‑balancing architectural models, see the NGINX Plus Admin Guide.

Converting F5 Configurations


				# create pool test_pool members add { 10.10.10.10:80 10.10.10.20:80 }
# create virtual test_virtual { destination 192.168.10.10:80 pool test_pool source-address-translation { type automap } ip-protocol tcp profiles add { http } }
# save sys config


				upstream test_pool {
    server 10.10.10.10:80;
    server 10.10.10.20:80;
}

server {
    listen 192.168.10.10:80;

    location / {
        proxy_pass http://test_pool;
    }
    ...
}

Converting F5 SSL Offload


				# create pool ssl_test_pool members add { 10.10.10.10:443 10.10.10.20:443 } 
# create virtual test_ssl_virtual { destination 192.168.10.10:443 pool ssl_test_pool source-address-translation { type automap } ip-protocol tcp profiles add { http } }
# save /sys config
# create profile client-ssl test_ssl_client_profile cert test.crt key test.key
# modify virtual test_ssl_virtual profiles add { test_ssl_client_profile }
# save /sys config
# create profile server-ssl test_ssl_server_profile cert test.crt key test.key
# modify virtual test_ssl_virtual profiles add { test_ssl_server_profile }
# save /sys config


				upstream ssl_test_pool {
    server 10.10.10.10:443;
    server 10.10.10.20:443;
}

server {
    listen 192.168.10.10:443 ssl;
    ssl_certificate     /etc/nginx/ssl/test.crt;
    ssl_certificate_key /etc/nginx/ssl/test.key;
    location / {
        proxy_pass http://ssl_test_pool;
    }
}

iRule Translations

Request Redirect:
```
return
```
Request Rewrite:
```
rewrite
```
Response Rewrite:
```
sub_filter
```
Searching Files:
```
try_files
```

Request Redirect


			#F5 iRule 
when HTTP_REQUEST { 
	HTTP::redirect "https://[getfield [HTTP::host] ":" 1][HTTP::uri]" 
}
----------------------------------------------------------------------
#NGINX
location / { 
	return 301 https://$host$request_uri; 
}

Request Rewrite


			#F5 iRule
when HTTP_REQUEST { 
	if {[string tolower [HTTP::uri]] matches_regex {^/music/([a-z]+)/([a-z]+)/?$} }  { 
		set myuri [string tolower [HTTP::uri]] 
		HTTP::uri [regsub {^/music/([a-z]+)/([a-z]+)/?$} $myuri "/mp3/\\1-\\2.mp3"] 
	}
 }
 -------------------------------------------------------------------------------

#NGINX
location ~*^/music/[a-z]+/[a-z]+/?$ { 
	rewrite ^/music/([a-z]+)/([a-z]+)/?$ /mp3/$1-$2.mp3 break; 
	proxy_pass http://music_backend; 
}

Response Rewrite


			#F5 iRule	
when HTTP_RESPONSE { 
	if  {[HTTP::header value Content-Type] contains "text"} { 
		STREAM::expression {@/mp3/@/music/@} 
		STREAM::enable 
	}
 }
--------------------------------------------------------------

#NGINX
location / { 
	sub_filter '/mp3/' '/music/'; 
	proxy_pass http://default_backend; 
}

Migrating from NetScaler

NetScaler	NGINX+
NetScaler IP (NSIP)	NGINX+ host IP
Subnet IP (SNIP)	NGINX+ host IP
Virtual IP (VIP)	Same Concept
Virtual Servers	server , server_name , and location
Server, Service, Service Group	upstream
High Availability	nginx-ha-keepalived

Documentation: Migrating Layer 7 Logic

NetScaler IP address (NSIP) – Management IP address of a specific NetScaler appliance. The NGINX Plus equivalent is the host IP address of the NGINX Plus instance.
Subnet IP address (SNIP) – The source (client) IP address seen by backend servers in the load balancing configuration. By default, the NGINX Plus equivalent is the host IP address of the NGINX Plus instance as well.
Both NGINX Plus and NetScaler use their routing table to choose the best IP address to use. With NetScaler, you manage the routing table in the NetScaler CLI or GUI, but with NGINX Plus you need to edit the system‑level routing for your Linux or FreeBSD OS. You can also use the proxy_bind directive in the NGINX Plus configuration to specify the source address used for a specific application.
Virtual IP address (VIP) – Address advertised to clients for the service provided by the backend servers. The VIP functions in the same way for both NetScaler and NGINX Plus: it shifts from the primary device or instance to the secondary in the case of a failover.
Virtual Servers -- NetScaler uses only the combination of IP address and port to select the virtual server for a request. If you want to consider information in the Host header as well, you use AppExpert policies or a Content Switch Virtual Server to inspect it.
In contrast, the NGINX Plus definition of a virtual server in a server block can include both IP address‑port combinations and values in the Host header. Include the server_name directive to specify the values to match in the Host header. The list of parameters to the server_name directive can include multiple hostnames, wildcards, and regular expressions. You can include multiple server_name directives and multiple listening IP address‑port combinations within one NGINX Plus server block.
Server, Service, Service Group -- It is straightforward to migrate the NetScaler entities that make up Services and Service Groups to NGINX Plus. NetScaler uses three major entity types:
- Server – IP address or hostname of a specific backend server
- Service – Association of a server entity with a listening port and monitor
- Service Group – Association of a pool of server entities and listening ports with a monitor
NGINX Plus uses the upstream block to represent a pool of backend (upstream) application servers. In the most basic configuration, a server directive for each server specifies its IP address or hostname.

For more information about how to migrate specific logic, check the article link at the bottom of the page

Converting NetScaler Configurations


				add lb vserver myvserver HTTP 10.0.0.99 80


			server {
    listen 10.0.0.99:80;
    server_name .example.com;
    ...
}

Converting NetScaler Service Group Entities


				add serviceGroup myapp HTTP
bind serviceGroup myapp 10.0.0.100 80
bind serviceGroup myapp 10.0.0.101 80
bind serviceGroup myapp 10.0.0.102 80


			upstream myapp {
    server 10.0.0.100:80;
    server 10.0.0.101:80;
    server 10.0.0.102:80;
}

Documentation: Converting NetScaler

Deployment Scenario 1

NGINX does ALL Load Balancing

Clients connect directly to NGINX Plus which then acts as a reverse proxy, load balancing requests to pools of backend servers. This scenario has the benefit of simplicity with just one platform to manage, and can be the end result of a migration process that starts with one of the other deployment scenarios we will discuss next.

If SSL/TLS is being used, NGINX Plus can offload SSL/TLS processing from the backend servers. This not only frees up resources on the backend servers, but centralizing SSL/TLS processing also increases SSL/TLS session reuse. Creating SSL/TLS sessions is the most CPU‑intensive part of SSL/TLS processing, so increasing session reuse can have a major positive impact on performance.

Both NGINX and NGINX Plus can be used as a cache for static and dynamic content, and NGINX Plus adds the ability to purge items from the cache, especially useful for dynamic content.

NGINX Plus offers additional ADC functions, such as application health checks, session persistence, response rate limiting, bandwidth limiting, connection limiting, and more.

To support high availability (HA) in this scenario requires clustering of the NGINX Plus instances.

Deployment Scenario 2

NGINX Works in Parallel with Legacy Hardware

Deployment Scenario 3

NGINX Sits behind Legacy Hardware

NGINX Plus is added to an environment with a legacy hardware‑based load balancer, but here it sits behind the legacy load balancer. Clients connect to The hardware‑based load balancer accepts client requests and load balances them to a pool of NGINX Plus instances, which load balance them across the group of actual backend servers. In this scenario NGINX Plus performs all Layer‑7 application load balancing and caching.

Because the NGINX Plus instances are being load balanced by the hardware load balancer, HA can be achieved by having the hardware load balancer do health checks on the NGINX Plus instances and stop sending traffic to instances that are down.

disadvantages: There can be multiple reasons for deploying NGINX Plus in this way. One is because of corporate structure. In a multi‑tenant environment where many internal application teams share a device or set of devices, the hardware load balancers are often owned and managed by the network team. The application teams probably would like access to the load balancers to add application‑specific logic, but the complexity of true multi‑tenancy means that even sophisticated solutions can still not provide complete isolation between one application and another. If the application teams were given free access to shared devices, one team might make configuration changes that negatively impact other teams.

To avoid the potential problems, the network team often retains sole control over the hardware load balancers. The application teams have to submit requests to make any configuration changes. In addition, because of the potential for configuration conflicts between teams, the network team is likely to limit which advanced ADC features are exposed, meaning that application teams can’t take advantage of all the functionally available on the hardware load balancer.

One solution to this problem is to deploy a set of smaller load balancers, such as NGINX Plus, so that each application team can have its own. Completely isolated from one another, the application teams can each take full advantage of all the features they need without risking negative consequences for other teams. It’s not cost effective to give each application team a set of hardware appliances, so this a great use case for a software‑based load balancer like NGINX Plus.

The hardware load balancers remain in place, still owned and managed by the network team, but they no longer have to deal with complex multi‑tenant issues or application logic; their only job is to get the requests to the right NGINX Plus instances where the application logic resides, and NGINX Plus routes the requests to the right backend servers. This provides the network team with the control they need while also enabling the application teams to take full advantage of the ADC functionality.

Searching For Files

if

directive is bad practice

try_files

directive is a better choice

if

Directive

Can cause NGINX to SIGSEGV
Essentially creates a nested
```
location
```
block that has to run on every request
Only 100% safe use cases:
- ```
return...;
```
- ```
rewrite ... last/permanent;
```


				if ($request_method = POST ) { 
    return 405; 
}
---------------------------------------------------

if ($args ~ post=140){ 
	rewrite ^ http://example.com/ permanent; 
}

With the If Directive the specified nested condition is evaluated. If it’s true, the directives specified inside those braces are executed, and that initial request is assigned to the configuration details inside the if directive.

It’s worth mentioning here that configurations inside the if directives are inherited from the previous configuration level, so you can basically thinkg of an if directive as a nested location

This is where the problems begin, if the 'if' directive has problems when used in a location context, in some cases it doesn’t do what you expect but rather something completely different instead.

In some cases it even segfaults. (i.e. segmentation fault (aka segfault)

This is common condition with systems under significant load and will cause programs to crash; they are often associated with a file named core . Segfaults are caused by a program trying to read or write an illegal memory location) It’s generally a good idea to avoid this if possible.

There are cases where you simply cannot avoid using an if, for example, if you need to test a variable which has no equivalent directive, like in this example which tests the condition of a Post request and returns a internal redirect code, or if an argument equals a certain post value and then the url can be rewritten as a permanent redirect.

It is important to note that the behaviour of if is not inconsistent, given two identical requests it will not randomly fail on one and work on the other, with proper testing and understanding ifs ‘’‘can’‘’ be used. It’s recommend to thoroughly read through the two links posted in this slide but if you’re in doubt one should use the return, rewrite, or try_files directives instead.

What we should do instead is use the try_files directive

try_files

Directive

NGINX checks for the existence of files and/or directories in order
Commonly uses the
```
$uri
```
variable
If no file or directory exists, NGINX performs an
```
internal
```
redirect


location / { 
	try_files $uri $uri/ @proxy; 
}

location @proxy { 
	proxy_pass http://backend/index.php;
}

Like the return and rewrite directives, the try_files directive is placed in a server or locationblock. As parameters, it takes a list of one or more files and directories and a final URI:

try_files file … uri;

NGINX checks for the existence of the files and directories in order (constructing the full path to each file from the settings of the root and alias directives), and serves the first one it finds. To indicate a directory, add a slash at the end of the element name. If none of the files or directories exist, NGINX performs an internal redirect to the URI defined by the final element (uri).

For the try_files directive to work, you also need to define a location block that captures the internal redirect, as shown in the following example. The final element can be a named location, indicated by an initial at‑sign (@).

The try_files directive commonly uses the $uri variable, which represents the part of the URL after the domain name.

In the following example, NGINX serves a default GIF file if the file requested by the client doesn’t exist. When the client requests (for example) http://www.domain.com/images/image1.gif, NGINX first looks for image1.gif in the local directory specified by the root or alias directive that applies to the location (not shown in the snippet). If image1.gif doesn’t exist, NGINX looks for image1.gif/, and if that doesn’t exist, it redirects to /images/default.gif. That value exactly matches the second location directive, so processing stops and NGINX serves that file and marks it to be cached for 30 seconds.

error_page

Directive

Create and reference custom error pages
Best practices:
- Set
```
root
```
  for
```
error_page
```
- Separate messages for each code or range


	error_page 404 /404.html;
location = /404.html {
    root /usr/share/nginx/html;
}

error_page 500 502 503 504 /50x.html;
location /50x.html {
    root /usr/share/nginx/html;
}

TCP/UDP Load Balancing

Module Objectives

This module enables you to:

Explore L7 and L4 differences with NGINX Plus
Differentiate between
```
stream
```
and
```
http
```
context
Configure logging for TCP/UDP upstream
Create Active Health Checks for
```
stream
```
context

http

vs.

stream

http

Parses
```
http
```
request
L7 Layer

Header injection
Location routing
SSL termination

stream

Raw IP packets
L3/L4 Layer

Pass SSL certs
Lower overhead
Network visibility

Layer 7: operates at the high-level application layer, which deals with the actual content of each message. HTTP is the predominant Layer 7 protocol for website traffic on the Internet. Layer 7 load balancers route network traffic in a much more sophisticated way than Layer 4 load balancers, particularly applicable to TCP-based traffic such as HTTP. A Layer 7 load balancer terminates the network traffic and reads the message within. It can make a load-balancing decision based on the content of the message (the URL or cookie, for example). It then makes a new TCP connection to the selected upstream server (or reuses an existing one, by means of HTTP keepalives) and writes the request to the server.

Layer 4: operates at the intermediate transport layer, which deals with delivery of messages with no regard to the content of the messages. Transmission Control Protocol (TCP) is the Layer 4 protocol for Hypertext Transfer Protocol (HTTP) traffic on the Internet. Layer 4 load balancers simply forward network packets to and from the upstream server without inspecting the content of the packets. They can make limited routing decisions by inspecting the first few packets in the TCP stream.

stream

Context

Key Differences

```
proxy_pass
```
relegated to
```
server
```
context
Active
```
health_checks
```
work differently than
```
http
```
load balancer
IP Transparency,
```
proxy_protocol
```
, and Direct Server Return (DSR) instead of
```
proxy_set_header
```
Logging only available with verison r11 or higher

IP Transparency

The Problem

Retain source IP during a TCP (or HTTP) reverse proxy to an application server

The Solution

proxy_bind

directive +

transparent

paramerter


	stream {		
    server {
        listen 3306;

        location / {
            proxy_bind $remote_addr transparent;
            proxy_pass http://mysql_db_upstream;
        }
    }
}

The intention of IP Transparency is to conceal the presence of the reverse proxy so that the origin server observes that the IP packets originate from the client’s IP address. IP Transparency can be used with TCP‑based and UDP‑based protocols.

In short here you're spoofing the source address of upstream traffic by including the transparent parameter to the proxy_bind directive. Most commonly, you set the source address to that of the remote client:

Now the challenge here is to correctly handle the conneciton details, in other words you need to ensure that response (egress) traffic to the remote client is correctly handled. The response traffic must be routed to the NGINX Plus reverse proxy, and NGINX Plus must terminate the TCP connection. NGINX Plus then sends the response traffic to the remote client using the client TCP connection:

IP Transparency Diagram

Documentation: IP Transparency

Documentation: Masquerading Traffic

So looking at it a little deeper, The IP Transparency mode of operation uses the TPROXY kernel module, which is a standard feature of most modern Linux kernels. You need to make several configuration changes:

On the NGINX Plus server, configure the worker processes to run as root, so that they can bind upstream sockets to arbitrary addresses. In the main (top‑level) context in /etc/nginx/nginx.conf, we can include the user directive to set the NGINX Plus worker processes’ ID to root:
On the NGINX Plus server, ensure that each connection originates from the remote client address (Step 2 in the diagram). Add the proxy_bind directive with the transparent parameter to the configuration for the virtual server:
And on the upstream servers we have to configure the routing so that all return traffic is forwarded to NGINX.
Remove any pre‑existing default routes, and check that the routing table looks sensible:
If your upstream servers need to be able to connect to external servers, you need to configure the new NGINX Plus gateway to forward and masquerade traffic
On the NGINX Plus server, we also need to configure the TPROXY kernel module to capture the return packets from the upstream servers and deliver them to NGINX Plus (Step 5 in the diagram). In the example in the documentation, we run the iptables and ip rule commands to capture all TCP traffic on port 80 from the servers represented by an IP range

As you can gather based on this diagram, IP transparency is very complicated, so I recommend setting up a test project to configure this setup, and also make sure you have control over the upstream servers so you can congifure the necessary ip routing capabilities you may need

proxy_protocol

Directive

Allows NGINX to accept client information via
```
proxy_protocol
```
from proxy servers/load balancers
Examples origin services:
- HAProxy
- Amazon ELB
- GCE Active LB


		stream {
    server {
        listen 12345;
        proxy_pass example.com:12345;
        proxy_protocol on;
    }
}

Documentation: Proxy Protocol Guide

proxy_protocol

Example


	log_format combined '$proxy_protocol_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';	

    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;

        set_real_ip_from 192.168.1.0/24;
        real_ip_header proxy_protocol;

	    proxy_set_header X-Real-IP       $proxy_protocol_addr;
	    proxy_set_header X-Forwarded-For $proxy_protocol_addr;
    }
}

The example assumes that there is a load balancer in front of NGINX, for example, Amazon ELB that balances all incoming HTTPS traffic. NGINX accepts the HTTPS traffic on port 443 (listen 443 ssl;), and accepts the client’s IP address passed from the load balancer via the PROXY protocol as well (the proxy_protocol parameter). The load balancer address is specified in the set_real_ip_from directive. The client’s IP address is passed in the proxy_protocol parameter from the real_ip_header directive.

NGINX terminates the HTTPS traffic (the ssl_certificate and ssl_certificate_key directives) and proxies the decrypted data to a backend server (proxy_pass http://backend1;) including the client’s IP address and port (the values of the proxy_set_header directives).

Execution Steps

The proxy_protocol_addr variable specified in the log_format directive also passes the client’s IP address to the log.
Configure NGINX to accept the PROXY protocol headers. Add the proxy_protocol parameter to the listen directive:
In the set_real_ip_from directive, specify the IP address or the CIDR range of addresses of the TCP proxy or load balancer:
In the real_ip_header directive, add the proxy_protocol parameter that will keep the client’s IP address and port number:
Pass the client IP address from NGINX to an upstream server with the proxy_set_header directive and the $proxy_protocol_addr variable:
Add the $proxy_protocol_addr variable to the log_format directive on the http level:

DSR

Responses (return packets) bypass Load Balancer
Takes load off of load balancer
```
health_checks
```
no longer work
Requires further configuration (iptables, Router configuration etc.)


		server {
    listen 53 udp;

    proxy_bind $remote_addr:$remote_port transparent;
    proxy_responses 0;
    # proxy_timeout 1s;
}

Documentation: DSR Guide

Direct Server Return (DSR) is an extension of the IP Transparency concept. In DSR, the upstream server receives packets that appear to originate from the remote client, and responds directly to the remote client. The return packets bypass the load balancer completely.

DSR can deliver a small performance benefit because it reduces the load on the load balancer, but it does carry a number of limitations:

The load balancer never sees the return packets, so it cannot detect whether the upstream server is responding or has failed.
The load balancer cannot inspect a request beyond the first packet before selecting an upstream, so its ability to make load‑balancing decisions (content‑based routing) is very limited.
The load balancer cannot participate in any form of negotiation or stateful processing, such as SSL/TLS.
Most other application delivery controller (ADC) features are not possible with DSR, such as caching, HTTP multiplexing, and logging.

How does it differ from IP transparency?

NGINX Plus must spoof both the remote client IP address and port when sending datagrams to upstream servers (proxy_bind port configuration).
NGINX Plus must not be configured to expect response datagrams from upstream servers (the proxy_responses 0 directive).
An additional step is necessary to rewrite the source address of the return datagrams to match the public address of the load balancer.

SSL Server Name Routing

The
```
preread
```
feature can inspect incoming SSL/TLS and determine target
Can also use the
```
map
```
to determine complex routing method


		stream {
    server {
        listen 443;
        ssl_preread on;
        proxy_pass $ssl_preread_server_name;
    }
}

You can now use NGINX Plus’ TCP/UDP load balancer to load balance SSL/TLS connections without decrypting them. This is useful in a secure or high‑traffic environment where you want to forward SSL/TLS‑encrypted connections to a remote server. With the new SSL server name preread feature, NGINX Plus R11 can inspect each incoming SSL/TLS connection and determine the target domain (such as the Server Name Indication [SNI] value) to which to route the connection.

The SSL server name is provided in the new $ssl_preread_server_name variable. It contains the name of the target host as extracted from the SNI field of the SSL/TLS handshake.

You can use the variable as the argument to the proxy_pass directive or as a field in the virtual server access log. Note that to enable this feature you must include the ssl_preread directive in the configuration, as shown in this example:

Logging for

stream

Use
```
access_log
```
to inspect data rates, protocols, error conditions, etc.
Only available in r11


			log_format tcp_log '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received' '$upstream_session_time $upstream_addr $proxy_protocol_addr’;

TCP/UDP Considerations

Access Control Limits

```
allow
```
,
```
deny
```
```
proxy_download_rate
```
,
```
proxy_upload_rate
```
```
limit_conn
```
,
```
limit_zone
```

Use
```
slow-start
```
to prevent overload
Use maintenance parameters to handle failover, updates, migrations etc.
- ```
drain
```
- ```
backup
```
- ```
down
```

Lab 3.1: Create TCP Upstream

In the
```
tcp
```
directory, create/open
```
lb.conf
```
Create a
```
server
```
that listens on port
```
90
```
and proxies to
```
tcp_backend
```


		    	stream {
    upstream tcp_backend {
        zone tcp_upstream 64k;
        server backend1:8080;
        server backend2:8080;
        server backend3:8080;
    }

    server {
        listen 90;
        proxy_pass tcp_backend;
    }
}

Lab 3.2: Create a UDP Upstream

Create a
```
server
```
that listens on
```
53
```
, and append the
```
udp
```
parameter
Use a
```
proxy_pass
```
to proxy to a new
```
upstream udp_backend
```


 upstream udp_backend {
        zone udp_upstream 64k;
        server ec-2:53;
        server ec-2:53;
        server ec-2:53;
    }

    server {
        listen 53 udp;
        proxy_pass udp_backend;
    }

TCP/UDP Health Checks

Passive health check use
```
health_check
```
Active health check use parameters:
- ```
interval
```
  ,
```
passes
```
  ,
```
fails
```
Sophisticated health check use
```
match
```
block
- ```
send
```
  : text string or hexidecimals
- ```
expect
```
  : literal string or regex data response

Lab 3.3: TCP Health Check

Configure a passive
```
health_check
```
for
```
udp
```
and
```
tcp
```
upstreams
Test using
```
status.html
```
Create a
```
match
```
block the uses a
```
GET
```
request to confirm TCP connection


 match http {
    send "GET / HTTP/1.0\r\nHost: localhost:8080\r\n\r\n";
    expect ~* "200 OK";
}

    server {
        listen 90;
        health_check interval=10 passes=5 fails=5 match=http;
        proxy_pass tcp_backend;
    }

COMPLETE EXAMPLE


stream {

    #Logging only supported in NGINX Version 1.11.5 or higher
    log_format basic " '$remote_addr [$time_local]' '$protocol $status $bytes_sent $bytes_received''$session_time";

    upstream tcp_backend {
        zone tcp_upstream 64k;
        server server:8080;
        server server:8080;
        server server:8080;
    }

    upstream udp_backend {
        zone udp_upstream 64k;
        server server:53;
        server server:53;
        server server:53;
    }

    match http {
        send "GET / HTTP/1.0\r\nHost: localhost:8080\r\n\r\n";
        expect ~* "200 OK";
     }

    server {
        listen 90;
        access_log /var/log/nginx/nginx_tcp_access.log basic buffer=32k;
        error_log /var/log/nginx/nginx_tcp_error.log info;
        health_check interval=10 passes=5 fails=5 match=http;
        proxy_pass tcp_backend;
    }

    server {
        listen 53;
        proxy_pass udp_backend;
    }
}

MySQL Load Balancing

Configure load balancer and make a SQL query to confirm behavior
Listening port must use MySQL server port (default
```
3306
```
)


 stream { 
	upstream db { 
		server db1:3306; 
		server db2:3306; 
		server db3:3306; }
	server { 
		listen 3306; 
		proxy_pass db; 
	}
}

Avoding Parrallel DB Updates

Failover:
```
db2
```
acts as a
```
backup
```
and
```
db1
```
receives connections to replicate across other nodes
Silent Partner:
```
db3
```
is a silent partner to
```
db1
```
and
```
db2
```
Failure Detection:
```
proxy_connect_timeout
```
set to low value (
```
1
```
second or less) to catch early failures


upstream db {
	server db1:3306; 
	server db2:3306 backup; 
	server db3:3306 down; 
}

server {
	listen 3306;	
	proxy_pass db;
	proxy_connect_timeout 1s;
}

High Availability

Module Objectives

This module enables you to:

Explore
```
nginx-ha-keepalived
```
solution
Understand syntax and basics of VRRP
Enable session affinity to override load balancer
Explore and demo various cloud solutions

High Availability

On Prem

```
keepalived
```

Uses Virtual Router Redundancy Protocol (VRRP)

Cloud Solutions

Google Cloud Compute
Floating IPs
AWS:
- Elastic IP
- ELB
- Route 53
- Lambda

The keepalived open source project provides the keepalive daemon for Linux servers, Use the Virtual Router Redundancy Protocol (VRRP) to manage virtual routers (virtual IP addresses), and a health check facility to determine whether a service (for example, a web server, PHP back end, or database server) is up and operational. If a service on a node fails the configured number of health checks, keepalived reassigns the virtual IP address from the master (active) node to the backup (passive) node.

VRRP also ensures that there is a master node at all times. The backup node listens for VRRP specified packets from the master node. If it does not receive an specified packet for a period longer than three times the configured specified interval, the backup node takes over as master and assigns the configured virtual IP addresses to itself.

nginx-ha-keepalived

Separate daemon from NGINX

manages shared virtual IPs
designates master NGINX node
Sends VRRP advertisement messages

VRRP

health check facility to determine service availability
requires 3 consective advertisments from
```
keepalived
```
Basic active-passive setup

keepalived

Configuration

Node values
- ```
unicast_src_ip
```
- ```
unicast_peer
```
```
priority
```
```
notify
```
```
vrrp_instance
```


		global_defs {
    vrrp_version 3
}

vrrp_script chk_manual_failover {
    script   "/usr/libexec/keepalived/nginx-ha-manual-failover"
    interval 10
    weight   50

vrrp_script chk_nginx_service {
    script   "/usr/libexec/keepalived/nginx-ha-check"
    interval 3
    weight   50
}

vrrp_instance VI_1 {
    interface                  eth0
    priority                   101
    virtual_router_id          51
    advert_int                 1
    accept
    garp_master_refresh        5
    garp_master_refresh_repeat 1
    unicast_src_ip             192.168.100.100

    unicast_peer {
        192.168.100.101
    }

    virtual_ipaddress {
        192.168.100.150
    }

    track_script {
        chk_nginx_service
        chk_manual_failover
    }

    notify "/usr/libexec/keepalived/nginx-ha-notify"
}

Describing the entire configuration is beyond the scope of this class, but a few items are worth noting:

Each node in the HA setup needs its own copy of the configuration file, with values for the priority, unicast_src_ip, and unicast_peer directives that are appropriate to the node’s role (master or backup).
unicast_src_ip is the IP of the interface keepalived listens on
unicast_peer is the IP of the peer instance
The priority directive controls which host becomes the master, explained in the next slide
The notify directive names the notification script included in the distribution, which can be used to generate syslog messages (or other notifications) when a state transition or fault occurs.
The value 51 for the virtual_router_id directive in the vrrp_instance VI_1 block is a sample value; change it as necessary to be unique in your environment. If you have multiple pairs of Keepalived instances (or other VRRP instances) running in your local network, create a vrrp_instance block for each one, with a unique name (like VI_1 in the sample) and virtual_router_id number.

Defining Mastership

No fencing mechanism

```
chk_nginx_service
```
```
weight
```
```
interval
```
```
rise
```
```
fall
```


		vrrp_script chk_manual_failover {
    script   "/usr/libexec/keepalived
        /nginx-ha-manual-failover"
    interval 10
    weight   50

vrrp_script chk_nginx_service {
    script   "/usr/libexec/keepalived
        /nginx-ha-check"
    interval 3
    weight   50
}

Note:

script

path should be on one line

There is no fencing mechanism in Keepalived. If the two nodes in a pair are not aware of each other, each assumes it is the master and assigns the virtual IP address to itself. To prevent this situation, the configuration file defines a script-execution mechanism called chk_nginx_service that runs a script regularly to check whether NGINX Plus is operational, and adjusts the local node’s priority based on the script’s return code. Code 0 (zero) indicates correct operation, and code 1 (or any nonzero code) indicates an error.

In the sample configuration of the script, the weight directive is set to 50, which means that when the check script succeeds (returning code 0):

The priority of the first node (which has a base priority of 101) is set to 151.
The priority of the second node (which has a base priority of 100) is set to 150.
The first node has higher priority (151 in this case) and becomes master.

The interval directive specifies how often the check script executes, in seconds (3 seconds in the sample configuration) file. Note that the check fails if the timeout is reached (by default, the timeout is the same as the check interval).

The rise and fall directives (not used in the sample configuration file) specify how many times the script must succeed or fail before action is taken.

The nginx-ha-check script provided with the nginx-ha-keepalived package checks if NGINX is up. We recommend creating additional scripts as appropriate for your local setup.

Active-Passive with keepalived

Install the package and run the setup:


$ apt-get install nginx-ha-keepalived
$ nginx-ha-setup

Configure the nginx-ha-check script


vrrp_script chk_nginx_service {
	 script "/usr/libexec/keepalived/nginx-ha-check" 
	interval 3 
	weight 50 
}

Documentation:

Adding More VIPs

virtual_ipaddress

block replicates

ip

utility


virtual_ipaddress {
    192.168.100.150
    192.168.100.200
}

Troubleshooting keepalived


		Feb 27 14:42:04 centos7-1 systemd: Starting LVS and VRRP High Availability Monitor...
Feb 27 14:42:04 Keepalived [19242]: Starting Keepalived v1.2.15 (02/26,2015)
Feb 27 14:42:04 Keepalived [19243]: Starting VRRP child process, pid=19244
Feb 27 14:42:04 Keepalived_vrrp [19244]: Registering Kernel netlink reflector
Feb 27 14:42:04 Keepalived_vrrp [19244]: Registering Kernel netlink command channel
Feb 27 14:42:04 Keepalived_vrrp [19244]: Registering gratuitous ARP shared channel
Feb 27 14:42:05 systemd: Started LVS and VRRP High Availability Monitor.
Feb 27 14:42:05 Keepalived_vrrp [19244]: Opening file '/etc/keepalived/keepalived.conf '.
Feb 27 14:42:05 Keepalived_vrrp [19244]: Truncating auth_pass to 8 characters
Feb 27 14:42:05 Keepalived_vrrp [19244]: Configuration is using: 64631 Bytes
Feb 27 14:42:05 Keepalived_vrrp [19244]: Using LinkWatch kernel netlink reflector...
Feb 27 14:42:05 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) Entering BACKUP STATE
Feb 27 14:42:05 Keepalived_vrrp [19244]: VRRP sockpool: [ifindex(2), proto(112), unicast(1), fd(14,15)]
Feb 27 14:42:05 nginx -ha-keepalived: Transition to state 'BACKUP ' on VRRP instance 'VI_1 '.
Feb 27 14:42:05 Keepalived_vrrp [19244]: VRRP_Script(chk_nginx_service) succeeded
Feb 27 14:42:06 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) forcing a new MASTER election
Feb 27 14:42:06 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) forcing a new MASTER election
Feb 27 14:42:07 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) Transition to MASTER STATE
Feb 27 14:42:08 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) Entering MASTER STATE
Feb 27 14:42:08 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) setting protocol VIPs.
Feb 27 14:42:08 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.100.150
Feb 27 14:42:08 nginx -ha-keepalived: Transition to state 'MASTER ' on VRRP instance 'VI_1 '.
Feb 27 14:42:13 Keepalived_vrrp [19244]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.100.150

Active-Active


		vrrp_script chk_nginx_service {
    script  "/usr/lib/keepalived/nginx-ha-check"
    interval 3
    weight   50
}

vrrp_instance VI_1 {
    interface         eth0
    state             BACKUP
    priority          101
    virtual_router_id 51
    advert_int        1
    accept
    unicast_src_ip    192.168.10.10

    unicast_peer {
        192.168.10.11
    }

    virtual_ipaddress {
        192.168.10.100
    }

    track_script {
        chk_nginx_service
    }

    notify "/usr/lib/keepalived/nginx-ha-notify"
}

vrrp_instance VI_2 {
    interface         eth0
    state             BACKUP
    priority          100
    virtual_router_id 61
    advert_int        1
    accept
    unicast_src_ip    192.168.10.10

    unicast_peer {
        192.168.10.11
    }

    virtual_ipaddress {
        192.168.10.101
    }

    track_script {
        chk_nginx_service
    }

    notify "/usr/lib/keepalived/nginx-ha-notify"
}

Documentation: Admin Guide

Use Cases:

DNS Resolution per service
Share IPs between services
Round Robin to map single DNS to multiple IPs
Layer 3 load-balancing device to distribute L3 traffic between IP addresses

You can run NGINX Plus in an “active-active” setup, where two or more nodes handle traffic at the same time. Usually this is achieved by defining multiple active IP addresses, and each IP address is hosted on a single NGINX instance. Then the Keepalived configuration ensures that these IP addresses are spread across two or more active nodes.

When hosting multiple services, each service’s DNS name should resolve to one of the IP addresses. Share the IP addresses between the services. Use round-robin DNS to map a single DNS name to multiple IP addresses.

Use a L3 load-balancing device such as a datacenter edge load balancer to distribute L3 traffic between the IP addresses.

Active-active may be used to increase the capacity of your load-balanced cluster, but if a single node in the pair were fails, your capacity would be reduced by half. You can use Active-Active as a form of safety, to provide sufficient resource to absorb unexpected spikes of traffic when both nodes are active, and you can use active-active in larger clusters to provide more redundancy.

However be aware that NGINX instances in a load-balanced cluster do not share configuration or state. For best performance in an active-active scenario, make sure that connections from the same client are routed to the same active IP address, and use session persistence methods such as sticky cookie that do not rely on server-side state--which we will cover in a few slides

Cloud Solutions

Must have facility to determine mastership

Floating IPs
GCE Active LB
AWS:
- ELB, Route 53, Elastic IPs

AWS HA Deployment

Method	HA Type	Address Type
ELB	Active‑active	Dynamic; requires CNAME delegation
Route 53	Active‑active or active‑passive	Static; DNS hosted in Route 53
Elastic IPs ( keepalived )	Active-passive	Static; DNS hosted anywhere
Elastic IP w/Lambda	Active-passive	Static; DNS hosted anywhere

Elastic Load Balancer

Disadvantages

Doesn't expose static IP
Cannot map a root domain
Doesn't support UDP LB (need Route 53)

You configure ELB to load balance traffic among all NGINX Plus instances (which then load balance the traffic to your application instances). If one of the NGINX Plus instances fails, ELB detects the failure and stops sending traffic to it. To help ELB quickly detect failure of NGINX Plus instances, it is important to configure ELB health checks.

ELB provides active‑active HA, meaning that all NGINX Plus instances are in use and receive their share of traffic. Not deploying standby NGINX Plus instances reduces the overall cost.

However, ELB deployed as an HTTP load balancer does not support the HTTP/2 or WebSocket protocols. If you need to use those protocols with NGINX Plus, you have to deploy ELB as a TCP (not HTTP) load balancer. In this case, if you need to pass the client IP address from ELB to NGINX Plus, you also have to configure the PROXY protocol in both ELB and NGINX Plus.

There are several downsides to using ELB for an HA deployment of NGINX Plus:

ELB does not expose a static IP address, which is critical requirement for some applications.
ELB adds an additional tier of load balancing, which increases complexity and cost.
As ELB does not support UDP, you cannot use NGINX Plus to load balance UDP traffic.
ELB doesn’t scale quickly, so large traffic spikes can result in dropped traffic.
ELB IP addresses are published using a DNS CNAME record; you cannot map a root domain (for example, example.com) to a CNAME unless you delegate all DNS to Route 53.

Route 53

Disadvantages

Doesn't update DNS Cached/Client records

The AWS Domain Name System (DNS) service, Amazon Route 53, performs global server load balancing by responding to a DNS query from a client with the DNS record for the region that is closest to the client and hosts the domain. For best performance and predictable failover between regions, “closeness” is measured in terms of network latency rather than the actual geographic location of the client.

The way it works with NGINX+ is you create a hosted zone and add records to enable routing traffic to all NGINX Plus instances. Then you configure Route 53 health checks of NGINX Plus instances. If one of the NGINX Plus instances fails, Route 53 excludes the record associated with that instance from its responses to DNS queries. However, clients and intermediate DNS servers will cache records in accordance with the time‑to‑live (TTL) value in each record, which is set by the authoritative DNS server. The clients and/or DNS Servers do not update the record until the TTL expires, and so do not immediately notice when the authoritative server removes the record for an NGINX Plus instance from its responses. To get around this, you need to set a minimal TTL value for NGINX Plus records.

Route 53 is probably the most powerful AWS HA load balancing method and in addition to an active‑active deployment, Route 53 allows you to create an active‑passive deployment or even more complicated failover combinations, as explained in the AWS documentation.

Elastic IPs

Disadvantages

Backup instance under-utilized
Slow IP Association
Complicated deployment

Perhaps you have a reason not to rely on ELB or Route 53 to make NGINX Plus highly available. One alternative is to use AWS’ Elastic IP address feature in an active‑passive NGINX Plus deployment.

To achieve this you expose a static IP addresses for NGINX Plus and associate it with the primary (active or master) NGINX Plus instance. Then define a second NGINX Plus instance in standby or backup mode does not handle traffic, but is ready to take over when the master NGINX Plus instance fails. During the takeover (failover), the Elastic IP address is reassigned to the second instance, which is now considered the master.

The downsides of the Elastic IP address method are as follows:

The backup instance is not used most of the time, which increases the cost of your deployment.
In contrast with a typical on‑premises deployment, re‑association of a static IP address is not instant. Our tests showed that it took up to 6 seconds to re‑associate the IP address with the new master instance.
The deployment is more complicated than the ELB or Route 53 methods, as you must install and properly configure software that monitors the health of the master NGINX Plus instance and re‑associates the Elastic IP address if the master fails.

Session Affinity

For applications that require state data on backend servers

NGINX supports the following methods:

```
sticky cookie
```
```
sticky learn
```
```
sticky route
```

sticky cookie

Syntax:

sticky cookie name


	    upstream myServers {
	server backend1;
	server backend2; 
	server backend3;
	
	sticky cookie my_srv expires=1h domain=example.com path=/cart;
}

sticky learn

Syntax:

sticky cookie name


	    upstream myServers {
	server backend1; 
	server backend2; 
	server backend3;
	
	sticky learn create=$upstream_cookie_sessionid lookup=$cookie_sessionid zone=client_sessions:1m; 
}

server {
	location / {
	    proxy_pass http://myServers;
}
}

sticky learn

Part 1

sticky learn

Part 2

sticky learn

Part 3

sticky route

	
upstream myServers {
	zone backend 64k;
	
	server backend1 route=backend1; 
	server backend2 route=backend2; 
	server backend3 route=backend3;

	sticky route $route_cookie $route_uri;
}

Tomcat Example

Routing Variables


			map $cookie_JSESSIONID $route_cookie {
	~.+\.(?P<route>\w+)$ $route;
}

map $request_uri $route_uri {
	~JSESSIONID=.+\.(?P<route>\w+)$ $route;
}

Lab 4.1: Tomcat Route

Open

main.conf

. In the

http

context, create a

 log_format

called

sticky

that logs the following:


		    	log_format sticky "$request \t $status \t 
           Client: $remote_addr \t 
           Upstream IP: $upstream_addr \t 
           Route URI: $route_uri \t 
           Route Cookie: $route_cookie \t";

Change the

access_log

level to

sticky


 	access_log /var/log/nginx/main.access.log sticky;

Note to instructor, if you haven't already please spinup the following Tomcat instances that are already pre-configured with the jvmRoute parameter

Command to access AMI (assuming you have the public key)


		  		ssh -i AWS_service_rocket_key.pub ec2-user@ec2-**-***-***-**.compute-1.amazonaws.com

(Lookup the password) and Spinup the following machines:


		  		ngx-launch-class ubuntu-backend1 1
ngx-launch-class ubuntu-backend2 1
ngx-launch-class ubuntu-backend3 1

Copy the URL for each machine and give it to the students. You won't have to log into any of these machines so don't worry about the student numbers and passwords

Lab Solution


        log_format sticky  "$request \t Upstream: $upstream_addr \t Route URI: $route_uri \t Routing Cookie: $route_cookie \t All Cookies: $http_cookie \t ";

    server {
    ...
    access_log /var/log/nginx/main.access.log sticky;
    #access_log /var/log/nginx/main.access.log combined;
    ...
}

Lab 4.2: Tomcat Route

Enable
```
sticky route
```
with two variables:
```
$route_cookie $route_uri;
```

Add the

route

parameter and a shared memory zone


	zone backend 64k;
server <backend_url>:8080 route=backend1;
server <backend_url>:8080 route=backend2;
server <backend_url>:8080 route=backend3;

Add the following

maps


map $cookie_jsessionid $route_cookie { 	
	~.+\.(?P<route>\w+)$ $route; 
}

map $request_uri $route_uri { 	
	~jsessionid=.+\.(?P<route>\w+)$ $route; 
}

ask your instructor about changing the upstream backend urls!!!

Lab 4.3: Tomcat Route Test

In your shell, make the following
```
curl
```
requests:
```
curl http://<localhost>:8080
```
In a separate shell, run a
```
tail -f
```
command on your
```
upstream_access.log
```
Do you notice the IP address changing?
Open a browser, and step through the app via the following URI:
```
<localhost>/examples/servlets/servlet/SessionExample
```
Execute the application, and refresh your browser several times. What can you observe in the log now? Which IP address is the request hitting?

Solution

You can test the below URIs in a browser (recommended so you can show students the source code once you hit the SessionExample), or you can use the curl requests below.

Make sure you run a tail command to show the new log_format in a separate or tabbed shell


		  		sudo tail -f /var/log/nginx/upstream.access.log


      curl http://localhost:8080/
      curl http://localhost:8080/examples/
      curl http://localhost:8080/examples/servlets/
      curl http://localhost:8080/examples/servlets/servlet/
      curl http://localhost:8080/examples/servlets/servlet/SessionExample/

Lab 4.4: All Active GCE LB Demo

Active-Active + sticky sessions

Documentation: Admin Guide

Service Discovery and Scaling NGINX

Module Objectives

This module enables you to:

Understand Service Discovery within the context of Microservice design
Explore Zookeeper demo
Deploy NGINX using a container service
Explore
```
resolver
```
and
```
resolve
```
directives

Monolith to Microservices

Monolithic Architecture

Microservices Architecture

Service Discovery

Services need to know locations of each other
Registries work in differenty ways
Register and read information

resolver

and

resolve

```
resolver
```
will re-resolve the domain name
```
valid
```
parameter overrides frequency
```
resolve
```
queries individual servers in an
```
upstream
```


		  		resolver 10.0.0.2 valid=10s;
#example 1
server {
    location / {
        set $backend_servers backends.example.com;
        proxy_pass http://$backend_servers:8080;
    }
#example 2
upstream myServers {
	server backend1 resolve;
	server backend2 resolve;
}

The resolver directive forces NGINX to re-resolve the domain name after the TTL expires by querying the DNS server

For those who are unfamiliar or want a refresher, the TTL value tells local resolving name servers how long a record should be stored locally before a new copy of the record must be retrieved from DNS. The record storage is known as the DNS cache, and the act of storing records is called caching.

TTL is part of the Domain Name System.
TTLs are set by an authoritative nameserver for each resource record.
TTLs are used for caching purpose. For example, www.dnsknowledge.com TTL value is 86400 seconds, which is 24 hours. The higher a record’s TTL, the longer the information will be cached, and the less queries a client will have to make in order to find the domain.
TTLs will be used by the resolving name server to speed up name resolving by caching results locally.

The valid parameter overrides TTL value and forces NGINX to re-resolve based on the frequency you set

Resolver is useful for when you set your domain name to a variable and also when using a service discovery method such as zookeeper or consul

The resolve parameter acts similar to resolver just for individual servers in your upstream

resolver

Example


		http {  
    resolver 10.xxx.xxx.2 valid=30s;

server {
    set $elb "{{ lp_app_elb }}";
    location / { 
        proxy_pass http://$elb/;
    }
}

A lot of customers use NGINX to reverse proxy traffic to the app servers in a few places (For example Cache servers that also use Nginx as a reverse proxy for traffic to the backend servers and the backend servers themselves reverse proxy traffic to other application servers). Many times in a service architecture like this all these services are containerized and hosted in private clous such as AWS, and to take advantage of some of the cool features that AWS offers such as an auto-scale we will throw an Elastic Load Balancer in front of everything.

So just like a lot of people on the inter webs you may have found out that NGINX will only resolve the ELB DNS once on startup and then won't refreshing it....holy crap right? This will cause you to thinkg you've been hacked or maybe that some of our servers break because AWS changes IPs for ELBs on the fly.

The usually suggested work around is to use the NGINX resolver directive to define a DNS server for NGINX to do the name lookup on AND, more importantly, a time of validity for the DNS lookups! using the valid parameter.

We also get a lot of questions about DNS spoofing. You must define a trusted DNS infastructure if you're going to lock it down. NGINX recommends to only use Active Directory Domain Services Zones and make sure NGINX points directly to that name server IP address.

High Quality LB

Precise distribution of traffic to services
Developer Configurable

Zookeeper Demo

How the Demo Works

Zookeeper performs service discovery
Registrator registers services with Zookeeper
Web app
```
hello
```
simulates backends
NGINX+ load balances the services

GitHub: zookeeper-demo

Other Service Discovery Demos

Containerized Services

Streamlined Deployment
Relatively Secure
Infrastructure agnostic

Docker Overview

Develop apps along with components
Docker Engine

Server
REST API
CLI

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. Docker Engine is a client-server application with these major components: A server which is a type of long-running program called a daemon process (the dockerd command). A REST API which specifies interfaces that programs can use to talk to the daemon and instruct it what to do. A command line interface (CLI) client (the docker command). The CLI uses the Docker REST API to control or interact with the Docker daemon through scripting or direct CLI commands. Many other Docker applications use the underlying API and CLI. The daemon creates and manages Docker objects, such as images, containers, networks, and volumes.

Intro Docker Images

Images are comprised of multiple layers
A layer is also just another image
Every image contains a base layer
Docker uses a copy on write system
Layers are read only

Docker uses copy-on-write, which essentially means that every instance of your docker image uses the same files until one of them needs to change a file. At that point, it copies the files and creates its own version. This means that often a docker image will not need to write anything to disk to spawn its process. That makes docker fast! We're talking "100 milliseconds" fast.

Once you start a process in Docker from an Image, Docker fetches the image and its Parent Image, and repeats the process until it reaches the Base Image. Then the Union File System adds a read-write layer on top. That read-write layer, plus the information about its Parent Image and some additional information like its unique id, networking configuration, and resource limits is called a container.

Dockerfile Overview

Dockerfile when used with docker build command, automates command line instructions:


 FROM ubuntu:12.04

MAINTAINER jtack4970 version: 0.1
ADD ./mysql-setup.sh /tmp/mysql-setup.sh
RUN /bin/sh /tmp/mysql-setup.sh
EXPOSE 3306

1. Dockerfiles are always named ‘Dockerfile’ because when the docker build command kicks off it creates a context of all the files in our directory and sub-directories, build daemon will then search for a file called ‘Dockerfile’ to build the image (hopefully in that same directory where you have all the necessary files to build the images) 2. Each line in a Docker file starts with a command 1. Example: 1. FROM command tells Dockerfile on which base image to build our new image ‘from’ (best practices use alpine Linux because it’s small, has a package manager, and allows you to debug containers in prod—it’s also used by default in Docker images) 2. MAINTAINER designates the author and maintainer for this docker image 3. ADD takes files/directories from our host machine and adds them to the file system of the container at the specified location 4. EXPOSE exposes the external facing port of the container image 4. ENTRYPOINT (not listed here) sets our container as an executable, so when it starts it will run whatever app you specify

Docker Commands


$ docker pull
$ docker run
$ docker build
$ docker create
$ docker push

NGINX and Docker

Use NGINX as a containerized LB Service
Configure NGINX and build with .conf files
Pull From or Push To Dockerhub

Dockerhub

Lab 6: NGINX + Docker

Install Docker and pull NGINX Image


$ sudo apt-get install docker.io
$ sudo docker images
$ sudo docker pull nginx:1.12.0
$ sudo docker run -d nginx:1.12.0

Try installing other versions of NGINX


$ sudo docker run -d nginx:1.11.0
$ sudo docker run -d nginx:1.10.0
$ sudo docker ps

Gather IPs and Hit NGINX


$ sudo docker ps
$ sudo docker inspect <container ID>
$ curl <container ip>

Lab 6: Cleanup Containers

Stop Containers
```
$ sudo docker stop <ID>
	
```
Remove Containers
```
$ sudo docker rm <ID>
```
Tear down All containers


$ sudo docker stop $(docker ps -a -q)
$ sudo docker rm -v $(docker ps -a -q)
# -v flag removes volumes on file system

Scaling with Kubernetes

Manage Containerized Services in Cluster
Easier App Management
- Service Discovery
- Configuration Files
- Rolling Updates
- Monitoring
High Level Abstraction
- i.e. Describe apps, what they should, let Kubernetes figure out the 'how'

Deep dive into architecture

problems with containerized apps (besides the constant need to make it smaller)
- 1. Structure of the Organization guides the structure of the work (code, that they produce). In other words, you can’t build microservices with a waterfall organization…too many bottlenecks.
- 2. Requires much more automation because of the moving parts, and systems that dynamically discover, track, and monitor issues
- 3. No need for Developers to build these systems themselves, too many good options like kubernetes
Kubernetes - an abstraction that makes sense
- Manage applications at a high level (in cluster
- Manage containers at a high level (in cluster)
- Docker makes it easy to develop + deploy apps in containers, Kubernetes makes it easier to manage other problems with application management:
  - 1. App Configuration
  - 2. Service Discovery
  - 3. Managing Updates (to apps)
  - 4. Monitoring (containers i.e. apps)
Just using Docker locks us into deploying to individual machines, which limits are workflow yes, but also limits the amount at which we can scale. Kubernetes abstracts the details from the individual machine and teaches the cluster like an entire logical machine
Application becomes a first class citizen, which allows Kubernetes to mange it using high-level abstractions i.e. describe a set of apps, how they ‘should’ interact with each other, and let kubernetes figure out how to make that happen through (service discovery, container management, container monitoring)

Intro to Kubernetes

Lab 7: Launch NGINX as K8 Service (with GCE)

Grab and Set Availability Zones


$ gcloud compute zones list
$ gcloud config set compute/zone <VALUE>

Create a Cluster


$ gcloud container clusters create <my-cluster>

Launch and expose NGINX


$ kubectl run nginx --image=nginx:1.12.0
$ kubectl expose deployment nginx --port 80 --type LoadBalancer

List the Services
```
$ kubectl get services
```

WTH is a Pod?

Core of Kubernetes, represents a logical application
Represents one or more containers
- i.e. containers with a hard dependencies on each other
- Can also contain volumes (data discs that live as long as the pod does)
- Any container can use a volume (because it’s a shared volume)
Pods also share a network namespace (e.g. one IP per pod)

Creating Pods

Uses .yaml file (manifest file like DockerFile)
To create a pod use:
```
$ kubectl create -f <path/to/.yaml>
```

To display details a pod/pods use


$ kubectl get pods
$ kubectl describe pods <POD NAME>

Service Discovery with Kubernetes

Containers, especially in a cluster, have dynamic IPs. A service like Kubernetes can implement service discovery to make sure incoming traffic from Load Balancer is routed correctly

OR we can use this:


resolver kube-dns.kube-system.svc.cluster.local valid=5s;

upstream backend {
    zone upstream-backend 64k;
    server webapp-svc.default.svc.cluster.local service=_http._tcp resolve;
}

resolver – Defines the DNS server that NGINX Plus uses to periodically re‑resolve the domain name we use to identify our upstream servers In this example, We identify this DNS server by its domain name, kube‑dns.kube‑system.svc.cluster.local. The valid parameter tells NGINX Plus to send the re‑resolution request every five seconds. this domain name is resolved only when NGINX starts or reloads, and NGINX Plus uses the system DNS server or servers defined in the some other conf file to resolve it.) Because both Kubernetes DNS and NGINX Plus (R10 and later) support DNS Service (SRV) records, NGINX Plus can get the port numbers of upstream servers via DNS. We include the service parameter to have NGINX Plus request SRV records, specifying the name (_http) and the protocol (_tcp) for the ports exposed by our service. We declare those values in whatever yaml file is associated with our services

Ingress Controller

Advanced Use Case of using NGINX as an Ingress ReplicationController

GitHub Repo: NGINX-Ingress

Secure and Fast

Encryption at the transmission layer
SSL handshake slows down communication
Encryption is CPU intensive

Scaling Options

Dynamic Re-Configuration Recap

```
upstream_conf
```
```
server
```
parameters

Example:


				curl -D http://server/upstream_conf?upstream=myServers&id=0&weight=5

Sample App

Proxy Model

This model focuses entirely on in-bound traffic and ignores the whole inter process communication problem. Basically think of it as putting NGINX on a publc agent and letting the services on the private agents fend for themselves. The good thing is that:

You get all the goodness of HTTP traffic management in this system that you normally get with NGINX
SSL termination
Traffic shaping and security
Caching
With NGINX plus you get robust load-balancing and service discovery

This model works well for a simple and flat API or a monolith with some basic microservices attached. For Kubernetes we have an open source Ingress Controller that allows you to easily implement this system using our OSS or commercial version

NGINX + gives you dynamic upstreams, active health checks, and robust monitoring WAF

Router Mesh

The next model is called the router mesh, like the proxy model, it has NGINX running in front of the system to manage in bound traffic and gives you all of the benefits of the proxy model

Where it differs is in the introduction of a centralized load balancer between the services. When services need to communicate with other services, they route through the this centralized load balancer and the traffic is distributed to other instances

The Dies Router with NGINX/NGINX Plus work in this manner

Service discovery through DNS and monitoring the service event stream in the registry, but the disadvantage her is it exacerbates the performance problem by adding another hop in the network connection thus requiring another SSL handshake to make it work

So instead of a 9 step SSL handshake, you need to do an 18 step SSL handshake

Fabric Model

Normal Process

So let’s look why the fabric model is so good, but first looking at the normal process

Let’s say you have two services that need to talk to each other In this diagram the Investment manager needs to talk to the user manager to get user data

The investment manager will create a new instance of an HTTP client The client will doa DNS request to the service registry (Mesos DNS sitting on top of Zookeeper)

It will get an ip address back of the service It will then go through the 9-step SSL handshake

Once the data is transferred, it will close down the connection and garbage collect the HTTP client Service discovery is dependent on the the application to be able to query and understand DNS requests – good luck with SRV records

The load-balancing is dependent on the service registry and is typically the dumbest load balancing option, round-robin DNS Each and every request has to go through the SSL negotiation process – even if you don’t do CA authentication, it is a 4 step process at minimum.

Detail Process

Service Discovery

LB and Persistent SSL

Load Balancing

When NGINX Plus gets back the list of User Manager instances, it puts them in the load balancing pool to distribute requests

NGINX has a variety of load balancing schemes that are user definable

Least time balancing will send data to the fast responding service, whether you choose header response or full data response
If your system has to connect to a monolith or stateful system, we have connection persistence to make sure requests go back to the proper instance

Persistent SSL

But here is where the real benefit comes in – stateful, persistent connections between microservices

Remember the first diagram and how the service instance goes through the process of:

Creating an HTTP client
Negotiating the SSL connection
Making the request
Closing down the connection

Here NGINX creates a connection to the other microservices and, using keepalive connection functionality, maintains that connection across application code requests

Essentially, there are mini VPNs that are created from service to service

In our initial testing we have seen a 77% increase in connection speed

Circuit Breakers

Network Considerations

Docker Best Practices
Process Failure means Container Failure
Adding Another Layer to the Stack
Dev Team Have Too Much Power
Tooling to Make the Fabric Model, Simple to Create and Deploy

Additional Resources

Further Information

NGINX Documentation

NGINX Admin Guides

NGINX Blog

Q&A

Survey!

Sales: nginx-inquiries@nginx.com