What is an Ingress?

Kubernetes cluster provides a mechanism to orchestrate containers and run microservices. But how do external services, outside the kubernetes cluster communicate with these microservices?

A Kubernetes Ingress resource can be used to make microservice(s) running inside Kubernetes externally accessible. Ingress is an API resource, it is backed by a Kubernetes CRD or API that defines the Ingress type of resource

An Ingress is the entry point to traffic inside the Kubernetes cluster. The traffic is destined to one or many services inside the cluster. Routing rules can be specified to route the traffic to different services.

An Ingress can route both HTTP and HTTP(s) type of traffic inside the cluster. Additionally a domain name can also be specified for an Ingress resource. Specifying a domain name makes it easier to access the service without specifying the IP address. This also makes the url to access the service portable since it does not change, even when the Kubernetes cluster is deployed with a different set of IPs

Several routing rules can be specified in a single Ingress resource. Say you want to provide a couple of versions of service - one for paid users and another one for free users. You can serve the service using two urls - example.com/free-access and example.com/paid-access

Since Ingress is a single point of routing traffic inside the cluster, it makes it easy to manage and troubleshoot routing issues if any. If no Ingress is used, the service may have to be exposed manually (say making service of type NodePort that gets traffic from an external load balancer) or making each of the services of type LoadBalancer (more on it later)

‍

Traffic routing patterns with Ingress

We show with an example the different patterns  for Ingress, starting from a very simple Ingress that routes all traffic to one service, to one ingress with multiple routing rules, then using a domain name and finally multiple domain names

Simple Ingress

‍

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  defaultBackend:
    service:
      name: httpbin
      port:
        number: 80

‍

Here is a brief explanation of fields used in the Ingress resource -

• defaultBackend - the default service to send all traffic to when there are no rules or none of the rules match
• service - the service running inside the kubernetes cluster
• port.number - the port on which the service accepts traffic

Simple Ingress with multiple routing rules

‍

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  rules:
  - http:
      paths:
      - path: /post
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port:
              number: 80
      - path: /get
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port:
              number: 80

‍

This ingress example provides an example for L7 routing. For this example the destination of both the routes is the same service, but it could be a different service or different versions of the service

Simple Ingress with domain and multiple routing rules

‍

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  labels:
    app: httpbin
spec:
  rules:
  - host: httpbin.enroutedemo.com
    http:
      paths:
      - path: /post
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80
      - path: /get
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80

‍

The above example presents a way to match hostname for the incoming request. There should be an external DNS setup (httpbin.enroutedemo.com → Public IP) to send traffic to the public IP associated with the external load balancer that sends traffic to this Ingress.

Name based virtual host routing with multiple domains and multiple routing rules

‍

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  labels:
    app: httpbin
spec:
  rules:
  - host: primary.httpbin.enroutedemo.com
    http:
      paths:
      - path: /post
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80
      - path: /get
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80
  - host: replica.httpbin.enroutedemo.com
    http:
      paths:
      - path: /post
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80
      - path: /get
        pathType: Prefix
        backend:
          service:
            name: httpbin
            port: 80

This example is an extension of the above with multiple names specified in the Ingress resource. Both the names primary.httpbin.enroutedemo.com and replica.httpbin.enroutedemo.com may have the same IP in DNS, but get routed on basis of the domain name

Kubernetes Ingress v/s LoadBalancer v/s NodePort

When a service is created in Kubernetes, it runs in its own networking stack and it is only accessible to other services in the cluster. This is the default behavior where the service type is ClusterIP. It means, the IP address provided to the service is only “visible” to other services inside the cluster.

‍

apiVersion: v1
kind: Service
metadata:
  labels:
    app: httpbin
  name: httpbin
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: httpbin
  type: ClusterIP

---
# Expose the service using Ingress resource

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  defaultBackend:
    service:
      name: httpbin
      port:
        number: 80

Without changing the service type, such a service can be externally exposed using an Ingress type of resource.

Setting the service type to NodePort , opens up a port on the Nodes (in the cluster). Any traffic sent to this port is routed to the service.

‍

apiVersion: v1
kind: Service
metadata:
  labels:
    app: httpbin
  name: httpbin
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: httpbin
  type: NodePort

# Service exposed on the kubernetes Node locally

‍

‍

Setting the service to LoadBalancer , opens up a port on the external Load Balancer on public cloud (eg: NLB or ALB on AWS). Any traffic sent to the external load balancer is routed to the service

‍

apiVersion: v1
kind: Service
metadata:
  labels:
    app: httpbin
  name: httpbin
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: httpbin
  type: LoadBalancer

# service is exposed to the external world
# through a public IP allocated on external load balancer

‍

Why Use an Ingress?

Part of what an Ingress provides, can be achieved by directly exposing a service. What are the advantages of using an Ingress?

Typically Ingress provides a more controlled way of exposing a service. For example, in some cases we may only want to allow access to the service over a specific path.

Alternatively, access to a service should only be allowed when it is over a specific domain and not otherwise.

Ingress also provides a mechanism to centralize all routing rules to one location inside the cluster. This can be done for all services that are externally exposed. This provides a mechanism to standardize how external services communicate with services inside the cluster.

There are several occasions where more functionality is required over what is provided compared to Ingress. Since Ingress spec is limited, such added functionality may be provided using annotations. Annotations are way to provide key/value pairs or additional config. In case of Ingress, this additional configuration may be indications of how to interact with the external load balancer. For example, the following annotations provide hints on how the external load balancer should be programmed on AWS -

‍

service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Preferred

‍

The above annotation provides a hint that http2 is preferred as a protocol. There are several other annotations provided for AWS.

Similarly there are annotations that provide hints to the azure kubernetes service load balancer -

service.beta.kubernetes.io/azure-load-balancer-ipv4

‍

Where an annotation above specifies that a V4 address should be used.

There are several such requirements which are not a part of the Ingress specificiation. A Kubernetes Ingress Controller that can provide these additional services can augment or replace an Ingress. Example of this may be specifying more stringent rules to access the service eg: for the example above, both the method GET should match along with the path /get

Kubernetes Ingress Controller

An Ingress Controller is a LoadBalancer for microservices running inside Kubernetes. It is like Ingress but can do a lot more compared to the Ingress.

An Ingress Controller is installed separately and runs a reverse proxy like Envoy, nginx or HAProxy. An Ingress Controller is deployed in Kubernetes like any other workload, say for instance using a Deployment. Then a LoadBalancer service can be used to expose the proxy of the Ingress Controller

EnRoute is a rich Ingress Controller that is built on Envoy proxy. Another example is nginx ingress controller.

‍

‍

Ingress Controllers provide a wide variety of features in addition to basic path based routing. If services need certificate management using an ACME provide like Let’s Encrypt, an Ingress Controller can help facilitate installing such certificates. When using certificates and TLS, it may be necessary to disable some ciphers to make it more secure, Ingress Controllers also provide options for this kind of config.

Ingress Controller like EnRoute support a majority of protocols and are not limited to just HTTP and HTTP(s). EnRoute has support to route both TCP and UDP when that is what is necessary.

The Gateway API Specification is a detailed spec which is the next version of Ingress specification. An Ingress Controller like EnRoute (and others too), provide an implementation of the Gateway API specification.

Ingress Controllers are a critical component of a Kubernetes Microservices based architecture.