Secure Azure Kubernetes with Let’s Encrypt certificates

Kubernetes has become a standard when it comes to automating deployment, scaling, and management of containerized applications. Azure Kubernetes Service (AKS) offers serverless Kubernetes. If you run many applications on a AKS cluster, you can secure the connection to the applications automatically by using Let’s Encrypt SSL certificates.
Of course, it would also work with traditional SSL certificates. However, automatic issuance and renewal by Let’s Encrypt is easier in the ongoing operations.
In this article I will show what to prepare on Azure Kubernetes Service and how to implement the automatic issuance by Let’s Encrypt.

Initially I used several resources on the web. Most of the sources had outdated versions or incorrect configuration files. One of the tutorials which was working fine is written by dev.to-Blogger Chris. Also I used the official cert-manager documentation. I made some changes to the configuration and finally got it working.

Background of this Article

Azure Kubernetes Service

I recommend that you connect to AKS via Azure Cloud Shell (Bash or PowerShell). The reason to do so, is, that you can store all configuration files in your mounted cloud drive.

az aks get-credentials -g "ResourceGroupName" -n "AKS-Cluster-Name"
kubectl config set-context "AKS-Cluster-Name"

Install & Configure CERT-MANAGER

You can easily install cert-manager with regular manifests or with Helm charts. It is deployed using regular YAML manifests, like any other application on Kubernetes.

kubectl create namespace cert-manager
# Kubernetes 1.16+
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager.yaml

# Kubernetes <1.16
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager-legacy.yaml

Alternatively you can install it also via the official Helm Chart:

helm repo add jetstack https://charts.jetstack.io
helm repo update
# Helm v3+
helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v1.1.0 \
  # --set installCRDs=true

# Helm v2
helm install \
  --name cert-manager \
  --namespace cert-manager \
  --version v1.1.0 \
  jetstack/cert-manager \
  # --set installCRDs=true

Verify the installation with the following command. Check the output which should look like this:

kubectl get pods --namespace cert-manager
---------------------------------------------------------------------------------------
NAME                                                   READY   STATUS    RESTARTS   AGE
certmanager-cert-manager-7565b00076-dqlhw              1/1     Running   0          30h
certmanager-cert-manager-cainjector-6f5xx0xbb5-zggbj   1/1     Running   1          30h
certmanager-cert-manager-webhook-76ff000f8d-bmc7r      1/1     Running   0          30h

Beyond, we do an extra-step to test if the cert-manager is working correctly. Create a file [test-cert-manager.yaml]. Apply it and verify the output.

# test-cert-manager.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager-test
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test-selfsigned
  namespace: cert-manager-test
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-cert
  namespace: cert-manager-test
spec:
  commonName: example.com
  secretName: selfsigned-cert-tls
  issuerRef:
    name: test-selfsigned

Next let us verify the output and if it is OK. Afterwards delete the resources.

kubectl apply -f test-cert-manager.yaml
kubectl describe certificate -n cert-manager-test

Possible output:

  Type    Reason          Age      From          Message
  ----    ------          ----     ----          -------
  ...
  ...
  Normal  CertIssued      55m      cert-manager  Certificate issued successfully

kubectl delete -f test-cert-manager.yaml

Configure Cluster Issuer

In addition, I assume that an NGINX Ingress Controller is already present (website-ingress.yaml). It is listening on HTTP port 80, there are several different host names configured as well.

To begin with, we create two Cluster Issuers. The first file is to validate our configuration with the Let’s Encrypt staging environment. Otherwise if there is an issue in the config, we will run into rate-limits at the Let’s Encrypt servers and get blocked.

In addition, adjust your eMail Configuration in these 2 files and create them.

$ kubectl create -f staging_issuer.yaml
$ kubectl create -f prod_issuer.yaml

# staging_issuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: [email protected]
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging-private-key
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: nginx
      selector: {}

# prod_issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: [email protected]
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-private-key
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: nginx
      selector: {}

Verify SSL certificates

To begin with, we adapt our “website-ingress.yaml” file. Therefore add the “letsencrypt-staging” secret-name to test the configuration. After that, if there are no errors with staging, we change it to letsencrypt-prod.

# website-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cert-manager
  annotations:
    kubernetes.io/ingress.class: "nginx"
    #certmanager.k8s.io/acme-challenge-type: http01
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
  tls:
  - hosts:
    - www.patrickriedl.at
    - patrickriedl.at
    secretName: letsencrypt-staging
  rules:
  - host: www.patrickriedl.at
    http:
      paths:
      - backend:
          serviceName: patrickriedl
          servicePort: 80
  - host: patrickriedl.at
    http:
      paths:
      - backend:
          serviceName: patrickriedl
          servicePort: 80

Now you have to apply the changes. As always, validate the output.

$ kubectl apply -f website-ingress.yaml
$ kubectl describe certificate letsencrypt-staging

After that verify the output, if the certificates get deployed. This could take several minutes as well, so please be patient 🙂

After that, you will see the message “Certificates issued successfully”. Also check in your browser and you will see the Let’s Encrypt Fake Authority. Next, you change the website-ingress.yaml with “letsencrypt-prod” issuer. The final file should look like this:

# website-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cert-manager
  annotations:
    kubernetes.io/ingress.class: "nginx"
    #certmanager.k8s.io/acme-challenge-type: http01
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - www.patrickriedl.at
    - patrickriedl.at
    secretName: letsencrypt-prod
  rules:
  - host: www.patrickriedl.at
    http:
      paths:
      - backend:
          serviceName: patrickriedl
          servicePort: 80
  - host: patrickriedl.at
    http:
      paths:
      - backend:
          serviceName: patrickriedl
          servicePort: 80

Finally, you have to apply the changes with “kubectl apply” command and verify the final results with the command:

$ kubectl describe certificate letsencrypt-prod

Wonderful!! 🙂 You have now deployed Let’s Encrypt Certificates successfully to your Kubernetes ingress. The sites are secured now. You can easily manage the domains from now on by updating the “hosts” settings in the website-ingress.yaml file.

Sources

• https://cert-manager.io
• https://docs.microsoft.com/en-us/azure/aks/ingress-tls
• https://dev.to/chrisme/setting-up-nginx-ingress-w-automatically-generated-letsencrypt-certificates-on-kubernetes-4f1k
• https://stackoverflow.com/questions/57506474/upgrading-from-helm-stable-cert-manager-to-jetstack-cert-manager
• https://stackoverflow.com/questions/58553510/cant-get-certs-working-with-cert-manager
• https://stackoverflow.com/questions/58423312/how-do-i-test-a-clusterissuer-solver/58436097