Run WASM At Ingress To Validate, Verify And Transform

Use WebAssembly (WASM) to steer and control traffic at Ingress

Introduction

When developer get tools they like, they tend to be more productive. An extensibility in a language of choice, provides the convenience of extensibility without learning a new language. WebAssembly runtime executes bytecode, generated by compiling code written in high level languages - javascript, C, C++, Python, Rust or Golang.

We explore WebAssembly support in EnRoute gateway, and how we can load bytecode generated from custom logic written in a language of choice. into the WebAssembly runtime using EnRoute.

We start with a goal to verify a request, validate it’s schema and transform the request. We use custom code that is compiled and loaded into the WebAssembly runtime, to achieve this

WASM support is a part of EnRoute Community Edition which is free to use without any enterprise license.

What This Article Covers

The following steps walk through what’s needed to work with a request

  • Install EnRoute with WASM support
  • Install example workload
  • Program EnRoute to make the service externally available
  • Create a WebAssembly filter
  • Enable WebAssembly filter for service
  • Ensure request MAC is validated and transform request to redact sensitive information
  • Ensure JSON schema is validated

Next we go through the above mentioned steps in detail. We also look at how to build a WebAssembly image in compat variant which is then loaded into the Envoy WebAssembly runtime by EnRoute

Install EnRoute With WASM Support

WASM support is provided in a separate EnRoute image. It can be installed using the following helm command after adding the helm repo and installing the enroute image with wasm tag


helm repo add saaras https://charts.getenroute.io
helm install enroute-demo saaras/enroute \
	--set serviceAccount.create=true \
	--create-namespace \
	--namespace enroutedemo \
	--set images.enrouteService.tag=wasm

Install Example Workload


kubectl create namespace httpbin
kubectl apply -f https://raw.githubusercontent.com/saarasio/enroute/master/helm-chart/httpbin-bin-service.yaml

Program EnRoute To Make The Service Externally Available


helm install httpbin-service-policy saaras/service-policy \
  --set service.name=httpbin \
  --set service.prefix=/post \
  --set service.port=80 \
  --namespace httpbin

Send request to External-IP of LoadBalancer service to verify it works as expected. The getting started guide has details on setting this up.


curl -X POST -H 'Content-Type: application/json' -d '{}' 212.2.241.255/post


{
  "args": {},
  "data": "{}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Content-Length": "2",
    "Content-Type": "application/json",
    "Host": "212.2.241.255",
    "User-Agent": "curl/7.68.0",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Envoy-External-Address": "192.168.1.5"
  },
  "json": {},
  "origin": "152.70.114.65,192.168.1.5",
  "url": "http://212.2.241.255/post"
}

Create A WebAssembly Filter


cat <<EOF | kubectl apply -f -
apiVersion: enroute.saaras.io/v1
kind: HttpFilter
metadata:
  name: httpbin-wasm-wasmfilter-oci
  namespace: httpbin
spec:
  httpFilterConfig:
    config: |
      {
              "url" : "oci://saarasio/vvx-json"
      }
  name: httpbin-wasm-wasmfilter-oci
  type: http_filter_wasm
EOF

Ensure MAC Is Verified And Request Is Transformed With Personal Information Redacted

We first create a payload JSON that we send to the server using a POST request. Here is the json payload in a file called payload.json


{
  "data": [{
    "type": "articles",
    "id": "1",
    "attributes": {
      "title": "JSON:API paints my bikeshed!",
      "body": "The shortest article. Ever.",
      "created": "2006-01-02 15:04",
      "updated": "2011-01-19 22:15",
      "dob": "2022-01-19 22:15"
    },
    "relationships": {
      "author": {
        "data": {"id": "42", "type": "people"}
      }
    }
  }],
  "included": [
    {
      "type": "people",
      "id": "42",
      "attributes": {
        "name": "John",
        "age": 80,
        "gender": "male"
      }
    }
  ]
}

For verification, we calculate the SHA256 of the payload and compare it to the value we compute and send in the Digest header


cat payload.json | tr -d "\n" | openssl dgst -sha256 -binary | base64


h3wEb7jrjmwD8O7TDvOD3WGG23lfnzsfmODcbwLmdlk=

To send a request


curl -vv 212.2.241.255/post -H 'Content-Type: application/json' -H "Digest: sha256=h3wEb7jrjmwD8O7TDvOD3WGG23lfnzsfmODcbwLmdlk="  --data @/home/ubuntu/payload.json



{
  "args": {},
  "data": "{}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Content-Length": "2",
    "Content-Type": "application/json",
    "Host": "212.2.241.255",
    "User-Agent": "curl/7.68.0",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Envoy-External-Address": "192.168.1.5"
  },
  "json": {},
  "origin": "152.70.114.65,192.168.1.5",
  "url": "http://212.2.241.255/post"
}


Ensure JSON Body Schema Is Validated

The JSON validation checks for presence of a few JSON elements in the payload and the format of certain fields. In the above payload, it checks for type, id, title, body and verifies the format of created and updated files to ensure valid dates are provided.

Code

Here is the code and it’s flow to achieve validate, verify, transform functions. The code can also be found on the saarasio github repo

Accumulate Request Body And Call Validate, Verify And Transform Functions


func (ctx *validateVerifyTransformContext) OnHttpRequestBody(bodySize int, endOfStream bool) types.Action {
        proxywasm.LogInfo("OnHttpRequestBody from Go!")
        ctx.totalRequestBodySize += bodySize
        if !endOfStream {
                // Wait until we see the entire body to replace.
                return types.ActionPause
        }

        jsonBody, err := proxywasm.GetHttpRequestBody(0, ctx.totalRequestBodySize)
        if err != nil {
                proxywasm.LogErrorf("failed to get request body: %v", err)
                return types.ActionContinue
        }

        if ctx.digestValueRead {
                secretKey := ""
                vererr, received, computed := Verify(jsonBody, ctx.digestValue, secretKey)

                proxywasm.LogInfof("WASM:OnHttpRequestBody() Received Digest: [%v] Computed Digest [%v]", received, computed)

                if vererr != nil {
                        if err := proxywasm.SendHttpResponse(400, nil, []byte(vererr.Error()), -1); err != nil {
                                panic(err)
                        }
                        return types.ActionPause
                }

                if received != computed {
                        if err := proxywasm.SendHttpResponse(400, nil, []byte("Received digest doesn't match computed digest\n"), -1); err != nil {
                                panic(err)
                        }
                        return types.ActionPause
                }
        }

        valerr := ValidateJSON(&jsonBody)

        if valerr != nil {
                if err := proxywasm.SendHttpResponse(400, nil, []byte(valerr.Error()), -1); err != nil {
                        panic(err)
                }
                return types.ActionPause

        }

        TransformJSON(&jsonBody)

        return types.ActionContinue
}

Verify HMAC


//////// Verify Digest ////////

func Verify(bytesIn []byte, encodedHash string, secretKey string) (error, string, string) {
        var valerr error

        var payload string
        var prefix string

        if strings.HasPrefix(encodedHash, "sha1=") {
                payload = strings.TrimPrefix(encodedHash, "sha1=")
                prefix = "sha1"
        } else if strings.HasPrefix(encodedHash, "sha256=") {
                payload = strings.TrimPrefix(encodedHash, "sha256=")
                prefix = "sha256"
        } else {
                valerr = fmt.Errorf("Unknown prefix in hash: [%v]\n", encodedHash)
                return valerr, "", ""
        }

        messageMAC := payload

        messageMACBuf2, _ := b64.StdEncoding.DecodeString(messageMAC)

        var receivedHash, computedHash string

        if prefix == "sha1" {
                sum := sha1.Sum(bytesIn)
                receivedHash = b64.StdEncoding.EncodeToString(messageMACBuf2[:])
                computedHash = b64.StdEncoding.EncodeToString(sum[:])
        } else if prefix == "sha256" {
                sum := sha256.Sum256(bytesIn)
                receivedHash = b64.StdEncoding.EncodeToString(messageMACBuf2[:])
                computedHash = b64.StdEncoding.EncodeToString(sum[:])
        } else {
                valerr = fmt.Errorf("Unknown prefix in hash: [%v]\n", encodedHash)
        }

        return valerr, receivedHash, computedHash
}

Validate JSON


func ValidateJSON(jsonBody *[]byte) error {

        // Verify elements are present
        paths := [][]string{
                []string{"data", "[0]", "type"},
                []string{"data", "[0]", "id"},
                []string{"data", "[0]", "attributes", "title"},
                []string{"data", "[0]", "attributes", "body"},
                []string{"data", "[0]", "attributes", "created"},
                []string{"data", "[0]", "attributes", "updated"},
        }
        match := 0
        var created, updated []byte
        jsonparser.EachKey(*jsonBody, func(idx int, value []byte, vt jsonparser.ValueType, err error) {
                switch idx {
                case 0: //
                        match += 1
                case 1: //
                        match += 1
                case 2: //
                        match += 1
                case 3: //
                        match += 1
                case 4: //
                        match += 1
                        created = value
                case 5: //
                        match += 1
                        updated = value
                }
        }, paths...)

        proxywasm.LogInfof("JSON Schema Validation Success, Match Count [%d]\n", match)

        var ret error

        // Verify created and updated are in date format
        _, err := time.Parse(string(created), string(updated))
        if err != nil {
                proxywasm.LogError("Failed to parse created, updated")
                ret = fmt.Errorf("Failed to parse created, updated")

        }

        if match != 6 {
                ret = fmt.Errorf("Failed to validate json schema")
        }

        proxywasm.LogInfo("JSON Field Validation Success, Validated Created, Updated \n")
        return ret
}

Transform JSON Body


//////// Transform JSON ////////

func TransformJSON(jsonBody *[]byte) {

        paths := [][]string{
                []string{"data", "[0]", "attributes", "dob"},
        }
        jsonparser.EachKey(*jsonBody, func(idx int, value []byte, vt jsonparser.ValueType, err error) {
                switch idx {
                case 0: //
                        *jsonBody, _ = jsonparser.Set(*jsonBody, []byte("\"0000-00-00 00:00\""), "data", "[0]", "attributes", "dob")
                        proxywasm.ReplaceHttpRequestBody(*jsonBody)
                }
        }, paths...)

}

Conclusion

WebAssembly provides a flexible extension mechanism to customize request verification, validation and transformation. WebAssembly in a proxy implements the proxy-wasm abi spec, that is implemented by SDKs. The SDKs in different languages provide a mechanism to interface with WebAssembly runtime. We use the go sdk that implements the proxy-wasm abi spec to achieve this.

The code generated in the above step is compiled to WASM using the WASI libc when compiling using tinygo The bytecode generated after the compilation is packaged into a WASM artifact in compat variant that is then loaded into the Envoy WebAssembly runtime.

In the next article we cover how to build a compat variant of an image that can be loaded by EnRoute and details on proxy-wasm ABI that let’s us recieve callbacks for request processing.