Feedback from the implementation in Otoroshi, the more extensible Scala reverse proxy

One of the main announcements of recent years is WebAssembly (abbreviated Wasm). Sometimes called the fourth language of the web, or the competitor to Javascript, it is, according to the W3C, a binary instruction format for a stack-based virtual machine.

Wasm is designed to be fast, secured and enable the possibility to browsers and others runtimes to run a ton of languages.

For APIs management, FAAS (Function-as-a-Service), and others cloud service providers, it is the best opportunity to simply extend models without exposing themselves to vulnerabilities in their own infrastructures.

For us, developers of reverse proxy and api management, it was a big challenge to introduce Wasm to our stack.

We are working on Otoroshi, which is an API management on top of reverse proxy. Our main goals are to offer an efficient and fast product, capable of responding to dozens of requests. We also want to offer the most complete list of plugins to allow users to customize and secure all their services exposed via Otoroshi : we want to be modular.

Before Wasm, our unique choice were to allow users to load their own plugins (built in JAR file) and to execute them. This approach have many disadvantages :

• no security : we are unable to control the code present in the JAR file. It can block Otoroshi or contain any computer virus

• performance issues : it must be load when Otoroshi starts, which can cause some latency issues

• JVM languages: it can be view as an advantage but with this approach, we can only write plugins using Java and Scala, leaving aside others languages

All of these things led us think about implementing Wasm into our stack allowing users to develop with any languages.

An efficient and customizable way

The first challenge was to find a runtime to execute Wasm files on JVM. For the moment, no runtime are directly implemented this. So we turned to a framework, able to use existing runtimes (like Wasmtime in Rust, Wazero in Go, etc …) and callable from Java.

Extism checks all of these expectations because it is a cross-language framework for building with WebAssembly. It supports many languages to write and execute Wasm files. Their approach are written to be extensible and used by any languages that supports FFI : Foreign function interface. FFI mechanism allows languages to call routines or services written or compiled in another one.

The Extism plug-in system is composed of SDKs and PDKs : SDKs are used to execute Wasm files and PDKs are used to write and compile language to Wasm binaries.

To fit Otoroshi stack languages, we chose and started implementing the Java-SDK into our model.

An orchestrator of Wasm Virtual Machine

As a reverse-proxy, Otoroshi has to be fast as possible to reduce its overhead time on each request. It needs to execute fews plugins in milliseconds to respond to queries in the shortest possible time. These plugins, initially written in the Scala core, meet this expectation. So to introduce Wasm plugins in the flows, we had to meet the same need : load and run them quickly.

Wasm runtimes come with this disadvantage for a reverse-proxy : they are slow to load a Wasm file, which means more than few milliseconds. To cover this need, we have working on an orchestrator of Wasm Virtual Machine, based on Extism. The best operation in the orchestrator is to reuse the virtual machine as much as possible by resetting the internal memory of each virtual machine. This allows Otoroshi to reuse the virtual machine dozens of time, without loading Wasm more than once.

Now we have all requirements to allow users to write their plugins without lost in performance. So let’s see how simple it is to create and use our first Wasm plugin in Otoroshi.

Proof by example

Let’s start an Otoroshi by downloading and running the JAR file.

curl -L -o otoroshi.jar \
'https://github.com/MAIF/otoroshi/releases/download/v16.14.0/otoroshi.jar' && \
java -Dotoroshi.adminPassword=password -jar otoroshi.jar

Once started, we can create our first Wasm plugin. Wasm plugin is a structure used by Otoroshi to describe content for running a virtual machine with a specific Wasm binary file. It contains at least three things : a name, the binary file and the function to call.

Let’s use the admin API of Otoroshi and a prebuilt Wasm file to create our first Wasm plugin. This wasm file contains a function named execute which checks for the presence of the foo header of a request.

curl -X POST "http://otoroshi-api.oto.tools:8080/apis/plugins.otoroshi.io/v1/wasm-plugins" \
-u admin-api-apikey-id:admin-api-apikey-secret \
-H 'Content-Type: application/json; charset=utf-8' \
-d @- <<'EOF'
{
  "id": "my-first-wasm-plugin",
  "name": "New wasm plugin",
  "description": "New wasm plugin",
  "config": {
    "source": {
      "kind": "Http",
      "path": "https://github.com/MAIF/wasmo/raw/main/wasm/validator-plugin.wasm",
      "opts": {}
    },
    "memoryPages": 100,
    "functionName": "execute",
    "config": {},
    "allowedHosts": [],
    "allowedPaths": {},
    "wasi": true,
    "opa": false,
    "instances": 1
  },
  "steps": [
    "ValidateAccess"
  ],
  "_loc": {
    "tenant": "default",
    "teams": ["default"]
  }
}
EOF

We just created the wasm plugin called New Wasm Plugin which will run the execute function of the Wasm file. This plugin is configured to run on the specific ValidateAccess step of a route. Depending on the criteria, this step validates the request content like headers and body and allows the request to continue its flow.

Let’s now create the route to expose a simple API secured by our wasm plugin.

curl -X POST "http://otoroshi-api.oto.tools:8080/apis/proxy.otoroshi.io/v1/routes" \
-u admin-api-apikey-id:admin-api-apikey-secret \
-H 'Content-Type: application/json; charset=utf-8' \
-d @- <<'EOF'
{
  "name": "My first route",
  "frontend": {
    "domains": ["my-route.oto.tools"]
  },
  "backend": {
    "targets": [
      {
        "hostname": "mirror.otoroshi.io",
        "port": 443,
        "tls": true
      }
    ]
  },
  "plugins": [
    {
      "enabled": true,
      "debug": false,
      "plugin": "cp:otoroshi.next.plugins.OverrideHost",
      "include": [],
      "exclude": [],
      "config": {},
      "plugin_index": {
        "transform_request": 0
      }
    },
    {
      "enabled": true,
      "debug": false,
      "plugin": "cp:otoroshi.next.plugins.WasmAccessValidator",
      "include": [],
      "exclude": [],
      "config": {
        "source": {
          "kind": "local",
          "path": "my-first-wasm-plugin",
          "opts": {}
        },
        "memoryPages": 20,
        "functionName": "",
        "config": {},
        "allowedHosts": [],
        "allowedPaths": {},
        "wasi": false,
        "opa": false,
        "instances": 1,
        "killOptions": {
          "immortal": false,
          "max_calls": 2147483647,
          "max_memory_usage": 0,
          "max_avg_call_duration": 0,
          "max_unused_duration": 300000
        }
      },
      "plugin_index": {
        "validate_access": 0
      }
    }
  ]
}
EOF

We just created a route to forward new-route.oto.tools calls to the mirror.otoroshi.io . The mirror service is commonly used to verify and display content send by Otoroshi to a downstream service. Between the client and the downstream service, Otoroshi will run the Wasm Access Validator plugin to ensure the foo header is present.

Let’s make a first call to our route.

curl http://new-route.oto.tools:8080 --include

In response, Otoroshi should return a you’re not authorized error.

This time, if we call the service passing the foo header with the bar value, we should be able to pass validation.

curl http://new-route.oto.tools:8080 -H foo:bar --include


HTTP/1.1 200 OK
Date: ...
Sozu-Id: 01HPJ3GAA3HM5JA9CWDNPYQMPA..
Opun-Gateway-State-Resp: none
Transfer-Encoding: chunked
Content-Type: application/json

{"method":"GET","path":"/", ...}

Congratulations 🎉. Our first route is running Wasm successfully!

But, what about performance between native Scala plugins and Wasm plugins ?

The environment of this test can really be improved but it can be used to give you an idea of the good performance of running Wasm in Otoroshi.

At the left we have the native execution, mostly execute in 24 ms. At the right, we have the Wasm execution in 31 ms.

Unsurprisingly, the route with the Wasm execution is slower than the native execution of Scala plugins, but it brings a huge new possibility to users : being able to write their own code using Javascript, Typescript, Go, Rust or Open Policy Agent (OPA).

This article is the first in a series around Wasm, Otoroshi and Extism.

If you followed this article to the end, add 👏 .

You can join discussions about Wasm and Otoroshi by clicking here.

About the projects used in this article :

Otoroshi : https://www.otoroshi.io/

The orchestrator of Wasm : https://github.com/MAIF/wasm4s

Extism : https://extism.org/