Engineering 13 min read

WAFio v2 preview: rewritten from Go to Rust, 90.61 % detection on SafeLine's blazehttp samples

WAFio v2 moves the WAF agent from Go to Rust and relies on its semantic engines alone. We measured it with blazehttp, the test tool from the makers of SafeLine, on the same 33,669 samples.

Bar chart of detection rate and false positive rate on 33,669 blazehttp samples for CloudFlare, ModSecurity, SafeLine (README and version 9.4.4 measured) and WAFio

WAFio v2 comes with a new WAF agent. The agent of WAFio v1 is written in Go. The agent of v2 is written in Rust, and it judges every request with our semantic engines alone, without the OWASP Core Rule Set. This article explains why we made the move, what changed in the engines after it, and how we measured the result. WAFio v2 is planned for release on 20 November 2026 at the latest, if nothing delays it.

For the measurement we used blazehttp, a test tool for web application firewalls published by Chaitin, the company behind SafeLine. It sends a fixed set of attack requests and normal requests through a WAF and counts what gets blocked. On 15 November 2026 we ran it with the same 33,669 samples against WAFio v2, against WAFio v1 and against the current SafeLine release, 9.4.4, all on one machine.

In short:

  • WAFio v2 blocked 521 of the 575 attack samples, 90.61 %. WAFio v1 with its semantic engines alone blocked 425, 73.91 %.
  • WAFio v2 blocked 20 of the 33,094 normal samples by mistake, 0.06 %. WAFio v1 blocked 282 with the semantic engines alone and 2,768 with the Core Rule Set switched on.
  • SafeLine 9.4.4 with its default settings blocked 356 attack samples and 17 normal samples. On false blocks and on response time it did better than WAFio v2.

The rest of the article gives the reasons, the full tables, the machine, the commands and the things this test does not prove.

Why we ran this test, and what it is not

This test is not meant to show which product is better. We ran it because we needed a yardstick for our own work. WAFio v2 changes how requests are judged, and we wanted to know, with numbers anyone can check, whether the new engines block more attacks and disturb less normal traffic than the old ones.

blazehttp fits that purpose for three reasons. The tool and its samples are public, so anyone can repeat the run. The samples are fixed, so two runs can be compared. And the makers of SafeLine published their own result with it, which gives a reference point that does not come from us. We are grateful that they made the tool and the samples open.

Every product has its strengths and its weak spots, and one sample set shows only a small part of them. In this run SafeLine blocked fewer normal requests than WAFio and answered faster. It also has parts this test does not touch at all. Read the numbers below as a measurement of one thing on one day, not as a ranking.

Why the WAF agent moved from Go to Rust

The WAF agent in the current release, WAFio v1, is written in Go. The build tested here is its replacement, written in Rust, and it is the WAF agent of WAFio v2. To see what the move bought, we ran both agents through the blazehttp test described further down: the same machine, the same limits of 2 CPUs and 4 GB, the same backend, the same samples.

RowWAFio v1 (Go), default rules with CRSWAFio v1 (Go), semantic engines onlyWAFio v2 (Rust)
Total samples33,66933,66933,669
Errors000
Detection rate75.13 % (432 of 575)73.91 % (425 of 575)90.61 % (521 of 575)
False positive rate8.36 % (2,768 of 33,094)0.85 % (282 of 33,094)0.06 % (20 of 33,094)
Accuracy91.35 %98.72 %99.78 %
Average time32.89 ms12.16 ms6.63 to 7.44 ms

The Go agent in this run is built from our main branch, the code line of the 1.10 releases, with the OWASP Core Rule Set 4.9.0 that the installer fetches. Like the WAFio v2 run, it went without a control plane and with the block policy and the L7 flood limit switched off.

WAFio v2 no longer depends on the OWASP Core Rule Set. In WAFio v1 a request passed through two judges: the Core Rule Set and our semantic engines. The table shows what each one added on these samples. The rule set brought 7 more blocked attacks, 432 against 425, and 2,486 more blocked normal requests, 2,768 against 282. It also took most of the time. The Core Rule Set is careful, widely used work, and it catches things by pattern that a meaning-based engine has to learn one by one. But after the engine work described below, our semantic engines block more of these attacks on their own than both judges did together, with far fewer false blocks. That is why we are now confident enough to rely on them alone.

Besides detection, three more things decided the move to Rust.

  • CPU per request. On 13 November 2026 we ran both agents on the same machine, limited to 1.5 CPUs. To analyze the blazehttp samples, the Rust agent used 0.149 times the CPU time of the Go agent, about one seventh.
  • Requests per second. With two worker threads each, the Rust agent served 2.8 times as many requests per second as the Go agent before it saturated.
  • One detection path. With the rule set gone there is one place where a request is judged and one place to fix when a verdict is wrong.

The move was done in two steps. First the Rust agent had to give the same verdict as the Go agent on 130,509 recorded requests, so that a difference could only come from a change we made on purpose. Only then did we start to change the engines.

It was not free. Under load the Rust agent used more memory at its peak, 201 to 209 MiB against 155 to 157 MiB for the Go agent. And the 99th percentile of the response time was 2.7 ms where our target is 1 ms. Both are open work.

What changed in the semantic engines

Before the engine work, the Rust agent decided like the Go agent with its semantic engines only. On the newer blazehttp sample set (658 attack samples, 33,219 normal samples), with the samples fed straight to the analyzer, that was 452 attacks blocked, 68.69 %, and 299 normal samples blocked, 0.90 %. After the work it is 604 attacks blocked, 91.79 %, and 20 normal samples blocked, 0.06 %. Over HTTP the same build blocks 603 and 20.

  • SQL. A signal counts only when the value goes on as SQL. A lone keyword in a sentence does not. False blocks went from 165 to 1.
  • PHP. Only code that PHP would run counts, not PHP-looking fragments in prose or data. False blocks went from 39 to 0.
  • XSS. Only markup and script that a browser would run counts. False blocks went from 38 to 5.
  • Commands, paths, URLs, templates. Look-alikes of shell, path, URL and template syntax are no longer read as attacks. False blocks in these engines went from 56 to 5.
  • Hidden payloads. Text under base64 is decoded and scanned, within a fixed budget per request, and calls through a computed index on a global object are recognized. Together with the other detection work this took the blocked attacks from 452 to 604.

The detection work also added 8 false blocks that were not there before: 7 on text with LDAP filter syntax and 1 on an XML document.

After the changes a second reviewer read the diff with one task: find attacks that the false positive fixes now let through. It found seven gaps. One was serious: a file:// URL that ends in an image or stylesheet name was treated as harmless even when it named a remote host. That one and three others are closed, each with a test that failed before the fix. Three minor ones are still open.

Where the SafeLine numbers come from

The numbers we compare against are not ours. They are the results table in the blazehttp README: README_EN.md in chaitin/blazehttp. The SafeLine README repeats the detection, false positive and accuracy rows: chaitin/SafeLine.

That table was written for 33,669 samples: 575 attack samples and 33,094 normal samples. The sample set in the repository has grown since then, so we took the samples from commit dde33c98 of 6 June 2024, the last commit before the samples changed. It holds exactly 575 files ending in .black and 33,094 ending in .white.

We did not install ModSecurity or CloudFlare. Their numbers below are copied from the README. For SafeLine we give both: the README numbers of 2024, and our own measurement of version 9.4.4 on the machine that ran WAFio.

The result

blazehttp, 33,669 samples (575 attack, 33,094 normal). Bars share one scale, 0 to 100 %.

Detection rate: share of attack samples blocked (higher is better)

CloudFlare, free (README)
10.70 %
ModSecurity, level 1 (README)
69.74 %
ModSecurity, level 4 (README)
94.61 %
SafeLine, balance (README)
71.65 %
SafeLine, strict (README)
76.17 %
SafeLine 9.4.4, default (measured)
61.91 %
SafeLine 9.4.4, strict (measured)
69.22 %
WAFio v1 (Go), with CRS (measured)
75.13 %
WAFio v1 (Go), semantic only (measured)
73.91 %
WAFio v2 (measured)
90.61 %

False positive rate: share of normal samples blocked (lower is better)

CloudFlare, free (README)
0.07 %
ModSecurity, level 1 (README)
17.58 %
ModSecurity, level 4 (README)
52.46 %
SafeLine, balance (README)
0.07 %
SafeLine, strict (README)
0.22 %
SafeLine 9.4.4, default (measured)
0.05 %
SafeLine 9.4.4, strict (measured)
0.14 %
WAFio v1 (Go), with CRS (measured)
8.36 %
WAFio v1 (Go), semantic only (measured)
0.85 %
WAFio v2 (measured)
0.06 %

Accuracy: samples handled correctly, of all samples (higher is better)

CloudFlare, free (README)
98.40 %
ModSecurity, level 1 (README)
82.20 %
ModSecurity, level 4 (README)
48.34 %
SafeLine, balance (README)
99.45 %
SafeLine, strict (README)
99.38 %
SafeLine 9.4.4, default (measured)
99.30 %
SafeLine 9.4.4, strict (measured)
99.34 %
WAFio v1 (Go), with CRS (measured)
91.35 %
WAFio v1 (Go), semantic only (measured)
98.72 %
WAFio v2 (measured)
99.78 %

Rows marked README: results table of the blazehttp README. Rows marked measured: our runs of 15 November 2026 on one machine.

Row of the README tableSafeLine, balance (README, 2024)SafeLine 9.4.4, default (measured)SafeLine 9.4.4, strict (measured)WAFio v2 (measured)
Total samples33,66933,66933,66933,669
Successful33,66933,66933,66933,669
Errors0000
Detection rate71.65 % (412 of 575)61.91 % (356 of 575)69.22 % (398 of 575)90.61 % (521 of 575)
False positive rate0.07 % (23 of 33,094)0.05 % (17 of 33,094)0.14 % (45 of 33,094)0.06 % (20 of 33,094)
Accuracy99.45 %99.30 %99.34 %99.78 %
Average time70.05 ms6.20 ms4.95 ms6.63 to 7.44 ms

What the table says, row by row:

  • Detection. WAFio v2 blocks 521 of the 575 attack samples. SafeLine 9.4.4 blocks 356 with its default settings and 398 in strict mode. The README reports 412.
  • False positives. SafeLine 9.4.4 with default settings is ahead: it blocks 17 normal samples, WAFio v2 blocks 20. Against the README number, 23, WAFio v2 is ahead. The difference is three samples either way.
  • Accuracy. WAFio v2 handles 99.78 % of all samples correctly, SafeLine 9.4.4 handles 99.30 %.
  • Average time. SafeLine 9.4.4 answered in 4.95 to 6.20 ms, WAFio v2 in 6.63 to 7.44 ms. On this row SafeLine is ahead.
  • One more column from the README. ModSecurity at paranoia level 4 blocks more attacks than WAFio v2, 94.61 %, but it also blocks 52.46 % of the normal samples.

What was tested

  • The build. The WAF agent of WAFio v2, its proxy and its detection engines, built from our main branch on 15 November 2026. This build is not in a release yet. The release you can download today is WAFio v1 with the Go agent, and it does not reach these numbers.
  • The mode. Block mode. A blocked request gets status 403 and never reaches the backend.
  • The rules. The default rules a new app gets from the control plane: block at score 150, log at score 80, the default thresholds per engine, and no OWASP Core Rule Set. Nothing was tuned for the test.
  • Two protections were switched off. The block policy (an address is blocked after 100 blocked requests) and the L7 flood limit (500 requests per 60 seconds per address). blazehttp sends every sample from one address, so with these on, the run measures the rate limit and not the detection.
  • Not in the path. The proxy ran without a control plane. The bad bot list, GeoIP and IP reputation were not loaded.
  • The backend. nginx 1.29.8 that answers 200 to every request.

The test environment

  • A KVM virtual machine with 4 virtual CPUs (AMD EPYC 9354P) and 15 GiB of memory. The machine also runs other services.
  • Ubuntu 24.04.5 LTS, kernel 6.8.0, Docker 29.8.1.
  • The WAFio proxy in a container limited to 2 CPUs and 4 GB of memory, built with Rust 1.95.0.
  • blazehttp built from source with Go 1.22.12, run with its default of 10 concurrent workers.
  • blazehttp, the proxy and the backend on the same machine, talking over the loopback interface.

The commands

Get blazehttp, build it, and export the sample set of the README table into its own directory:

git clone https://github.com/chaitin/blazehttp
cd blazehttp
docker run --rm -v "$PWD":/src:ro -v "$PWD/..":/out -w /src \
  -e CGO_ENABLED=0 -e GOFLAGS=-buildvcs=false golang:1.22-alpine \
  go build -o /out/blazehttp-bin ./cmd/blazehttp

mkdir ../samples
git archive dde33c98 testcases | tar -x -C ../samples
find ../samples/testcases -name '*.go' -delete
find ../samples/testcases -name '*.black' | wc -l    # 575
find ../samples/testcases -name '*.white' | wc -l    # 33094

Start a backend that answers 200 to everything. This is the nginx configuration we used, and the command that starts it:

server {
    listen 127.0.0.1:18081;
    client_max_body_size 0;
    large_client_header_buffers 8 64k;
    access_log off;
    location / { return 200 "ok\n"; }
}
docker run -d --net=host --cpus=1 --memory=256m \
  -v "$PWD/upstream.conf":/etc/nginx/conf.d/default.conf:ro nginx:alpine

Put WAFio in front of it in block mode. In our run the proxy listened on 127.0.0.1:18080 and forwarded to http://127.0.0.1:18081. Check both paths by hand before the run: a normal request must pass, an attack must be blocked.

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:18080/index.html
# 200
curl -s -o /dev/null -w '%{http_code}\n' \
  'http://127.0.0.1:18080/item?id=1%27%20UNION%20SELECT%20username,password%20FROM%20users--'
# 403

Then run blazehttp against the proxy with the exported samples:

../blazehttp-bin -t http://127.0.0.1:18080 -g ../samples/testcases

To repeat this against your own WAFio install, point -t at an app that is in block mode, and switch off the block policy and the L7 flood limit for that app during the run.

The output of the WAFio run

Terminal output of the run: 575 attack samples, 33,094 normal samples, status 200 for a normal request, 403 for an attack, then the blazehttp result
The commands of the run and what they printed, rendered from the recorded terminal output of 15 November 2026. The paths are those of our test machine.

blazehttp prints its result in Chinese. Line by line:

Total samples: 33669    successful: 33669    errors: 0
Detection rate: 90.61% (attack samples: 575, blocked: 521, missed: 54)
False positive rate: 0.06% (normal samples: 33094, passed: 33074, blocked by mistake: 20)
Accuracy: 99.78% (blocked attacks + passed normal requests) / total samples
Average time: 7.44 ms

The full run takes about 25 seconds. We ran it twice on this build. The counts were the same both times; the average time was 6.63 ms and 7.44 ms.

SafeLine 9.4.4 on the same machine

We installed SafeLine 9.4.4, the newest release on 15 November 2026, on the same machine, with the compose file of that release and the images chaitin/safeline-*-g:9.4.4. It is the free edition, a fresh install, with nothing tuned.

  • One application: any host name, port 18082, forwarding to the same nginx backend that WAFio used.
  • First run: the detection modules as installed. 15 of the 17 modules are in their default mode and two (m_response, m_scanner) are off. We read this as the balance mode of the README.
  • Second run: all 17 modules set to strict, the two that are off by default included. That may be stricter than the strict mode of the README.
  • The same blazehttp binary and the same sample directory as for WAFio.
curl -s -o /dev/null -w '%{http_code}
' http://127.0.0.1:18082/index.html
# 200
curl -s -o /dev/null -w '%{http_code}
'   'http://127.0.0.1:18082/item?id=1%27%20UNION%20SELECT%20username,password%20FROM%20users--'
# 403
../blazehttp-bin -t http://127.0.0.1:18082 -g ../samples/testcases

The result of the two runs, translated line by line:

SafeLine 9.4.4, default settings
Total samples: 33669    successful: 33669    errors: 0
Detection rate: 61.91% (attack samples: 575, blocked: 356, missed: 219)
False positive rate: 0.05% (normal samples: 33094, passed: 33077, blocked by mistake: 17)
Accuracy: 99.30%
Average time: 6.20 ms

SafeLine 9.4.4, all modules strict
Total samples: 33669    successful: 33669    errors: 0
Detection rate: 69.22% (attack samples: 575, blocked: 398, missed: 177)
False positive rate: 0.14% (normal samples: 33094, passed: 33049, blocked by mistake: 45)
Accuracy: 99.34%
Average time: 4.95 ms

SafeLine 9.4.4 detected less than its own README reports: 61.91 % against 71.65 %. We do not know why. The sample set is the one the README was written for, so the difference is in the product version or in a setting we did not find. If you know which setting the README run used, tell us and we will repeat the run.

How blazehttp decides

  • Before the run it sends one harmless request and one obvious attack to the target. If both get the same status code, it stops: there is no WAF in front. Otherwise the status code of the attack becomes the block code. For WAFio that is 403.
  • Every sample is one raw HTTP request in a file. A file ending in .black is an attack, one ending in .white is normal traffic.
  • A sample counts as blocked when the answer carries the block code. A sample with no readable answer counts as an error.
  • Detection rate is blocked attacks divided by all attacks. False positive rate is blocked normal samples divided by all normal samples. Accuracy is blocked attacks plus passed normal samples, divided by all samples.

What these numbers do not say

  • It is not a blind test. We used this sample set while working on the detection engines. A result on samples the engines have never met may be lower. The same holds for any product whose maker owns the sample set.
  • Our SafeLine run is one configuration. A fresh install of the free edition with default settings. Someone who knows the product may get more out of it.
  • The average times are rough. All measured runs went over the loopback interface on one machine that was also doing other work. The 70 ms of the README comes from a setup it does not describe.
  • A sample set is not your traffic. 33,094 normal samples are a small slice of what real applications send. Run a WAF in monitor mode on your own traffic before you switch it to block.
  • The settings matter. With thresholds lower than the default rules, the same build blocked 120 normal samples, 0.36 %. A benchmark number belongs to one configuration.

What it still gets wrong

54 attack samples passed. Among them: file uploads whose content was not flagged, SQL one-liners that read a system table or call a database function (SELECT * FROM all_tables, dbms_pipe.receive_message), HTML that loads a remote resource without any script (<body background>, <meta http-equiv="refresh">), and requests whose test parameter is empty in the request line.

20 normal samples were blocked: 7 texts that carry LDAP filter syntax, 5 pieces of markup with a script tag or a javascript: URL as content, 3 paths that look like directory traversal, 2 URLs that point at localhost, 1 text read as SQL, 1 XML document with an entity, and 1 request over the body size limit. These are the next things on our list.

How the detection works is described on the Semantic WAF page, and why WAFio scans by meaning explains the reasoning behind it.

Sources