docs: analyzers

2024-09-21 12:14:08 +08:00 · 2024-01-24 20:01:53 -08:00 · 2024-01-24 20:01:53 -08:00 · d7d3437d3c
commit d7d3437d3c
parent 7441d24aea
3 changed files with 282 additions and 12 deletions
--- a/README.md
+++ b/README.md
@ -3,6 +3,7 @@
 [![License][1]][2]

 [1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
+
 [2]: LICENSE

 **[中文文档](README.zh.md)**
@ -19,11 +20,11 @@ Linux that's in many ways more powerful than the real thing. It's cyber sovereig
 ## Features

 - Full IP/TCP reassembly, various protocol analyzers
-  - HTTP, TLS, DNS, SSH, and many more to come
-  - "Fully encrypted traffic" detection for Shadowsocks,
-    etc. (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
-  - Trojan (proxy protocol) detection based on Trojan-killer (https://github.com/XTLS/Trojan-killer)
-  - [WIP] Machine learning based traffic classification
+    - HTTP, TLS, DNS, SSH, and many more to come
+    - "Fully encrypted traffic" detection for Shadowsocks,
+      etc. (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
+    - Trojan (proxy protocol) detection based on Trojan-killer (https://github.com/XTLS/Trojan-killer)
+    - [WIP] Machine learning based traffic classification
 - Full IPv4 and IPv6 support
 - Flow-based multicore load balancing
 - Connection offloading
@ -72,8 +73,7 @@ workers:

 ### Example rules

-Documentation on all supported protocols and what field each one has is not yet ready. For now, you have to check the
-code under "analyzer" directory directly.
+[Analyzer properties](docs/Analyzers.md)

 For syntax of the expression language, please refer
 to [Expr Language Definition](https://expr-lang.org/docs/language-definition).
--- a/README.zh.md
+++ b/README.zh.md
@ -3,6 +3,7 @@
 [![License][1]][2]

 [1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
+
 [2]: LICENSE

 OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedia.org/wiki/%E9%98%B2%E7%81%AB%E9%95%BF%E5%9F%8E)
@ -17,10 +18,10 @@ OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedi
 ## 功能

 - 完整的 IP/TCP 重组，各种协议解析器
-  - HTTP, TLS, DNS, SSH, 更多协议正在开发中
-  - Shadowsocks 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
-  - 基于 Trojan-killer 的 Trojan 检测 (https://github.com/XTLS/Trojan-killer)
-  - [开发中] 基于机器学习的流量分类
+    - HTTP, TLS, DNS, SSH, 更多协议正在开发中
+    - Shadowsocks 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
+    - 基于 Trojan-killer 的 Trojan 检测 (https://github.com/XTLS/Trojan-killer)
+    - [开发中] 基于机器学习的流量分类
 - 同等支持 IPv4 和 IPv6
 - 基于流的多核负载均衡
 - 连接 offloading
@ -69,7 +70,7 @@ workers:

 ### 样例规则

-关于规则具体支持哪些协议，以及每个协议包含哪些字段的文档还没有写。目前请直接参考 "analyzer" 目录下的代码。
+[解析器属性](docs/Analyzers.md)

 规则的语法请参考 [Expr Language Definition](https://expr-lang.org/docs/language-definition)。

--- a/docs/Analyzers.md
+++ b/docs/Analyzers.md
@ -0,0 +1,269 @@
+# Analyzers
+
+Analyzers are one of the main components of OpenGFW. Their job is to analyze a connection, see if it's a protocol they
+support, and if so, extract information from that connection and provide properties for the rule engine to match against
+user-provided rules. OpenGFW will automatically analyze which analyzers are referenced in the given rules and enable
+only those that are needed.
+
+This document lists the properties provided by each analyzer that can be used by rules.
+
+## DNS (TCP & UDP)
+
+For queries:
+
+```json
+{
+  "dns": {
+    "aa": false,
+    "id": 41953,
+    "opcode": 0,
+    "qr": false,
+    "questions": [
+      {
+        "class": 1,
+        "name": "www.google.com",
+        "type": 1
+      }
+    ],
+    "ra": false,
+    "rcode": 0,
+    "rd": true,
+    "tc": false,
+    "z": 0
+  }
+}
+```
+
+For responses:
+
+```json
+{
+  "dns": {
+    "aa": false,
+    "answers": [
+      {
+        "a": "142.251.32.36",
+        "class": 1,
+        "name": "www.google.com",
+        "ttl": 255,
+        "type": 1
+      }
+    ],
+    "id": 41953,
+    "opcode": 0,
+    "qr": true,
+    "questions": [
+      {
+        "class": 1,
+        "name": "www.google.com",
+        "type": 1
+      }
+    ],
+    "ra": true,
+    "rcode": 0,
+    "rd": true,
+    "tc": false,
+    "z": 0
+  }
+}
+```
+
+Example for blocking DNS queries for `www.google.com`:
+
+```yaml
+- name: Block Google DNS
+  action: drop
+  expr: dns != nil && !dns.qr && any(dns.questions, {.name == "www.google.com"})
+```
+
+## FET (Fully Encrypted Traffic)
+
+Check https://www.usenix.org/system/files/usenixsecurity23-wu-mingshi.pdf for more information.
+
+```json
+{
+  "fet": {
+    "ex1": 3.7560976,
+    "ex2": true,
+    "ex3": 0.9512195,
+    "ex4": 39,
+    "ex5": false,
+    "yes": false
+  }
+}
+```
+
+Example for blocking fully encrypted traffic:
+
+```yaml
+- name: Block suspicious proxy traffic
+  action: block
+  expr: fet != nil && fet.yes
+```
+
+## HTTP
+
+```json
+{
+  "http": {
+    "req": {
+      "headers": {
+        "accept": "*/*",
+        "host": "ipinfo.io",
+        "user-agent": "curl/7.81.0"
+      },
+      "method": "GET",
+      "path": "/",
+      "version": "HTTP/1.1"
+    },
+    "resp": {
+      "headers": {
+        "access-control-allow-origin": "*",
+        "content-length": "333",
+        "content-type": "application/json; charset=utf-8",
+        "date": "Wed, 24 Jan 2024 05:41:44 GMT",
+        "referrer-policy": "strict-origin-when-cross-origin",
+        "server": "nginx/1.24.0",
+        "strict-transport-security": "max-age=2592000; includeSubDomains",
+        "via": "1.1 google",
+        "x-content-type-options": "nosniff",
+        "x-envoy-upstream-service-time": "2",
+        "x-frame-options": "SAMEORIGIN",
+        "x-xss-protection": "1; mode=block"
+      },
+      "status": 200,
+      "version": "HTTP/1.1"
+    }
+  }
+}
+```
+
+Example for blocking HTTP requests to `ipinfo.io`:
+
+```yaml
+- name: Block ipinfo.io HTTP
+  action: block
+  expr: http != nil && http.req != nil && http.req.headers != nil && http.req.headers.host == "ipinfo.io"
+```
+
+## SSH
+
+```json
+{
+  "ssh": {
+    "server": {
+      "comments": "Ubuntu-3ubuntu0.6",
+      "protocol": "2.0",
+      "software": "OpenSSH_8.9p1"
+    },
+    "client": {
+      "comments": "IMHACKER",
+      "protocol": "2.0",
+      "software": "OpenSSH_8.9p1"
+    }
+  }
+}
+```
+
+Example for blocking all SSH connections:
+
+```yaml
+- name: Block SSH
+  action: block
+  expr: ssh != nil
+```
+
+## TLS
+
+```json
+{
+  "tls": {
+    "req": {
+      "alpn": [
+        "h2",
+        "http/1.1"
+      ],
+      "ciphers": [
+        4866,
+        4867,
+        4865,
+        49196,
+        49200,
+        159,
+        52393,
+        52392,
+        52394,
+        49195,
+        49199,
+        158,
+        49188,
+        49192,
+        107,
+        49187,
+        49191,
+        103,
+        49162,
+        49172,
+        57,
+        49161,
+        49171,
+        51,
+        157,
+        156,
+        61,
+        60,
+        53,
+        47,
+        255
+      ],
+      "compression": "AA==",
+      "random": "UqfPi+EmtMgusILrKcELvVWwpOdPSM/My09nPXl84dg=",
+      "session": "jCTrpAzHpwrfuYdYx4FEjZwbcQxCuZ52HGIoOcbw1vA=",
+      "sni": "ipinfo.io",
+      "supported_versions": [
+        772,
+        771
+      ],
+      "version": 771
+    },
+    "resp": {
+      "cipher": 4866,
+      "compression": 0,
+      "random": "R/Cy1m9pktuBMZQIHahD8Y83UWPRf8j8luwNQep9yJI=",
+      "session": "jCTrpAzHpwrfuYdYx4FEjZwbcQxCuZ52HGIoOcbw1vA=",
+      "supported_versions": 772,
+      "version": 771
+    }
+  }
+}
+```
+
+Example for blocking TLS connections to `ipinfo.io`:
+
+```yaml
+- name: Block ipinfo.io TLS
+  action: block
+  expr: tls != nil && tls.req != nil && tls.req.sni == "ipinfo.io"
+```
+
+## Trojan (proxy protocol)
+
+Check https://github.com/XTLS/Trojan-killer for more information.
+
+```json
+{
+  "trojan": {
+    "down": 4712,
+    "up": 671,
+    "yes": true
+  }
+}
+```
+
+Example for blocking Trojan connections:
+
+```yaml
+- name: Block Trojan
+  action: block
+  expr: trojan != nil && trojan.yes
+```