docs: analyzers

This commit is contained in:
Toby 2024-01-24 20:01:53 -08:00
parent 7441d24aea
commit d7d3437d3c
3 changed files with 282 additions and 12 deletions

View File

@ -3,6 +3,7 @@
[![License][1]][2] [![License][1]][2]
[1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg [1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
[2]: LICENSE [2]: LICENSE
**[中文文档](README.zh.md)** **[中文文档](README.zh.md)**
@ -19,11 +20,11 @@ Linux that's in many ways more powerful than the real thing. It's cyber sovereig
## Features ## Features
- Full IP/TCP reassembly, various protocol analyzers - Full IP/TCP reassembly, various protocol analyzers
- HTTP, TLS, DNS, SSH, and many more to come - HTTP, TLS, DNS, SSH, and many more to come
- "Fully encrypted traffic" detection for Shadowsocks, - "Fully encrypted traffic" detection for Shadowsocks,
etc. (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf) etc. (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
- Trojan (proxy protocol) detection based on Trojan-killer (https://github.com/XTLS/Trojan-killer) - Trojan (proxy protocol) detection based on Trojan-killer (https://github.com/XTLS/Trojan-killer)
- [WIP] Machine learning based traffic classification - [WIP] Machine learning based traffic classification
- Full IPv4 and IPv6 support - Full IPv4 and IPv6 support
- Flow-based multicore load balancing - Flow-based multicore load balancing
- Connection offloading - Connection offloading
@ -72,8 +73,7 @@ workers:
### Example rules ### Example rules
Documentation on all supported protocols and what field each one has is not yet ready. For now, you have to check the [Analyzer properties](docs/Analyzers.md)
code under "analyzer" directory directly.
For syntax of the expression language, please refer For syntax of the expression language, please refer
to [Expr Language Definition](https://expr-lang.org/docs/language-definition). to [Expr Language Definition](https://expr-lang.org/docs/language-definition).

View File

@ -3,6 +3,7 @@
[![License][1]][2] [![License][1]][2]
[1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg [1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
[2]: LICENSE [2]: LICENSE
OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedia.org/wiki/%E9%98%B2%E7%81%AB%E9%95%BF%E5%9F%8E) OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedia.org/wiki/%E9%98%B2%E7%81%AB%E9%95%BF%E5%9F%8E)
@ -17,10 +18,10 @@ OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedi
## 功能 ## 功能
- 完整的 IP/TCP 重组,各种协议解析器 - 完整的 IP/TCP 重组,各种协议解析器
- HTTP, TLS, DNS, SSH, 更多协议正在开发中 - HTTP, TLS, DNS, SSH, 更多协议正在开发中
- Shadowsocks 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf) - Shadowsocks 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
- 基于 Trojan-killer 的 Trojan 检测 (https://github.com/XTLS/Trojan-killer) - 基于 Trojan-killer 的 Trojan 检测 (https://github.com/XTLS/Trojan-killer)
- [开发中] 基于机器学习的流量分类 - [开发中] 基于机器学习的流量分类
- 同等支持 IPv4 和 IPv6 - 同等支持 IPv4 和 IPv6
- 基于流的多核负载均衡 - 基于流的多核负载均衡
- 连接 offloading - 连接 offloading
@ -69,7 +70,7 @@ workers:
### 样例规则 ### 样例规则
关于规则具体支持哪些协议,以及每个协议包含哪些字段的文档还没有写。目前请直接参考 "analyzer" 目录下的代码。 [解析器属性](docs/Analyzers.md)
规则的语法请参考 [Expr Language Definition](https://expr-lang.org/docs/language-definition)。 规则的语法请参考 [Expr Language Definition](https://expr-lang.org/docs/language-definition)。

269
docs/Analyzers.md Normal file
View File

@ -0,0 +1,269 @@
# Analyzers
Analyzers are one of the main components of OpenGFW. Their job is to analyze a connection, see if it's a protocol they
support, and if so, extract information from that connection and provide properties for the rule engine to match against
user-provided rules. OpenGFW will automatically analyze which analyzers are referenced in the given rules and enable
only those that are needed.
This document lists the properties provided by each analyzer that can be used by rules.
## DNS (TCP & UDP)
For queries:
```json
{
"dns": {
"aa": false,
"id": 41953,
"opcode": 0,
"qr": false,
"questions": [
{
"class": 1,
"name": "www.google.com",
"type": 1
}
],
"ra": false,
"rcode": 0,
"rd": true,
"tc": false,
"z": 0
}
}
```
For responses:
```json
{
"dns": {
"aa": false,
"answers": [
{
"a": "142.251.32.36",
"class": 1,
"name": "www.google.com",
"ttl": 255,
"type": 1
}
],
"id": 41953,
"opcode": 0,
"qr": true,
"questions": [
{
"class": 1,
"name": "www.google.com",
"type": 1
}
],
"ra": true,
"rcode": 0,
"rd": true,
"tc": false,
"z": 0
}
}
```
Example for blocking DNS queries for `www.google.com`:
```yaml
- name: Block Google DNS
action: drop
expr: dns != nil && !dns.qr && any(dns.questions, {.name == "www.google.com"})
```
## FET (Fully Encrypted Traffic)
Check https://www.usenix.org/system/files/usenixsecurity23-wu-mingshi.pdf for more information.
```json
{
"fet": {
"ex1": 3.7560976,
"ex2": true,
"ex3": 0.9512195,
"ex4": 39,
"ex5": false,
"yes": false
}
}
```
Example for blocking fully encrypted traffic:
```yaml
- name: Block suspicious proxy traffic
action: block
expr: fet != nil && fet.yes
```
## HTTP
```json
{
"http": {
"req": {
"headers": {
"accept": "*/*",
"host": "ipinfo.io",
"user-agent": "curl/7.81.0"
},
"method": "GET",
"path": "/",
"version": "HTTP/1.1"
},
"resp": {
"headers": {
"access-control-allow-origin": "*",
"content-length": "333",
"content-type": "application/json; charset=utf-8",
"date": "Wed, 24 Jan 2024 05:41:44 GMT",
"referrer-policy": "strict-origin-when-cross-origin",
"server": "nginx/1.24.0",
"strict-transport-security": "max-age=2592000; includeSubDomains",
"via": "1.1 google",
"x-content-type-options": "nosniff",
"x-envoy-upstream-service-time": "2",
"x-frame-options": "SAMEORIGIN",
"x-xss-protection": "1; mode=block"
},
"status": 200,
"version": "HTTP/1.1"
}
}
}
```
Example for blocking HTTP requests to `ipinfo.io`:
```yaml
- name: Block ipinfo.io HTTP
action: block
expr: http != nil && http.req != nil && http.req.headers != nil && http.req.headers.host == "ipinfo.io"
```
## SSH
```json
{
"ssh": {
"server": {
"comments": "Ubuntu-3ubuntu0.6",
"protocol": "2.0",
"software": "OpenSSH_8.9p1"
},
"client": {
"comments": "IMHACKER",
"protocol": "2.0",
"software": "OpenSSH_8.9p1"
}
}
}
```
Example for blocking all SSH connections:
```yaml
- name: Block SSH
action: block
expr: ssh != nil
```
## TLS
```json
{
"tls": {
"req": {
"alpn": [
"h2",
"http/1.1"
],
"ciphers": [
4866,
4867,
4865,
49196,
49200,
159,
52393,
52392,
52394,
49195,
49199,
158,
49188,
49192,
107,
49187,
49191,
103,
49162,
49172,
57,
49161,
49171,
51,
157,
156,
61,
60,
53,
47,
255
],
"compression": "AA==",
"random": "UqfPi+EmtMgusILrKcELvVWwpOdPSM/My09nPXl84dg=",
"session": "jCTrpAzHpwrfuYdYx4FEjZwbcQxCuZ52HGIoOcbw1vA=",
"sni": "ipinfo.io",
"supported_versions": [
772,
771
],
"version": 771
},
"resp": {
"cipher": 4866,
"compression": 0,
"random": "R/Cy1m9pktuBMZQIHahD8Y83UWPRf8j8luwNQep9yJI=",
"session": "jCTrpAzHpwrfuYdYx4FEjZwbcQxCuZ52HGIoOcbw1vA=",
"supported_versions": 772,
"version": 771
}
}
}
```
Example for blocking TLS connections to `ipinfo.io`:
```yaml
- name: Block ipinfo.io TLS
action: block
expr: tls != nil && tls.req != nil && tls.req.sni == "ipinfo.io"
```
## Trojan (proxy protocol)
Check https://github.com/XTLS/Trojan-killer for more information.
```json
{
"trojan": {
"down": 4712,
"up": 671,
"yes": true
}
}
```
Example for blocking Trojan connections:
```yaml
- name: Block Trojan
action: block
expr: trojan != nil && trojan.yes
```