MigrateRepos/OpenGFW
1
0
Fork 0
mirror of https://github.com/apernet/OpenGFW.git synced 2025-04-20 03:49:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: MigrateRepos:master
Branches Tags
MigrateRepos:master
MigrateRepos:wip-noenobufs
MigrateRepos:v0.4.1
MigrateRepos:v0.4.0
MigrateRepos:v0.3.4
MigrateRepos:v0.3.3
MigrateRepos:v0.3.2
MigrateRepos:v0.3.1
MigrateRepos:v0.3.0
MigrateRepos:v0.2.3
MigrateRepos:v0.2.2-b
MigrateRepos:v0.2.2
MigrateRepos:v0.2.1
MigrateRepos:v0.2.0
MigrateRepos:v0.1.1
MigrateRepos:v0.1.0
MigrateRepos:v0.0.5
MigrateRepos:v0.0.4
MigrateRepos:v0.0.3
MigrateRepos:v0.0.2
MigrateRepos:v0.0.1
..
compare: MigrateRepos:v0.0.2
Branches Tags
MigrateRepos:master
MigrateRepos:wip-noenobufs
MigrateRepos:v0.4.1
MigrateRepos:v0.4.0
MigrateRepos:v0.3.4
MigrateRepos:v0.3.3
MigrateRepos:v0.3.2
MigrateRepos:v0.3.1
MigrateRepos:v0.3.0
MigrateRepos:v0.2.3
MigrateRepos:v0.2.2-b
MigrateRepos:v0.2.2
MigrateRepos:v0.2.1
MigrateRepos:v0.2.0
MigrateRepos:v0.1.1
MigrateRepos:v0.1.0
MigrateRepos:v0.0.5
MigrateRepos:v0.0.4
MigrateRepos:v0.0.3
MigrateRepos:v0.0.2
MigrateRepos:v0.0.1

No commits in common. "master" and "v0.0.2" have entirely different histories.
master ... v0.0.2

47 changed files with 651 additions and 5246 deletions
Show Stats Expand all files Collapse all files

47
.github/workflows/check.yaml vendored
View File

@ -1,47 +0,0 @@
name: Quality check
on:
push:
branches:
- "*"
pull_request:
permissions:
contents: read
jobs:
static-analysis:
name: Static analysis
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: 'stable'
- run: go vet ./...
- name: staticcheck
uses: dominikh/staticcheck-action@v1.3.0
with:
install-go: false
tests:
name: Tests
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: 'stable'
- run: go test ./...

33
.github/workflows/release.yaml vendored
View File

@ -1,7 +1,6 @@
name: Release
on:
release:
types: [published]
types: [ created ]
permissions:
contents: write
@ -13,27 +12,15 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
goos: [linux]
goarch: ["386", amd64, arm64]
goos: [ linux ]
goarch: [ "386", amd64, arm64 ]
steps:
- uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
- uses: wangyoucao577/go-release-action@v1
with:
go-version: "1.22"
- name: Build
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
CGO_ENABLED: 0
run: |
mkdir -p build
go build -o build/OpenGFW-${GOOS}-${GOARCH} -ldflags "-s -w" .
- name: Upload
uses: softprops/action-gh-release@v2
with:
files: build/*
github_token: ${{ secrets.GITHUB_TOKEN }}
goos: ${{ matrix.goos }}
goarch: ${{ matrix.goarch }}
goversion: "https://go.dev/dl/go1.21.6.linux-amd64.tar.gz"
binary_name: "OpenGFW"
extra_files: LICENSE README.md README.zh.md

3
.gitignore vendored
View File

@ -205,6 +205,3 @@ $RECYCLE.BIN/
*.lnk
# End of https://www.toptal.com/developers/gitignore/api/windows,macos,linux,go,goland+all,visualstudiocode
# Internal tools not ready for public use yet
tools/flowseq/

41
README.ja.md
View File

@ -1,41 +0,0 @@
# ![OpenGFW](docs/logo.png)
[![Quality check status](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml/badge.svg)](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml)
[![License][1]][2]
[1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
[2]: LICENSE
OpenGFW は、あなた専用の DIY 中国のグレートファイアウォール (https://en.wikipedia.org/wiki/Great_Firewall) です。Linux 上で利用可能な柔軟で使いやすいオープンソースプログラムとして提供されています。なぜ権力者だけが楽しむのでしょうか？権力を人々に与え、検閲を民主化する時が来ました。自宅のルーターにサイバー主権のスリルをもたらし、プロのようにフィルタリングを始めましょう - あなたもビッグブラザーになることができます。
**ドキュメントウェブサイト: https://gfw.dev/**
Telegram グループ: https://t.me/OpGFW
> [!CAUTION]
> 本プロジェクトはまだ初期開発段階にあります。テスト時のリスクは自己責任でお願いします。私たちは、このプロジェクトを一緒に改善するために貢献者を探しています。
## 特徴
- フル IP/TCP 再アセンブル、各種プロトコルアナライザー
- HTTP、TLS、QUIC、DNS、SSH、SOCKS4/5、WireGuard、OpenVPN、その他多数
- Shadowsocks、VMess の「完全に暗号化されたトラフィック」の検出など (https://gfw.report/publications/usenixsecurity23/en/)
- Trojan プロキシプロトコルの検出
- [WIP] 機械学習に基づくトラフィック分類
- IPv4 と IPv6 をフルサポート
- フローベースのマルチコア負荷分散
- 接続オフロード
- [expr](https://github.com/expr-lang/expr) に基づく強力なルールエンジン
- ルールのホットリロード (`SIGHUP` を送信してリロード)
- 柔軟なアナライザ＆モディファイアフレームワーク
- 拡張可能な IO 実装 (今のところ NFQueue のみ)
- [WIP] ウェブ UI
## ユースケース
- 広告ブロック
- ペアレンタルコントロール
- マルウェア対策
- VPN/プロキシサービスの不正利用防止
- トラフィック分析（ログのみモード）
- 独裁的な野心を実現するのを助ける

102
README.md
View File

@ -1,36 +1,33 @@
# ![OpenGFW](docs/logo.png)
[![Quality check status](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml/badge.svg)](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml)
[![License][1]][2]
[1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
[2]: LICENSE
**[中文文档](README.zh.md)**
**[日本語ドキュメント](README.ja.md)**
OpenGFW is your very own DIY Great Firewall of China (https://en.wikipedia.org/wiki/Great_Firewall), available as a flexible, easy-to-use open source program on Linux. Why let the powers that be have all the fun? It's time to give power to the people and democratize censorship. Bring the thrill of cyber-sovereignty right into your home router and start filtering like a pro - you too can play Big Brother.
**Documentation site: https://gfw.dev/**
Telegram group: https://t.me/OpGFW
OpenGFW is a flexible, easy-to-use, open source implementation of [GFW](https://en.wikipedia.org/wiki/Great_Firewall) on
Linux that's in many ways more powerful than the real thing. It's cyber sovereignty you can have on a home router.
> [!CAUTION]
> This project is still in very early stages of development. Use at your own risk. We are looking for contributors to help us improve and expand the project.
> This project is still in very early stages of development. Use at your own risk.
> [!NOTE]
> We are looking for contributors to help us with this project, especially implementing analyzers for more protocols!!!
## Features
- Full IP/TCP reassembly, various protocol analyzers
- HTTP, TLS, QUIC, DNS, SSH, SOCKS4/5, WireGuard, OpenVPN, and many more to come
- "Fully encrypted traffic" detection for Shadowsocks, VMess,
etc. (https://gfw.report/publications/usenixsecurity23/en/)
- Trojan (proxy protocol) detection
- [WIP] Machine learning based traffic classification
- Full IPv4 and IPv6 support
- HTTP, TLS, DNS, SSH, and many more to come
- "Fully encrypted traffic" detection for Shadowsocks,
etc. (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
- Trojan (proxy protocol) detection based on Trojan-killer (https://github.com/XTLS/Trojan-killer)
- [WIP] Machine learning based traffic classification
- Flow-based multicore load balancing
- Connection offloading
- Powerful rule engine based on [expr](https://github.com/expr-lang/expr)
- Hot-reloadable rules (send `SIGHUP` to reload)
- Flexible analyzer & modifier framework
- Extensible IO implementation (only NFQueue for now)
- [WIP] Web UI
@ -42,4 +39,77 @@ Telegram group: https://t.me/OpGFW
- Malware protection
- Abuse prevention for VPN/proxy services
- Traffic analysis (log only mode)
- Help you fulfill your dictatorial ambitions
## Usage
### Build
```shell
go build
```
### Run
```shell
export OPENGFW_LOG_LEVEL=debug
./OpenGFW -c config.yaml rules.yaml
```
### Example config
```yaml
io:
queueSize: 1024
local: true # set to false if you want to run OpenGFW on FORWARD chain
workers:
count: 4
queueSize: 16
tcpMaxBufferedPagesTotal: 4096
tcpMaxBufferedPagesPerConn: 64
udpMaxStreams: 4096
```
### Example rules
Documentation on all supported protocols and what field each one has is not yet ready. For now, you have to check the
code under "analyzer" directory directly.
For syntax of the expression language, please refer
to [Expr Language Definition](https://expr-lang.org/docs/language-definition).
```yaml
- name: block v2ex http
action: block
expr: string(http?.req?.headers?.host) endsWith "v2ex.com"
- name: block v2ex https
action: block
expr: string(tls?.req?.sni) endsWith "v2ex.com"
- name: block shadowsocks
action: block
expr: fet != nil && fet.yes
- name: block trojan
action: block
expr: trojan != nil && trojan.yes
- name: v2ex dns poisoning
action: modify
modifier:
name: dns
args:
a: "0.0.0.0"
aaaa: "::"
expr: dns != nil && dns.qr && any(dns.questions, {.name endsWith "v2ex.com"})
```
#### Supported actions
- `allow`: Allow the connection, no further processing.
- `block`: Block the connection, no further processing. Send a TCP RST if it's a TCP connection.
- `drop`: For UDP, drop the packet that triggered the rule, continue processing future packets in the same flow. For
TCP, same as `block`.
- `modify`: For UDP, modify the packet that triggered the rule using the given modifier, continue processing future
packets in the same flow. For TCP, same as `allow`.

95
README.zh.md
View File

@ -1,32 +1,30 @@
# ![OpenGFW](docs/logo.png)
[![Quality check status](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml/badge.svg)](https://github.com/apernet/OpenGFW/actions/workflows/check.yaml)
[![License][1]][2]
[1]: https://img.shields.io/badge/License-MPL_2.0-brightgreen.svg
[2]: LICENSE
OpenGFW 是一个 Linux 上灵活、易用、开源的 DIY [GFW](https://zh.wikipedia.org/wiki/%E9%98%B2%E7%81%AB%E9%95%BF%E5%9F%8E) 实现，并且在许多方面比真正的 GFW 更强大。为何让那些掌权者独享乐趣？是时候把权力归还给人民，人人有墙建了。立即安装可以部署在家用路由器上的网络主权 - 你也能是老大哥。
**文档网站： https://gfw.dev/**
Telegram 群组： https://t.me/OpGFW
OpenGFW 是一个 Linux 上灵活、易用、开源的 [GFW](https://zh.wikipedia.org/wiki/%E9%98%B2%E7%81%AB%E9%95%BF%E5%9F%8E)
实现，并且在许多方面比真正的 GFW 更强大。可以部署在家用路由器上的网络主权。
> [!CAUTION]
> 本项目仍处于早期开发阶段。测试时自行承担风险。我们正在寻求贡献者一起完善本项目。
> 本项目仍处于早期开发阶段。测试时自行承担风险。
> [!NOTE]
> 我们正在寻求贡献者一起完善本项目，尤其是实现更多协议的解析器！
## 功能
- 完整的 IP/TCP 重组，各种协议解析器
- HTTP, TLS, QUIC, DNS, SSH, SOCKS4/5, WireGuard, OpenVPN, 更多协议正在开发中
- Shadowsocks, VMess 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/zh/)
- Trojan 协议检测
- [开发中] 基于机器学习的流量分类
- 同等支持 IPv4 和 IPv6
- HTTP, TLS, DNS, SSH, 更多协议正在开发中
- Shadowsocks 等 "全加密流量" 检测 (https://gfw.report/publications/usenixsecurity23/data/paper/paper.pdf)
- 基于 Trojan-killer 的 Trojan 检测 (https://github.com/XTLS/Trojan-killer)
- [开发中] 基于机器学习的流量分类
- 基于流的多核负载均衡
- 连接 offloading
- 基于 [expr](https://github.com/expr-lang/expr) 的强大规则引擎
- 规则可以热重载 (发送 `SIGHUP` 信号)
- 灵活的协议解析和修改框架
- 可扩展的 IO 实现 (目前只有 NFQueue)
- [开发中] Web UI
@ -38,4 +36,73 @@ Telegram 群组： https://t.me/OpGFW
- 恶意软件防护
- VPN/代理服务滥用防护
- 流量分析 (纯日志模式)
- 助你实现你的独裁野心
## 使用
### 构建
```shell
go build
```
### 运行
```shell
export OPENGFW_LOG_LEVEL=debug
./OpenGFW -c config.yaml rules.yaml
```
### 样例配置
```yaml
io:
queueSize: 1024
local: true # 如果需要在 FORWARD 链上运行 OpenGFW，请设置为 false
workers:
count: 4
queueSize: 16
tcpMaxBufferedPagesTotal: 4096
tcpMaxBufferedPagesPerConn: 64
udpMaxStreams: 4096
```
### 样例规则
关于规则具体支持哪些协议，以及每个协议包含哪些字段的文档还没有写。目前请直接参考 "analyzer" 目录下的代码。
规则的语法请参考 [Expr Language Definition](https://expr-lang.org/docs/language-definition)。
```yaml
- name: block v2ex http
action: block
expr: string(http?.req?.headers?.host) endsWith "v2ex.com"
- name: block v2ex https
action: block
expr: string(tls?.req?.sni) endsWith "v2ex.com"
- name: block shadowsocks
action: block
expr: fet != nil && fet.yes
- name: block trojan
action: block
expr: trojan != nil && trojan.yes
- name: v2ex dns poisoning
action: modify
modifier:
name: dns
args:
a: "0.0.0.0"
aaaa: "::"
expr: dns != nil && dns.qr && any(dns.questions, {.name endsWith "v2ex.com"})
```
#### 支持的 action
- `allow`: 放行连接，不再处理后续的包。
- `block`: 阻断连接，不再处理后续的包。如果是 TCP 连接，会发送 RST 包。
- `drop`: 对于 UDP，丢弃触发规则的包，但继续处理同一流中的后续包。对于 TCP，效果同 `block`。
- `modify`: 对于 UDP，用指定的修改器修改触发规则的包，然后继续处理同一流中的后续包。对于 TCP，效果同 `allow`。

224
analyzer/internal/tls.go
View File

@ -1,224 +0,0 @@
package internal
import (
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/utils"
)
// TLS record types.
const (
RecordTypeHandshake = 0x16
)
// TLS handshake message types.
const (
TypeClientHello = 0x01
TypeServerHello = 0x02
)
// TLS extension numbers.
const (
extServerName = 0x0000
extALPN = 0x0010
extSupportedVersions = 0x002b
extEncryptedClientHello = 0xfe0d
)
func ParseTLSClientHelloMsgData(chBuf *utils.ByteBuffer) analyzer.PropMap {
var ok bool
m := make(analyzer.PropMap)
// Version, random & session ID length combined are within 35 bytes,
// so no need for bounds checking
m["version"], _ = chBuf.GetUint16(false, true)
m["random"], _ = chBuf.Get(32, true)
sessionIDLen, _ := chBuf.GetByte(true)
m["session"], ok = chBuf.Get(int(sessionIDLen), true)
if !ok {
// Not enough data for session ID
return nil
}
cipherSuitesLen, ok := chBuf.GetUint16(false, true)
if !ok {
// Not enough data for cipher suites length
return nil
}
if cipherSuitesLen%2 != 0 {
// Cipher suites are 2 bytes each, so must be even
return nil
}
ciphers := make([]uint16, cipherSuitesLen/2)
for i := range ciphers {
ciphers[i], ok = chBuf.GetUint16(false, true)
if !ok {
return nil
}
}
m["ciphers"] = ciphers
compressionMethodsLen, ok := chBuf.GetByte(true)
if !ok {
// Not enough data for compression methods length
return nil
}
// Compression methods are 1 byte each, we just put a byte slice here
m["compression"], ok = chBuf.Get(int(compressionMethodsLen), true)
if !ok {
// Not enough data for compression methods
return nil
}
extsLen, ok := chBuf.GetUint16(false, true)
if !ok {
// No extensions, I guess it's possible?
return m
}
extBuf, ok := chBuf.GetSubBuffer(int(extsLen), true)
if !ok {
// Not enough data for extensions
return nil
}
for extBuf.Len() > 0 {
extType, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension type
return nil
}
extLen, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension length
return nil
}
extDataBuf, ok := extBuf.GetSubBuffer(int(extLen), true)
if !ok || !parseTLSExtensions(extType, extDataBuf, m) {
// Not enough data for extension data, or invalid extension
return nil
}
}
return m
}
func ParseTLSServerHelloMsgData(shBuf *utils.ByteBuffer) analyzer.PropMap {
var ok bool
m := make(analyzer.PropMap)
// Version, random & session ID length combined are within 35 bytes,
// so no need for bounds checking
m["version"], _ = shBuf.GetUint16(false, true)
m["random"], _ = shBuf.Get(32, true)
sessionIDLen, _ := shBuf.GetByte(true)
m["session"], ok = shBuf.Get(int(sessionIDLen), true)
if !ok {
// Not enough data for session ID
return nil
}
cipherSuite, ok := shBuf.GetUint16(false, true)
if !ok {
// Not enough data for cipher suite
return nil
}
m["cipher"] = cipherSuite
compressionMethod, ok := shBuf.GetByte(true)
if !ok {
// Not enough data for compression method
return nil
}
m["compression"] = compressionMethod
extsLen, ok := shBuf.GetUint16(false, true)
if !ok {
// No extensions, I guess it's possible?
return m
}
extBuf, ok := shBuf.GetSubBuffer(int(extsLen), true)
if !ok {
// Not enough data for extensions
return nil
}
for extBuf.Len() > 0 {
extType, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension type
return nil
}
extLen, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension length
return nil
}
extDataBuf, ok := extBuf.GetSubBuffer(int(extLen), true)
if !ok || !parseTLSExtensions(extType, extDataBuf, m) {
// Not enough data for extension data, or invalid extension
return nil
}
}
return m
}
func parseTLSExtensions(extType uint16, extDataBuf *utils.ByteBuffer, m analyzer.PropMap) bool {
switch extType {
case extServerName:
ok := extDataBuf.Skip(2) // Ignore list length, we only care about the first entry for now
if !ok {
// Not enough data for list length
return false
}
sniType, ok := extDataBuf.GetByte(true)
if !ok || sniType != 0 {
// Not enough data for SNI type, or not hostname
return false
}
sniLen, ok := extDataBuf.GetUint16(false, true)
if !ok {
// Not enough data for SNI length
return false
}
m["sni"], ok = extDataBuf.GetString(int(sniLen), true)
if !ok {
// Not enough data for SNI
return false
}
case extALPN:
ok := extDataBuf.Skip(2) // Ignore list length, as we read until the end
if !ok {
// Not enough data for list length
return false
}
var alpnList []string
for extDataBuf.Len() > 0 {
alpnLen, ok := extDataBuf.GetByte(true)
if !ok {
// Not enough data for ALPN length
return false
}
alpn, ok := extDataBuf.GetString(int(alpnLen), true)
if !ok {
// Not enough data for ALPN
return false
}
alpnList = append(alpnList, alpn)
}
m["alpn"] = alpnList
case extSupportedVersions:
if extDataBuf.Len() == 2 {
// Server only selects one version
m["supported_versions"], _ = extDataBuf.GetUint16(false, true)
} else {
// Client sends a list of versions
ok := extDataBuf.Skip(1) // Ignore list length, as we read until the end
if !ok {
// Not enough data for list length
return false
}
var versions []uint16
for extDataBuf.Len() > 0 {
ver, ok := extDataBuf.GetUint16(false, true)
if !ok {
// Not enough data for version
return false
}
versions = append(versions, ver)
}
m["supported_versions"] = versions
}
case extEncryptedClientHello:
// We can't parse ECH for now, just set a flag
m["ech"] = true
}
return true
}

7
analyzer/tcp/fet.go
View File

@ -143,11 +143,8 @@ func isTLSorHTTP(bytes []byte) bool {
if len(bytes) < 3 {
return false
}
// "We observe that the GFW exempts any connection whose first
// three bytes match the following regular expression:
// [\x16-\x17]\x03[\x00-\x09]" - from the paper in Section 4.3
if bytes[0] >= 0x16 && bytes[0] <= 0x17 &&
bytes[1] == 0x03 && bytes[2] <= 0x09 {
if bytes[0] == 0x16 && bytes[1] == 0x03 && bytes[2] <= 0x03 {
// TLS handshake for TLS 1.0-1.3
return true
}
// HTTP request

64
analyzer/tcp/http_test.go
View File

@ -1,64 +0,0 @@
package tcp
import (
"reflect"
"strings"
"testing"
"github.com/apernet/OpenGFW/analyzer"
)
func TestHTTPParsing_Request(t *testing.T) {
testCases := map[string]analyzer.PropMap{
"GET / HTTP/1.1\r\n": {
"method": "GET", "path": "/", "version": "HTTP/1.1",
},
"POST /hello?a=1&b=2 HTTP/1.0\r\n": {
"method": "POST", "path": "/hello?a=1&b=2", "version": "HTTP/1.0",
},
"PUT /world HTTP/1.1\r\nContent-Length: 4\r\n\r\nbody": {
"method": "PUT", "path": "/world", "version": "HTTP/1.1", "headers": analyzer.PropMap{"content-length": "4"},
},
"DELETE /goodbye HTTP/2.0\r\n": {
"method": "DELETE", "path": "/goodbye", "version": "HTTP/2.0",
},
}
for tc, want := range testCases {
t.Run(strings.Split(tc, " ")[0], func(t *testing.T) {
tc, want := tc, want
t.Parallel()
u, _ := newHTTPStream(nil).Feed(false, false, false, 0, []byte(tc))
got := u.M.Get("req")
if !reflect.DeepEqual(got, want) {
t.Errorf("\"%s\" parsed = %v, want %v", tc, got, want)
}
})
}
}
func TestHTTPParsing_Response(t *testing.T) {
testCases := map[string]analyzer.PropMap{
"HTTP/1.0 200 OK\r\nContent-Length: 4\r\n\r\nbody": {
"version": "HTTP/1.0", "status": 200,
"headers": analyzer.PropMap{"content-length": "4"},
},
"HTTP/2.0 204 No Content\r\n\r\n": {
"version": "HTTP/2.0", "status": 204,
},
}
for tc, want := range testCases {
t.Run(strings.Split(tc, " ")[0], func(t *testing.T) {
tc, want := tc, want
t.Parallel()
u, _ := newHTTPStream(nil).Feed(true, false, false, 0, []byte(tc))
got := u.M.Get("resp")
if !reflect.DeepEqual(got, want) {
t.Errorf("\"%s\" parsed = %v, want %v", tc, got, want)
}
})
}
}

508
analyzer/tcp/socks.go
View File

@ -1,508 +0,0 @@
package tcp
import (
"net"
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/utils"
)
const (
SocksInvalid = iota
Socks4
Socks4A
Socks5
Socks4Version = 0x04
Socks5Version = 0x05
Socks4ReplyVN = 0x00
Socks4CmdTCPConnect = 0x01
Socks4CmdTCPBind = 0x02
Socks4ReqGranted = 0x5A
Socks4ReqRejectOrFailed = 0x5B
Socks4ReqRejectIdentd = 0x5C
Socks4ReqRejectUser = 0x5D
Socks5CmdTCPConnect = 0x01
Socks5CmdTCPBind = 0x02
Socks5CmdUDPAssociate = 0x03
Socks5AuthNotRequired = 0x00
Socks5AuthPassword = 0x02
Socks5AuthNoMatchingMethod = 0xFF
Socks5AuthSuccess = 0x00
Socks5AuthFailure = 0x01
Socks5AddrTypeIPv4 = 0x01
Socks5AddrTypeDomain = 0x03
Socks5AddrTypeIPv6 = 0x04
)
var _ analyzer.Analyzer = (*SocksAnalyzer)(nil)
type SocksAnalyzer struct{}
func (a *SocksAnalyzer) Name() string {
return "socks"
}
func (a *SocksAnalyzer) Limit() int {
// Socks4 length limit cannot be predicted
return 0
}
func (a *SocksAnalyzer) NewTCP(info analyzer.TCPInfo, logger analyzer.Logger) analyzer.TCPStream {
return newSocksStream(logger)
}
type socksStream struct {
logger analyzer.Logger
reqBuf *utils.ByteBuffer
reqMap analyzer.PropMap
reqUpdated bool
reqLSM *utils.LinearStateMachine
reqDone bool
respBuf *utils.ByteBuffer
respMap analyzer.PropMap
respUpdated bool
respLSM *utils.LinearStateMachine
respDone bool
version int
authReqMethod int
authUsername string
authPassword string
authRespMethod int
}
func newSocksStream(logger analyzer.Logger) *socksStream {
s := &socksStream{logger: logger, reqBuf: &utils.ByteBuffer{}, respBuf: &utils.ByteBuffer{}}
s.reqLSM = utils.NewLinearStateMachine(
s.parseSocksReqVersion,
)
s.respLSM = utils.NewLinearStateMachine(
s.parseSocksRespVersion,
)
return s
}
func (s *socksStream) Feed(rev, start, end bool, skip int, data []byte) (u *analyzer.PropUpdate, d bool) {
if skip != 0 {
return nil, true
}
if len(data) == 0 {
return nil, false
}
var update *analyzer.PropUpdate
var cancelled bool
if rev {
s.respBuf.Append(data)
s.respUpdated = false
cancelled, s.respDone = s.respLSM.Run()
if s.respUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{"resp": s.respMap},
}
s.respUpdated = false
}
} else {
s.reqBuf.Append(data)
s.reqUpdated = false
cancelled, s.reqDone = s.reqLSM.Run()
if s.reqUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{
"version": s.socksVersion(),
"req": s.reqMap,
},
}
s.reqUpdated = false
}
}
return update, cancelled || (s.reqDone && s.respDone)
}
func (s *socksStream) Close(limited bool) *analyzer.PropUpdate {
s.reqBuf.Reset()
s.respBuf.Reset()
s.reqMap = nil
s.respMap = nil
return nil
}
func (s *socksStream) parseSocksReqVersion() utils.LSMAction {
socksVer, ok := s.reqBuf.GetByte(true)
if !ok {
return utils.LSMActionPause
}
if socksVer != Socks4Version && socksVer != Socks5Version {
return utils.LSMActionCancel
}
s.reqMap = make(analyzer.PropMap)
s.reqUpdated = true
if socksVer == Socks4Version {
s.version = Socks4
s.reqLSM.AppendSteps(
s.parseSocks4ReqIpAndPort,
s.parseSocks4ReqUserId,
s.parseSocks4ReqHostname,
)
} else {
s.version = Socks5
s.reqLSM.AppendSteps(
s.parseSocks5ReqMethod,
s.parseSocks5ReqAuth,
s.parseSocks5ReqConnInfo,
)
}
return utils.LSMActionNext
}
func (s *socksStream) parseSocksRespVersion() utils.LSMAction {
socksVer, ok := s.respBuf.GetByte(true)
if !ok {
return utils.LSMActionPause
}
if (s.version == Socks4 || s.version == Socks4A) && socksVer != Socks4ReplyVN ||
s.version == Socks5 && socksVer != Socks5Version || s.version == SocksInvalid {
return utils.LSMActionCancel
}
if socksVer == Socks4ReplyVN {
s.respLSM.AppendSteps(
s.parseSocks4RespPacket,
)
} else {
s.respLSM.AppendSteps(
s.parseSocks5RespMethod,
s.parseSocks5RespAuth,
s.parseSocks5RespConnInfo,
)
}
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5ReqMethod() utils.LSMAction {
nMethods, ok := s.reqBuf.GetByte(false)
if !ok {
return utils.LSMActionPause
}
methods, ok := s.reqBuf.Get(int(nMethods)+1, true)
if !ok {
return utils.LSMActionPause
}
// For convenience, we only take the first method we can process
s.authReqMethod = Socks5AuthNoMatchingMethod
for _, method := range methods[1:] {
switch method {
case Socks5AuthNotRequired:
s.authReqMethod = Socks5AuthNotRequired
return utils.LSMActionNext
case Socks5AuthPassword:
s.authReqMethod = Socks5AuthPassword
return utils.LSMActionNext
default:
// TODO: more auth method to support
}
}
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5ReqAuth() utils.LSMAction {
switch s.authReqMethod {
case Socks5AuthNotRequired:
s.reqMap["auth"] = analyzer.PropMap{"method": s.authReqMethod}
case Socks5AuthPassword:
meta, ok := s.reqBuf.Get(2, false)
if !ok {
return utils.LSMActionPause
}
if meta[0] != 0x01 {
return utils.LSMActionCancel
}
usernameLen := int(meta[1])
meta, ok = s.reqBuf.Get(usernameLen+3, false)
if !ok {
return utils.LSMActionPause
}
passwordLen := int(meta[usernameLen+2])
meta, ok = s.reqBuf.Get(usernameLen+passwordLen+3, true)
if !ok {
return utils.LSMActionPause
}
s.authUsername = string(meta[2 : usernameLen+2])
s.authPassword = string(meta[usernameLen+3:])
s.reqMap["auth"] = analyzer.PropMap{
"method": s.authReqMethod,
"username": s.authUsername,
"password": s.authPassword,
}
default:
return utils.LSMActionCancel
}
s.reqUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5ReqConnInfo() utils.LSMAction {
/* preInfo struct
+----+-----+-------+------+-------------+
|VER | CMD | RSV | ATYP | DST.ADDR(1) |
+----+-----+-------+------+-------------+
*/
preInfo, ok := s.reqBuf.Get(5, false)
if !ok {
return utils.LSMActionPause
}
// verify socks version
if preInfo[0] != Socks5Version {
return utils.LSMActionCancel
}
var pktLen int
switch int(preInfo[3]) {
case Socks5AddrTypeIPv4:
pktLen = 10
case Socks5AddrTypeDomain:
domainLen := int(preInfo[4])
pktLen = 7 + domainLen
case Socks5AddrTypeIPv6:
pktLen = 22
default:
return utils.LSMActionCancel
}
pkt, ok := s.reqBuf.Get(pktLen, true)
if !ok {
return utils.LSMActionPause
}
// parse cmd
cmd := int(pkt[1])
if cmd != Socks5CmdTCPConnect && cmd != Socks5CmdTCPBind && cmd != Socks5CmdUDPAssociate {
return utils.LSMActionCancel
}
s.reqMap["cmd"] = cmd
// parse addr type
addrType := int(pkt[3])
var addr string
switch addrType {
case Socks5AddrTypeIPv4:
addr = net.IPv4(pkt[4], pkt[5], pkt[6], pkt[7]).String()
case Socks5AddrTypeDomain:
addr = string(pkt[5 : 5+pkt[4]])
case Socks5AddrTypeIPv6:
addr = net.IP(pkt[4 : 4+net.IPv6len]).String()
default:
return utils.LSMActionCancel
}
s.reqMap["addr_type"] = addrType
s.reqMap["addr"] = addr
// parse port
port := int(pkt[pktLen-2])<<8 | int(pkt[pktLen-1])
s.reqMap["port"] = port
s.reqUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5RespMethod() utils.LSMAction {
method, ok := s.respBuf.Get(1, true)
if !ok {
return utils.LSMActionPause
}
s.authRespMethod = int(method[0])
s.respMap = make(analyzer.PropMap)
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5RespAuth() utils.LSMAction {
switch s.authRespMethod {
case Socks5AuthNotRequired:
s.respMap["auth"] = analyzer.PropMap{"method": s.authRespMethod}
case Socks5AuthPassword:
authResp, ok := s.respBuf.Get(2, true)
if !ok {
return utils.LSMActionPause
}
if authResp[0] != 0x01 {
return utils.LSMActionCancel
}
authStatus := int(authResp[1])
s.respMap["auth"] = analyzer.PropMap{
"method": s.authRespMethod,
"status": authStatus,
}
default:
return utils.LSMActionCancel
}
s.respUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks5RespConnInfo() utils.LSMAction {
/* preInfo struct
+----+-----+-------+------+-------------+
|VER | REP | RSV | ATYP | BND.ADDR(1) |
+----+-----+-------+------+-------------+
*/
preInfo, ok := s.respBuf.Get(5, false)
if !ok {
return utils.LSMActionPause
}
// verify socks version
if preInfo[0] != Socks5Version {
return utils.LSMActionCancel
}
var pktLen int
switch int(preInfo[3]) {
case Socks5AddrTypeIPv4:
pktLen = 10
case Socks5AddrTypeDomain:
domainLen := int(preInfo[4])
pktLen = 7 + domainLen
case Socks5AddrTypeIPv6:
pktLen = 22
default:
return utils.LSMActionCancel
}
pkt, ok := s.respBuf.Get(pktLen, true)
if !ok {
return utils.LSMActionPause
}
// parse rep
rep := int(pkt[1])
s.respMap["rep"] = rep
// parse addr type
addrType := int(pkt[3])
var addr string
switch addrType {
case Socks5AddrTypeIPv4:
addr = net.IPv4(pkt[4], pkt[5], pkt[6], pkt[7]).String()
case Socks5AddrTypeDomain:
addr = string(pkt[5 : 5+pkt[4]])
case Socks5AddrTypeIPv6:
addr = net.IP(pkt[4 : 4+net.IPv6len]).String()
default:
return utils.LSMActionCancel
}
s.respMap["addr_type"] = addrType
s.respMap["addr"] = addr
// parse port
port := int(pkt[pktLen-2])<<8 | int(pkt[pktLen-1])
s.respMap["port"] = port
s.respUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks4ReqIpAndPort() utils.LSMAction {
/* Following field will be parsed in this state:
+-----+----------+--------+
| CMD | DST.PORT | DST.IP |
+-----+----------+--------+
*/
pkt, ok := s.reqBuf.Get(7, true)
if !ok {
return utils.LSMActionPause
}
if pkt[0] != Socks4CmdTCPConnect && pkt[0] != Socks4CmdTCPBind {
return utils.LSMActionCancel
}
dstPort := uint16(pkt[1])<<8 | uint16(pkt[2])
dstIp := net.IPv4(pkt[3], pkt[4], pkt[5], pkt[6]).String()
// Socks4a extension
if pkt[3] == 0 && pkt[4] == 0 && pkt[5] == 0 {
s.version = Socks4A
}
s.reqMap["cmd"] = pkt[0]
s.reqMap["addr"] = dstIp
s.reqMap["addr_type"] = Socks5AddrTypeIPv4
s.reqMap["port"] = dstPort
s.reqUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks4ReqUserId() utils.LSMAction {
userIdSlice, ok := s.reqBuf.GetUntil([]byte("\x00"), true, true)
if !ok {
return utils.LSMActionPause
}
userId := string(userIdSlice[:len(userIdSlice)-1])
s.reqMap["auth"] = analyzer.PropMap{
"user_id": userId,
}
s.reqUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks4ReqHostname() utils.LSMAction {
// Only Socks4a support hostname
if s.version != Socks4A {
return utils.LSMActionNext
}
hostnameSlice, ok := s.reqBuf.GetUntil([]byte("\x00"), true, true)
if !ok {
return utils.LSMActionPause
}
hostname := string(hostnameSlice[:len(hostnameSlice)-1])
s.reqMap["addr"] = hostname
s.reqMap["addr_type"] = Socks5AddrTypeDomain
s.reqUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) parseSocks4RespPacket() utils.LSMAction {
pkt, ok := s.respBuf.Get(7, true)
if !ok {
return utils.LSMActionPause
}
if pkt[0] != Socks4ReqGranted &&
pkt[0] != Socks4ReqRejectOrFailed &&
pkt[0] != Socks4ReqRejectIdentd &&
pkt[0] != Socks4ReqRejectUser {
return utils.LSMActionCancel
}
dstPort := uint16(pkt[1])<<8 | uint16(pkt[2])
dstIp := net.IPv4(pkt[3], pkt[4], pkt[5], pkt[6]).String()
s.respMap = analyzer.PropMap{
"rep": pkt[0],
"addr": dstIp,
"addr_type": Socks5AddrTypeIPv4,
"port": dstPort,
}
s.respUpdated = true
return utils.LSMActionNext
}
func (s *socksStream) socksVersion() int {
switch s.version {
case Socks4, Socks4A:
return Socks4Version
case Socks5:
return Socks5Version
default:
return SocksInvalid
}
}

314
analyzer/tcp/tls.go
View File

@ -2,7 +2,6 @@ package tcp
import (
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/internal"
"github.com/apernet/OpenGFW/analyzer/utils"
)
@ -44,12 +43,12 @@ type tlsStream struct {
func newTLSStream(logger analyzer.Logger) *tlsStream {
s := &tlsStream{logger: logger, reqBuf: &utils.ByteBuffer{}, respBuf: &utils.ByteBuffer{}}
s.reqLSM = utils.NewLinearStateMachine(
s.tlsClientHelloPreprocess,
s.parseClientHelloData,
s.tlsClientHelloSanityCheck,
s.parseClientHello,
)
s.respLSM = utils.NewLinearStateMachine(
s.tlsServerHelloPreprocess,
s.parseServerHelloData,
s.tlsServerHelloSanityCheck,
s.parseServerHello,
)
return s
}
@ -89,132 +88,261 @@ func (s *tlsStream) Feed(rev, start, end bool, skip int, data []byte) (u *analyz
return update, cancelled || (s.reqDone && s.respDone)
}
// tlsClientHelloPreprocess validates ClientHello message.
//
// During validation, message header and first handshake header may be removed
// from `s.reqBuf`.
func (s *tlsStream) tlsClientHelloPreprocess() utils.LSMAction {
// headers size: content type (1 byte) + legacy protocol version (2 bytes) +
// + content length (2 bytes) + message type (1 byte) +
// + handshake length (3 bytes)
const headersSize = 9
// minimal data size: protocol version (2 bytes) + random (32 bytes) +
// + session ID (1 byte) + cipher suites (4 bytes) +
// + compression methods (2 bytes) + no extensions
const minDataSize = 41
header, ok := s.reqBuf.Get(headersSize, true)
func (s *tlsStream) tlsClientHelloSanityCheck() utils.LSMAction {
data, ok := s.reqBuf.Get(9, true)
if !ok {
// not a full header yet
return utils.LSMActionPause
}
if header[0] != internal.RecordTypeHandshake || header[5] != internal.TypeClientHello {
if data[0] != 0x16 || data[5] != 0x01 {
// Not a TLS handshake, or not a client hello
return utils.LSMActionCancel
}
s.clientHelloLen = int(header[6])<<16 | int(header[7])<<8 | int(header[8])
if s.clientHelloLen < minDataSize {
s.clientHelloLen = int(data[6])<<16 | int(data[7])<<8 | int(data[8])
if s.clientHelloLen < 41 {
// 2 (Protocol Version) +
// 32 (Random) +
// 1 (Session ID Length) +
// 2 (Cipher Suites Length) +_ws.col.protocol == "TLSv1.3"
// 2 (Cipher Suite) +
// 1 (Compression Methods Length) +
// 1 (Compression Method) +
// No extensions
// This should be the bare minimum for a client hello
return utils.LSMActionCancel
}
// TODO: something is missing. See:
// const messageHeaderSize = 4
// fullMessageLen := int(header[3])<<8 | int(header[4])
// msgNo := fullMessageLen / int(messageHeaderSize+s.serverHelloLen)
// if msgNo != 1 {
// // what here?
// }
// if messageNo != int(messageNo) {
// // what here?
// }
return utils.LSMActionNext
}
// tlsServerHelloPreprocess validates ServerHello message.
//
// During validation, message header and first handshake header may be removed
// from `s.reqBuf`.
func (s *tlsStream) tlsServerHelloPreprocess() utils.LSMAction {
// header size: content type (1 byte) + legacy protocol version (2 byte) +
// + content length (2 byte) + message type (1 byte) +
// + handshake length (3 byte)
const headersSize = 9
// minimal data size: server version (2 byte) + random (32 byte) +
// + session ID (>=1 byte) + cipher suite (2 byte) +
// + compression method (1 byte) + no extensions
const minDataSize = 38
header, ok := s.respBuf.Get(headersSize, true)
func (s *tlsStream) tlsServerHelloSanityCheck() utils.LSMAction {
data, ok := s.respBuf.Get(9, true)
if !ok {
// not a full header yet
return utils.LSMActionPause
}
if header[0] != internal.RecordTypeHandshake || header[5] != internal.TypeServerHello {
if data[0] != 0x16 || data[5] != 0x02 {
// Not a TLS handshake, or not a server hello
return utils.LSMActionCancel
}
s.serverHelloLen = int(header[6])<<16 | int(header[7])<<8 | int(header[8])
if s.serverHelloLen < minDataSize {
s.serverHelloLen = int(data[6])<<16 | int(data[7])<<8 | int(data[8])
if s.serverHelloLen < 38 {
// 2 (Protocol Version) +
// 32 (Random) +
// 1 (Session ID Length) +
// 2 (Cipher Suite) +
// 1 (Compression Method) +
// No extensions
// This should be the bare minimum for a server hello
return utils.LSMActionCancel
}
// TODO: something is missing. See example:
// const messageHeaderSize = 4
// fullMessageLen := int(header[3])<<8 | int(header[4])
// msgNo := fullMessageLen / int(messageHeaderSize+s.serverHelloLen)
// if msgNo != 1 {
// // what here?
// }
// if messageNo != int(messageNo) {
// // what here?
// }
return utils.LSMActionNext
}
// parseClientHelloData converts valid ClientHello message data (without
// headers) into `analyzer.PropMap`.
//
// Parsing error may leave `s.reqBuf` in an unusable state.
func (s *tlsStream) parseClientHelloData() utils.LSMAction {
func (s *tlsStream) parseClientHello() utils.LSMAction {
chBuf, ok := s.reqBuf.GetSubBuffer(s.clientHelloLen, true)
if !ok {
// Not a full client hello yet
return utils.LSMActionPause
}
m := internal.ParseTLSClientHelloMsgData(chBuf)
if m == nil {
s.reqUpdated = true
s.reqMap = make(analyzer.PropMap)
// Version, random & session ID length combined are within 35 bytes,
// so no need for bounds checking
s.reqMap["version"], _ = chBuf.GetUint16(false, true)
s.reqMap["random"], _ = chBuf.Get(32, true)
sessionIDLen, _ := chBuf.GetByte(true)
s.reqMap["session"], ok = chBuf.Get(int(sessionIDLen), true)
if !ok {
// Not enough data for session ID
return utils.LSMActionCancel
} else {
s.reqUpdated = true
s.reqMap = m
}
cipherSuitesLen, ok := chBuf.GetUint16(false, true)
if !ok {
// Not enough data for cipher suites length
return utils.LSMActionCancel
}
if cipherSuitesLen%2 != 0 {
// Cipher suites are 2 bytes each, so must be even
return utils.LSMActionCancel
}
ciphers := make([]uint16, cipherSuitesLen/2)
for i := range ciphers {
ciphers[i], ok = chBuf.GetUint16(false, true)
if !ok {
return utils.LSMActionCancel
}
}
s.reqMap["ciphers"] = ciphers
compressionMethodsLen, ok := chBuf.GetByte(true)
if !ok {
// Not enough data for compression methods length
return utils.LSMActionCancel
}
// Compression methods are 1 byte each, we just put a byte slice here
s.reqMap["compression"], ok = chBuf.Get(int(compressionMethodsLen), true)
if !ok {
// Not enough data for compression methods
return utils.LSMActionCancel
}
extsLen, ok := chBuf.GetUint16(false, true)
if !ok {
// No extensions, I guess it's possible?
return utils.LSMActionNext
}
extBuf, ok := chBuf.GetSubBuffer(int(extsLen), true)
if !ok {
// Not enough data for extensions
return utils.LSMActionCancel
}
for extBuf.Len() > 0 {
extType, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension type
return utils.LSMActionCancel
}
extLen, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension length
return utils.LSMActionCancel
}
extDataBuf, ok := extBuf.GetSubBuffer(int(extLen), true)
if !ok || !s.handleExtensions(extType, extDataBuf, s.reqMap) {
// Not enough data for extension data, or invalid extension
return utils.LSMActionCancel
}
}
return utils.LSMActionNext
}
// parseServerHelloData converts valid ServerHello message data (without
// headers) into `analyzer.PropMap`.
//
// Parsing error may leave `s.respBuf` in an unusable state.
func (s *tlsStream) parseServerHelloData() utils.LSMAction {
func (s *tlsStream) parseServerHello() utils.LSMAction {
shBuf, ok := s.respBuf.GetSubBuffer(s.serverHelloLen, true)
if !ok {
// Not a full server hello yet
return utils.LSMActionPause
}
m := internal.ParseTLSServerHelloMsgData(shBuf)
if m == nil {
s.respUpdated = true
s.respMap = make(analyzer.PropMap)
// Version, random & session ID length combined are within 35 bytes,
// so no need for bounds checking
s.respMap["version"], _ = shBuf.GetUint16(false, true)
s.respMap["random"], _ = shBuf.Get(32, true)
sessionIDLen, _ := shBuf.GetByte(true)
s.respMap["session"], ok = shBuf.Get(int(sessionIDLen), true)
if !ok {
// Not enough data for session ID
return utils.LSMActionCancel
} else {
s.respUpdated = true
s.respMap = m
}
cipherSuite, ok := shBuf.GetUint16(false, true)
if !ok {
// Not enough data for cipher suite
return utils.LSMActionCancel
}
s.respMap["cipher"] = cipherSuite
compressionMethod, ok := shBuf.GetByte(true)
if !ok {
// Not enough data for compression method
return utils.LSMActionCancel
}
s.respMap["compression"] = compressionMethod
extsLen, ok := shBuf.GetUint16(false, true)
if !ok {
// No extensions, I guess it's possible?
return utils.LSMActionNext
}
extBuf, ok := shBuf.GetSubBuffer(int(extsLen), true)
if !ok {
// Not enough data for extensions
return utils.LSMActionCancel
}
for extBuf.Len() > 0 {
extType, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension type
return utils.LSMActionCancel
}
extLen, ok := extBuf.GetUint16(false, true)
if !ok {
// Not enough data for extension length
return utils.LSMActionCancel
}
extDataBuf, ok := extBuf.GetSubBuffer(int(extLen), true)
if !ok || !s.handleExtensions(extType, extDataBuf, s.respMap) {
// Not enough data for extension data, or invalid extension
return utils.LSMActionCancel
}
}
return utils.LSMActionNext
}
func (s *tlsStream) handleExtensions(extType uint16, extDataBuf *utils.ByteBuffer, m analyzer.PropMap) bool {
switch extType {
case 0x0000: // SNI
ok := extDataBuf.Skip(2) // Ignore list length, we only care about the first entry for now
if !ok {
// Not enough data for list length
return false
}
sniType, ok := extDataBuf.GetByte(true)
if !ok || sniType != 0 {
// Not enough data for SNI type, or not hostname
return false
}
sniLen, ok := extDataBuf.GetUint16(false, true)
if !ok {
// Not enough data for SNI length
return false
}
m["sni"], ok = extDataBuf.GetString(int(sniLen), true)
if !ok {
// Not enough data for SNI
return false
}
case 0x0010: // ALPN
ok := extDataBuf.Skip(2) // Ignore list length, as we read until the end
if !ok {
// Not enough data for list length
return false
}
var alpnList []string
for extDataBuf.Len() > 0 {
alpnLen, ok := extDataBuf.GetByte(true)
if !ok {
// Not enough data for ALPN length
return false
}
alpn, ok := extDataBuf.GetString(int(alpnLen), true)
if !ok {
// Not enough data for ALPN
return false
}
alpnList = append(alpnList, alpn)
}
m["alpn"] = alpnList
case 0x002b: // Supported Versions
if extDataBuf.Len() == 2 {
// Server only selects one version
m["supported_versions"], _ = extDataBuf.GetUint16(false, true)
} else {
// Client sends a list of versions
ok := extDataBuf.Skip(1) // Ignore list length, as we read until the end
if !ok {
// Not enough data for list length
return false
}
var versions []uint16
for extDataBuf.Len() > 0 {
ver, ok := extDataBuf.GetUint16(false, true)
if !ok {
// Not enough data for version
return false
}
versions = append(versions, ver)
}
m["supported_versions"] = versions
}
case 0xfe0d: // ECH
// We can't parse ECH for now, just set a flag
m["ech"] = true
}
return true
}
func (s *tlsStream) Close(limited bool) *analyzer.PropUpdate {

69
analyzer/tcp/tls_test.go
View File

@ -1,69 +0,0 @@
package tcp
import (
"reflect"
"testing"
"github.com/apernet/OpenGFW/analyzer"
)
func TestTlsStreamParsing_ClientHello(t *testing.T) {
// example packet taken from <https://tls12.xargs.org/#client-hello/annotated>
clientHello := []byte{
0x16, 0x03, 0x01, 0x00, 0xa5, 0x01, 0x00, 0x00, 0xa1, 0x03, 0x03, 0x00,
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c,
0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x00, 0x00, 0x20, 0xcc, 0xa8,
0xcc, 0xa9, 0xc0, 0x2f, 0xc0, 0x30, 0xc0, 0x2b, 0xc0, 0x2c, 0xc0, 0x13,
0xc0, 0x09, 0xc0, 0x14, 0xc0, 0x0a, 0x00, 0x9c, 0x00, 0x9d, 0x00, 0x2f,
0x00, 0x35, 0xc0, 0x12, 0x00, 0x0a, 0x01, 0x00, 0x00, 0x58, 0x00, 0x00,
0x00, 0x18, 0x00, 0x16, 0x00, 0x00, 0x13, 0x65, 0x78, 0x61, 0x6d, 0x70,
0x6c, 0x65, 0x2e, 0x75, 0x6c, 0x66, 0x68, 0x65, 0x69, 0x6d, 0x2e, 0x6e,
0x65, 0x74, 0x00, 0x05, 0x00, 0x05, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00,
0x0a, 0x00, 0x0a, 0x00, 0x08, 0x00, 0x1d, 0x00, 0x17, 0x00, 0x18, 0x00,
0x19, 0x00, 0x0b, 0x00, 0x02, 0x01, 0x00, 0x00, 0x0d, 0x00, 0x12, 0x00,
0x10, 0x04, 0x01, 0x04, 0x03, 0x05, 0x01, 0x05, 0x03, 0x06, 0x01, 0x06,
0x03, 0x02, 0x01, 0x02, 0x03, 0xff, 0x01, 0x00, 0x01, 0x00, 0x00, 0x12,
0x00, 0x00,
}
want := analyzer.PropMap{
"ciphers": []uint16{52392, 52393, 49199, 49200, 49195, 49196, 49171, 49161, 49172, 49162, 156, 157, 47, 53, 49170, 10},
"compression": []uint8{0},
"random": []uint8{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31},
"session": []uint8{},
"sni": "example.ulfheim.net",
"version": uint16(771),
}
s := newTLSStream(nil)
u, _ := s.Feed(false, false, false, 0, clientHello)
got := u.M.Get("req")
if !reflect.DeepEqual(got, want) {
t.Errorf("%d B parsed = %v, want %v", len(clientHello), got, want)
}
}
func TestTlsStreamParsing_ServerHello(t *testing.T) {
// example packet taken from <https://tls12.xargs.org/#server-hello/annotated>
serverHello := []byte{
0x16, 0x03, 0x03, 0x00, 0x31, 0x02, 0x00, 0x00, 0x2d, 0x03, 0x03, 0x70,
0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x7b, 0x7c,
0x7d, 0x7e, 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88,
0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x00, 0xc0, 0x13, 0x00, 0x00,
0x05, 0xff, 0x01, 0x00, 0x01, 0x00,
}
want := analyzer.PropMap{
"cipher": uint16(49171),
"compression": uint8(0),
"random": []uint8{112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143},
"session": []uint8{},
"version": uint16(771),
}
s := newTLSStream(nil)
u, _ := s.Feed(true, false, false, 0, serverHello)
got := u.M.Get("resp")
if !reflect.DeepEqual(got, want) {
t.Errorf("%d B parsed = %v, want %v", len(serverHello), got, want)
}
}

500
analyzer/tcp/trojan.go
View File

@ -9,15 +9,22 @@ import (
var _ analyzer.TCPAnalyzer = (*TrojanAnalyzer)(nil)
// CCS stands for "Change Cipher Spec"
var ccsPattern = []byte{20, 3, 3, 0, 1, 1}
var trojanCCS = []byte{20, 3, 3, 0, 1, 1}
// TrojanAnalyzer uses length-based heuristics to detect Trojan traffic based on
// its "TLS-in-TLS" nature. The heuristics are trained using a decision tree with
// about 20k Trojan samples and 30k non-Trojan samples. The tree is then converted
// to code using a custom tool and inlined here (isTrojanSeq function).
// Accuracy: 1% false positive rate, 10% false negative rate.
// We do NOT recommend directly blocking all positive connections, as this may
// break legitimate TLS connections.
const (
trojanUpLB = 650
trojanUpUB = 1000
trojanDownLB1 = 170
trojanDownUB1 = 180
trojanDownLB2 = 3000
trojanDownUB2 = 7500
)
// TrojanAnalyzer uses a very simple packet length based check to determine
// if a TLS connection is actually the Trojan proxy protocol.
// The algorithm is from the following project, with small modifications:
// https://github.com/XTLS/Trojan-killer
// Warning: Experimental only. This method is known to have significant false positives and false negatives.
type TrojanAnalyzer struct{}
func (a *TrojanAnalyzer) Name() string {
@ -25,7 +32,7 @@ func (a *TrojanAnalyzer) Name() string {
}
func (a *TrojanAnalyzer) Limit() int {
return 512000
return 16384
}
func (a *TrojanAnalyzer) NewTCP(info analyzer.TCPInfo, logger analyzer.Logger) analyzer.TCPStream {
@ -33,12 +40,10 @@ func (a *TrojanAnalyzer) NewTCP(info analyzer.TCPInfo, logger analyzer.Logger) a
}
type trojanStream struct {
logger analyzer.Logger
first bool
count bool
rev bool
seq [4]int
seqIndex int
logger analyzer.Logger
active bool
upCount int
downCount int
}
func newTrojanStream(logger analyzer.Logger) *trojanStream {
@ -52,466 +57,35 @@ func (s *trojanStream) Feed(rev, start, end bool, skip int, data []byte) (u *ana
if len(data) == 0 {
return nil, false
}
if s.first {
s.first = false
// Stop if it's not a valid TLS connection
if !(!rev && len(data) >= 3 && data[0] >= 0x16 && data[0] <= 0x17 &&
data[1] == 0x03 && data[2] <= 0x09) {
return nil, true
}
if !rev && !s.active && len(data) >= 6 && bytes.Equal(data[:6], trojanCCS) {
// Client CCS encountered, start counting
s.active = true
}
if !rev && !s.count && len(data) >= 6 && bytes.Equal(data[:6], ccsPattern) {
// Client Change Cipher Spec encountered, start counting
s.count = true
}
if s.count {
if rev == s.rev {
// Same direction as last time, just update the number
s.seq[s.seqIndex] += len(data)
if s.active {
if rev {
// Down direction
s.downCount += len(data)
} else {
// Different direction, bump the index
s.seqIndex += 1
if s.seqIndex == 4 {
// Up direction
if s.upCount >= trojanUpLB && s.upCount <= trojanUpUB &&
((s.downCount >= trojanDownLB1 && s.downCount <= trojanDownUB1) ||
(s.downCount >= trojanDownLB2 && s.downCount <= trojanDownUB2)) {
return &analyzer.PropUpdate{
Type: analyzer.PropUpdateReplace,
M: analyzer.PropMap{
"seq": s.seq,
"yes": isTrojanSeq(s.seq),
"up": s.upCount,
"down": s.downCount,
"yes": true,
},
}, true
}
s.seq[s.seqIndex] += len(data)
s.rev = rev
s.upCount += len(data)
}
}
return nil, false
// Give up when either direction is over the limit
return nil, s.upCount > trojanUpUB || s.downCount > trojanDownUB2
}
func (s *trojanStream) Close(limited bool) *analyzer.PropUpdate {
return nil
}
func isTrojanSeq(seq [4]int) bool {
length1 := seq[0]
length2 := seq[1]
length3 := seq[2]
length4 := seq[3]
if length2 <= 2431 {
if length2 <= 157 {
if length1 <= 156 {
if length3 <= 108 {
return false
} else {
return false
}
} else {
if length1 <= 892 {
if length3 <= 40 {
return false
} else {
if length3 <= 788 {
if length4 <= 185 {
if length1 <= 411 {
return true
} else {
return false
}
} else {
if length2 <= 112 {
return false
} else {
return true
}
}
} else {
if length3 <= 1346 {
if length1 <= 418 {
return false
} else {
return true
}
} else {
return false
}
}
}
} else {
if length2 <= 120 {
if length2 <= 63 {
return false
} else {
if length4 <= 653 {
return false
} else {
return false
}
}
} else {
return false
}
}
}
} else {
if length1 <= 206 {
if length1 <= 185 {
if length1 <= 171 {
return false
} else {
if length4 <= 211 {
return false
} else {
return false
}
}
} else {
if length2 <= 251 {
return true
} else {
return false
}
}
} else {
if length2 <= 286 {
if length1 <= 1123 {
if length3 <= 70 {
return false
} else {
if length1 <= 659 {
if length3 <= 370 {
return true
} else {
return false
}
} else {
if length4 <= 272 {
return false
} else {
return true
}
}
}
} else {
if length4 <= 537 {
if length2 <= 276 {
if length3 <= 1877 {
return false
} else {
return false
}
} else {
return false
}
} else {
if length1 <= 1466 {
if length1 <= 1435 {
return false
} else {
return true
}
} else {
if length2 <= 193 {
return false
} else {
return false
}
}
}
}
} else {
if length1 <= 284 {
if length1 <= 277 {
if length2 <= 726 {
return false
} else {
if length2 <= 768 {
return true
} else {
return false
}
}
} else {
if length2 <= 782 {
if length4 <= 783 {
return true
} else {
return false
}
} else {
return false
}
}
} else {
if length2 <= 492 {
if length2 <= 396 {
if length2 <= 322 {
return false
} else {
return false
}
} else {
if length4 <= 971 {
return false
} else {
return true
}
}
} else {
if length2 <= 2128 {
if length2 <= 1418 {
return false
} else {
return false
}
} else {
if length3 <= 103 {
return false
} else {
return false
}
}
}
}
}
}
}
} else {
if length2 <= 6232 {
if length3 <= 85 {
if length2 <= 3599 {
return false
} else {
if length1 <= 613 {
return false
} else {
return false
}
}
} else {
if length3 <= 220 {
if length4 <= 1173 {
if length1 <= 874 {
if length4 <= 337 {
if length4 <= 68 {
return true
} else {
return true
}
} else {
if length1 <= 667 {
return true
} else {
return true
}
}
} else {
if length3 <= 108 {
if length1 <= 1930 {
return true
} else {
return true
}
} else {
if length2 <= 5383 {
return false
} else {
return true
}
}
}
} else {
return false
}
} else {
if length1 <= 664 {
if length3 <= 411 {
if length3 <= 383 {
if length4 <= 346 {
return true
} else {
return false
}
} else {
if length1 <= 445 {
return true
} else {
return false
}
}
} else {
if length2 <= 3708 {
if length4 <= 307 {
return true
} else {
return false
}
} else {
if length2 <= 4656 {
return false
} else {
return false
}
}
}
} else {
if length1 <= 1055 {
if length3 <= 580 {
if length1 <= 724 {
return true
} else {
return false
}
} else {
if length1 <= 678 {
return false
} else {
return true
}
}
} else {
if length2 <= 5352 {
if length3 <= 1586 {
return false
} else {
return false
}
} else {
if length4 <= 2173 {
return true
} else {
return false
}
}
}
}
}
}
} else {
if length2 <= 9408 {
if length1 <= 670 {
if length4 <= 76 {
if length3 <= 175 {
return true
} else {
return true
}
} else {
if length2 <= 9072 {
if length3 <= 314 {
if length3 <= 179 {
return false
} else {
return false
}
} else {
if length4 <= 708 {
return false
} else {
return false
}
}
} else {
return true
}
}
} else {
if length1 <= 795 {
if length2 <= 6334 {
if length2 <= 6288 {
return true
} else {
return false
}
} else {
if length4 <= 6404 {
if length2 <= 8194 {
return true
} else {
return true
}
} else {
if length2 <= 8924 {
return false
} else {
return true
}
}
}
} else {
if length3 <= 732 {
if length1 <= 1397 {
if length3 <= 179 {
return false
} else {
return false
}
} else {
if length1 <= 1976 {
return false
} else {
return false
}
}
} else {
if length1 <= 2840 {
if length1 <= 2591 {
return false
} else {
return true
}
} else {
return false
}
}
}
}
} else {
if length4 <= 30 {
return false
} else {
if length2 <= 13314 {
if length4 <= 1786 {
if length2 <= 13018 {
if length4 <= 869 {
return false
} else {
return false
}
} else {
return true
}
} else {
if length3 <= 775 {
return false
} else {
return false
}
}
} else {
if length4 <= 73 {
return false
} else {
if length3 <= 640 {
if length3 <= 237 {
return false
} else {
return false
}
} else {
if length2 <= 43804 {
return false
} else {
return false
}
}
}
}
}
}
}
}
}

31
analyzer/udp/internal/quic/LICENSE
View File

@ -1,31 +0,0 @@
Author:: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Copyright:: Copyright (c) 2023, Cuong Manh Le
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
* Neither the name of the @organization@ nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL LE MANH CUONG
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

1
analyzer/udp/internal/quic/README.md
View File

@ -1 +0,0 @@
The code here is from https://github.com/cuonglm/quicsni with various modifications.

105
analyzer/udp/internal/quic/header.go
View File

@ -1,105 +0,0 @@
package quic
import (
"bytes"
"encoding/binary"
"errors"
"io"
"github.com/quic-go/quic-go/quicvarint"
)
// The Header represents a QUIC header.
type Header struct {
Type uint8
Version uint32
SrcConnectionID []byte
DestConnectionID []byte
Length int64
Token []byte
}
// ParseInitialHeader parses the initial packet of a QUIC connection,
// return the initial header and number of bytes read so far.
func ParseInitialHeader(data []byte) (*Header, int64, error) {
br := bytes.NewReader(data)
hdr, err := parseLongHeader(br)
if err != nil {
return nil, 0, err
}
n := int64(len(data) - br.Len())
return hdr, n, nil
}
func parseLongHeader(b *bytes.Reader) (*Header, error) {
typeByte, err := b.ReadByte()
if err != nil {
return nil, err
}
h := &Header{}
ver, err := beUint32(b)
if err != nil {
return nil, err
}
h.Version = ver
if h.Version != 0 && typeByte&0x40 == 0 {
return nil, errors.New("not a QUIC packet")
}
destConnIDLen, err := b.ReadByte()
if err != nil {
return nil, err
}
h.DestConnectionID = make([]byte, int(destConnIDLen))
if err := readConnectionID(b, h.DestConnectionID); err != nil {
return nil, err
}
srcConnIDLen, err := b.ReadByte()
if err != nil {
return nil, err
}
h.SrcConnectionID = make([]byte, int(srcConnIDLen))
if err := readConnectionID(b, h.SrcConnectionID); err != nil {
return nil, err
}
initialPacketType := byte(0b00)
if h.Version == V2 {
initialPacketType = 0b01
}
if (typeByte >> 4 & 0b11) == initialPacketType {
tokenLen, err := quicvarint.Read(b)
if err != nil {
return nil, err
}
if tokenLen > uint64(b.Len()) {
return nil, io.EOF
}
h.Token = make([]byte, tokenLen)
if _, err := io.ReadFull(b, h.Token); err != nil {
return nil, err
}
}
pl, err := quicvarint.Read(b)
if err != nil {
return nil, err
}
h.Length = int64(pl)
return h, err
}
func readConnectionID(r io.Reader, cid []byte) error {
_, err := io.ReadFull(r, cid)
if err == io.ErrUnexpectedEOF {
return io.EOF
}
return nil
}
func beUint32(r io.Reader) (uint32, error) {
b := make([]byte, 4)
if _, err := io.ReadFull(r, b); err != nil {
return 0, err
}
return binary.BigEndian.Uint32(b), nil
}

193
analyzer/udp/internal/quic/packet_protector.go
View File

@ -1,193 +0,0 @@
package quic
import (
"crypto"
"crypto/aes"
"crypto/cipher"
"crypto/sha256"
"crypto/tls"
"encoding/binary"
"errors"
"fmt"
"hash"
"golang.org/x/crypto/chacha20"
"golang.org/x/crypto/chacha20poly1305"
"golang.org/x/crypto/cryptobyte"
"golang.org/x/crypto/hkdf"
)
// NewProtectionKey creates a new ProtectionKey.
func NewProtectionKey(suite uint16, secret []byte, v uint32) (*ProtectionKey, error) {
return newProtectionKey(suite, secret, v)
}
// NewInitialProtectionKey is like NewProtectionKey, but the returned protection key
// is used for encrypt/decrypt Initial Packet only.
//
// See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-initial-secrets
func NewInitialProtectionKey(secret []byte, v uint32) (*ProtectionKey, error) {
return NewProtectionKey(tls.TLS_AES_128_GCM_SHA256, secret, v)
}
// NewPacketProtector creates a new PacketProtector.
func NewPacketProtector(key *ProtectionKey) *PacketProtector {
return &PacketProtector{key: key}
}
// PacketProtector is used for protecting a QUIC packet.
//
// See: https://www.rfc-editor.org/rfc/rfc9001.html#name-packet-protection
type PacketProtector struct {
key *ProtectionKey
}
// UnProtect decrypts a QUIC packet.
func (pp *PacketProtector) UnProtect(packet []byte, pnOffset, pnMax int64) ([]byte, error) {
if isLongHeader(packet[0]) && int64(len(packet)) < pnOffset+4+16 {
return nil, errors.New("packet with long header is too small")
}
// https://www.rfc-editor.org/rfc/rfc9001.html#name-header-protection-sample
sampleOffset := pnOffset + 4
sample := packet[sampleOffset : sampleOffset+16]
// https://www.rfc-editor.org/rfc/rfc9001.html#name-header-protection-applicati
mask := pp.key.headerProtection(sample)
if isLongHeader(packet[0]) {
// Long header: 4 bits masked
packet[0] ^= mask[0] & 0x0f
} else {
// Short header: 5 bits masked
packet[0] ^= mask[0] & 0x1f
}
pnLen := packet[0]&0x3 + 1
pn := int64(0)
for i := uint8(0); i < pnLen; i++ {
packet[pnOffset:][i] ^= mask[1+i]
pn = (pn << 8) | int64(packet[pnOffset:][i])
}
pn = decodePacketNumber(pnMax, pn, pnLen)
hdr := packet[:pnOffset+int64(pnLen)]
payload := packet[pnOffset:][pnLen:]
dec, err := pp.key.aead.Open(payload[:0], pp.key.nonce(pn), payload, hdr)
if err != nil {
return nil, fmt.Errorf("decryption failed: %w", err)
}
return dec, nil
}
// ProtectionKey is the key used to protect a QUIC packet.
type ProtectionKey struct {
aead cipher.AEAD
headerProtection func(sample []byte) (mask []byte)
iv []byte
}
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-aead-usage
//
// "The 62 bits of the reconstructed QUIC packet number in network byte order are
// left-padded with zeros to the size of the IV. The exclusive OR of the padded
// packet number and the IV forms the AEAD nonce."
func (pk *ProtectionKey) nonce(pn int64) []byte {
nonce := make([]byte, len(pk.iv))
binary.BigEndian.PutUint64(nonce[len(nonce)-8:], uint64(pn))
for i := range pk.iv {
nonce[i] ^= pk.iv[i]
}
return nonce
}
func newProtectionKey(suite uint16, secret []byte, v uint32) (*ProtectionKey, error) {
switch suite {
case tls.TLS_AES_128_GCM_SHA256:
key := hkdfExpandLabel(crypto.SHA256.New, secret, keyLabel(v), nil, 16)
c, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
aead, err := cipher.NewGCM(c)
if err != nil {
panic(err)
}
iv := hkdfExpandLabel(crypto.SHA256.New, secret, ivLabel(v), nil, aead.NonceSize())
hpKey := hkdfExpandLabel(crypto.SHA256.New, secret, headerProtectionLabel(v), nil, 16)
hp, err := aes.NewCipher(hpKey)
if err != nil {
panic(err)
}
k := &ProtectionKey{}
k.aead = aead
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-aes-based-header-protection
k.headerProtection = func(sample []byte) []byte {
mask := make([]byte, hp.BlockSize())
hp.Encrypt(mask, sample)
return mask
}
k.iv = iv
return k, nil
case tls.TLS_CHACHA20_POLY1305_SHA256:
key := hkdfExpandLabel(crypto.SHA256.New, secret, keyLabel(v), nil, chacha20poly1305.KeySize)
aead, err := chacha20poly1305.New(key)
if err != nil {
return nil, err
}
iv := hkdfExpandLabel(crypto.SHA256.New, secret, ivLabel(v), nil, aead.NonceSize())
hpKey := hkdfExpandLabel(sha256.New, secret, headerProtectionLabel(v), nil, chacha20.KeySize)
k := &ProtectionKey{}
k.aead = aead
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-chacha20-based-header-prote
k.headerProtection = func(sample []byte) []byte {
nonce := sample[4:16]
c, err := chacha20.NewUnauthenticatedCipher(hpKey, nonce)
if err != nil {
panic(err)
}
c.SetCounter(binary.LittleEndian.Uint32(sample[:4]))
mask := make([]byte, 5)
c.XORKeyStream(mask, mask)
return mask
}
k.iv = iv
return k, nil
}
return nil, errors.New("not supported cipher suite")
}
// decodePacketNumber decode the packet number after header protection removed.
//
// See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-transport-32#section-appendix.a
func decodePacketNumber(largest, truncated int64, nbits uint8) int64 {
expected := largest + 1
win := int64(1 << (nbits * 8))
hwin := win / 2
mask := win - 1
candidate := (expected &^ mask) | truncated
switch {
case candidate <= expected-hwin && candidate < (1<<62)-win:
return candidate + win
case candidate > expected+hwin && candidate >= win:
return candidate - win
}
return candidate
}
// Copied from crypto/tls/key_schedule.go.
func hkdfExpandLabel(hash func() hash.Hash, secret []byte, label string, context []byte, length int) []byte {
var hkdfLabel cryptobyte.Builder
hkdfLabel.AddUint16(uint16(length))
hkdfLabel.AddUint8LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes([]byte("tls13 "))
b.AddBytes([]byte(label))
})
hkdfLabel.AddUint8LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(context)
})
out := make([]byte, length)
n, err := hkdf.Expand(hash, secret, hkdfLabel.BytesOrPanic()).Read(out)
if err != nil || n != length {
panic("quic: HKDF-Expand-Label invocation failed unexpectedly")
}
return out
}

94
analyzer/udp/internal/quic/packet_protector_test.go
View File

@ -1,94 +0,0 @@
package quic
import (
"bytes"
"crypto"
"crypto/tls"
"encoding/hex"
"strings"
"testing"
"unicode"
"golang.org/x/crypto/hkdf"
)
func TestInitialPacketProtector_UnProtect(t *testing.T) {
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-server-initial
protect := mustHexDecodeString(`
c7ff0000200008f067a5502a4262b500 4075fb12ff07823a5d24534d906ce4c7
6782a2167e3479c0f7f6395dc2c91676 302fe6d70bb7cbeb117b4ddb7d173498
44fd61dae200b8338e1b932976b61d91 e64a02e9e0ee72e3a6f63aba4ceeeec5
be2f24f2d86027572943533846caa13e 6f163fb257473d0eda5047360fd4a47e
fd8142fafc0f76
`)
unProtect := mustHexDecodeString(`
02000000000600405a020000560303ee fce7f7b37ba1d1632e96677825ddf739
88cfc79825df566dc5430b9a045a1200 130100002e00330024001d00209d3c94
0d89690b84d08a60993c144eca684d10 81287c834d5311bcf32bb9da1a002b00
020304
`)
connID := mustHexDecodeString(`8394c8f03e515708`)
packet := append([]byte{}, protect...)
hdr, offset, err := ParseInitialHeader(packet)
if err != nil {
t.Fatal(err)
}
initialSecret := hkdf.Extract(crypto.SHA256.New, connID, getSalt(hdr.Version))
serverSecret := hkdfExpandLabel(crypto.SHA256.New, initialSecret, "server in", []byte{}, crypto.SHA256.Size())
key, err := NewInitialProtectionKey(serverSecret, hdr.Version)
if err != nil {
t.Fatal(err)
}
pp := NewPacketProtector(key)
got, err := pp.UnProtect(protect, offset, 1)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(got, unProtect) {
t.Error("UnProtect returns wrong result")
}
}
func TestPacketProtectorShortHeader_UnProtect(t *testing.T) {
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-chacha20-poly1305-short-hea
protect := mustHexDecodeString(`4cfe4189655e5cd55c41f69080575d7999c25a5bfb`)
unProtect := mustHexDecodeString(`01`)
hdr := mustHexDecodeString(`4200bff4`)
secret := mustHexDecodeString(`9ac312a7f877468ebe69422748ad00a1 5443f18203a07d6060f688f30f21632b`)
k, err := NewProtectionKey(tls.TLS_CHACHA20_POLY1305_SHA256, secret, V1)
if err != nil {
t.Fatal(err)
}
pnLen := int(hdr[0]&0x03) + 1
offset := len(hdr) - pnLen
pp := NewPacketProtector(k)
got, err := pp.UnProtect(protect, int64(offset), 654360564)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(got, unProtect) {
t.Error("UnProtect returns wrong result")
}
}
func mustHexDecodeString(s string) []byte {
b, err := hex.DecodeString(normalizeHex(s))
if err != nil {
panic(err)
}
return b
}
func normalizeHex(s string) string {
return strings.Map(func(c rune) rune {
if unicode.IsSpace(c) {
return -1
}
return c
}, s)
}

122
analyzer/udp/internal/quic/payload.go
View File

@ -1,122 +0,0 @@
package quic
import (
"bytes"
"crypto"
"errors"
"fmt"
"io"
"sort"
"github.com/quic-go/quic-go/quicvarint"
"golang.org/x/crypto/hkdf"
)
func ReadCryptoPayload(packet []byte) ([]byte, error) {
hdr, offset, err := ParseInitialHeader(packet)
if err != nil {
return nil, err
}
// Some sanity checks
if hdr.Version != V1 && hdr.Version != V2 {
return nil, fmt.Errorf("unsupported version: %x", hdr.Version)
}
if offset == 0 || hdr.Length == 0 {
return nil, errors.New("invalid packet")
}
initialSecret := hkdf.Extract(crypto.SHA256.New, hdr.DestConnectionID, getSalt(hdr.Version))
clientSecret := hkdfExpandLabel(crypto.SHA256.New, initialSecret, "client in", []byte{}, crypto.SHA256.Size())
key, err := NewInitialProtectionKey(clientSecret, hdr.Version)
if err != nil {
return nil, fmt.Errorf("NewInitialProtectionKey: %w", err)
}
pp := NewPacketProtector(key)
// https://datatracker.ietf.org/doc/html/draft-ietf-quic-tls-32#name-client-initial
//
// "The unprotected header includes the connection ID and a 4-byte packet number encoding for a packet number of 2"
if int64(len(packet)) < offset+hdr.Length {
return nil, fmt.Errorf("packet is too short: %d < %d", len(packet), offset+hdr.Length)
}
unProtectedPayload, err := pp.UnProtect(packet[:offset+hdr.Length], offset, 2)
if err != nil {
return nil, err
}
frs, err := extractCryptoFrames(bytes.NewReader(unProtectedPayload))
if err != nil {
return nil, err
}
data := assembleCryptoFrames(frs)
if data == nil {
return nil, errors.New("unable to assemble crypto frames")
}
return data, nil
}
const (
paddingFrameType = 0x00
pingFrameType = 0x01
cryptoFrameType = 0x06
)
type cryptoFrame struct {
Offset int64
Data []byte
}
func extractCryptoFrames(r *bytes.Reader) ([]cryptoFrame, error) {
var frames []cryptoFrame
for r.Len() > 0 {
typ, err := quicvarint.Read(r)
if err != nil {
return nil, err
}
if typ == paddingFrameType || typ == pingFrameType {
continue
}
if typ != cryptoFrameType {
return nil, fmt.Errorf("encountered unexpected frame type: %d", typ)
}
var frame cryptoFrame
offset, err := quicvarint.Read(r)
if err != nil {
return nil, err
}
frame.Offset = int64(offset)
dataLen, err := quicvarint.Read(r)
if err != nil {
return nil, err
}
frame.Data = make([]byte, dataLen)
if _, err := io.ReadFull(r, frame.Data); err != nil {
return nil, err
}
frames = append(frames, frame)
}
return frames, nil
}
// assembleCryptoFrames assembles multiple crypto frames into a single slice (if possible).
// It returns an error if the frames cannot be assembled. This can happen if the frames are not contiguous.
func assembleCryptoFrames(frames []cryptoFrame) []byte {
if len(frames) == 0 {
return nil
}
if len(frames) == 1 {
return frames[0].Data
}
// sort the frames by offset
sort.Slice(frames, func(i, j int) bool { return frames[i].Offset < frames[j].Offset })
// check if the frames are contiguous
for i := 1; i < len(frames); i++ {
if frames[i].Offset != frames[i-1].Offset+int64(len(frames[i-1].Data)) {
return nil
}
}
// concatenate the frames
data := make([]byte, frames[len(frames)-1].Offset+int64(len(frames[len(frames)-1].Data)))
for _, frame := range frames {
copy(data[frame.Offset:], frame.Data)
}
return data
}

59
analyzer/udp/internal/quic/quic.go
View File

@ -1,59 +0,0 @@
package quic
const (
V1 uint32 = 0x1
V2 uint32 = 0x6b3343cf
hkdfLabelKeyV1 = "quic key"
hkdfLabelKeyV2 = "quicv2 key"
hkdfLabelIVV1 = "quic iv"
hkdfLabelIVV2 = "quicv2 iv"
hkdfLabelHPV1 = "quic hp"
hkdfLabelHPV2 = "quicv2 hp"
)
var (
quicSaltOld = []byte{0xaf, 0xbf, 0xec, 0x28, 0x99, 0x93, 0xd2, 0x4c, 0x9e, 0x97, 0x86, 0xf1, 0x9c, 0x61, 0x11, 0xe0, 0x43, 0x90, 0xa8, 0x99}
// https://www.rfc-editor.org/rfc/rfc9001.html#name-initial-secrets
quicSaltV1 = []byte{0x38, 0x76, 0x2c, 0xf7, 0xf5, 0x59, 0x34, 0xb3, 0x4d, 0x17, 0x9a, 0xe6, 0xa4, 0xc8, 0x0c, 0xad, 0xcc, 0xbb, 0x7f, 0x0a}
// https://www.ietf.org/archive/id/draft-ietf-quic-v2-10.html#name-initial-salt-2
quicSaltV2 = []byte{0x0d, 0xed, 0xe3, 0xde, 0xf7, 0x00, 0xa6, 0xdb, 0x81, 0x93, 0x81, 0xbe, 0x6e, 0x26, 0x9d, 0xcb, 0xf9, 0xbd, 0x2e, 0xd9}
)
// isLongHeader reports whether b is the first byte of a long header packet.
func isLongHeader(b byte) bool {
return b&0x80 > 0
}
func getSalt(v uint32) []byte {
switch v {
case V1:
return quicSaltV1
case V2:
return quicSaltV2
}
return quicSaltOld
}
func keyLabel(v uint32) string {
kl := hkdfLabelKeyV1
if v == V2 {
kl = hkdfLabelKeyV2
}
return kl
}
func ivLabel(v uint32) string {
ivl := hkdfLabelIVV1
if v == V2 {
ivl = hkdfLabelIVV2
}
return ivl
}
func headerProtectionLabel(v uint32) string {
if v == V2 {
return hkdfLabelHPV2
}
return hkdfLabelHPV1
}

384
analyzer/udp/openvpn.go
View File

@ -1,384 +0,0 @@
package udp
import (
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/utils"
)
var (
_ analyzer.UDPAnalyzer = (*OpenVPNAnalyzer)(nil)
_ analyzer.TCPAnalyzer = (*OpenVPNAnalyzer)(nil)
)
var (
_ analyzer.UDPStream = (*openvpnUDPStream)(nil)
_ analyzer.TCPStream = (*openvpnTCPStream)(nil)
)
// Ref paper:
// https://www.usenix.org/system/files/sec22fall_xue-diwen.pdf
// OpenVPN Opcodes definitions from:
// https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/ssl_pkt.h
const (
OpenVPNControlHardResetClientV1 = 1
OpenVPNControlHardResetServerV1 = 2
OpenVPNControlSoftResetV1 = 3
OpenVPNControlV1 = 4
OpenVPNAckV1 = 5
OpenVPNDataV1 = 6
OpenVPNControlHardResetClientV2 = 7
OpenVPNControlHardResetServerV2 = 8
OpenVPNDataV2 = 9
OpenVPNControlHardResetClientV3 = 10
OpenVPNControlWkcV1 = 11
)
const (
OpenVPNMinPktLen = 6
OpenVPNTCPPktDefaultLimit = 256
OpenVPNUDPPktDefaultLimit = 256
)
type OpenVPNAnalyzer struct{}
func (a *OpenVPNAnalyzer) Name() string {
return "openvpn"
}
func (a *OpenVPNAnalyzer) Limit() int {
return 0
}
func (a *OpenVPNAnalyzer) NewUDP(info analyzer.UDPInfo, logger analyzer.Logger) analyzer.UDPStream {
return newOpenVPNUDPStream(logger)
}
func (a *OpenVPNAnalyzer) NewTCP(info analyzer.TCPInfo, logger analyzer.Logger) analyzer.TCPStream {
return newOpenVPNTCPStream(logger)
}
type openvpnPkt struct {
pktLen uint16 // 16 bits, TCP proto only
opcode byte // 5 bits
_keyId byte // 3 bits, not used
// We don't care about the rest of the packet
// payload []byte
}
type openvpnStream struct {
logger analyzer.Logger
reqUpdated bool
reqLSM *utils.LinearStateMachine
reqDone bool
respUpdated bool
respLSM *utils.LinearStateMachine
respDone bool
rxPktCnt int
txPktCnt int
pktLimit int
reqPktParse func() (*openvpnPkt, utils.LSMAction)
respPktParse func() (*openvpnPkt, utils.LSMAction)
lastOpcode byte
}
func (o *openvpnStream) parseCtlHardResetClient() utils.LSMAction {
pkt, action := o.reqPktParse()
if action != utils.LSMActionNext {
return action
}
if pkt.opcode != OpenVPNControlHardResetClientV1 &&
pkt.opcode != OpenVPNControlHardResetClientV2 &&
pkt.opcode != OpenVPNControlHardResetClientV3 {
return utils.LSMActionCancel
}
o.lastOpcode = pkt.opcode
return utils.LSMActionNext
}
func (o *openvpnStream) parseCtlHardResetServer() utils.LSMAction {
if o.lastOpcode != OpenVPNControlHardResetClientV1 &&
o.lastOpcode != OpenVPNControlHardResetClientV2 &&
o.lastOpcode != OpenVPNControlHardResetClientV3 {
return utils.LSMActionCancel
}
pkt, action := o.respPktParse()
if action != utils.LSMActionNext {
return action
}
if pkt.opcode != OpenVPNControlHardResetServerV1 &&
pkt.opcode != OpenVPNControlHardResetServerV2 {
return utils.LSMActionCancel
}
o.lastOpcode = pkt.opcode
return utils.LSMActionNext
}
func (o *openvpnStream) parseReq() utils.LSMAction {
pkt, action := o.reqPktParse()
if action != utils.LSMActionNext {
return action
}
if pkt.opcode != OpenVPNControlSoftResetV1 &&
pkt.opcode != OpenVPNControlV1 &&
pkt.opcode != OpenVPNAckV1 &&
pkt.opcode != OpenVPNDataV1 &&
pkt.opcode != OpenVPNDataV2 &&
pkt.opcode != OpenVPNControlWkcV1 {
return utils.LSMActionCancel
}
o.txPktCnt += 1
o.reqUpdated = true
return utils.LSMActionPause
}
func (o *openvpnStream) parseResp() utils.LSMAction {
pkt, action := o.respPktParse()
if action != utils.LSMActionNext {
return action
}
if pkt.opcode != OpenVPNControlSoftResetV1 &&
pkt.opcode != OpenVPNControlV1 &&
pkt.opcode != OpenVPNAckV1 &&
pkt.opcode != OpenVPNDataV1 &&
pkt.opcode != OpenVPNDataV2 &&
pkt.opcode != OpenVPNControlWkcV1 {
return utils.LSMActionCancel
}
o.rxPktCnt += 1
o.respUpdated = true
return utils.LSMActionPause
}
type openvpnUDPStream struct {
openvpnStream
curPkt []byte
// We don't introduce `invalidCount` here to decrease the false positive rate
// invalidCount int
}
func newOpenVPNUDPStream(logger analyzer.Logger) *openvpnUDPStream {
s := &openvpnUDPStream{
openvpnStream: openvpnStream{
logger: logger,
pktLimit: OpenVPNUDPPktDefaultLimit,
},
}
s.respPktParse = s.parsePkt
s.reqPktParse = s.parsePkt
s.reqLSM = utils.NewLinearStateMachine(
s.parseCtlHardResetClient,
s.parseReq,
)
s.respLSM = utils.NewLinearStateMachine(
s.parseCtlHardResetServer,
s.parseResp,
)
return s
}
func (o *openvpnUDPStream) Feed(rev bool, data []byte) (u *analyzer.PropUpdate, d bool) {
if len(data) == 0 {
return nil, false
}
var update *analyzer.PropUpdate
var cancelled bool
o.curPkt = data
if rev {
o.respUpdated = false
cancelled, o.respDone = o.respLSM.Run()
if o.respUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateReplace,
M: analyzer.PropMap{"rx_pkt_cnt": o.rxPktCnt, "tx_pkt_cnt": o.txPktCnt},
}
o.respUpdated = false
}
} else {
o.reqUpdated = false
cancelled, o.reqDone = o.reqLSM.Run()
if o.reqUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateReplace,
M: analyzer.PropMap{"rx_pkt_cnt": o.rxPktCnt, "tx_pkt_cnt": o.txPktCnt},
}
o.reqUpdated = false
}
}
return update, cancelled || (o.reqDone && o.respDone) || o.rxPktCnt+o.txPktCnt > o.pktLimit
}
func (o *openvpnUDPStream) Close(limited bool) *analyzer.PropUpdate {
return nil
}
// Parse OpenVPN UDP packet.
func (o *openvpnUDPStream) parsePkt() (p *openvpnPkt, action utils.LSMAction) {
if o.curPkt == nil {
return nil, utils.LSMActionPause
}
if !OpenVPNCheckForValidOpcode(o.curPkt[0] >> 3) {
return nil, utils.LSMActionCancel
}
// Parse packet header
p = &openvpnPkt{}
p.opcode = o.curPkt[0] >> 3
p._keyId = o.curPkt[0] & 0x07
o.curPkt = nil
return p, utils.LSMActionNext
}
type openvpnTCPStream struct {
openvpnStream
reqBuf *utils.ByteBuffer
respBuf *utils.ByteBuffer
}
func newOpenVPNTCPStream(logger analyzer.Logger) *openvpnTCPStream {
s := &openvpnTCPStream{
openvpnStream: openvpnStream{
logger: logger,
pktLimit: OpenVPNTCPPktDefaultLimit,
},
reqBuf: &utils.ByteBuffer{},
respBuf: &utils.ByteBuffer{},
}
s.respPktParse = func() (*openvpnPkt, utils.LSMAction) {
return s.parsePkt(true)
}
s.reqPktParse = func() (*openvpnPkt, utils.LSMAction) {
return s.parsePkt(false)
}
s.reqLSM = utils.NewLinearStateMachine(
s.parseCtlHardResetClient,
s.parseReq,
)
s.respLSM = utils.NewLinearStateMachine(
s.parseCtlHardResetServer,
s.parseResp,
)
return s
}
func (o *openvpnTCPStream) Feed(rev, start, end bool, skip int, data []byte) (u *analyzer.PropUpdate, d bool) {
if skip != 0 {
return nil, true
}
if len(data) == 0 {
return nil, false
}
var update *analyzer.PropUpdate
var cancelled bool
if rev {
o.respBuf.Append(data)
o.respUpdated = false
cancelled, o.respDone = o.respLSM.Run()
if o.respUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateReplace,
M: analyzer.PropMap{"rx_pkt_cnt": o.rxPktCnt, "tx_pkt_cnt": o.txPktCnt},
}
o.respUpdated = false
}
} else {
o.reqBuf.Append(data)
o.reqUpdated = false
cancelled, o.reqDone = o.reqLSM.Run()
if o.reqUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{"rx_pkt_cnt": o.rxPktCnt, "tx_pkt_cnt": o.txPktCnt},
}
o.reqUpdated = false
}
}
return update, cancelled || (o.reqDone && o.respDone) || o.rxPktCnt+o.txPktCnt > o.pktLimit
}
func (o *openvpnTCPStream) Close(limited bool) *analyzer.PropUpdate {
o.reqBuf.Reset()
o.respBuf.Reset()
return nil
}
// Parse OpenVPN TCP packet.
func (o *openvpnTCPStream) parsePkt(rev bool) (p *openvpnPkt, action utils.LSMAction) {
var buffer *utils.ByteBuffer
if rev {
buffer = o.respBuf
} else {
buffer = o.reqBuf
}
// Parse packet length
pktLen, ok := buffer.GetUint16(false, false)
if !ok {
return nil, utils.LSMActionPause
}
if pktLen < OpenVPNMinPktLen {
return nil, utils.LSMActionCancel
}
pktOp, ok := buffer.Get(3, false)
if !ok {
return nil, utils.LSMActionPause
}
if !OpenVPNCheckForValidOpcode(pktOp[2] >> 3) {
return nil, utils.LSMActionCancel
}
pkt, ok := buffer.Get(int(pktLen)+2, true)
if !ok {
return nil, utils.LSMActionPause
}
pkt = pkt[2:]
// Parse packet header
p = &openvpnPkt{}
p.pktLen = pktLen
p.opcode = pkt[0] >> 3
p._keyId = pkt[0] & 0x07
return p, utils.LSMActionNext
}
func OpenVPNCheckForValidOpcode(opcode byte) bool {
switch opcode {
case OpenVPNControlHardResetClientV1,
OpenVPNControlHardResetServerV1,
OpenVPNControlSoftResetV1,
OpenVPNControlV1,
OpenVPNAckV1,
OpenVPNDataV1,
OpenVPNControlHardResetClientV2,
OpenVPNControlHardResetServerV2,
OpenVPNDataV2,
OpenVPNControlHardResetClientV3,
OpenVPNControlWkcV1:
return true
}
return false
}

81
analyzer/udp/quic.go
View File

@ -1,81 +0,0 @@
package udp
import (
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/internal"
"github.com/apernet/OpenGFW/analyzer/udp/internal/quic"
"github.com/apernet/OpenGFW/analyzer/utils"
)
const (
quicInvalidCountThreshold = 4
)
var (
_ analyzer.UDPAnalyzer = (*QUICAnalyzer)(nil)
_ analyzer.UDPStream = (*quicStream)(nil)
)
type QUICAnalyzer struct{}
func (a *QUICAnalyzer) Name() string {
return "quic"
}
func (a *QUICAnalyzer) Limit() int {
return 0
}
func (a *QUICAnalyzer) NewUDP(info analyzer.UDPInfo, logger analyzer.Logger) analyzer.UDPStream {
return &quicStream{logger: logger}
}
type quicStream struct {
logger analyzer.Logger
invalidCount int
}
func (s *quicStream) Feed(rev bool, data []byte) (u *analyzer.PropUpdate, done bool) {
// minimal data size: protocol version (2 bytes) + random (32 bytes) +
// + session ID (1 byte) + cipher suites (4 bytes) +
// + compression methods (2 bytes) + no extensions
const minDataSize = 41
if rev {
// We don't support server direction for now
s.invalidCount++
return nil, s.invalidCount >= quicInvalidCountThreshold
}
pl, err := quic.ReadCryptoPayload(data)
if err != nil || len(pl) < 4 { // FIXME: isn't length checked inside quic.ReadCryptoPayload? Also, what about error handling?
s.invalidCount++
return nil, s.invalidCount >= quicInvalidCountThreshold
}
if pl[0] != internal.TypeClientHello {
s.invalidCount++
return nil, s.invalidCount >= quicInvalidCountThreshold
}
chLen := int(pl[1])<<16 | int(pl[2])<<8 | int(pl[3])
if chLen < minDataSize {
s.invalidCount++
return nil, s.invalidCount >= quicInvalidCountThreshold
}
m := internal.ParseTLSClientHelloMsgData(&utils.ByteBuffer{Buf: pl[4:]})
if m == nil {
s.invalidCount++
return nil, s.invalidCount >= quicInvalidCountThreshold
}
return &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{"req": m},
}, true
}
func (s *quicStream) Close(limited bool) *analyzer.PropUpdate {
return nil
}

58
analyzer/udp/quic_test.go
View File

@ -1,58 +0,0 @@
package udp
import (
"reflect"
"testing"
"github.com/apernet/OpenGFW/analyzer"
)
func TestQuicStreamParsing_ClientHello(t *testing.T) {
// example packet taken from <https://quic.xargs.org/#client-initial-packet/annotated>
clientHello := make([]byte, 1200)
clientInitial := []byte{
0xcd, 0x00, 0x00, 0x00, 0x01, 0x08, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05,
0x06, 0x07, 0x05, 0x63, 0x5f, 0x63, 0x69, 0x64, 0x00, 0x41, 0x03, 0x98,
0x1c, 0x36, 0xa7, 0xed, 0x78, 0x71, 0x6b, 0xe9, 0x71, 0x1b, 0xa4, 0x98,
0xb7, 0xed, 0x86, 0x84, 0x43, 0xbb, 0x2e, 0x0c, 0x51, 0x4d, 0x4d, 0x84,
0x8e, 0xad, 0xcc, 0x7a, 0x00, 0xd2, 0x5c, 0xe9, 0xf9, 0xaf, 0xa4, 0x83,
0x97, 0x80, 0x88, 0xde, 0x83, 0x6b, 0xe6, 0x8c, 0x0b, 0x32, 0xa2, 0x45,
0x95, 0xd7, 0x81, 0x3e, 0xa5, 0x41, 0x4a, 0x91, 0x99, 0x32, 0x9a, 0x6d,
0x9f, 0x7f, 0x76, 0x0d, 0xd8, 0xbb, 0x24, 0x9b, 0xf3, 0xf5, 0x3d, 0x9a,
0x77, 0xfb, 0xb7, 0xb3, 0x95, 0xb8, 0xd6, 0x6d, 0x78, 0x79, 0xa5, 0x1f,
0xe5, 0x9e, 0xf9, 0x60, 0x1f, 0x79, 0x99, 0x8e, 0xb3, 0x56, 0x8e, 0x1f,
0xdc, 0x78, 0x9f, 0x64, 0x0a, 0xca, 0xb3, 0x85, 0x8a, 0x82, 0xef, 0x29,
0x30, 0xfa, 0x5c, 0xe1, 0x4b, 0x5b, 0x9e, 0xa0, 0xbd, 0xb2, 0x9f, 0x45,
0x72, 0xda, 0x85, 0xaa, 0x3d, 0xef, 0x39, 0xb7, 0xef, 0xaf, 0xff, 0xa0,
0x74, 0xb9, 0x26, 0x70, 0x70, 0xd5, 0x0b, 0x5d, 0x07, 0x84, 0x2e, 0x49,
0xbb, 0xa3, 0xbc, 0x78, 0x7f, 0xf2, 0x95, 0xd6, 0xae, 0x3b, 0x51, 0x43,
0x05, 0xf1, 0x02, 0xaf, 0xe5, 0xa0, 0x47, 0xb3, 0xfb, 0x4c, 0x99, 0xeb,
0x92, 0xa2, 0x74, 0xd2, 0x44, 0xd6, 0x04, 0x92, 0xc0, 0xe2, 0xe6, 0xe2,
0x12, 0xce, 0xf0, 0xf9, 0xe3, 0xf6, 0x2e, 0xfd, 0x09, 0x55, 0xe7, 0x1c,
0x76, 0x8a, 0xa6, 0xbb, 0x3c, 0xd8, 0x0b, 0xbb, 0x37, 0x55, 0xc8, 0xb7,
0xeb, 0xee, 0x32, 0x71, 0x2f, 0x40, 0xf2, 0x24, 0x51, 0x19, 0x48, 0x70,
0x21, 0xb4, 0xb8, 0x4e, 0x15, 0x65, 0xe3, 0xca, 0x31, 0x96, 0x7a, 0xc8,
0x60, 0x4d, 0x40, 0x32, 0x17, 0x0d, 0xec, 0x28, 0x0a, 0xee, 0xfa, 0x09,
0x5d, 0x08, 0xb3, 0xb7, 0x24, 0x1e, 0xf6, 0x64, 0x6a, 0x6c, 0x86, 0xe5,
0xc6, 0x2c, 0xe0, 0x8b, 0xe0, 0x99,
}
copy(clientHello, clientInitial)
want := analyzer.PropMap{
"alpn": []string{"ping/1.0"},
"ciphers": []uint16{4865, 4866, 4867},
"compression": []uint8{0},
"random": []uint8{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31},
"session": []uint8{},
"sni": "example.ulfheim.net",
"supported_versions": []uint16{772},
"version": uint16(771),
}
s := quicStream{}
u, _ := s.Feed(false, clientHello)
got := u.M.Get("req")
if !reflect.DeepEqual(got, want) {
t.Errorf("%d B parsed = %v, want %v", len(clientHello), got, want)
}
}

217
analyzer/udp/wireguard.go
View File

@ -1,217 +0,0 @@
package udp
import (
"container/ring"
"encoding/binary"
"slices"
"sync"
"github.com/apernet/OpenGFW/analyzer"
)
var (
_ analyzer.UDPAnalyzer = (*WireGuardAnalyzer)(nil)
_ analyzer.UDPStream = (*wireGuardUDPStream)(nil)
)
const (
wireguardUDPInvalidCountThreshold = 4
wireguardRememberedIndexCount = 6
wireguardPropKeyMessageType = "message_type"
)
const (
wireguardTypeHandshakeInitiation = 1
wireguardTypeHandshakeResponse = 2
wireguardTypeData = 4
wireguardTypeCookieReply = 3
)
const (
wireguardSizeHandshakeInitiation = 148
wireguardSizeHandshakeResponse = 92
wireguardMinSizePacketData = 32 // 16 bytes header + 16 bytes AEAD overhead
wireguardSizePacketCookieReply = 64
)
type WireGuardAnalyzer struct{}
func (a *WireGuardAnalyzer) Name() string {
return "wireguard"
}
func (a *WireGuardAnalyzer) Limit() int {
return 0
}
func (a *WireGuardAnalyzer) NewUDP(info analyzer.UDPInfo, logger analyzer.Logger) analyzer.UDPStream {
return newWireGuardUDPStream(logger)
}
type wireGuardUDPStream struct {
logger analyzer.Logger
invalidCount int
rememberedIndexes *ring.Ring
rememberedIndexesLock sync.RWMutex
}
func newWireGuardUDPStream(logger analyzer.Logger) *wireGuardUDPStream {
return &wireGuardUDPStream{
logger: logger,
rememberedIndexes: ring.New(wireguardRememberedIndexCount),
}
}
func (s *wireGuardUDPStream) Feed(rev bool, data []byte) (u *analyzer.PropUpdate, done bool) {
m := s.parseWireGuardPacket(rev, data)
if m == nil {
s.invalidCount++
return nil, s.invalidCount >= wireguardUDPInvalidCountThreshold
}
s.invalidCount = 0 // Reset invalid count on valid WireGuard packet
messageType := m[wireguardPropKeyMessageType].(byte)
propUpdateType := analyzer.PropUpdateMerge
if messageType == wireguardTypeHandshakeInitiation {
propUpdateType = analyzer.PropUpdateReplace
}
return &analyzer.PropUpdate{
Type: propUpdateType,
M: m,
}, false
}
func (s *wireGuardUDPStream) Close(limited bool) *analyzer.PropUpdate {
return nil
}
func (s *wireGuardUDPStream) parseWireGuardPacket(rev bool, data []byte) analyzer.PropMap {
if len(data) < 4 {
return nil
}
if slices.Max(data[1:4]) != 0 {
return nil
}
messageType := data[0]
var propKey string
var propValue analyzer.PropMap
switch messageType {
case wireguardTypeHandshakeInitiation:
propKey = "handshake_initiation"
propValue = s.parseWireGuardHandshakeInitiation(rev, data)
case wireguardTypeHandshakeResponse:
propKey = "handshake_response"
propValue = s.parseWireGuardHandshakeResponse(rev, data)
case wireguardTypeData:
propKey = "packet_data"
propValue = s.parseWireGuardPacketData(rev, data)
case wireguardTypeCookieReply:
propKey = "packet_cookie_reply"
propValue = s.parseWireGuardPacketCookieReply(rev, data)
}
if propValue == nil {
return nil
}
m := make(analyzer.PropMap)
m[wireguardPropKeyMessageType] = messageType
m[propKey] = propValue
return m
}
func (s *wireGuardUDPStream) parseWireGuardHandshakeInitiation(rev bool, data []byte) analyzer.PropMap {
if len(data) != wireguardSizeHandshakeInitiation {
return nil
}
m := make(analyzer.PropMap)
senderIndex := binary.LittleEndian.Uint32(data[4:8])
m["sender_index"] = senderIndex
s.putSenderIndex(rev, senderIndex)
return m
}
func (s *wireGuardUDPStream) parseWireGuardHandshakeResponse(rev bool, data []byte) analyzer.PropMap {
if len(data) != wireguardSizeHandshakeResponse {
return nil
}
m := make(analyzer.PropMap)
senderIndex := binary.LittleEndian.Uint32(data[4:8])
m["sender_index"] = senderIndex
s.putSenderIndex(rev, senderIndex)
receiverIndex := binary.LittleEndian.Uint32(data[8:12])
m["receiver_index"] = receiverIndex
m["receiver_index_matched"] = s.matchReceiverIndex(rev, receiverIndex)
return m
}
func (s *wireGuardUDPStream) parseWireGuardPacketData(rev bool, data []byte) analyzer.PropMap {
if len(data) < wireguardMinSizePacketData {
return nil
}
if len(data)%16 != 0 {
// WireGuard zero padding the packet to make the length a multiple of 16
return nil
}
m := make(analyzer.PropMap)
receiverIndex := binary.LittleEndian.Uint32(data[4:8])
m["receiver_index"] = receiverIndex
m["receiver_index_matched"] = s.matchReceiverIndex(rev, receiverIndex)
m["counter"] = binary.LittleEndian.Uint64(data[8:16])
return m
}
func (s *wireGuardUDPStream) parseWireGuardPacketCookieReply(rev bool, data []byte) analyzer.PropMap {
if len(data) != wireguardSizePacketCookieReply {
return nil
}
m := make(analyzer.PropMap)
receiverIndex := binary.LittleEndian.Uint32(data[4:8])
m["receiver_index"] = receiverIndex
m["receiver_index_matched"] = s.matchReceiverIndex(rev, receiverIndex)
return m
}
type wireGuardIndex struct {
SenderIndex uint32
Reverse bool
}
func (s *wireGuardUDPStream) putSenderIndex(rev bool, senderIndex uint32) {
s.rememberedIndexesLock.Lock()
defer s.rememberedIndexesLock.Unlock()
s.rememberedIndexes.Value = &wireGuardIndex{
SenderIndex: senderIndex,
Reverse: rev,
}
s.rememberedIndexes = s.rememberedIndexes.Prev()
}
func (s *wireGuardUDPStream) matchReceiverIndex(rev bool, receiverIndex uint32) bool {
s.rememberedIndexesLock.RLock()
defer s.rememberedIndexesLock.RUnlock()
var found bool
ris := s.rememberedIndexes
for it := ris.Next(); it != ris; it = it.Next() {
if it.Value == nil {
break
}
wgidx := it.Value.(*wireGuardIndex)
if wgidx.Reverse == !rev && wgidx.SenderIndex == receiverIndex {
found = true
break
}
}
return found
}

4
analyzer/utils/lsm.go
View File

@ -44,10 +44,6 @@ func (lsm *LinearStateMachine) Run() (cancelled bool, done bool) {
return false, true
}
func (lsm *LinearStateMachine) AppendSteps(steps ...func() LSMAction) {
lsm.Steps = append(lsm.Steps, steps...)
}
func (lsm *LinearStateMachine) Reset() {
lsm.index = 0
lsm.cancelled = false

205
cmd/root.go
View File

@ -5,9 +5,8 @@ import (
"fmt"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/tcp"
@ -17,7 +16,6 @@ import (
"github.com/apernet/OpenGFW/modifier"
modUDP "github.com/apernet/OpenGFW/modifier/udp"
"github.com/apernet/OpenGFW/ruleset"
"github.com/apernet/OpenGFW/ruleset/builtins/geo"
"github.com/spf13/cobra"
"github.com/spf13/viper"
@ -43,7 +41,6 @@ var logger *zap.Logger
// Flags
var (
cfgFile string
pcapFile string
logLevel string
logFormat string
)
@ -90,14 +87,10 @@ var logFormatMap = map[string]zapcore.EncoderConfig{
var analyzers = []analyzer.Analyzer{
&tcp.FETAnalyzer{},
&tcp.HTTPAnalyzer{},
&tcp.SocksAnalyzer{},
&tcp.SSHAnalyzer{},
&tcp.TLSAnalyzer{},
&tcp.TrojanAnalyzer{},
&udp.DNSAnalyzer{},
&udp.OpenVPNAnalyzer{},
&udp.QUICAnalyzer{},
&udp.WireGuardAnalyzer{},
}
var modifiers = []modifier.Modifier{
@ -119,7 +112,6 @@ func init() {
func initFlags() {
rootCmd.PersistentFlags().StringVarP(&cfgFile, "config", "c", "", "config file")
rootCmd.PersistentFlags().StringVarP(&pcapFile, "pcap", "p", "", "pcap file (optional)")
rootCmd.PersistentFlags().StringVarP(&logLevel, "log-level", "l", envOrDefaultString(appLogLevelEnv, "info"), "log level")
rootCmd.PersistentFlags().StringVarP(&logFormat, "log-format", "f", envOrDefaultString(appLogFormatEnv, "console"), "log format")
}
@ -168,39 +160,19 @@ func initLogger() {
type cliConfig struct {
IO cliConfigIO `mapstructure:"io"`
Workers cliConfigWorkers `mapstructure:"workers"`
Ruleset cliConfigRuleset `mapstructure:"ruleset"`
Replay cliConfigReplay `mapstructure:"replay"`
}
type cliConfigIO struct {
QueueSize uint32 `mapstructure:"queueSize"`
QueueNum *uint16 `mapstructure:"queueNum"`
Table string `mapstructure:"table"`
ConnMarkAccept uint32 `mapstructure:"connMarkAccept"`
ConnMarkDrop uint32 `mapstructure:"connMarkDrop"`
ReadBuffer int `mapstructure:"rcvBuf"`
WriteBuffer int `mapstructure:"sndBuf"`
Local bool `mapstructure:"local"`
RST bool `mapstructure:"rst"`
}
type cliConfigReplay struct {
Realtime bool `mapstructure:"realtime"`
QueueSize uint32 `mapstructure:"queueSize"`
Local bool `mapstructure:"local"`
}
type cliConfigWorkers struct {
Count int `mapstructure:"count"`
QueueSize int `mapstructure:"queueSize"`
TCPMaxBufferedPagesTotal int `mapstructure:"tcpMaxBufferedPagesTotal"`
TCPMaxBufferedPagesPerConn int `mapstructure:"tcpMaxBufferedPagesPerConn"`
TCPTimeout time.Duration `mapstructure:"tcpTimeout"`
UDPMaxStreams int `mapstructure:"udpMaxStreams"`
}
type cliConfigRuleset struct {
GeoIp string `mapstructure:"geoip"`
GeoSite string `mapstructure:"geosite"`
Count int `mapstructure:"count"`
QueueSize int `mapstructure:"queueSize"`
TCPMaxBufferedPagesTotal int `mapstructure:"tcpMaxBufferedPagesTotal"`
TCPMaxBufferedPagesPerConn int `mapstructure:"tcpMaxBufferedPagesPerConn"`
UDPMaxStreams int `mapstructure:"udpMaxStreams"`
}
func (c *cliConfig) fillLogger(config *engine.Config) error {
@ -209,35 +181,14 @@ func (c *cliConfig) fillLogger(config *engine.Config) error {
}
func (c *cliConfig) fillIO(config *engine.Config) error {
var ioImpl io.PacketIO
var err error
if pcapFile != "" {
// Setup IO for pcap file replay
logger.Info("replaying from pcap file", zap.String("pcap file", pcapFile))
ioImpl, err = io.NewPcapPacketIO(io.PcapPacketIOConfig{
PcapFile: pcapFile,
Realtime: c.Replay.Realtime,
})
} else {
// Setup IO for nfqueue
ioImpl, err = io.NewNFQueuePacketIO(io.NFQueuePacketIOConfig{
QueueSize: c.IO.QueueSize,
QueueNum: c.IO.QueueNum,
Table: c.IO.Table,
ConnMarkAccept: c.IO.ConnMarkAccept,
ConnMarkDrop: c.IO.ConnMarkDrop,
ReadBuffer: c.IO.ReadBuffer,
WriteBuffer: c.IO.WriteBuffer,
Local: c.IO.Local,
RST: c.IO.RST,
})
}
nfio, err := io.NewNFQueuePacketIO(io.NFQueuePacketIOConfig{
QueueSize: c.IO.QueueSize,
Local: c.IO.Local,
})
if err != nil {
return configError{Field: "io", Err: err}
}
config.IO = ioImpl
config.IOs = []io.PacketIO{nfio}
return nil
}
@ -246,7 +197,6 @@ func (c *cliConfig) fillWorkers(config *engine.Config) error {
config.WorkerQueueSize = c.Workers.QueueSize
config.WorkerTCPMaxBufferedPagesTotal = c.Workers.TCPMaxBufferedPagesTotal
config.WorkerTCPMaxBufferedPagesPerConn = c.Workers.TCPMaxBufferedPagesPerConn
config.WorkerTCPTimeout = c.Workers.TCPTimeout
config.WorkerUDPMaxStreams = c.Workers.UDPMaxStreams
return nil
}
@ -281,19 +231,19 @@ func runMain(cmd *cobra.Command, args []string) {
if err != nil {
logger.Fatal("failed to parse config", zap.Error(err))
}
defer engineConfig.IO.Close() // Make sure to close IO on exit
defer func() {
// Make sure to close all IOs on exit
for _, i := range engineConfig.IOs {
_ = i.Close()
}
}()
// Ruleset
rawRs, err := ruleset.ExprRulesFromYAML(args[0])
if err != nil {
logger.Fatal("failed to load rules", zap.Error(err))
}
rsConfig := &ruleset.BuiltinConfig{
Logger: &rulesetLogger{},
GeoMatcher: geo.NewGeoMatcher(config.Ruleset.GeoSite, config.Ruleset.GeoIp),
ProtectedDialContext: engineConfig.IO.ProtectedDialContext,
}
rs, err := ruleset.CompileExprRules(rawRs, analyzers, modifiers, rsConfig)
rs, err := ruleset.CompileExprRules(rawRs, analyzers, modifiers)
if err != nil {
logger.Fatal("failed to compile rules", zap.Error(err))
}
@ -305,42 +255,14 @@ func runMain(cmd *cobra.Command, args []string) {
logger.Fatal("failed to initialize engine", zap.Error(err))
}
// Signal handling
ctx, cancelFunc := context.WithCancel(context.Background())
go func() {
// Graceful shutdown
shutdownChan := make(chan os.Signal, 1)
signal.Notify(shutdownChan, os.Interrupt, syscall.SIGTERM)
<-shutdownChan
sigChan := make(chan os.Signal)
signal.Notify(sigChan, os.Interrupt, os.Kill)
<-sigChan
logger.Info("shutting down gracefully...")
cancelFunc()
}()
go func() {
// Rule reload
reloadChan := make(chan os.Signal, 1)
signal.Notify(reloadChan, syscall.SIGHUP)
for {
<-reloadChan
logger.Info("reloading rules")
rawRs, err := ruleset.ExprRulesFromYAML(args[0])
if err != nil {
logger.Error("failed to load rules, using old rules", zap.Error(err))
continue
}
rs, err := ruleset.CompileExprRules(rawRs, analyzers, modifiers, rsConfig)
if err != nil {
logger.Error("failed to compile rules, using old rules", zap.Error(err))
continue
}
err = en.UpdateRuleset(rs)
if err != nil {
logger.Error("failed to update ruleset", zap.Error(err))
} else {
logger.Info("rules reloaded")
}
}
}()
logger.Info("engine started")
logger.Info("engine exited", zap.Error(en.Run(ctx)))
}
@ -373,26 +295,12 @@ func (l *engineLogger) TCPStreamPropUpdate(info ruleset.StreamInfo, close bool)
}
func (l *engineLogger) TCPStreamAction(info ruleset.StreamInfo, action ruleset.Action, noMatch bool) {
if noMatch {
logger.Debug("TCP stream no match",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()))
} else {
logger.Info("TCP stream action",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()))
}
}
func (l *engineLogger) TCPFlush(workerID, flushed, closed int) {
logger.Debug("TCP flush",
zap.Int("workerID", workerID),
zap.Int("flushed", flushed),
zap.Int("closed", closed))
logger.Info("TCP stream action",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()),
zap.Bool("noMatch", noMatch))
}
func (l *engineLogger) UDPStreamNew(workerID int, info ruleset.StreamInfo) {
@ -413,19 +321,20 @@ func (l *engineLogger) UDPStreamPropUpdate(info ruleset.StreamInfo, close bool)
}
func (l *engineLogger) UDPStreamAction(info ruleset.StreamInfo, action ruleset.Action, noMatch bool) {
if noMatch {
logger.Debug("UDP stream no match",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()))
} else {
logger.Info("UDP stream action",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()))
}
logger.Info("UDP stream action",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.String("action", action.String()),
zap.Bool("noMatch", noMatch))
}
func (l *engineLogger) MatchError(info ruleset.StreamInfo, err error) {
logger.Error("match error",
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.Error(err))
}
func (l *engineLogger) ModifyError(info ruleset.StreamInfo, err error) {
@ -457,29 +366,17 @@ func (l *engineLogger) AnalyzerErrorf(streamID int64, name string, format string
zap.String("msg", fmt.Sprintf(format, args...)))
}
type rulesetLogger struct{}
func (l *rulesetLogger) Log(info ruleset.StreamInfo, name string) {
logger.Info("ruleset log",
zap.String("name", name),
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.Any("props", info.Props))
}
func (l *rulesetLogger) MatchError(info ruleset.StreamInfo, name string, err error) {
logger.Error("ruleset match error",
zap.String("name", name),
zap.Int64("id", info.ID),
zap.String("src", info.SrcString()),
zap.String("dst", info.DstString()),
zap.Error(err))
}
func envOrDefaultString(key, def string) string {
if v := os.Getenv(key); v != "" {
return v
}
return def
}
func envOrDefaultBool(key string, def bool) bool {
if v := os.Getenv(key); v != "" {
b, _ := strconv.ParseBool(v)
return b
}
return def
}

BIN
docs/logo.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 44 KiB

48
engine/engine.go
View File

@ -15,7 +15,7 @@ var _ Engine = (*engine)(nil)
type engine struct {
logger Logger
io io.PacketIO
ioList []io.PacketIO
workers []*worker
}
@ -34,7 +34,6 @@ func NewEngine(config Config) (Engine, error) {
Ruleset: config.Ruleset,
TCPMaxBufferedPagesTotal: config.WorkerTCPMaxBufferedPagesTotal,
TCPMaxBufferedPagesPerConn: config.WorkerTCPMaxBufferedPagesPerConn,
TCPTimeout: config.WorkerTCPTimeout,
UDPMaxStreams: config.WorkerUDPMaxStreams,
})
if err != nil {
@ -43,7 +42,7 @@ func NewEngine(config Config) (Engine, error) {
}
return &engine{
logger: config.Logger,
io: config.IO,
ioList: config.IOs,
workers: workers,
}, nil
}
@ -58,30 +57,28 @@ func (e *engine) UpdateRuleset(r ruleset.Ruleset) error {
}
func (e *engine) Run(ctx context.Context) error {
workerCtx, workerCancel := context.WithCancel(ctx)
defer workerCancel() // Stop workers
// Register IO shutdown
ioCtx, ioCancel := context.WithCancel(ctx)
e.io.SetCancelFunc(ioCancel)
defer ioCancel() // Stop IO
defer ioCancel() // Stop workers & IOs
// Start workers
for _, w := range e.workers {
go w.Run(workerCtx)
go w.Run(ioCtx)
}
// Register IO callback
errChan := make(chan error, 1)
err := e.io.Register(ioCtx, func(p io.Packet, err error) bool {
// Register callbacks
errChan := make(chan error, len(e.ioList))
for _, i := range e.ioList {
ioEntry := i // Make sure dispatch() uses the correct ioEntry
err := ioEntry.Register(ioCtx, func(p io.Packet, err error) bool {
if err != nil {
errChan <- err
return false
}
return e.dispatch(ioEntry, p)
})
if err != nil {
errChan <- err
return false
return err
}
return e.dispatch(p)
})
if err != nil {
return err
}
// Block until IO errors or context is cancelled
@ -90,13 +87,12 @@ func (e *engine) Run(ctx context.Context) error {
return err
case <-ctx.Done():
return nil
case <-ioCtx.Done():
return nil
}
}
// dispatch dispatches a packet to a worker.
func (e *engine) dispatch(p io.Packet) bool {
// This must be safe for concurrent use, as it may be called from multiple IOs.
func (e *engine) dispatch(ioEntry io.PacketIO, p io.Packet) bool {
data := p.Data()
ipVersion := data[0] >> 4
var layerType gopacket.LayerType
@ -106,19 +102,17 @@ func (e *engine) dispatch(p io.Packet) bool {
layerType = layers.LayerTypeIPv6
} else {
// Unsupported network layer
_ = e.io.SetVerdict(p, io.VerdictAcceptStream, nil)
_ = ioEntry.SetVerdict(p, io.VerdictAcceptStream, nil)
return true
}
// Convert to gopacket.Packet
packet := gopacket.NewPacket(data, layerType, gopacket.DecodeOptions{Lazy: true, NoCopy: true})
packet.Metadata().Timestamp = p.Timestamp()
// Load balance by stream ID
index := p.StreamID() % uint32(len(e.workers))
packet := gopacket.NewPacket(data, layerType, gopacket.DecodeOptions{Lazy: true, NoCopy: true})
e.workers[index].Feed(&workerPacket{
StreamID: p.StreamID(),
Packet: packet,
SetVerdict: func(v io.Verdict, b []byte) error {
return e.io.SetVerdict(p, v, b)
return ioEntry.SetVerdict(p, v, b)
},
})
return true

6
engine/interface.go
View File

@ -2,7 +2,6 @@ package engine
import (
"context"
"time"
"github.com/apernet/OpenGFW/io"
"github.com/apernet/OpenGFW/ruleset"
@ -19,14 +18,13 @@ type Engine interface {
// Config is the configuration for the engine.
type Config struct {
Logger Logger
IO io.PacketIO
IOs []io.PacketIO
Ruleset ruleset.Ruleset
Workers int // Number of workers. Zero or negative means auto (number of CPU cores).
WorkerQueueSize int
WorkerTCPMaxBufferedPagesTotal int
WorkerTCPMaxBufferedPagesPerConn int
WorkerTCPTimeout time.Duration
WorkerUDPMaxStreams int
}
@ -38,12 +36,12 @@ type Logger interface {
TCPStreamNew(workerID int, info ruleset.StreamInfo)
TCPStreamPropUpdate(info ruleset.StreamInfo, close bool)
TCPStreamAction(info ruleset.StreamInfo, action ruleset.Action, noMatch bool)
TCPFlush(workerID, flushed, closed int)
UDPStreamNew(workerID int, info ruleset.StreamInfo)
UDPStreamPropUpdate(info ruleset.StreamInfo, close bool)
UDPStreamAction(info ruleset.StreamInfo, action ruleset.Action, noMatch bool)
MatchError(info ruleset.StreamInfo, err error)
ModifyError(info ruleset.StreamInfo, err error)
AnalyzerDebugf(streamID int64, name string, format string, args ...interface{})

30
engine/tcp.go
View File

@ -60,6 +60,13 @@ func (f *tcpStreamFactory) New(ipFlow, tcpFlow gopacket.Flow, tcp *layers.TCP, a
rs := f.Ruleset
f.RulesetMutex.RUnlock()
ans := analyzersToTCPAnalyzers(rs.Analyzers(info))
if len(ans) == 0 {
ctx := ac.(*tcpContext)
ctx.Verdict = tcpVerdictAcceptStream
f.Logger.TCPStreamAction(info, ruleset.ActionAllow, true)
// a tcpStream with no activeEntries is a no-op
return &tcpStream{}
}
// Create entries for each analyzer
entries := make([]*tcpStreamEntry, 0, len(ans))
for _, a := range ans {
@ -102,7 +109,6 @@ type tcpStream struct {
ruleset ruleset.Ruleset
activeEntries []*tcpStreamEntry
doneEntries []*tcpStreamEntry
lastVerdict tcpVerdict
}
type tcpStreamEntry struct {
@ -113,16 +119,8 @@ type tcpStreamEntry struct {
}
func (s *tcpStream) Accept(tcp *layers.TCP, ci gopacket.CaptureInfo, dir reassembly.TCPFlowDirection, nextSeq reassembly.Sequence, start *bool, ac reassembly.AssemblerContext) bool {
if len(s.activeEntries) > 0 || s.virgin {
// Make sure every stream matches against the ruleset at least once,
// even if there are no activeEntries, as the ruleset may have built-in
// properties that need to be matched.
return true
} else {
ctx := ac.(*tcpContext)
ctx.Verdict = s.lastVerdict
return false
}
// Only accept packets if we still have active entries
return len(s.activeEntries) > 0
}
func (s *tcpStream) ReassembledSG(sg reassembly.ScatterGather, ac reassembly.AssemblerContext) {
@ -148,12 +146,13 @@ func (s *tcpStream) ReassembledSG(sg reassembly.ScatterGather, ac reassembly.Ass
s.virgin = false
s.logger.TCPStreamPropUpdate(s.info, false)
// Match properties against ruleset
result := s.ruleset.Match(s.info)
result, err := s.ruleset.Match(s.info)
if err != nil {
s.logger.MatchError(s.info, err)
}
action := result.Action
if action != ruleset.ActionMaybe && action != ruleset.ActionModify {
verdict := actionToTCPVerdict(action)
s.lastVerdict = verdict
ctx.Verdict = verdict
ctx.Verdict = actionToTCPVerdict(action)
s.logger.TCPStreamAction(s.info, action, false)
// Verdict issued, no need to process any more packets
s.closeActiveEntries()
@ -161,7 +160,6 @@ func (s *tcpStream) ReassembledSG(sg reassembly.ScatterGather, ac reassembly.Ass
}
if len(s.activeEntries) == 0 && ctx.Verdict == tcpVerdictAccept {
// All entries are done but no verdict issued, accept stream
s.lastVerdict = tcpVerdictAcceptStream
ctx.Verdict = tcpVerdictAcceptStream
s.logger.TCPStreamAction(s.info, ruleset.ActionAllow, true)
}

30
engine/udp.go
View File

@ -61,6 +61,12 @@ func (f *udpStreamFactory) New(ipFlow, udpFlow gopacket.Flow, udp *layers.UDP, u
rs := f.Ruleset
f.RulesetMutex.RUnlock()
ans := analyzersToUDPAnalyzers(rs.Analyzers(info))
if len(ans) == 0 {
uc.Verdict = udpVerdictAcceptStream
f.Logger.UDPStreamAction(info, ruleset.ActionAllow, true)
// a udpStream with no activeEntries is a no-op
return &udpStream{}
}
// Create entries for each analyzer
entries := make([]*udpStreamEntry, 0, len(ans))
for _, a := range ans {
@ -161,7 +167,6 @@ type udpStream struct {
ruleset ruleset.Ruleset
activeEntries []*udpStreamEntry
doneEntries []*udpStreamEntry
lastVerdict udpVerdict
}
type udpStreamEntry struct {
@ -172,15 +177,8 @@ type udpStreamEntry struct {
}
func (s *udpStream) Accept(udp *layers.UDP, rev bool, uc *udpContext) bool {
if len(s.activeEntries) > 0 || s.virgin {
// Make sure every stream matches against the ruleset at least once,
// even if there are no activeEntries, as the ruleset may have built-in
// properties that need to be matched.
return true
} else {
uc.Verdict = s.lastVerdict
return false
}
// Only accept packets if we still have active entries
return len(s.activeEntries) > 0
}
func (s *udpStream) Feed(udp *layers.UDP, rev bool, uc *udpContext) {
@ -201,7 +199,10 @@ func (s *udpStream) Feed(udp *layers.UDP, rev bool, uc *udpContext) {
s.virgin = false
s.logger.UDPStreamPropUpdate(s.info, false)
// Match properties against ruleset
result := s.ruleset.Match(s.info)
result, err := s.ruleset.Match(s.info)
if err != nil {
s.logger.MatchError(s.info, err)
}
action := result.Action
if action == ruleset.ActionModify {
// Call the modifier instance
@ -211,7 +212,6 @@ func (s *udpStream) Feed(udp *layers.UDP, rev bool, uc *udpContext) {
s.logger.ModifyError(s.info, errInvalidModifier)
action = ruleset.ActionMaybe
} else {
var err error
uc.Packet, err = udpMI.Process(udp.Payload)
if err != nil {
// Modifier error, fallback to maybe
@ -221,9 +221,8 @@ func (s *udpStream) Feed(udp *layers.UDP, rev bool, uc *udpContext) {
}
}
if action != ruleset.ActionMaybe {
verdict, final := actionToUDPVerdict(action)
s.lastVerdict = verdict
uc.Verdict = verdict
var final bool
uc.Verdict, final = actionToUDPVerdict(action)
s.logger.UDPStreamAction(s.info, action, false)
if final {
s.closeActiveEntries()
@ -232,7 +231,6 @@ func (s *udpStream) Feed(udp *layers.UDP, rev bool, uc *udpContext) {
}
if len(s.activeEntries) == 0 && uc.Verdict == udpVerdictAccept {
// All entries are done but no verdict issued, accept stream
s.lastVerdict = udpVerdictAcceptStream
uc.Verdict = udpVerdictAcceptStream
s.logger.UDPStreamAction(s.info, ruleset.ActionAllow, true)
}

30
engine/worker.go
View File

@ -2,7 +2,6 @@ package engine
import (
"context"
"time"
"github.com/apernet/OpenGFW/io"
"github.com/apernet/OpenGFW/ruleset"
@ -15,12 +14,9 @@ import (
const (
defaultChanSize = 64
defaultTCPMaxBufferedPagesTotal = 65536
defaultTCPMaxBufferedPagesPerConnection = 16
defaultTCPTimeout = 10 * time.Minute
defaultTCPMaxBufferedPagesTotal = 4096
defaultTCPMaxBufferedPagesPerConnection = 64
defaultUDPMaxStreams = 4096
tcpFlushInterval = 1 * time.Minute
)
type workerPacket struct {
@ -37,7 +33,6 @@ type worker struct {
tcpStreamFactory *tcpStreamFactory
tcpStreamPool *reassembly.StreamPool
tcpAssembler *reassembly.Assembler
tcpTimeout time.Duration
udpStreamFactory *udpStreamFactory
udpStreamManager *udpStreamManager
@ -52,7 +47,6 @@ type workerConfig struct {
Ruleset ruleset.Ruleset
TCPMaxBufferedPagesTotal int
TCPMaxBufferedPagesPerConn int
TCPTimeout time.Duration
UDPMaxStreams int
}
@ -66,9 +60,6 @@ func (c *workerConfig) fillDefaults() {
if c.TCPMaxBufferedPagesPerConn <= 0 {
c.TCPMaxBufferedPagesPerConn = defaultTCPMaxBufferedPagesPerConnection
}
if c.TCPTimeout <= 0 {
c.TCPTimeout = defaultTCPTimeout
}
if c.UDPMaxStreams <= 0 {
c.UDPMaxStreams = defaultUDPMaxStreams
}
@ -107,7 +98,6 @@ func newWorker(config workerConfig) (*worker, error) {
tcpStreamFactory: tcpSF,
tcpStreamPool: tcpStreamPool,
tcpAssembler: tcpAssembler,
tcpTimeout: config.TCPTimeout,
udpStreamFactory: udpSF,
udpStreamManager: udpSM,
modSerializeBuffer: gopacket.NewSerializeBuffer(),
@ -121,10 +111,6 @@ func (w *worker) Feed(p *workerPacket) {
func (w *worker) Run(ctx context.Context) {
w.logger.WorkerStart(w.id)
defer w.logger.WorkerStop(w.id)
tcpFlushTicker := time.NewTicker(tcpFlushInterval)
defer tcpFlushTicker.Stop()
for {
select {
case <-ctx.Done():
@ -136,17 +122,12 @@ func (w *worker) Run(ctx context.Context) {
}
v, b := w.handle(wPkt.StreamID, wPkt.Packet)
_ = wPkt.SetVerdict(v, b)
case <-tcpFlushTicker.C:
w.flushTCP(w.tcpTimeout)
}
}
}
func (w *worker) UpdateRuleset(r ruleset.Ruleset) error {
if err := w.tcpStreamFactory.UpdateRuleset(r); err != nil {
return err
}
return w.udpStreamFactory.UpdateRuleset(r)
return w.tcpStreamFactory.UpdateRuleset(r)
}
func (w *worker) handle(streamID uint32, p gopacket.Packet) (io.Verdict, []byte) {
@ -192,11 +173,6 @@ func (w *worker) handleTCP(ipFlow gopacket.Flow, pMeta *gopacket.PacketMetadata,
return io.Verdict(ctx.Verdict)
}
func (w *worker) flushTCP(timeout time.Duration) {
flushed, closed := w.tcpAssembler.FlushCloseOlderThan(time.Now().Add(-timeout))
w.logger.TCPFlush(w.id, flushed, closed)
}
func (w *worker) handleUDP(streamID uint32, ipFlow gopacket.Flow, udp *layers.UDP) (io.Verdict, []byte) {
ctx := &udpContext{
Verdict: udpVerdictAccept,

11
go.mod
View File

@ -1,22 +1,18 @@
module github.com/apernet/OpenGFW
go 1.21
go 1.20
require (
github.com/bwmarrin/snowflake v0.3.0
github.com/coreos/go-iptables v0.7.0
github.com/expr-lang/expr v1.16.3
github.com/expr-lang/expr v1.15.7
github.com/florianl/go-nfqueue v1.3.2-0.20231218173729-f2bdeb033acf
github.com/google/gopacket v1.1.20-0.20220810144506-32ee38206866
github.com/google/gopacket v1.1.19
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/mdlayher/netlink v1.6.0
github.com/quic-go/quic-go v0.41.0
github.com/spf13/cobra v1.8.0
github.com/spf13/viper v1.18.2
go.uber.org/zap v1.26.0
golang.org/x/crypto v0.19.0
golang.org/x/sys v0.17.0
google.golang.org/protobuf v1.31.0
gopkg.in/yaml.v3 v3.0.1
)
@ -41,6 +37,7 @@ require (
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
)

46
go.sum
View File

@ -6,29 +6,19 @@ github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46t
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/expr-lang/expr v1.16.3 h1:NLldf786GffptcXNxxJx5dQ+FzeWDKChBDqOOwyK8to=
github.com/expr-lang/expr v1.16.3/go.mod h1:uCkhfG+x7fcZ5A5sXHKuQ07jGZRl6J0FCAaf2k4PtVQ=
github.com/expr-lang/expr v1.15.7 h1:BK0JcWUkoW6nrbLBo6xCKhz4BvH5DSOOu1Gx5lucyZo=
github.com/expr-lang/expr v1.15.7/go.mod h1:uCkhfG+x7fcZ5A5sXHKuQ07jGZRl6J0FCAaf2k4PtVQ=
github.com/florianl/go-nfqueue v1.3.2-0.20231218173729-f2bdeb033acf h1:NqGS3vTHzVENbIfd87cXZwdpO6MB2R1PjHMJLi4Z3ow=
github.com/florianl/go-nfqueue v1.3.2-0.20231218173729-f2bdeb033acf/go.mod h1:eSnAor2YCfMCVYrVNEhkLGN/r1L+J4uDjc0EUy0tfq4=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gopacket v1.1.20-0.20220810144506-32ee38206866 h1:NaJi58bCZZh0jjPw78EqDZekPEfhlzYE01C5R+zh1tE=
github.com/google/gopacket v1.1.20-0.20220810144506-32ee38206866/go.mod h1:riddUzxTSBpJXk3qBHtYr4qOhFhT6k/1c0E3qkQjQpA=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 h1:yAJXTCF9TqKcTiHJAE8dj7HMvPfh66eeA2JYW7eFpSE=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=
github.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
@ -38,9 +28,7 @@ github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLf
github.com/josharian/native v1.0.0 h1:Ts/E8zCSEsG17dUqv7joXJFybuMLjQfWE04tsBODTxk=
github.com/josharian/native v1.0.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mdlayher/netlink v1.6.0 h1:rOHX5yl7qnlpiVkFWoqccueppMtXzeziFjWAjLg6sz0=
@ -49,19 +37,11 @@ github.com/mdlayher/socket v0.1.1 h1:q3uOGirUPfAV2MUoaC7BavjQ154J7+JOkTWyiV+intI
github.com/mdlayher/socket v0.1.1/go.mod h1:mYV5YIZAfHh4dzDVzI8x8tWLWCliuX8Mon5Awbj+qDs=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/onsi/ginkgo/v2 v2.9.5 h1:+6Hr4uxzP4XIUyAkg61dWBw8lb/gc4/X5luuxN/EC+Q=
github.com/onsi/ginkgo/v2 v2.9.5/go.mod h1:tvAoo1QUJwNEU2ITftXTpR7R1RbCzoZUOs3RonqW57k=
github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE=
github.com/onsi/gomega v1.27.6/go.mod h1:PIQNjfQwkP3aQAH7lf7j87O/5FiNr+ZR8+ipb+qQlhg=
github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/quic-go v0.41.0 h1:aD8MmHfgqTURWNJy48IYFg2OnxwHT3JL7ahGs73lb4k=
github.com/quic-go/quic-go v0.41.0/go.mod h1:qCkNjqczPEvgsOnxZ0eCD14lv+B2LHlFAB++CNOh9hA=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.4.0 h1:HApY1R9zGo4DBgr7dqsTH/JJxLTTsOt7u6keLGt6kNQ=
github.com/sagikazarmark/locafero v0.4.0/go.mod h1:Pe1W6UlPYUk/+wc/6KFhbORCfqzgYEpgQ3O5fPuL3H4=
@ -88,19 +68,13 @@ github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcU
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/vishvananda/netlink v1.1.0/go.mod h1:cTgwzPIzzgDAYoQrMm0EdrjRUBkTqKYppBueQtXaqoE=
github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df/go.mod h1:JP3t17pCcGlemwknint6hfoeCVQrEMVwxRLRjXpq+BU=
github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.19.0 h1:ENy+Az/9Y1vSrlrvBSyna3PITt4tiZLf7sgCjZBX7Wo=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 h1:GoHiUyI/Tp2nVkLI2mCxVkOjsbSXD66ic0XW0js0R9g=
golang.org/x/exp v0.0.0-20230905200255-921286631fa9/go.mod h1:S2oDrQGGwySpoQPVqRShND87VCbxmc6bL1Yd2oYrm6k=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
@ -117,16 +91,14 @@ golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606203320-7fc4e5ec1444/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@ -136,17 +108,11 @@ golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.13.0 h1:Iey4qkscZuv0VvIt8E0neZjtPVQFSc870HQ448QgEmQ=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

12
io/interface.go
View File

@ -2,8 +2,6 @@ package io
import (
"context"
"net"
"time"
)
type Verdict int
@ -25,14 +23,13 @@ const (
type Packet interface {
// StreamID is the ID of the stream the packet belongs to.
StreamID() uint32
// Timestamp is the time the packet was received.
Timestamp() time.Time
// Data is the raw packet data, starting with the IP header.
Data() []byte
}
// PacketCallback is called for each packet received.
// Return false to "unregister" and stop receiving packets.
// It must be safe for concurrent use.
type PacketCallback func(Packet, error) bool
type PacketIO interface {
@ -42,15 +39,8 @@ type PacketIO interface {
Register(context.Context, PacketCallback) error
// SetVerdict sets the verdict for a packet.
SetVerdict(Packet, Verdict, []byte) error
// ProtectedDialContext is like net.DialContext, but the connection is "protected"
// in the sense that the packets sent/received through the connection must bypass
// the packet IO and not be processed by the callback.
ProtectedDialContext(ctx context.Context, network, address string) (net.Conn, error)
// Close closes the packet IO.
Close() error
// SetCancelFunc gives packet IO access to context cancel function, enabling it to
// trigger a shutdown
SetCancelFunc(cancelFunc context.CancelFunc) error
}
type ErrInvalidPacket struct {

377
io/nfqueue.go
View File

@ -4,88 +4,39 @@ import (
"context"
"encoding/binary"
"errors"
"fmt"
"net"
"os/exec"
"strconv"
"strings"
"syscall"
"time"
"github.com/coreos/go-iptables/iptables"
"github.com/florianl/go-nfqueue"
"github.com/mdlayher/netlink"
"golang.org/x/sys/unix"
)
const (
nfqueueDefaultQueueNum = 100
nfqueueNum = 100
nfqueueMaxPacketLen = 0xFFFF
nfqueueDefaultQueueSize = 128
nfqueueDefaultConnMarkAccept = 1001
nftFamily = "inet"
nftDefaultTable = "opengfw"
nfqueueConnMarkAccept = 1001
nfqueueConnMarkDrop = 1002
)
func (n *nfqueuePacketIO) generateNftRules() (*nftTableSpec, error) {
if n.local && n.rst {
return nil, errors.New("tcp rst is not supported in local mode")
}
table := &nftTableSpec{
Family: nftFamily,
Table: n.table,
}
table.Defines = append(table.Defines, fmt.Sprintf("define ACCEPT_CTMARK=%d", n.connMarkAccept))
table.Defines = append(table.Defines, fmt.Sprintf("define DROP_CTMARK=%d", n.connMarkDrop))
table.Defines = append(table.Defines, fmt.Sprintf("define QUEUE_NUM=%d", n.queueNum))
if n.local {
table.Chains = []nftChainSpec{
{Chain: "INPUT", Header: "type filter hook input priority filter; policy accept;"},
{Chain: "OUTPUT", Header: "type filter hook output priority filter; policy accept;"},
}
} else {
table.Chains = []nftChainSpec{
{Chain: "FORWARD", Header: "type filter hook forward priority filter; policy accept;"},
}
}
for i := range table.Chains {
c := &table.Chains[i]
c.Rules = append(c.Rules, "meta mark $ACCEPT_CTMARK ct mark set $ACCEPT_CTMARK") // Bypass protected connections
c.Rules = append(c.Rules, "ct mark $ACCEPT_CTMARK counter accept")
if n.rst {
c.Rules = append(c.Rules, "ip protocol tcp ct mark $DROP_CTMARK counter reject with tcp reset")
}
c.Rules = append(c.Rules, "ct mark $DROP_CTMARK counter drop")
c.Rules = append(c.Rules, "counter queue num $QUEUE_NUM bypass")
}
return table, nil
var iptRulesForward = []iptRule{
{"filter", "FORWARD", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkAccept), "-j", "ACCEPT"}},
{"filter", "FORWARD", []string{"-p", "tcp", "-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "REJECT", "--reject-with", "tcp-reset"}},
{"filter", "FORWARD", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "DROP"}},
{"filter", "FORWARD", []string{"-j", "NFQUEUE", "--queue-num", strconv.Itoa(nfqueueNum), "--queue-bypass"}},
}
func (n *nfqueuePacketIO) generateIptRules() ([]iptRule, error) {
if n.local && n.rst {
return nil, errors.New("tcp rst is not supported in local mode")
}
var chains []string
if n.local {
chains = []string{"INPUT", "OUTPUT"}
} else {
chains = []string{"FORWARD"}
}
rules := make([]iptRule, 0, 4*len(chains))
for _, chain := range chains {
// Bypass protected connections
rules = append(rules, iptRule{"filter", chain, []string{"-m", "mark", "--mark", strconv.Itoa(n.connMarkAccept), "-j", "CONNMARK", "--set-mark", strconv.Itoa(n.connMarkAccept)}})
rules = append(rules, iptRule{"filter", chain, []string{"-m", "connmark", "--mark", strconv.Itoa(n.connMarkAccept), "-j", "ACCEPT"}})
if n.rst {
rules = append(rules, iptRule{"filter", chain, []string{"-p", "tcp", "-m", "connmark", "--mark", strconv.Itoa(n.connMarkDrop), "-j", "REJECT", "--reject-with", "tcp-reset"}})
}
rules = append(rules, iptRule{"filter", chain, []string{"-m", "connmark", "--mark", strconv.Itoa(n.connMarkDrop), "-j", "DROP"}})
rules = append(rules, iptRule{"filter", chain, []string{"-j", "NFQUEUE", "--queue-num", strconv.Itoa(n.queueNum), "--queue-bypass"}})
}
var iptRulesLocal = []iptRule{
{"filter", "INPUT", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkAccept), "-j", "ACCEPT"}},
{"filter", "INPUT", []string{"-p", "tcp", "-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "REJECT", "--reject-with", "tcp-reset"}},
{"filter", "INPUT", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "DROP"}},
{"filter", "INPUT", []string{"-j", "NFQUEUE", "--queue-num", strconv.Itoa(nfqueueNum), "--queue-bypass"}},
return rules, nil
{"filter", "OUTPUT", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkAccept), "-j", "ACCEPT"}},
{"filter", "OUTPUT", []string{"-p", "tcp", "-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "REJECT", "--reject-with", "tcp-reset"}},
{"filter", "OUTPUT", []string{"-m", "connmark", "--mark", strconv.Itoa(nfqueueConnMarkDrop), "-j", "DROP"}},
{"filter", "OUTPUT", []string{"-j", "NFQUEUE", "--queue-num", strconv.Itoa(nfqueueNum), "--queue-bypass"}},
}
var _ PacketIO = (*nfqueuePacketIO)(nil)
@ -93,74 +44,31 @@ var _ PacketIO = (*nfqueuePacketIO)(nil)
var errNotNFQueuePacket = errors.New("not an NFQueue packet")
type nfqueuePacketIO struct {
n *nfqueue.Nfqueue
local bool
rst bool
rSet bool // whether the nftables/iptables rules have been set
queueNum int
table string // nftable name
connMarkAccept int
connMarkDrop int
// iptables not nil = use iptables instead of nftables
ipt4 *iptables.IPTables
ipt6 *iptables.IPTables
protectedDialer *net.Dialer
n *nfqueue.Nfqueue
local bool
ipt4 *iptables.IPTables
ipt6 *iptables.IPTables
}
type NFQueuePacketIOConfig struct {
QueueSize uint32
QueueNum *uint16
Table string
ConnMarkAccept uint32
ConnMarkDrop uint32
ReadBuffer int
WriteBuffer int
Local bool
RST bool
QueueSize uint32
Local bool
}
func NewNFQueuePacketIO(config NFQueuePacketIOConfig) (PacketIO, error) {
if config.QueueSize == 0 {
config.QueueSize = nfqueueDefaultQueueSize
}
if config.QueueNum == nil {
queueNum := uint16(nfqueueDefaultQueueNum)
config.QueueNum = &queueNum
ipt4, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
if err != nil {
return nil, err
}
if config.Table == "" {
config.Table = nftDefaultTable
}
if config.ConnMarkAccept == 0 {
config.ConnMarkAccept = nfqueueDefaultConnMarkAccept
}
if config.ConnMarkDrop == 0 {
config.ConnMarkDrop = config.ConnMarkAccept + 1
if config.ConnMarkDrop == 0 {
// Overflow
config.ConnMarkDrop = 1
}
}
if config.ConnMarkAccept == config.ConnMarkDrop {
return nil, errors.New("connMarkAccept and connMarkDrop cannot be the same")
}
var ipt4, ipt6 *iptables.IPTables
var err error
if nftCheck() != nil {
// We prefer nftables, but if it's not available, fall back to iptables
ipt4, err = iptables.NewWithProtocol(iptables.ProtocolIPv4)
if err != nil {
return nil, err
}
ipt6, err = iptables.NewWithProtocol(iptables.ProtocolIPv6)
if err != nil {
return nil, err
}
ipt6, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
if err != nil {
return nil, err
}
n, err := nfqueue.Open(&nfqueue.Config{
NfQueue: *config.QueueNum,
NfQueue: nfqueueNum,
MaxPacketLen: nfqueueMaxPacketLen,
MaxQueueLen: config.QueueSize,
Copymode: nfqueue.NfQnlCopyPacket,
@ -169,52 +77,26 @@ func NewNFQueuePacketIO(config NFQueuePacketIOConfig) (PacketIO, error) {
if err != nil {
return nil, err
}
if config.ReadBuffer > 0 {
err = n.Con.SetReadBuffer(config.ReadBuffer)
if err != nil {
_ = n.Close()
return nil, err
}
io := &nfqueuePacketIO{
n: n,
local: config.Local,
ipt4: ipt4,
ipt6: ipt6,
}
if config.WriteBuffer > 0 {
err = n.Con.SetWriteBuffer(config.WriteBuffer)
if err != nil {
_ = n.Close()
return nil, err
}
err = io.setupIpt(config.Local, false)
if err != nil {
_ = n.Close()
return nil, err
}
return &nfqueuePacketIO{
n: n,
local: config.Local,
rst: config.RST,
queueNum: int(*config.QueueNum),
table: config.Table,
connMarkAccept: int(config.ConnMarkAccept),
connMarkDrop: int(config.ConnMarkDrop),
ipt4: ipt4,
ipt6: ipt6,
protectedDialer: &net.Dialer{
Control: func(network, address string, c syscall.RawConn) error {
var err error
cErr := c.Control(func(fd uintptr) {
err = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_MARK, int(config.ConnMarkAccept))
})
if cErr != nil {
return cErr
}
return err
},
},
}, nil
return io, nil
}
func (n *nfqueuePacketIO) Register(ctx context.Context, cb PacketCallback) error {
err := n.n.RegisterWithErrorFunc(ctx,
return n.n.RegisterWithErrorFunc(ctx,
func(a nfqueue.Attribute) int {
if ok, verdict := n.packetAttributeSanityCheck(a); !ok {
if a.PacketID != nil {
_ = n.n.SetVerdict(*a.PacketID, verdict)
}
if a.PacketID == nil || a.Ct == nil || a.Payload == nil || len(*a.Payload) < 20 {
// Invalid packet, ignore
// 20 is the minimum possible size of an IP packet
return 0
}
p := &nfqueuePacket{
@ -222,57 +104,11 @@ func (n *nfqueuePacketIO) Register(ctx context.Context, cb PacketCallback) error
streamID: ctIDFromCtBytes(*a.Ct),
data: *a.Payload,
}
// Use timestamp from attribute if available, otherwise use current time as fallback
if a.Timestamp != nil {
p.timestamp = *a.Timestamp
} else {
p.timestamp = time.Now()
}
return okBoolToInt(cb(p, nil))
},
func(e error) int {
if opErr := (*netlink.OpError)(nil); errors.As(e, &opErr) {
if errors.Is(opErr.Err, unix.ENOBUFS) {
// Kernel buffer temporarily full, ignore
return 0
}
}
return okBoolToInt(cb(nil, e))
})
if err != nil {
return err
}
if !n.rSet {
if n.ipt4 != nil {
err = n.setupIpt(false)
} else {
err = n.setupNft(false)
}
if err != nil {
return err
}
n.rSet = true
}
return nil
}
func (n *nfqueuePacketIO) packetAttributeSanityCheck(a nfqueue.Attribute) (ok bool, verdict int) {
if a.PacketID == nil {
// Re-inject to NFQUEUE is actually not possible in this condition
return false, -1
}
if a.Payload == nil || len(*a.Payload) < 20 {
// 20 is the minimum possible size of an IP packet
return false, nfqueue.NfDrop
}
if a.Ct == nil {
// Multicast packets may not have a conntrack, but only appear in local mode
if n.local {
return false, nfqueue.NfAccept
}
return false, nfqueue.NfDrop
}
return true, -1
}
func (n *nfqueuePacketIO) SetVerdict(p Packet, v Verdict, newPacket []byte) error {
@ -286,62 +122,25 @@ func (n *nfqueuePacketIO) SetVerdict(p Packet, v Verdict, newPacket []byte) erro
case VerdictAcceptModify:
return n.n.SetVerdictModPacket(nP.id, nfqueue.NfAccept, newPacket)
case VerdictAcceptStream:
return n.n.SetVerdictWithConnMark(nP.id, nfqueue.NfAccept, n.connMarkAccept)
return n.n.SetVerdictWithConnMark(nP.id, nfqueue.NfAccept, nfqueueConnMarkAccept)
case VerdictDrop:
return n.n.SetVerdict(nP.id, nfqueue.NfDrop)
case VerdictDropStream:
return n.n.SetVerdictWithConnMark(nP.id, nfqueue.NfDrop, n.connMarkDrop)
return n.n.SetVerdictWithConnMark(nP.id, nfqueue.NfDrop, nfqueueConnMarkDrop)
default:
// Invalid verdict, ignore for now
return nil
}
}
func (n *nfqueuePacketIO) ProtectedDialContext(ctx context.Context, network, address string) (net.Conn, error) {
return n.protectedDialer.DialContext(ctx, network, address)
}
func (n *nfqueuePacketIO) Close() error {
if n.rSet {
if n.ipt4 != nil {
_ = n.setupIpt(true)
} else {
_ = n.setupNft(true)
}
n.rSet = false
}
return n.n.Close()
}
// nfqueue IO does not issue shutdown
func (n *nfqueuePacketIO) SetCancelFunc(cancelFunc context.CancelFunc) error {
return nil
}
func (n *nfqueuePacketIO) setupNft(remove bool) error {
rules, err := n.generateNftRules()
if err != nil {
return err
}
rulesText := rules.String()
if remove {
err = nftDelete(nftFamily, n.table)
func (n *nfqueuePacketIO) setupIpt(local, remove bool) error {
var rules []iptRule
if local {
rules = iptRulesLocal
} else {
// Delete first to make sure no leftover rules
_ = nftDelete(nftFamily, n.table)
err = nftAdd(rulesText)
}
if err != nil {
return err
}
return nil
}
func (n *nfqueuePacketIO) setupIpt(remove bool) error {
rules, err := n.generateIptRules()
if err != nil {
return err
rules = iptRulesForward
}
var err error
if remove {
err = iptsBatchDeleteIfExists([]*iptables.IPTables{n.ipt4, n.ipt6}, rules)
} else {
@ -353,23 +152,24 @@ func (n *nfqueuePacketIO) setupIpt(remove bool) error {
return nil
}
func (n *nfqueuePacketIO) Close() error {
err := n.setupIpt(n.local, true)
_ = n.n.Close()
return err
}
var _ Packet = (*nfqueuePacket)(nil)
type nfqueuePacket struct {
id uint32
streamID uint32
timestamp time.Time
data []byte
id uint32
streamID uint32
data []byte
}
func (p *nfqueuePacket) StreamID() uint32 {
return p.streamID
}
func (p *nfqueuePacket) Timestamp() time.Time {
return p.timestamp
}
func (p *nfqueuePacket) Data() []byte {
return p.data
}
@ -382,61 +182,6 @@ func okBoolToInt(ok bool) int {
}
}
func nftCheck() error {
_, err := exec.LookPath("nft")
if err != nil {
return err
}
return nil
}
func nftAdd(input string) error {
cmd := exec.Command("nft", "-f", "-")
cmd.Stdin = strings.NewReader(input)
return cmd.Run()
}
func nftDelete(family, table string) error {
cmd := exec.Command("nft", "delete", "table", family, table)
return cmd.Run()
}
type nftTableSpec struct {
Defines []string
Family, Table string
Chains []nftChainSpec
}
func (t *nftTableSpec) String() string {
chains := make([]string, 0, len(t.Chains))
for _, c := range t.Chains {
chains = append(chains, c.String())
}
return fmt.Sprintf(`
%s
table %s %s {
%s
}
`, strings.Join(t.Defines, "\n"), t.Family, t.Table, strings.Join(chains, ""))
}
type nftChainSpec struct {
Chain string
Header string
Rules []string
}
func (c *nftChainSpec) String() string {
return fmt.Sprintf(`
chain %s {
%s
%s
}
`, c.Chain, c.Header, strings.Join(c.Rules, "\n\x20\x20\x20\x20"))
}
type iptRule struct {
Table, Chain string
RuleSpec []string

136
io/pcap.go
View File

@ -1,136 +0,0 @@
package io
import (
"context"
"hash/crc32"
"io"
"net"
"os"
"sort"
"strings"
"time"
"github.com/google/gopacket"
"github.com/google/gopacket/pcapgo"
)
var _ PacketIO = (*pcapPacketIO)(nil)
type pcapPacketIO struct {
pcapFile io.ReadCloser
pcap *pcapgo.Reader
timeOffset *time.Duration
ioCancel context.CancelFunc
config PcapPacketIOConfig
dialer *net.Dialer
}
type PcapPacketIOConfig struct {
PcapFile string
Realtime bool
}
func NewPcapPacketIO(config PcapPacketIOConfig) (PacketIO, error) {
pcapFile, err := os.Open(config.PcapFile)
if err != nil {
return nil, err
}
handle, err := pcapgo.NewReader(pcapFile)
if err != nil {
return nil, err
}
return &pcapPacketIO{
pcapFile: pcapFile,
pcap: handle,
timeOffset: nil,
ioCancel: nil,
config: config,
dialer: &net.Dialer{},
}, nil
}
func (p *pcapPacketIO) Register(ctx context.Context, cb PacketCallback) error {
go func() {
packetSource := gopacket.NewPacketSource(p.pcap, p.pcap.LinkType())
for packet := range packetSource.Packets() {
p.wait(packet)
networkLayer := packet.NetworkLayer()
if networkLayer != nil {
src, dst := networkLayer.NetworkFlow().Endpoints()
endpoints := []string{src.String(), dst.String()}
sort.Strings(endpoints)
id := crc32.Checksum([]byte(strings.Join(endpoints, ",")), crc32.IEEETable)
cb(&pcapPacket{
streamID: id,
timestamp: packet.Metadata().Timestamp,
data: packet.LinkLayer().LayerPayload(),
}, nil)
}
}
// Give the workers a chance to finish everything
time.Sleep(time.Second)
// Stop the engine when all packets are finished
p.ioCancel()
}()
return nil
}
// A normal dialer is sufficient as pcap IO does not mess up with the networking
func (p *pcapPacketIO) ProtectedDialContext(ctx context.Context, network, address string) (net.Conn, error) {
return p.dialer.DialContext(ctx, network, address)
}
func (p *pcapPacketIO) SetVerdict(pkt Packet, v Verdict, newPacket []byte) error {
return nil
}
func (p *pcapPacketIO) SetCancelFunc(cancelFunc context.CancelFunc) error {
p.ioCancel = cancelFunc
return nil
}
func (p *pcapPacketIO) Close() error {
return p.pcapFile.Close()
}
// Intentionally slow down the replay
// In realtime mode, this is to match the timestamps in the capture
func (p *pcapPacketIO) wait(packet gopacket.Packet) {
if !p.config.Realtime {
return
}
if p.timeOffset == nil {
offset := time.Since(packet.Metadata().Timestamp)
p.timeOffset = &offset
} else {
t := time.Until(packet.Metadata().Timestamp.Add(*p.timeOffset))
time.Sleep(t)
}
}
var _ Packet = (*pcapPacket)(nil)
type pcapPacket struct {
streamID uint32
timestamp time.Time
data []byte
}
func (p *pcapPacket) StreamID() uint32 {
return p.streamID
}
func (p *pcapPacket) Timestamp() time.Time {
return p.timestamp
}
func (p *pcapPacket) Data() []byte {
return p.data
}

18
ruleset/builtins/cidr.go
View File

@ -1,18 +0,0 @@
package builtins
import (
"net"
)
func MatchCIDR(ip string, cidr *net.IPNet) bool {
ipAddr := net.ParseIP(ip)
if ipAddr == nil {
return false
}
return cidr.Contains(ipAddr)
}
func CompileCIDR(cidr string) (*net.IPNet, error) {
_, ipNet, err := net.ParseCIDR(cidr)
return ipNet, err
}

128
ruleset/builtins/geo/geo_loader.go
View File

@ -1,128 +0,0 @@
package geo
import (
"io"
"net/http"
"os"
"time"
"github.com/apernet/OpenGFW/ruleset/builtins/geo/v2geo"
)
const (
geoipFilename = "geoip.dat"
geoipURL = "https://cdn.jsdelivr.net/gh/Loyalsoldier/v2ray-rules-dat@release/geoip.dat"
geositeFilename = "geosite.dat"
geositeURL = "https://cdn.jsdelivr.net/gh/Loyalsoldier/v2ray-rules-dat@release/geosite.dat"
geoDefaultUpdateInterval = 7 * 24 * time.Hour // 7 days
)
var _ GeoLoader = (*V2GeoLoader)(nil)
// V2GeoLoader provides the on-demand GeoIP/MatchGeoSite database
// loading functionality required by the ACL engine.
// Empty filenames = automatic download from built-in URLs.
type V2GeoLoader struct {
GeoIPFilename string
GeoSiteFilename string
UpdateInterval time.Duration
DownloadFunc func(filename, url string)
DownloadErrFunc func(err error)
geoipMap map[string]*v2geo.GeoIP
geositeMap map[string]*v2geo.GeoSite
}
func NewDefaultGeoLoader(geoSiteFilename, geoIpFilename string) *V2GeoLoader {
return &V2GeoLoader{
GeoIPFilename: geoIpFilename,
GeoSiteFilename: geoSiteFilename,
DownloadFunc: func(filename, url string) {},
DownloadErrFunc: func(err error) {},
}
}
func (l *V2GeoLoader) shouldDownload(filename string) bool {
info, err := os.Stat(filename)
if os.IsNotExist(err) {
return true
}
dt := time.Since(info.ModTime())
if l.UpdateInterval == 0 {
return dt > geoDefaultUpdateInterval
} else {
return dt > l.UpdateInterval
}
}
func (l *V2GeoLoader) download(filename, url string) error {
l.DownloadFunc(filename, url)
resp, err := http.Get(url)
if err != nil {
l.DownloadErrFunc(err)
return err
}
defer resp.Body.Close()
f, err := os.Create(filename)
if err != nil {
l.DownloadErrFunc(err)
return err
}
defer f.Close()
_, err = io.Copy(f, resp.Body)
l.DownloadErrFunc(err)
return err
}
func (l *V2GeoLoader) LoadGeoIP() (map[string]*v2geo.GeoIP, error) {
if l.geoipMap != nil {
return l.geoipMap, nil
}
autoDL := false
filename := l.GeoIPFilename
if filename == "" {
autoDL = true
filename = geoipFilename
}
if autoDL && l.shouldDownload(filename) {
err := l.download(filename, geoipURL)
if err != nil {
return nil, err
}
}
m, err := v2geo.LoadGeoIP(filename)
if err != nil {
return nil, err
}
l.geoipMap = m
return m, nil
}
func (l *V2GeoLoader) LoadGeoSite() (map[string]*v2geo.GeoSite, error) {
if l.geositeMap != nil {
return l.geositeMap, nil
}
autoDL := false
filename := l.GeoSiteFilename
if filename == "" {
autoDL = true
filename = geositeFilename
}
if autoDL && l.shouldDownload(filename) {
err := l.download(filename, geositeURL)
if err != nil {
return nil, err
}
}
m, err := v2geo.LoadGeoSite(filename)
if err != nil {
return nil, err
}
l.geositeMap = m
return m, nil
}

113
ruleset/builtins/geo/geo_matcher.go
View File

@ -1,113 +0,0 @@
package geo
import (
"net"
"strings"
"sync"
)
type GeoMatcher struct {
geoLoader GeoLoader
geoSiteMatcher map[string]hostMatcher
siteMatcherLock sync.Mutex
geoIpMatcher map[string]hostMatcher
ipMatcherLock sync.Mutex
}
func NewGeoMatcher(geoSiteFilename, geoIpFilename string) *GeoMatcher {
return &GeoMatcher{
geoLoader: NewDefaultGeoLoader(geoSiteFilename, geoIpFilename),
geoSiteMatcher: make(map[string]hostMatcher),
geoIpMatcher: make(map[string]hostMatcher),
}
}
func (g *GeoMatcher) MatchGeoIp(ip, condition string) bool {
g.ipMatcherLock.Lock()
defer g.ipMatcherLock.Unlock()
matcher, ok := g.geoIpMatcher[condition]
if !ok {
// GeoIP matcher
condition = strings.ToLower(condition)
country := condition
if len(country) == 0 {
return false
}
gMap, err := g.geoLoader.LoadGeoIP()
if err != nil {
return false
}
list, ok := gMap[country]
if !ok || list == nil {
return false
}
matcher, err = newGeoIPMatcher(list)
if err != nil {
return false
}
g.geoIpMatcher[condition] = matcher
}
parseIp := net.ParseIP(ip)
if parseIp == nil {
return false
}
ipv4 := parseIp.To4()
if ipv4 != nil {
return matcher.Match(HostInfo{IPv4: ipv4})
}
ipv6 := parseIp.To16()
if ipv6 != nil {
return matcher.Match(HostInfo{IPv6: ipv6})
}
return false
}
func (g *GeoMatcher) MatchGeoSite(site, condition string) bool {
g.siteMatcherLock.Lock()
defer g.siteMatcherLock.Unlock()
matcher, ok := g.geoSiteMatcher[condition]
if !ok {
// MatchGeoSite matcher
condition = strings.ToLower(condition)
name, attrs := parseGeoSiteName(condition)
if len(name) == 0 {
return false
}
gMap, err := g.geoLoader.LoadGeoSite()
if err != nil {
return false
}
list, ok := gMap[name]
if !ok || list == nil {
return false
}
matcher, err = newGeositeMatcher(list, attrs)
if err != nil {
return false
}
g.geoSiteMatcher[condition] = matcher
}
return matcher.Match(HostInfo{Name: site})
}
func (g *GeoMatcher) LoadGeoSite() error {
_, err := g.geoLoader.LoadGeoSite()
return err
}
func (g *GeoMatcher) LoadGeoIP() error {
_, err := g.geoLoader.LoadGeoIP()
return err
}
func parseGeoSiteName(s string) (string, []string) {
parts := strings.Split(s, "@")
base := strings.TrimSpace(parts[0])
attrs := parts[1:]
for i := range attrs {
attrs[i] = strings.TrimSpace(attrs[i])
}
return base, attrs
}

27
ruleset/builtins/geo/interface.go
View File

@ -1,27 +0,0 @@
package geo
import (
"fmt"
"net"
"github.com/apernet/OpenGFW/ruleset/builtins/geo/v2geo"
)
type HostInfo struct {
Name string
IPv4 net.IP
IPv6 net.IP
}
func (h HostInfo) String() string {
return fmt.Sprintf("%s|%s|%s", h.Name, h.IPv4, h.IPv6)
}
type GeoLoader interface {
LoadGeoIP() (map[string]*v2geo.GeoIP, error)
LoadGeoSite() (map[string]*v2geo.GeoSite, error)
}
type hostMatcher interface {
Match(HostInfo) bool
}

213
ruleset/builtins/geo/matchers_v2geo.go
View File

@ -1,213 +0,0 @@
package geo
import (
"bytes"
"errors"
"net"
"regexp"
"sort"
"strings"
"github.com/apernet/OpenGFW/ruleset/builtins/geo/v2geo"
)
var _ hostMatcher = (*geoipMatcher)(nil)
type geoipMatcher struct {
N4 []*net.IPNet // sorted
N6 []*net.IPNet // sorted
Inverse bool
}
// matchIP tries to match the given IP address with the corresponding IPNets.
// Note that this function does NOT handle the Inverse flag.
func (m *geoipMatcher) matchIP(ip net.IP) bool {
var n []*net.IPNet
if ip4 := ip.To4(); ip4 != nil {
// N4 stores IPv4 addresses in 4-byte form.
// Make sure we use it here too, otherwise bytes.Compare will fail.
ip = ip4
n = m.N4
} else {
n = m.N6
}
left, right := 0, len(n)-1
for left <= right {
mid := (left + right) / 2
if n[mid].Contains(ip) {
return true
} else if bytes.Compare(n[mid].IP, ip) < 0 {
left = mid + 1
} else {
right = mid - 1
}
}
return false
}
func (m *geoipMatcher) Match(host HostInfo) bool {
if host.IPv4 != nil {
if m.matchIP(host.IPv4) {
return !m.Inverse
}
}
if host.IPv6 != nil {
if m.matchIP(host.IPv6) {
return !m.Inverse
}
}
return m.Inverse
}
func newGeoIPMatcher(list *v2geo.GeoIP) (*geoipMatcher, error) {
n4 := make([]*net.IPNet, 0)
n6 := make([]*net.IPNet, 0)
for _, cidr := range list.Cidr {
if len(cidr.Ip) == 4 {
// IPv4
n4 = append(n4, &net.IPNet{
IP: cidr.Ip,
Mask: net.CIDRMask(int(cidr.Prefix), 32),
})
} else if len(cidr.Ip) == 16 {
// IPv6
n6 = append(n6, &net.IPNet{
IP: cidr.Ip,
Mask: net.CIDRMask(int(cidr.Prefix), 128),
})
} else {
return nil, errors.New("invalid IP length")
}
}
// Sort the IPNets, so we can do binary search later.
sort.Slice(n4, func(i, j int) bool {
return bytes.Compare(n4[i].IP, n4[j].IP) < 0
})
sort.Slice(n6, func(i, j int) bool {
return bytes.Compare(n6[i].IP, n6[j].IP) < 0
})
return &geoipMatcher{
N4: n4,
N6: n6,
Inverse: list.InverseMatch,
}, nil
}
var _ hostMatcher = (*geositeMatcher)(nil)
type geositeDomainType int
const (
geositeDomainPlain geositeDomainType = iota
geositeDomainRegex
geositeDomainRoot
geositeDomainFull
)
type geositeDomain struct {
Type geositeDomainType
Value string
Regex *regexp.Regexp
Attrs map[string]bool
}
type geositeMatcher struct {
Domains []geositeDomain
// Attributes are matched using "and" logic - if you have multiple attributes here,
// a domain must have all of those attributes to be considered a match.
Attrs []string
}
func (m *geositeMatcher) matchDomain(domain geositeDomain, host HostInfo) bool {
// Match attributes first
if len(m.Attrs) > 0 {
if len(domain.Attrs) == 0 {
return false
}
for _, attr := range m.Attrs {
if !domain.Attrs[attr] {
return false
}
}
}
switch domain.Type {
case geositeDomainPlain:
return strings.Contains(host.Name, domain.Value)
case geositeDomainRegex:
if domain.Regex != nil {
return domain.Regex.MatchString(host.Name)
}
case geositeDomainFull:
return host.Name == domain.Value
case geositeDomainRoot:
if host.Name == domain.Value {
return true
}
return strings.HasSuffix(host.Name, "."+domain.Value)
default:
return false
}
return false
}
func (m *geositeMatcher) Match(host HostInfo) bool {
for _, domain := range m.Domains {
if m.matchDomain(domain, host) {
return true
}
}
return false
}
func newGeositeMatcher(list *v2geo.GeoSite, attrs []string) (*geositeMatcher, error) {
domains := make([]geositeDomain, len(list.Domain))
for i, domain := range list.Domain {
switch domain.Type {
case v2geo.Domain_Plain:
domains[i] = geositeDomain{
Type: geositeDomainPlain,
Value: domain.Value,
Attrs: domainAttributeToMap(domain.Attribute),
}
case v2geo.Domain_Regex:
regex, err := regexp.Compile(domain.Value)
if err != nil {
return nil, err
}
domains[i] = geositeDomain{
Type: geositeDomainRegex,
Regex: regex,
Attrs: domainAttributeToMap(domain.Attribute),
}
case v2geo.Domain_Full:
domains[i] = geositeDomain{
Type: geositeDomainFull,
Value: domain.Value,
Attrs: domainAttributeToMap(domain.Attribute),
}
case v2geo.Domain_RootDomain:
domains[i] = geositeDomain{
Type: geositeDomainRoot,
Value: domain.Value,
Attrs: domainAttributeToMap(domain.Attribute),
}
default:
return nil, errors.New("unsupported domain type")
}
}
return &geositeMatcher{
Domains: domains,
Attrs: attrs,
}, nil
}
func domainAttributeToMap(attrs []*v2geo.Domain_Attribute) map[string]bool {
m := make(map[string]bool)
for _, attr := range attrs {
// Supposedly there are also int attributes,
// but nobody seems to use them, so we treat everything as boolean for now.
m[attr.Key] = true
}
return m
}

44
ruleset/builtins/geo/v2geo/load.go
View File

@ -1,44 +0,0 @@
package v2geo
import (
"os"
"strings"
"google.golang.org/protobuf/proto"
)
// LoadGeoIP loads a GeoIP data file and converts it to a map.
// The keys of the map (country codes) are all normalized to lowercase.
func LoadGeoIP(filename string) (map[string]*GeoIP, error) {
bs, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
var list GeoIPList
if err := proto.Unmarshal(bs, &list); err != nil {
return nil, err
}
m := make(map[string]*GeoIP)
for _, entry := range list.Entry {
m[strings.ToLower(entry.CountryCode)] = entry
}
return m, nil
}
// LoadGeoSite loads a GeoSite data file and converts it to a map.
// The keys of the map (site keys) are all normalized to lowercase.
func LoadGeoSite(filename string) (map[string]*GeoSite, error) {
bs, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
var list GeoSiteList
if err := proto.Unmarshal(bs, &list); err != nil {
return nil, err
}
m := make(map[string]*GeoSite)
for _, entry := range list.Entry {
m[strings.ToLower(entry.CountryCode)] = entry
}
return m, nil
}

745
ruleset/builtins/geo/v2geo/v2geo.pb.go
View File

@ -1,745 +0,0 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.31.0
// protoc v4.24.4
// source: v2geo.proto
package v2geo
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
// Type of domain value.
type Domain_Type int32
const (
// The value is used as is.
Domain_Plain Domain_Type = 0
// The value is used as a regular expression.
Domain_Regex Domain_Type = 1
// The value is a root domain.
Domain_RootDomain Domain_Type = 2
// The value is a domain.
Domain_Full Domain_Type = 3
)
// Enum value maps for Domain_Type.
var (
Domain_Type_name = map[int32]string{
0: "Plain",
1: "Regex",
2: "RootDomain",
3: "Full",
}
Domain_Type_value = map[string]int32{
"Plain": 0,
"Regex": 1,
"RootDomain": 2,
"Full": 3,
}
)
func (x Domain_Type) Enum() *Domain_Type {
p := new(Domain_Type)
*p = x
return p
}
func (x Domain_Type) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (Domain_Type) Descriptor() protoreflect.EnumDescriptor {
return file_v2geo_proto_enumTypes[0].Descriptor()
}
func (Domain_Type) Type() protoreflect.EnumType {
return &file_v2geo_proto_enumTypes[0]
}
func (x Domain_Type) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use Domain_Type.Descriptor instead.
func (Domain_Type) EnumDescriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{0, 0}
}
// Domain for routing decision.
type Domain struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Domain matching type.
Type Domain_Type `protobuf:"varint,1,opt,name=type,proto3,enum=Domain_Type" json:"type,omitempty"`
// Domain value.
Value string `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"`
// Attributes of this domain. May be used for filtering.
Attribute []*Domain_Attribute `protobuf:"bytes,3,rep,name=attribute,proto3" json:"attribute,omitempty"`
}
func (x *Domain) Reset() {
*x = Domain{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Domain) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Domain) ProtoMessage() {}
func (x *Domain) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Domain.ProtoReflect.Descriptor instead.
func (*Domain) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{0}
}
func (x *Domain) GetType() Domain_Type {
if x != nil {
return x.Type
}
return Domain_Plain
}
func (x *Domain) GetValue() string {
if x != nil {
return x.Value
}
return ""
}
func (x *Domain) GetAttribute() []*Domain_Attribute {
if x != nil {
return x.Attribute
}
return nil
}
// IP for routing decision, in CIDR form.
type CIDR struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// IP address, should be either 4 or 16 bytes.
Ip []byte `protobuf:"bytes,1,opt,name=ip,proto3" json:"ip,omitempty"`
// Number of leading ones in the network mask.
Prefix uint32 `protobuf:"varint,2,opt,name=prefix,proto3" json:"prefix,omitempty"`
}
func (x *CIDR) Reset() {
*x = CIDR{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *CIDR) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*CIDR) ProtoMessage() {}
func (x *CIDR) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[1]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use CIDR.ProtoReflect.Descriptor instead.
func (*CIDR) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{1}
}
func (x *CIDR) GetIp() []byte {
if x != nil {
return x.Ip
}
return nil
}
func (x *CIDR) GetPrefix() uint32 {
if x != nil {
return x.Prefix
}
return 0
}
type GeoIP struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
CountryCode string `protobuf:"bytes,1,opt,name=country_code,json=countryCode,proto3" json:"country_code,omitempty"`
Cidr []*CIDR `protobuf:"bytes,2,rep,name=cidr,proto3" json:"cidr,omitempty"`
InverseMatch bool `protobuf:"varint,3,opt,name=inverse_match,json=inverseMatch,proto3" json:"inverse_match,omitempty"`
// resource_hash instruct simplified config converter to load domain from geo file.
ResourceHash []byte `protobuf:"bytes,4,opt,name=resource_hash,json=resourceHash,proto3" json:"resource_hash,omitempty"`
Code string `protobuf:"bytes,5,opt,name=code,proto3" json:"code,omitempty"`
}
func (x *GeoIP) Reset() {
*x = GeoIP{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GeoIP) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GeoIP) ProtoMessage() {}
func (x *GeoIP) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[2]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GeoIP.ProtoReflect.Descriptor instead.
func (*GeoIP) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{2}
}
func (x *GeoIP) GetCountryCode() string {
if x != nil {
return x.CountryCode
}
return ""
}
func (x *GeoIP) GetCidr() []*CIDR {
if x != nil {
return x.Cidr
}
return nil
}
func (x *GeoIP) GetInverseMatch() bool {
if x != nil {
return x.InverseMatch
}
return false
}
func (x *GeoIP) GetResourceHash() []byte {
if x != nil {
return x.ResourceHash
}
return nil
}
func (x *GeoIP) GetCode() string {
if x != nil {
return x.Code
}
return ""
}
type GeoIPList struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Entry []*GeoIP `protobuf:"bytes,1,rep,name=entry,proto3" json:"entry,omitempty"`
}
func (x *GeoIPList) Reset() {
*x = GeoIPList{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GeoIPList) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GeoIPList) ProtoMessage() {}
func (x *GeoIPList) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[3]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GeoIPList.ProtoReflect.Descriptor instead.
func (*GeoIPList) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{3}
}
func (x *GeoIPList) GetEntry() []*GeoIP {
if x != nil {
return x.Entry
}
return nil
}
type GeoSite struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
CountryCode string `protobuf:"bytes,1,opt,name=country_code,json=countryCode,proto3" json:"country_code,omitempty"`
Domain []*Domain `protobuf:"bytes,2,rep,name=domain,proto3" json:"domain,omitempty"`
// resource_hash instruct simplified config converter to load domain from geo file.
ResourceHash []byte `protobuf:"bytes,3,opt,name=resource_hash,json=resourceHash,proto3" json:"resource_hash,omitempty"`
Code string `protobuf:"bytes,4,opt,name=code,proto3" json:"code,omitempty"`
}
func (x *GeoSite) Reset() {
*x = GeoSite{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GeoSite) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GeoSite) ProtoMessage() {}
func (x *GeoSite) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[4]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MatchGeoSite.ProtoReflect.Descriptor instead.
func (*GeoSite) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{4}
}
func (x *GeoSite) GetCountryCode() string {
if x != nil {
return x.CountryCode
}
return ""
}
func (x *GeoSite) GetDomain() []*Domain {
if x != nil {
return x.Domain
}
return nil
}
func (x *GeoSite) GetResourceHash() []byte {
if x != nil {
return x.ResourceHash
}
return nil
}
func (x *GeoSite) GetCode() string {
if x != nil {
return x.Code
}
return ""
}
type GeoSiteList struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Entry []*GeoSite `protobuf:"bytes,1,rep,name=entry,proto3" json:"entry,omitempty"`
}
func (x *GeoSiteList) Reset() {
*x = GeoSiteList{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GeoSiteList) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GeoSiteList) ProtoMessage() {}
func (x *GeoSiteList) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[5]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GeoSiteList.ProtoReflect.Descriptor instead.
func (*GeoSiteList) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{5}
}
func (x *GeoSiteList) GetEntry() []*GeoSite {
if x != nil {
return x.Entry
}
return nil
}
type Domain_Attribute struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"`
// Types that are assignable to TypedValue:
//
// *Domain_Attribute_BoolValue
// *Domain_Attribute_IntValue
TypedValue isDomain_Attribute_TypedValue `protobuf_oneof:"typed_value"`
}
func (x *Domain_Attribute) Reset() {
*x = Domain_Attribute{}
if protoimpl.UnsafeEnabled {
mi := &file_v2geo_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Domain_Attribute) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Domain_Attribute) ProtoMessage() {}
func (x *Domain_Attribute) ProtoReflect() protoreflect.Message {
mi := &file_v2geo_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Domain_Attribute.ProtoReflect.Descriptor instead.
func (*Domain_Attribute) Descriptor() ([]byte, []int) {
return file_v2geo_proto_rawDescGZIP(), []int{0, 0}
}
func (x *Domain_Attribute) GetKey() string {
if x != nil {
return x.Key
}
return ""
}
func (m *Domain_Attribute) GetTypedValue() isDomain_Attribute_TypedValue {
if m != nil {
return m.TypedValue
}
return nil
}
func (x *Domain_Attribute) GetBoolValue() bool {
if x, ok := x.GetTypedValue().(*Domain_Attribute_BoolValue); ok {
return x.BoolValue
}
return false
}
func (x *Domain_Attribute) GetIntValue() int64 {
if x, ok := x.GetTypedValue().(*Domain_Attribute_IntValue); ok {
return x.IntValue
}
return 0
}
type isDomain_Attribute_TypedValue interface {
isDomain_Attribute_TypedValue()
}
type Domain_Attribute_BoolValue struct {
BoolValue bool `protobuf:"varint,2,opt,name=bool_value,json=boolValue,proto3,oneof"`
}
type Domain_Attribute_IntValue struct {
IntValue int64 `protobuf:"varint,3,opt,name=int_value,json=intValue,proto3,oneof"`
}
func (*Domain_Attribute_BoolValue) isDomain_Attribute_TypedValue() {}
func (*Domain_Attribute_IntValue) isDomain_Attribute_TypedValue() {}
var File_v2geo_proto protoreflect.FileDescriptor
var file_v2geo_proto_rawDesc = []byte{
0x0a, 0x0b, 0x76, 0x32, 0x67, 0x65, 0x6f, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x97, 0x02,
0x0a, 0x06, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x20, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x0c, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x2e,
0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61,
0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65,
0x12, 0x2f, 0x0a, 0x09, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x18, 0x03, 0x20,
0x03, 0x28, 0x0b, 0x32, 0x11, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x2e, 0x41, 0x74, 0x74,
0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x52, 0x09, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74,
0x65, 0x1a, 0x6c, 0x0a, 0x09, 0x41, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x12, 0x10,
0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79,
0x12, 0x1f, 0x0a, 0x0a, 0x62, 0x6f, 0x6f, 0x6c, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02,
0x20, 0x01, 0x28, 0x08, 0x48, 0x00, 0x52, 0x09, 0x62, 0x6f, 0x6f, 0x6c, 0x56, 0x61, 0x6c, 0x75,
0x65, 0x12, 0x1d, 0x0a, 0x09, 0x69, 0x6e, 0x74, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x03,
0x20, 0x01, 0x28, 0x03, 0x48, 0x00, 0x52, 0x08, 0x69, 0x6e, 0x74, 0x56, 0x61, 0x6c, 0x75, 0x65,
0x42, 0x0d, 0x0a, 0x0b, 0x74, 0x79, 0x70, 0x65, 0x64, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22,
0x36, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, 0x12, 0x09, 0x0a, 0x05, 0x50, 0x6c, 0x61, 0x69, 0x6e,
0x10, 0x00, 0x12, 0x09, 0x0a, 0x05, 0x52, 0x65, 0x67, 0x65, 0x78, 0x10, 0x01, 0x12, 0x0e, 0x0a,
0x0a, 0x52, 0x6f, 0x6f, 0x74, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x10, 0x02, 0x12, 0x08, 0x0a,
0x04, 0x46, 0x75, 0x6c, 0x6c, 0x10, 0x03, 0x22, 0x2e, 0x0a, 0x04, 0x43, 0x49, 0x44, 0x52, 0x12,
0x0e, 0x0a, 0x02, 0x69, 0x70, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x02, 0x69, 0x70, 0x12,
0x16, 0x0a, 0x06, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0d, 0x52,
0x06, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x22, 0xa3, 0x01, 0x0a, 0x05, 0x47, 0x65, 0x6f, 0x49,
0x50, 0x12, 0x21, 0x0a, 0x0c, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x72, 0x79, 0x5f, 0x63, 0x6f, 0x64,
0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x72, 0x79,
0x43, 0x6f, 0x64, 0x65, 0x12, 0x19, 0x0a, 0x04, 0x63, 0x69, 0x64, 0x72, 0x18, 0x02, 0x20, 0x03,
0x28, 0x0b, 0x32, 0x05, 0x2e, 0x43, 0x49, 0x44, 0x52, 0x52, 0x04, 0x63, 0x69, 0x64, 0x72, 0x12,
0x23, 0x0a, 0x0d, 0x69, 0x6e, 0x76, 0x65, 0x72, 0x73, 0x65, 0x5f, 0x6d, 0x61, 0x74, 0x63, 0x68,
0x18, 0x03, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0c, 0x69, 0x6e, 0x76, 0x65, 0x72, 0x73, 0x65, 0x4d,
0x61, 0x74, 0x63, 0x68, 0x12, 0x23, 0x0a, 0x0d, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65,
0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0c, 0x72, 0x65, 0x73,
0x6f, 0x75, 0x72, 0x63, 0x65, 0x48, 0x61, 0x73, 0x68, 0x12, 0x12, 0x0a, 0x04, 0x63, 0x6f, 0x64,
0x65, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x63, 0x6f, 0x64, 0x65, 0x22, 0x29, 0x0a,
0x09, 0x47, 0x65, 0x6f, 0x49, 0x50, 0x4c, 0x69, 0x73, 0x74, 0x12, 0x1c, 0x0a, 0x05, 0x65, 0x6e,
0x74, 0x72, 0x79, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x06, 0x2e, 0x47, 0x65, 0x6f, 0x49,
0x50, 0x52, 0x05, 0x65, 0x6e, 0x74, 0x72, 0x79, 0x22, 0x86, 0x01, 0x0a, 0x07, 0x47, 0x65, 0x6f,
0x53, 0x69, 0x74, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x72, 0x79, 0x5f,
0x63, 0x6f, 0x64, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x63, 0x6f, 0x75, 0x6e,
0x74, 0x72, 0x79, 0x43, 0x6f, 0x64, 0x65, 0x12, 0x1f, 0x0a, 0x06, 0x64, 0x6f, 0x6d, 0x61, 0x69,
0x6e, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e,
0x52, 0x06, 0x64, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x23, 0x0a, 0x0d, 0x72, 0x65, 0x73, 0x6f,
0x75, 0x72, 0x63, 0x65, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52,
0x0c, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x48, 0x61, 0x73, 0x68, 0x12, 0x12, 0x0a,
0x04, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x63, 0x6f, 0x64,
0x65, 0x22, 0x2d, 0x0a, 0x0b, 0x47, 0x65, 0x6f, 0x53, 0x69, 0x74, 0x65, 0x4c, 0x69, 0x73, 0x74,
0x12, 0x1e, 0x0a, 0x05, 0x65, 0x6e, 0x74, 0x72, 0x79, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32,
0x08, 0x2e, 0x47, 0x65, 0x6f, 0x53, 0x69, 0x74, 0x65, 0x52, 0x05, 0x65, 0x6e, 0x74, 0x72, 0x79,
0x42, 0x09, 0x5a, 0x07, 0x2e, 0x2f, 0x76, 0x32, 0x67, 0x65, 0x6f, 0x62, 0x06, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x33,
}
var (
file_v2geo_proto_rawDescOnce sync.Once
file_v2geo_proto_rawDescData = file_v2geo_proto_rawDesc
)
func file_v2geo_proto_rawDescGZIP() []byte {
file_v2geo_proto_rawDescOnce.Do(func() {
file_v2geo_proto_rawDescData = protoimpl.X.CompressGZIP(file_v2geo_proto_rawDescData)
})
return file_v2geo_proto_rawDescData
}
var file_v2geo_proto_enumTypes = make([]protoimpl.EnumInfo, 1)
var file_v2geo_proto_msgTypes = make([]protoimpl.MessageInfo, 7)
var file_v2geo_proto_goTypes = []interface{}{
(Domain_Type)(0), // 0: Domain.Type
(*Domain)(nil), // 1: Domain
(*CIDR)(nil), // 2: CIDR
(*GeoIP)(nil), // 3: GeoIP
(*GeoIPList)(nil), // 4: GeoIPList
(*GeoSite)(nil), // 5: MatchGeoSite
(*GeoSiteList)(nil), // 6: GeoSiteList
(*Domain_Attribute)(nil), // 7: Domain.Attribute
}
var file_v2geo_proto_depIdxs = []int32{
0, // 0: Domain.type:type_name -> Domain.Type
7, // 1: Domain.attribute:type_name -> Domain.Attribute
2, // 2: GeoIP.cidr:type_name -> CIDR
3, // 3: GeoIPList.entry:type_name -> GeoIP
1, // 4: MatchGeoSite.domain:type_name -> Domain
5, // 5: GeoSiteList.entry:type_name -> MatchGeoSite
6, // [6:6] is the sub-list for method output_type
6, // [6:6] is the sub-list for method input_type
6, // [6:6] is the sub-list for extension type_name
6, // [6:6] is the sub-list for extension extendee
0, // [0:6] is the sub-list for field type_name
}
func init() { file_v2geo_proto_init() }
func file_v2geo_proto_init() {
if File_v2geo_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_v2geo_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Domain); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*CIDR); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*GeoIP); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*GeoIPList); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*GeoSite); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*GeoSiteList); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_v2geo_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Domain_Attribute); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
file_v2geo_proto_msgTypes[6].OneofWrappers = []interface{}{
(*Domain_Attribute_BoolValue)(nil),
(*Domain_Attribute_IntValue)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_v2geo_proto_rawDesc,
NumEnums: 1,
NumMessages: 7,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_v2geo_proto_goTypes,
DependencyIndexes: file_v2geo_proto_depIdxs,
EnumInfos: file_v2geo_proto_enumTypes,
MessageInfos: file_v2geo_proto_msgTypes,
}.Build()
File_v2geo_proto = out.File
file_v2geo_proto_rawDesc = nil
file_v2geo_proto_goTypes = nil
file_v2geo_proto_depIdxs = nil
}

76
ruleset/builtins/geo/v2geo/v2geo.proto
View File

@ -1,76 +0,0 @@
syntax = "proto3";
option go_package = "./v2geo";
// This file is copied from
// https://github.com/v2fly/v2ray-core/blob/master/app/router/routercommon/common.proto
// with some modifications.
// Domain for routing decision.
message Domain {
// Type of domain value.
enum Type {
// The value is used as is.
Plain = 0;
// The value is used as a regular expression.
Regex = 1;
// The value is a root domain.
RootDomain = 2;
// The value is a domain.
Full = 3;
}
// Domain matching type.
Type type = 1;
// Domain value.
string value = 2;
message Attribute {
string key = 1;
oneof typed_value {
bool bool_value = 2;
int64 int_value = 3;
}
}
// Attributes of this domain. May be used for filtering.
repeated Attribute attribute = 3;
}
// IP for routing decision, in CIDR form.
message CIDR {
// IP address, should be either 4 or 16 bytes.
bytes ip = 1;
// Number of leading ones in the network mask.
uint32 prefix = 2;
}
message GeoIP {
string country_code = 1;
repeated CIDR cidr = 2;
bool inverse_match = 3;
// resource_hash instruct simplified config converter to load domain from geo file.
bytes resource_hash = 4;
string code = 5;
}
message GeoIPList {
repeated GeoIP entry = 1;
}
message GeoSite {
string country_code = 1;
repeated Domain domain = 2;
// resource_hash instruct simplified config converter to load domain from geo file.
bytes resource_hash = 3;
string code = 4;
}
message GeoSiteList {
repeated GeoSite entry = 1;
}

230
ruleset/expr.go
View File

@ -1,15 +1,10 @@
package ruleset
import (
"context"
"fmt"
"net"
"os"
"reflect"
"strings"
"time"
"github.com/expr-lang/expr/builtin"
"github.com/expr-lang/expr"
"github.com/expr-lang/expr/ast"
@ -19,14 +14,12 @@ import (
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/modifier"
"github.com/apernet/OpenGFW/ruleset/builtins"
)
// ExprRule is the external representation of an expression rule.
type ExprRule struct {
Name string `yaml:"name"`
Action string `yaml:"action"`
Log bool `yaml:"log"`
Modifier ModifierEntry `yaml:"modifier"`
Expr string `yaml:"expr"`
}
@ -49,119 +42,83 @@ func ExprRulesFromYAML(file string) ([]ExprRule, error) {
// compiledExprRule is the internal, compiled representation of an expression rule.
type compiledExprRule struct {
Name string
Action *Action // fallthrough if nil
Log bool
Action Action
ModInstance modifier.Instance
Program *vm.Program
Analyzers map[string]struct{}
}
var _ Ruleset = (*exprRuleset)(nil)
type exprRuleset struct {
Rules []compiledExprRule
Ans []analyzer.Analyzer
Logger Logger
Rules []compiledExprRule
Ans []analyzer.Analyzer
}
func (r *exprRuleset) Analyzers(info StreamInfo) []analyzer.Analyzer {
return r.Ans
}
func (r *exprRuleset) Match(info StreamInfo) MatchResult {
func (r *exprRuleset) Match(info StreamInfo) (MatchResult, error) {
env := streamInfoToExprEnv(info)
for _, rule := range r.Rules {
v, err := vm.Run(rule.Program, env)
if err != nil {
// Log the error and continue to the next rule.
r.Logger.MatchError(info, rule.Name, err)
continue
return MatchResult{
Action: ActionMaybe,
}, fmt.Errorf("rule %q failed to run: %w", rule.Name, err)
}
if vBool, ok := v.(bool); ok && vBool {
if rule.Log {
r.Logger.Log(info, rule.Name)
}
if rule.Action != nil {
return MatchResult{
Action: *rule.Action,
ModInstance: rule.ModInstance,
}
}
return MatchResult{
Action: rule.Action,
ModInstance: rule.ModInstance,
}, nil
}
}
// No match
return MatchResult{
Action: ActionMaybe,
}
}, nil
}
// CompileExprRules compiles a list of expression rules into a ruleset.
// It returns an error if any of the rules are invalid, or if any of the analyzers
// used by the rules are unknown (not provided in the analyzer list).
func CompileExprRules(rules []ExprRule, ans []analyzer.Analyzer, mods []modifier.Modifier, config *BuiltinConfig) (Ruleset, error) {
func CompileExprRules(rules []ExprRule, ans []analyzer.Analyzer, mods []modifier.Modifier) (Ruleset, error) {
var compiledRules []compiledExprRule
fullAnMap := analyzersToMap(ans)
fullModMap := modifiersToMap(mods)
depAnMap := make(map[string]analyzer.Analyzer)
funcMap := buildFunctionMap(config)
// Compile all rules and build a map of analyzers that are used by the rules.
for _, rule := range rules {
if rule.Action == "" && !rule.Log {
return nil, fmt.Errorf("rule %q must have at least one of action or log", rule.Name)
action, ok := actionStringToAction(rule.Action)
if !ok {
return nil, fmt.Errorf("rule %q has invalid action %q", rule.Name, rule.Action)
}
var action *Action
if rule.Action != "" {
a, ok := actionStringToAction(rule.Action)
if !ok {
return nil, fmt.Errorf("rule %q has invalid action %q", rule.Name, rule.Action)
}
action = &a
}
visitor := &idVisitor{Variables: make(map[string]bool), Identifiers: make(map[string]bool)}
patcher := &idPatcher{FuncMap: funcMap}
visitor := &depVisitor{Analyzers: make(map[string]struct{})}
program, err := expr.Compile(rule.Expr,
func(c *conf.Config) {
c.Strict = false
c.Expect = reflect.Bool
c.Visitors = append(c.Visitors, visitor, patcher)
for name, f := range funcMap {
c.Functions[name] = &builtin.Function{
Name: name,
Func: f.Func,
Types: f.Types,
}
}
c.Visitors = append(c.Visitors, visitor)
},
)
if err != nil {
return nil, fmt.Errorf("rule %q has invalid expression: %w", rule.Name, err)
}
if patcher.Err != nil {
return nil, fmt.Errorf("rule %q failed to patch expression: %w", rule.Name, patcher.Err)
}
for name := range visitor.Identifiers {
// Skip built-in analyzers & user-defined variables
if isBuiltInAnalyzer(name) || visitor.Variables[name] {
continue
}
if f, ok := funcMap[name]; ok {
// Built-in function, initialize if necessary
if f.InitFunc != nil {
if err := f.InitFunc(); err != nil {
return nil, fmt.Errorf("rule %q failed to initialize function %q: %w", rule.Name, name, err)
}
}
} else if a, ok := fullAnMap[name]; ok {
// Analyzer, add to dependency map
depAnMap[name] = a
for name := range visitor.Analyzers {
a, ok := fullAnMap[name]
if !ok && !isBuiltInAnalyzer(name) {
return nil, fmt.Errorf("rule %q uses unknown analyzer %q", rule.Name, name)
}
depAnMap[name] = a
}
cr := compiledExprRule{
Name: rule.Name,
Action: action,
Log: rule.Log,
Program: program,
Name: rule.Name,
Action: action,
Program: program,
Analyzers: visitor.Analyzers,
}
if action != nil && *action == ActionModify {
if action == ActionModify {
mod, ok := fullModMap[rule.Modifier.Name]
if !ok {
return nil, fmt.Errorf("rule %q uses unknown modifier %q", rule.Name, rule.Modifier.Name)
@ -180,9 +137,8 @@ func CompileExprRules(rules []ExprRule, ans []analyzer.Analyzer, mods []modifier
depAns = append(depAns, a)
}
return &exprRuleset{
Rules: compiledRules,
Ans: depAns,
Logger: config.Logger,
Rules: compiledRules,
Ans: depAns,
}, nil
}
@ -252,126 +208,12 @@ func modifiersToMap(mods []modifier.Modifier) map[string]modifier.Modifier {
return modMap
}
// idVisitor is a visitor that collects all identifiers in an expression.
// This is for determining which analyzers are used by the expression.
type idVisitor struct {
Variables map[string]bool
Identifiers map[string]bool
type depVisitor struct {
Analyzers map[string]struct{}
}
func (v *idVisitor) Visit(node *ast.Node) {
if varNode, ok := (*node).(*ast.VariableDeclaratorNode); ok {
v.Variables[varNode.Name] = true
} else if idNode, ok := (*node).(*ast.IdentifierNode); ok {
v.Identifiers[idNode.Value] = true
}
}
// idPatcher patches the AST during expr compilation, replacing certain values with
// their internal representations for better runtime performance.
type idPatcher struct {
FuncMap map[string]*Function
Err error
}
func (p *idPatcher) Visit(node *ast.Node) {
switch (*node).(type) {
case *ast.CallNode:
callNode := (*node).(*ast.CallNode)
if callNode.Callee == nil {
// Ignore invalid call nodes
return
}
if f, ok := p.FuncMap[callNode.Callee.String()]; ok {
if f.PatchFunc != nil {
if err := f.PatchFunc(&callNode.Arguments); err != nil {
p.Err = err
return
}
}
}
}
}
type Function struct {
InitFunc func() error
PatchFunc func(args *[]ast.Node) error
Func func(params ...any) (any, error)
Types []reflect.Type
}
func buildFunctionMap(config *BuiltinConfig) map[string]*Function {
return map[string]*Function{
"geoip": {
InitFunc: config.GeoMatcher.LoadGeoIP,
PatchFunc: nil,
Func: func(params ...any) (any, error) {
return config.GeoMatcher.MatchGeoIp(params[0].(string), params[1].(string)), nil
},
Types: []reflect.Type{reflect.TypeOf(config.GeoMatcher.MatchGeoIp)},
},
"geosite": {
InitFunc: config.GeoMatcher.LoadGeoSite,
PatchFunc: nil,
Func: func(params ...any) (any, error) {
return config.GeoMatcher.MatchGeoSite(params[0].(string), params[1].(string)), nil
},
Types: []reflect.Type{reflect.TypeOf(config.GeoMatcher.MatchGeoSite)},
},
"cidr": {
InitFunc: nil,
PatchFunc: func(args *[]ast.Node) error {
cidrStringNode, ok := (*args)[1].(*ast.StringNode)
if !ok {
return fmt.Errorf("cidr: invalid argument type")
}
cidr, err := builtins.CompileCIDR(cidrStringNode.Value)
if err != nil {
return err
}
(*args)[1] = &ast.ConstantNode{Value: cidr}
return nil
},
Func: func(params ...any) (any, error) {
return builtins.MatchCIDR(params[0].(string), params[1].(*net.IPNet)), nil
},
Types: []reflect.Type{reflect.TypeOf(builtins.MatchCIDR)},
},
"lookup": {
InitFunc: nil,
PatchFunc: func(args *[]ast.Node) error {
var serverStr *ast.StringNode
if len(*args) > 1 {
// Has the optional server argument
var ok bool
serverStr, ok = (*args)[1].(*ast.StringNode)
if !ok {
return fmt.Errorf("lookup: invalid argument type")
}
}
r := &net.Resolver{
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
if serverStr != nil {
address = serverStr.Value
}
return config.ProtectedDialContext(ctx, network, address)
},
}
if len(*args) > 1 {
(*args)[1] = &ast.ConstantNode{Value: r}
} else {
*args = append(*args, &ast.ConstantNode{Value: r})
}
return nil
},
Func: func(params ...any) (any, error) {
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
defer cancel()
return params[1].(*net.Resolver).LookupHost(ctx, params[0].(string))
},
Types: []reflect.Type{
reflect.TypeOf((func(string, *net.Resolver) []string)(nil)),
},
},
func (v *depVisitor) Visit(node *ast.Node) {
if idNode, ok := (*node).(*ast.IdentifierNode); ok {
v.Analyzers[idNode.Value] = struct{}{}
}
}

16
ruleset/interface.go
View File

@ -1,13 +1,11 @@
package ruleset
import (
"context"
"net"
"strconv"
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/modifier"
"github.com/apernet/OpenGFW/ruleset/builtins/geo"
)
type Action int
@ -92,17 +90,5 @@ type Ruleset interface {
Analyzers(StreamInfo) []analyzer.Analyzer
// Match matches a stream against the ruleset and returns the result.
// It must be safe for concurrent use by multiple workers.
Match(StreamInfo) MatchResult
}
// Logger is the logging interface for the ruleset.
type Logger interface {
Log(info StreamInfo, name string)
MatchError(info StreamInfo, name string, err error)
}
type BuiltinConfig struct {
Logger Logger
GeoMatcher *geo.GeoMatcher
ProtectedDialContext func(ctx context.Context, network, address string) (net.Conn, error)
Match(StreamInfo) (MatchResult, error)
}