Skip to content

Latest commit

 

History

History
159 lines (120 loc) · 8.12 KB

README_ZH.md

File metadata and controls

159 lines (120 loc) · 8.12 KB
Raw

ForMe

English

Rust CDC 客户端。 是MySQL binlog解析器的干净、方便的 Rust实现，

包括对MySQL 5.6/5.7/8.0中引入的JSONB类型的支持。

其主要目的是处理基于行的日志记录消息， 已经针对（MySQL）5.6、5.7和8.0进行了测试。

这个库试图在解析完整的binlog文件后允许定制各种不同输出端的实现，所有感兴趣的数据类型都可以使用Serde进行序列化，

因此很容易连接到其他数据处理流中。

Limitations

请注意，目前有以下限制：

  • 仅支持标准身份验证插件mysql_native_password和caching_sha2_password。
  • 目前，该库不支持SSL加密。
  • 不处理拆分数据包（16MB及以上）。

Development environment

为了降低执行错误的概率并改进功能特征， 我们统一了Rust工具链的版本，并切换了以下命令：

 rustup default nightly

您可以在 rustup toolchain list 中查看它。如果没有，它将自动下载。

Architecture

mysql-cdc-rs-architecture

模块依赖图设计图

模块职能设计


+-- binlog： binlog 事件解析的能力实现
+-- binlog-Adapter： binlog 事件数据结构转中立数据输出实现
    -- log: 默认的binlog数据的日志输出
    -- relay_log: 默认的binlog数据的中继日志输出
+-- binlog_cli： CLI 客户端
+-- common: 基本类型定义
+-- conf: 工程默认配置文件
+-- connection: 提供 MySQL/PostgreSQL/MariaDB 的连接能力和binlog订阅能力
+-- doc: 文档
+-- memory: 内存分配器
+-- raft: raft 协议(Broker Impl)
+-- relay_log: 中继日志
+-- replayer: 启动入口
+-- rpc: rpc 协议
+-- sink: 中继数据推送至Broker的服务
+-- slave: 提供mysql slave 伪装能力与dump能力
+-- tests: 测试用例

How to Use

# cargo tree
cargo build

FAQ

Windows环境编译报错。 error: failed to run custom build command for openssl-sys v0.9.102

Caused by:
  process didn't exit successfully: `mysql-cdc-rs\target\debug\build\openssl-sys-94071a3d762a0669\build-script-main` (exit code: 101)
  --- stdout
  cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR
  X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_VENDOR
  OPENSSL_NO_VENDOR unset
  running "perl" "./Configure" "--prefix=/mysql-cdc-rs/target/debug/build/openssl-sys-5ad9f46fc53da764/out/openssl-build/install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"

  Error configuring OpenSSL build:
      Command: "perl" "./Configure" "--prefix=/mysql-cdc-rs/target/debug/build/openssl-sys-5ad9f46fc53da764/out/openssl-build/install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"
      Failed to execute: program not found

说明使用perl程序进行编译配置。 从 https://strawberryperl.com/ 下载windows安装包。 安装后，重启命令行终端， 再次build。

strawberry-perl-5.38.2.2-64bit.msi

CLI

See BinlogCLI README.md

演示

http://s.codealy.com/rust_us/mysql_cdc_rs/2024.02%20binlog%20cli%20view.webm

Support Event

See Binlog README.md

是一个基于 Rust 实现的 MySQL binlog 文件解析库， 纯 Rust 实现，无需 mysql-server 库， 但同时也可以订阅 mysql-master-server。

项目的目标是尽量解析 binlog 事件的每个字段。

Parsed events matrix:

Hex Event Name Support Tested Noted
0x00 UNKNOWN_EVENT support and not tested
0x01 START_EVENT_V3 too old and support not tested
0x02 QUERY_EVENT support tested
0x03 STOP_EVENT support not tested
0x04 ROTATE_EVENT support tested
0x05 INTVAR_EVENT support tested
0x06 LOAD_EVENT not fully support not tested
0x07 SLAVE_EVENT not fully support not tested
0x08 CREATE_FILE_EVENT not fully support not tested
0x09 APPEND_BLOCK_EVENT not fully support not tested
0x0a EXEC_LOAD_EVENT not fully support not tested
0x0b DELETE_FILE_EVENT not fully support not tested
0x0c NEW_LOAD_EVENT support not tested
0x0d RAND_EVENT support not tested
0x0e USER_VAR_EVENT support not fully tested
0x0f FORMAT_DESCRIPTION_EVENT support tested
0x10 XID_EVENT not fully support tested
0x11 BEGIN_LOAD_QUERY_EVENT not fully support tested
0x12 EXECUTE_LOAD_QUERY_EVENT not fully support not tested
0x13 TABLE_MAP_EVENT support tested
0x14 PreGaWriteRowsEvent(v0) not support not tested
0x15 PreGaUpdateRowsEvent(v0) not support not tested
0x16 PreGaDeleteRowsEvent(v0) not support not tested
0x17 WRITE_ROWS_EVENTv1 support tested
0x18 UPDATE_ROWS_EVENTv1 support tested
0x19 DELETE_ROWS_EVENTv1 support tested
0x1a INCIDENT_EVENT not fully support not tested
0x1b HEARTBEAT_EVENT not fully support not tested
0x1c IGNORABLE_EVENT support not tested
0x1d ROWS_QUERY_EVENT not fully support not fully tested
0x1e WRITE_ROWS_EVENTv2 support not fully tested
0x1f UPDATE_ROWS_EVENTv2 support not fully tested
0x20 DELETE_ROWS_EVENTv2 support not fully tested
0x21 GTID_EVENT support tested
0x22 ANONYMOUS_GTID_EVENT support tested
0x23 PREVIOUS_GTIDS_EVENT support tested
0x24 TRANSACTION_CONTEXT_EVENT not support not tested
0x25 VIEW_CHANGE_EVENT not support not tested
0x26 XA_PREPARE_LOG_EVENT not support not tested
0x27 PARTIAL_UPDATE_ROWS_EVENT not support not tested
0x28 TRANSACTION_PAYLOAD_EVENT not support not tested
0x29 HEARTBEAT_LOG_EVENT_V2 not support not tested