Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate config item max-memory and add items server-memory-quota and memory-usage-alarm-ratio #4977

Merged
merged 24 commits into from
Dec 2, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
correct indentation
  • Loading branch information
TomShawn committed Dec 2, 2020
commit 140a4484fb9eb7132f685a8eb6100308e5080424
40 changes: 22 additions & 18 deletions configure-memory-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ set @@tidb_mem_quota_query = 8 << 10;

例如,配置 tidb-server 实例的内存使用总量,将其设置成为 32 GB:

{{< copyable "" >}}

```toml
[performance]
server-memory-quota = 34359738368
wshwsh12 marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -81,8 +83,8 @@ server-memory-quota = 34359738368

> **警告:**
TomShawn marked this conversation as resolved.
Show resolved Hide resolved
>
> `server-memory-quota` 默认值为 0,表示无内存限制
> `server-memory-quota` 目前为实验性特性,不建议在生产环境中使用
> + `server-memory-quota` 目前为实验性特性,不建议在生产环境中使用
> + `server-memory-quota` 默认值为 0,表示无内存限制

## tidb-server 内存占用过高时的报警

Expand All @@ -94,29 +96,31 @@ server-memory-quota = 34359738368

1. 配置报警比例为 `0.8`:

```toml
mem-quota-query = 34359738368 // 将单条 SQL 内存限制调高,以便于构造占用内存较大的 SQL
[performance]
memory-usage-alarm-ratio = 0.8
```
{{< copyable "" >}}

```toml
mem-quota-query = 34359738368 // 将单条 SQL 内存限制调高,以便于构造占用内存较大的 SQL
[performance]
memory-usage-alarm-ratio = 0.8
```

2. 创建单表 `CREATE TABLE t(a int);` 并插入 1000 行数据。

3. 执行 `select * from t t1 join t t1 join t t3 order by t1.a`。该 SQL 语句会输出 1000000000 条记录,占用巨大的内存,进而触发报警。

4. 检查 `tidb.log` 文件,其中会记录系统总内存、系统当前内存使用量、tidb-server 实例的内存使用量以及状态文件所在目录。

```
[2020/11/30 15:25:17.252 +08:00] [WARN] [memory_usage_alarm.go:141] ["tidb-server has the risk of OOM. Running SQLs and heap profile will be recorded in record path"] ["is server-memory-quota set"=false] ["system memory total"=33682427904] ["system memory usage"=27142864896] ["tidb-server memory usage"=22417922896] [memory-usage-alarm-ratio=0.8] ["record path"="/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record"]
```
```
[2020/11/30 15:25:17.252 +08:00] [WARN] [memory_usage_alarm.go:141] ["tidb-server has the risk of OOM. Running SQLs and heap profile will be recorded in record path"] ["is server-memory-quota set"=false] ["system memory total"=33682427904] ["system memory usage"=27142864896] ["tidb-server memory usage"=22417922896] [memory-usage-alarm-ratio=0.8] ["record path"="/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record"]
```

以上 Log 字段的含义如下:
以上 Log 字段的含义如下:

* `is server-memory-quota set`:表示配置项 [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) 是否被设置
* `system memory total`:表示当前系统的总内存
* `system memory usage`:表示当前系统的内存使用量
* `tidb-server memory usage`:表示 tidb-server 实例的内存使用量
* `memory-usage-alarm-ratio`:表示配置项 [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio) 的值
* `record path`:表示状态文件存放的目录
* `is server-memory-quota set`:表示配置项 [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) 是否被设置
* `system memory total`:表示当前系统的总内存
* `system memory usage`:表示当前系统的内存使用量
* `tidb-server memory usage`:表示 tidb-server 实例的内存使用量
* `memory-usage-alarm-ratio`:表示配置项 [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio) 的值
* `record path`:表示状态文件存放的目录

5. 通过访问状态文件所在目录(该示例中的目录为 `/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record`),可以得到一组文件,其中包括 `goroutinue`、`heap`、`running_sql` 3 个文件,文件以记录状态文件的时间为后缀。这 3 个文件分别用来记录报警时的 goroutine 栈信息,堆内存使用状态,及正在运行的 SQL 信息。其中 `running_sql` 文件内的日志格式请参考 [`expensive-queries`](/identify-expensive-queries.md)
5. 通过访问状态文件所在目录(该示例中的目录为 `/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record`),可以得到一组文件,其中包括 `goroutinue`、`heap`、`running_sql` 3 个文件,文件以记录状态文件的时间为后缀。这 3 个文件分别用来记录报警时的 goroutine 栈信息,堆内存使用状态,及正在运行的 SQL 信息。其中 `running_sql` 文件内的日志格式请参考 [`expensive-queries`](/identify-expensive-queries.md)。
4 changes: 2 additions & 2 deletions tidb-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,12 +286,12 @@ TiDB 配置文件比命令行参数支持更多的选项。你可以在 [config/

+ tidb-server 实例内存的使用限制,单位为字节。(原配置项 `max-memory` 已被该配置项代替)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

原配置项 max-memory 被一个实验特性配置项所替代?


+ 默认值:0
+ 默认值:0
+ 默认值为 0 表示无内存限制。

### `memory-usage-alarm-ratio`

+ tidb-server 实例内存使用占总内存的比例超过一定阈值时会报警。该配置项的有效范围为 01。如果配置该选项为 01,则表示关闭内存阈值报警功能。
+ tidb-server 实例内存使用占总内存的比例超过一定阈值时会报警。该配置项的有效范围为 `0``1`。如果配置该选项为 `0``1`,则表示关闭内存阈值报警功能。
+ 默认值:0.8
+ 当内存阈值报警功能开启时,如果配置项 [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) 未设置,则内存报警阈值为 `memory-usage-alarm-ratio * 系统内存大小`;如果 `server-memory-quota` 被设置且大于 0,则内存报警阈值为 `memory-usage-alarm-ratio * server-memory-quota`。
+ 当 TiDB 检测到 tidb-server 的内存使用超过了阈值,则会认为存在内存溢出的风险,会将当前正在执行的所有 SQL 语句中内存使用最高的 10 条语句和运行时间最长的 10 条语句以及 heap profile 记录到目录 [`tmp-storage-path/record`](/tidb-configuration-file.md#tmp-storage-path) 中,并输出一条包含关键字 `tidb-server has the risk of OOM` 的日志。
Expand Down