Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

titan doc update for release 7.6.0 #15986

Merged
merged 34 commits into from
Jan 25, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
64ecfce
titan doc update for release 7.6.0
tonyxuqqi Jan 5, 2024
9e58e74
lint issue
Jan 5, 2024
19eb7ef
Apply suggestions from code review
hfxsd Jan 8, 2024
9546630
Apply suggestions from code review
hfxsd Jan 8, 2024
b6dacd0
Update tikv-configuration-file.md
hfxsd Jan 8, 2024
481b007
Apply suggestions from code review
hfxsd Jan 9, 2024
c43a35c
change the default value of blob-file-compression to zstd
hfxsd Jan 9, 2024
3ca3d45
Update tikv-configuration-file.md
hfxsd Jan 9, 2024
a47c5bf
Update tikv-configuration-file.md
hfxsd Jan 9, 2024
02d78bb
Apply suggestions from code review
hfxsd Jan 16, 2024
331bfe1
polish titan doc
tonyxuqqi Jan 17, 2024
8bb32cb
Merge branch 'titan_7.6' of https://github.com/tonyxuqqi/docs into ti…
tonyxuqqi Jan 17, 2024
b86de77
address comments
tonyxuqqi Jan 17, 2024
adb3363
update gc thread count
tonyxuqqi Jan 22, 2024
d8b48fd
update num-threads
tonyxuqqi Jan 22, 2024
cc5fecf
titan: update titan doc for v7.6.0 (enable titan by default)
benmaoer Jan 23, 2024
17c7f43
Merge pull request #1 from benmaoer/15986-titan-doc-updates
tonyxuqqi Jan 23, 2024
3937b19
Merge remote-tracking branch 'upstream/master' into pr/15986
hfxsd Jan 24, 2024
8b8a477
synced cn changes
hfxsd Jan 24, 2024
8bb38a4
Update tikv-configuration-file.md
hfxsd Jan 24, 2024
b6554f7
Update titan-configuration.md
hfxsd Jan 24, 2024
c93019e
Update titan-configuration.md
hfxsd Jan 24, 2024
4b9baf6
Update storage-engine/titan-overview.md
hfxsd Jan 24, 2024
a1bbf0a
Apply suggestions from code review
hfxsd Jan 24, 2024
51e07da
Update storage-engine/titan-configuration.md
hfxsd Jan 24, 2024
4c89679
Update storage-engine/titan-configuration.md
hfxsd Jan 24, 2024
5664bc7
add min blob size link
hfxsd Jan 24, 2024
44f95a4
Apply suggestions from code review
hfxsd Jan 24, 2024
5c6ae2c
Update tikv-configuration-file.md
hfxsd Jan 24, 2024
92ff46a
Apply suggestions from code review
hfxsd Jan 24, 2024
a6ba25f
Update storage-engine/titan-configuration.md
hfxsd Jan 24, 2024
204afcd
Update storage-engine/titan-configuration.md
hfxsd Jan 24, 2024
aa0a9a4
Update tikv-configuration-file.md
hfxsd Jan 24, 2024
665c9f9
Update tikv-configuration-file.md
hfxsd Jan 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Apply suggestions from code review
Co-authored-by: Ran <huangran.alex@gmail.com>
  • Loading branch information
hfxsd and ran-huang authored Jan 16, 2024
commit 02d78bbc2cc8029b4cb8290cc6a59185a8c4e940
6 changes: 3 additions & 3 deletions storage-engine/titan-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ To disable Titan, you can configure the `rocksdb.defaultcf.titan.blob-run-mode`
- When the option is set to `read-only`, all newly written values are written into RocksDB, regardless of the value size.
- When the option is set to `fallback`, all newly written values are written into RocksDB, regardless of the value size. Also, all compacted values stored in the Titan blob file are automatically moved back to RocksDB.

To fully disable Titan for all existing and future data, you can follow these steps. Note that in general you can skip Step 2 because it can greatly impact online traffic performance. In fact even without Step 2, the data convertion consumes extra I/O and CPU resources, and performance will degrade (sometimes as much as 50%) when TiKV's I/O or CPU resources are limited.
To fully disable Titan for all existing and future data, you can follow these steps. Note that in general you can skip Step 2 because it can greatly impact online traffic performance. In fact even without Step 2, the data compaction consumes extra I/O and CPU resources when it migrates data from Titan to RocksDB, and performance will degrade (sometimes as much as 50%) when TiKV's I/O or CPU resources are limited.

1. Update the configuration of the TiKV nodes you wish to disable Titan for. You can update configuration in two methods:

Expand All @@ -144,9 +144,9 @@ To fully disable Titan for all existing and future data, you can follow these st

> **Note:**
>
> Use the default value `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio) when there is not enough disk space to hold both Titan and RocksDB data. In general, it is recommended to use the default value if the free disk space is less than 50%. This is because when `discardable-ratio = 1.0`, the RocksDB data keeps growing. At the same time, the recovery of Titan's original blob file requires all the data in that file to be migrated to RocksDB, which is a slow process. If the disk size is big enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth.
> When there is insufficient disk space to accommodate both Titan and RocksDB data, it is recommended to use the default value of `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio). In general, if the free disk space is below 50%, it is advisable to use the default value. This is because when `discardable-ratio = 1.0`, the RocksDB data keeps growing. At the same time, the recovery of Titan's original blob file requires all the data in that file to be migrated to RocksDB, which is a slow process. However, if the disk size is large enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth.

2. [Optional] Perform a full compaction using tikv-ctl. This process will consume a large amount of I/O and CPU resources.
2. (Optional) Perform a full compaction using tikv-ctl. This process will consume a large amount of I/O and CPU resources.

```bash
tikv-ctl --pd <PD_ADDR> compact-cluster --bottommost force
Expand Down
2 changes: 1 addition & 1 deletion storage-engine/titan-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The prerequisites for enabling Titan are as follows:
- No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB.
- Sufficient disk space (consider reserving a space twice of the RocksDB disk consumption with the same data volume). This is because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB.

Starting from v7.6.0, Titan is improved and is enabled by default for newly created clusters. Because small KV datasets will still be stored in RocksDB even Titan is enabled, you can still enable Titan in the configuration.
Starting from v7.6.0, Titan is improved and is enabled by default for newly created clusters. Because small KV datasets will still be stored in RocksDB even when Titan is enabled, you can still enable Titan in the configuration.

If you want to improve the performance of Titan, see the blog post [Titan: A RocksDB Plugin to Reduce Write Amplification](https://pingcap.com/blog/titan-storage-engine-design-and-implementation/).

Expand Down
Loading