diff --git a/v1.0/manual-deployment-insecure.md b/v1.0/manual-deployment-insecure.md index 7b32ab4b6a2..da2b8c93f25 100644 --- a/v1.0/manual-deployment-insecure.md +++ b/v1.0/manual-deployment-insecure.md @@ -57,7 +57,7 @@ This tutorial shows you how to manually deploy an insecure multi-node CockroachD --host= ~~~ - This command sets the node to insecure and identifies the address at which other nodes can reach it, in this case an internal address since you likely don't want applications outside your network reaching an insecure cluster. Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + This commands starts an insecure node and identifies the address at which other nodes can reach it, in this case an internal address since you likely don't want applications outside your network reaching an insecure cluster. Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). ## Step 2. Add nodes to the cluster diff --git a/v1.1/deploy-cockroachdb-on-aws-insecure.md b/v1.1/deploy-cockroachdb-on-aws-insecure.md index 3e1dc3a1f55..55c5bc1b096 100644 --- a/v1.1/deploy-cockroachdb-on-aws-insecure.md +++ b/v1.1/deploy-cockroachdb-on-aws-insecure.md @@ -28,7 +28,7 @@ You must have SSH access ([key pairs](http://docs.aws.amazon.com/AWSEC2/latest/U - Any user, connecting as `root`, can read or write any data in your cluster. - There is no network encryption or authentication, and thus no confidentiality. -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - All instances running CockroachDB should be members of the same Security Group. @@ -118,13 +118,20 @@ AWS offers fully-managed load balancing to distribute traffic between instances. $ sudo mv cockroach /usr/local/bin ~~~ -3. Start a new CockroachDB cluster with a single node, which will communicate with other nodes on its internal IP address: +3. Start a new CockroachDB cluster with a single node: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This commands starts an insecure node and identifies the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 5. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -151,11 +158,15 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: - ~~~ shell - $ cockroach start --insecure \ - --background \ - --join=:26257 ~~~ + $ cockroach start --insecure \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background + ~~~ + + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. 4. Repeat these steps for each instance you want to use as a node. diff --git a/v1.1/deploy-cockroachdb-on-aws.md b/v1.1/deploy-cockroachdb-on-aws.md index 156aca026e7..4abad7d7713 100644 --- a/v1.1/deploy-cockroachdb-on-aws.md +++ b/v1.1/deploy-cockroachdb-on-aws.md @@ -24,7 +24,7 @@ If you are only testing CockroachDB, or you are not concerned with protecting ne ## Recommendations -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - All instances running CockroachDB should be members of the same Security Group. @@ -257,15 +257,22 @@ Locally, you'll need to [create the following certificates and keys](create-secu $ sudo mv cockroach /usr/local/bin ~~~ -3. Start a new CockroachDB cluster with a single node, specifying the location of certificates and the address at which other nodes can reach it: +3. Start a new CockroachDB cluster with a single node: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --advertise-host= + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This command specifies the location of certificates and the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 6. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -301,13 +308,18 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --advertise-host= \ - --join=:26257 + --advertise-host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each instance you want to use as a node. ## Step 7. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-digital-ocean-insecure.md b/v1.1/deploy-cockroachdb-on-digital-ocean-insecure.md index e576372dc7a..45d1474e94e 100644 --- a/v1.1/deploy-cockroachdb-on-digital-ocean-insecure.md +++ b/v1.1/deploy-cockroachdb-on-digital-ocean-insecure.md @@ -28,7 +28,7 @@ You must have [SSH access](https://www.digitalocean.com/community/tutorials/how- - Any user, connecting as `root`, can read or write any data in your cluster. - There is no network encryption or authentication, and thus no confidentiality. -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Set up your Droplets using [private networking](https://www.digitalocean.com/community/tutorials/how-to-set-up-and-use-digitalocean-private-networking). @@ -100,12 +100,18 @@ For guidance, you can use Digital Ocean's guide to configuring firewalls based o 3. Start a new CockroachDB cluster with a single node, which will communicate with other nodes on its internal IP address: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background \ - --advertise-host= + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This commands starts an insecure node and identifies the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 5. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -132,13 +138,16 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background \ - --advertise-host= \ - --join=:26257 + --advertise-host= \ + --join=:26257 \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each Droplet you want to use as a node. ## Step 6. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-digital-ocean.md b/v1.1/deploy-cockroachdb-on-digital-ocean.md index 38208458279..52455853f81 100644 --- a/v1.1/deploy-cockroachdb-on-digital-ocean.md +++ b/v1.1/deploy-cockroachdb-on-digital-ocean.md @@ -24,7 +24,7 @@ If you are only testing CockroachDB, or you are not concerned with protecting ne ## Recommendations -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Set up your Droplets using [private networking](https://www.digitalocean.com/community/tutorials/how-to-set-up-and-use-digitalocean-private-networking). @@ -237,12 +237,19 @@ Locally, you'll need to [create the following certificates and keys](create-secu 3. Start a new CockroachDB cluster with a single node, specifying the location of certificates and the address at which other nodes can reach it: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --advertise-host= + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This command specifies the location of certificates and the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 6. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -278,13 +285,18 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --advertise-host= \ - --join=:26257 + --advertise-host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each Droplet you want to use as a node. ## Step 7. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-google-cloud-platform-insecure.md b/v1.1/deploy-cockroachdb-on-google-cloud-platform-insecure.md index c208d6f9d2e..228920bf392 100644 --- a/v1.1/deploy-cockroachdb-on-google-cloud-platform-insecure.md +++ b/v1.1/deploy-cockroachdb-on-google-cloud-platform-insecure.md @@ -28,7 +28,7 @@ You must have [SSH access](https://cloud.google.com/compute/docs/instances/conne - Any user, connecting as `root`, can read or write any data in your cluster. - There is no network encryption or authentication, and thus no confidentiality. -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Decide how you want to access your Admin UI: - Only from specific IP addresses, which requires you to set firewall rules to allow communication on port `8080` *(documented on this page)*. @@ -119,11 +119,18 @@ To use GCE's TCP Proxy Load Balancing service: 3. Start a new CockroachDB cluster with a single node: - ~~~ shell + ~~~ $ cockroach start --insecure \ + --host= \ + --cache=25% \ + --max-sql-memory=25% \ --background ~~~ + This commands starts an insecure node and identifies the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 5. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -150,12 +157,17 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background \ - --join=:26257 + --host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each instance you want to use as a node. ## Step 6. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-google-cloud-platform.md b/v1.1/deploy-cockroachdb-on-google-cloud-platform.md index 5d2fc5acea0..988a40b5641 100644 --- a/v1.1/deploy-cockroachdb-on-google-cloud-platform.md +++ b/v1.1/deploy-cockroachdb-on-google-cloud-platform.md @@ -24,7 +24,7 @@ If you are only testing CockroachDB, or you are not concerned with protecting ne ## Recommendations -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Decide how you want to access your Admin UI: - Only from specific IP addresses, which requires you to set firewall rules to allow communication on port `8080` *(documented on this page)*. @@ -254,13 +254,21 @@ Locally, you'll need to [create the following certificates and keys](create-secu $ sudo mv cockroach /usr/local/bin ~~~ -3. Start a new CockroachDB cluster with a single node, specifying the location of certificates and the address at which other nodes can reach it: +3. Start a new CockroachDB cluster with a single node: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ - --certs-dir=certs ~~~ + $ cockroach start \ + --host= \ + --certs-dir=certs \ + --cache=25% \ + --max-sql-memory=25% \ + --background + ~~~ + + This command specifies the location of certificates and the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). ## Step 6. Add nodes to the cluster @@ -297,12 +305,18 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --join=:26257 + --host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each instance you want to use as a node. ## Step 7. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-microsoft-azure-insecure.md b/v1.1/deploy-cockroachdb-on-microsoft-azure-insecure.md index d6b3f46fa41..d4fadead73f 100644 --- a/v1.1/deploy-cockroachdb-on-microsoft-azure-insecure.md +++ b/v1.1/deploy-cockroachdb-on-microsoft-azure-insecure.md @@ -28,7 +28,7 @@ You must have [SSH access](https://docs.microsoft.com/en-us/azure/virtual-machin - Any user, connecting as `root`, can read or write any data in your cluster. - There is no network encryption or authentication, and thus no confidentiality. -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Decide how you want to access your Admin UI: - Only from specific IP addresses, which requires you to set firewall rules to allow communication on port `8080` *(documented on this page)*. @@ -126,12 +126,18 @@ Microsoft Azure offers fully-managed load balancing to distribute traffic betwee 3. Start a new CockroachDB cluster with a single node: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background \ - --advertise-host= + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This commands starts an insecure node and identifies the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + {{site.data.alerts.callout_info}}You can find the VM's internal IP address listed in the Resource Group's Virtual Network.{{site.data.alerts.end}} ## Step 5. Add nodes to the cluster @@ -160,13 +166,17 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --background \ --advertise-host= \ - --join=:26257 + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each VM you want to use as a node. ## Step 6. Test your cluster diff --git a/v1.1/deploy-cockroachdb-on-microsoft-azure.md b/v1.1/deploy-cockroachdb-on-microsoft-azure.md index 674562c8f90..5cb41d50f68 100644 --- a/v1.1/deploy-cockroachdb-on-microsoft-azure.md +++ b/v1.1/deploy-cockroachdb-on-microsoft-azure.md @@ -24,7 +24,7 @@ If you are only testing CockroachDB, or you are not concerned with protecting ne ## Recommendations -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). - Decide how you want to access your Admin UI: - Only from specific IP addresses, which requires you to set firewall rules to allow communication on port `8080` *(documented on this page)*. @@ -256,16 +256,22 @@ Locally, you'll need to [create the following certificates and keys](create-secu $ sudo mv cockroach /usr/local/bin ~~~ -3. Start a new CockroachDB cluster with a single node, specifying the location of certificates and the address at which other nodes can reach it: +3. Start a new CockroachDB cluster with a single node: {% include copy-clipboard.html %} - ~~~ shell + ~~~ $ cockroach start \ - --background \ --certs-dir=certs \ - --advertise-host= + --advertise-host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + This command specifies the location of certificates and the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). + ## Step 6. Add nodes to the cluster At this point, your cluster is live and operational but contains only a single node. Next, scale your cluster by setting up additional nodes that will join the cluster. @@ -301,14 +307,18 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's internal IP address: {% include copy-clipboard.html %} - ~~~ shell + ~~~ $ cockroach start \ - --background \ --certs-dir=certs \ - --advertise-host= \ - --join=:26257 + --advertise-host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ + The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. + 4. Repeat these steps for each instance you want to use as a node. ## Step 7. Test your cluster diff --git a/v1.1/manual-deployment-insecure.md b/v1.1/manual-deployment-insecure.md index 9e6ddcfdee4..3cf590c4650 100644 --- a/v1.1/manual-deployment-insecure.md +++ b/v1.1/manual-deployment-insecure.md @@ -30,7 +30,7 @@ This tutorial shows you how to manually deploy an insecure multi-node CockroachD - Any user, connecting as `root`, can read or write any data in your cluster. - There is no network encryption or authentication, and thus no confidentiality. -- For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +- For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). ## Step 1. Start the first node @@ -52,12 +52,17 @@ This tutorial shows you how to manually deploy an insecure multi-node CockroachD 3. Start a new CockroachDB cluster with a single node: - ~~~ shell + ~~~ $ cockroach start --insecure \ - --host= + --host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ - This command sets the node to insecure and identifies the address at which other nodes can reach it, in this case an internal address since you likely don't want applications outside your network reaching an insecure cluster. Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). + This commands starts an insecure node and identifies the address at which other nodes can reach it, in this case an internal address since you likely don't want applications outside your network reaching an insecure cluster. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). + + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, listens for internal and client communication on port 26257, and listens for HTTP requests from the Admin UI on port 8080. To set these options manually, see [Start a Node](start-a-node.html). ## Step 2. Add nodes to the cluster @@ -81,10 +86,13 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's address: - ~~~ shell + ~~~ $ cockroach start --insecure \ --host= \ - --join=:26257 + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. diff --git a/v1.1/manual-deployment.md b/v1.1/manual-deployment.md index 01704f5b3d2..c4d031a60b2 100644 --- a/v1.1/manual-deployment.md +++ b/v1.1/manual-deployment.md @@ -25,7 +25,7 @@ If you are only testing CockroachDB, or you are not concerned with protecting ne ## Recommendations -For guidance on cluster topology, clock synchronization, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). +For guidance on cluster topology, clock synchronization, cache and SQL memory size, and file descriptor limits, see [Recommended Production Settings](recommended-production-settings.html). ## Step 1. Generate certificates @@ -185,13 +185,18 @@ Locally, you'll need to [create the following certificates and keys](create-secu 3. Start a new CockroachDB cluster with a single node: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ - --certs-dir=certs \ - --host= - ~~~ + ~~~ + $ cockroach start \ + --certs-dir=certs \ + --host= \ + --cache=25% \ + --max-sql-memory=25% \ + --background + ~~~ + + This command specifies the location of certificates and the address at which other nodes can reach it. It also increases the node's cache and temporary SQL memory size to 25% of available system memory to improve read performance and increase capacity for in-memory SQL processing (see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details). - This command specifies the location of certificates and the address at which other nodes can reach it. Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). + Otherwise, it uses all available defaults. For example, the node stores data in the `cockroach-data` directory, binds internal and client communication to port 26257, and binds Admin UI HTTP requests to port 8080. To set these options manually, see [Start a Node](start-a-node.html). ## Step 3. Add nodes to the cluster @@ -223,11 +228,14 @@ At this point, your cluster is live and operational but contains only a single n 3. Start a new node that joins the cluster using the first node's address: {% include copy-clipboard.html %} - ~~~ shell - $ cockroach start --background \ + ~~~ + $ cockroach start \ --certs-dir=certs \ - --host= \ - --join=:26257 + --host= \ + --join=:26257 \ + --cache=25% \ + --max-sql-memory=25% \ + --background ~~~ The only difference when adding a node is that you connect it to the cluster with the `--join` flag, which takes the address and port of the first node. Otherwise, it's fine to accept all defaults; since each node is on a unique machine, using identical ports won't cause conflicts. diff --git a/v1.1/operational-faqs.md b/v1.1/operational-faqs.md index 26c866aaefb..e68aff8dfd4 100644 --- a/v1.1/operational-faqs.md +++ b/v1.1/operational-faqs.md @@ -31,7 +31,7 @@ flag. ## Why is memory usage increasing despite lack of traffic? -If you start a CockroachDB node on your computer and let it run for hours or days, you might notice that its memory usage steadily grows for a while before plateauing at around 25% of your computer's total memory. This is expected behavior -- like most databases, CockroachDB caches the most recently accessed data in memory so that it can provide faster reads, and [its periodic writes of timeseries data](#why-is-disk-usage-increasing-despite-lack-of-writes) cause that cache size to increase until it hits its configured limit. The cache size limit defaults to 25% of the machine's memory, but can be controlled by setting the `--cache` flag when running [`cockroach start`](start-a-node.html). +Like most databases, CockroachDB caches the most recently accessed data in memory so that it can provide faster reads, and [its periodic writes of timeseries data](#why-is-disk-usage-increasing-despite-lack-of-writes) cause that cache size to increase until it hits its configured limit. For information about manually controlling the cache size, see [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1). ## Why is disk usage increasing despite lack of writes? diff --git a/v1.1/recommended-production-settings.md b/v1.1/recommended-production-settings.md index 00201ddf2cf..0a1780bc6a4 100644 --- a/v1.1/recommended-production-settings.md +++ b/v1.1/recommended-production-settings.md @@ -44,15 +44,18 @@ For details about controlling the number and location of replicas, see [Configur {% include faq/clock-synchronization.html %} -## Cache Size +## Cache and SQL Memory Size Changed in v1.1 -If you run multiple applications on the same machine as a CockroachDB node, you might consider manually setting the cache size instead of using the default 25% of available memory. +By default, each node's cache size and temporary SQL memory size is `128MiB` respectively. These defaults were chosen to facilitate development and testing, where users are likely to run multiple CockroachDB nodes on a single computer. When running a production cluster with one node per host, however, it's recommended to increase these values: -To manually set the limit of the cache size, start the node using the [`--cache` flag](start-a-node.html#flags). For example, the following command limits a node's cache to 5GB: +- Increasing a node's **cache size** will improve the node's read performance. +- Increasing a node's **SQL memory size** will increase the number of simultaneous client connections it allows (the `128MiB` default allows a maximum of 6200 simultaneous connections) as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. -```shell -$ cockroach start --cache=5GB -``` +To manually increase a node's cache size and SQL memory size, start the node using the [`--cache`](start-a-node.html#flags) and [`--max-sql-memory`](start-a-node.html#flags) flags: + +~~~ shell +$ cockroach start --cache=25% --max-sql-memory=25% +~~~ ## File Descriptors Limit diff --git a/v1.1/secure-a-cluster.md b/v1.1/secure-a-cluster.md index 8a969a608cf..b0651a31114 100644 --- a/v1.1/secure-a-cluster.md +++ b/v1.1/secure-a-cluster.md @@ -87,8 +87,6 @@ This command starts a node in secure mode, accepting most [`cockroach start`](st - Node data is stored in the `cockroach-data` directory. - The [standard output](start-a-node.html#standard-output) gives you helpful details such as the CockroachDB version, the URL for the admin UI, and the SQL URL for clients. -{{site.data.alerts.callout_success}}By default, each node's cache is limited to 25% of available memory. This default is reasonable when running one node per host. When you run multiple nodes on a single host, however, this default may lead to out-of-memory errors, especially if you test in a serious way. To avoid such errors, you can limit each node's cache size by setting the --cache flag in the start command.{{site.data.alerts.end}} - ## Step 3. Add nodes to the cluster At this point, your cluster is live and operational. With just one node, you can already connect a SQL client and start building out your database. In real deployments, however, you'll always want 3 or more nodes to take advantage of CockroachDB's [automatic replication](demo-data-replication.html), [rebalancing](demo-automatic-rebalancing.html), and [fault tolerance](demo-fault-tolerance-and-recovery.html) capabilities. This step helps you simulate a real deployment locally. diff --git a/v1.1/start-a-local-cluster-in-docker.md b/v1.1/start-a-local-cluster-in-docker.md index f6ffbc98bed..0edcf12a31a 100644 --- a/v1.1/start-a-local-cluster-in-docker.md +++ b/v1.1/start-a-local-cluster-in-docker.md @@ -66,8 +66,6 @@ This command creates a container and starts the first CockroachDB node inside it - `-v "//c/Users//cockroach-data/roach1:/cockroach/cockroach-data"`: This flag mounts a host directory as a data volume. This means that data and logs for this node will be stored in `Users//cockroach-data/roach1` on the host and will persist after the container is stopped or deleted. For more details, see Docker's Mount a host directory as a data volume topic. - `cockroachdb/cockroach:{{page.release_info.version}} start --insecure`: The CockroachDB command to [start a node](start-a-node.html) in the container in insecure mode. - {{site.data.alerts.callout_success}}By default, each node's cache is limited to 25% of available memory. This default is reasonable when running one container/node per host. When running multiple containers/nodes on a single host, however, it may lead to out of memory errors, especially when testing against the cluster in a serious way. To avoid such errors, you can manually limit each node's cache size by setting the --cache flag in the start command.{{site.data.alerts.end}} - ## Step 3. Add nodes to the cluster At this point, your cluster is live and operational. With just one node, you can already connect a SQL client and start building out your database. In real deployments, however, you'll always want 3 or more nodes to take advantage of CockroachDB's [automatic replication](demo-data-replication.html), [rebalancing](demo-automatic-rebalancing.html), and [fault tolerance](demo-fault-tolerance-and-recovery.html) capabilities. diff --git a/v1.1/start-a-local-cluster.md b/v1.1/start-a-local-cluster.md index 9ec5b26821f..d04d89317b8 100644 --- a/v1.1/start-a-local-cluster.md +++ b/v1.1/start-a-local-cluster.md @@ -54,8 +54,6 @@ This command starts a node in insecure mode, accepting most [`cockroach start`]( - Node data is stored in the `cockroach-data` directory. - The [standard output](start-a-node.html#standard-output) gives you helpful details such as the CockroachDB version, the URL for the admin UI, and the SQL URL for clients. -{{site.data.alerts.callout_success}}By default, each node's cache is limited to 25% of available memory. This default is reasonable when running one node per host. When you run multiple nodes on a single host, however, this default may lead to out-of-memory errors, especially if you test in a serious way. To avoid such errors, you can limit each node's cache size by setting the --cache flag in the start command.{{site.data.alerts.end}} - ## Step 2. Add nodes to the cluster At this point, your cluster is live and operational. With just one node, you can already connect a SQL client and start building out your database. In real deployments, however, you'll always want 3 or more nodes to take advantage of CockroachDB's [automatic replication](demo-data-replication.html), [rebalancing](demo-automatic-rebalancing.html), and [fault tolerance](demo-fault-tolerance-and-recovery.html) capabilities. This step helps you simulate a real deployment locally. diff --git a/v1.1/start-a-node.md b/v1.1/start-a-node.md index e32b275d250..800635af925 100644 --- a/v1.1/start-a-node.md +++ b/v1.1/start-a-node.md @@ -36,7 +36,7 @@ Flag | Description `--advertise-host` | The hostname or IP address to advertise to other CockroachDB nodes. If it is a hostname, it must be resolvable from all nodes; if it is an IP address, it must be routable from all nodes.

When this flag is not set, the node advertises the address in the `--host` flag. `--attrs` | Arbitray strings, separated by colons, specifying node capability, which might include specialized hardware or number of cores, for example:

`--attrs=ram:64gb`

These can be used to influence the location of data replicas. See [Configure Replication Zones](configure-replication-zones.html#replication-constraints) for full details. `--background` | Set this to start the node in the background. This is better than appending `&` to the command because control is returned to the shell only once the node is ready to accept requests. -`--cache` | The total size for caches, shared evenly if there are multiple storage devices. This can be in any bytes-based unit, for example:

`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

**Default:** 25% of total system memory (excluding swap), or 512MiB if the memory size cannot be determined +`--cache` | The total size for caches, shared evenly if there are multiple storage devices. This can be a percentage or any bytes-based unit, for example:

`--cache=25%`
`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

Changed in v1.1: **Default:** `128MiB`

The default cache size is reasonable for local development clusters. For production deployments, this should be increased to 25% or higher. See [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details. `--certs-dir` | The path to the [certificate directory](create-security-certificates.html). The directory must contain valid certificates if running in secure mode.

**Default:** `${HOME}/.cockroach-certs/` `--host` | The hostname or IP address to listen on for intra-cluster and client communication. The node will also advertise itself to other nodes using this address only if `--advertise-host` is not specified; in this case, if it is a hostname, it must be resolvable from all nodes, and if it is an IP address, it must be routable from all nodes.

**Default:** Listen on all interfaces, but this flag can be set to listen on an external address `--http-host` | The hostname or IP address to listen on for Admin UI HTTP requests.

**Default:** same as `--host` @@ -45,8 +45,8 @@ Flag | Description `--join`
`-j` | The address for connecting the node to an existing cluster. When starting the first node, leave this flag out. When starting subsequent nodes, set this flag to the address of any existing node.

Optionally, you can specify the addresses of multiple existing nodes as a comma-separated list, using multiple `--join` flags, or using a combination of these approaches, for example:

`--join=localhost:1234,localhost:2345`
`--join=localhost:1234 --join=localhost:2345`
`--join=localhost:1234,localhost:2345 --join=localhost:3456` `--listening-url-file` | The file to which the node's SQL connection URL will be written on successful startup, in addition to being printed to the [standard output](#standard-output).

This is particularly helpful in identifying the node's port when an unused port is assigned automatically (`--port=0`). `--locality` | Arbitrary key-value pairs that describe the locality of the node. Locality might include country, region, datacenter, rack, etc.

CockroachDB attempts to spread replicas evenly across the cluster based on locality, with the order determining the priority. The keys themselves and the order of key-value pairs must be the same on all nodes, for example:

`--locality=region=east,datacenter=us-east-1`
`--locality=region=west,datacenter=us-west-1`

These can be used to influence the location of data replicas. See [Configure Replication Zones](configure-replication-zones.html#replication-constraints) for full details. -`--max-offset` | The maximum allowed clock offset for the cluster. If observed clock offsets exceed this limit, servers will crash to minimize the likelihood of reading inconsistent data. Increasing this value will increase the time to recovery of failures as well as the frequency of uncertainty-based read restarts.

Note that this value must be the same on all nodes in the cluster and cannot be changed with a [rolling upgrade](upgrade-cockroach-version.html#perform-a-rolling-upgrade). In order to change it, first stop every node in the cluster. Then once the entire cluster is offline, restart each node with the new value.

**Default:** 500ms -`--max-sql-memory` | The total size for storage of temporary data for SQL clients, including prepared queries and intermediate data rows during query execution. This can be in any bytes-based unit, for example:

`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

**Default:** 25% of total system memory (excluding swap), or 512MiB if the memory size cannot be determined +`--max-offset` | The maximum allowed clock offset for the cluster. If observed clock offsets exceed this limit, servers will crash to minimize the likelihood of reading inconsistent data. Increasing this value will increase the time to recovery of failures as well as the frequency of uncertainty-based read restarts.

Note that this value must be the same on all nodes in the cluster and cannot be changed with a [rolling upgrade](upgrade-cockroach-version.html#perform-a-rolling-upgrade). In order to change it, first stop every node in the cluster. Then once the entire cluster is offline, restart each node with the new value.

**Default:** `500ms` +`--max-sql-memory` | The total size for storage of temporary data for SQL clients, including prepared queries and intermediate data rows during query execution. This can be a percentage or any bytes-based unit, for example:

`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

Changed in v1.1: **Default:** `128MiB`

The default SQL memory size is reasonable for local development clusters. For production deployments, this should be increased to 25% or higher. See [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size-changed-in-v1-1) for more details. `--pid-file` | The file to which the node's process ID will be written on successful startup. When this flag is not set, the process ID is not written to file. `--port`
`-p` | The port to bind to for internal and client communication.

To have an unused port assigned automatically, pass `--port=0`.

**Env Variable:** `COCKROACH_PORT`
**Default:** `26257` `--store`
`-s` | The file path to a storage device and, optionally, store attributes and maximum size. When using multiple storage devices for a node, this flag must be specified separately for each device, for example:

`--store=/mnt/ssd01 --store=/mnt/ssd02`

For more details, see [`store`](#store) below. @@ -116,43 +116,114 @@ Field | Description ### Start a local cluster -This example demonstrates starting up three nodes locally. See [Start a Local Cluster (Insecure)](start-a-local-cluster.html) and [Start a Local Cluster (Secure)](secure-a-cluster.html) for a detailed walkthrough. +
+ + +
+This example demonstrates starting up three nodes locally. See [Start a Local Cluster](secure-a-cluster.html) for a detailed walkthrough. + +
+~~~ shell +$ cockroach start \ +--certs-dir=certs \ +--host=localhost + +$ cockroach start \ +--certs-dir=certs \ +--store=node2 \ +--host=localhost \ +--port=26258 \ +--http-port=8081 \ +--join=localhost:26257 + +$ cockroach start \ +--certs-dir=certs \ +--store=node3 \ +--host=localhost \ +--port=26259 \ +--http-port=8082 \ +--join=localhost:26257 +~~~ +
+ +
~~~ shell -# Insecure: -$ cockroach start --insecure -$ cockroach start --insecure --store=node2 --port=26258 --http-port=8081 --join=localhost:26257 -$ cockroach start --insecure --store=node3 --port=26259 --http-port=8082 --join=localhost:26257 - -# Secure: -$ cockroach start --certs-dir=certs --http-host=localhost --background -$ cockroach start --certs-dir=certs --store=node2 --port=26258 --http-host=localhost --http-port=8081 --join=localhost:26257 -$ cockroach start --certs-dir=certs --store=node3 --port=26259 --http-host=localhost --http-port=8082 --join=localhost:26257 +$ cockroach start --insecure \ +--host=localhost \ + +$ cockroach start --insecure \ +--store=node2 \ +--host=localhost \ +--port=26258 \ +--http-port=8081 \ +--join=localhost:26257 + +$ cockroach start --insecure \ +--store=node3 \ +--host=localhost \ +--port=26259 \ +--http-port=8082 \ +--join=localhost:26257 ~~~ +
### Start a distributed cluster -This example demonstrates starting up three nodes on different machines. Because each is on a different machine, default ports can be used without causing conflict. See [Manual Deployment](manual-deployment.html) for a detailed walkthrough. +
+ + +
+ +This example demonstrates starting up three nodes on different machines. Because each is on a different machine, default ports can be used without causing conflict, but we've increased memory settings to improve read performance and capacity for in-memory SQL processing. See [Manual Deployment](manual-deployment.html) for a detailed walkthrough. + +
+~~~ +$ cockroach start \ +--certs-dir=certs \ +--host= \ +--http-host= \ +--cache=25% \ +--max-sql-memory=25% + +$ cockroach start \ +--certs-dir=certs \ +--host= \ +--http-host= \ +--join=:26257 \ +--cache=25% \ +--max-sql-memory=25% + +$ cockroach start \ +--certs-dir=certs \ +--host= \ +--http-host= \ +--join=:26257 \ +--cache=25% \ +--max-sql-memory=25% +~~~ +
- +
~~~ shell -# Insecure: -$ cockroach start --insecure --host= -$ cockroach start --insecure --host= --join=:26257 -$ cockroach start --insecure --host= --join=:26257 - -# Secure: -$ cockroach start --certs-dir=certs --host= --http-host= -$ cockroach start --certs-dir=certs --host= --http-host= --join=:26257 -$ cockroach start --certs-dir=certs --host= --http-host= --join=:26257 +$ cockroach start --insecure \ +--host= \ +--cache=25% \ +--max-sql-memory=25% + +$ cockroach start --insecure \ +--host= \ +--join=:26257 \ +--cache=25% \ +--max-sql-memory=25% + +$ cockroach start --insecure \ +--host= \ +--join=:26257 \ +--cache=25% \ +--max-sql-memory=25% ~~~ +
## See Also