Consider using s3cme. More complete template Go app repo with image build/publish pipelines, SBOM, and SLSA provenance using OIDC and Artifact Registry.
Template to bootstrap a fully functional, multi-region, REST service on GCP with a developer release pipeline.
List of resources provisioned using this project
- Container registry (GCR) to store images
- Service account to use for image publishing from GitHub to GCR
- Workload identity pool provider for GitHub Action workflows to access GCR without the needing to store the GCP credentials as long-lived GitHub secrets
Cloud Run service provisioned into n-number of regions with:
- Custom identity (service account)
- Custom capacity and autoscaling strategy
- Accessible only via Internal and Load Balancer traffic (i.e. no external access)
- Secret Manager service with sample secret
- Revision config using secret variable
HTTPS load balancer configured with:
- External IP
- SSL certificate
- Cloud DNS service with A record for custom domain
- Cloud Armor service with throttling and Canary CVE policies
- Serverless NEGs for load balancer to point to Cloud Run service in each region
- Service uptime and SSL cert expiration alerts
Go code exposing following sample services:
- Request info - client request, headers and environment variables
- Echo message - simple echo message
Note, values obfuscated and generalized for illustration purposes
{
"request": {
"host": "172.18.0.3:30080",
"id": "2994eb6e-b0dd-11eb-acc7-3e6bed2cd376",
"method": "GET",
"path": "/v1/request",
"protocol": "HTTP/1.1",
"time": "2021-05-09T15:42:29.682509631Z",
"version": "v0.8.11"
},
"headers": {
"accept": "*/*",
"content-type": "application/json",
"user-agent": "curl/7.68.0"
},
"env_vars": {
"HOME": "/home/nonroot",
"HOSTNAME": "restme-86888bf66-9khgb",
...
}
}
Request:
curl -i \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ..." \
http://localhost:8080/v1/echo \
-d '{ "on": $(shell date +%s), "msg": "hello?" }'
Response:
{ "on": 1620438323, "msg": "hello?" }
Good how-to on using terraform with GCP is located here
- terraform CLI
- GCP Project
- gcloud CLI
- Make sure to authenticate to GCP using
gcloud auth application-default login
- Make sure to authenticate to GCP using
- In terminal, from the root of the project, first initialize terraform
Note, this flow uses local terraform state, make sure you do not check that into source control and consider using persistent state provider like GCS
terraform -chdir=infra/1-dev-flow init
- Apply the configuration
When promoted for the
GCP Project ID
, enter your existing project ID (not the name),GitHub Repo
name (in username/repo-name format), and confirm withyes
the terraform displayed plan. Alternatively you can use either the command-line variables or a terraform variable file. More on that here.
terraform -chdir=infra/1-dev-flow apply
- Create GitHub Secrets
The result of the apply
command above will look something like this:
PROJECT_ID = "<id-of-your-project>"
SERVICE_ACCOUNT = "github-action-publisher@<id-of-your-project>.iam.gserviceaccount.com"
IDENTITY_PROVIDER = "projects/<project-number>/locations/global/workloadIdentityPools/github-pool/providers/github-provider"
Navigate to your GitHub project secrets and create the following entries with the values returned by the apply command:
PROJECT_ID
SERVICE_ACCOUNT
IDENTITY_PROVIDER
- Test GitHub Workflow
Now, whenever you create a version tag (v*
) in your repo, GitHub will run the image-on-tag.yaml action which will build, publish, and sign (using cosign) your container image in GCR.
This assumes you have not already created a git tag with this v0.0.1 version
git tag v0.0.1
git push origin v0.0.1
Once the action completes, you should see the new image in GCR (gcr.io/<project_id>/restme:v0.0.1
).
This assumes an already published image in GCR
- Initialize terraform
Note, this flow uses local terraform state, make sure you do not check that into source control and consider using persistent state provider like GCS
The service deployment step uses few new Terraform modules, so start by initializing the deployment.
terraform -chdir=infra/2-service init --upgrade
- Apply the configuration
This deployment will prompt for a lot of variables, you can create variables.tf
with the following entries in infra/2-service
folder to avoid these prompts. Edit these as necessary.
project_id = "your-project-id"
name = "restme"
domain = "your.domain.dev"
regions = ["us-west1", "europe-west1", "asia-east1"]
image = "restme"
secret_version = "latest"
log_level = "info"
alert_email = "you@domain.com"
environment = "prod"
Note, the domain must be something you can control DNS for as you will have to create an
A
entry to point to theIP
in Terraform output for this step.
terraform -chdir=infra/2-service apply -var-file=terraform.tfvars
The result of apply
should be a list of Cloud Run services (URLs) for each one of the regions you deployed to, as well as the external IP and URL on the load balancer by which you can access these services.
Note, you will not be able to access the Cloud Run services directly as their ingress (trigger) is internal and Cloud load balancer only.
cloud_run_services = toset([
"https://restme--asia-east1-fr3j36toba-de.a.run.app",
"https://restme--europe-west1-fr3j36toba-ew.a.run.app",
"https://restme--us-west1-fr3j36toba-uw.a.run.app",
])
external_ip = "x.x.x.x"
external_url = "https://your.domain.dev/"
It will take a few min for the SSL certificate to be provisioned. As soon as the apply
step completes, use the IP in external_ip
to create an A
record in your DNS to point to the domain in external_url
. For example:
Host Name: demo
# the your
portion of your.domain.dev
Type: A
TTL: 60
Data: x.x.x.x
# the actual IP returned by the above step
Note, the SSL cert provisioning in the above action will take a few min.
Assuming everything went OK, you should now be able to test the deployment by using curl
to invoke the address returned by the external_url
output
url https://your.domain.dev/
The sample service included with this repo includes token authentication to prevent just anyone on the internet from invoking your demo service. To create new user/token pair run the make-token script and add the resulting key-value pair to the specific environment configuration (e.g. prod).
tools/make-token demo-user
Will return something similar to this:
"demo-user": "96f83affb9ab923262efdf7922f890ef"
Add that to the auth.tokens
list in configs/dev.json or whatever other configuration file you are using.
Note, this file is used as content for the Secret Manager which is then configured in Cloud Run service. By default this demo uses the
prod
for Cloud Run deployment but you can change it during the deployment when prompted for theenvironment
variable.
To clean up each of these deployments run the following command:
Note, the project itself and the external IP address will not be deleted and all APIs enabled as part of these deployments will stay enabled.
terraform -chdir=infra/2-service destroy
terraform -chdir=infra/1-dev-flow destroy