Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

collect historical download data #2003

Merged
merged 1 commit into from
Jul 4, 2019
Merged

collect historical download data #2003

merged 1 commit into from
Jul 4, 2019

Conversation

garyverhaegen-da
Copy link
Contributor

I'd like to start collecting data around our project.

This PR creates a new GCS bucket to hold that data, and adds, as a simple first step, a job to our existing cron to collect download statistics from the GitHub releases API, which will allow us to take a historical view of our downloads to answer questions like how quickly new versions overtake previous ones.

Moving forward I would like to add CI build times to this bucket too, which will come in a separate PR.

hurryabit
hurryabit approved these changes Jul 4, 2019
View reviewed changes
Copy link
Contributor

@hurryabit hurryabit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Thanks.

@garyverhaegen-da
Copy link
Contributor Author

Terraform apply:

An execution plan has been generated and is shown below.                                                                                                     
Resource actions are indicated with the following symbols:                                                                                                   
  + create                                                                    
                                                                                                                                                             
Terraform will perform the following actions:                                                                                                                
                                                                                                                                                             
  + google_storage_bucket.data                                                                                                                               
      id:                     <computed>                                                                                                                     
      force_destroy:          "false"                                                                                                                        
      labels.%:               "5"                                             
      labels.cost-allocation: "daml-language"                                 
      labels.host-group:      "buildpipeline"                                                                                                                
      labels.infra-owner:     "daml-language"                                 
      labels.managed:         "true"                                          
      labels.target:          "infra"                                         
      location:               "US-EAST4"                                                                                                                     
      name:                   "daml-data"                                     
      project:                "da-dev-gcp-daml-language"        
      self_link:              <computed>                                                                                                                     
      storage_class:          "REGIONAL"                                                                                                                     
      url:                    <computed>                                                                                                                     
                                                                                                                                                             
  + google_storage_bucket_acl.data                                                                                                                           
      id:                     <computed>                                      
      bucket:                 "daml-data"                                                                                                                    
      role_entity.#:          "3"                                                                                     
      role_entity.0:          "OWNER:project-owners-462460456285"                                                     
      role_entity.1:          "OWNER:project-editors-462460456285"                                                    
      role_entity.2:          "READER:project-viewers-462460456285"                                                   
                                                                                                                      
  + google_storage_bucket_iam_member.data                                                                             
      id:                     <computed>                                                                              
      bucket:                 "daml-data"                                                                             
      etag:                   <computed>                                                                                                                                                                                                     
      member:                 "serviceAccount:daml-ci-writer@da-dev-gcp-daml-language.iam.gserviceaccount.com"                                                                                                                               
      role:                   "roles/storage.objectAdmin"                                                             

                                                                                                                      
Plan: 3 to add, 0 to change, 0 to destroy.                                                                            
                                                           
Do you want to perform these actions?                                                                                 
  Terraform will perform the actions described above.                                                                 
  Only 'yes' will be accepted to approve.                                                                             
                                                           
  Enter a value: yes                                       
                                                                                                                      
google_storage_bucket.data: Creating...                                                                               
  force_destroy:          "" => "false"                                                                               
  labels.%:               "" => "5"                                                                                   
  labels.cost-allocation: "" => "daml-language"                                                                       
  labels.host-group:      "" => "buildpipeline"                                                                       
  labels.infra-owner:     "" => "daml-language"                                                                       
  labels.managed:         "" => "true"                                                                                
  labels.target:          "" => "infra"                                                                               
  location:               "" => "US-EAST4"                                                                            
  name:                   "" => "daml-data"                                                                           
  project:                "" => "da-dev-gcp-daml-language"                                                            
  self_link:              "" => "<computed>"                                                                          
  storage_class:          "" => "REGIONAL"                                                                            
  url:                    "" => "<computed>"                                                                          
google_storage_bucket.data: Creation complete after 1s (ID: daml-data)                                                
google_storage_bucket_iam_member.data: Creating...                                                                    
  bucket: "" => "daml-data"                                
  etag:   "" => "<computed>"                                                                                                                                                                                                                 
  member: "" => "serviceAccount:daml-ci-writer@da-dev-gcp-daml-language.iam.gserviceaccount.com"                                                                                                                                             
  role:   "" => "roles/storage.objectAdmin"                                                                           
google_storage_bucket_acl.data: Creating...                                                                           
  bucket:        "" => "daml-data"                         
  role_entity.#: "" => "3"                                                                                            
  role_entity.0: "" => "OWNER:project-owners-462460456285"                                                            
  role_entity.1: "" => "OWNER:project-editors-462460456285"                                                           
  role_entity.2: "" => "READER:project-viewers-462460456285"                                                                                                                                                                                 
google_storage_bucket_acl.data: Creation complete after 0s (ID: daml-data-acl)                                                                                                                                                               
google_storage_bucket_iam_member.data: Creation complete after 5s (ID: daml-data/roles/storage.objectAdmin/ser...-daml-language.iam.gserviceaccount.com)                                                                                     
                                                                                                                      
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.                                                           
Releasing state lock. This may take a few moments...                                                                  
                                                           
Outputs:                                                   
                                                           
bazel_cache_ip = 35.190.28.114                             
hoogle_address = 34.98.122.184                             
nix_cache_ip = 35.186.227.171

@mergify mergify bot merged commit 3607047 into master Jul 4, 2019
@mergify mergify bot deleted the data-gcs-bucket branch July 4, 2019 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hurryabit hurryabit hurryabit approved these changes

Assignees
No one assigned
Labels
Standard-Change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@garyverhaegen-da @hurryabit