Repo template and various other fixes.

CMSgov · Jul 26, 2023 · 08b4f02 · 08b4f02
1 parent 8200fc0
commit 08b4f02
Show file tree

Hide file tree

Showing 10 changed files with 720 additions and 43 deletions.
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md
@@ -0,0 +1,15 @@
+## Issue Report
+
+module-name: One line summary of the issue (less than 72 characters)
+
+### Expected behavior
+
+As concisely as possible, describe the expected behavior.
+
+### Actual behavior
+
+As concisely as possible, describe the observed behavior.
+
+### Steps to reproduce the behavior
+
+List all relevant steps to reproduce the observed behavior.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,49 @@
+<!--
+Thank you for sending the PR! We appreciate you spending the time to work on
+these changes.
+
+Help us understand your motivation by explaining why you decided to make this change.
+
+Happy contributing!
+
+- Comments should be formatted to a width no greater than 80 columns.
+
+- Files should be exempt of trailing spaces.
+
+- We adhere to a specific format for commit messages. Please write your commit
+messages along these guidelines. Please keep the line width no greater than 80
+columns (You can use `fmt -n -p -w 80` to accomplish this).
+
+
+-->
+
+## module-name: One line description of your change (less than 72 characters)
+
+## Problem
+
+Explain the context and why you're making that change. What is the problem
+you're trying to solve? In some cases there is not a problem and this can be
+thought of being the motivation for your change.
+
+## Solution
+
+Describe the modifications you've done.
+
+## Result
+
+What will change as a result of your pull request? Note that sometimes this
+section is unnecessary because it is self-explanatory based on the solution.
+
+Some important notes regarding the summary line:
+
+* Describe what was done; not the result 
+* Use the active voice 
+* Use the present tense 
+* Capitalize properly 
+* Do not end in a period — this is a title/subject 
+* Prefix the subject with its scope
+
+## Test Plan
+
+(Write your test plan here. If you changed any code, please provide us with
+clear instructions on how you verified your changes work.)
diff --git a/.gitignore b/.gitignore
@@ -1,4 +1,4 @@
-history.ndjson
-gource.log
+*.ndjson
+*.log
 *.swp
-.DS_Store
+.DS_Store
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,56 @@
+# How to Contribute
+
+We're so thankful you're considering contributing to an [open source project of
+the U.S. government](https://code.gov/)! If you're unsure about anything, just
+ask -- or submit the issue or pull request anyway. The worst that can happen is
+you'll be politely asked to change something. We appreciate all friendly
+contributions.
+
+We encourage you to read this project's CONTRIBUTING policy (you are here), its
+[LICENSE](LICENSE.md), and its [README](README.md).
+
+## Getting Started
+
+### Coding Style and Linters
+
+If making changes, please format code with [Prettier](https://prettier.io) using the default settings.
+
+## Policies
+
+### Open Source Policy
+
+We adhere to the [CMS Open Source
+Policy](https://github.com/CMSGov/cms-open-source-policy). If you have any
+questions, just [shoot us an email](mailto:opensource@cms.hhs.gov).
+
+### Security and Responsible Disclosure Policy
+
+The Centers for Medicare & Medicaid Services is committed to ensuring the
+security of the American public by protecting their information from
+unwarranted disclosure. We want security researchers to feel comfortable
+reporting vulnerabilities they have discovered so we can fix them and keep our
+users safe. We developed our disclosure policy to reflect our values and uphold
+our sense of responsibility to security researchers who share their expertise
+with us in good faith.
+
+_Submit a vulnerability:_ Unfortunately, we cannot accept secure submissions via
+email or via GitHub Issues. Please use our website to submit vulnerabilities at
+[https://hhs.responsibledisclosure.com](https://hhs.responsibledisclosure.com).
+HHS maintains an acknowledgements page to recognize your efforts on behalf of
+the American public, but you are also welcome to submit anonymously.
+
+Review the HHS Disclosure Policy and websites in scope:
+[https://www.hhs.gov/vulnerability-disclosure-policy/index.html](https://www.hhs.gov/vulnerability-disclosure-policy/index.html).
+
+This policy describes _what systems and types of research_ are covered under this
+policy, _how to send_ us vulnerability reports, and _how long_ we ask security
+researchers to wait before publicly disclosing vulnerabilities.
+
+If you have other cybersecurity related questions, please contact us at
+[csirc@hhs.gov](mailto:csirc@hhs.gov).
+
+## Public domain
+
+This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
+
+All contributions to this project will be released under the CC0 dedication. By submitting a pull request or issue, you are agreeing to comply with this waiver of copyright interest.
diff --git a/LICENSE.md b/LICENSE.md
@@ -0,0 +1,34 @@
+# License
+
+As a work of the [United States government](https://www.usa.gov/), this project
+is in the public domain within the United States of America.
+
+Additionally, we waive copyright and related rights in the work worldwide
+through the CC0 1.0 Universal public domain dedication.
+
+## CC0 1.0 Universal Summary
+
+This is a human-readable summary of the [Legal Code (read the full
+text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode).
+
+### No Copyright
+
+The person who associated a work with this deed has dedicated the work to the
+public domain by waiving all of their rights to the work worldwide under
+copyright law, including all related and neighboring rights, to the extent
+allowed by law.
+
+You can copy, modify, distribute, and perform the work, even for commercial
+purposes, all without asking permission.
+
+### Other Information
+
+In no way are the patent or trademark rights of any person affected by CC0, nor
+are the rights that other persons may have in the work or in how the work is
+used, such as publicity or privacy rights.
+
+Unless expressly stated otherwise, the person who associated a work with this
+deed makes no warranties about the work, and disclaims liability for all uses
+of the work, to the fullest extent permitted by applicable law. When using or
+citing the work, you should not imply endorsement by the author or the
+affirmer.
diff --git a/README.md b/README.md
@@ -12,13 +12,39 @@ The code here is split between Google Apps Script to get the raw activity data a
 
 0. Make sure you have Node.js and Gource installed. The latter is available as `gource` from Homebrew and most Linux package managers; Windows builds of Gource are also available, but this script hasn't been tested there.
 1. Find the ID of the Google Drive folder you want to visualize. This will look something like `1RmiS97L0x0k_CQRbK8__Vuqt4X7ijWr7`; it can be found by navigating to the folder in the Google Drive web UI and looking at the URL.
-2. Clone this repo locally, and substitute in the folder ID at the top of `get-history.js` and `history-to-gource.mjs`, eg `const root = "items/1RmiS97L0x0k_CQRbK8__Vuqt4X7ijWr7";`
-3. [Create a Google Apps Script project](https://script.google.com/home) and paste in the contents of `get-history.js`.
-4. In "Services" on the left, add "Drive Activity API" and "Peopleapi" (the latter is needed to get human-readable names for editors), keeping the default options.
-5. In Google Apps Script, click `Deploy > Test deployment`. Click the gear next to `Select type`, and select `Web app`. Copy the URL that appears and navigate there. It'll take a while to load, depending on the length of history; for our folder with about 6 months of work, it takes around a minute.
+2. [Create a Google Apps Script project](https://script.google.com/home) and paste in the contents of `get-history.js`, substituting in the folder ID from the previous step at the top.
+3. In "Services" on the left, add "Drive Activity API" and "Peopleapi" (the latter is needed to get human-readable names for editors), keeping the default options.
+4. Click "Run" at the top.
+   - If this is the first time, click through the permission prompt. (The script requires permission to read from Google Drive and Google Drive activity for obvious reason; write to Google Drive so we can store the output somewhere, and "access your contacts" to get the names of other editors.)
+5. When it's done, it'll save a `.ndjson` file to your My Drive folder, and print the download link; download it.
+6. Run `node history-to-gource.mjs RootFolderId path/to/log.ndjson "Your Name" > gource.log`.
+   - The last parameter (your name) is only required if you appear in the activity log; Google Apps Script makes it easy enough to get the names of other editors but surprisingly difficult to get your own, so we ask for it manually.
+7. Run `gource gource.log` (plus whatever other Gource parameters you'd like).
+   - See [Gource's docs](https://github.com/acaudwell/Gource) for a full list of parameters. You'll probably want to set `--seconds-per-day` to something less than the default of 10, and `--max-file-lag` (which limits how long, in seconds, Gource can delay showing an event to make the animation look better) to something close to your `--seconds-per-day` to prevent the animation from getting too far out of sync. `--high-dpi` (Gource v0.53+) is well worth setting if you have a monitor that supports it.
+   - To generate a video (as opposed to viewing the visualization directly in Gource), see [Gource's docs](https://github.com/acaudwell/Gource/wiki/Videos#ffmpeg-using-x264-codec).
 
-   - If this is the first time, click through the Google OAuth prompt.
+## Contributing
 
-6. Save the resulting output as `history.ndjson` in the cloned repo.
-7. Run `node history-to-gource.mjs > gource.log`.
-8. Run `gource gource.log` (plus whatever other Gource parameters you'd like).
+Thank you for considering contributing to an Open Source project of the US
+Government! For more information about our contribution guidelines, see
+[CONTRIBUTING.md](CONTRIBUTING.md)
+
+## Security
+
+For more information about our Security, Vulnerability, and Responsible
+Disclosure Policies, see [SECURITY.md](SECURITY.md).
+
+## Authors and Maintainers
+
+A full list of contributors can be found on [https://github.cms.gov/dsacms/drive2gource/graphs/contributors](https://github.cms.gov/dsacms/drive2gource/graphs/contributors).
+
+## Public domain
+
+This project is licensed within in the public domain within the United States,
+and copyright and related rights in the work worldwide are waived through the
+[CC0 1.0 Universal public domain
+dedication](https://creativecommons.org/publicdomain/zero/1.0/).
+
+All contributions to this project will be released under the CC0 dedication. By
+submitting a pull request or issue, you are agreeing to comply with this waiver
+of copyright interest.
diff --git a/SECURITY.md b/SECURITY.md
@@ -0,0 +1,17 @@
+# Security and Responsible Disclosure Policy
+
+*Submit a vulnerability:* Unfortunately, we cannot accept secure submissions via
+email or via GitHub Issues. Please use our website to submit vulnerabilities at
+[https://hhs.responsibledisclosure.com](https://hhs.responsibledisclosure.com).
+HHS maintains an acknowledgements page to recognize your efforts on behalf of
+the American public, but you are also welcome to submit anonymously.
+
+Review the HHS Disclosure Policy and websites in scope:
+[https://www.hhs.gov/vulnerability-disclosure-policy/index.html](https://www.hhs.gov/vulnerability-disclosure-policy/index.html).
+
+This policy describes *what systems and types of research* are covered under this
+policy, *how to send* us vulnerability reports, and *how long* we ask security
+researchers to wait before publicly disclosing vulnerabilities.
+
+If you have other cybersecurity related questions, please contact us at
+[csirc@hhs.gov.](mailto:csirc@hhs.gov).
diff --git a/get-history.js b/get-history.js
@@ -1,8 +1,10 @@
+// SPDX-License-Identifier: CC0-1.0
+
 // This is meant to be run in Google Apps Script: see README.
 
-const root = "items/YOUR_DIRECTORY_ID";
+const root = "items/YOUR_FOLDER_ID";
 
-function doGet(e) {
+function exportActivity() {
   let ret = DriveActivity.Activity.query({
     pageSize: 100,
     ancestorName: root,
@@ -25,20 +27,53 @@ function doGet(e) {
 
     activities = ret.activities.concat(activities);
 
-    // out += activities.length + "\n";
+    console.log(`Fetched ${activities.length} activities...`);
   }
 
   let users = {};
+  let parents = {};
+
+  console.log(
+    "Activities fetched. Fetching additional data (usernames and file parents)..."
+  );
 
-  activities.forEach((act) => {
+  // at this stage we do a progress report every 10 seconds
+  let lastProgressReport = Date.now();
+
+  activities.forEach((act, i) => {
     const userId = act.actors[0].user.knownUser.personName;
     if (!users[userId]) {
       users[userId] = People.People.get(userId, { personFields: "names" });
     }
-    act.actors[0].user.info = users[userId];
+    if (act.primaryActionDetail.create) {
+      const id = act.targets[0].driveItem.name;
+      const shortId = id.split("/")[1];
+      if (!parents[id]) {
+        const parentIter = DriveApp.getFileById(shortId).getParents();
+        parents[id] = [];
+        while (parentIter.hasNext()) {
+          parents[id].push("items/" + parentIter.next().getId());
+        }
+      }
+      act.targets[0]._d2g_parents = parents[id];
+    }
+
+    act.actors[0].user._d2g_info = users[userId];
+
+    if (lastProgressReport + 1000 * 10 <= Date.now()) {
+      console.log(`Fetched additional data for ${i + 1} activities...`);
+      lastProgressReport = Date.now();
+    }
   });
 
-  return ContentService.createTextOutput(
-    activities.map(JSON.stringify).join("\n")
+  const file = DriveApp.createFile(
+    Utilities.newBlob(
+      activities.map(JSON.stringify).join("\n"),
+      "application/json",
+      `${root.split("/")[1]}-${new Date().toISOString()}.ndjson`
+    )
   );
+
+  console.log("Log generated and saved to Google Drive:");
+  console.log(file.getDownloadUrl());
 }