DEV Community: Jody Heavener

A tip on using peer dependencies with TypeScript

Jody Heavener — Tue, 30 Aug 2022 04:00:05 +0000

I encountered this issue recently and felt pretty silly when I realized the simple mistake I was making, so allow me to share, in hopes that it saves someone else time...

When you're developing an NPM package that makes use of a dependency also used by the main application, you might consider listing it as a peer dependency.

Take this example, where we move React from "dependencies" to "peerDependencies":

-  "dependencies": {
-    "react": "^16.8.4 || ^17.0.0"
-  },
   "devDependencies": {
     "@docusaurus/theme-classic": "^2.0.1",
     "@types/react": "^18.0.17"
+  },
+  "peerDependencies": {
+    "react": "^16.8.4 || ^17.0.0"
   }

React is now a peer dependency, which means the main application needs to list it in its own dependencies. Additionally, we're able to keep developing this package with no issues from TypeScript (can you see why?).

Now notice that other package, @docusaurus/theme-classic. I wanted to make this one a peer dependency as well, so I did just that:

   "devDependencies": {
-    "@docusaurus/theme-classic": "^2.0.1",
     "@types/react": "^18.0.17"
   },
   "peerDependencies": {
+    "@docusaurus/theme-classic": "^2.0.1",
     "react": "^16.8.4 || ^17.0.0"
   }
 }

But after I made this change, TypeScript wasn't happy. 😔 When I tried importing from that module I got the typical "Cannot find module or its corresponding type declarations" error. I spent quite a while scratching my head, trying to understand peer dependencies. I knew package manager CLIs don't automatically install peer dependencies, but I couldn't figure out why other packages, such as React, were working while this one wasn't.

And this is where I felt silly after figuring it out: the @docusaurus/theme-classic package was supplying its own type declarations, so moving it over to peer dependencies was eliminating its types altogether.

To address this, the simplest solution I've found is to duplicate that dependency over to "devDependencies". Doing this makes sure that it is installed locally while you develop the package, while also maintaining its status as a peer dependency when the main application consumes it.

   "devDependencies": {
+    "@docusaurus/theme-classic": "^2.0.1",
     "@types/react": "^18.0.17"
   },
   "peerDependencies": {

I've also tried playing with the install-peers package, that claims to install all your peer dependencies as dev dependencies, but wasn't having much success with it.

If you have your own solution for this problem, I'd love to hear it!

Brick by brick: why Docusaurus is a powerful documentation framework

Jody Heavener — Wed, 06 Jul 2022 15:33:34 +0000

At 2022’s AGConf (1Password’s annual employee conference), every employee received a goodie box to celebrate the event and the company’s successes over the past year. Our theme this year was “space”, so the goodie box included a kit for a Lego rocket ship (very appropriate considering our own CEO is a Lego aficionado).

Building the spaceship brought me back to when I was younger and played endlessly with those little bricks.

For me, though, it wasn’t so much about building the specific items in a kit. Sure, I absolutely loved putting together the houses and planes and cars, but what I was most fascinated by was how I could use tiny bricks to expand my creation and build anything I could dream up. The possibilities were endless, my imagination ran wild, and sometimes – usually through through dumb luck – I built something way cooler than what the kit offered in the first place.

Late last year, I started exploring the React-based documentation framework Docusaurus, and spent a good chunk of time going through the documentation. (Surprise! They use their own product!) I got pretty familiar with how it works under the hood, and the ways in which it can be expanded on. It's also got a bustling community, which is unsurprising since it’s entirely open source.

When I joined 1Password earlier this year, where I would be driving the effort to stand up a developer portal for our new developer offerings, I was excited to learn that we’d chosen Docusaurus v2 as the framework to power it all. I’ve had a chance to really dig in since then, learning as much as I could about this powerful little static site generator.

And it occurred to me recently that, with the way they’ve set it up, I’m reminded of those Lego creations: at its core it’s really just a bunch of individual pieces cleverly interlocked to create something far greater. It’s also built on a foundation designed to be entirely extensible.

So I’d like to look at how Docusaurus is put together, and why it’s so great for the 1Password developer portal.

Plugins all the way down

Plugins are the building blocks of features in a Docusaurus 2 site. Each plugin handles its own individual feature.

Docusaurus has handy plugin lifecycle APIs. When you start up the development server or generate a static bundle, each plugin kicks in and traverses through every stage of the lifecycle. With it, you can pull in data across all plugins simultaneously, register routes, validate configuration, and inject HTML tags, among many other things. Docusaurus leverages these same APIs to build up the overall user-facing functionality of the framework through their own collection of plugins.

Consider the primary use case for Docusaurus: documentation. The @docusaurus/plugin-content-docs plugin powers this central feature for the framework. Its more immediate functionality comes from using the loadContent method to look for potentially localized and versioned sets of documentation on the filesystem, and contentLoaded to provide the structured route data for the core to register and produce HTML files. It also extends Docusaurus’ CLI to allow for tagging a new docs version, and even tells the dev server which files to watch, and in turn run the lifecycles again.

The documentation plugin is obviously a huge part of Docusaurus, but they don’t stop there. Everything from the docs, to blogging and individual pages, all the way down to setting up Google Analytics and generating sitemaps are all powered by plugins.

So, why is this important?

If you’ll allow me to borrow my Lego analogy again: Docusaurus’ plugin APIs mean that, while they provide you with a kit you can set up and build something really cool with, they’ve also provided you with the ability to extend the framework in any direction to build something to suit your exact needs (at least as far as static sites go).

Great examples of this can be found on their community plugins page, where others have built plugins for offline/local search (we even use this today), adding SASS styles loading, and consuming OpenAPI specs to generate full API documentation pages. And it couldn’t be easier to roll your own.

Let’s say you wanted to load in some Google Fonts. Here’s what a plugin that does this by using the injectHtmlTags method might look like:

module.exports = function pluginGoogleFonts(context, options) {
  return {
    name: "plugin-google-fonts",

    injectHtmlTags: () => ({
    // Tell the browser we're going to be loading resources from these origins
      headTags: [
        {
          tagName: "link",
          attributes: {
            rel: "preconnect",
            href: "https://fonts.googleapis.com",
          },
        },
        {
          tagName: "link",
          attributes: {
            rel: "preconnect",
            href: "https://fonts.gstatic.com",
            crossorigin: "anonymous",
          },
        },
        // Load the Lobster font
        {
          tagName: "link",
          attributes: {
            rel: "stylesheet",
            href: "https://fonts.googleapis.com/css2?family=Lobster&display=swap",
          },
        },
      ],
    })
  }
};

With this plugin in place, you can now freely use the Lobster font in your CSS. If you wanted to take it a step further and package this plugin up for distribution, you could even allow it to take an array of font names and weights as options to make it truly dynamic.

In the future, as we expand our developer portal, you’re likely to see us build plugins for things like importing and rendering developer blog posts, highlighting integrations built by our developer community, and a whole lot more.

Need to customize it? Swizzle away.

Plugins aren’t limited to just extending functionality, either. They’re what also delivers the look of the framework. Using the getThemePath method your plugin can tell Docusaurus where to find the React components that make up a theme, selectively overriding components from an existing theme or building your own theme from the ground up.

One of the neatest features of Docusaurus is the ability to Swizzle a component.

[Swizzling] comes from Objective-C and Swift-UI: method swizzling is the process of changing the implementation of an existing selector (method). For Docusaurus, component swizzling means providing an alternative component that takes precedence over the component provided by the theme.

What does this mean in practice? Well, our developer portal currently uses the default Classic theme, but if you check out our footer you’ll notice that it looks nothing like the footer in that theme. We wanted ours to share a consistent look with the one on 1Password.com, so we swizzled the existing Footer component by running the following command:

npm run swizzle @docusaurus/theme-classic Footer -- --eject

This cloned the component out of the Docusaurus package and into our workspace. Now we've got full agency over the look and feel of the site’s footer, while still being able to rely on the rest of the theme’s components, which also includes future updates. We’re going to be swizzling a fair bit this year as the developer portal evolves.

The framework ships with the Classic theme, and out of the box it does a fantastic job. As of April 2022 the theme selection is fairly limited for v2 of Docusaurus, with only the Classic theme and some extensions to it available. More are coming, though. One that I’m particularly looking forward to, a Tailwind-powered theme, is also a great example of why I appreciate that they’re an open source project: it started as a community request, grew in popularity, and over time evolved into part of the roadmap.

Markup or Markdown - how about both?

As with every static site generator, it’s expected that Docusaurus would support Markdown - and they took it a step further, using MDX to parse content. MDX allows you to write JSX (React components) alongside your Markdown, allowing seamless native integration with the rest of the React app, which eventually gets all compiled down to HTML. This concept of static site generators interlacing Markdown with another syntax to extend the capabilities of its documentation is not new, but what gets me excited is the power that JSX affords us. You’re not limited to static HTML or shortcodes. Instead you get the full power of JSX components, meaning it’s possible to build fully styled, rich components that you can embed right in your content.

MDX also supports Remark and Rehype plugins, allowing you to augment the syntax and replace content on the fly. What can we do with this? Docusaurus demonstrates this well by creating its own plugins for admonitions, table of contents generation, and creating heading links.

There’s already a huge collection of plugins available for both Remark and Rehype, but if you need something a little more tailored to your specific use case creating these types of plugins is really straightforward, too. Consider this 13-liner that defaults Markdown code blocks to using Shell highlighting:

const visit = require("unist-util-visit");

module.exports = function pluginRemarkShellCode(context, options) {
  return (tree) => {
    visit(tree, (node) => {
      // If the node is a code block, but the language is not set
      if (node.type === "code" && !node.lang) {
        // Set it to Shell
        node.lang = "shell";
      }
    });
  };
};

Using unist-util-visit we can iterate across all nodes and their children to selectively modify the properties or contents of any node that matches our criteria. Now our Markdown files only need to specify language for those code blocks that aren't using Shell.

Fully Open Source

I’ve been heads down in Docusaurus for quite some time now, and it’s proven to be incredibly robust. But beyond the framework itself, I’ve also really appreciated the community behind it. From contributing my own PRs to the core, to getting help from team members themselves and other eager developers in their Discord server, it’s been a pleasure creating with this extraordinary tool.

Go check out the 1Password developer portal, built with Docusaurus. I’m looking forward to showing off the cool things we’ve got planned for it down the road as we use these building blocks to create something really, really cool.

Introducing 1Password for Visual Studio Code

Jody Heavener — Thu, 23 Jun 2022 15:22:22 +0000

In writing software, we’re used to embedding secrets and other configurable values right in the codebase. They might be Stripe keys to power your online shop, webhooks for a custom Slack bot, a Docker username and password for a CI config, AWS credentials, or an API token and host to set up 1Password Connect.

Secrets are used everywhere in our code. Sometimes, though, we forget when we’ve been using real secrets in our work. Maybe there’s a leftover token you dropped in to build that one feature, or maybe you didn’t delete the .env file you set up to test drive the app. Now you’ve got to rotate your secrets because you accidentally committed and pushed sensitive values for the whole world to see. Yikes.

We’ve all been there. That’s why I’m delighted that I get to announce the launch of the all-new 1Password for VS Code extension.

Go ahead, commit your secrets references

With 1Password Secrets Automation, the 1Password Developer Products team introduced the concept of secret references. It starts by storing a sensitive value, such as an API credential or client ID, in 1Password. That item and the field you’d like to get the value from can then be retrieved through a special op:// URL scheme that 1Password’s tooling knows how to parse. It’s made up of three parts: vault, item, and field. This is known as a “secret reference”.

Now, instead of using a real value in your configs, environment variable files, or anywhere else in the codebase, just drop in the secret reference in VS Code. When you do, you can rest easy knowing that the real value will never accidentally make its way into your codebase.

The best part? Through our suite of tools and integrations, you can work with references in both local and deployed environments.

To help make sure you’re not accidentally leaving secrets in your code, you can move them over to 1Password with just a couple clicks. The extension uses a series of secret detection techniques to look for values that might be sensitive. With these matches, it makes inline suggestions to store them in 1Password, automatically replacing them with secret references.

Secret reference integration doesn’t stop there. You can hover a reference to inspect the item and field details, click it to open the item in the desktop app, and even preview the real values of an entire file full of references.

Beyond secret detection suggestions, 1Password for VS Code makes it easy to retrieve items for use in your code, as well as store any bits of code you’d like in 1Password. If you’ve got multiple values you want stored in the same item – perhaps a username, password, and email – it supports that as well. Just select each value and run the “Save in 1Password” command.

Built using tools available to everyone

I’ll let you in on a little secret: we didn’t plan to build this extension. It wasn’t requested by our developer community, and wasn’t part of any roadmap. Instead this extension began as a side project for myself. I wanted to scratch my own itch and integrate 1Password more closely into my development workflow, and to generally learn more about developing with 1Password.

So you can imagine my excitement when, after a quick demo at an internal call, I was invited to polish it up and get it slated for release.

To my delight, after demoing the extension and then going on vacation, Dave posted a video of the presentation from his CLI launch blog post and it was met with some pretty wild enthusiasm from the developer community. There was even some love for it at our 1Password 8 for Mac Reddit AMA:

Although not a goal from the outset, an interesting aspect of this project is that it’s built using only tools available to the public – there’s nothing internal or proprietary powering the features of the extension. We’ve even open-sourced the whole project on our GitHub, so if you want to help iterate on it or report an issue, that’s a great place to start.

VS Code extensions run in a Node environment, and we wanted to interact with the new CLI. So we built and open-sourced an entirely new package for doing exactly this: op-js. It wraps the CLI with a simple-to-use JavaScript interface and ships with TypeScript declarations, making 60+ commands, including those that support biometrics unlock, available to your Node-based application.

Ultimately my hope is that this extension demonstrates some of the neat ways you can extend the power of 1Password by building your own integrations, whether it be for yourself or others. And you should have fun doing it. We’re in early days here, with plenty more developer offerings coming down the line.

I’d love to hear what you think, and we’ll be iterating on the extension as feedback rolls in. Learn more about 1Password for VS Code and our other developer tools by checking out our developer portal. While you’re there, consider joining our Developer Slack workspace, where you’ll find myself and others on the Developer Products team who are eager to hear how you’re incorporating 1Password into your development workflow. And if you’re building something cool, be sure to tag it #BuildWith1Password!

Finally, we owe a tremendous debt of gratitude to Mike Selander, Chris Dunn-Birch, Floris van der Grinten, the incredibly helpful folks over in the VS Code Extension community, and so many more for providing endless help and guidance while working on this project. Thank you!

Building a browser extension for PM2

Jody Heavener — Mon, 14 Jun 2021 00:03:21 +0000

Today I'm launching PM2 DevTools, a browser extension that allows you to manage PM2 processes and logs from your browser. This has been a fun side project for me, and while I realize the demographic for a tool like this is likely pretty narrow I'm still excited to show it off.

PM2 DevTools was born out of a desire to interact and do something with the logs FxA produced without needing to leave the browser. Specifically, when you sign up for a new account you need to enter an account verification code in the browser, and locally this is printed in one of the process logs.

PM2 DevTools will allow me to now script my way through this flow -- from CLI output to browser input.

Pretty specific use-case, right? Maybe, but depending on what your PM2 logs produce and your imagination I'm sure there are a bunch of things you could automate.

You can go download it from Firefox Add-ons right now and try it out. I even put together a little example PM2 project you can use.

Manage processes, observe logs, automate page actions

As soon as you run PM2 DevTools it starts a background script that attempts to connect to the PM2 WebSocket server. The WebSocket is responsible for delivering all the data used by the extension, such as listing processes and streaming log data. Once that connection is made it is maintained until the browser is closed. If configured, logs may start flowing in and the script evaluates each log event against any of the tabs you have open with any Log Scripts you have saved (more on this later). Otherwise, the background script lies in wait for a connection to be made from the part of the extension that you'll interact with: the panel.

The main part of PM2 DevTools is, well, the dev tool! This extension adds a new panel to your Developer Tools, and it's where most of the features can be observed and settings managed. When you open it the panel connects to our background script and asks it to retrieve all the PM2 processes, which in turn asks the WebSocket for this information. As you can see the background script acts as a relay between the WebSocket and Developer Tools panel.

The extension supports both dark and light modes. Documentation on all the settings can be found here.

Simple process-management UI

With this extension all your processes are available in a simple sidebar UI. This UI is kept up-to-date at all times via the WebSocket, reflecting new processes, as well as changes to their status. Each process cell provides important details like name, identifier, and status.

To help distinguish between large numbers of processes and their logs, each process is also automatically color-coded.

Managing a process is simple - just click the play or pause button in its cell, depending on its status, to turn it on or off. Click the eye icon to exclude its logs from the output window, including search results; this is helpful for services that are a little too chatty.

Inspect, search, and automate with your logs

As you might expect, the main component of PM2 DevTools is the log output, and it's the visual equivelent of pm2 logs in the CLI, with some added functionality.

When the "Start logs" play button is toggled color-coded process logs will stream in via WebSocket connection. As noted earlier, specific processes can be excluded as desired. You'll also see extension-level logs appear here, such as when a connection is made or lost to the WebScoket server, or if an error occurs under the hood.

You can of course clear all logs, as well as search through them. The search field supports process name and log content, and it also treats a string between two forward slashes as a regex query.

Log Scripts

In the bottom right you'll see a script tags button; clicking this will take you to Log Scripts.

Log Scripts are JavaScript snippets that can be executed when PM2 logs are broadcast. When the extension's background script receives a new log event it will check to see if any Log Scripts have matching URLs and then execute the script's code, which will have access to the log's data, directly on the page.

These scripts can be really powerful when flows in the browser rely on output from process logs. If you think back to the scenerio described above, where we want to look up a verification code in process logs and use it in a browser field, we can now automate this with a Log Script:

if (data.name !== "inbox") { return; }

const field = document.querySelector(".otp-code");
const button = document.querySelector("#submit-btn");
const regexp = /^.+Signin code (\d+).+$/;
const match = data.message.match(regexp);
const code = match[1];

if (field && button && code) {
  field.value = code;
  button.click();
}

So what's happening here? Well, it's a function body that's evaluated directly on any page (so be careful!) with a matching URL.

Any time a process log arrives the background script looks up all available Log Scripts, checks if any of the URLs from pages you've got open match (this can also be a glob), and if so executes that Script's function body with the log data passed in as the data object directly on the page. This object looks like:

{
  "message": "some output from the service",
  "timestamp": 1619810341487,
  "pmId": 15,
  "name": "auth-db"
}

With this log script saved now all I have to do is run through the flow locally.

Documentation on setting up Log Scripts can be found here.

Ultimately, as I mentioned, the audience for this is probably pretty small, but I had so much fun diving into the world of browser extensions. I'd love to know what you think, and if you have any suggestions!

Show appreciation for your team’s code with Bonusly for GitHub

Jody Heavener — Fri, 18 Sep 2020 02:50:19 +0000

Have you heard of Bonusly? If not, you’re missing out. It’s a tool “dedicated to building recognition-rich cultures in workplaces across the globe”. One of the core concepts of the tool is to award your teammates with “points”. This could be for a job well done, a birthday, a work anniversary, anything you can think of. These points can then be redeemed for physical and digital goods, gift cards, donations to charity, and more.

I’ve long been a fan, and thought there must be a way to combine Bonusly’s recognition point system with the hard work developers do. That’s why I’m happy to present my submission to the GitHub Actions x DEV Hackathon: Bonusly for GitHub.

My Workflow

You can find and install the Bonusly for GitHub Action from the GitHub Marketplace.

Follow the README Usage instructions to get started, including prerequisites for secrets. An example workflow might look something like this:

name: Bonusly

on:
  pull_request:
    types: [closed]

jobs:
  merge-allocate:
    runs-on: ubuntu-latest
    name: Merge allocate
    steps:
      - uses: actions/checkout@v2
      - uses: jodyheavener/bonusly-github@0.1.0
        if: github.event.pull_request.merged
        with:
          bonusly-token: ${{ secrets.BONUSLY_API_TOKEN }}
          github-token: ${{ secrets.GH_API_TOKEN }}
          default-hashtag: '#awesome'

You’ll notice that this runs only on Pull Request closure, and is further narrowed by the if statement under “steps” to indicate we only want to act on merged Pull Requests.

Once you’re up and running, all that’s left to do is to start awarding points to your co-workers via comments on their PRs! Check out the repo README for supported comment formats, as well as some gotchas that may occur.

Once the Pull Request is merged the Action will run and allocate any Bonusly points to the commit authors of the PR. This even works for co-authored commits!

Done! Points awarded and teammates celebrated for their hard work. 🥂

Submission Category:

Wacky Wildcards

I might even say Maintainer Must-Haves as well, depending on how your team is set up.

Yaml File or Link to Code

jodyheavener / bonusly-github

A GitHub Action for Bonusly

Bonusly for GitHub

Award Bonusly points for successful Pull Request merges.

Usage

First, set the following secrets on your repo:

# A Bonusly API Access Token
# https://bonus.ly/api
BONUSLY_API_TOKEN
# A GitHub Personal Access Token
# https://github.com/settings/tokens/new
GH_API_TOKEN

Then set up a new workflow:

name: Bonusly

on:
  pull_request:
    types: [closed]

jobs:
  merge-allocate:
    runs-on: ubuntu-latest
    name: Merge allocate
    steps:
      - uses: actions/checkout@v2
      - uses: jodyheavener/bonusly-github@0.1.0
        if: github.event.pull_request.merged
        with:
          bonusly-token: ${{ secrets.BONUSLY_API_TOKEN }}
          github-token: ${{ secrets.GH_API_TOKEN }}
          default-hashtag: '#awesome'

Now, anyone who would like to award the commit authors of a merged Pull Request can indicate so by leaving a comment on it in one of the following formats:

@bonusly +[amount] [message]
@bonusly [amount] [message]
@bonusly [amount] points [message]

When the Pull Request is merged the points will be awarded and a comment will…

View on GitHub

Additional Resources / Info

GitHub Actions are a lot of fun. I actually went through a whole lot of ideas before landing on this one. It was totally worth it because Bonusly is a pretty powerful tool for bringing teams together, especially these days when a lot of folks might not be connecting with their teammates as much as they’d like.

Hope you liked this submission. All thoughts and feedback welcome.

CircleCI for VS Code (beta)

Jody Heavener — Sun, 17 May 2020 21:15:22 +0000

🎉 Hey all! I’m excited to announce that I’ve released the first beta version of CircleCI for VS Code.

It’s pretty easy to jump right in if you want to try it out, just install it in VS Code and follow the instructions. I’m going to give a brief rundown and go over what the development experience was like below.

Quick note before I dive in: while I certainly hope they like it, this extension is not built or endorsed by CircleCI.

Highlights

A full breakdown of setup and everything you can do can be found in the Marketplace overview, but here are a few areas I wanted to call out.

Builds for your current branch, master, and more

The extension will always poll for builds on your current Git branch — it’ll automatically update when you change branches. But if you need to you can additionally specify any custom branch name that you’d like to list in the extension view. I personally like to always be watching master.

Customize build counts and re-check intervals

The extension options give you the ability to specify how many builds it should retrieve per pipeline, and, if a build is in an active state, how frequently it should poll CircleCI for updates to it.

Keep an eye out for new builds

Unfortunately we can’t use webhooks to inform the extension of new builds, but we do provide you the option to regularly poll stale pipelines for new builds.

Explore artifacts in VS Code

Each build has the option to look up artifacts generated by the workflow. All you have to do is click the “Look up artifacts” row and they’ll populate below. Click an artifact to view it directly in VS Code.

Retry, cancel, copy details, and more

Most UI rows have context menu options. Pipelines and builds can be opened directly in the browser, but builds can also be retried and canceled directly from the extension. Workflow and commit rows allow you to copy various items to your clipboard.

Development

This was my first exploration into the world of VS Code extensions, and I’ve got to say it was pretty fun. There’s a huge amount of flexibility in what you can build for VS Code, so that paired with excellent documentation and a CLI tool made developing and publishing a breeze.

For this extension I chose to go with a TreeView layout, with each pipeline and its builds rendered in collapsable rows. Using the circleci Node package it was as simple as looking up builds for the current local branch (which, along with the username and repository name, make up a “pipeline”). If a pipeline has running builds, check in on them every so often, and re-render the latest data. Sprinkle in some context menu commands to retry failing builds, cancel running builds, and open everything in the browser, and we’ve got an extension. Things get a little more exciting when we start to look at the additional properties the CircleCI API returns; we can see commit data, the associated workflow, time spent running, and a lot more. I tried to include as many relevant build details as possible, for a UI that is informative but not cluttered.

There were a few areas of note that I particularly enjoyed:

The ability to download a build’s artifact and load it directly into a VS Code window. No need to leave the IDE at all in some cases.
Setting up configuration options was very nice; they’re defined in the package.json, support Markdown descriptions, have field types, and can be retrieved and updated within the extension execution.
Because it’s VS Code, of course everything comes out of the box fully-typed. This made development so much smoother.

What do you think?

That’s all for now. If you end up trying out the extension please do let me know what you think by commenting below, or if you’re running into problems you can either ask here or file an issue. I’d love to improve on it.

Also, a huge thanks to James Van Dyke and his CircleCI extension. It’s a few years old and uses the command palette and status bar for its primary interactions, whereas I wanted a few additional features and a visual UI. It provided for an excellent reference point, though, so please go check it out!

If you’re interested in building your own extension for VS Code, the Extension API is a great way to get started.

Won’t you be my Pal? (Twilio Hackathon)

Jody Heavener — Tue, 28 Apr 2020 04:34:55 +0000

In the never-ending pursuit to Always Be CLeveling up my coding skills I’ve recently been going through some Udemy courses to learn a few new tech-niques (get it?). So when I saw that Twilio and DEV were putting on a hackathon I thought it would be the perfect opportunity to bust out my new skills to build something fun and maybe even a little helpful.

Believe it or not I actually spent the first half of the month working on a completely different idea for a service that would allow you to set up a nearby contact (like a neighbour) to check in on you if you contracted an illness like COVID-19. After a couple weeks I decided it was a little too involved for such a short development window, so I scrapped the idea and went for something a little simpler.

Not all was lost, though, because I was able to move a lot of foundational setup and code to this new idea. So here we are…

What I built

Pals: A simple SMS service that allows you to chat with a random person anonymously.

Category Submission:

COVID-19 Communications

Why did I build it?

When I was in elementary school I had the opportunity to have a pen pal. Actually it was an email pal. But the Power Macintosh G3 wasn’t the fastest and dial-up internet was slow enough that it might as well have been a proper pen pal. Anyway, I remember always looking forward to learning more about my new friend on the other side of the planet. So I figured in these crazy times, in this world where we’re isolated in our homes, maybe we could all use a pal of our own.

What does it do?

It’s simple, really. You just send a text to the service’s phone number. From there it’ll ask you for a name that you want to go by, and then you’ll be paired with someone else: your new Pal.

Once you’re connected any message you send will go directly to your Pal. Except one: text > MENU to access commands that you can use to disconnect and more.

That’s pretty much it. It’s not meant to be complicated, no fancy UI or drawn-out signup process. The beauty in it is that it’s connecting you to someone you would likely never get to learn about otherwise, while at the same time keeping the participants’ phone numbers private.

How I built it

As I mentioned above I was using this hackathon as an opportunity to play around with some stuff I’d be learning on Udemy. I’m currently enrolled in TypeScript course and I recently finished a Serverless course. So with that in mind, here’s a breakdown of the stack:

Twilio’s Programmable SMS to deliver messages
Serverless AWS to provision most things and deploy code
- This included setting up the IAM Role statement, setting up VPC configuration, and employing both API Gateway and CloudWatch Events.
AWS Lambda functions written in Node.js
MySQL database (AWS RDS Aurora)
Sequelize for the ORM
TypeScript on all the things (or at least I tried… sort of)

I guess I should also note this was my first time using Sequelize as well. New things all around! Don’t you dare critique my code! 😁

There’s plenty that could still be done to make it shine, but I wanted to get an MVP in before the deadline.

Finally, I recognize that this probably isn’t a very sustainable model, considering you’re essentially proxying messages between two people. But hey, it was fun to make.

Demo

~~All you need to do is send a text to ███-███-████ and follow the prompts.~~

Uhhh…

Ok, I had this whole post planned and was all excited for everyone to start texting in to meet their new Pals. Everything works as expected when developing and testing locally, but for reasons I am still unsure of I can’t get anything to happen when I execute the Twilio client’s create message function. I’ve spent hours trying to debug. At this point I think it has to do with my VPC/Subnet configuration, but I don’t know enough about it to figure it out just yet. After all, this is my first shot at a Serverless stack. 🙃

If I can figure it out before the deadline, great. If not and it disqualifies the submission, no big deal, I'm still proud of it. Check out the source below if you want to set it up locally and try it for yourself; maybe you'll have better luck than me.

Link to Code

Full source is available on GitHub at jodyheavener/pals.

This was a fun project. I really enjoyed applying some new Serverless knowledge, and at least a little bit of TypeScript. I’d love to hear what you think; what could be added to make Pals even better?

Protecting your frontend with a Content Security Policy

Jody Heavener — Thu, 27 Feb 2020 20:24:35 +0000

Today we’re going to look at how a Content Security Policy can be configured to protect your web pages from Cross-Site Scripting attacks.

A primer on Cross-Site Scripting

Anyone who touches the front end of a web page should have at least a basic understanding of Cross-Site Scripting — and I mean anyone, whether you’re the React-SPA-“CSS in JS is life” front end ninja, the project manager who figured out how to open a PR to add yet another CTA modal, or the backender that just wants to render some simple HTML. If you’re not sure what Cross-Site Scripting (from here on out known as XSS) is, it’s time to get informed.

A very basic breakdown of XSS is that it’s a method with which someone executes unauthorized script(s) on a web page without the web page owners knowing or the user consenting. In some cases this could result in something silly, maybe the words “U’ve been hAcKeD” in big flaming letters across the screen, but it could also look a legitimate part of the page, perhaps by rendering what appears to be an official form that captures your sensitive information. There are a variety of other possible outcomes:

Stealing your cookies and other browser storage data, such as your session cookies.
Attempting to enable revealing browser APIs (geolocation, webcam). Sure you might be prompted, but when was the last time you dismissed a location request from a website?
Keylogging, mouse movement tracking, other session snooping.
Loading additional scripts to perform additional actions (hello crypto miners).

If you’re not entirely new to XSS you might be familiar with the three types of XSS that have long been the convention, Stored, Reflected, and DOM-based. Because these types actually can have quite a bit of overlap we’ve seen a new set of terms for XSS types come to prominence:

Server XSS occurs when someone is able to successful store nefarious data on the server, and when a user makes a request that results in the data being returned from the server and the browser renders it as HTML, allowing scripts to be called. This can be mitigated by sanitizing user output from the server (e.g. never rendering user output as HTML). It wouldn’t hurt to also sanitize it before it hits the server, but if it’s already in there your best bet is to catch it on the way out.

Client XSS occurs when an unsafe JavaScript call is used to update the DOM. The result is that the data being used in the call is treated as safe to execute, which allows scripts to be called. This could be the result of an AJAX call, a value from browser storage, or some other in-DOM method. This can be mitigated by not using unsafe JavaScript (e.g. document.write, el.innerHTML, eval), but you’ll see below why even these measures can be tricky to implement.

Just in case you had any doubts about whether or not XSS is a real problem, have a look at these findings:

The Open Web Application Security Project lists XSS as one of the Top Ten security risks to web applications (as of 2017).
The Edgescan Vulnerability Stats Report 2019 notes “[XSS], both reflected and stored, was the most common vulnerability in 2018 at 14.69%”.
Noted in HackerOne’s The 2019 Hacker Report, “over 38% of hackers surveyed said they prefer searching for cross-site scripting vulnerabilities” when asked about their favourite attack methods and vectors.

How would a Content Security Policy help?

Content-Security-Policy (CSP) is one of the fundamental controls for XSS prevention. You can generally set it up once and it’ll look out for you until the end of days.

Let’s look at it from this angle: XSS is hard to prevent. You don’t really see what’s going on because it can all occur on the client side, doesn’t necessarily have to hit the server, and when it does it might not be so easy to detect. So it helps to have a lot of different security controls. As I noted above, this can include input validation and output encoding.

Output encoding is of course a valid method to help prevent against XSS attacks, but it is not always reliable. Let’s use output that is used in HTML as an example:

<a href="{{ output }}">Some link</a>

In the above example, if output is <script>alert(1)</script> and it is rendered without escaping the HTML characters, the JavaScript would execute. However if you escaped them, this would be fine to print in the HTML. But, what if it’s not an HTML context? What if output is javascript:alert(1), which is another valid form of executing JavaScript in various HTML attributes. Now that we’re in a JavaScript context escaping HTML would do nothing here and the code would run as designed.

This was just one example, but it’s meant to emphasize how important it is to think about the context in which your output is printed and executed. You might be convinced that your user output is covered, or you might not even have user-generated output at this point in your application’s life, but a CSP is something that can help you now (everyone makes mistakes) as well as help future-proof your product.

Enter CSP

In essence a Content Security Policy (CSP) is a set of rules that tell the browser which types of files or content can be loaded on the page and in what contexts, and anything that does not match that criteria will be discarded. This helps prevent XSS payloads from being rendered on the page. If the browser supports CSP it’ll enforce it, if not it doesn’t harm anything.

There are many types of files or content that can be allowed or restricted using a CSP, including scripts, styles, fonts, manifests, media (video, audio), embedded frames, and more.

A CSP can be set two ways. Either by the web server, where it is sent over via an HTTP header, or by a meta tag in the page’s markup. Here’s an example that restricts loading all resources to HTTPS:

// header
Content-Security-Policy: default-src https:

// meta tag
<meta http-equiv="Content-Security-Policy" content="default-src https:">

So think back to the example in the previous section. If we were to set our CSP to default-src 'self'; script-src 'self' and tried to run the above inline JavaScript context, it would fail. Why? Because this policy does not allow inline execution of JavaScript.

If you need help building a custom CSP you can use CSP is Awesome, and if you want to analyze another website’s CSP, check out this tool from Report URI.

Here are some more examples, from the Content Security Policy Reference:

// Allow everything but only from the same origin
default-src 'self';
// Only Allow Scripts from the same origin
script-src 'self';
// Allow Google Analytics, Google AJAX CDN and Same Origin
script-src 'self' www.google-analytics.com ajax.googleapis.com;
// Allow images, scripts, AJAX, and CSS from the same origin, and does not allow any other resources to load.
default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self';

Bypass vector

A common practice with CSP is to include wildcards URLs for common asset hosting origins.

An example, where you might want to load scripts from an AWS S3 bucket:

Content-Security-Policy: default-src 'self'; script-src 'self' *.amazonaws.com

If someone were to discover that your site is vulnerable to XSS, all they would have to do is put their script in an S3 bucket and use that address to load it on your web site.

Using nonces with your CSPs

CSPs support restricting the execution of scripts with a nonce.

If you’re not sure what a nonce is, it’s just a single-use set of characters (number used once) generated by the server with which you can use to verify that a request is legitimate. Often this involves validating the nonce from one request to another on the server, but in this case we're actually going just validate it in the browser.

As an example, let’s say your server generates the nonce value of 123456. You would then set a CSP header with that value:

Content-Security-Policy: default-src 'self'; script-src 'nonce-123456'

Now, when you render a script you would also use this same nonce value as the nonce attribute on the script tag:

<!-- these would be discarded -->
<script>document.write('super haxed');</script>
<script nonce="123457">document.body.innerHTML = '<h1>Got u!!!</h1>';</script>

<!-- this would execute -->
<script nonce="123456">console.log('hello!')</script>

The browser will then check the header value against the script tag value; if they don't match the script will not execute.

Reporting XSS attacks

Not only can a CSP help stop intruders, but it can also narc on them.

The browser that a CSP violation occurs on can send a JSON report (via POST) to a specified URI for you to then analyze. Here’s what this might look like:

Content-Security-Policy: default-src 'self'; report-uri http://example.com/csp-reports

Note this uses the report-to directive, and while a handful of browsers do support this it’s no longer the recommended method. The new directive is report-to, but as of writing it’s only supported by about 66% of browser users, so it’s best to include both directives in your policy.

Wrapping up

I hope this helped shine a light on how important it is to protect against XSS attacks, and how a Content Security Policy can greatly help with this.

If you’re still curious about this topic and want to learn more, here are some fantastic posts from others in the DEV community:

Additionally, here are some of the resources that I used to help develop this post (🙏):

Default to safe force pushing in Git (without aliases)

Jody Heavener — Thu, 27 Feb 2020 14:45:43 +0000

A few days ago I posted some Git tips and tricks, one of which was that you should use --force-with-lease any time you want to use --force.

Why should you do this? Basically --force can be destructive and --force-with-lease is a little more careful before changing history. Let's hear from this Atlassian blog post again:

What --force-with-lease does is refuse to update a branch unless it is the state that we expect; i.e. nobody has updated the branch upstream. In practice this works by checking that the upstream ref is what we expect, because refs are hashes, and implicitly encode the chain of parents into their value.

If you're already familiar with this concept you might be using an alias to invoke --force-with-lease. Mine looks like this:

lease = push --force-with-lease

This allows me to type git lease and have the same effect as git push --force-with-lease. If this works for you then perfect! You don't really need to read any further.

The problem I have is that in a lot of my command line usage, Git and otherwise, I regularly type --force for various reasons. It's muscle memory that I've developed, and I regularly find myself force pushing the wrong way. It's bad and I feel bad. So bad that I spent some time recently working on this quick and dirty (but also kind of neat) solution to short circuit my use of the bad force altogether:

(Writing shell commands is not my strong suit, so if you see ways this can be improved I am all ears.)

This is a shell function I've added to my ZSH profile. Here's a quick breakdown of what it does:

First, it overrides the git command. (I know it's probably not a great idea overriding git, don't email me, do this at your own risk, whatever)
It looks for use of the push. If that isn't found, continue on regularly.
If you're using push and not --force, continue on regularly.
If you are using --force, replace that argument with --force-with-lease.
If you're using --force and you really need it, you can add --seriously to avoid having it replaced with --force-with-lease.
In any case, any other arguments are still applied regularly.

So now we can do the following:

# Regular use of --force-with-lease unaffected
➜  example git:(master) ✗ git push --force-with-lease
Everything up-to-date

# Use of --force overridden 
➜  example git:(master) ✗ git push --force
Detected use of --force! Using --force-with-lease instead. If you're absolutely sure you can override with --force --seriously.
Everything up-to-date

# Use of --force not overridden if --seriously
➜  example git:(master) ✗ git push --force --seriously
Heads up! Using --force and not --force-with-lease.
Everything up-to-date

That's it! Now I can lazily avoid having to change my ways. Would love to hear what you think.

Using Lerna to manage your JavaScript monorepo

Jody Heavener — Wed, 26 Feb 2020 14:34:53 +0000

Let’s dig into Lerna!

Lerna is a handy command line utility that you can use to manage JavaScript projects with multiple packages. It can be handy for both open source and private projects.

This post is intended to be more of a primer, and not a deep dive. That being said, if you have any suggestions to make this post better, or just have a question, please leave a comment below!

What’s a package?

Projects often contain many components; perhaps a front-end, server, maybe some micro services, you get the point. They can be individually worked on, versioned, and released, all while being housed under the same repository, which in turn makes development access and project tracking easier. Think of these standalone components as your packages.

How does Lerna help?

Lerna uses Git and NPM to create an optimal workflow for your many project packages, via the lerna CLI. This could include versioning, distribution, testing, and so much more.

This tool is exciting to me because it removes the hassle of having to manually go through each package of a project to perform mundane operations. On the surface it’s a fairly simple tool, but it can be configured in some pretty neat ways, and used effectively can make working with a monorepo a huge time saver.

Fun fact:

The seven-headed monster you’ll see at the top of the project’s website and repo depict the Greek mythological creature known as the Lernaean Hydra, or Hydra of Lerna. From the Wikipedia article:

The Hydra possessed many heads, the exact number of which varies according to the source. Later versions of the Hydra story add a regeneration feature to the monster: for every head chopped off, the Hydra would regrow two heads.

I find this to be a pretty suitable mascot for the tool. 🐉

Getting set up

As I mentioned above this tool has a few ways in which in can be configured to suit your needs, but in this post we’re just going to get up and running without too much trouble.

Before you go any further, install the command line utility globally:

# via npm
npm install -g lerna
# via yarn
yarn global add lerna

Now you’ll need to decide how and where you’d like to set it up. Lerna can be initialized in a bare directory, or it can be added to an existing project. Either way, run lerna init in your project folder and you’ll get the following additions:

lerna.json
package.json
/packages

If you ran it on a bare directory it’ll run git init for you. Make sure you add a remote so you can push versions up.

If you already had a package.json it won’t overwrite it, but it will add lerna as a dev dependency. Be sure to npm install afterward.

The newly added packages/ dir will serve as the place for all your packages to live. “packages” is just the default. Speaking of which…

Your new lerna.json is the configuration that will be used when performing commands. You can see a full list of options here, but let’s run through a few:

packages This tells Lerna where to find the packages it can operate on. By default this is ["packages/*"], which globs any directory under packages/, hence why it was auto-generated.
version This value tells Lerna which version of the project we’re at. By default this is in “fixed” mode which uses a semver value and auto-increments when you run version commands, but you can optionally change this value to independent if you’d like to manage package versions independent from one another. You can set this automatically by running lerna init with --independent.
npmClient The client to run commands with. Defaults to npm, but you can change it to yarn if you’d like.
command.publish.ignoreChanges This can be an array of globs that Lerna will ignore when looking for changes to generate a new version. Examples could be your README, CHANGELOG, any tests or benchmarks, etc.

And that’s mostly it! From here you can add new packages under packages/. If you initialized Lerna in an existing root-level project you may need to do some reconfiguring. Just make sure that each package has a valid package.json that Lerna can invoke with its commands. For example:

/packages
  /cli
    index.js
    package.json

Note that Lerna does not need to exist as a dependency in each package.

Quick tip:

You may feel more comfortable manually setting up your packages, but if you’re setting up new ones a great command to use is lerna create [name]. This will handle it all for you by creating the appropriate structure and assisting you in creating a package.json (same as npm init).

Commonly used commands

I went over lerna init and lerna create above to help you get started, so now let’s go over some of the commands you might use in a regular development workflow.

Note that commands support filter options, such as --scope to only execute on certain packages, --no-private to exclude private packages, and --since [ref] to only run on packages that have seen changes since a specified Git ref.

`lerna run`

This command will run, in each package, the script name affixed to it with the configured npmClient.

Let’s look at an example from the Firefox Accounts monorepo:

// package.json
"scripts": {
  "format": "lerna run test",
}

// packages/fxa-content-server/package.json
"scripts": {
  "test": "node tests/intern.js --unit=true",
}

// packages/fxa-payments-server/package.json
"scripts": {
  "test": "npm-run-all test:*",
}

// packages/fxa-payments-server/package.json
"scripts": {
  "test": "scripts/test-local.sh",
}

These are just a handful of the packages listed in that repo, but you get the point. Now, when you run npm run test at the root, Lerna will iterate across each package and effectively run npm run test in each directory.

There are handful of ways to customize how this runs:

# Pass args to the scripts
lerna run <script> -- [..args]
# Run scripts simultaneously, good for long-running processes 
lerna run <script> --parallel
# Ignore non-zero exit codes when running
lerna run --no-bail <script>

`lerna exec`

Use this command to run an arbitrary command inside each package. This might seem familiar to the run command, and if you wanted to you could probably set it to perform the same operation, but this command will run anything, and not just an NPM script.

Take a look at how Antony Budianto’s Create React App Universal CLI uses it:

// package.json
"scripts": {
  "clean:build": "lerna exec -- rimraf lib",
  "demo:start:client": "lerna exec --scope cra-universal-demo -- npm start",
}

You can see that for all packages running npm run clean:build would execute the command rimraf lib in each package directory, and for subsequent commands like demo:start:client the execution of npm start is using the --scope filter option to run only in the package named cra-universal-demo.

`lerna version`

This command performs a version bump on packages that have changed since the last release. It won’t run on any packages that have not seen changes, and if you’ve configured your ignoreChanges it will ignore those files as well.

By default just running lerna version will provide you with prompts as to how you’d like to perform each version bump. You can also provide a semver keyword (e.g. prerelease) or an explicit value (e.g. 1.4.8).

Once that’s done Lerna will run lifecycle scripts, commit and tag the changes, and then push everything to your Git remote.

You can configure this command in a variety of ways. For example:

# Create a release on GitHub or GitLab
lerna version --create-release github
# Perform the changes on the current commit and don't push
lerna version --amend
# Use Convention Commits Specification to determine the version bump and generate CHANGELOG.md files
# https://www.conventionalcommits.org/en/v1.0.0/
lerna version --conventional-commits
# Provide your own version bump message, where %v is replaced with the new version
lerna version -m "chore(release): publish %v"

`lerna publish`

You might be able to guess what this one does. As the name suggests, this command will publish a new version of each package to the NPM registry. By default it will call the version command behind the scenes and publish updates since the last release, but there are a couple other options for this:

# Publish packages that have changed since the last release
lerna publish
# Explicitly publish packages tagged in the current commit
lerna publish from-git
# Handy for when you’d like to manually increment package versions but want to be able to automate publishing
# Explicitly publish packages where the latest version is not present in the registry
# Handy for when you need to retry a publish (e.g. it previously failed to publish)
lerna publish from-package

This too comes with a handful of options to suit your needs. Some examples include:

# Publish to NPM with a given dist-tag (instead of `latest`)
# https://docs.npmjs.com/cli/dist-tag 
lerna publish --dist-tag next
# If you have two-factor authentication set up to publish
lerna publish --otp 818391
# Automatically accept prompts that come up during the publish process
lerna publish --yes

Wrapping up

I hope I was able to show you just how cool Lerna is in this post. It’s something I’m looking forward to using in all my projects with multiple components.

If you're in need of some examples of how others are using Lerna, the Lerna website has a nice big list of repos, including Jest, Chrome Workbox, WordPress Gutenberg, Vue CLI, ESLint Typescript, and more.

If this post was interesting and you’re eager to Lerna more (sorry) from the DEV community, check out these fine posts from others:

Also, if you’ve got 13 minutes you should definitely watch this great video from Ben Awad where he demonstrates using Lerna. It helped me get a better grasp on how things work, and I’m sure I’ve used some of that info in this post.

I’d love to see how you’re using Lerna! If you’ve got any comments or questions, hit me up in the comments 😎.

Another list of tips and tricks to improve your Git workflow

Jody Heavener — Mon, 24 Feb 2020 15:24:05 +0000

I did somewhat of a deep dive into Git recently, and I figured I could share some of the things that helped me greatly improve my workflow and overall understanding of how Git works. This article draws from various videos and articles, so be sure to check out the resources at the bottom of this post for even more details into each topic!

This post assumes a basic understanding of Git.

Sign your commits

As I started working on open source projects I discovered that a common rule for contributing is that you need to sign your commits. This makes sense, as projects want to maintain the integrity of commits and the code contributed to it. It makes sense to do it even if you’re not working on open source projects, too.

Commit signing is a way to verify that the person who pushed commits to the repository is in fact the actual person who created the commit. Here’s a more detailed explanation of what commit signing helps protect against:

There are a number of ways in which a git repository can be compromised (this isn’t a security flaw, just a fact of life—one should not avoid using git because of this). For example, someone may have pushed to your repository claiming to be you. Or for that matter, someone could have pushed to someone else’s repository claiming to be you (someone could push to their own repository claiming to be you too).

Rather than rewrite the wheel here I’m going to link to a fantastic post from Rémi Lavedrine, where they break down how to set up commit signing and getting it set up with the major Git services.

Find the commit that broke everything by bisecting

The feature’s broken? It was working just fine two months ago! What changed?

In order to effectively use bisect you need three things:

A test to determine if things are broken. If it’s a manual test that’s fine, but this is also where good test coverage can save you.
A commit where things were working. You can use git log to go back in history, checking out older commits to see where the thing that is broken works again.
A commit where things are broken. This would generally be the most recent commit.

We can now use bisect to perform a binary search that will find the commit where things went from good to bad. Step by step, here’s how it works:

# Tell Git we’re entering bisect mode
git bisect start
# Check out the commit where things are bad
git checkout 8ebbad
# Tell Git that this is a commit where things are bad
git bisect bad
# Check out the commit where things are good
git checkout a526ff
# Tell Git that this is a commit where things are good
git bisect good

Now Git will check out the commits in between the two you’ve provided, asking you to test it and determine it’s working or still broken each time. If it’s working, you’ll run git bisect good, and if not you’ll run git bisect bad. From this you gain more information about which commit introduced a breaking change.

If you’ve got an automated test you can actually set up a script to pass to Git (git bisect run test.sh) and it will use this to automatically find the culprit.

Stage and commit in a single command:

Instead of adding and committing in two steps:

git add .
git commit -m "Big changes"

For commits to changes on existing files it’s as simple as:

git commit -am "Big changes"

The -a flag will “tell the command to automatically stage files that have been modified and deleted,” but importantly, “new files you have not told Git about are not affected.”

Quickly update your most recent commit

Let’s say you forgot to add the ticket number to a commit message. Or maybe you forgot to include a file in your commit. You could create a new commit, or interactive rebase to reword the message, sure, but if it’s a simple change just “amend” the commit:

git commit -am "Big changes" # original commit
git add some-file.txt # add your missing file, if there is one
git commit --amend -m "#2947 - Big changes" # amended commit

This will simply update the message of the commit, but not any of the changes.

Fun tip:
Every commit in your repository has a long string of characters assigned to it, called a commit hash. The commit hash is generated based on the parent, the contents, and the message of that commit. In theory, because of the specificity involved in generating it, no two commits ever in existence should have the same hash.

Stop forcing it. Lease it, instead.

You’ve probably been burned by git push —force. If you haven’t maybe you should. Or at the very least someone should have yelled at you about it by now.

When you force push you’re effectively saying, “I don’t care what you think happened here, and I don’t care if others have pushed here since I last pulled - the timeline I’m presenting is fact.” This can sometimes be helpful (maybe you’ve just rebased), but at worst can be damaging as there is little recourse for this action. Unless you want to spend time trawling around the reflog you might be out of luck if you force push when you shouldn’t have.

Instead, you should consider force pushing with a lease. I think this Atlassian blog post put it best:

What --force-with-lease does is refuse to update a branch unless it is the state that we expect; i.e. nobody has updated the branch upstream. In practice this works by checking that the upstream ref is what we expect, because refs are hashes, and implicitly encode the chain of parents into their value.

What this means is that when you go to force push Git will check the local ref head against the remote ref and if they do not match the force will not continue. Magic.

Move something you accidentally committed to master over to a feature branch

Let’s say you were working on a cool new feature, meant for a feature branch, but you accidentally committed it to master. Oh no!

Fear not. You just need to cherry pick it. The first thing you’ll want to do is find the commit hash of the commit you just made. You can use git log to see this.

Once you’ve copied your commit hash, go ahead and check out your feature branch. The next step is to perform something called a cherry pick, which essentially clones your commit to the branch you’re on:

git log # retrieve your commit hash
# Let’s say your commit hash is
# 3e59227225678186a93efe0f3bc207d6483989af
git checkout my-new-feature # check out the feature branch
git cherry-pick 3e5922

You’ll notice we only supplied the first 6 characters of the commit hash. This is because Git is smart enough to identify your commit with just the first few characters of its hash.

Note that we only just cloned the commit to the feature branch, we did not delete the original commit on master. So how do we remove that accidental commit from the master branch? Let’s reset!

git reset changes the branch pointer to point to a different commit in your repository. We can use the HEAD reference to determine which commit it should point to. HEAD represents the commit you’re currently sitting on, and we can use the carat (^) to step back in commits:

HEAD # the commit we’re currently sitting on
HEAD^ # the parent of the commit we’re currently sitting on
HEAD^^ # the grandparent of the commit we’re currently sitting on

# You can use ^ to step back further and further in commits

Another, sometimes quicker, way to step back in time is to use the tilde (~) with HEAD:

HEAD~5 # the same as HEAD^^^^^

In our case, since we just want to remove the most recent commit, we can run git reset --hard HEAD^ to take us one step back. Our erroneous commit is no longer being pointed at.

Fun tip:
In Git, there is no such thing as a branch object. Branches are merely labels that point to commits. Over time, using various commands, you can change where those labels point to in order to change how those branches are structured.

Get a little interactive with your commits

There are some ideas discussed in this post about rewording and rearranging commits, and they are indeed helpful in their own way, but there’s an even more advanced way of modifying and manipulating your commits, and it’s called an Interactive Rebase.

You can enter interactive rebase mode by passing the -i flag to your rebase command: git rebase -i HEAD~5. This will open a list of the last 5 commits in your designated Git editor.

You can rearrange these this list of commits if you need to change the order in which they occur. The commits will be re-executed from top to bottom. Below your list of commits there will also be a list of keywords you can prepend your commit with in order to perform that action. Here are a few of them in a little more detail:

p, pick You’ll notice this is the default keyword before each commit, and it’s simply meant to signify that this commit is good to use. You can leave it in place if you’re not performing another action on that commit.
r, reword Place this keyword before a commit to reword the commit message. When you hit save on the file Git will re-open again, asking you to enter a new commit message for the specified commit.
e, edit You want to use this commit, but you want to make changes first. When you hit save on the file Git will, in the command line, prompt you to make your changes and then continue with git rebase --continue.
s, squash If you’re familiar with squashing this will sound familiar; use this keyword to squash this commit into the previous commit. Once you hit save on the file Git will prompt you to enter a new commit message for this newly melded commit.
f, fixup This is similar to squash, except it automatically uses the previous commit’s message.
d, drop This will drop the commit entirely. You can alternatively delete the line in your editor. As a reminder, interactive rebases are re-executed from top to bottom, so you need to be careful that the file changes in the commit you’re dropping aren’t further operated on where the initial change no longer exists.

There are additional options, x, exec, l, label, t, reset, and m, merge, that can also be handy, but are a little more advanced and probably won’t be something you use every day, so I’ll leave those for now.

Bonus: Git Extras

I’m probably preaching to the choir with this, but if you’re not already using Git Extras in your workflow, do yourself a favour and give it a whirl. This extension to the git CLI provides you with over 60 additional tools that range from novelty to daily use.

Here are some examples:

# Run git commands without typing 'git'
git repl
# Delete branches that have been merged
git delete-merged-branches
# Merge commits from new_feature into master
git graft new_feature master
# Output jodyheavener’s contributions to a project:
git contrib jodyheavener

Bonus: Node GH

Yeah, yeah, not everyone uses GitHub. We get it. But if you do you might benefit from this nifty little CLI that allows you to interact directly with GitHub.

The utility has a variety of helpful commands that allow you to drastically reduce the need to leave your command line. Here are some examples:

# List all your open Pull Requests
gh pr --list --me
# Comment on PR #52
gh pr 52 --comment "This is amazing!"
# Create an issue using your default editor to type the message
gh is --new --title "Error occurs when…" --message
# Close issue #81
gh is 81 --close
# List all gists
gh gi --list
# Create a new repo and clone it into the current dir
gh re --new booyah --clone

Update: Right as I posted this I saw that the official GitHub CLI entered beta. Give it a spin for me, would ya?

There appears to also be a gitlab CLI! But I haven't tried it, so GLHF.

Final Bonus: Conventional Commits

This isn’t a tool so much as it’s an approach to uniform, understandable Git commits (well, there is a tool). The Conventional Commits method defines a spec so that each commit’s message belongs to a type and an optional scope, followed by your commit’s description, body, and footer.

Some common types include fix, feat, chore, docs, style, refactor, perf, test, and others.

This pattern is useful in any repository’s history, but can be especially useful in open source projects where it helps to have everyone held to the same contribution standard.

As the project’s website outlines, commits should be structured like so:

<type>[optional scope]: <description>

[optional body]

[optional footer]

Here are what these could look like in practice (pulled from mozilla/fxa):

feat(oauth): added redis scripts to store oauth access tokens
fix(email): Minor CSS tweaks for subscription email
chore(CI): update three jobs to use node 12

Resources and further reading:

Is it possible to build plugins for multiple design tools with a single code base?

Jody Heavener — Sun, 23 Feb 2020 13:42:12 +0000

tl;dr DTPM is a proof of concept command line utility that would allow you to develop plugins for Sketch, Figma, and Adobe XD using a single API. Check it out here.

If you work in the design space you probably use tools like Sketch, Affinity Designer, Figma, Adobe XD, Photoshop, Illustrator, InVision Studio, Procreate… the list goes on and on. You're also more than likely familiar with the massive landscape of plugins that come with certain platforms. Photoshop has had them for nearly two decades, Sketch has had a pretty vibrant community for the last few years, and a few other tools have also started supporting them over the last couple of years, with more on the horizon. Plugins can be novel, they can handy at times, and in some cases they can be critical to ones design process. If you're not familiar with plugins, do check them out (Sketch, Figma, Adobe XD).

These days they’re also pretty fun to make, and the possibilities are endless. The development environments tend to be based in JavaScript with a handful of a host properties and methods, so it’s usually not too difficult to start working on your own plugin.

I’ve built a handful of plugins myself, and while the barrier to entry isn’t necessarily huge if you’re familiar with some basic JavaScript, I’ve noticed that as more design tools adopt the ability to expand their platforms with plugins they tend to implement the setup, manifests, and APIs in vastly different ways. I suppose it makes sense; they have no incentive to work with one another to create a consistent development environment, and in some cases they might want to expose unique functionality that their tool offers. It’s great that they’re supporting plugins, but the inconsistencies can sometimes suck as a developer if you’re trying to build the same plugin for multiple tools.

So what can be done about these inconsistencies?

I’ve been pondering this for the last few months, and today I want to demonstrate a proof of concept solution for a tool that would allow someone to be able to develop a plugin one time — that is, use a single API to interact with the platform’s components and capabilities — for multiple design tools.

What am I hoping will come from this?

I like to think of it this way: web browsers provide the same level of basic functionality. You can open a web page, interact with it, share it, and much more. Some browsers do things a little better than others and some provide bells and whistles to make the experience better, but at the end of the day, they’re all generally doing the same thing. The top browsers, even though they’re in competition with one another, for the most part all implement a similar structure of Browser Extensions. There’s even a W3C Draft Spec for it.

Much like browsers, these tools generally provide the same set of functionality, each with their own unique presentation, and a handful of speciality areas. So why shouldn’t they also support a consistent structure for plugins?

My goal with this project is to normalize development across multiple tools, find overlaps in the structures and APIs, and perhaps plant a seed that could one day lead to something of a “plugin spec” that all design tools could adhere to.

But at the very least, a neat little proof of concept is always fun.

Introducing DTPM

The Design Tool Plugin Manager, or DTPM, at its core is two things: a bridging API that provides a common interface for interacting with plugin environments, and a CLI that allows you to generate templates, compile your plugin files, and load it into each respective tool. Both of these things come from the dtpm command line, available on npm.

I’m aware I use the words “tool” and “platform” interchangeably. I’m also aware “Plugin Manager” isn’t the best name for this utility. Do not email me.

If you build plugins and this utility looks at all familiar it might be because I got a lot of inspiration from xdpm and skpm.

As of writing, DTPM supports generating plugins for Sketch, Figma, and Adobe XD. I initially was trying to also support InVision Studio, but their plugin environment isn’t ready.

Why these platforms?

As I mentioned previously most plugin environments are based in JavaScript, and these three are no exception. This makes it that much easier to provide a common development setup, without needing to wrap additional programming languages around each platform.

You may have also guessed from the JavaScript, but each platform also supports some degree of HTML and CSS when building out interfaces. This is a little more nuanced, though, so we’ll get to that later.

These platforms also all have bustling plugin development communities! This is one of my favourite parts, because it’s important to have others that you can bounce ideas off of, get help from, and in some cases even provide feedback to the platform creators.

Finally, all three design tools are rapidly evolving, and their plugin environments (for the most part) are keeping pace.

If they’re all based in JavaScript, what’s the problem?

For me, the biggest hurdle when building plugins for multiple platforms is that each one has different expectations.

Here are just a few examples:

Sketch and Figma both have document API environments separate from UI environment APIs, while XD exposes both APIs in a single environment.
Figma expects you to signal when your plugin is finished running, but Sketch and XD don’t seem to care.
XD and Sketch both support instantiating and manipulating layers before inserting them into the document, while Figma inserts on creation.
Sketch expects you to attach your command function to the window instance, XD expects them to be exported in module.exports, and Figma expects you to invoke your function based on the value of figma.command.

It doesn’t take long before you get a sense of how different each platform can be. Just take a look at what’s necessary to create a simple Rectangle layer:

So what does this thing actually do?

`dtpm new`

For starters, it removes the hassle of getting set up for each platform. The new command will generate a set of template files that you can use to start building plugins with.

`dtpm build`

The build command (optionally with the -watch flag) builds your plugin files into their respective plugin formats. Where possible the utility will also load them into the platform.

The API

This is the core of DTPM. It allows you to import various plugin, platform, and document related properties and methods, and will take care of executing them as each platform expects.

Let’s take a look at that Rectangle example again, but this time using DTPM to build for Sketch, Figma, and XD:

I’ll be the first to admit that right now this API is extremely slim. But hey, it’s a proof of concept. Don’t email me.

The small things

Lastly, DTPM is going to take care of generating a manifest for each plugin, with the properties that each platform expects, as well as generate your plugin icon into the required sizes.

You don’t have to give up full control to DTPM for manifests, though. For each platform you can specify override keys.

Here’s an example manifest:

Roadmap: plugin UI

One of the biggest inconsistencies across the three platforms, and therefor one of the biggest hurdles that this utility will need to overcome, is the ability to display plugin UI. It’s something I’m hoping to accomplish soon.

Why is it so challenging? You’ll need to dig into the code to really understand it, but the gist is that:

Figma supports a single HTML file that has to be listed in your manifest and invoked in your plugin environment code. It cannot talk directly to the document API environment, and instead relies on post-messages to transfer information back and forth between document and UI.
XD only supports using JavaScript to interact with an augmented DOM in order to create dialogue boxes. No external HTML files allowed. However, this also means that the document API environment and the UI API environment are one and the same.
Sketch does not provide any native HTML UI support, but the third-party sketch-module-web-view provides an Electron-style interface that allows you to effectively invoke a browser window. As such, and similar to Figma, this means you need to communicate back and forth via post-message setup.

Final thoughts

Thank you for coming to my TED talk. I’m not even sure if multi-platform-plugin-development-inconsistency is a problem that many developers face, but I certainly do, and this has been a fun experiment.

If you like DTPM, have any thoughts or suggestions about how to improve it, or just want to chat more about the wild world of design tool plugins, drop me a note.