DEV Community: Eyas

Server Environments

Eyas — Thu, 02 Feb 2023 00:46:44 +0000

When maintaining large software systems, you will likely have multiple environments with names like Prod, Staging, Dev, Eval, UAT, Daily, Nightly, or some remix of these names. To distinguish this type of environment from the dozen other things in software development that we give that same name, these are often formally referred to as Deployment Environments.

One question I've never been asked directly is: "What is an Environment?" This is surprising; because not understanding what Deployment Environments actually are is one of the most common pitfalls I see in my day-to-day work.

What is a Deployment Environment

A Deployment Environment is a consistently connected set of

processes
datastores, and
any ecosystem around them (e.g., cron jobs, analytics, etc.)¹

making up a fully functioning software system.

This definition is quite intuitive, but the devil is in the details. In this case, that detail is the phrase "consistently connected".

What consistently connected entails

Ideally, environments should be perfectly isolated; no data or RPC should leak between processes and data stores across environments during normal operation².

A Case Study

This concrete example convinces the uninitiated that you should never mix dependencies across environments (e.g., Dev instances should only call other Dev instances; never prod instances). If this is already intuitive for you, you can
skip this section and go straight to the principles of consistent connectedness.

Imagine the architecture above represents a system you maintain. Each box here represents either a service or a datastore. Right now, you have one instance of each of these endpoints, and they're connected as you see above. Let's say these systems are connected by directly addressing each other (e.g., by calling specific URIs for each service). Real people are about to use this service, but you want to continue deploying more recent versions.

You might decide to create a set of Dev instances to help you out.

You might wonder: How many instances do I need to set up and configure to have a viable Dev environment?

What is viable will certainly depend on which endpoints you care about testing.

Let's assume you want to test TodoApi. This service:

Calls PeopleApi
Reads and writes to TodoStore
Is called by PeopleApi
Is called by HttpBackend

Generally, if you can exercise the service you're interested in directly, you might not care about its callers.

Next, you have to decide:

which PeopleApi should this instance call, and
which TodoStore should this instance read and write to?

Remember that PeopleApi will call TodoApi back. It's very tough (and almost always wrong) to try to get away with calling the Production instance of PeopleApi from your Dev instance of TodoApi; the production instance you call might mutate the production state, or it might call back the production instance of TodoApi instead of you. You might convince yourself it's harmless, but more often than not, you'll be met with subtle glitchy behavior at best and serious bugs or user data leaks at worst.

Instead, you'll want an entirely separate Dev instance of PeopleApi in its own right. As you configure this dev instance of PeopleApi, you will have only one correct choice for which TodoApi to call: the Dev instance we just created.

We will also likely want a separate TodoStore database to be available to the Dev instance of TodoApi, with totally separate tasks, etc. This allows us to make sure none of our read/write testing has the potential to affect production users.

This line of reasoning applies recursively and can help you arrive at some general principles.

Principles of consistent connectedness

Read more about the principles of consistent connectedness in the full post.

I include the rest of the ecosystem around this for completeness, but often it's sufficient to think of a Deployment Environment simply as a consistently connected set of processes & datastores. ↩
Explicit processes that exist beyond the bounds of any environment may purposely interact with multiple environments. For example, it might be desirable to sync or seed some test data between environments, etc. ↩

A Brief Tour of the Unity Editor

Eyas — Wed, 24 Feb 2021 03:25:50 +0000

Those picking up Unity will likely have a lot of questions about the Unity Editor. How do I navigate it? What is a good development workflow using the Editor as my IDE? Can I circumvent the editor and just do things programmatically? This article will give you a brief tour of the Unity Editor, helping orient you around the environment. We'll explore how keeping everything in code can work, and why you might want to fight that urge.

This article originally appeared in my blog as part of a series called Unity for Software Engineers, targeting folks familiar with software development best practices seeking an accelerated introduction to Unity as an engine and editor. We already covered foundational concepts, high-level best practices, and the Input System. The series is specially tailored for those who learn best as I do: starting with first principles and working your way upwards.

A Tour of the Editor

The Unity Editor offers a customizable layout of dockable windows. You'll find many hidden gems buried in the dozens of windows Unity provides. In this article, I'll focus on a few key windows you'll need to understand when getting started:

Project View

The Project View represents your project's state on-disk, mapping 1:1 to the filesystem structure your project is under. This view allows you to explore your assets as they exist in your source project.

Within this view, you'll spend most of your time at the top level ./Assets/ folder and its children. This is where your scenes, prefabs, code, and authored code will likely live. This is also where assets you download from the Unity Asset Store will be added. Selecting an item in this view will show it in the Inspector. Specific file types, such as C# files, will open when double-clicked. Code will open in your favorite¹ editor, while other assets might trigger internal Unity Editor Windows to open. Double-clicking a scene will open it in the Scene View.

Scene View

The Scene View provides a visual of the open scene. You can pan, look, and zoom in the scene, move and rotate objects around, and select individual objects to modify.

The Editor Toolbar exposes several modes for interaction with the scene view. If you have a mouse with a middle button, you'll rarely want to explicitly use the "grab" mode; it allows you to pan around in the scene using your mouse's primary button, but you're almost always better off just holding the middle mouse button to pan. You can also rotate around the scene with your secondary (usually, right) mouse button, and zoom in/out with the scroll wheel. You'll find corresponding inputs for other pointing devices.

The other interaction modes like Move, Rotate Scale, and others, allow you to manipulate individual Game Objects in your scene. If no object is currently selected while in this mode, you can select any object by tapping it in the scene. When an object is selected, you'll see a Move/Scale/Rotate gizmo that lets you control the object.

A simple Scene view showing a Player (character asset by Supercyan) with custom gizmos for a camera controller. Assets in the background are courtesy of Leon LEE.

You can add new objects into your scene by either dragging a prefab from the project view or through the context menu in the hierarchy view.

Hierarchy View

The Hierarchy View is a tree view of objects in your current scene. Use this to quickly select an object, view or manipulate the object nesting structure, create/delete objects in the scene, etc.

Inspector

The Inspector gives you an editable view of the currently-selected object. The currently selected object can be any Game Object within a Scene or any asset within your project.

When selecting a Game Object (or prefab), every Component is represented as a section in the inspector window. Each serializable property of that component can be edited from the inspector.

When selecting other assets, you will see an editing experience for those. Assets will similarly expose serialized properties that can be editable from within, such as changing an input system settings or a 3D model's import settings.

Inspector Editors

The Inspector experience is driven through Editors. In a GameObject, each Component has an Editor responsible for displaying its properties (and any other options) in the GUI. Your assets are also driven by an Editor, just like individual components.

Engineers are encouraged to write their own custom editors for
MonoBehaviors and ScriptableObjects they create, where helpful. You'll find that extending the editor to write your own tooling is crucial for your development experience.

Property Drawers

Within an Inspector Editor, each serializable property exposed can be editable. A class that renders (and handles) this is called a property drawer. Unity provides built-in property drawers for primitives, Unity first-class constructs like Vector2, Vector3, UnityEvent, etc., and object pickers for all children of UnityEngine.Object². Unity also provides a default implementation for [Serializable] classes and structs, which basically recursively renders any Unity-handled sub-properties.

Edit Mode and Play Mode

You can also test-play your game from within the Editor. This is because the Unity Editor itself runs within the Unity Engine; the same process that serializes your project assets into your scene view for editing is what loads assets and scenes in-memory for playing. The result of this is a rich play+edit experience within the same IDE.

By default, the editor is in Edit Mode, which functions as described above. All serializable properties are writeable (an object's position and rotation are serializable properties, which is why you can place objects in a scene). Changing these properties allows you to save your project's new state.

Choosing the "Play" button lets you enter Play Mode in your current scene. This allows you to quickly test your game at a specific point. Play Mode shifts focus to a "Game" window, which is essentially the view of your game from your Main Camera (it can also take mouse inputs, etc. as implemented in your game).

You can always undock the Game View and be able to see both Game and Scene view simultaneously. If you do that, you'll see that objects in your scene move around and reflect your game's current state as you play it; this is because the same in-memory state of the objects in your game is shared between the various Editor components. The inspector will also still show the up-to-date state of a selected object's serializable properties.

You can even update a serializable property from the Inspector. This won't do any magic (e.g., retriggering events or initialization), but simply update a property's in-memory value. If, for example, you control an Object's speed in the Update() or FixedUpdate() loop, you can manually update a serializable "Speed" property on that object in the inspector and quickly try out different speed values before choosing what works best.

Note that when exiting Play Mode, the state of all of your serializable properties is reset to whatever initial value is set in your serialized scene. This makes sense since you don't want moving players/enemies while playing to actually apply that state when you're done playtesting.

Note that the low-level performance characteristics of playing your game within the Editor's play mode will be different from a built game. Some parts will be slower in play mode given the Editor's overhead, but others, such as loading an asset, might be faster in play mode than a built game. This is because the Editor might already have your asset deserialized from disk and loaded into memory. In general, you'll always want to thoroughly test your built game on the intended clients, but Play Mode remains a powerful tool for rapid development and testing.

What the Editor Does (and Doesn't) Give You

As a programmer using the Unity Editor, you first and foremost have a powerful level editor. Placing enemies or platforms around the level makes more sense to do visually. Setting an enemy patrol path similarly makes more sense to do within the visual 2- or 3-dimensional layout of a level.

Given Unity's serialization system and the Editor's ability to read and write to serializable properties, your Editor is both a powerful debugging engine as well as a tuning tool. If a jump between two platforms doesn't feel right, move it the platforms around while still in play mode until you find the perfect jump.

The Unity Inspector is also a manual dependency injection framework of sorts. Suppose you need objects to depend on each other. In that case, you can move out common logic into a ScriptableObject asset (e.g. an Inventory Manager, etc.) and pass that to objects that need it, without ever using static globals that make testing harder. The idea of a "manual dependency injection framework" sounds like an oxymoron: if you're just passing dependencies manually, what's the point? There are a couple:

We can still hook up our dependencies differently in tests by ditching fixed-in-code statics.
By always utilizing prefabs, it means that we'll often "inject" these dependencies once per type/class of object.

You will always need to programmatically create and place objects within your scene: e.g., instantiating explosions/effects, launching arrows or projectiles, or switching active weapons. Knowing what to hand-craft in the Editor's Scene View and what to create programmatically will take some intuition. Still, the first step, I think, is to try to rebel against the instinct to do everything programmatically: games are often inherently visual, and designing levels and placements visually often makes sense.

The trade-off looks different depending on your needs; expect to do way more in code when designing a procedural game. However, even in a procedural game, your intuitions might betray you if you don't consider the editor. Small pieces of a procedural game can still be designed visually and connected together in a procedural generation program that stitches those.

You can select the editor to open your C# code via Edit > Preferences... > External Tools and choosing your favorite editor from the External Script Editor drop-down. ↩
Children of UnityEngine.Object include GameObject, MonoBehavior, and ScriptableObject. Any individual subclass (such as a user defined MonoBehavior) exposed as a serialized field will be represented as an object picker. ↩

Unity Input System, from Basic Principles

Eyas — Mon, 02 Nov 2020 00:11:22 +0000

In the third installment of the Unity for Software Engineers series, we cover Unity's Input System from basic principles.

Handling player input will likely be among the first game systems you implement. Unity Technologies unveiled a new Input System in 2019 intended to replace their previous system. While the new Input System allows building on more idiomatic Unity patterns, it can also be more challenging for a newcomer to pick these up. This is especially true because the Input System leans on configurable assets and emphasizes Editor usage. The Input System's Quick start guide is helpful, but only once the Input System and Unity Events' basic concepts are understood.

Much of the newer pieces of the Unity Engine are being released as encapsulated packages. Unity packages are installed from the Unity Package Manager UI in the editor (open it in Window > Package Manager). The package updates a manifest.json file that has a "dependencies" entry identical to a package.json. The
package manager resolves dependencies and writes to a package-lock.json. You can install the Unity Input System from the Package manager, which adds a dependency on com.unity.inputsystem in your manifest.

The Input System makes it easy to create player-configurable cross-platform game controls through assets known as Input Action Maps. The system also exposes an Editor- and event-based interface to specify Input behavior.

Programmatic Use

If you'd like to make progress on a proof of concept quickly, you might initially choose to sidestep Input Action maps and programmatically interrogate the inputs.

Here's a basic example covering the most important concepts:

using UnityEngine;
using UnityEngine.InputSystem;

public class PlayerControlHandler : MonoBehaviour
{
  /// <summary>
  /// The speed of a moving player in m/s.
  /// </summary>
  [SerializeField] private float _moveSpeed = 5f;

  void Update()
  {
    // The last used (or last connected) Gamepad.
    // null if there is no connected gamepad.
    Gamepad gamepad = Gamepad.current;
    // The return value of `.current` cann be null.
    if (gamepad == null)
    {
      return;
    }

    // buttonSouth is a ButtonControl.
    if (gamepad.buttonSouth.wasPressedThisFrame)
    {
      Fire();
    }

    // leftStick is a StickControl, which eventually
    // inherits from InputControl<Vector2>.
    //
    // InputControl<T> exposes a ReadValue() method
    // returning T. Typically, this is a zero-value
    // when the control has not been actuated in a
    // given frame.
    //
    // Here, move is a Vector2 between (-1, -1) and
    // (+1, +1) indicating the most recent direction
    // of the stick (as of this frame).
    Vector2 move = gamepad.leftStick.ReadValue();

    Vector3 moveThisFrame =
      // Make movement speed frame-rate independent
      Time.deltaTime * _moveSpeed *
      (
        (Vector3.right * move.x) +
        (Vector3.forward * move.y)
      );

    transform.position += moveThisFrame;
  }

  void Fire() {} // TODO: Implement.
}

or:

using UnityEngine;
using UnityEngine.InputSystem;

public class PlayerControlHandler : MonoBehaviour
{
  /// <summary>
  /// The speed of a moving player in m/s.
  /// </summary>
  [SerializeField] private float _moveSpeed = 5f;

  void Update()
  {
    // Control input with WASD + Mouse
    Keyboard keyboard = Keyboard.current;
    Mouse mouse = Mouse.current;

    if (keyboard == null || mouse == null)
    {
      return;
    }

    // Mouse.leftButton is also a Button control.
    if (mouse.leftButton.wasPressedThisFrame)
    {
      Fire();
    }

    // Every keyboard key is a KeyControl, which
    // inherits ButtonControl.
    //
    // ButtonControl extends InputControl<float>
    // which represents a control with a value 0
    // when not pressed and 1 when pressed.
    //
    // Here we represent our movement as vertical
    // and horizontal axes, each from -1 to +1.
    float forward = keyboard.wKey.ReadValue()
                  - keyboard.sKey.ReadValue();
    float horizontal = keyboard.dKey.ReadValue()
                     - keyboard.aKey.ReadValue();
    Vector3 moveThisFrame =
      // Make movement speed frame-rate independent
      Time.deltaTime * _moveSpeed *
      (
        (Vector3.right * horizontal) +
        (Vector3.forward * forward)
      );

    transform.position += moveThisFrame;
  }

  void Fire() {} // TODO: Implement.
}

As you can see, UnityEngine.InputSystem exposes various input devices, each with a static current member returning the device if connected¹. See the documentation for UnityEngine.InputSystem.Gamepad, for example.

Each Input Device exposes a number of Controls. These all implement the InputControl<T> class. Significantly, InputControl<T> exposes T ReadValue() which returns the current value. This can be repeatedly called in Update() or FixedUpdate() to respond to input.

Most buttons (Gamepad buttons, Mouse clicks, Keyboard keys) eventually inherit ButtonControl, which also introduces useful properties such as isPressed, wasPressedThisFrame, and wasReleasedThisFrame. A ButtonControl eventually inherits from InputControl<float>. On digital devices, controls can have a value of either 0 or 1, but on analog- or pressure-sensitive devices, this can smoothly vary between 0 and 1.

Other controls, like DpadControl and StickControl eventually inherit from InputControl<Vector2>, which gives us the combined player input in 2D space.

You can define a composite Vector2 control using Vector2Composite from UnityEngine.InputSystem.Composites. Doing so is helpful, for example, to create WASD as a single Vector2 composite. At that point, however, you might as well go for the more expressive approach of using Input Actions and Input Action Map assets, rather than the purely programmatic approach.

An advantage of programmatic handling of input is that you can get started quickly. The moment you want to make controls player-configurable or handle multiple devices, this code becomes unwieldy.

Input Actions

Last week, I warned against the impulse of keeping everything in code, when the editor is at your disposal. With a basic understanding of the fundamentals from the code above, I recommend using the higher-level concept of "Input Actions".

A recently-created Player Input component allows its Game Object to respond to inputs from a given Input Action Map asset.

To start making a Game Object controllable, let's add a Player Input component to it. A component takes an Input Actions
asset and uses it to define behaviors for each Input Action.

To start, we'll use the Create Actions... button to create a default Input Action Asset. Unity will pedagogically create an asset pre-populated with a cross-platform control scheme, including movement (on PC: WASD), look (on PC: mouse movement), and firing.

A populated Input Action asset loosely modified from the default asset. ![A populated Input Action Asset. Shows two "Action Maps" for Player and UI. The Player map is selected, with definitions for Move, Look, Fire, and Jump actions.](https://dev-to-uploads.s3.amazonaws.com/i/334nkpv5ghcibq1zjsy1.png)

An Input Action Asset describes a game's entire control schemes. The asset comprises multiple Action Maps, each corresponding to a control scheme active in a given context. By default, Unity creates a "Player" (in-game) and a "UI" (e.g., in menus) map.

Here, you can see that Move is a Vector2 action. The Gamepad control scheme uses a stick control, while the Keyboard/Mouse scheme uses a composite Vector2. Keyboard/mouse bindings here allow the player to use either WASD or arrow keys.

Modify your Action Map as needed. For now, don't worry about Interactions or Processors; these allow you to constrain or pre-process action values before passing them to consumers.

When a Player Input component has an input action asset, it allows you to set a Default Scheme and a Default Map.

A Player Input component has four potential behaviors:

Send Messages: For each Input Action that actuates, calls SendMessage on the Game Object, with the action name (e.g., OnFire, OnLook, OnMove) and an InputValue.

An InputValue exposes boolean property isPressed, and the generic TValue Get<TValue>() method. You will need to make sure the generic type you pass is the same as the Action Type in the Input Action Map.

I alluded to Unity Messages in the section on Components in this series. We already saw message functions such as Update(), OnEnable(), and others. Here, the component calls messages that any other component in this GameObject can implement.
Broadcast Messages: The exact behavior discussed above, but using BroadcastMessage instead.

All MonoBehaviours receive a broadcasted message on the GameObject or any of its child objects.
Invoke Unity Events: Exposes a UnityEvent<InputAction.CallbackContext> for each action in the asset. UnityEvents have a friendly API and Editor interface and allow you to specify the exact set of methods to trigger during an action.

InputAction.CallbackContext is also richer than InputValue, exposing the corresponding InputAction. Further details such as whether the event corresponds to an action that was canceled (e.g., a button that was released), performed, or started (e.g., a button that was just pressed). Like InputValue, the Callback Context exposes a method to read its value: TValue ReadValue<TValue>().

A Unity Event is triggered every time an action is started, performed, and canceled. While an InputAction is actuated, it will trigger a performed event each time the underlying value changes.

An action such as a digital button will typically fire a single started, performed, and canceled event. Holding a button for 5 seconds, for example, would merely delay the canceled event.

On the other hand, a more variable value, such as a gamepad stick Vector2 value or a Mouse delta Vector2, would trigger a started, typically followed by many performed events, and finally, a canceled event.
Invoke C# Events: Exposes an onActionTriggered C# Event on the component which similarly triggers an InputAction.CallbackContext, as described above.

Unlike Unity Events, C# events cannot be modified from the editor. Therefore, components that want to subscribe to an event will need to reference the PlayerAction component to subscribe to the event. To avoid leaking memory, you should always remember to subscribe in OnEnable and unsubscribe in OnDisable.

Also, unlike Unity Events, there is only a single onActionTriggered event that will trigger the callback for any action that takes place. This means your callback should switch/condition on the Callback Context's action property.

Using Unity Events

I usually prefer using Unity Events with the Input System for a few reasons:

I find the InputAction.CallbackContext API more helpful than InputValue;
In SendMessage and BroadcastMessage, a typo in the message name fails silently, resulting in a method that never gets called;
Compared to C# events, I find it helpful to visually separate the handlers for each action, not only across methods in the same component but across components (e.g., a Move component and an Attack component).

A Unity Event can trigger callbacks on any Unity Object, whether in Asset form (e.g., a ScriptableObject) or as a Scene GameObject². In most cases, you will probably want to be passing your events to a MonoBehaviour on a scene object. Make sure to select the object in the "Scene" tab rather than
"Assets" (i.e., do not select a prefab)³.

Therefore, the typical process will be to

select the scene object that contains the component you want to call,
select the sub-menu corresponding to the specific component, and
pick your method from the menu of available public methods.

In cases where your public method signature takes in a single
InputAction.CallbackContext parameter, it will show up in the "Dynamic CallbackContext" section, and automatically bind that input when the event triggers.

Inputs and Frame-rate Independence

Unity will call the Update method as often as it can. Each update method represents a single frame in your game. This means that you should not use a constant "speed per frame" to control how much the player moves per frame, for example, because that will alter the player's apparent speed as the frame rate fluctuates.

Unity exposes Time.deltaTime largely for this purpose, which gives you the elapsed time since the last frame. If you want your player to move by a constant speed, for example, you will want to make sure to multiple by Time.deltaTime:

Vector3 moveVelocity = _moveSpeed * (
    _currentMove.x * Vector3.right +
    _currentMove.y * Vector3.forward
  );
Vector3 moveThisFrame = Time.deltaTime * moveVelocity;

transform.position += moveThisFrame;

With Unity Events, a common way to handle button movement is to do something like this:

using UnityEngine;
using UnityEngine.InputSystem;

namespace Game.Controls
{
  public class PlayerMover : MonoBehaviour
  {
    [SerializeField] private float _moveSpeed;
    private Vector2 _currentMove;

    public void OnMove(InputAction.CallbackContext context)
    {
      // This returns Vector2.zero when context.canceled
      // is true, so no need to handle these separately.
      _currentMove = context.ReadValue<Vector2>();
    }

    private void Update()
    {
      Vector3 moveVelocity = _moveSpeed * (
          _currentMove.x * Vector3.right +
          _currentMove.y * Vector3.forward
        );
      Vector3 moveThisFrame = Time.deltaTime * moveVelocity;
      transform.position += moveThisFrame;
    }
  }
}

Frame-rate independence is also helpful with action cooldowns (e.g., not being able to Fire multiple times too frequently), changing rotations, etc. But you should also watch for pitfalls of when not use Time.deltaTime in a computation.

A great example to watch out for is mouse delta. Mouse Delta tells you how much a mouse has moved.

If you have a pointer to an InputAction in your Update method, the Vector2 returned from a mouse delta action will give you the total distance moved this frame. A longer frame will have a larger value (in other words, Time.deltaTime is already accounted for), and multiplying by Time.deltaTime will further cause the value to be disproportionately larger for longer frames, and disproportionately smaller for shorter frames.

If you are using UnityEvents, a mouse delta action will result in your callback potentially being called multiple times per frame with a different delta value. These are separate deltas that need to compose additively. Suppose you want to handle player movement and rotation (or camera rotation) only in Update(), LateUpdate(), or FixedUpdate(). In that case, you'll need to keep track of the delta cumulatively for each frame, resetting it between frames. For example:

using UnityEngine;
using UnityEngine.InputSystem;

namespace Game.Controls
{
  /// <summary>
  /// Moves the object according to input. Typically attached to a top-down
  /// camera overlooking the scene.
  /// </summary>
  public class ViewportMover : MonoBehaviour
  {
    [SerializeField] private float _moveSensitivity;
    private Vector2 _panThisFrame;

    public void OnMove(InputAction.CallbackContext context)
    {
      // This works for delta values, since it is reset at the end of each
      // frame.
      //
      // If the game was cross-platform, and other panning controls used
      // console sticks, for example, then those events should replace
      // each others, rather than add. We can check context.control.
      _panThisFrame += context.ReadValue<Vector2>();
    }

    private void Update()
    {
      // For Mouse delta, _panThisFrame is already frame-rate independent, and
      // only needs to be multiplied by a sensitivity value.
      //
      // If the game was cross-platform, we'd need to multiply this value when
      // using a non-delta control. We can check context.control.
      Vector3 panThisFrame = _moveSensitivity * (
          _panThisFrame.x * Vector3.right +
          _panThisFrame.y * Vector3.forward
        );

      transform.position += panThisFrame;

      // Reset pan at the end of each frame.
      _panThisFrame = Vector2.zero;
    }
  }
}

Final Thoughts

There is much to learn when mastering smooth player inputs, but I hope the resources above give you the tools you need to start. Whether it is thinking about frame-rate independence (and knowing when not to blindly multiply by Time.deltaTime), creating easily overridable player inputs using Input Action Maps, or cross-platform input, I hope input in Unity feels less mysterious by
now.

I also hope I convinced you that Unity Events are a great way to substitute hard cross-component references (statics or using GetComponent) into injectable connections via the Editor. Keep them in mind next time you design a system that needs to interact across components.

Local multiplayer games can't count on these static accessors, as the current Gamepad is whichever device most recently used. ↩
Read more about the difference in the first installment, Basic Concepts in Unity for Software Engineers. ↩
For example, if you want to move your player, you should pass the Player from the scene tabs. If your player is prefab-ed and you pass the prefab, you'll see some unexpected behavior when trying to move the player. ↩

6 Software Practices to Keep, Shed, and Adopt in Unity

Eyas — Fri, 23 Oct 2020 21:06:34 +0000

This is the second installment in my article series, Unity for Software Engineers. Check out the first article about six fundamental concepts in Unity. I'll be releasing additional installments over the next few weeks, so make sure to subscribe for updates. The series especially tailored for those who learn best as I do: starting with first principles and working your way upwards.

Software Engineers starting with game development are often looking for best practices and idiomatic techniques. You'll find some authoritative sources of idiomatic development from talks in Unity's Unite conference, posts on the Unity blog, and members of the community like Jason Weimann. But game development isn't just engineering; it's also an art. This sometimes means that getting something to work takes center stage. You might find a lot of advice along the lines of "do what works", which, while it is valid and helpful, doesn't quite send you down the right path when you're still learning.

Below, I put a brief list of Software Engineering practices you should shed when starting with game development, those you should adopt, and those you should keep.

Shed: Keep it all in code

In Software Engineering, it often feels like the more you represent in code, the better off you are: you can analyze your code to find references, jump to definitions, etc. A Software Engineer might be inclined to represent their scenes, objects, and most of the game in code.

But you'll probably want to get used to leaning more on the Unity Editor UI than you expected.

By simply setting a "Patrol Position" to an empty Game Object, you'll already have a neat visual game editing experience.

The Unity Engine does a lot of heavy lifting in its serialization/deserialization code and in asset management. If you're programming in Unity (at least when getting started), you'll want to swim with the current.

Game design is also inherently visual, usually. Placing your objects in 3D space, constructing guard patrol paths, or setting up a field-of-view for those guards are all easier done while visually editing a scene than by writing down the coordinates imperatively. What's more, if you need to modify these values later, visually doing so is less error-prone.

Adopt: Write your own Custom Editors and Property Drawers

Along the same lines, one of the first things you should make a habit of doing is to make sure your custom components, property types, and assets feel like first-class citizens in the Unity Editor.

Have your own "Float Range" struct? Why not make sure you can manipulate it in the Editor with sliders that guarantee the min is always smaller than the max?

Have a MonoBehaviour with mutually-exclusive fields? Write a custom editor for the MonoBehaviour that displays them how you want.

Rolling your own Dialogue Tree? Represent that as an asset and write a custom editor for it.

Custom Editors are not just a great way to make your objects editable as first-class citizens; they can also be a way to make sure your objects are debuggable and testable as first-class objects. Write an editor that shows you some of your object's internal state when in play mode, or add a button to your editor that triggers a state in a controlled way (e.g., an "I got hit" button).

An example "Interactable" component allows setting effects when the player initiates an interaction.

See more in the Unity docs on Custom Editors and Property Drawers. The Brackeys Editor Window video is also helpful.

Keep: Singletons tend to be a code smell

If you're following game development tutorials or seeing discussions in forums, you'll see singletons everywhere. But for much the same reasons as in Software Development, the singleton pattern is often inflexible, untestable, and has the potential to tangle your dependencies.

See more here and here.

Shed: Get comfortable with some globals

It might be tempting to be very strict about always encapsulating state and fully separating concerns. In game development, though, I find it helpful to get comfortable with the idea of slightly more state sharing than you would like in some other software system.

Part of this might be because there truly is more global state in games (e.g., has the first level boss been defeated, has the first town been saved, etc.?)

But I argue that even states that you can encapsulate might benefit from being a little more global. For example, you can encapsulate an HP variable within your Player component and use it to drive logic. You would also need your Player component to control HP display in the UI, and maybe screen-shake effects when the player takes a hit. On the other hand, having a global "Player HP" variable that you can pass to the player, a HUD object, and a screen-shake VFX object might be a cleaner alternative.

Adopt: The Inspector can be your injection framework

I alluded to this above, but I think the way you can have clean globals can be a streamlined form of dependency injection. If a player requires a global Player HP value, let the component reference an HP serialized field and pass it in the inspector. You'll get some of the nice things about DI (stubbing in a test, isolation), without much of the "magic" that makes it so intimidating to use.

See more in Ryan Hipple's talk on Scriptable Objects. If you'd like to learn more about Scriptable Objects, I wrote about their serialization and surveying their uses in a Unity tutorial.

Keep: Unit- and Integration Tests are Always Good

It's easy to spend many months reading about game development & best practices without reading discussions on test frameworks. Unity provides the Unity Test Framework to this end. This tutorial by Anthony Uccello provides an overview of how to get started.

Photo by Kelly Sikkema via Unsplash

Closing

So far, in the series, we covered Unity fundamentals and some intuition on the practices to lean into. Next, we'll cover how to orient yourself around the Unity Editor and go over the bare minimum you need to know to unblock yourself as a solo programmer working with lighting, animations, input, and other Unity tools.

Basic Concepts in Unity for Software Engineers

Eyas — Thu, 22 Oct 2020 03:50:54 +0000

If you're trying to get into game development as a Software Engineer, finding learning materials with the right level of context can be challenging. You'll likely face a choice between following materials introducing you to basic C# and OOP concepts while also describing Unity concepts, or starting with advanced tutorials and be left to figure out the core concepts deductively.

To fill that gap, I'm writing a series called Unity for Software Engineers. This is the first piece, and I'll be releasing additional installments over the next few weeks, so make sure to subscribe for updates. The series is intended for folks already comfortable with programming and software architecture, especially those who learn best as I do: starting with first principles and working your way upwards.

This article originally appeared on blog.eyas.sh.

I started my programming journey around 17 years ago by picking up
Game Maker. Countless hours spent coding little games and tools led me to a bigger passion for programming. Eventually, I was at a point where I focused mainly on Software Engineering. From my peers, I know this is quite a common path that many of us took to find programming.

Yet the game development scene has changed significantly from those days. When I went to pick up Unity after a long absence from game development, I was mostly interested in understanding the basic concepts: what are the fundamental building blocks of a game? What do I need to know about how these building blocks are represented in memory or on disk? How is idiomatic code organized? What patterns are preferred?

In the first article in the series, we'll focus on those first two questions.

Scenes

A Scene is the largest unit of organizing your objects in-memory. Scenes contain the objects making up your game.

In basic use, one scene represents a single level in your game, where one scene is loaded at any given point. In more "advanced" use, you can have two or more active scenes at a time. Scenes can be loaded additively and unloaded ¹. Having multiple scenes loaded during gameplay comes especially handy when building a massive world; keeping far-away areas on-disk rather than in-memory will help you stay within your performance budget.

Unity Scene editor opening a default empty scene in 3D mode. Empty Scenes in Unity3D will by default include _Main Camera_ and _Directional light_ objects.

Unity Scene editor showing an example scene, with a few objects selected. You can use the scene view to edit levels in your game.

Every game object in Unity needs to be in a scene.

Game Objects

A Game Object (in code, GameObject) is one of the basic building blocks of a game.

Game Objects can represent both physical things you see in the game (e.g., a player, the ground, a tree, a terrain, lights, a weapon, a bullet, an explosion) as well as metaphysical things (e.g., an inventory manager, a multiplayer controller, etc.) in your game.

Every Game Object has a position and rotation. For metaphysical objects, this doesn't matter.

Game Objects can nest under each other. Each object's position and rotation is relative to its parent object. An object directly in the scene is relative to "world space" coordinates.

A group of objects nested together in a scene under an empty "Interior_Props" object, for organizational purposes

You might choose to nest your objects for many reasons. For example, you might decide it organizationally makes sense to put all your "environment" (e.g., individual pieces that make up a city or village) objects under an empty parent object. This way, it can be collapsed in the scene view and easily moved together when building your game.

A group of objects nested under the player. These include the player's weapon, avatar, and various UI elements rendered around the player.

Game Object nesting can also have functional significance. For example, a "Car" can be an object, with code that controls the car's speed and rotation as a whole. But individual child objects might represent the four wheels (these would spin independently), the car body, windows, etc. Moving the parent car object would move all the child objects, keeping their relative orientation to the parent (and each other). We might want the player to interact with a door separately from the rest of the car, for instance.

Components (& MonoBehaviors)

The Warrior object from the previous screenshot is shown above in Unity's "Inspector" window. Each illustrated section (e.g., Animator, Rigidbody, Collider) are _Components_ making up this object.

Every Game Object is comprised of Components.

A Component implements a well-defined set of behaviors for a GameObject to execute. Everything that makes an object what it is would come from the components that make it up:

A single "visible" piece of a car will have a Renderer component that paints it, and likely a Collider component that sets up its collision bounds.
If a car represents the player, the car object itself might have a Player Input Controller that takes key input events and translates these to code moving the car around.

While you can write large and complex Components that correspond 1:1 to an object (e.g., a player component codes the entire player, while an enemy component codes the enemy as a whole), it's typically common to factor out your logic into streamlined pieces corresponding to individual traits. For example:

All objects with health, whether a Player or an Enemy, might have a LivingObject component that sets an initial health value, takes damage, and triggers a death event once it dies.
A player might additionally have an input component controlling its movement, while the enemy might have an AI component that controls its movement instead.

Components receive various callbacks throughout their lifetime, known in Unity as Messages. Examples of Messages include OnEnable/OnDisable, Start, OnDestroy, Update, and others. If an object implements an Update() method, this method will automagically be called by Unity in every frame of the game loop while the object is active, and the given component is enabled. These methods can be marked private; the Unity engine will still call them.

Components can also expose public methods as you'd expect. Other components can take a reference to a component and call these public methods.

Assets

Assets are the on-disk resources that make up your game project. These include meshes (models), textures, sprites, sounds, and other resources.

When serialized to the disk, your scenes are represented as assets made up of the Game Objects inside of them. We'll also discuss in the next section how you can make Game Objects that are reused often into an asset known as a Prefab. ²

Assets can also represent less tangible things, such as Input Control Maps, Graphics Settings, i18n string databases, and more. You can also create your own custom asset types using ScriptableObjects. I wrote about how these are saved
here.

For your in-development project, assets form the key representation of your project's codebase, alongside your code.

Your built-and-bundled game will include most ³ of your assets. These assets will be saved on disk on the device where the game is installed.

Prefabs

Game Objects, their Components, and their input parameters exist as individual instances in a scene. But what if a particular class of objects is commonly repeated? Such objects can be made into a Prefab, which is effectively the object in asset form.

Instances of a prefab in a scene can have local modifications that distinguish it (e.g., if a tree object is a prefab, you can have tree instances of different heights). Instances of a prefab all inherit and override data from their prefab.

Nested Prefabs

Starting with Unity 2018.3, Prefabs can be nested just as you expect:

A parent object with prefab child objects can be a prefab itself. In the parent prefab, the child prefab instance can have its own modifications. In the scene, the entire prefab hierarchy is instantiated, and scene-specific modifications can layer on top.
A prefab instance in a scene with its own local modifications can be saved as its own "Prefab Variant" asset. A variant is a prefab asset that inherits from another prefab, applying additional modifications on top.

These concepts compose; a prefab variant of a nested prefab, or a prefab variant of a prefab variant, for instance.

Serialization & Deserialization

Your project's assets, scenes, and objects are all persisted on-disk. When editing your game, these objects are loaded in memory and saved back to disk using Unity's serialization system. When playtesting your game, the objects and scenes in-memory are loaded through the same serialization system. This system also maps between assets in your compiled bundle and the loaded/unloaded scene objects in-memory.

The Unity Engine's Serialization/Deserialization flow loads on-disk assets into memory (in your project: for editing/playtesting; in-game, when loading a scene) and is responsible for saving the state of your edited objects & components back into their scenes and prefabs.

Therefore, the serialization system is also at the core of the Unity Editor experience itself. For a MonoBehavior to take an input on construction when instantiated in a scene, those fields must be serialized.

Most core Unity types such as GameObjects, MonoBehaviours, and asset resources are Serializable and can receive initial values on creation from within the Unity Editor. Public fields on your MonoBehavior are serialized by default (if they're of a serializable type), and private fields need to be marked with Unity's [SerializeField] attribute to be serialized as well.

Conclusion

These six concepts cover essential structural pieces for architecting games in Unity. Knowing more about these and how assets on-disk map to in-memory representation should give you the intuition needed to follow some of the more advanced tutorials.

There are still significant areas to wrap your head around in Unity. Understanding the Editor and mapping your Software Engineering best practices to game development best practices will help hone in your skill. Even more, understanding broad areas such as lighting, animation controllers, navigation meshes, input handling will help you go a long way as well.

I will cover many of these topics in future iterations of this series, but I'm hopeful you're armed with the knowledge that makes the rest of your game development journey more intuitive.

Still, there can only ever be one "main" active scene at a time. ↩
The term asset is a bit overloaded. Scenes are Prefabs are serialized and organized in your projects like other assets. You can browse these in the Asset view alongside code and other assets. If you're deep in Editor code manipulating an asset, Prefabs and Scenes cannot usually be treated as assets. ↩
You can further optimize your game by loading some assets over the network or employing other asset-management techniques. ↩

My Journey Through Tech Volunteering: Anticipation, Passion, Burnout, and Looking Ahead

Eyas — Mon, 25 May 2020 17:39:53 +0000

Cover Image: Winding Path by Phil Bulleyment, via Flickr. CC BY-2.0

I had been working in New York City for just over a year when I sat down at one of my favorite cafes in my neighborhood to write a personal journal entry. I gave it the title "On the crossroads between goal-oriented and process-oriented" and I wrote down stream-of-consciousness reflections on my life, career, and how I wanted to do things differently.

It was October 2015, and I had finished grad school and moved to NYC to work full-time as a Software Developer in a fin-tech company. I was having what I have come to see as the seminal quarter-life crisis many folks go through when they finish their formal years of education. I had been chasing goals all my life up until then, and now I had the luxury and privilege of deciding whether I should set another goal or do things radically different than what I had done so far. Goal-oriented versus Process-oriented, I called it.

A lot of my thoughts at the time (and those in that journal) have been foundational to my thinking in this new life chapter. Most of those are a story for another time. One I kept coming back to, however, was knowing that I needed to re-engage with doing good in the world.

There are many ways to do good in the world, but I eventually settled on finding skill based volunteering opportunities that used my tech skills as potentially most effective and satisfying. My thinking was: If tech companies put a high dollar amount on my time and skill, wouldn't giving some of it away be the most effective thing I can do?

This is the story about how I found my path volunteering in tech, how it gave that new chapter of my life new meaning, how I burned out, and what's next.

Photo by _Women of Color in Tech stock images_ ([via Flickr](https://www.flickr.com/photos/wocintechchat/25720969670/))

Finding my path in Volunteering in Tech

Being passionate about tech and justice, I was interested especially in areas where I can help in issues of equity, representation, and access. I also liked teaching, so my first intuition was looking at programming bootcamps, as well as programs like Girls Who Code, Black Girls Code. Bootcamps appealed to me because they gave an opportunity to work with folks looking to start a career in tech soon and provided access to minoritized people often excluded from the "traditional" STEM pathway. School programs were exciting for different reasons: it's way more indirect, but it gave the opportunity to inspire a student to consider a totally new path.

One important piece of my situation back then is immigration: I was on an H1-B visa, which meant that my continued ability to live in the U.S. was tied to my job. Therefore, I wasn't really looking for full-time or sabbatical-type service opportunities.

This ended up excluding most bootcamps at that time. It also excluded the more intense summer programs like the Girls Who Code Summer Immersion Program.

I decided the best match for me at the time was the Girls Who Code Clubs program, which gave me the ability to work with a school in the NYC-area as an instructor in their after-school program. Finally, after a lot of indecision, I ended up applying on Dec 31.

The background screen started on Jan 3. Five days later, I received an e-mail saying I was matched with Chloe Taylor, who was just co-starting a Club at her school. For the coming spring term, I would be helping as an instructor for an after-school club for 5^th- and 6^th-grade girls interested in coding.

Your Nth reminder that teaching is hard

Everyone tells you teaching is hard, but boy was it hard. I always felt like teaching and mentoring was a core part of me and my passions. I deeply cared about doing a good job. I practiced my very first class again and again, but I don't think I was truly ready; I underestimated how difficult it would be to command the attention of a classroom. I was used to TA-ing in college where the students needed me, or mentoring co-workers asking for help, but those were adults who had figured out how to manage their attention and distractions. I remember feeling a hard-to-pinpoint—almost embarrassed—sensation after that first club meeting, like I failed at something and was anxious to show my face the next week.

Some of the girls were incredibly excited to code. For others, their parents wanted them to get into it, but they were not yet convinced. They all were eager to try coding! The first good day, where I felt I truly taught and excited them, I was over the moon. But the hard days where I felt like a total failure never really went away.

The clubs were set up (at least back then) to meet once a week, have the instructor briefly present a new concept, then work on practice exercises. None of the girls had prior coding experience, so we were using the Scratch programming language¹. I would walk around the classroom, with the other teachers (who were themselves picking up coding and doing some of the exercises) and we'd answer questions, check-in on the students, and guide them through their projects.

It was a bit of a trip re-learning how a tween in junior high acts: How they go off day-dreaming mid-sentence or be very single-mindedly focused when something excites them, moving seamlessly between both. It's incredibly charming! When they got stuck in their coding project, they demanded immediate attention, but if my answer ran a bit too long, they'd have already wandered to the next thought.

We had awesome field trips to the BuzzFeed and Facebook² offices, heard from folks in tech about their experiences getting into it. BuzzFeed had a panel of people with all sorts of backgrounds entering the field, told us about challenges faced by women in tech, and answered all sorts of questions. Facebook's free food, snacks, and office design probably made the students more excited about being future engineers than anything I did that year. Whatever does the job, I guess!

As the semester came to a close, we said our goodbyes. I was thankful for the experience and hopeful I might have helped. At that point, I had the sense that teaching younger students was likely not in my wheelhouse. I wanted to get better at it but also knew a trained educator is really what this program needed.

Meanwhile, the Election...

You almost forgot all of this is happening in 2016, didn't you? All this time I had been thinking about "How do I do more good in the world?" I was also facing a feeling of impending doom. The U.S., arguably the world's first successful experiment in multicultural democracy, was faced with the possibility of heading towards nationalistic, isolationist, and racist populism. Election anxiety was getting the best of me, an experience that was proving to be very, very common.

In August, the answer came to me: DevProgress, a group of volunteers in tech working in loose partnership with the Hillary Clinton campaign to help with progressive causes. DevProgress was effectively a Slack organization, Trello board, and GitHub organization of interested individuals budding off and forming projects with some level of community. Some worked on apps to help organize carpools to vote, others worked with artists making Hillary-inspired art, etc. I applied to join, and by September, I had joined in earnest.

As someone living in the US for 6 years at that point, I cared deeply about where the country is headed. Yet as a visa-holder, I was not allowed to donate to political campaigns, which is traditionally what someone anxious about the election could do. I could, however, volunteer my skills for a campaign.

As someone working in a closed-source company, my involvement in DevProgress can be seen very starkly in that brief period in 2016.

I contributed to a few of these projects, and spent countless nights especially helping on a project called "I like Hillary, but...", aimed at combatting what we perceived as misinformation and innuendo about Hillary Clinton³.

Working with the talented folks of DevProgress was one of the few ways I managed my anxieties in 2016. Things were uncertain, and they were scary—but I could tell myself that I'm doing my best. In retrospect I still wonder if I did my best, or if there's any small action by one person that could have had a butterfly-effect on the whole election. In the midst of it all, though, it was a particularly good coping mechanism.

Volunteering with DevProgress was always going to end abruptly on November 8, but I was hoping it wouldn't end devastatingly too. It did, but a lot of the experience is something I keep with me always: Working with a diverse group of passionate people and knowing that my tech skills had value for social issues.

While DevProgress itself became defunct, one of its main leaders Brady Kriss founded Ragtag, which in many ways became the spiritual successor of DevProgress. They're always looking for volunteers, I suggest you join!

For me, part of regrouping and picking the pieces after 2016 involved a lot of soul searching. I wasn't ready to face post-Election realities yet, so I held off on joining Ragtag in earnest. But the experience of writing code for good, coupled with another transformative experience that happened around the same time (which I'll talk about next), lead me down what became my passion for the coming few years.

Building Websites for Good

Around the time I started looking into DevProgress in 2016, I came across a call for volunteers. A non-profit called Out in Tech was launching a new hackathon-type event:
Digital Corps. Their announcement read:

Out in Tech is teaming up with Squarespace to build websites for ten (10) organizations fighting to protect LGBTQ+ rights in their home countries, from Pakistan to Botswana.

They announced a one-day event where teams of techy volunteers would build a website for an activist organization working to protect the rights of LGBTQ+ people in their area. I forward this to a few friends, adding "That's pretty interesting. I'm considering signing up.."

The event was scheduled for a Saturday in September. I wasn't quite sure how impactful building a website would be, but the idea of doing good using tech appealed to me. They also expressed a need for Arabic speakers, which made it all the more compelling to me to try and help.

That day, what quickly became apparent as my team started working on our website, was just how impactful an online presence can be. These activists were all either in countries where it was illegal or extremely dangerous to be LGBTQ, in incredibly underserved areas, or both.

A website is a safe way for both the brave activists to reach their communities, and for at-risk individuals to get the information they need. A website is a bold assertion of existence in countries that silence and deny the existence of citizens that do not conform. In many cases, the websites we build will be the first affirming resource written in a given native language or dialect. Those websites, with their content written by local activists, will often be the first resource from a local perspective on issues of sexual orientations and gender identity. Those websites, however small they might seem, end up countering the narrative that the modern LGBTQ movement is merely a Western movement.

For people in the west, amplifying voices of local activists is also important to counter homonationalism in Western countries.

Lifting up those voices is important, meaningful, consequential work. I felt that deeply.

The Digital Corps become an army

A few months into 2017, an opportunity came knocking. I was still buried deep in the pits of despair following the 2016 election, too anxious to think about politics and activism. Out in Tech sent out a call asking people to sign up to volunteer on the organizing side.

Digital Crops stood out to me as a dream project I'd love to work on: It is globally minded, fairly activist and political, yet not necessarily directly engaging with the realities of the US political climate which I was hoping not to have to think about it all the time.

And just like that (fast-forwarding a few weeks), I became part of the first group of organizers taking on Digital Corps as a repeatable long-term program. We ended up partnering with Automattic the creators of WordPress. Automattic proved to be the best partner a non-profit can ask for: they offered our partner organizations free Premium and Business hosting and tech support, and they offered our volunteers excellent support as they built these websites, sending Automatticians to provide front-line support as we're building these sites.

We set up the organizing team into federated groups of organizers each working with one or more activists on understanding their needs prior to the event. We figured out requirements and collected anything we might need: reference information, content, photos, etc., ahead of the event, then worked on selecting volunteers and getting them up-to-speed on the day of.

My first ever project was working with the Massachusetts Transgender Political Coalition (MTPC) on creating a Name & Gender Marker Change Guide for people navigating this incredibly difficult process in Massachusetts. The result was gorgeous and thoughtful, in no small part to Rita de Almeida a volunteer on the team. Rita is also an artist, UX designer, and researcher. She soon also became the goddess of Digital Corps, heading the entire organizing effort.

The first event I helped organize in June 2017 was a success. It gave me great confidence that the "magic" I felt working on one website was repeatable, and with it the potential for a positive impact on the activists around it.

Things weren't perfect right away, we were still trying to understand: how to scope projects so they fit in a day; what level of maintenance should we expect the organizations to do themselves; what level of support should we offer them; and more. We tried working with a large multinational organization to extend their Drupal/Salesforce pipeline and quickly learned that self-contained projects have a way higher chance to succeed than incremental pieces in large projects.

We also learned that WordPress powers 30% of the world's websites for a reason: It's maintainable and accessible for all users, while still highly customizable by web developers.

With that, from 2017 until 2020, I've worked on 10 or so of these events (Digital Corps tries to hold 3-4 events/year across the world). Each of those events is quite a big production: From logistical event planning, marketing, volunteer recruitment and selection, to finding and vetting partner organizations, working with them on their requirements, and making sure we have the needed requisites to build them a high-quality presence.

Working with these organizations over the last few years was among the most meaningful work I had done.

I worked with Roopbaan, Bangladesh's first and only LGBT magazine. It was founded by hero and activist Xulhaz Mannan. Mannan first published Roopbaan in 2014, and the magazine rose to prominence (as well as infamy) in Bangladesh. Xulhaz was murdered in 2016, along with fellow activist and Roopbaan member K Mahbub Rabbi, abruptly putting a stop to the magazine. I learned all this from Mannan's activist friends and colleagues, who wanted to continue the Roopbaan effort online where it could not be silenced by those who wanted to.

I worked with the African Queer Youth Initiative on creating their website and online presence, and watched them grow their website and blog into a respected authoritative perspective on issues impacting queer youth in Africa.

I worked with Outcasts Tunisia in developing one of the first Arabic-language transgender-affirming resources in the Maghreb region.

Those, and many more, inspired me daily to the best I can do between working hours and on weekends. I took calls at odd times, working across many time zones, meeting new organizers and activists hoping to make their local communities better, against all odds.

I also met countless volunteers who are passionate about making a difference. Folks who decided to show up at 9:00 AM on a Saturday⁴ at some brightly-lit tech office to work on some websites. They're internationally-minded, many hailing from all over the world, and all passionate about giving back. I'm grateful today to call many of those people good friends.

What makes Digital Corps successful?

Volunteer efforts almost always suffer from friction, a lack of motivation, and fundamental challenges with initiative-taking. It's easy to sign up to do something, but it's just as easy to procrastinate doing it. Saying "I'll do this thing" feels good on its own, so when it comes time to reply to that e-mail or decide what project to work on, you might just wait for someone else to do it—or just postpone it a little.

From my experience, every volunteer-based organization I've seen suffers from this.

From my perspective, the Digital Corps program sidesteps this by making so much of the action self-contained within a single build day. We just ask our volunteers to show up on a given day (and have a certain skill set), and we count on them to work as they're energized by the colleagues around them.

Seeing these organizations and the impact associated with their work is incredibly meaningful. People are proud of the work they're done, rate the event highly in surveys, and will often apply for future events and tell their friends about it.

Effectively, we retain a highly motivated, highly skilled population of volunteers by shifting a lot of the day-to-day engagement work into organizers. The organizers in turn work to define a self-contained single-day deliverable for these volunteers.

Burning Out

One downside to the Digital Corps model is that it might be easy to burn out as an organizer. At least that's what happened to me. Working on something meaningful on tight deadlines with a rather short periodicity (once every few months) is a bit of an emotional rollercoaster. You get to know a set of activists and invest your relationship with them. You race against the clock figuring out requirements, structure, content, etc. It culminates in a huge event that hopefully goes swimmingly. Then you're back to square one.

The periodicity of it really affected me. Especially as I tried to balance work and other volunteer commitments.

My work with Digital Corps helped me survive and come to terms with the new realities of the post-2016 era. It spanned an apartment move, two jobs, and a few promotions. I joined Digital Corps when I was just about checked out with the world around me, and it snapped me out of it.

Yet as 2019 went by I was becoming more fatigued and checked out. It's a really weird experience when the thing that brought you back from detachment is making you more detached.

Part of this was that my first few years with Digital Corps involved me coasting at a fin-tech job then joining Google and ramping up. Ramping up isn't quite
the same as coasting, but there are overlaps with it often being less high-stakes which helps keeping volunteering involvement front-and-center. It felt easy to be an engaged volunteer coasting at work, but it was much harder trying to do both. It felt like my full-time job, though not quite a do-good profession in the same way, had its turn to come first.

Thinking Ahead

Today, I maintain some minimal involvement in Digital Corps. I keep wanting to explore being more active, but with the pandemic upon us in all its glory, it's now harder than ever to think about how to stay motivated with what you're already doing.

As I wrote this, I had questions on my mind that I hoped spelling this out would answer: What are the decision points for re-engaging now or later? What circumstances might need to be in place for me to re-engage in a different capacity?

I was disappointed the answer didn't come to me. I sent an early draft of this to friends hoping for perspective. Here are some proposed parameters:

Dive back in again and then accept the inevitability of burning out (again)
Angle for a career shift within Google to get paid to do "good" (not just do-no-harm work) work?
Does having a green card mean you can donate to political causes? If so, it might be time to take the rich person offramp and donate as a way of exercising your values.
Can you look for avenues in Digital Corps for a smaller amount of contribution?
Focus on a different area of volunteering so you can chase the high of being a newbie again and ride that wave until the next trough finally hits.

Embracing burn-out as a (sometimes) inevitable part of the process and working with it resonates. We all burn out and work, but it rarely means we're forced into early retirement.

Google, for its part, does have the Google.org Fellowship, where employees effectively take a sabbatical from their role to work with non-profits that need that work. This, and other types of sabbaticals, are definitely appealing. Working out the timing (especially when putting work on hold might slow down your career advancement) and figuring out the right trade-off remains something I haven't mastered there.

Donating should absolutely become part of the wider picture of wanting to do good. Donating competes with buying other goods for money, but it doesn't really compete on time with other commitments per se. If I can, I'd still love to do both.

I also need to answer questions about what and how much. Can I find smaller meaningful chunks that I contribute? Would those fit in the common volunteering model where it is harder to come up with self-contained tasks that need work than it is to do the work for those tasks? Will working in a new area re-inspire me in ways where I might have gotten numb⁵?

I'm still not sure what looks ahead. But soon enough the pendulum will swing from burnt out to restless, and the cycle will start over.

If my discussion on Digital Corps excites you, please consider applying for their coming events! Their mailing list will inform you of upcoming opportunities.

If this resonates, I'd love to hear what you think. Tweet at @EyasSH. Or, be sure to sign up to get updates on future articles.

A more advanced course with Python was also available. ↩
They made sure to tell all their friends they were going to the Instagram offices. It was cooler. ↩
We later found out how much of that misinformation was carefully amplified through bot rings, fake profiles, and strategic online advertising targeting both right- and left-leaning voters. ↩
This is tech we're talking about, so 9:00 AM on a weekend is early. ↩
Analogous to compassion fatigue, if you will. ↩

Errors, Assertions, and Test Coverage

Eyas — Tue, 28 Jan 2020 00:54:52 +0000

I wrote about the Unexpected Lessons from 100% Test Coverage, inspired by people like Isaac Schlueter, who challenge the conventional wisdom that 100% coverage is bad. Part of this post involved reflections on thinking about delineating assertions and errors that I thought warranted being split up into its own post.

Lines that check for error and conditions and make assertions are notoriously some of the hardest to cover during testing. Attempting to test these can be very helpful in understanding our code.

schema-dts transforms RDF triples into an intermediate representation of "classes", and eventually TypeScript typings. The codebase for these transformations included checking for a number of boundary conditions, and a number of other assertions. When looking at my test coverage, I saw that most of those cases were not being tested.

In an effort to testing them, I looked at some use cases and saw some common themes.

A: Intra-function Impossibility

There was a line in my code that my tests never covered. It looked something like this:

// Top-level module:
const wellKnownTypes = [ ... ];  // Top-level var with more than one "Type".
const dataType = ...;

function ForwardDeclareClasses(topics: ReadonlyArray<TypedTopic>): ClassMap {
  const classes = new Map<string, Class>();
  for (const wk of wellKnownTypes) {
    classes.set(wk.subject.toString(), wk);
  }
  classes.set(dataType.subject.toString(), wk);
  for (const topic of topics) {
    // ...
  }

  if (classes.size === 0) {
    throw new Error('Expected Class topics to exist.');
  }

  return classes;
}

As you can see just reading this function and knowing: (classes.size === 0) will never happen. For one, there’s a classes.set(K, V) a few sections above. And we set a few other key-value pairs from this wellKnownTypes array that is hard-coded to always have a set number of elements.

One can try to understand the point of this error: It could be to show that none of the RDF triples we got are turned into classes (in which case we might want to compare classes.size with wellKnownTypes.length + 1 instead). Alternatively, it can be a poorly-placed verification for when we had less confidence that we were building classes properly, and had no clear concept of such “well known” types.

In my case, creating a map with just the well knows seemed fine. If the ontology is empty or missing data, we’ll likely find it at earlier steps or later ones. And the error gives no clear context as to what’s going wrong. So in my case, the answer was to kill it:

-  if (classes.size === 0) {
-    throw new Error('Expected Class topics to exist.');
-  }

B: Inter-Function Assertion

Another error I saw looked like this:

function ForwardDeclareClasses(topics: ReadonlyArray<TypedTopic>): ClassMap {
  // ...
  // ...
  for (const topic of topics) {
    if (!IsClass(topic)) continue;
    // ...
    classes.set(
        topic.Subject.toString(), new Class(topic.Subject, /* ... */));
  }
  // ...
  return classes;
}
function BuildClasses(topics: ReadonlyArray<TypedTopic>, classes: ClassMap) {
  for (const topic of topics) {
    if (!IsClass(topic)) continue;
    const cls = classes.get(topic.Subject.toString());
    if (!cls) {
      // Line (A) is next ---------------
      throw new Error(/**... class should have been forward declared */);
    }
    toClass(cls, topic, classes);
  }

In this case, line A never happened (and the (!cls) condition was always false). This should make sense: ForwardDeclareClasses literally checks if a TypedTopic satisfies IsClass(), and, if so, unconditionally adds it to the map. BuildClasses assert that a topic matching IsClass exists in the map.

One way to get test coverage for this line is to export BuildClasses and test it. But that seems like it goes against the spirit of making the codebase better. A better approach is ask what this line is trying to do:

Interlude: Expectations, Errors, and Assertions

Sometimes, we assert things because they either…

are error conditions that can happen in the wild due to poor data or input,
shouldn’t happen, and if they did it’s a sign of a bug in our code, or
shouldn’t happen, and if they did it’s a sign of cosmic radiation.

I decided to differentiate these. If my test coverage report complains about an uncovered...

error condition, I should test it. If I can’t, I should refactor my code to make it testable;
assertion that might indicate a bug, some code golf to make these always run might be in order (more on that in a bit);
assertion that is totally impossible, maybe I should delete it.

I refer to #1 as an error condition. Test these. For assertions, I often found that the line between #2 and #3 is often the function boundary (this isn’t always true). Cases of intra-function assertions (like case study A above) seem so useless that we’re better off removing them. Cases of inter-function assertions (like this case) seem useful enough to stay.

The Fix

I found that this distinction is not just helpful to split hairs: it’s also very helpful for someone reading the code: Is this error something that can happen in normal operation? or, is it a sign of a bug? I decided to make this clear:

Normal error conditions: if + throw, or similar.
Bug assertions: assert and assertXyz variants.

With that, I ended up with this change:

+import {ok} from 'assert';
+
 // ... more imports ...
+const assert: <T>(item: T|undefined) => asserts
+

 function BuildClasses(topics: ReadonlyArray<TypedTopic>, classes: ClassMap) {
   for (const topic of topics) {
     if (!IsClass(topic)) continue;

     const cls = classes.get(topic.Subject.toString());
+    assert(cls);
-    if (!cls) {
-      throw new Error(/**... class should have been forward declared */);
-    }
     toClass(cls, topic, classes);
   }
 }

Here, thinking about covering a line of code fundamentally helped me communicate what my code does more effectively. A lot of the “moving things around” that I had to do is very much semantic code golf (that happily happens to give a better test coverage score), but I’d like to think it’s net positive.

Read more about this experience here

Cover image is in in the public domain, licensed under CC0.

Use trackBy in Angular ngFor Loops and MatTables

Eyas — Mon, 25 Nov 2019 23:37:06 +0000

A missing trackBy in an ngFor block or a data table can often result in hard-to-track and seemingly glitchy behaviors in your web app. Today, I’ll discuss the signs that you need to use trackBy. But first—some context:

More often than not, you’ll want to render some repeated element in Angular. You’ll see code that looks like this:

<ng-container *ngFor="let taskItem of getTasks(category)">

In cases where the ngFor is looping over the results of a function that are created anew each time (e.g. an array being constructed using .map and .filter), you’ll run into some issues.

Every time the template is re-rendered, a new array is created with new elements. While newly-created array elements might be equivalent to the previous ones, Angular uses strict equality on each element to determine how to handle it.

In cases where the elements are an object type, strict equality will show that each element of the array is new. This means that a re-render would have a few side-effects:

Angular determines all the old elements are no longer a part of the block, and
- destroys their components recursively,
- unsubscribes from all Observables accessed through an | async pipe from within the ngFor body.
Angular finds newly-added elements, and
- creates their components from scratch,
- subscribing to new Observables (i.e. by making a new HTTP request) to each Observable it accesses via an | async pipe.

This also leads to a bunch of state being lost:

selection state inside the ngFor is lost on re-render,
state like a link being in focus, or a text-box having filled-in values, would go away.
if you have side-effects in your Observable pipes, you’ll see those happen again.

The Solution

trackBy gives you the ability to define custom equality operators for the values you’re looping over. This allows Angular to better track insertions, deletions, and reordering of elements and components within an ngFor block.

<ng-container *ngFor="let taskItem of getTasks(category); trackBy: trackTask">

... where trackTask is a TrackByFunction<Task>, such as:

  trackTask(index: number, item: Task): string {
    return `${item.id}`;
  }

If you run into situations where you have Observables that are being subscribed more often that you expect, seemingly duplicate HTTP calls being made, DOM elements that lose interaction and selection state sporadically, you might be missing a trackBy somewhere.

It’s not just For Loops

Any kind of data source that corresponds to repeated rows or items, especially ones that are fetched via Observables, should ideally allow you to use trackBy-style APIs. Angular’s MatTable (and the more general CdkTable) support their own version of trackBy for that purpose.

Since a table’s dataSource will often by an Observable or Observable-like source of periodically-updating data, understanding row-sameness across updates is very important.

Symptoms of not specifying trackBy in data tables are similar to ngFor loops; lost selections and interaction states when items are reloaded, and any nested components rendered will be destroyed and re-created. The experience of trackBy-less tables might be even worse, in some cases: changing a table sort or filtering will often be implemented at the data source level, causing a new array of data to render once more, with all the side effects entailed.

For a table of tasks fetched as Observables, we can have:

<table mat-table [dataSource]="category.tasksObs" [trackBy]="trackTask">

Where trackTask is implemented identically as a TrackByFunction<Task>.

This article originally appeared in my blog.

Learning by Implementing: Observables

Eyas — Wed, 02 Oct 2019 20:03:34 +0000

Sometimes, the best way to learn a new concept is to try to implement it. With my journey with reactive programming, my attempts at implementing Observables were key to to my ability to intuit how to best use them. In this post, we’ll be trying various strategies of implementing an Observable and see if we can make get to working solution.

I’ll be using TypeScript and working to implement something similar to RxJS Observables in these examples, but the intuition should be broadly applicable.

First thing’s first, though: what are we trying to implement? My favorite way or motivating Observables is by analogy. If you have some type, T, you might represent it in asynchronous programming as Future<T> or Promise<T>. Just as futures and promises are the asynchronous analog of a plain type, an Observable<T> is the asynchronous construct representing as collection of T.

The basic API for Observable is a subscribe method that takes as bunch of callbacks, each triggering on a certain event:

interface ObservableLike<T> {
  subscribe(
      onNext?: (item: T) => void,
      onError?: (error: unknown) => void,
      onDone?: () => void): Subscription;
}

interface Subscription {
  unsubscribe(): void;
}

With that, let’s get to work!

First Attempt: Mutable Observables

One way of implementing an Observable is to make sure it keeps tracks of it’s subscribers (in an array) and have the object send events to listeners as they happen.

For the purpose of this and other implementations, we’ll define an internal representation of a Subscription as follows:

interface SubscriptionInternal<T> {
  onNext?: (item: T) => void;
  onError?: (error: unknown) => void;
  onDone?: () => void;
}

Therefore, we could define an Observable as such:

(BAD)

class Observable<T> implements ObservableLike<T> {
  private readonly subscribers: Array<SubscriptionInternal<T>> = [];

  triggerNext(item: T) {
    this.subscribers.forEach(sub => sub.onNext && sub.onNext(item));
  }

  triggerError(err: unknown) {
    this.subscribers.forEach(sub => sub.onError && sub.onError(err));
  }

  triggerDone() {
    this.subscribers.forEach(sub => sub.onDone && sub.onDone());
    this.subscribers.splice(0, this.subscribers.length);
  }

  subscribe(
    onNext?: (item: T) => void,
    onError?: (error: unknown) => void,
    onDone?: () => void
  ): Subscription {
    const subInternal: SubscriptionInternal<T> = {
      onNext,
      onError,
      onDone
    };

    this.subscribers.push(subInternal);
    return {
      unsubscribe: () => {
        const index = this.subscribers.indexOf(subInternal);
        if (index !== -1) {
          onDone && onDone(); // Maybe???
          this.subscribers.splice(index, 1);
        }
      }
    };
  }
}

This would be used as follows:

// Someone creates an observable:
const obs = new Observable<number>();
obs.triggerNext(5);
obs.triggerDone();

// Someone uses an observable
obs.subscribe(next => alert(`I got ${next}`), undefined, () => alert("done"));

Problems

There are a few fundamental problems going on here:

The implementer doesn’t know when subscribers will start listening, and thus won’t know if triggering an event will be heard by no one,
Related to the above, this implementation always creates hot observables; the Observable can start triggering events immediately after creation, depending on the creator, and
Mutable: Anyone who receives the Observable can call triggerNext, triggerError, and triggerDone on it, which would interfere with everyone else.

There are some limitations of the current implementation: can error multiple times, a “done” Observable can trigger again, and an Observable can move back and forth between “done”, triggering, and “errored” states. But state tracking here wouldn’t be fundamentally more complicated. We also need to think more about errors in the callback, and what the effect of that should be on other subscribers.

Second Attempt: Hot Immutable Observables

Let’s first solve the mutability problem. One approach is to pass a ReadonlyObservable interface around which hides the mutating methods. But any downstream user up-casting the Observable could wreck havoc, never mind plain JS users who just see these methods on an object.

A cleaner approach in JavaScript is to borrow from the Promise constructor’s executor pattern, where the constructor is must be passed a user-defined function that defines when an Observable triggers:

(Still BAD)

class Observable<T> implements ObservableLike<T> {
  private readonly subscribers: Array<SubscriptionInternal<T>> = [];

    constructor(
      executor: (
        next: (item: T) => void,
        error: (err: unknown) => void,
        done: () => void
      ) => void
    ) {
      const next = (item: T) => {
        this.subscribers.forEach(sub => sub.onNext && sub.onNext(item));
      };

      const error = (err: unknown) => {
        this.subscribers.forEach(sub => sub.onError && sub.onError(err));
      };

      const done = () => {
        this.subscribers.forEach(sub => sub.onDone && sub.onDone());
        this.subscribers.splice(0, this.subscribers.length);
      };

      executor(next, error, done);
    }

  subscribe(
    onNext?: (item: T) => void,
    onError?: (error: unknown) => void,
    onDone?: () => void
  ): Subscription {
    const subInternal: SubscriptionInternal<T> = {
      onNext,
      onError,
      onDone
    };

    this.subscribers.push(subInternal);
    return {
      unsubscribe: () => {
        const index = this.subscribers.indexOf(subInternal);
        if (index !== -1) {
          onDone && onDone(); // Maybe???
          this.subscribers.splice(index, 1);
        }
      }
    };
  }
}

Much better! We can use this as such:

// Someone creates an observable:
const obs = new Observable<number>((next, error, done) => {
  next(5);
  done();
});

// Someone uses an observable
obs.subscribe(next => alert(`I got ${next}`), undefined, () => alert("done"));

This cleans up the API quite a bit. But in this example, calling this code in this order will still cause the subscriber to see no events.

Good Examples

We can already use this type of code to create helpful Observables:

// Create an Observable of a specific event in the DOM.
function fromEvent<K extends keyof HTMLElementEventMap>(
  element: HTMLElement,
  event: K
): Observable<HTMLElementEventMap[K]> {
  return new Observable<HTMLElementEventMap[K]>((next, error, done) => {
    element.addEventListener(event, next);
    // Never Done.
  });
}

const clicks: Observable<MouseEvent> = fromEvent(document.body, "click");

(More examples in original article.)

Even these examples have some issues: they keep running even when no one is listening. That’s sometimes fine, if we know we’ll only have one Observable, or we’re sure callers are listening and so tracking that state is unnecessary overhead, but it’s starting to point to certain smells.

Bad Examples

One common Observable factory is of, which creates an Observable that emits one item. The assumption being that:

const obs: Observable<number> = of(42);
obs.subscribe(next => alert(`The answer is ${next}`));

... would work, and result in “The answer is 42” being alerted. But a naive implementation, such as:

function of<T>(item: T): Observable<T> {
  return new Observable<T>((next, error, done) => {
    next(item);
    done();
  };
}

... would result in the event happening before anyone has the chance to subscribe. Tricks like setTimeout work for code that subscribes immediately after, but is fundamentally broken if we want to generalize this to someone who subscribes at a later point.

The case for Cold Observables

We can try to make our Observables lazy, meaning they only start acting on the world once subscribed to. Note that by lazy I don’t just mean that a shared Observable will only start triggering once someone subscribes to it — I mean something stronger: an Observable will trigger for each subscriber.

For example, we’d like this to work properly:

const obs: Observable<number> = of(42);
obs.subscribe(next => alert(`The answer is ${next}`));
obs.subscribe(next => alert(`The second answer is ${next}`)); 
setTimeout(() => {
  obs.subscribe(next => alert(`The third answer is ${next}`)); 
}, 1000);

Where we get 3 alert messages the contents of the event.

Third Attempt: Cold Observables (v1)

(Still BAD)

type UnsubscribeCallback = (() => void) | void;

class Observable<T> implements ObservableLike<T> {
  constructor(
    private readonly executor: (
      next: (item: T) => void,
      error: (err: unknown) => void,
      done: () => void
    ) => UnsubscribeCallback
  ) {}

  subscribe(
    onNext?: (item: T) => void,
    onError?: (error: unknown) => void,
    onDone?: () => void
  ): Subscription {
    const noop = () => {};
    const unsubscribe = this.executor(
      onNext || noop,
      onError || noop,
      onDone || noop
    );

    return {
      unsubscribe: unsubscribe || noop
    };
  }
}

In this attempt, each Subscription will run the executor separately, triggering onNext, onError, and onDone for each subscriber as needed. This is pretty cool! The naive implementation of of works just fine. I also snuck in a pretty simple method to allow us to add cleanup logic to our executors.

fromEvent would benefit from that, for example:

// Create an Observable of a specific event in the DOM.
function fromEvent<K extends keyof HTMLElementEventMap>(
  element: HTMLElement,
  event: K
): Observable<HTMLElementEventMap[K]> {
  return new Observable<HTMLElementEventMap[K]>((next, error, done) => {
    element.addEventListener(event, next);
    // Never Done.

    return () => {
      element.removeEventListener(event, next);
    };
  });
}

The nice thing about this is that we remove our listeners when a particular subscriber unsubscribes. Except now, we open as many listeners as subscribers. That might be okay for this one case, but we’ll want to figure out how to let users “multicast” (reuse underlying events, etc.) when they want to.

We still haven’t figured out error handling and proper cleanup and error handling. For example:

It is generally regarded that a subscription that errors is closed (just like how throwing an error while iterating over a for loop will terminate that loop)
When a subscriber unsubscribes, we should probably get that “onDone” event.
When there’s an error, we should probably do some cleanup.

Better Cold Observables

Here’s a re-implementation of subscribe that might satisfy these conditions:

class Observable<T> implements ObservableLike<T> {
  constructor(
    private readonly executor: (
      next: (item: T) => void,
      error: (err: unknown) => void,
      done: () => void
    ) => UnsubscribeCallback
  ) {}

  subscribe(
    onNext?: (item: T) => void,
    onError?: (error: unknown) => void,
    onDone?: () => void
  ): Subscription {
    let dispose: UnsubscribeCallback;
    let running = true;
    const unsubscribe = () => {
      // Do not allow retriggering:
      onNext = onError = undefined;

      onDone && onDone();
      // Don't notify someone of "done" again if we unsubscribe.
      onDone = undefined;

      if (dispose) {
        dispose();
        // Don't dispose twice if we unsubscribe.
        dispose = undefined;
      }

      running = false;
    };

    const error = (err: unknown) => {
      onError && onError(err);
      unsubscribe();
    };

    const done = () => {
      unsubscribe();
    };

    const next = (item: T) => {
      try {
        onNext && onNext(item);
      } catch (e) {
        onError && onError(e);
        error(e);
      }
    };

    dispose = this.executor(next, error, done);

    // We just assigned dispose. If the executor itself already
    // triggered done() or fail(), then unsubscribe() has gotten called
    // before assigning dispose.
    // To guard against those cases, call dispose again in that case.
    if (!running) {
      dispose && dispose();
    }

    return {
      unsubscribe: () => unsubscribe()
    };
  }
}

Using Observables

Taking the “Better Cold Observables” example, let’s see how we can use Observables. These are described in the original article, and they include:

Useful factories, like throwError or zip
Useful operators like map or filter

Conclusion

I didn’t really try to sell you, dear reader, on why you should use Observables as helpful tools in your repertoire. Some of my other writing showing their use cases (here, here, and here) might be helpful. But really, what I wanted to demonstrate is some of the intuition on how Observables work. The implementation I shared isn’t a complete one, for that, you better consult with Observable.ts in the RxJS implementation. This implementation is notably missing a few things:

We could still do much better on error handling (especially in my operators)
RxJS observables include the pipe() method, which makes applying one or more of those operators to transform an Observable much more ergonomic
Lot’s of things here and there

See the original article in my blog

Modeling Schema.org JSON-LD in TypeScript: A Story in Four Parts

Eyas — Tue, 09 Jul 2019 04:33:15 +0000

Recently, I published schema-dts, an open source library that models JSON-LD Schema.org in TypeScript. A big reason I wanted to do this project is because I knew some TypeScript type system features, such as discriminated type unions, powerful type inference, nullability checking, and type intersections, present an opportunity to both model what Schema.org-conformant JSON-LD looks like, while also providing ergonomic completions to the developer.

I wrote a series of posts, describing some of the Structured Data concepts that lent themselves well to TypeScript’s type system, and those concepts that didn’t.

1. Modeling Schema.org Class Hierarchy using Structural Typing

Schema.org JSON-LD node objects are always typed (that is, they have a @type property that points to some IRI–a string–describing it). Given a @type you know all the properties that are defined on a particular object. Object types inherit from each other. For example, Thing in Schema.org has a property called name, and Person is a subclass of Thing that defines additional properties such as birthDate, and inherits all the properties of Thing such as name. Thing has other sub-classes, like Organization, with it’s own properties, like logo.

In the first installment, we end up uncovering a recursive TypeScript inheritance hierarchy we can use to model the full complexity of Schema.org class inheritance.

2. Schema.org Enumerations in TypeScript

When trying to model Enumerations, we looked at a ton of examples from the Schema.org website to discover that absolute IRIs or @context-relative IRIs are expected to model the value of an enumeration. But we also found out that Enumerations can be arbitrary nodes, and take part in the class hierarchy.

3. Schema.org DataType in TypeScript

The Schema.org DataType hierarchy is far richer than TypeScript’s type system can accommodate. In the third installment, we figured out what trade-offs can we make.

4. Class Properties and Special Cases

Properties -- all the stuff that actually lives within a JSON node -- turn out to be more complicated than we thought: they're all optional, they're all repeated, they can supersede one another, and then can subclass one another.

The End Result

The end result is schema-dts itself. We can create programmatic TypeScript definitions that express much of Schema.org. For example, the top-level Thing type in Schema.org can be represented as:

type ThingBase = {
    /** An additional type for the item, typically used for adding more specific types from external vocabularies in microdata syntax. This is a relationship between something and a class that the thing is in. In RDFa syntax, it is better to use the native RDFa syntax - the 'typeof' attribute - for multiple types. Schema.org tools may have only weaker understanding of extra types, in particular those defined externally. */
    "additionalType"?: URL | URL[];
    /** An alias for the item. */
    "alternateName"?: Text | Text[];
    /** A description of the item. */
    "description"?: Text | Text[];
    /** A sub property of description. A short description of the item used to disambiguate from other, similar items. Information from other properties (in particular, name) may be necessary for the description to be useful for disambiguation. */
    "disambiguatingDescription"?: Text | Text[];
    /** The identifier property represents any kind of identifier for any kind of {@link http://schema.org/Thing Thing}, such as ISBNs, GTIN codes, UUIDs etc. Schema.org provides dedicated properties for representing many of these, either as textual strings or as URL (URI) links. See {@link /docs/datamodel.html#identifierBg background notes} for more details. */
    "identifier"?: (PropertyValue | Text | URL) | (PropertyValue | Text | URL)[];
    /** An image of the item. This can be a {@link http://schema.org/URL URL} or a fully described {@link http://schema.org/ImageObject ImageObject}. */
    "image"?: (ImageObject | URL) | (ImageObject | URL)[];
    /** Indicates a page (or other CreativeWork) for which this thing is the main entity being described. See {@link /docs/datamodel.html#mainEntityBackground background notes} for details. */
    "mainEntityOfPage"?: (CreativeWork | URL) | (CreativeWork | URL)[];
    /** The name of the item. */
    "name"?: Text | Text[];
    /** Indicates a potential Action, which describes an idealized action in which this thing would play an 'object' role. */
    "potentialAction"?: Action | Action[];
    /** URL of a reference Web page that unambiguously indicates the item's identity. E.g. the URL of the item's Wikipedia page, Wikidata entry, or official website. */
    "sameAs"?: URL | URL[];
    /** A CreativeWork or Event about this Thing.. */
    "subjectOf"?: (CreativeWork | Event) | (CreativeWork | Event)[];
    /** URL of the item. */
    "url"?: URL | URL[];
};
/** The most generic type of item. */
export type Thing = ({
    "@type": "Thing";
} & ThingBase) | (Action | CreativeWork | Event | Intangible | MedicalEntity | Organization | Person | Place | Product);

View the entire series at https://blog.eyas.sh/tag/schema.org

react-schemaorg: Strongly-typed Schema.org JSON-LD for React

Eyas — Wed, 06 Feb 2019 19:37:20 +0000

I made this library (and the related, lower level schema-dts) after I was working inserting JSON-LD in a site and noticing a lack of TypeScript-y, developer-friendly way to do it. Most of the tooling to validate Schema.org JSON-LD is to use these online "validators", but that breaks (or lengthens) the development write-build-test loop.

google / react-schemaorg

Type-checked Schema.org JSON-LD for React

react-schemaorg

Easily insert valid Schema.org JSON-LD in your React apps.

This library provides <JsonLd> for plain React apps, and helmetJsonLdProp() for use with <Helmet>.

Uses schema-dts for Schema.org TypeScript definitions.

Note: This is not an officially supported Google product.

Usage

Install react-schemaorg and your desired version of schema-dts:

npm install schema-dts
npm install react-schemaorg

Plain React Usage

To insert a simple JSON-LD snippet:

import { Person } from "schema-dts";
import { JsonLd } from "react-schemaorg";
export function GraceHopper() {
  return (
    <JsonLd<Person>
      item={{
        "@context": "https://schema.org",
        "@type": "Person",
        name: "Grace Hopper",
        alternateName: "Grace Brewster Murray Hopper",
        alumniOf: {
          "@type": "CollegeOrUniversity",
          name: ["Yale University", "Vassar College"],
        },
        knowsAbout: ["Compilers", "Computer Science"],
      }}
    />
  );

…

View on GitHub

This library allows you to insert JSON-LD that is type-checked and offers completions, etc., just by taking advantage of TypeScript's type system. react-schemaorg is a simple wrapper around schema-dts, that inserts <script type="application/ld+json"> tags and requires you to specify a top-level "@context".

Schema-DTS: Schema.org Linked Data Types & Code Generation

Eyas — Tue, 15 Jan 2019 15:58:10 +0000

Schema-DTS is a Google Open Source Initiative project that makes it possible to write JSON-LD literals in the Schema.org vocabulary.

google / schema-dts

JSON-LD TypeScript types for Schema.org vocabulary

schema-dts

JSON-LD TypeScript types for Schema.org vocabulary.

schema-dts provides TypeScript definitions for Schema.org vocabulary in JSON-LD format. The typings are exposed as complete sets of discriminated type unions, allowing for easy completions and stricter validation.

This repository contains two NPM packages:

schema-dts-gen Providing a command-line tool to generate TypeScript files based on a specific Schema version and layer.
schema-dts Pre-packaged TypeScript typings of latest Schema.org schema, without pending and other non-core layers.

Note: This is not an officially supported Google product.

Usage

To use the typings for your project, simply add the schema-dts NPM package to your project:

npm install schema-dts

Then you can use it by importing "schema-dts".

Root context

You will usually want your top-level item to include a @context, like https://schema.org. In order for your object type to accept this property, you can augment it with WithContext, e.g.:

import { Person, WithContext

…

View on GitHub

It consists of two NPM packages:

schema-dts - A set of default types and properties representing schema.org, including deprecated types, pending types, and certain extension vocabularies.
schema-dts-gen - A CLI that dynamically generates TypeScript definitions given a Schema.org layer (e.g. "schema" or "all layers") with custom flags and configuration (e.g. a custom "@context", skipping deprecated types and properties, etc.).

DEV Community: Eyas

Server Environments

What is a Deployment Environment

What consistently connected entails

A Case Study

Principles of consistent connectedness

A Brief Tour of the Unity Editor

A Tour of the Editor

Project View

Scene View

Hierarchy View

Inspector

Inspector Editors

Property Drawers

Edit Mode and Play Mode

What the Editor Does (and Doesn't) Give You

Unity Input System, from Basic Principles

Programmatic Use

Input Actions

Using Unity Events

Inputs and Frame-rate Independence

Final Thoughts

6 Software Practices to Keep, Shed, and Adopt in Unity

Shed: Keep it all in code

Adopt: Write your own Custom Editors and Property Drawers

Keep: Singletons tend to be a code smell

Shed: Get comfortable with some globals

Adopt: The Inspector can be your injection framework

Keep: Unit- and Integration Tests are Always Good

Closing

Basic Concepts in Unity for Software Engineers

Scenes

Game Objects

Components (& MonoBehaviors)

Assets

Prefabs

Nested Prefabs

Serialization & Deserialization

Conclusion

My Journey Through Tech Volunteering: Anticipation, Passion, Burnout, and Looking Ahead

Finding my path in Volunteering in Tech

Your *N*th reminder that teaching is hard

Meanwhile, the Election...

Building Websites for Good

The Digital Corps become an army

What makes Digital Corps successful?

Burning Out

Thinking Ahead

Errors, Assertions, and Test Coverage

A: Intra-function Impossibility

B: Inter-Function Assertion

Interlude: Expectations, Errors, and Assertions

The Fix

Use trackBy in Angular ngFor Loops and MatTables

The Solution

It’s not just For Loops

Learning by Implementing: Observables

First Attempt: Mutable Observables

Problems

Second Attempt: Hot Immutable Observables

Good Examples

Bad Examples

The case for Cold Observables

Third Attempt: Cold Observables (v1)

Better Cold Observables

Using Observables

Conclusion

Modeling Schema.org JSON-LD in TypeScript: A Story in Four Parts

1. Modeling Schema.org Class Hierarchy using Structural Typing

2. Schema.org Enumerations in TypeScript

3. Schema.org DataType in TypeScript

4. Class Properties and Special Cases

The End Result

react-schemaorg: Strongly-typed Schema.org JSON-LD for React

google / react-schemaorg

Type-checked Schema.org JSON-LD for React

react-schemaorg

Usage

Plain React Usage

Schema-DTS: Schema.org Linked Data Types & Code Generation

Your Nth reminder that teaching is hard