Overview

This is an RFC on our approach to introducing basic well liquid/volume tracking to the Python protocol api.

The Python protocol api currently only tracks volume as far as the pipette, so the hardware controller knows how much to move the pipette's plunger for any given atomic pipetting action.

The recently introduced Thermocycler Module needs to know the volumes of the samples in its loaded labware in order to precisely heat and cool its wells. Now is the right time to consider how to track the position and volume of liquids on the deck of the OT2 at runtime.

Requirements

A successful first implementation should address the following goals:

• track the total liquid volume in every well on the deck through a protocol run
• produce identical results to the Protocol Designer's underlying step-generation liquid tracking
• provide opt-in functionality for PAPIv2 users that gracefully degrades (e.g. if initial well volume not supplied)

The implementation should also consider the following longer-term feature additions:

• tracking liquid identity, metadata, and volume ratios per well (ala Protocol Designer)
• tracking liquid properties, such as viscosity, evaporation, cohesion, volatility (not currently in Protocol Designer)
• Run time checks, potentially associated with other sensors or feedback

Proposal

This RFC proposes that we implement basic well-level volume tracking in the protocol api.

In opentrons/protocol-designer/step-generation (currently only used by Protocol Designer), lives all of the logic to track the state of liquids on the deck of the OT2 across protocol commands. It is important that whatever Python implementation we use not functionally diverge from the Javascript implementation.
We can enforce this relationship in a number of ways:

1. Create a method-to-method carbon-copy of step-generation in Python. Write robust tests that consume the same artifacts and enforce identical outputs.

• Pros:
  • changes to either implementation would be straight-forward to link to necessary changes in the other
  • writing shared regression/unit tests would be straight-forward, as they too could match one to one.
• Cons:
  • cost of maintaining two different modules in two different languages that perform the exact same task
  • could add development friction to updating either module, as every change would require 2x implementation resources

2. Create a Python module that accomplishes the same goals in a style that is closer to the rest of our Python logic. Write robust tests that consume the same artifacts and enforce identical outputs.

• Pros:
  • each module can maintain it's own coding style and approach that makes the most sense in it's language and context
  • regression/unit tests could still consume the same artifacts
  • less tight coupling across the stack (compared with option 1)
• Cons:
  • cost of maintaining two different modules in two different languages that perform the exact same task
  • could add development friction to updating either module, as every change would require 2x implementation resources

3. From the existing Python code, execute the step-generation logic in a Javascript interpretter, probably in a different thread running Node.

• Pros:
  • truly one source of truth for liquid state simulation
• Cons:
  • messiness of depending on multiple programming languages running the robot
  • because PD and the Robot Software would both rely on the same code, we'd either have to split it out into it's own versioned dependency, or deal with the consequences of possible discrepancies between platform releases (which undermines the purpose of this whole approach)

Given these three rough approaches, I think Option 2 is the best fit for our needs at the moment and will serve us going forward.

Option 3 increases the testing area of all our robot software, and has some serious implications surrounding tight coupling and implicit co-versioning across, PD, Run App, and robot software. Option 1 on the other-hand is less tightly coupled, but still has a hazy contract of implementation details. Our JS and Python have enough conventional differences, that I suspect we would be fighting from both sides to compromise whenever possible.

Option 2 is only as strong as the test suite that ensures parity between the two implementations. This feels like a fine place to assert sameness, as each implementation may develop as its domain requirements grow. The tests can serve as the contract that protects against cross-stack breaking changes. This brings up another relevant point. How does this effect our various versioning mechanisms?

As the tests in Option 2 only assert behavioral uniformity, not necessarily structural similarity, each project should be able to append functionality without "breaking" that contract. Editing existing functionality is a little bit tougher. For instance: if the JS step-generation were to be updated in a way that effected multichannel access rules, we'd want to make sure that the Python implementation would mirror that update. We're then faced with the possibility of people creating protocols in PD that simulate liquid state in the UI (via step-generation) differently than the robot software's run/simulation time representation of liquid state.

Without adding a new version for the step-generation module to the mix, we may be able to use PD's existing versioning with some new UI in the Run App. In PD, a Major or Minor bump constitutes a change that may be breaking (currently at the PD import file migration level). We could establish the convention that a change to step-generation that gets included into a release of PD always requires at least a Minor version bump. On the Run App side, we can surface an error (or warning), at simulate time, to express that the protocol will still function properly, but may differ from your expectations based on the PD's rendering of the protocol.

Discussion/Future Work

At its smallest scope this python implementation of liquid tracking would just have to cover aspirate, dispense, blowout, and potentially droptip. This approach would be sufficient in many cases to supply the thermocycler with accurate volume information at runtime. Those cases that it wouldn't cover, (e.g. initial liquid volume in TC wells > 0), are common and not adequately addressed with the minimal solution.

Because of this, I propose that our first implementation aim for basic parity with Protocol Designer's underlying step-generation liquid tracking. This similarly only requires coverage of aspirate, dispense, blowout, and droptip. The main difference is that instead of just tracking gross volume in wells, we also introduce the concept of liquid identity. This is the mechanism that allows PD to track mixtures/locations of reagents across a protocol. This sort of information paired with liquid metadata would allow for a rich runtime experience that is much more adaptable and extendable for features that are down the road.

This approach gives us much more leverage for creating connective tissue between our two protocol creating api's. For example, using a PD-esque UI to populate your starting deck state, and exporting that to be referenced in a Python protocol. This would save us from strict reliance on the inevitable suite of utility functions required for an average Python user to populate their deck and its labware with reagents at the beginning of every protocol.

The opposite direction could also be fruitful by allowing the robot software to send live updates of the current liquid state at runtime to clients in a consistent format to the way it is described under the hood in PD.

┆Issue is synchronized with this Wrike Task by Unito