NoFlo is a simple flow-based programming implementation for Node.js. From WikiPedia:

In computer science, flow-based programming (FBP) is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.

Developers used to the Unix philosophy should be immediately familiar with FBP:

This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

It also fits well in Alan Kay's original idea of object-oriented programming:

I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages (so messaging came at the very beginning -- it took a while to see how to do messaging in a programming language efficiently enough to be useful).

NoFlo has been written in CoffeeScript for simplicity. The system is heavily inspired by J. Paul Morrison's book Flow-Based Programming.

Currently NoFlo is still in quite early stages. It has already been used in some real-world applications, but the small number of available components still limits the utility of the system.

NoFlo requires a reasonably recent version of Node.js, and some npm packages. You can install everything needed by a simple:

$ npm link

NoFlo is available from GitHub under the MIT license.

There are two ways to use NoFlo:

• Independent: Building the whole control logic of your software as a NoFlo graph, and running it with the noflo tool
• Embedded: Using NoFlo as a library and calling some NoFlo graphs whenever your software needs workflows

File line count using embedded NoFlo:

$ coffee ./examples/linecount/count.coffee somefile.txt

File line count as an individual NoFlo application:

$ noflo
NoFlo>> load examples/linecount/count 

Simple "Hello, world" web service with Basic authentication using embedded NoFlo:

$ coffee ./examples/http/hello.coffee

Then just point your browser to http://localhost:8003/. Note that this example needs to have connect NPM package installed. Username is user and password is pass.

• Component: individual, pluggable and reusable piece of software. In this case a NoFlo-compatible CommonJS module
• Graph: the control logic of a FBP application, can be either in programmatical or file format
• Inport: inbound port of a component
• Network: collection of processes connected by sockets. A running version of a graph
• Outport: outbound port of a component
• Process: an instance of a component that is running as part of a graph

A component is the main ingredient of flow-based programming. Component is a CommonJS module providing a set of input and output port handlers. These ports are used for connecting components to each other.

NoFlo processes (the boxes of a flow graph) are instances of a component, with the graph controlling connections between ports of components.

Functionality a component provides:

• List of inports (named inbound ports)
• List of outports (named outbound ports)
• Handler for component initialization that accepts configuration
• Handler for connections for each inport

Minimal component written in CoffeeScript would look like the following:

noflo = require "noflo"

class Forwarder extends noflo.Component
    description: "This component receives data on a single input port and sends the same data out to the output port"

    constructor: ->
        # Register ports
        @inPorts =
            in: new noflo.Port()
        @outPorts =
            out: new noflo.Port()

        @inPorts.in.on "data", (data) =>
            # Forward data when we receive it.
            # Note: send() will connect automatically if needed
            @outPorts.out.send data

        @inPorts.in.on "disconnect", =>
            # Disconnect output port when input port disconnects
            @outPorts.out.disconnect()

exports.getComponent = ->
    new Forwarder()

This example component register two ports: in and out. When it receives data in the in port, it opens the out port and sends the same data there. When the in connection closes, it will also close the out connection. So basically this component would be a simple repeater.

You can find more examples of components in the components folder shipping with NoFlo.

Components should aim to be reusable, to do one thing and do it well. This is why often it is a good idea to split functionality traditionally done in one function to multiple components. For example, counting lines in a text file could happen in the following way:

• Filename is sent to a Read File component
• Read File reads it and sends the contents onwards to Split String component
• Split String splits the contents by newlines, and sends each line separately to a Count component
• Count counts the number of packets it received, and sends the total to a Output component
• Output displays the number

This way the whole logic of the application is in the graph, in how the components are wired together. And each of the components is easily reusable for other purposes.

If a component requires configuration, the good approach is to set sensible defaults in the component, and to allow them to be overridden via an input port. This method of configuration allows the settings to be kept in the graph itself, or for example to be read from a file or database, depending on the needs of the application.

The components should not depend on a particular global state, either, but instead attempt to keep the input and output ports their sole interface to the external world. There may be some exceptions, like a component that listens for HTTP requests or Redis pub-sub messages, but even in these cases the server, or subscription should be set up by the component itself.

Being a flow-based programming environment, the main action in NoFlo happens through ports and their connections. There are five events that can be associated with ports:

• Attach: there is a connection to the port
• Connect: the port has started sending or receiving a data transmission
• Data: an individual data packet in a transmission. There might be multiple depending on how a component operates
• Disconnect: end of data transmission
• Detach: A connection to the port has been removed

It depends on the nature of the component how these events may be handled. Most typical components do operations on a whole transmission, meaning that they should wait for the disconnect event on inports before they act, but some components can also act on single data packets coming in.

When a port has no connections, meaning that it was initialized without a connection, or a detach event has happened, it should do no operations regarding that port.

NoFlo comes with a command shell that you can use to load, run and manipulate NoFlo graphs. For example, the line count graph that was explained in Component design could be built with the shell in the following way:

$ noflo
NoFlo>> new countlines
countlines>> add read ReadFile
countlines>> add split SplitStr
countlines>> add count Counter
countlines>> add display Output
countlines>> connect read out split in
countlines>> connect split out count in
countlines>> connect count count display in
countlines>> dot
digraph {
  read [shape=box]
  split [shape=box]
  count [shape=box]
  display [shape=box]
  read -> split[label='out']
  split -> count[label='out']
  count -> display[label='count']
}
countlines>> send read source somefile

You can run help to see all available NoFlo shell commands, and quit to get out of the shell.

In addition to the shell, NoFlo also comes with a web interface that allows loaded graphs to be monitored. To start it, load a graph into the NoFlo shell, and run:

>> startserver 8080

This will start the NoFlo monitor on port 8080 of your system, so browsers can connect to it with http://localhost:8080. You can also use another port number.

At the moment the monitor only displays the graph, showing the processes and connections between them. Real-time statistics of data flow, and support for visual graph editing are planned.

In addition to using NoFlo in embedded mode where you create the FBP graph programmatically (see example), you can also initialize and run graphs defined using a JSON file.

The NoFlo JSON files declare the processes used in the FBP graph, and the connections between them. They look like the following:

{
    "properties": {
        "name": "Count lines in a file"
    },
    "processes": {
        "Read File": {
            "component": "ReadFile"
        },
        "Split by Lines": {
            "component": "SplitStr"
        },
        ...
    },
    "connections": [
        {
            "data": "package.json",
            "tgt": {
                "process": "Read File",
                "port": "source"
            }
        },
        {
            "src": {
                "process": "Read File",
                "port": "out"
            },
            "tgt": {
                "process": "Split by Lines",
                "port": "in"
            }
        },
        ...
    ]
}

To run a graph file, you can either use the load command of the NoFlo shell, or do it programmatically:

noflo = require "noflo"
noflo.loadFile "example.json", (network) ->
    console.log "Graph loaded"
    console.log network.graph.toDOT()

NoFlo development happens on GitHub. Just fork the main repository, make modifications and send a pull request.

To run the unit tests you need nodeunit. Run the tests with:

$ nodeunit test/*

• Real-time status of the NoFlo graph via socket.io, see where data is flowing
• Web Workers based multiprocess runner
• Sockets-based multi-computer runner