Skip to content

Commit

Permalink
Cleanup rebased (Sovereign-Labs#351)
Browse files Browse the repository at this point in the history 
* WIP: update RPC

* Improve ledger RPC UX

* Add ledger RPC docs to demo-rollup

* Replace Module::genesis with  trait

* Improve docs for Alpha (Sovereign-Labs#345)

* Fix module RPC namespaces

* Update docs

* Minor fix to nft readme

* Capitalize Module System everywhere

* minor doc improvements

* Apply small nits from review

* Change header sizes for demo-nft readme

* Fix nits in const-rollup-config

* Fix nits in demo-simple-stf readme

* Fix nits in full-node readmes

* Fix nits in mod-impl examples and int tests

* Fix nits in module-implementations READMEs

* Fix nits in module system sub dir READMEs

---------

Co-authored-by: Cem Özer <cem@sovlabs.io>

* Revert "Replace Module::genesis with  trait"

This reverts commit 6bb36dc.

* Blanket implement genesis for Modules

---------

Co-authored-by: Cem Özer <cem@sovlabs.io>
  • Loading branch information
@preston-evans98 @cemozerr
preston-evans98 and cemozerr authored May 29, 2023
1 parent fef717e commit 34364b7
Show file tree
Hide file tree
Showing 30 changed files with 362 additions and 324 deletions.
16 changes: 8 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,29 +52,29 @@ it has no built-in notion of state, accounts, tokens, and the like. This means t
a rollup as easy as deploying a smart contract. So, we've built out an additional set of tools for defining your state transition function
called the Module System.

At the heart of the module system is the package [`sov-modules-api`](./module-system/sov-modules-api/). This package defines
At the heart of the Module System is the package [`sov-modules-api`](./module-system/sov-modules-api/). This package defines
a group of core traits which express how functionality implemented in separate modules can be combined into a `Runtime`
capable of processing transactions and serving RPC requests. It also defines macros for implementing most of these traits.
For many applications, defining your state transition function using the module system should be as simple as picking
For many applications, defining your state transition function using the Module System should be as simple as picking
some modules off the shelf and defining a struct which glues them together.
To deliver this experience, the module system relies on a set of common types and traits that are used in every module. The
To deliver this experience, the Module System relies on a set of common types and traits that are used in every module. The
`sov-modules-api` crate defines these traits (like `Context` and `MerkleTreeSpec`) and types like `Address`.

On top of the module API, we provide a [state storage layer](./module-system/sov-state/) backed by a [Jellyfish Merkle Tree](https://github.com/penumbra-zone/jmt)
and a bunch of helpful utilities for working with stateful transactions. Finally, we provide a set of modules implementing common
blockchain functionality like `Accounts`, and fungible `Tokens`.

For more information on the Module System, see its [README](./module-system/README.md). You can also find a tutorial on
implementing and deploying a custom module [here](./examples/demo-nft-module/README.md)
implementing and deploying a custom module [here](./examples/demo-nft-module/README.md).

### The Full Node

The final component of this repository is The Full Node, which is a client implementation capable of running any
rollup which implements the Rollup Interface. The Full Node provides an easy way to deploy
The final component of this repository is the Full Node, which is a client implementation capable of running any
rollup that implements the Rollup Interface. The Full Node provides an easy way to deploy
and run your rollup. With the default configuration, it can automatically store chain data in its database,
serve RPC requests for chain data and application state, and interact with the DA layer to sync its state and send transactions.
While the full node implementation should be compatible with custom state transition functions, it is currently only tested for
rollups built with the module system. If you encounter any difficulties running the full node, please reach out or open an
rollups built with the Module System. If you encounter any difficulties running the full node, please reach out or open an
issue! All of the core developers can be reached via [Discord](https://discord.gg/kbykCcPrcA).

## Getting Started
Expand All @@ -88,7 +88,7 @@ state transition, and [`demo-rollup`](./examples/demo-rollup/), which shows how
get a complete rollup implementation.

If you want even more control over your rollup's functionality, you can implement a completely custom State Transition Function
without using the module system. You can find a tutorial [here](./examples/demo-simple-stf/).
without using the Module System. You can find a tutorial [here](./examples/demo-simple-stf/).

### Adding a new Data Availability Layer

Expand Down
6 changes: 3 additions & 3 deletions examples/const-rollup-config/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

In Sovereign, many state transition functions require consensus critical configuration. For example, rollups on Celestia
need to configure a namespace which they check for data. This consensus critical configuration needs to be available
to packages at compile time, so that it is baked into the binary which is fed to the zkvm. Otherwise, a malicious
prover might be able to overwrite this configuration at runtime and create valid-looking proofs which were run
to packages at compile time, so that it is baked into the binary which is fed to the ZKVM. Otherwise, a malicious
prover might be able to overwrite this configuration at runtime and create valid-looking proofs that were run
over the wrong namespace.

This package demonstrates how you can accomplish such configuration. You can see its usage in the [`main` function of demo-rollup](../demo-rollup/main.rs)
This package demonstrates how you can accomplish such configuration. You can see its usage in the [`main` function of demo-rollup](../demo-rollup/src/main.rs).
120 changes: 58 additions & 62 deletions examples/demo-nft-module/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to Create a New Module Using the Module System

## Understanding the Module System
### Understanding the Module System

The Sovereign Software Development Kit (SDK) includes a [Module System](../../module-system/README.md),
which serves as a catalog of concrete and opinionated implementations for the rollup interface.
Expand All @@ -11,19 +11,18 @@ These modules are the fundamental building blocks of a rollup and include:
- **Application-level logic**: This is akin to smart contracts on Ethereum or pallets on Polkadot.
These modules often use state, modules-API, and macros modules to simplify their development and operation.

## Creating a Non-Fungible Token (NFT) Module
### Creating a Non-Fungible Token (NFT) Module

In this tutorial, we will focus on developing an application-level module. Users of this module will be able to mint
unique tokens, transfer them to each other, or burn them. Users can also check the ownership of a particular token. For
simplicity, each token represents only an ID and won't hold any metadata.

# Getting Started
## Getting Started

## Structure and dependencies
### Structure and dependencies

The Sovereign SDK provides [module-template](../../module-system/module-implementations/module-template/README.md).
It provides a boilerplate which can be customised.
The purpose of each rust module is explained later.
The Sovereign SDK provides a [module-template](../../module-system/module-implementations/module-template/README.md),
which is boilerplate that can be customised to easily build modules.

```
Expand All @@ -46,12 +45,12 @@ sov-modules-api = { git = "https://github.com/Sovereign-Labs/sovereign.git", bra
sov-modules-macros = { git = "https://github.com/Sovereign-Labs/sovereign.git", branch = "main" }
```

## Establishing the Root Module Structure
### Establishing the Root Module Structure

A module is a distinct crate that implements the `sov_modules_api::Module` trait
and defines its own change logic based on input messages.
A module is a distinct crate that implements the `sov_modules_api::Module` trait. Each module
has private state, which it updates in response to input messages.

## Module definition
### Module definition

NFT module is defined as the following:

Expand Down Expand Up @@ -81,26 +80,22 @@ This module includes:
1. **Address**: Every module must have an address, like a smart contract address in Ethereum. This ensures that:
- The module address is unique.
- The private key that generates this address is unknown.
2. **State** attributes: In this case, the state attributes are the admin's address and a map of token IDs to owner
2. **State attributes**: In this case, the state attributes are the admin's address and a map of token IDs to owner
addresses.
For simplicity, the token ID is an u64.
3. **Optional module reference**: This is used if the module needs to refer to another module.

## State and Context
### State and Context

### State
#### State

`State` values are stored in a Merkle Tree and can be read and written from there.
The module struct itself doesn't hold values but indicates what values it operates in a working set.
The `WorkingSet` populates the data.
The provided state implementation from the `sov-state` crate uses
the [Jellyfish Merkle Tree](https://github.com/penumbra-zone/jmt) (JMT).
`#[state]` values declared in a module are not physically stored in the module. Instead, the module definition
simply declares the _types_ of the values that it will access. The values themselves live in a special struct
called a `WorkingSet`, which abstracts away the implementation details of storage. In the default implementation, the actual state values live in a [Jellyfish Merkle Tree](https://github.com/penumbra-zone/jmt) (JMT).
This separation between functionality (defined by the `Module`) and state (provided by the `WorkingSet`) explains
why so many module methods take a `WorkingSet` as an argument.

The state operates in full node and zero-knowledge modes.
In full node mode, the entire Merkle tree is maintained and modified, while in zero-knowledge mode,
the state only provides access to Merkle proofs for leaves that were modified in a batch.

### Context
#### Context

The `Context` trait allows the runtime to pass verified data to modules during execution.
Currently, the only required method in Context is sender(), which returns the address of the individual who initiated
Expand All @@ -111,21 +106,13 @@ Storage, digital Signatures, and Addresses. The Spec trait allows rollups to eas
VMs. By being generic over a Spec, a rollup can ensure that any potentially SNARK-unfriendly cryptography can be easily
swapped out.

# Implementing `sov_modules_api::Module` trait

Before we start implementing the `Module` trait, there are several preparatory steps to take:
## Implementing `sov_modules_api::Module` trait

1. Add new dependencies:
[serde](https://serde.rs/),
[borsh](https://github.com/near/borsh-rs),
[sov-state](../../module-system/sov-state/README.md)
2. Define a 'native' feature flag to separate logic that isn't needed in zero-knowledge mode.
3. Define `Call` messages.
4. Define `Config`.
### Preparation

## Preparation
Before we start implementing the `Module` trait, there are several preparatory steps to take:

1. Define `native` feature in `Cargo.toml`:
1. Define `native` feature in `Cargo.toml` and add additional dependencies:

```toml
[dependencies]
Expand All @@ -147,11 +134,12 @@ Before we start implementing the `Module` trait, there are several preparatory s
This step is necessary to optimize the module for execution in ZK mode, where none of the RPC-related logic is
needed.
Zero Knowledge mode uses a different serialization format, so serde is not needed.
The `sov-state` module maintains same logic, so its `native` flag only enabled in that case.
The `sov-state` module maintains the same logic, so its `native` flag is only enabled in that case.

2. Define `Call` messages, which are used to change the state of the module.
2. Define `Call` messages, which are used to change the state of the module:

```rust
// in call.rs
use sov_modules_api::Context;

#[cfg_attr(feature = "native", derive(serde::Serialize), derive(serde::Deserialize))]
Expand All @@ -174,17 +162,24 @@ Before we start implementing the `Module` trait, there are several preparatory s

```

3. Define Config. In this case, config will contain admin and initial tokens:
As you can see, we derive the `borsh` serialization format for these messages. Unlike most serialization libraries,
`borsh` guarantees that all messages have a single "canonical" serialization, which makes it easy to reliably
hash and compare serialized messages.

3. Create a `Config` struct for the genesis configuration. In this case, the admin address and initial token distribution
are configurable:

```rust
// in lib.rs
pub struct NonFungibleTokenConfig<C: Context> {
pub admin: C::Address,
pub owners: Vec<(u64, C::Address)>,
}
```

# Stub implementation of the Module trait
## Stub implementation of the Module trait

Plug together all types and features
Plugging together all types and features, we get this `Module` trait implementation in `lib.rs`:

```rust
impl<C: Context> Module for NonFungibleToken<C> {
Expand Down Expand Up @@ -213,16 +208,17 @@ impl<C: Context> Module for NonFungibleToken<C> {
}
```

# Implementing state change logic
## Implementing state change logic

## Initialization
### Initialization

Initialization is performed by the genesis method,
Initialization is performed by the `genesis` method,
which takes a config argument specifying the initial state to configure.
Since it modifies state, genesis also takes a working set as an argument.
Genesis is called only once, during the rollup deployment.
Since it modifies state, `genesis` also takes a working set as an argument.
`Genesis` is called only once, during the rollup deployment.

```rust
// in genesis.rs
impl<C: Context> NonFungibleToken<C> {
pub(crate) fn init_module(
&self,
Expand Down Expand Up @@ -256,9 +252,9 @@ impl<C: Context> Module for NonFungibleToken<C> {
}
```

## Call message
### Call message

First, need to implement actual logic of handling different cases, let's add `mint`, `transfer` and `burn` methods:
First, we need to implement actual logic of handling different cases. Let's add `mint`, `transfer` and `burn` methods:

```rust

Expand Down Expand Up @@ -318,7 +314,7 @@ impl<C: Context> NonFungibleToken<C> {
}
```

And then map it in the trait implementation:
And then make them accessible to users via the `call` function:

```rust
impl<C: Context> Module for NonFungibleToken<C> {
Expand All @@ -340,10 +336,10 @@ impl<C: Context> Module for NonFungibleToken<C> {
}
```

## Enabling Queries
### Enabling Queries

We also want other modules to be able to query the owner of a token, so we add a public method for that. This method is only available to modules,
but is not currently exposed via RPC.
We also want other modules to be able to query the owner of a token, so we add a public method for that.
This method is only available to other modules: it is not currently exposed via RPC.

```rust
impl<C: Context> NonFungibleToken<C> {
Expand All @@ -359,19 +355,19 @@ impl<C: Context> NonFungibleToken<C> {
}
```

# Testing
## Testing

To make sure that module is implemented correctly, integration tests are recommended, so all public APIs work as
expected.
Integration tests are recommended to ensure that the module is implemented correctly. This helps confirm
that all public APIs function as intended.

For testing temporary storage is needed, so `temp` feature for `sov-state` module is required:
Temporary storage is needed for testing, so we enable the `temp` feature of `sov-state` as a `dev-dependency`

```toml
[dev-dependencies]
sov-state = { git = "https://github.com/Sovereign-Labs/sovereign.git", branch = "main", features = ["temp"] }
```

Here is boilerplate for NFT module integration tests
Here is some boilerplate for NFT module integration tests:

```rust
use demo_nft_module::call::CallMessage;
Expand Down Expand Up @@ -434,9 +430,9 @@ fn transfer() {
}
```

# Plugging in the rollup
## Plugging in the rollup

Now this module can be added to rollup's Runtime:
Now this module can be added to rollup's `Runtime`:

```rust
#[derive(Genesis, DispatchCall, MessageCodec)]
Expand All @@ -453,7 +449,7 @@ pub struct Runtime<C: Context> {
}
```

And then this runtime can be used in the State Transition Function runner to execute transactions.
And then this `Runtime` can be used in the State Transition Function runner to execute transactions.
Here's an example of how to do it with `AppTemplate` from `sov-default-stf`:

```rust
Expand All @@ -473,5 +469,5 @@ Here's an example of how to do it with `AppTemplate` from `sov-default-stf`:
}
```

`AppTemplate` uses runtime to dispatch call during execution of `apply_batch` method.
More details on how to setup rollup is available in [demo-rollup documentation](../demo-rollup/README.md)
The `AppTemplate` uses `runtime` to dispatch calls during execution of the `apply_batch` method.
Detailed instructions on how to set up a rollup can be found in the [`demo-rollup` documentation](../demo-rollup/README.md).
8 changes: 4 additions & 4 deletions examples/demo-prover/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@ It contains known security flaws and numerous inefficiencies.
## What is it?

This demo shows how to integrate RiscZero prover into rollup workflow.
This code reads blocks from Celestia, executes them and proves it inside RiscZero ZK VM.
This code reads blocks from Celestia, executes them and inside the RiscZero ZKVM, and creates a proof of the result.

This package implements the same logic as [`demo-rollup`](../demo-rollup/), but it splits the logic between
the "host" and "guest" (prover and zk-circuit) to create actual zk-proofs. This separation makes it slightly
harder to follow at first glance, so we recommend diving into the `demo-rollup` before attempting to use this package.

## Getting Started

1. Make sure Celestia light node is running as described in [Demo Rollup README](../demo-rollup/README.md)
2. Execute `cargo run`
1. Make sure Celestia light node is running as described in [Demo Rollup README](../demo-rollup/README.md).
2. Execute `cargo run`.

## Development

[IDE integration](./ide_setup.md) described in separate document.

# License
## License

Licensed under the [Apache License, Version
2.0](../../LICENSE).
Expand Down
18 changes: 7 additions & 11 deletions examples/demo-rollup/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This is a demo full node running a simple Sovereign SDK rollup on [Celestia](htt

## What is it?

This demo shows how to integrate a state-transition function with a DA layer and a Zkvm to create a full
This demo shows how to integrate a State Transition Function with a DA layer and a ZKVM to create a full
zk-rollup. The code in this repository corresponds to running a full-node of the rollup, which executes
every transaction. If you want to see the logic for _proof generation_, check out the [demo-prover](../demo-prover/)
package instead.
Expand All @@ -31,14 +31,14 @@ communicate with the DA layer's RPC endpoints.

If you're using Celestia as your DA layer, you can follow the instructions at the end
of this document to set up a local full node, or connect to
a remote node. Whichever option you pick, simply place the connection
information in the `rollup_config.toml` file and it will be
a remote node. Whichever option you pick, simply place the URL and authentication token
in the `rollup_config.toml` file and it will be
automatically picked up by the node implementation.

### Step 2: Initialize the State Transition Function

The next step is to initialize your state transition function. If it implements the StateTransitionRunner interface, you can use that
for easy initialization.
The next step is to initialize your state transition function. If it implements the [StateTransitionRunner](../../rollup-interface/src/state_machine/stf.rs)
interface, you can use that for easy initialization.

```rust
let mut stf_runner = NativeAppRunner::<Risc0Host>::new(rollup_config);
Expand Down Expand Up @@ -72,16 +72,12 @@ related to the chain's history - batches, transactions, receipts, etc.
This is a prototype. It contains known vulnerabilities and should not be used in production under any
circumstances.

## Celestia Integration

The current prototype runs against Celestia-node version `v0.7.1`. This is the version used on the `arabica` testnet
as of Mar 18, 2023.

## Getting Started

### Set up Celestia

Sync a Celestia light node running on the Arabica testnet
The current prototype runs against Celestia-node version `v0.7.1`. This is the version used on the `arabica` testnet
as of Mar 18, 2023. To get started, you'll need to sync a Celestia light node running on the Arabica testnet

1. Clone the repository: `git clone https://github.com/celestiaorg/celestia-node.git`.
1. `cd celestia-node`
Expand Down
Loading

0 comments on commit 34364b7

Please sign in to comment.