Review 1
Score
1 (weak accept)
Text
This work proposes SCEW a programming framework for P2P (web)
applications that allows for the manipulation of individual assets in
the presence of byzantine faults. This is achieved by encoding the state
of each asset via a Convergent Replicated Data Type that provides atomic
register semantics via a BFT-consensus algorithm.
The paper defines the abstract interfaces of the logical components of
the framework, exemplifying its use via a simple, yet clear, use case.
It also includes an experimental study showing that the proposed
framework can be used to build (simple) applications that scale up to
100 nodes.
The paper is well written and addresses a timely and interesting topic,
at the intersection between smart-contracts/block-chain, CRDTs and
programming models.
My main criticism towards the paper is that the choice of NOT providing
support for transactions involving multiple assets should be supported
by strong experimental evidence that "the pain is worth the gain". In
fact, the paper argues that the choice of limiting the programming model
allows for efficient synchronization. However, it does not present any
experimental data to quantify the gains (e.g., in terms of latency or
scalability) that are enabled by their proposed approach with respect to
alternative systems that do not impose these limitations of the
programming model.
Another baseline that would be interesting to include in the
experimental study is FSolidM, in order to quantify the benefits
deriving from reusing the transition results as the state of the
register.
Other comments & Knitpicks:
- The CvRDT acronym in the abstract is used before being defined.
-
In page 3, left column, when discussing the benefits of assets/tickets
(i.e., enabling synchronization at the individual level), it would be
good to mention also the limitations deriving from this choice.
-
In the description of Atomic Register CvRDTs, page 3, I found it
unclear how/why the merge method "enables peers to discover new
proposals".
-
A few lines below you write: "Paying attention to size, the atomic
register CvRDT can be implemented with the same storage requirements
and efficiency as Tickets"
=> This claim requires a citation or some additional discussion.
-
The evaluation section is very vague on the implementation of the
framework. E.g., which quorum protocol was used to implement the
atomic register?
Review 2
Score
1 (weak accept)
Text
This work investigates an interesting application of smart contracts in
P2P web applications. The proposed framework nicely integrates prior
technologies (e.g., BFT consensus, atomic register, primitive contracts)
to provide a reasonable abstraction with moderate performance penalty.
The key enabling component, in my opinion, is the atomic register CvRDT.
I’m not an expert in the application domain, but I think the problem is
interesting to the workshop, write-up is well-written, and the
evaluation is reasonable. Hence, I recommend an accept.
The reason that I only recommend a weak accept is due to the following
reasons. I encourage the authors to address them so that the work is
more accessible for non-expert like me.
-
The targeting application is not clear to me. The evaluation shows ~3
sec delay. I'm not sure clients are willing to suffer this kind of
delay in using web app.
-
BMachines is a potentially useful abstraction But again without a
clear picture of targeting applications, I can't evaluate whether it
is necessary to have such a strong guarantees.
-
Even though the authors cite Lamport's original work on atomic
registers, the usage/semantic doesn't seem to match what Lamport has
in mind. In particular, the authors seem to imply that there are
several registers and need to use a consensus protocol to merge them.
This was not supported in the original semantic. For a cleaner
discussion on the semantic of atomic registers, I recommend “Sharing
Memory Robustly in Message-Passing Systems” JACM '95. Clearly
identifying what is provided and adapted will help readers graph the
key idea.
-
The way the CvRDT is described reminds me the a recent work
“FabricCRDT: A Conflict-Free Replicated Datatypes Approach to
Permissioned Blockchains” Middleware 2019. It'd be useful.
-
Evaluation is sufficient for the workshop; however, I'd want to see
more discussion to help readers. For example, is there any random
network delay? Is 60ms a reasonable number for 4G? Is 1 tx/s a
reasonable rate? (This seems too small for modelling “real-world
interaction”) How big is the contract? (Is there any data exchanged
involved?) How are malicious and crash behavior modeled? It seems to
me that if malicious faults tampered the other parts (instead of
violating only the smart contract part), the delay could potentially
be larger?
Review 3
Score
1 (weak accept)
Text
This paper entitled "SCEW: Programmable BFT-Consensus with Smart
Contracts for Client-Centric P2P Web Applications" proposed a framework
to build simple decentralized P2P web-based applications that don't have
very latency-critical requirements and can't afford the blockchain
storage/computational cost. The framework allows the developer to
implement some application logic on top of pre-defined CRDT-like data
types to do the merging, however after consensus (a quorum) is reached.
This hides the BFT logic which is embedded in a smart contract-like
abstraction.
Strengths: the idea is nice and timely, and the paper is well written
Weakness: the guarantees and assumptions are not clear as compared to
blockchain/CRDT solutions.
I liked this paper. Although it leaves many unanswered questions (which
would need 14pages), the idea is worth publishing and the paper will
spark nice discussion at PAPOC. The paper is also well-written and
self-contained. In case the paper is accepted, please consider tackling
the fixable comments below and prepare your answers for the talk :)
Major things that must not be overlooked (at least in a future longer
paper): the proposed solution requires some assumptions and does not
guarantee the same security as blockchains. These are not defined nor
discussed. In addition, some details like the consensus used in merging
is very important and deserves more emphasis (in a longer version). The
evaluation does not compare with state of the art solutions.
More detailed comments:
Abstract
-
Well, not all of them: Contemporary systems provide fault tolerance
and consistency by using Conflict-free Replicated Data Types for
synchronization.
Intro:
-
Please consider more State of the Art works (especially on
blockchain/BFT part)
-
On which blockchain platform smart contracts are experimented?
Experimental results indicate that SCEW can support interac- tive
collaboration in client-centric P2P web apps, supporting networks with
up to 100 users, keeping latencies below 3.2 and 2 seconds for 99% of
all transactions in scenarios with and without Byzantine faults
respectively.
-
This is very web-application-dependent: in some applications 1sec is
too high (e.g., post editing&commenting, postcard apps, ..): keeping
latencies below 3.2 and 2 seconds for 99% of all transactions in
scenarios
Section 2
-
Some verbs or commas are missing: The first application Loyalty
Programs demonstrates the use of BFT in protecting shared loyalty
points
-
Loyalty Programs: it works, but it is not really a decentralization
demanding application. Loyalty credits are often low and the client
usually trust the merchant to handle it in a database.
-
This application is also not very useful in a decentralized manner
because a common issue in "physical" things is the guarantee of their
state (undamaged) or tracking (delivered or not?). For this reason, a
central authority works well because it can ask for pre-credit,
penalize the malicious users or sentence them before a court... The
application is OK, technically, but I would look for more realistic
applications.
Section 3:
-
What are managers? Say simply replicas or processes: Replica managers
- internet -> Internet
-
This is not accurate, see the delta-version of CRDTs by Almeida et al.
(Delta state replicated data types): Note that attention should be
paid at design time to minimize the size of the CvRDT, as the entire
state is sent over the network during synchronization
-
This is debatable in the community, just say it is building block, or
at the core of, or part of, ...: Blockchains [31] are an important
application for (BFT) consensus.
-
Please fix the phrase: When executed, smart contracts can read and
write state to the ledger
-
date -> time: making it computationally in- feasible to change the
contents of blocks or their order at a later date.
-
Any may others : Typical choices for consensus algorithms include
Proof-of-Work (PoW) and BFT, depending on the context [31].
-
Not accurate, only full nodes need to store the blockchain: peers must
also store the en- tire blockchain to be able to validate the state of
the ledger.
-
Tickets are not known and provide different guarantees than
blockchains. This deserves more technical details/summary about how
Tickets works and what are the guarantees vs blockchains
Section 4
- Please add a typical workflow in the overview
-
Problem definition is missing. Please say exactly what is the problem
you solve. This will lead to defining the guarantees you maintain vs
blockchains
-
I don't know what security guarantees and assumptions you assume? If
the asset is managed individually, where is the role of consensus? How
do you guarantee immutability/non-repudiation? : This behavior is
desirable, as assets can be synchronized on an individual level,
limiting the size of the state that is exchanged during
synchronization.
-
How do you do this important part? The consensus protocol is
implemented by the merge method,
-
BMachines section is too compressed. What matters more is really how
integrity is achieved across the different BMachines or peer versions?
- Please use another term: BFT machinery
Section 5
- Evaluation lacks any comparison with other solutions
Section 6
-
How is it related to your work not smart contracts? This stands in
contrast with smart contracts, which all perform the same computation
and reach a consensus on the result of a single invocation.
-
Again, this is not accurate since you can use delta CRDTs by Almeida
et al.: However, this does not work well with a state-based approach
where the entire object, including superfluous fields, needs to be
sent over the network during synchronization.
-
Missing related works: Atomic Register CvRDTs vs MV-register; smart
contracts vs Primitive contracts vs Tickets; BFT consensus vs SCEW
consensus (in the "merge")