fred/wallet
1
0
Fork 0
forked from cheng/wallet
Code Pull Requests Activity

clarified use of public broadcast channel and merkle patricia trees

Browse Source
This commit is contained in:
reaction.la 2024-02-26 03:21:53 +00:00
parent a6ae125e3d
commit 05da22896f
No known key found for this signature in database
1 changed files with 28 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
docs/manifesto/scalability.md
View File

@ -8,7 +8,7 @@ notmine: false
::: myabstract ::: myabstract
[abstract:]{.bigbold} [abstract:]{.bigbold}
Bitcoin does not scale to the required size. The Bitcoin reliable broadcast channel is a massively replicated public ledger of every transaction that ever there was, each of which has to be evaluated for correctness by every full peer. With recursive snarks, we can now instead have a massively replicated public sql index of private ledgers. Such a blockchain with as many transactions as bitcoin, will, after running for as long as Bitcoin, only occupy a few dozen megabytes of disk storage, rather than near a terabyte, and each peer and client wallet only has to evaluate the root recursive snark to prove the validity of every transaction that ever there was, including all those lost in the mists of time. Bitcoin does not scale to the required size. The Bitcoin reliable broadcast channel is a massively replicated public ledger of every transaction that ever there was, each of which has to be evaluated for correctness by every full peer. With recursive snarks, we can now instead have a massively replicated public SQL index of private ledgers. Such a blockchain with as many transactions as bitcoin, will, after running for as long as Bitcoin, only occupy a few dozen megabytes of disk storage, rather than near a terabyte, and each peer and client wallet only has to evaluate the root recursive snark to prove the validity of every transaction that ever there was, including all those lost in the mists of time.
::: :::
# Scaling, privacy, and recursive snarks # Scaling, privacy, and recursive snarks
@ -30,7 +30,7 @@ This explanation is going to require you to know what a graph,
vertex, edge, root, and leaf is, what a directed acyclic graph (dag) vertex, edge, root, and leaf is, what a directed acyclic graph (dag)
is, what a hash is, what a blockchain is, is, what a hash is, what a blockchain is,
and how hashes make blockchains possible. and how hashes make blockchains possible.
And what an sql index is and what it does, and what a primary sql index is and what it does. And what an SQL index is and what it does, and what a primary SQL index is and what it does.
You need to know what a transaction output is in the context of blockchains, You need to know what a transaction output is in the context of blockchains,
and what an unspent transaction output (utxo) is. and what an unspent transaction output (utxo) is.
Other terms will be briefly and cryptically explained as necessary. Other terms will be briefly and cryptically explained as necessary.
@ -91,13 +91,13 @@ of which the current root of the blockchain is the root hash.
## structs ## structs
A struct is simply some binary data laid out in well known and agreed format. A struct is simply some binary data laid out in well known and agreed format.
Almost the same thing as an sql row, except that Almost the same thing as an SQL row, except that an SQL
an sql row does not have a well known and agreed binary format, row does not have a well known and agreed binary format,
so does not have a well defined hash, and a struct is not so does not have a well defined hash, and a struct is not
necessarily part of an sql table, though obviously you can put a necessarily part of an SQL table, though obviously you can put a
bunch of structs of the same type in an sql table, and represent an bunch of structs of the same type in an SQL table, and represent an SQL
sql table as a bunch of structs, plus at least one primary index. table as a bunch of structs, plus at least one primary index.
An sql table is equivalent to a pile of structs, An SQL table is equivalent to a pile of structs,
plus at least one primary index of those structs. plus at least one primary index of those structs.
## merkle graphs and merkle trees ## merkle graphs and merkle trees
@ -129,12 +129,12 @@ If you publish information on a reliable broadcast channel,
everyone who looks at the channel is guaranteed to see it and to everyone who looks at the channel is guaranteed to see it and to
see the same thing, and if someone did not get the information see the same thing, and if someone did not get the information
that you were supposed to send over the channel, it is his fault, that you were supposed to send over the channel, it is his fault,
not yours. You performed the protocol correctly. not yours. You can prove you performed the protocol correctly.
A blockchain is a merkle chain *and* a reliable broadcast channel. A blockchain is a merkle chain *and* a reliable broadcast channel.
In Bitcoin, the reliable broadcast channel contains the entire In Bitcoin, the reliable broadcast channel contains the entire
merkle chain, which obviously does not scale, and suffers from a merkle chain, which obviously does not scale, and suffers from a
massive lack of privacy, so we have introduce the obscure massive lack of privacy, so we have to introduce the obscure
cryptographic terminology "reliable broadcast channel" to draw a cryptographic terminology "reliable broadcast channel" to draw a
distinction that does not exist in Bitcoin. In Bitcoin the merkle distinction that does not exist in Bitcoin. In Bitcoin the merkle
vertices are very large, each block is a single huge merkle vertex, vertices are very large, each block is a single huge merkle vertex,
@ -146,11 +146,11 @@ which is what is happening with Ethereum's use of recursive snarks.
So we need to structure the data as large dag of small merkle So we need to structure the data as large dag of small merkle
vertices, with all the paths through the dag for which we need to vertices, with all the paths through the dag for which we need to
generate proofs being logarithmic in the size of the contents of generate proofs being logarithmic in the size of the contents of
the reliable broadcast channel. the reliable broadcast channel and the height of the blockchain.
## merkle patricia tree ## merkle patricia tree
A merkle patricia tree is a representation of an sql index as a A merkle patricia tree is a representation of an SQL index as a
merkle tree. Each edge of a vertex is associated with a short merkle tree. Each edge of a vertex is associated with a short
bitstring, and as you go down the tree from the root (tree graphs bitstring, and as you go down the tree from the root (tree graphs
have their root at the top and their leaves at the bottom, just to have their root at the top and their leaves at the bottom, just to
@ -158,23 +158,26 @@ confuse the normies) you append that bitstring, and when you
reach the edge (hash) that points to a leaf, you have a bitstring reach the edge (hash) that points to a leaf, you have a bitstring
that corresponds to path you took through the merkle tree, and to that corresponds to path you took through the merkle tree, and to
the leading bits of the bitstring that make that key unique in the the leading bits of the bitstring that make that key unique in the
index. Thus the sql operation of looking up a key in an index index. Thus the SQL operation of looking up a key in an index
corresponds to a walk through the merkle patricia tree corresponds to a walk through the merkle patricia tree
guided by the key. guided by the key. So we can generate a recursive snark that proves
you found something in an index of the blockchain, or proves that
something, such as a previous spend of the output you want to spend,
does not exist in that index.
# Blockchain # Blockchain
Each block in the chain is an set of sql tables, represented as merkle dags. Each block in the chain is an set of SQL tables, represented as merkle dags.
So a merkle patricia tree and the structs that its leaf edges point So a merkle patricia tree and the structs that its leaf edges point
to is an sql table that you can generate recursive snarks for, to is an SQL table that you can generate recursive snarks for,
which can prove things about transactions in that table. We are which can prove things about transactions in that table. We are
unlikely to be programming the blockchain in sql, but to render unlikely to be programming the blockchain in SQL, but to render
what one is doing intelligible, it is useful to think and design in sql. what one is doing intelligible, it is useful to think and design in SQL.
So with recursive snarks you can prove that that your transaction So with recursive snarks you can prove that that your transaction
is valid because certain unspent transaction outputs were in the is valid because certain unspent transaction outputs were in the
sql index of unspent transaction outputs, and were recently spent SQL index of unspent transaction outputs, and were recently spent
in the index of commitments to transactions, without revealing in the index of commitments to transactions, without revealing
which outputs those were, or what was in your transaction. which outputs those were, or what was in your transaction.
@ -182,7 +185,7 @@ It is a widely shared public index. But what it is an index of is
private information about the transactions and outputs of those private information about the transactions and outputs of those
transactions, information known only to the parties of those transactions, information known only to the parties of those
transactions. It is not a public ledger. It is a widely shared transactions. It is not a public ledger. It is a widely shared
public sql index of private ledgers. And because it is a merkle public SQL index of private ledgers. And because it is a merkle
tree, it is possible to produce a single reasonably short recursive tree, it is possible to produce a single reasonably short recursive
snark for the current root of that tree that proves that every snark for the current root of that tree that proves that every
transaction in all those private ledgers was a valid transaction transaction in all those private ledgers was a valid transaction
@ -191,7 +194,7 @@ and every unspent transaction output is as yet unspent.
## performing a transaction ## performing a transaction
Oops, what I just described is a whole sequence of complete Oops, what I just described is a whole sequence of complete
immutable sql indexes, each new block a new complete index. immutable SQL indexes, each new block a new complete index.
But that would waste a whole lot of bandwidth. What you want is But that would waste a whole lot of bandwidth. What you want is
that each new block is only an index of new unspent transaction that each new block is only an index of new unspent transaction
outputs, and of newly spent transaction outputs, which spending outputs, and of newly spent transaction outputs, which spending
@ -236,8 +239,8 @@ registered, you can create a new unspent transaction output for
each transaction input to the failed transaction which effectively each transaction input to the failed transaction which effectively
rolls back the failed transaction. This time limit enables us to rolls back the failed transaction. This time limit enables us to
recover from failed transactions, and, perhaps, more importantly, recover from failed transactions, and, perhaps, more importantly,
enables us to clean up the mutable sql index that the immense enables us to clean up the mutable SQL index that the immense
chain of immutable sql indexes represents, and that the public chain of immutable SQL indexes represents, and that the public
broadcast channel contains. We eventually drop outputs that have broadcast channel contains. We eventually drop outputs that have
been committed to a particular transaction, and can then been committed to a particular transaction, and can then
eventually drop the commits of that output without risking eventually drop the commits of that output without risking
@ -248,8 +251,8 @@ public broadcast channel.
So that the public broadcast channel can eventually dump old So that the public broadcast channel can eventually dump old
blocks, and thus old spend events, every time we produce a new blocks, and thus old spend events, every time we produce a new
base level block containing new events (an sql index of new base level block containing new events (an SQL index of new
transaction outputs, and an sql index table with the same primary transaction outputs, and an SQL index table with the same primary
of spend commitments of past unspent transaction outputs to of spend commitments of past unspent transaction outputs to
transactions) we also produce a consolidation block, a summary transactions) we also produce a consolidation block, a summary
block that condenses two past blocks into one summary block, block that condenses two past blocks into one summary block,