From 05da22896f3f7be8c9816469bdf2a3531b082368 Mon Sep 17 00:00:00 2001 From: "reaction.la" Date: Mon, 26 Feb 2024 03:21:53 +0000 Subject: [PATCH] clarified use of public broadcast channel and merkle patricia trees --- docs/manifesto/scalability.md | 53 ++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 25 deletions(-) diff --git a/docs/manifesto/scalability.md b/docs/manifesto/scalability.md index 54620a3..da28069 100644 --- a/docs/manifesto/scalability.md +++ b/docs/manifesto/scalability.md @@ -8,7 +8,7 @@ notmine: false ::: myabstract [abstract:]{.bigbold} -Bitcoin does not scale to the required size. The Bitcoin reliable broadcast channel is a massively replicated public ledger of every transaction that ever there was, each of which has to be evaluated for correctness by every full peer. With recursive snarks, we can now instead have a massively replicated public sql index of private ledgers. Such a blockchain with as many transactions as bitcoin, will, after running for as long as Bitcoin, only occupy a few dozen megabytes of disk storage, rather than near a terabyte, and each peer and client wallet only has to evaluate the root recursive snark to prove the validity of every transaction that ever there was, including all those lost in the mists of time. +Bitcoin does not scale to the required size. The Bitcoin reliable broadcast channel is a massively replicated public ledger of every transaction that ever there was, each of which has to be evaluated for correctness by every full peer. With recursive snarks, we can now instead have a massively replicated public SQL index of private ledgers. Such a blockchain with as many transactions as bitcoin, will, after running for as long as Bitcoin, only occupy a few dozen megabytes of disk storage, rather than near a terabyte, and each peer and client wallet only has to evaluate the root recursive snark to prove the validity of every transaction that ever there was, including all those lost in the mists of time. ::: # Scaling, privacy, and recursive snarks @@ -30,7 +30,7 @@ This explanation is going to require you to know what a graph, vertex, edge, root, and leaf is, what a directed acyclic graph (dag) is, what a hash is, what a blockchain is, and how hashes make blockchains possible. -And what an sql index is and what it does, and what a primary sql index is and what it does. +And what an SQL index is and what it does, and what a primary SQL index is and what it does. You need to know what a transaction output is in the context of blockchains, and what an unspent transaction output (utxo) is. Other terms will be briefly and cryptically explained as necessary. @@ -91,13 +91,13 @@ of which the current root of the blockchain is the root hash. ## structs A struct is simply some binary data laid out in well known and agreed format. -Almost the same thing as an sql row, except that -an sql row does not have a well known and agreed binary format, +Almost the same thing as an SQL row, except that an SQL + row does not have a well known and agreed binary format, so does not have a well defined hash, and a struct is not -necessarily part of an sql table, though obviously you can put a -bunch of structs of the same type in an sql table, and represent an -sql table as a bunch of structs, plus at least one primary index. -An sql table is equivalent to a pile of structs, +necessarily part of an SQL table, though obviously you can put a +bunch of structs of the same type in an SQL table, and represent an SQL +table as a bunch of structs, plus at least one primary index. +An SQL table is equivalent to a pile of structs, plus at least one primary index of those structs. ## merkle graphs and merkle trees @@ -129,12 +129,12 @@ If you publish information on a reliable broadcast channel, everyone who looks at the channel is guaranteed to see it and to see the same thing, and if someone did not get the information that you were supposed to send over the channel, it is his fault, -not yours. You performed the protocol correctly. +not yours. You can prove you performed the protocol correctly. A blockchain is a merkle chain *and* a reliable broadcast channel. In Bitcoin, the reliable broadcast channel contains the entire merkle chain, which obviously does not scale, and suffers from a -massive lack of privacy, so we have introduce the obscure +massive lack of privacy, so we have to introduce the obscure cryptographic terminology "reliable broadcast channel" to draw a distinction that does not exist in Bitcoin. In Bitcoin the merkle vertices are very large, each block is a single huge merkle vertex, @@ -146,11 +146,11 @@ which is what is happening with Ethereum's use of recursive snarks. So we need to structure the data as large dag of small merkle vertices, with all the paths through the dag for which we need to generate proofs being logarithmic in the size of the contents of -the reliable broadcast channel. +the reliable broadcast channel and the height of the blockchain. ## merkle patricia tree -A merkle patricia tree is a representation of an sql index as a +A merkle patricia tree is a representation of an SQL index as a merkle tree. Each edge of a vertex is associated with a short bitstring, and as you go down the tree from the root (tree graphs have their root at the top and their leaves at the bottom, just to @@ -158,23 +158,26 @@ confuse the normies) you append that bitstring, and when you reach the edge (hash) that points to a leaf, you have a bitstring that corresponds to path you took through the merkle tree, and to the leading bits of the bitstring that make that key unique in the -index. Thus the sql operation of looking up a key in an index +index. Thus the SQL operation of looking up a key in an index corresponds to a walk through the merkle patricia tree -guided by the key. +guided by the key. So we can generate a recursive snark that proves +you found something in an index of the blockchain, or proves that +something, such as a previous spend of the output you want to spend, +does not exist in that index. # Blockchain -Each block in the chain is an set of sql tables, represented as merkle dags. +Each block in the chain is an set of SQL tables, represented as merkle dags. So a merkle patricia tree and the structs that its leaf edges point -to is an sql table that you can generate recursive snarks for, +to is an SQL table that you can generate recursive snarks for, which can prove things about transactions in that table. We are -unlikely to be programming the blockchain in sql, but to render -what one is doing intelligible, it is useful to think and design in sql. +unlikely to be programming the blockchain in SQL, but to render +what one is doing intelligible, it is useful to think and design in SQL. So with recursive snarks you can prove that that your transaction is valid because certain unspent transaction outputs were in the -sql index of unspent transaction outputs, and were recently spent +SQL index of unspent transaction outputs, and were recently spent in the index of commitments to transactions, without revealing which outputs those were, or what was in your transaction. @@ -182,7 +185,7 @@ It is a widely shared public index. But what it is an index of is private information about the transactions and outputs of those transactions, information known only to the parties of those transactions. It is not a public ledger. It is a widely shared -public sql index of private ledgers. And because it is a merkle +public SQL index of private ledgers. And because it is a merkle tree, it is possible to produce a single reasonably short recursive snark for the current root of that tree that proves that every transaction in all those private ledgers was a valid transaction @@ -191,7 +194,7 @@ and every unspent transaction output is as yet unspent. ## performing a transaction Oops, what I just described is a whole sequence of complete -immutable sql indexes, each new block a new complete index. +immutable SQL indexes, each new block a new complete index. But that would waste a whole lot of bandwidth. What you want is that each new block is only an index of new unspent transaction outputs, and of newly spent transaction outputs, which spending @@ -236,8 +239,8 @@ registered, you can create a new unspent transaction output for each transaction input to the failed transaction which effectively rolls back the failed transaction. This time limit enables us to recover from failed transactions, and, perhaps, more importantly, -enables us to clean up the mutable sql index that the immense -chain of immutable sql indexes represents, and that the public +enables us to clean up the mutable SQL index that the immense +chain of immutable SQL indexes represents, and that the public broadcast channel contains. We eventually drop outputs that have been committed to a particular transaction, and can then eventually drop the commits of that output without risking @@ -248,8 +251,8 @@ public broadcast channel. So that the public broadcast channel can eventually dump old blocks, and thus old spend events, every time we produce a new -base level block containing new events (an sql index of new -transaction outputs, and an sql index table with the same primary +base level block containing new events (an SQL index of new +transaction outputs, and an SQL index table with the same primary of spend commitments of past unspent transaction outputs to transactions) we also produce a consolidation block, a summary block that condenses two past blocks into one summary block,