From ad5c97e7dc0e07aad1914a44289963444e9f138a Mon Sep 17 00:00:00 2001 From: "reaction.la" Date: Sun, 6 Mar 2022 20:14:19 +1000 Subject: [PATCH] cleaned up svg code Realized that generalizing a Merkle-patricia tree to represent an arbitrary Merkle dag was just too messy. Led to complicated data structures forming complicated trees navigated in complicated ways. Treating each type of Merkle vertex as its own thing, and a Merkle-patricia vertex as a common and useful special case of a Merkle vetex leads to simpler data structures that more directly represent concepts. learning how to do paths without inkscape M point c point point point s point point s point point ... If you have more than one c triplet, you will have kinks Or m point, q point point t point t point ... M point q point will often suffice. You should never need many s or t values, and never have more than one c or q values. cleaned up consensus, made a start on cleaning up blockdags --- docs/blockdag_consensus.md | 45 +++++--- docs/merkle_patricia-dac.md | 224 ++++++++++++++++++++++-------------- 2 files changed, 169 insertions(+), 100 deletions(-) diff --git a/docs/blockdag_consensus.md b/docs/blockdag_consensus.md index ac7b309..b52b875 100644 --- a/docs/blockdag_consensus.md +++ b/docs/blockdag_consensus.md @@ -69,10 +69,10 @@ there is for a prong of a fork. ### Sampling the peers -So we have to sample the peers, or rather have each peer form the same -representative sample. And then we implement something similar to Paxos -and Raft within that small sample. And sometimes peers will disagree about -which sample, resulting in a fork, which has to be resolved. +So we have to sample the peers, or rather have each peer draw consensus +from the same representative sample. And then we implement something +similar to Paxos and Raft within that small sample. And sometimes peers +will disagree about which sample, resulting in a fork, which has to be resolved. For each peer that could be on the network, including those that have been sleeping in a cold wallet for years, each peer keeps a running cumulative @@ -105,13 +105,19 @@ there will be multiple next blocks, and entire sequences of blocks. In which case each peer has to rank them and weight them and choose the highest ranked and weighted block, or the highest weighted sequence of blocks. +A peer cannot tell the difference between network bisection and a whole lot +of peers being offline. + +We have to keep producing a result when a whole lot of peers are offline, +rather than just stop working as Practical Byzantine Fault Resistant +Consensus and its raft equivalent does. Which means that in a long lasting +network bisection, there is going to be fork which will have to be resolved +when the network recovers. + [running average]:./running_average.html -"how to calculate the running average\ - -dogs" +"how to calculate the running average" {target="_blank"} - The rank of a block is rank of the lowest rank peer of the group forming the block, or one plus the [running average] of previous blocks rounded up to the nearest integer, whichever is less. The weight of a block is $m*2^R$, where $m$ is the size of the group @@ -157,6 +163,14 @@ one group fail, another group will succeed. Correctly behaving peers will wait longer the lower their rank before attempting to participate in block formation, will wait longer before participating in an attempt to form a low weighted block, and will not attempt to form a new block if a block of which they already have a copy would be higher rank or weight. (Because they would abandon it immediately in favour of a block already formed.) +To ensure that forks and network bisections are resolved, the timeout before attempting to forming a new block increases exponentially with the difference between the proposed block and the running average, and inverse linearly with the running average. + +High ranking peers try to form a new block first, and in the event of +network bisection, the smaller part of the bisection will on average have +fewer peers, therefore the highest ranking peers available will on average +be lower rank, so the smaller portion of the bisection will form new blocks +more slowly on average. + In the course of attempting to form a group, peers will find out what other high ranking peers are online, so, if the Hedera assumption (that everyone always gets through eventually and that everyone knows who is online) is true, or true enough, there will only be one block formed, and it will be @@ -189,14 +203,19 @@ one or both prongs of the fork? Then we calculate the weight of the prong: -$$\displaystyle\sum\limits_{i}2^{R_i}$$ +$$\displaystyle\sum\limits_{i}{\large 2^{R_i}}$$ -Where $i$ is the peer, and R_i$ is the rank of the last block on that prong of -the fork that he has participated in forming. +Where $i$ is a peer that has formed blocks on that prong, and R_i$ is the +rank of the last block on that prong of the fork that he has participated in forming. -If two prongs have the same weight, hash all the public keys of the signatories, their ranks, and the block height of the root of the fork and take the prong with the largest hash. +Which, in the event of a fork, will on average reflect the total stake of +peers on that fork. -This value, the weight of a prong of the fork, will over time for large deep +If two prongs have the same weight, take the prong with the most transactions. If they have the same weight and the same number of transactions, hash all the public keys of the signatories that formed the +blocks, their ranks, and the block height of the root of the fork and take +the prong with the largest hash. + +This value, the weight of a prong of the fork will, over time for large deep forks, approximate the stake of the peers online on that prong, without the painful cost taking a poll of all peers online, and without the considerable risk that that poll will be jammed by hostile parties. diff --git a/docs/merkle_patricia-dac.md b/docs/merkle_patricia-dac.md index 937c2ec..d2f0734 100644 --- a/docs/merkle_patricia-dac.md +++ b/docs/merkle_patricia-dac.md @@ -856,100 +856,150 @@ solve the problem of the number of items not being a power of two? xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="29em" height="17em" - viewBox="0 209 360 102" + viewBox="0 202 230 102" style="background-color:ivory" stroke-width=".6" stroke-linecap="round" > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + - - - id + + + id - - - Immutable append only file as a Merkle chain - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Immutable append only file as a collection of + balanced binary Merkle trees + in postfix order + + - - - - - - - - - - - - - - - - Immutable append only file as a collection of - balanced binary Merkle trees - in postfix order - - - + + + + + + + + + + + + + + + + + + + + + Immutable append only file as a Merkle chain + + + + +This is a Merkle binary dag, not a Merkle-patricia tree. I found that +generalizing Merkle patricia trees to handle the case of immutable append +only files led to more complicated data structures and more complicated +descriptions. It has a great deal in common with a Merkle-patricia tree, but +a Merkle vertex is its own type, even though it convenient for it to have +much in common with a Merkle patricia vertex. Putting it into the patricia +straightjacket generated a lot of mess. A Merkle patricia vertex is best +handled as a special case of a Merkle vertex, and a Merkle patricia tree as +special vertices within a Merkle dag. The superstructure of balanced binary Merkle trees allows us to verify any part of it with only $O(log)$ hashes, and thus to verify that one version of