diff --git a/docs/blockdag_consensus.md b/docs/blockdag_consensus.md index b52b875..8bd9bef 100644 --- a/docs/blockdag_consensus.md +++ b/docs/blockdag_consensus.md @@ -344,8 +344,7 @@ to add another block, rather than attempt to replace an existing block. In Paxos, once one peer accepts one value, it will eventually become the consensus value, assuming that everyone eventually gets through and that the usual network problems do not foul things up. Thus Paxos can provide -a definitive result eventually, while Bitcoin's results are never definitive, -merely exponentially probable. +a definitive result eventually, while the results of Bitcoin type consensus are never definitive, merely exponentially probable. In Paxos, a peer learns of the definitive and final consensus when it discovers that a majority of peers have accepted one value. Which if @@ -357,14 +356,34 @@ other, but there is no moment when a peer discovers than one branch is definitive and final. It just finds that one branch is becoming more and more likely, and all the other branches less and less likely. -Thus paxos has a stronger liveness property than bitcoin, but this -difference is in practice not important, for paxos may take an indefinitely -long time before it can report a definite and final consensus, while Bitcoin -takes a fairly definite time to report it is nearly certain about the consensus -value and that value is unlikely is unlikely to change. +Thus Paxos type algorithms (such as Paxos, Raft, and their Byzantine fault +tolerant equivalents) have a stronger liveness property than bitcoin type +algorithms, but this difference is in practice not important, for paxos may +take an indefinitely long time before it can report a definite and final +consensus, while a Bitcoin like consensus algorithm takes a fairly definite +time to report it is nearly certain about the consensus value and that value +is unlikely to change. Further, in actual practice, particularly during a view change, Paxos and Raft are frequently in a state where a peer knows that one view is overwhelmingly likely to become final, and another view highly unlikely, but the state takes a while to finalize. And the client is waiting. So it winds up reporting a provisional result to the client. +Thus in practice, the difference between the liveness condition of Paxos +type algorithms, which eventually provide a definitive proof that +consensus has been achieved, is not usefully stronger than the weaker liveness condition of consensus algorithms that merely provide an ever +growing likelihood that consensus has been achieved, and instead of +telling the client that consensus has been achieved or not, merely tell the +client how strong the evidence is that consensus has been achieved. + +And in practice, Byzantine Fault Tolerant Paxos like algorithms tend to be +"optimized" by behaving like bitcoin type algorithms, making their +implementation considerably more complicated. + +Crypto currency consensus algorithms derived from Paxos equivalent +consensus fail to scale, because they actually have to find out that a +majority of peers are on the same consensus. And there could be a lot of +peers, so this is a lot of data, and it is quite easy to maliciously attack the +network in ways to make this data difficult to obtain, if someone found it +convenient to stall a transaction from going through. "The cheque is in the mail." + # Bitcoin does not scale to competing with fiat currency Bitcoin is limited to ten transactions per second. Credit card networks diff --git a/docs/merkle_patricia-dac.md b/docs/merkle_patricia-dac.md index d2f0734..fd59dc7 100644 --- a/docs/merkle_patricia-dac.md +++ b/docs/merkle_patricia-dac.md @@ -856,32 +856,47 @@ solve the problem of the number of items not being a power of two? xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="29em" height="17em" - viewBox="0 202 230 102" + viewBox="0 200 230 102" style="background-color:ivory" stroke-width=".6" stroke-linecap="round" > - - + + - - + id + + - + M30,254 C33,249 35,249 35,263 S43,262 40,267"/> + M30,254 C33,249 35,249 50,248 s20,-9 20,-9"/> - @@ -952,7 +963,7 @@ solve the problem of the number of items not being a power of two? diff --git a/docs/mkdocs.sh b/docs/mkdocs.sh index fb98630..63e339a 100644 --- a/docs/mkdocs.sh +++ b/docs/mkdocs.sh @@ -1,5 +1,6 @@ #!/bin/bash set -e +set -x cd `dirname $0` if [[ "$OSTYPE" == "linux-gnu"* ]]; then @@ -15,15 +16,19 @@ templates="./pandoc_templates" options=$osoptions"--toc -N --toc-depth=5 --wrap=preserve --metadata=lang:en --include-in-header=$templates/icon.pandoc --include-before-body=$templates/before.pandoc --css=$templates/style.css -o" for f in *.md do + echo $f len=${#f} base=${f:0:($len-3)} if [ $f -nt $base.html ]; then + set -x katex="" mine="--include-after-body=$templates/after.pandoc " - for i in 1 2 3 4 5 6 + for i in 1 2 3 4 do + set -x read line + echo $line if [[ $line =~ katex$ ]]; then katex=" --katex=./" @@ -40,11 +45,13 @@ if [[ $line =~ notmine$ ]]; # echo " $base.html up to date" fi done +echo normal exit cd libraries templates="../pandoc_templates" options=$osoptions"--toc -N --toc-depth=5 --wrap=preserve --metadata=lang:en --include-in-header=$templates/icondotdot.pandoc --include-before-body=$templates/beforedotdot.pandoc --css=$templates/style.css --include-after-body=$templates/after.pandoc -o" for f in *.md do + echo $f len=${#f} base=${f:0:($len-3)} if [ $f -nt $base.html ]; @@ -68,6 +75,7 @@ cd ../.. templates=docs/pandoc_templates for f in *.md do + echo $f len=${#f} base=${f:0:($len-3)} if [ $f -nt $base.html ]; diff --git a/docs/number_encoding.md b/docs/number_encoding.md index 5f8e23a..559b31b 100644 --- a/docs/number_encoding.md +++ b/docs/number_encoding.md @@ -1,5 +1,5 @@ --- -lang: en +# katex title: Number encoding --- @@ -10,6 +10,38 @@ in protocols tend to become obsolete. Therefore, for future upwards compatibility, we want to have variable precision numbers. +Secondly, to represent integers within a patricia merkle tree representing a database index, we want all values to be left field aligned, rather than right field aligned. + +## Compression algorithm preserving sort order + +We want to represent integers by byte strings whose lexicographic order reflects their order as integers, which is to say, when sorted as a left aligned field, sort like integers represented as a right aligned field. (Because a Merkle patricia tree has a hard time with right aligned fields) + +To do this we have a field that is a count of the number of bytes, and the size of that field is encoded in unary. + +Thus a single byte value, representing integers in the range $0\le n \lt 2^7$ starts with a leading zero bit + +A two byte value, representing integers in the range $2^7\le n \lt 2^{13}+2^7$ starts with the bits 100 + +A three byte value, representing integers in the range $2^{13}+2^7 \le n \lt 2^{21}+2^{13}+2^7$ starts with the bits 101 + +A four byte value representing integers in the range $2^{21}+2^{13}+2^7 \le n \lt 2^{27}+2^{21}+2^{13}+2^7$ starts with the bits 11000 + +A five byte value representing integers in the range $2^{21}+2^{13}+2^7 \le n \lt 2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 11001 + +A six byte value representing integers in the range $2^{35}+2^{21}+2^{13}+2^7 \le n \lt 2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 11010 + +A seven byte value representing integers in the range $2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 11011 + +An eight byte value representing integers in the range $2^{51}2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{57}+2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 1110000 + +A nine byte value representing integers in the range $2^{57}+2^{51}+2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{65}+2^{57}+2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 1110001 + +Similarly the bits 111 0111 indicate a fifteen byte value representing 113 bit integers. + +To represent signed integers so that signed integers sort correctly with each other (but not with unsigned integers) the leading bit indicates the sign, a one bit for positive signed integers, and a zero bit for negative integers, and the if the signed integer is negative, we invert the bits of the byte count. Thus signed integers in the range $-2^6\le n \lt 2^6$ are represented by the corresponding eight bit value with its leading bit inverted. + +This is perhaps a little too much cleverness except for the uncommon case where we actually need a representation that sorts correctly. + ## Use case QR codes and prefix free number encoding is useful in cases where we want data to be self describing – this bunch of bits is to be interpreted in a certain way, used in a certain action, means one thing, and not another thing. At present there is no standard for self description. QR codes are given meanings by the application, and could carry completely arbitrary data whose meaning and purpose comes from outside, from the context. diff --git a/docs/white_paper.md b/docs/white_paper.md index caa4428..c4faa34 100644 --- a/docs/white_paper.md +++ b/docs/white_paper.md @@ -32,6 +32,11 @@ and twist the arms of the miners to exclude transactions in those bitcoins, with the result bitcoins cease to be fungible and thus cease to be money, as uncut diamonds ceased to be money. +We need untraceability to prevent the blood diamonds attack. Even if you +do not need your transactions to be untraceable, if they are traceable, ham +fisted government intervention is likely to make your money disappear +under you, as the value of uncut diamonds was lost. + Bitcoin is vulnerable to the one third attack. If one third of miners exclude “tainted” bitcoins and refuse to add to a chain ending in a block containing “tainted” bitcoins, other miners have an incentive to exclude “tainted” @@ -65,7 +70,8 @@ recording the accumulated effect of many transactions. A true lightning network functions as full reserve correspondence banking with no central authority. The existing Bitcoin lightning network functions as marginal reserve banking with good conduct enforced by central -authority. +authority. Taproot was designed to make this fixable, and plans are afoot +to fix it. At the time of writing, it was not clear to me that it has been fixed. But even with sidechaining and the lightning network, we are going to need the capacity to put thousands of transactions per second on the @@ -106,7 +112,7 @@ connection to the internet, or someone’s computer goes down), or, if someone breaks the circle in a Byzantine failure suggestive of Byzantine defection, then, and only then, it goes through on the primary blockchain. -The problem with the existing Bitcoin lightning network is that it has a +The problem with the existing Bitcoin lightning network is, or recently was, that it has a hidden and unexplained central authority, whom you have to trust, and which does stuff that is never explained or revealed. This is not stable, and does not scale. Not only is it evil, it is incapable of connecting everyone diff --git a/docs/writing_and_editing_documentation.md b/docs/writing_and_editing_documentation.md index a4698cb..b8b3322 100644 --- a/docs/writing_and_editing_documentation.md +++ b/docs/writing_and_editing_documentation.md @@ -192,14 +192,76 @@ scalable vector graphics files (`svg` files) to html can get complicated, and interfacing the resulting complicated html to markdown can get more complicated. -Which means that any time I want to edit the `svg`, I have to extract it into a -temporary `svg` file, edit it in Inkscape and Visual Studio Code, minify it in +Inkscape files are unreadable, and once they are cleaned up, Inkscape +cannot read them. To (irreversibly) clean up an Inkscape file, minify it in Visual Studio Code to get rid of all confusing mystery cruft inserted by Inkscape, edit it back into markdown compatible form, and reinsert it in the markdown file. -Which assumes my images, once done, are rarely or never going -to change. +A sequence of straight lines is M point, L point, L point. + +Z and z draw a straight line back to the beginning, use in conjunction with +`fill="#red"` , for example `fill="#FF0000"`. If the line is open, `fill="none"` + +Drawing smooth curves by typing in text is painful and slow, but so is +drawing them in Inkscape. Inkscape is apt to do a series of C beziers with +sharp corners between them, and when I try to fix the sharp corners, the +bezier goes weird. + +What Inkscape should do is let you manipulate a sequence of control +points, and draw an M C S S ..S bezier to the point halfway between the +final even numbered control point and the preceding control point, +creating an additional control point by mirroring if the user only provides +an odd number of control points. + +What it does instead is complicated and mysterious. + +To convert a sequence of straight lines into a smooth curve, you encounter +much grief matching the end of one bezier to the start of the next.\ +And if you do not match them, you get corners between beziers. + +* find the midpoint of each edge, or the midpoint of every second edge\ + When I say midpoint, somewhere along the line, close to where + you want the turn to sharp, and far from where you want the turn to be gentle. + +* A smooth curve starts a midpoint of first edge then:\ + C first vertex, second vertex, next midpoint S second next vertex + next midpoint S second next vertex, next midpoint ... S second next + vertex next midpoint ...\ + When I say second next vertex, I mean you skip a vertex, and thus + do not have to find the preceding midpoint. But the implied vertex + is the mirror of the preceding vertex, which if your midpoint is off, + is not where you expect it to be.\ + If it fails to go through the midpoint parallel to what you think the + endpoints are, you need to tinker the control point to reduce the + discrepancy, or else it is apt to be off at subsequent midpoints. + +* Or starts at midpoint of first edge then:\ + Q vertex midpoint T midpoint T midpoint ....\ + But the Q T T T chain is apt to act weird unless your midpoints are + near the middle, because the implied vertex is the mirror of the + preceding vertex, which may not be the vertex that you intended if + your midpoint is not near the centre of the line segment to that vertex. + +Using S and T has the enormous advantage that if your midpoint is a bit +off to one side of the line segment or the other, you still get a smooth +curve, no kinks between beziers But if it is too far off from the centre, +your curve will be off from the line segments, often in weird, surprising, +and complicated ways. + +On the other hand, if you have a C or a Q bezier following a previous C +or Q bezier, getting the join smooth is a bitch. + +To find the midpoint of a line segment, I use `M` approximate midpoint +`v6 v-12 v6 h6 h-12 h6` to mark the location with a cross. + +To draw a long twisty curve, mark your intended path with a sequence of +line segments, with your final line segment passing through your +destination, because destination will be the midpoint of that line segment. +Then do a M first point, C second point, third point, midpoint, then S +even numbered vertex, midpoint following the even numbered vertex, starting +with the fourth vertex and the midpoint to the fifth vertex. The midpoint of +the final vertex will be your destination. Scalable vector graphics are dimensionless, and the `` tag's height, width, and ViewBox properties translate the dimensionless quantities into @@ -221,23 +283,17 @@ take its defaults from the element selected. (which can be an entire group, or the entire high top level group, in which case it will pick up sane and appropriate properties from the first relevant item in the group.) -And how did you set those sane and relevant properties in the first place? - -By editing that element as text, very likely in your markdown file. - The enormous advantage of scalable vector graphics is that it handles repetitious items in diagrams beautifully, because you can define an item by reference to another item, thus very large hierarchical structure can be defined by very small source code. -Scalable vector graphics is best edited as text, except that one needs to -draw lines and splines graphically, and in the process, all your nicely laid out text gets mangled into soup, and has to be unmangled. - You might decide to keep it around as soup, in an `svg` file that only Inkscape ever tries to read, but then you are going to have to edit it as text again. And I wound up embedding my vector graphics files in markdown rather than invoking them as separate graphics files because my last step -was apt to be editing them as text. +was apt to be editing them as text. Which irreversibly made them +uneditable by Inkscape It is convenient to construct and position all the graphical elements in Inkscape, then edit the resulting `svg` file in Visual Studio Code with the @@ -284,8 +340,8 @@ graphic in the markdown preview pane. font-weight="400" stroke-width="2"> + d="M14 101, c40 -20, 30 -56, + 54 -18, s60 15, 40 15"/> A simple scalable vector graphic