diff --git a/docs/libraries/mkdocs.sh b/docs/libraries/mkdocs.sh index e5a4609..01ac989 100644 --- a/docs/libraries/mkdocs.sh +++ b/docs/libraries/mkdocs.sh @@ -3,5 +3,6 @@ set -e echo `dirname $0` cd `dirname $0` docroot="../" +banner_height=banner_height:15ex templates=$docroot"pandoc_templates" . $templates"/mkdocs.cfg" diff --git a/docs/libraries/navbar b/docs/libraries/navbar new file mode 100644 index 0000000..710d180 --- /dev/null +++ b/docs/libraries/navbar @@ -0,0 +1,7 @@ +
+ \ No newline at end of file diff --git a/docs/manifesto/mkdocs.sh b/docs/manifesto/mkdocs.sh index 2cfb4a5..01ac989 100644 --- a/docs/manifesto/mkdocs.sh +++ b/docs/manifesto/mkdocs.sh @@ -1,41 +1,8 @@ #!/bin/bash set -e +echo `dirname $0` cd `dirname $0` docroot="../" banner_height=banner_height:15ex -if [[ "$OSTYPE" == "linux-gnu"* ]]; then - osoptions="" -elif [[ "$OSTYPE" == "darwin"* ]]; then - osoptions="" -elif [[ "$OSTYPE" == "cygwin" ]]; then - osoptions="--fail-if-warnings --eol=lf " -elif [[ "$OSTYPE" == "msys" ]]; then - osoptions="--fail-if-warnings --eol=lf " -fi -if [[ -z $targetDocroot ]]; then - targetDocroot=$docroot -fi -options=$osoptions"--toc --number-sections --toc-depth=5 --from markdown+smart+raw_html+fenced_divs+bracketed_spans --to html5 --wrap=preserve --metadata=lang:en --css=$targetDocroot"pandoc_templates/style.css" -Bnavbar -o" -for f in * -do - [[ -d $item && -x $item/mkdocs.sh ]] && $item/mkdocs.sh - if [[ $f =~ (.*)".md"$ ]];then - base=${BASH_REMATCH[1]} - if [[ $f -nt $destdir$base.html ]]; then - katex="" - line="" - for i in 1 2 3 4 5 6 - do - read line - - if [[ $line =~ katex ]];then - katex=" --katex="$docroot - fi - done <$f - pandoc --variable $banner_height --variable targetDocroot:$targetDocroot --template $docroot"pandoc_templates/pandoc.template" $katex $options $destdir$base.html $base.md - echo "$destdir$base.html from $f" - #else - # echo " $base.html up to date" - fi - fi -done \ No newline at end of file +templates=$docroot"pandoc_templates" +. $templates"/mkdocs.cfg" diff --git a/docs/manifesto/scalability.md b/docs/manifesto/scalability.md index 9fb5894..755b0a0 100644 --- a/docs/manifesto/scalability.md +++ b/docs/manifesto/scalability.md @@ -438,7 +438,12 @@ that fourteenth century businessmen imagined into reality. For sovereign corporations, a great deal of corporate governance can be done by the laws of mathematics, -rather than the laws of men +rather than the laws of men, +which was one of the original cypherpunk goals and slogans +that Satoshi was attempting to fulfil. +We always intended from the very beginning +to destroy postmodern capitalism and restore +the modern capitalism of Charles the Second. The commits form a directed acyclic graph. Each particular individual who knows the preimage of *some* @@ -460,7 +465,7 @@ single path through the merkle dag, and it is possible to prove the ledger consistent along this named path. *And* we want him to be able to prove that he is showing facts -about his ledger that are consistent with everyone else's ledgerss. +about his ledger that are consistent with everyone else's ledgers. To do that, triple entry accounting, where a journal entry that lists an obligation of a counterparty as an asset, or an obligation to a counterparty as a liability, @@ -469,16 +474,9 @@ jointly signed by the non fungible name tokens of both parties. Double entry accounting shows the books balance. Triple entry accounting shows that obligations between parties -recorded on their books balance. - -Thus for sovereign corporations, a great deal of corporate governance -can be done by the laws of mathematics, -rather than the laws of men, -which was one of the original cypherpunk goals and slogans -that Satoshi was attempting to fulfil. -We always intended from the very beginning -to destroy postmodern capitalism and restore -the modern capitalism of Charles the Second. +recorded on their books balance. Triple entry accounting was +originally developed by cypherpunks when we were attempting to +create internet money based on moving ownership of gold around. Such a non fungible name tokens would also be necessary for a reputation system, if we want to eat Amazon's and Ebay's lunch. diff --git a/docs/manifesto/social_networking.md b/docs/manifesto/social_networking.md index fb84df7..173012c 100644 --- a/docs/manifesto/social_networking.md +++ b/docs/manifesto/social_networking.md @@ -1446,3 +1446,35 @@ And, with the corporate [book]s better connected to reality, the shareholders will have better information on whether to do so. A necessary precondition for corporations to function at all is that we fix bookkeeping to work in a world of untrustworthy elites. + +## share holding and trading + +It used to be the case that shares were a direct relationship between the shareholder and the company. +The company issued a certificate directly to the shareholder. That was the modern corporate capitalism +of Charles the Second, and it still existed within living memory, within my memory. + +Today, in postmodern capitalism, there is a custodian in the middle. +You do not own shares. You own promises made by big finance. +And as we saw in the Gamestop shenanigans, +there can be a very big difference between these two things. + +Gamestop was trying to raise money from its shareholders for a change in business plan. +The net effect of these shenanigans was that most of the money went into the pockets of +people in big finance running the stock exchange. + +As our elites become ever less trustworthy, and we trust them ever less, +they insist on systems that force us to give them ever more trust. + +As elite virtue collapses, we strangely find ourselves forced to rely on systems that +strangely become ever more reliant on elite virtue. +Custodial shares to money and investment what absentee voting is to elections. + +The corporate form needs to be put on blockchains, so that your shares are +owned through private secrets derived from each party's master secret. + +The financial businesses of the stock exchange are creaming off a vast amount of wealth. + +Information wants to be free, but programmers want to paid. If we liberate corporations +and shareholders from this tyranny, we can arrange for a very small part of this mountain +of wealth to stick to us. And a very small part of this mountain is still a mighty big +mountain. diff --git a/docs/manifesto/vision.md b/docs/manifesto/vision.md index 7e257fc..fe86d02 100644 --- a/docs/manifesto/vision.md +++ b/docs/manifesto/vision.md @@ -10,7 +10,8 @@ reputation secured by the blockchain. Satoshi was after the money in the seignorage tax. There is also a lot of money in eBay and Amazon owning other people's reputations, a lot of money in the domain name -system, and a lot of money in limited liability company name registration. eBay is +system, a lot of money in limited liability company name registration, and a lot +of money in the financial institutions of the stockmarket. eBay is entirely a search engine for the reputation data of people doing transactions and Amazon is largely such a search engine, and the primary asset value of most businesses is the goodwill accrued by their registered company name. diff --git a/docs/manifesto/white_paper.md b/docs/manifesto/white_paper.md index 942132e..8e5a7a4 100644 --- a/docs/manifesto/white_paper.md +++ b/docs/manifesto/white_paper.md @@ -47,7 +47,7 @@ longest chain. until rather more than ten percent join the attack. One third, however will suffice. Calling it the blood diamonds attack is more accurate. The blood diamonds attack put a lot of people out of business, for they were really in -the money changer business, rather than the jewelry business.) +the money changer business, rather than the jewellery business.) [Bitcoin is already under the ten percent attack. If it approaches one third, time to flee bitcoin.](http://reaction.la/security/bitcoin_vulnerable_to_currency_controls.html) diff --git a/docs/mkdocs.sh b/docs/mkdocs.sh index 0d0af20..3eb79ae 100644 --- a/docs/mkdocs.sh +++ b/docs/mkdocs.sh @@ -1,11 +1,8 @@ #!/bin/bash set -e -cd `dirname $0` echo `dirname $0` -for item in *; do - [[ -d $item && -f $item/mkdocs.sh ]] && $item/mkdocs.sh -done -echo all subdirectories done +cd `dirname $0` docroot="./" +banner_height=banner_height:15ex templates=$docroot"pandoc_templates" . $templates"/mkdocs.cfg" diff --git a/docs/names/TCP.md b/docs/names/TCP.md index 8fc471f..a46a72e 100644 --- a/docs/names/TCP.md +++ b/docs/names/TCP.md @@ -1,5 +1,6 @@ --- title: Replacing TCP, SSL, DNS, CAs, and TLS +sidebar: true ... # related diff --git a/docs/names/mkdocs.sh b/docs/names/mkdocs.sh index e5a4609..01ac989 100644 --- a/docs/names/mkdocs.sh +++ b/docs/names/mkdocs.sh @@ -3,5 +3,6 @@ set -e echo `dirname $0` cd `dirname $0` docroot="../" +banner_height=banner_height:15ex templates=$docroot"pandoc_templates" . $templates"/mkdocs.cfg" diff --git a/docs/names/multisignature.md b/docs/names/multisignature.md index 120a8f9..361bde1 100644 --- a/docs/names/multisignature.md +++ b/docs/names/multisignature.md @@ -2,6 +2,7 @@ title: Multisignature # katex +sidebar: true ... To do a Schnorr multisignature, you just list all the signatures that diff --git a/docs/names/navbar b/docs/names/navbar new file mode 100644 index 0000000..710d180 --- /dev/null +++ b/docs/names/navbar @@ -0,0 +1,7 @@ + + \ No newline at end of file diff --git a/docs/names/petnames.md b/docs/names/petnames.md index bb37ad7..3616cf7 100644 --- a/docs/names/petnames.md +++ b/docs/names/petnames.md @@ -1,7 +1,8 @@ --- title: An Introduction to Petname Systems # notmine ---- +sidebar: true +... ::: {style="text-align: center;"} by diff --git a/docs/names/server.md b/docs/names/server.md index 9356db0..a5469ac 100644 --- a/docs/names/server.md +++ b/docs/names/server.md @@ -1,5 +1,6 @@ --- title: Server Data Representation +sidebar: true ... # related diff --git a/docs/navbar b/docs/navbar index a10f78c..710d180 100644 --- a/docs/navbar +++ b/docs/navbar @@ -1,7 +1,7 @@ \ No newline at end of file diff --git a/docs/notes/merkle_patricia_dag.md b/docs/notes/big_cirle_notation.md similarity index 97% rename from docs/notes/merkle_patricia_dag.md rename to docs/notes/big_cirle_notation.md index 20cc7b9..6d27753 100644 --- a/docs/notes/merkle_patricia_dag.md +++ b/docs/notes/big_cirle_notation.md @@ -1,6 +1,6 @@ --- -title: -Big Circ notation +title: >- + Big Circ notation # katex ... diff --git a/docs/notes/mkdocs.sh b/docs/notes/mkdocs.sh new file mode 100644 index 0000000..01ac989 --- /dev/null +++ b/docs/notes/mkdocs.sh @@ -0,0 +1,8 @@ +#!/bin/bash +set -e +echo `dirname $0` +cd `dirname $0` +docroot="../" +banner_height=banner_height:15ex +templates=$docroot"pandoc_templates" +. $templates"/mkdocs.cfg" diff --git a/docs/notes/navbar b/docs/notes/navbar new file mode 100644 index 0000000..710d180 --- /dev/null +++ b/docs/notes/navbar @@ -0,0 +1,7 @@ + + \ No newline at end of file diff --git a/docs/number_encoding.md b/docs/number_encoding.md index b572ed3..9f0fd5c 100644 --- a/docs/number_encoding.md +++ b/docs/number_encoding.md @@ -1,7 +1,40 @@ --- # katex title: Number encoding ---- +sidebar: true +... + +I have spent far too much time implementing and thinking about +variable length quantities. + +And became deeply confused about +suitable representation of such quantities in patricia trees +because I lost track of the levels of encapsulation. + +A patricia vertex represents a prefix with a bitcount. +A patricia tree represents a prefix free index. A patricia vertex +encapsulates a bitcount of the prefix, and is encapsulated +by a bytecount of the vertex. + +There is mud all over my web pages resulting from this confusion. + +If we ever have a patricia tree of integers, variable length number encoding +means that it is still just a bit string and a bit count. +It will be the common bits of the variable length encoded integers of its children. + +Because the variable length representation of the integers have normal byte order +we don't have to worry that some of them are different length, we don't have worry +about where the binary point is relative to start of the prefix, on which point +I was endlessly confused. + +# history of mucking around. + +I originally implemented variable length quantities following the standard. + +And then I realized that an sql index represented as a Merkle-patricia tree inherently sorts in byte string order. +Which is fine if we represent integers as fixed length integers in big endian format, +but does not correctly sort variable length quantities if we follow the standard: + # The problem to be solved As computers and networks grow, any fixed length fields @@ -9,66 +42,166 @@ in protocols tend to become obsolete. Therefore, for future upwards compatibility, we want to have variable precision numbers. -Secondly, to represent integers within a Merkle-patricia tree representing a database index, we want all values to be left field aligned, rather than right field aligned. +Secondly, to represent integers within a Merkle-patricia tree representing a database index, +we want all values to be left field aligned, rather than right field aligned. +which requires some form of variable length encoding that preserves the order +relationship between integers, so that the bytestring order is the same as +the integer order. ## Merkle-patricia dag +### patricia tree + +A patricia tree inherently represents a collection of prefix free bitstrings, +which means the represented bitstrings must be self terminating, +which for fixed length bitstrings is just reaching a certain number of bits, +and for cee strings is eight zero bits, byte aligned. + +But the vertices of a patricia tree represent a collection of prefixes. +Each vertex represents some left aligned bits and a count of those bits. +The count of the bits is extrinsic to the bits, they are inherently not self terminating, +and the vertices are themselves not self terminating, they have an extrinsic byte count or bit count +which is not itself part of the vertex, but part of the data structure within which +the vertex is stored or represented. + +We could *represent* the vertices by a self terminating bytestring or bitstring, but only +if we wanted to put the vertices of a patricia tree inside *another* patricia tree. Which +seems like a stupid thing to do under most circumstances. And what is being represented +is itself inherently not self terminating. + +The leaves of the patricia tree represent a data structure +whose *index* is in the patricia tree, the index being a fixed length set of fields, +each field itself a self terminating string of bits. Thus the leaf is not a +patricia vertex, but an object of a different kind. (Unless of course the +index fully represents all the information of the object) + +The links inside a vertex also represent a short string of bits, +the bits that the vertex pointed to has in addition to the bits +that vertex point already has. This string is typically very +short, one bit implicit in it being a left or right vertex, and +zero or few additional bits, but may be very long, thirty two bytes. + +Because it is of variable, and typically short, length, it is +conveniently represented by a short self terminating variable length data structure -- +the variable length representation of the positive integer +you obtain by prepending a one bit to the bit field, +or the signed integer you obtain by prepending zeroes to bitfield +beginning with a one bit to obtain a small positive integer, +and ones to a bitfield beginning with a zero bit to obtain a small +negative integer. + +But, regardless of how it is represented within the vertex, and how it is manipulated +by the code, what is being represented is a bit field with byte and field alignment +and a bit count. And for large bitfields, which do not fit inside the standard computer +word length, it is inconvenient to represent them as integers, you want to represent them +with field alignment. If using the signed representation, the integers zero and minus one +both represent the empty bit string, and we can reserve one of them to represent +switching to a representation more convenient for longer bitstrings. + +### Merkle dag We intend to have a vast Merkle dag, and a vast collection of immutable append only data structures. Each new block in the append only data structure is represented by a hash, whose preimage is a Merkle vertex. A -path through the Merkle dag is represented by sequence of integers, which -are zero terminated if the path is intended to end at that preimage. The -same path in different versions should lead to an object that is in some -sense equivalent to the entity that that path led to in a previous version of -that merkle vertex. Equivalence and relationship of objects between -different merkle vertices in the same immutable append only sequence is -represented by constant paths leading to different objects. - -If the sequence corresponds to valid unicode characters that are non control, non whitespace, and not #, @, (), or {}, it is so represented to the user. If one of the integers in the sequence is not, then the user will generally see a # followed by a sequence of base 58 characters representing a sequence of integers, until there is an integer corresponding to a non control, non whitespace, non numeric, non ascii alpha, and not #, @, (), or {}, at which point the human representation switches back to unicode. - -If an ascii blank, character 32, occurs within a sequence being represented -to the human as unicode, it is displayed as a blank and all subsequent -values are displayed as dot separated decimal numbers, the displayed -decimal number being the underlying number minus one. If the underlying -value is zero, this terminates the sequence and is therefore never displayed. - -A space introduces a sequence of version numbers intended to be -intelligible to both humans and computers as a version number. The -intended usage is package authentication and source control - the package -manager will know this is an updated package, rather than an unrelated -package. +path through the Merkle dag is represented by a consecutive sequence of integers, which +represent not a Merkle-patricia tree, but a sequence of immutable Merkle-patricia +trees that represent the mutable Merkle-patricia tree that is the current state of +the blockchain. ## Compression algorithm preserving sort order -We want to represent integers by byte strings whose lexicographic order reflects their order as integers, which is to say, when sorted as a left aligned field, sort like integers represented as a right aligned field. (Because a Merkle-patricia tree has a hard time with right aligned fields) +We want to represent integers by byte strings whose +lexicographic order reflects their order as integers, +which is to say, when sorted as a left aligned field, sort like integers +represented as a right aligned field. (Because a Merkle-patricia +tree has a hard time with right aligned fields, and we do not +want to represent integers by a fixed length field, because the fixed +length will always either be too big or too small, as has given TCP no end +of grief. -To do this we have a field that is a count of the number of bytes, and the size of that field is encoded in unary. +To do this we have a field that is a count of the bytes minus one, +and the size of that field is encoded in unary. -Thus a single byte value, representing integers in the range $0\le n \lt 2^7$ starts with a leading zero bit +### representation of unsigned integers as variable length quantities. -A two byte value, representing integers in the range $2^7\le n \lt 2^{13}+2^7$ starts with the bits 10 0 +An unsigned integer value that fits in a single byte starts with $0$, that being unary for a zero +width byte count field. This leaves seven bits to represent the unsigned integer. -A three byte value, representing integers in the range $2^{13}+2^7 \le n \lt 2^{21}+2^{13}+2^7$ starts with the bits 10 1 +Thus an unsigned integer value in the range $0\le n \lt 2^7$ is the +unsigned eight bit integer itself. -A four byte value representing integers in the range $2^{21}+2^{13}+2^7 \le n \lt 2^{27}+2^{21}+2^{13}+2^7$ starts with the bits 110 00 +An unsigned integer in the range $2^7\le n \lt 2^{13}$ starts with the bits $10\,0$, +$10$ being unary for a one bit wide field, and $0$ being that one bit, indicating +two bytes. Thus is represented by the integer itself $+0\text{x}\,8000$ +in big endian format. -A five byte value representing integers in the range $2^{21}+2^{13}+2^7 \le n \lt 2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 110 01 +If the representation is less than $0\text{x}8080$ then it does not represent +an integer value, and reading such data should result in an exception that +ends processing of the data or in special case handling for non integers. -A six byte value representing integers in the range $2^{35}+2^{21}+2^{13}+2^7 \le n \lt 2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 110 10 +An unsigned integer in the range $2^{13}\le n \lt 2^{21}$ starts with the bits $10\,1$, +$10$ being unary for a one bit wide field, and $1$ being that one bit, indicating +three bytes. Thus is represented by the integer itself $+0\text{x}\,88\,0000$ +in big endian format. -A seven byte value representing integers in the range $2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 110 11 +If the representation is less than $0\text{x}\,88\,2000$ then it does not represent +an integer value, and reading such data should result in an exception that +ends processing of the data or in special case handling for non integers. -An eight byte value representing integers in the range $2^{51}2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{57}+2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 1110 000 +An unsigned integer in the range $2^{21}\le n \lt 2^{27}$ starts with the bits $110\,00$, +$110$ being unary for a two bit wide field, and $00$ being those two bits, indicating +four bytes. Thus is represented by the integer itself $+0\text{x}\,c000\,0000$ +in big endian format. -A nine byte value representing integers in the range $2^{57}+2^{51}+2^{43}+2^{35}+2^{21}+2^{13}+2^7 \le n \lt2^{65}+2^{57}+2^{51}+2^{43}+2^{35}+2^{27}+2^{21}+2^{13}+2^7+2^{13}+2^7$ starts with the bits 1110 001 +If the representation is less than $0\text{x}\,c020\,0000$ then it does not represent +an integer value, and reading such data should result in an exception that +ends processing of the data or in special case handling for non integers. -Similarly the bits 1110 111 indicate a fifteen byte value representing 113 bit integers. +And similarly, five byte values start with $110\,01$, representing values in the range $2^{27}\le n \lt 2^{35}$, six byte values start with $110\,10$, representing values in the range $2^{35}\le n \lt 2^{43}$, seven byte values start with $110\,11$, representing values in the range $2^{43}\le n \lt 2^{51}$. -To represent signed integers so that signed integers sort correctly with each other (but not with unsigned integers) the leading bit indicates the sign, a one bit for positive signed integers, and a zero bit for negative integers, and the if the signed integer is negative, we invert the bits of the byte count. Thus signed integers in the range $-2^6\le n \lt 2^6$ are represented by the corresponding eight bit value with its leading bit inverted. +Similarly, eight byte values start with $1110\,000$, $1110$ being unary for +a three bit wide field, and $000$ being those three bits, representing values in the range $2^{51}\le n \lt 2^{57}$, nine byte values start with $1110\,001$, representing values in the range $2^{57}\le n \lt 2^{65}$, ten byte values start with $1110\,010$, and so on and so forth to $1110\,111$ representing a fifteen byte value in the range $2^{105}\le n \lt 2^{113}$. -This is perhaps a little too much cleverness except for the uncommon case -where we actually need a representation of signed integers that sorts -correctly. +And so on and so forth except that for some time to come the reader implementation is going to throw an exception if it attempts to read a value larger than $2^{64}-1$ + +Eventually we will allow 128 bit values, requiring a nine bit header, but not for quite some time. + +### representation of signed integers as variable length quantities. + +To represent unsigned integers, the header starts with a one bit for +positive quantities and a zero bit for negative quantities, to ensure that +bytestring order on the representation agrees with the order of the values +represented. Following that bit, we proceed as for unsigned integers, except +that for negative values, the header bits are inverted, to get correct +sort order, so that more negative values sort before less negative values +and more positive values sort after less positive values. + +Thus a single byte value, representing positive signed integers in the +range $0\le n \lt 2^6$ starts with a leading one bit, followed by zero bit +(unary for a zero width count field), and a single byte value representing +negative integers in the range $-2^6\le n \lt 0$ starts with a leading +zero bit, representing the sign, and a leading one bit, being inverted unary +for a zero width count field + +Thus the representation for a signed integer $n$ in the range $-2^6\le n \lt 2^6$ +is the one byte signed integer itself $\oplus 0\text{x}\,80$ + +A two byte value, representing signed integers in the range $2^6\le n \lt 2^{12}$ starts with the bits $1\,10\, 0$, 10 being unary for field one bit long, +as for unsigned integers + +A two byte value, representing signed integers in the range $-2^{12}\le n \lt -2^6$, +starts with the bits with the bits $0\,01\, 1$, as for unsigned integers but inverted. + +Thus the representation for a signed integer $n$ in the range $(-2^{12}\le n \lt 2^6)\,\lor\,(2^6\le n \lt 2^{12})$ +is the two byte signed integer itself $\oplus 0\text{x}\,c000$ +in big endian format. + +With, as usual "not an integer value" exceptions being thrown if the central excluded range is violated. + +And, similarly, the representation for a signed integer $n$ in the range $(-2^{20}\le n \lt 2^{12})\,\lor\,(2^{12}\le n \lt 2^{20})$ is the three byte signed integer itself +$\oplus 0\text{x}\,c4\,0000$ +in big endian format. + +And so on and so forth for signed integers of unlimited size. ## base 58 representation of a sequence of unsigned integers @@ -96,83 +229,29 @@ We display a value in the range $0\le n \lt 58/2$ as itself, a value $n$ in the range $58/2\le n \lt \lfloor 58*2^{-2}\rfloor*58 +58/2$ as the base 58 representation of $n+58*(58/2-1)$ -## Variable length bit fields +# bitstrings -To represent variable length bit fields in the postfix sort order, -such that a shorter bit field sorts after all longer bit fields -with same leading bits: +Bitstrings in Merkle-patricia tree representing an sql index +are typically very short, so should be represented by a +variable length quantity, except for the leaf edge, +which is fixed size and large, so should not be +represented by variable length quantity. -We break it into seven bit fields, with a final field representing zero to six bits. +We use the integer zero to represent this special case, +the integer one to represent the zero length bit string, +integers two and three to represent the one bit bitstring, +integers four to seven to represent the two bit bit string, +and so on and so forth. -A seven bit field is represented by a byte ending in a zero low order bit. +In other words, we represent it as the integer obtained +by prepending a leading one bit to the bit string. -A variable length $m$ bit field where m is 0 to 6 (seven possible -values) by is represented by a fixed width eight bit field: +# Dewey decimal sequences. -Where if\ -$j$ is the bitfield interpreted as a number\ -$m$ is the length of the bitfield\ -$c$ is a count of the set bits in the bitfield - -The value of the eight bit field is:\ -$j*(2^{(7-m)}-1)+2*c+1$ - ----------------------- -variable 7 bit -bit field bitfield ---------- ------------ -000000 0000 0001 - -000001 0000 0011 - -00000 0000 0101 - -000010 0000 0111 - -000011 0000 1001 - -00001 0000 1011 - -0000 0000 1101 - -000100 0000 1111 - -000101 0001 1001 - -00010 0001 0011 - -000110 0001 0101 - -000111 0001 0111 - -00011 0001 1001 - -0001 0001 1011 - -000 0001 1101 - -... ... - -111101 1110 1100 - -11110 1110 1101 - -111110 1110 1111 - -111111 1111 0001 - -11111 1111 0011 - -1111 1111 0101 - -111 1111 0111 - -11 1111 1001 - -1 1111 1011 - -empty 1111 1101 --------------------- +The only thing we ever want to do with Dewey decimal sequences is $<=>$, +and they are always positive numbers less than $10^{14}$, so we represent them as +a sequence of variable length numbers terminated by the number minus one +and compare them as bytestrings. ### SQL blobs. diff --git a/docs/pandoc_templates/mkdocs.cfg b/docs/pandoc_templates/mkdocs.cfg index a37db23..9d0f199 100644 --- a/docs/pandoc_templates/mkdocs.cfg +++ b/docs/pandoc_templates/mkdocs.cfg @@ -7,26 +7,33 @@ elif [[ "$OSTYPE" == "cygwin" ]]; then elif [[ "$OSTYPE" == "msys" ]]; then osoptions="--fail-if-warnings --eol=lf " fi -options=$osoptions"--toc --number-sections --toc-depth=5 --from markdown+smart+raw_html+native_divs+native_spans+fenced_divs+bracketed_spans --to html5 --wrap=preserve --metadata=lang:en --include-in-header=icon.pandoc --css=$templates/style.css -o" -for f in *.md +if [[ -z $targetDocroot ]]; then + targetDocroot=$docroot +fi +options=$osoptions"--toc --number-sections --toc-depth=5 --from markdown+smart+raw_html+fenced_divs+bracketed_spans --to html5 --wrap=preserve --metadata=lang:en --css=$targetDocroot"pandoc_templates/style.css" -Bnavbar -o" +for f in * do - len=${#f} - base=${f:0:($len-3)} - if [ $f -nt $destdir$base.html ]; - then - line="" - for i in 1 2 3 4 5 6 - do - read line - if [[ $line =~ katex$ ]]; - then - katex=" --katex=./" - fi - done <$f - pandoc $katex $options $destdir$base.html $base.md - echo "$base.html from $f" + if [[ -d $f && -x $f/mkdocs.sh ]]; then + echo "recursing into" $f + $f/mkdocs.sh +fi + if [[ $f =~ (.*)".md"$ ]];then + base=${BASH_REMATCH[1]} + if [[ $f -nt $destdir$base.html ]]; then + katex="" + line="" + for i in 1 2 3 4 5 + do + read line - #else - # echo " $base.html up to date" - fi + if [[ $line =~ katex ]];then + katex=" --katex="$docroot + fi + done <$f + pandoc --variable $banner_height --variable targetDocroot:$targetDocroot --template $docroot"pandoc_templates/pandoc.template" $katex $options $destdir$base.html $base.md + echo "$destdir$base.html from $f" + #else + # echo " $base.html up to date" + fi + fi done diff --git a/docs/pandoc_templates/pandoc.template b/docs/pandoc_templates/pandoc.template index b93eb5e..c61fba5 100644 --- a/docs/pandoc_templates/pandoc.template +++ b/docs/pandoc_templates/pandoc.template @@ -82,6 +82,17 @@ $if(title)$$subtitle$
$endif$ $endif$ +$if(sidebar)$ +$else$ +$if(toc)$ + +$endif$ +$endif$ $body$ $if(sidebar)$ diff --git a/docs/rootDocs/mkdocs.sh b/docs/rootDocs/mkdocs.sh index 8f098f2..35bb001 100644 --- a/docs/rootDocs/mkdocs.sh +++ b/docs/rootDocs/mkdocs.sh @@ -1,43 +1,10 @@ #!/bin/bash set -e +echo `dirname $0` cd `dirname $0` docroot="../" targetDocroot="docs/" destdir="../../" banner_height=banner_height:15ex -if [[ "$OSTYPE" == "linux-gnu"* ]]; then - osoptions="" -elif [[ "$OSTYPE" == "darwin"* ]]; then - osoptions="" -elif [[ "$OSTYPE" == "cygwin" ]]; then - osoptions="--fail-if-warnings --eol=lf " -elif [[ "$OSTYPE" == "msys" ]]; then - osoptions="--fail-if-warnings --eol=lf " -fi -if [[ -z $targetDocroot ]]; then - targetDocroot=$docroot -fi -options=$osoptions"--toc --number-sections --toc-depth=5 --from markdown+smart+raw_html+fenced_divs+bracketed_spans --to html5 --wrap=preserve --metadata=lang:en --css=$targetDocroot/pandoc_templates/style.css -Bnavbar -o" -for f in * -do - [[ -d $item && -x $item/mkdocs.sh ]] && $item/mkdocs.sh - if [[ $f =~ (.*)".md"$ ]];then - base=${BASH_REMATCH[1]} - if [[ $f -nt $destdir$base.html ]]; then - katex="" - line="" - for i in 1 2 3 4 5 6 - do - read line - - if [[ $line =~ katex ]];then - katex=" --katex="$docroot - fi - done <$f - pandoc --variable $banner_height --variable targetDocroot:$targetDocroot --template $docroot"pandoc_templates/pandoc.template" $katex $options $destdir$base.html $base.md - echo "$destdir$base.html from $f" - #else - # echo " $base.html up to date" - fi - fi -done +templates=$docroot"pandoc_templates" +. $templates"/mkdocs.cfg" diff --git a/docs/setup/contributor_code_of_conduct.md b/docs/setup/contributor_code_of_conduct.md index de40a9c..ba95af2 100644 --- a/docs/setup/contributor_code_of_conduct.md +++ b/docs/setup/contributor_code_of_conduct.md @@ -1,5 +1,6 @@ --- title: Contributor Code of Conduct +sidebar: true ... # Peace on Earth to all men of good will diff --git a/docs/setup/dovecot.md b/docs/setup/dovecot.md index 6bdad57..b24c742 100644 --- a/docs/setup/dovecot.md +++ b/docs/setup/dovecot.md @@ -1,6 +1,8 @@ --- title: Install Dovecot on Debian 10 +sidebar: true ... + # Purpose We want postfix working with Dovecot so that we can send and access our emails from email client such as thunderbird client on another computer. diff --git a/docs/setup/mkdocs.sh b/docs/setup/mkdocs.sh index e5a4609..01ac989 100644 --- a/docs/setup/mkdocs.sh +++ b/docs/setup/mkdocs.sh @@ -3,5 +3,6 @@ set -e echo `dirname $0` cd `dirname $0` docroot="../" +banner_height=banner_height:15ex templates=$docroot"pandoc_templates" . $templates"/mkdocs.cfg" diff --git a/docs/setup/navbar b/docs/setup/navbar new file mode 100644 index 0000000..c4314ed --- /dev/null +++ b/docs/setup/navbar @@ -0,0 +1,6 @@ + + \ No newline at end of file diff --git a/docs/setup/set_up_build_environments.md b/docs/setup/set_up_build_environments.md index 9844268..134cc09 100644 --- a/docs/setup/set_up_build_environments.md +++ b/docs/setup/set_up_build_environments.md @@ -1,8 +1,8 @@ --- title: Set up build environments +sidebar: true ... - # partitioning for linux For a gpt partition table, sixteen MiB fat32 partition with boot and efi flags diff --git a/docs/setup/wireguard.md b/docs/setup/wireguard.md index e75c09e..7b36fdd 100644 --- a/docs/setup/wireguard.md +++ b/docs/setup/wireguard.md @@ -1,8 +1,12 @@ --- title: Wireguard +sidebar: true +notmine: false ... -Setting up your own vpn using wireguard and a Debian 11 server in the cloud +::: myabstract +[abstract:]{.bigbold}Setting up your own vpn using wireguard and a Debian 11 server in the cloud +::: This tutorial largely stolen from [Linuxbabe](https://www.linuxbabe.com/debian/wireguard-vpn-server-debian){target="_blank"} It is slightly more up to date than her version at the time of writing. diff --git a/docs/triple_entry_accounting.md b/docs/triple_entry_accounting.md index 82e7cf9..dadb0d2 100644 --- a/docs/triple_entry_accounting.md +++ b/docs/triple_entry_accounting.md @@ -14,12 +14,12 @@ See [Sox accounting], for why we need to replace Sox accounting with triple entr # What is triple entry accounting - Double entry accounting ensures that the books of the corporation are - internally consistent. Triple entry accounting ensures that the books of the - corporation are not only internally consistent, but that its account of its - obligations and transactions with clients and partners is consistent with their account. +Double entry accounting ensures that the books of the corporation are +internally consistent. Triple entry accounting ensures that the books of the +corporation are not only internally consistent, but that its account of its +obligations and transactions with clients and partners is consistent with their account. - This requires a shared pool of data shared between several entities who +This requires a shared pool of data shared between several entities who have long lasting and repeated durable business relationships with each other, and if it is a lot of data, a blockchain. @@ -44,19 +44,22 @@ itself a real thing, it became a real thing, the real thing being those people acting as if with one will, the will of the enterprise. And then King Charles the second gave some of these accounting fictions -made real legal status as corporations, creating the modern joint stock publicly traded for-profit corporation, the modern corporation. +legal status as corporations, +creating the modern joint stock publicly traded for-profit corporation, +the modern corporation. + +And thus created modern capitalism, which worked spectacularly well. +(We now, unfortunately, have postmodern capitalism, which is working very badly.) Corporations are people, real people, because enterprises are real people, whose will is made one through bookkeeping tracking what value they -create and cost for each other. A corporation is not so much a legal fiction, as a book keeping fiction. +create and cost for each other. A corporation is not so much a legal fiction, +as a book keeping fiction, fictions imagined into reality by real people acting as one. -And then King Charles the second gave some of these accounting fictions -made real legal status as corporations, creating the modern joint stock -publicly traded for-profit corporation, the modern corporation. - -Corporations are people, real people, because enterprises are real people, -whose will is made one through bookkeeping tracking what value they -create and cost for each other. +It is not the buildings and the tools that are the corporation, but +the unity of action. This is what makes it possible to move +corporations onto the blockchain, to substitute cryptographic +algorithms for the laws of men. Double entry book keeping is social technology. It fundamentally shapes our society, even though almost no one understands it. The corporation @@ -118,6 +121,10 @@ liability subtypes of double entry accounting. It is triple entry, because the same digitally signed record shows up in many places – and not exactly two, nor for that matter, exactly three. +It is triple entry because double entry shows that entries in the books +agree with each other, while triple entry shows that they agree with +the books of other enterprises. + Sums over these records preserve the invariants of double entry accounting. Failure to preserve the double entry invariants indicates a communication failure, update failure, or disk corruption that forces an @@ -162,7 +169,7 @@ Debit entries are ones that account for the following effects: * Increase in assets * Increase in expense -*Decrease in liability +* Decrease in liability * Decrease in equity * Decrease in income diff --git a/docs/usury.md b/docs/usury.md index cfbbcf3..c685375 100644 --- a/docs/usury.md +++ b/docs/usury.md @@ -42,7 +42,7 @@ And if things don’t work out, he is free and clear if he returns the cattle or the house. The Islamic ban on usury is similar to the Christian ban, but their frame is -rather than the lender shares the risk, rather than the lender is charging +rather that the lender shares the risk, rather than that the lender is charging rental on a productive property. The dark enlightenment frame is game theoretic, that the lender often knows better than the borrower what is a bad loan for the borrower, and should not have incentive to trap the @@ -86,7 +86,7 @@ business when there is a slump in real estate prices, or the cattle business when there is a drought, he lobbies against the Christian rules on usury, and in favour of the Jewish rules. If you buy a house on a mortgage, and the price falls below the mortgage, he wants to sell the house over your head, -and then go after you for the difference. +*and* then go after you for the difference. Since it is rather dangerous to move gold around, and safer to move ownership of gold around, people, instead using gold pieces as money, @@ -99,7 +99,7 @@ the same time for the money they had stashed away for a rainy day, and you have a financial crisis. And, under the rules the bankers lobbied for, the angry depositors can take -all their stuff, and probably give the bankers a horsewhipping. +all their stuff, *and* give the bankers a horsewhipping. So the bankers rush to the government, and say, “financial crisis, bailout” diff --git a/docs/variable-length-quantity.md b/docs/variable-length-quantity.md deleted file mode 100644 index 309ea23..0000000 --- a/docs/variable-length-quantity.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -title: Variable Length Quantity ---- - -I originally implemented variable length quantities following the standard. - -And then I realized that an sql index represented as a Merkle-patricia tree inherently sorts in byte string order. -Which is fine if we represent integers as fixed length integers in big endian format, -but does not correctly sort variable length quantities if we follow the standard: - -So: To represent variable length signed numbers in sequential byte string sortable order so that the integer sequence corresponds one to one to the byte string sequence, a strictly sequential sequence of integers with no gaps corresponding to a strictly sequential sequence of byte strings with no gaps: - -# For positive signed integers - -If the leading bits are $10$, it represents a number in the range\ -$0$ ... $2^6-1$ So only one byte (two bits of header, six bits to represent $2^{6}$ different -values as the trailing six bits bits of an ordinary eight bit bit -positive integer). - -If the leading bits are $110$, it represents a number in the range\ -$2^6$ ... $2^6+2^{13}-1$ So two bytes - -if the leading bits are $1110$, it represents a number in the range\ -$2^6+2^{13}$ ... $2^6+2^{13}+2^{20}-1$ So three bytes long -(four bits of header, twenty bits bits to represent $2^{20}$ different -values as the trailing twenty bits of an ordinary thirty two bit -positive integer in big endian format). - -if the leading bits are $b1111\,0$, it represents a number in the range\ -$2^6+2^{13}+2^{20}$ ... $2^6+2^{13}+2^{20}+2^{27}-1$ So four bytes long -(five bits of header, twenty seven bits to represent $2^{27}$ different -values as the trailing twenty seven bits of an ordinary thirty two bit -positive integer in big endian format). - -if the leading bits are $1111\,10$, it represents a number in the range\ -$2^6+2^{13}+2^{20}+2^{27}$ ... $2^6+2^{13}+2^{20}+2^{27}+2^{34}-1$ -So five bytes long. - -if the leading bits are $1111\,110$, it represents a number in the range\ -$2^6+2^{13}+2^{20}+2^{27}+2^{34}-1$ ... $2^6+2^{13}+2^{20}+2^{27}+2^{34}+2^{41}-1$ -So six bytes long. - -if the leading bits are $1111\,1110$, it represents a number in the range\ -$2^6+2^{13}+2^{20}+2^{27}+2^{34}+2^{41}$ ... $2^6+2^{13}+2^{20}+2^{27}+2^{34}+2^{41}+2^{48}-1$ -So seven bytes long. - -The reason for these complicated offsets is to ensure that the byte string are strictly sequential. - -if the bits of the first byte are $1111\,1111$, we change representations. -Instead that number is represented by a variable -length quantity that is a count of -bytes in the rest of the byte string, which is the number itself in its -natural binary big endian form, with the leading zero bytes discarded. -So no longer using these complicated offsets for the number itself, -but are using them for the byte count. - -This change in representation simplifies coding and speeds up the transformation, -but costs an extra byte for numbers larger than $2^{48}$ and less than $2^{55}$. - -And so on and so forth in the same pattern for positive signed numbers of unlimited size. - -## examples - -The bytestring 0xCABC corresponds to the integer 0x0A7C.\ -The bytestring 0xEABEEF corresponds to the integer 0x0ABCAF. - -# For negative signed integers - -If the leading bits are $01$, it represents a number in the range\ -$-2^6$ ... $-1$ So only one byte (two bits of header, -six bits to represent $2^6$ different values as the -trailing six bits of an ordinary eight bit negative integer). - -If the leading bits are $001$, it represents a number in the range\ -$-2^{13}-2^6$ ... $2^6-1$ So two bytes (three bits of header, -thirteen bits to represent $2^{13}$ different values as the trailing -thirteen bits of an ordinary sixteen bit negative integer in big endian format). - -if the leading bits are $0001$, it represents a number in the range\ -$-2^6-2^{13}-2^{20}$ ... $-2^6-2^{13}-1$ So three bytes long. - -if the leading bits are $0000\,1$, it represents a number in the range\ -$-2^6-2^{13}-2^{20}-2^{27}$ ... $-2^6-2^{13}-2^{20}-1$ -So four bytes long (five bits of header, twenty seven bits to represent -$2^{27}$ different values as the trailing twenty seven bits of -an ordinary thirty two bit negative integer in big endian format). - -if the leading bits are $0000\,01$, it represents a number in the range\ - $-2^6-2^{13}-2^{20}-2^{27}-2^{34}$ ... $-2^6-2^{13}-2^{20}-2^{27}-1$ - So five bytes long. - -if the leading bits are $0000\,001$, it represents a number in the range\ - $-2^6-2^{13}-2^{20}-2^{27}-2^{34}-2^{41}-1$ ... $-2^6-2^{13}-2^{20}-2^{27}-2^{34}-1$ - So six bytes long. - -if the leading bits are $0000\,0001$, it represents a number in the range\ -$-2^6-2^{13}-2^{20}-2^{27}-2^{34}-2^{41}-2^{48}$ ... $-2^6-2^{13}-2^{20}-2^{27}-2^{34}-2^{41}$ -So seven bytes long. - -if the bits of the first byte are $0000\,0000$, we change representations. -Instead that number is represented by a variable length quantity that is -*zero minus the count* of bytes in the rest of the byte string, -which is the negative number itself in its natural binary big endian form, -with the leading minus one bytes discarded. -So no longer using these complicated offset for the number itself, -but are using them for the byte count. -We use the negative of the count, in order to get the correct -sort order on the underlying byte strings, so that they can be -represented in a Merkle-patricia tree representing and index. - -And so on and so forth in the same pattern for negative signed numbers of unlimited size. - -# bitstrings - -Bitstrings in Merkle-patricia tree representing an sql index -are typically very short, so should be represented by a -variable length quantity, except for the leaf edge, -which is fixed size and large, so should not be -represented by variable length quantity. - -We use the integer zero to represent this special case, -the integer one to represent the zero length bit string, -integers two and three to represent the one bit bitstring, -integers four to seven to represent the two bit bit string, -and so on and so forth. - -In other words, we represent it as the integer obtained -by prepending a leading one bit to the bit string. - -# Dewey decimal sequences. - -The only thing we ever want to do with Dewey decimal sequences is $<=>$, -and they are always positive numbers less than $10^{14}$, so we represent them as -a sequence of variable length numbers terminated by the number minus one -and compare them as bytestrings. diff --git a/docs/writing_and_editing_documentation.md b/docs/writing_and_editing_documentation.md index cd5ffca..b881d6f 100644 --- a/docs/writing_and_editing_documentation.md +++ b/docs/writing_and_editing_documentation.md @@ -2,6 +2,7 @@ # katex title: >- Writing and Editing Documentation +sidebar: true ... # Organization @@ -9,67 +10,6 @@ title: >- My stuff is currently an enormous pile of disorganized, and thus inaccessible, documents. -The clean and elegant style of open source documentation always has a -header bar at the top, through which all documents can be accessed. And a -moderately sized logo and title at the top. I am thinking a rho, surrounded -by a flaming halo of green blue radial flames, followed by "Rhocoin" - -Then a top level menu of five or six items, followed by lower level horizontal menus that depend on which higher level menus was selected. - -The best layout I have seen the [wxWidgets style](https://docs.wxwidgets.org/3.0/functions.html){target="_blank"} which has multiple bars, -and when you clicked on entry in one bar, the lower bars changed. -wxSqlite3 used to have this style. Now, has a [single bar with multilevel drop downs](https://utelle.github.io/wxsqlite3/docs/html/index.html){target="_blank"} , as does the [SQLite3 style](https://sqlite.org/docs.html){target="_blank"} , of which it is a direct copy, -but SQLite3 style lacks a way to handle the enormous number of SQLite3 -documents, which can only be handled by a multi level bar, or by a page -full of links - -All major software has on all its pages a constant logo and text, which is not a waste of space for someone stumbling on it over the internet, a button bar for high level navigation, which is different on each child page, though only wxWidgets does the right thing with multiple layers of button bars, which get deeper as you drill down, and then below this horizontal element is the normal substance of a web page, which structure tends to be rather random and adhoc. - -Libsodium, on the other hand, has the logo on top, but instead of a link bar, a [left hand bar with drop downs](https://doc.libsodium.org/secret-key_cryptography){target="_blank"}. Which -is probably easier to implement, but that there is no documentation locally -installed suggests that it too is in some way server generated. Apache2 and -nginx similarly, and handle the enormous number of documents by -bringing up pages full of links. Which is OK, but means you do not have -navigation at your fingertips.] - -This layout is in some way automatically generated on the server, which -sucks. Probably relies on server side include, which is the easiest way to -do it. The documentation needs to be in every install and every repository. -Thus wxWidgets documentation on the server has nice organizational -style, but on each person's individual installed copy, disorganized crap. - -Each bottom level subtree should be a directory, and each html document -in that directory should call the script which generates the horizontal bars -on the path from the root to it. The bash script that uses pandoc to generate -those documents from the markdown documents in that directory should -also generate the javascript, concatenating all the javascripts of the parent -directories into it. - -One tricky bit is that you want the path highlighted. In which case it is -probably easier for the bash script, which is recursing through the tree of -files and keeps track of the path by which it got there in an enormous -string variable, to insert a direct copy of the header bar into each html file, - -The directory name is what appears in the top level bars, and the final bar -is a possibly multiline bar that is the titles of all the documents in the directory -and any subdirectories. - -On reflection, we will not use any cleverness to have a single header bar -file that all html files use because each top bar of each html file will b -different, having different items highlighted, and according to its depth in -the tree, a different number of '../' prepended to the links in the top bar. - -Each markdown file and directory in a directory should have a short -human friendly name, which will correspond to the name in the top bar, -and for each directory `foo` there is a should be a file `foo.link` which is the -path from within that directory that will be the file that comes up when -that directory name in the top bar is clicked on. - -We code a script runs through each directory twice constructing the -necessary bar, and then inserts it directly as a 'before' element in pandoc. -The script will be in `bash`, to run on all systems, and will use `sed` to -generate the bar, to run on all systems, because every computer system -everywhere has `sed` and `bash`. # pandoc @@ -79,17 +19,34 @@ is easier to read, and allows superior control of appearance To convert Pandoc markdown to its final html form, invoke `Pandoc` by the bash shell file `./mkdoc.sh`, which generates html. +Directories with markdown files contain a shell script mkdocs.sh which compiles +them into html, and also executes the mkdocs.sh of each of its +subdirectories, if they have a mkdocs.sh + +The directories also contain a file called navbar, which +is the horizontal navigation menu. + +Each markdown files should start with some yaml data that sets +the title. + +Thus the markdown for this web page starts with + +```yaml +--- +# katex +title: >- + Writing and Editing Documentation +sidebar: true +... +``` + +- `# katex` tells the bash script and pandoc to support katex +- `sidebar: true` causes the table of contents to appear at the side +- `notmine: true` suppresses the copyright license. + In the windows 10 environment, shell scripts used in this project need to be associated with [Git Bash](libraries/git_bash_undocumented.html) or run from within Git Bash. -If the title in the markdown file is followed by `# katex`, as in -the markdown form of this file, the shell script will tell Pandoc to display -any formulae using katex in the html file. - -More precisely, if any of the first three lines in the yaml header specifying -the title at the start of the markdown file are `# katex`, the `./mkdoc.sh` -will tell Pandoc to use katex to display maths formula. - This vast pile of notes is out of control, and writing code and maths in html leads to intolerable overheads. @@ -103,7 +60,7 @@ Pandoc, however, will allow you to take control. To integrate html and css with markdown using Pandoc is a bit like rolling marbles with your elbows through a cage. One has to work through and around the entry points that Pandoc gives you, while if you were writing in html you could -just write what you damn well wanted directly, but having done the work, +just write what you wanted directly, but having done the work, Pandoc can then ensure it is done for every document in the same style in the same way, and you can change the final form of every document in the same way all at once. @@ -133,36 +90,10 @@ Which extensions do not work correctly in Visual Studio Code. These can be used to put an anchor in text, but the easiest and most intelligible way to insert an anchor is as a header. -Pandoc can do a good job of rendering math markdown without invoking -katex, and in such cases, one should generate the html - -```bash -fn=filename - pandoc --toc --eol=lf --wrap=preserve --from markdown+ascii_identifiers+smart+raw_html+fenced_divs+bracketed_spans --to html --metadata=lang:en --verbose --include-in-header=./pandoc_templates/header.pandoc --include-before-body=./pandoc_templates/before.pandoc --include-after-body=./pandoc_templates/after.Pandoc -o $fn.html $fn.md -``` - Since markdown has no concept of a title, Pandoc expects to find the title in a yaml inline, which is most conveniently put at the top, which renders it somewhat legible as a title. -Thus the markdown version of this document starts with: - -```markdown ---- -title: >- - Writing and Editing Documentation -# katex -... -``` - -## Converting html source to markdown source - -In bash - -```bash -fn=foobar -git mv $fn.html $fn.md && cp $fn.md $fn.html && pandoc -s --to markdown-smart+raw_html+native_divs+native_spans+fenced_divs+bracketed_spans --eol=lf --wrap=preserve --verbose -o $fn.md $fn.html -``` ## Math expressions and katex @@ -197,23 +128,6 @@ The square root of 100 is $\sqrt{100}=10$.\ The cubic root of 64 is $\sqrt[3]{64}=4$ $$\bigg\lfloor\frac{x+5)}{6}\bigg\rfloor = \bigg\lceil{\frac{x}{6}}\bigg\rceil$$ -So for documents requiring some heavy maths display, we convert from markdown -to html with, in the bash script `./mkdoc.sh`: - -```bash -fn=filename -pandoc --katex=./ --toc --eol=lf --wrap=preserve --from markdown --to html --metadata=lang:en --verbose --include-in-header=./pandoc_templates/header.pandoc --include-before-body=./pandoc_templates/before.pandoc --include-after-body=./pandoc_templates/after.pandoc -o $fn.html $fn.md -``` - -The `./` tells `pandoc` to expect to find the files - -```bash -./katex.min.css -./katex.min.js -``` - -That a file needs katex is flagged for `./mkdoc.sh` in the yaml header. - A file that does not need katex has the header: ```markdown @@ -308,15 +222,16 @@ In this table, edited in a fixed font, you are using whitespace and blank lines ### Grid tables -Allows multiline, and alignment, but visual studio does not like it, and you still have to count those spaces +Allows multiline, and alignment, but visual studio does not like it, and you still have to count those spaces. +Works fine if you are editing in a monospaced font, otherwise unusable +---------------+---------------+--------------------+ | Fruit | Price | Advantages | +===============+==============:+====================+ -| Bananas | $1.34 | Mary had a little lamb whose fleece was white as snow, and everywhere that | -| | | Mary went the lamb was sure to go | -| | | | -| | | bright color | +| Bananas | $1.34 | Mary had a little | +| | | lamb and every | +| | | where that mary | +| | | bright color | +---------------+---------------+--------------------+ | Oranges | $2.10 | - cures scurvy | | | | - tasty | @@ -341,6 +256,23 @@ Alignments can be specified as with pipe tables, by putting colons at the bounda | Right | Left | Centered | +---------------+---------------+--------------------+ +## footnotes + +Markdown permits footnotes. + +This is a useless feature present because people are moving legacy documents to the web. + +[^long] They can be inline ^[this is an inline footnote] +but an inline footnote appears at the bottom of the document, which is confusing in html +unless the document is short. + +Footnotes were designed to be used with paginated^[footnotes are an ancient hangover from paginated documents in order to do hypertext on paper] documents. +Html is designed for hypertext. + +[^long]: This is an out of line footnote, that can be inserted anywhere, which is confusing in an html document, and hard to edit, because they always wind up sequentially numbered at the bottom of the document. + + Reflect on how unusable wikipedia foot notes are. + # Images Images are preferably `*.webp`, and are expressed as the markdown code @@ -373,7 +305,7 @@ compatible form, and reinsert it in the markdown file. A sequence of straight lines is M point, L point, L point. Z and z draw a straight line back to the beginning, use in conjunction with -`fill="#red"` , for example `fill="#FF0000"`. If the line is open, `fill="none"` +`fill="#red"`, for example `fill="#FF0000"`. If the line is open, `fill="none"` H and h draw horizontal lines, V and v vertical lines. To see which line you are working on, convenient to temporarily begin and end it with a cross, h 3 h-6 h3 v3 v-6 v3 @@ -646,70 +578,4 @@ defined by very small source code. } ); - -# tables - -May Scale of monetary hardness | -|
Hardness | - - |
-
Hard | -|
1 | -Street cash, US dollars | -
2 | -Street cash, euro currencies, japan | -
3 | -Major crypto currencies, such as Bitcoin and Monaro | -
4 | -Street cash, other regions | -
5 | -Interbank transfers of various sorts (wires etc), - bank checks | -
6 | -personal checks | -
7 - | -Consumer-level electronic account transfers (eg - bPay) | -
8 | -Business-account-level retail transfer systems | -
Soft | -|
9 | -Paypal and similar 'new money' entities, beenz | -
10 | -Credit cards | -