A review of potentially useful libraries and utilities.
The material here is usually way out of date and frequently wrong.
It should be treated as a bunch of hints likely to point the reader
in the correct direction, so that the reader can do his homework
on the appropriate library. It should not be taken as gospel.
# Recursive snarks
A horde of libraries are rapidly appearing on GitHub,
most of which have stupendously slow performance,
can only generate proofs for absolutely trivial things,
and take a very long time to do so.
[Nova]:https://github.com/microsoft/Nova
{target="_blank"}
[Nova] claims to be fast, is being frequently updated, needs no trusted setup, and other people are writing toy programs using [Nova].
[Nova] claims you can plug in other elliptic curves, though it sounds like you
might need alarmingly considerable knowledge of elliptic curves in order to
do so.
Plonky had a special purpose hash, such that it was
easy to produce recursive proofs about Merkle trees.
I don't know if Nova can do hashes with useful speed, or hashes at all,
without which no recursive snark system is useful.
We need a hash that has a relatively small circuit.
And it appears that no such hash is known.
Nova is built out of commitments, which are about 256 times bigger than a hash.
A Nova proof is a proof about a merkle tree of commitments.
If we build our blockchain out of Nova commitments, it will about couple of
hundred times larger than one built out of regular hashes,
but will still only occupy about ten or twenty gigabytes of storage.
Bandwidth limits restrict us to about twenty transactions a second,
which is still faster than the bitcoin blockchain.
Plus, when we hit ten or twenty transactions per second,
we can shard the blockchain, which we can do because each shard can prove
it is telling the truth about transactions, whereas with bitcoin,
every peer has to evaluate every transaction,
lest one shard conspire to cheat the others.
[Nova] does not appear to have a language.
Representing a proof system as a Turing machine just seems like a bad idea.
It is not a Turing machine.
You don't calculate $a=b*c$ you instead prove that
$a=b*c$, when you already somehow knew $a$, $b$, and $c$.
A Turing machine is a state machine. A proof system is not.
It is often said, and is in a sense true, that prover produces a proof
that for a given computation he knows an input such that after a
correct execution of the computation he obtains a certain public output.
But no he his not. The proof system proves that relationships hold between values.
And because it can only prove certain rather arcane and special things about
relationships between values, you have to compute a very large number
of intermediate values such that the relationship you actually want to prove
between the input and the output corresponds to simple relationships between
these intermediate values. But computing those intermediate values belongs
in an another language, such as C++ or rust.
With Nova, we would get an algorithm such that you start out with your real input.
You create a bunch of intermediate values in a standard language like C++ or rust,
then you call the proof system to produce a data structure
that can be used to prove relationships between your input and those
intermediate values.
Then you produce the next set of intermediate values,
call your proof system to produce a data structure
that can be used to prove the next set of relationships,
fold those two proof generating data structures together,
rinse and repeat,
and at the end you generate a proof that the set of relationships
the fold represents is valid.
That is procedural, but expressing the relationships is not.
Since your fold is the size of the largest hamiltonian circuit so far,
you want the steps to be all of similar size.
This suggests a functional language (sql). There are, in reality,
no purely functional languages for Turing machines.
Haskell has its monads, sql has update, insert, and delete.
But the natural implementation for a proof system would be a truly purely functional language, an sql without update, insert, or delete, without any operations that actually wrote anything to memory or disk, that simply defined relationships without a state machine that changes state to write data into memory consistent with those changes.
The proof script has to be intellible, and the same for prover and verifier,
the difference being that the prover interleaves the proof language with ordinary code
in ordinary language, to produce the values that are going to be proven. The prover
drives the script along with ordinary language code, and verifier drives it along
with different ordinary language code, but the proof definition that is common
to both of them has no concept of being sequential and driven along,
no concept that things are done in any particular order.
It is a graph of relationships.
The proof language, as is typical of purely functional languages,
should consist of assertions of about relationships between immutable
data structures, without expressing the idea that some of these
data structures were created at one time, and destroyed at another.
Some of these values are defined recursively, which means that what
is actually going to happen in practice is that they are going to be
created by a loop, written in the ordinary procedural language
such as Rust or C++, but the proof language should have no concept of that.
if the proof language asserts that $1 \leq 0 \lor n<20 \impliesf(n-1)=g(f(n))$,
the ordinary procedural language will likely need to
generate the values of $f(n) for n=1 to 19,
and will need to cause the proof language to generate proofs for each value
of $n$ for 1 to 19, but the resulting proof will be independent
of the order in which these these proofs were generated.
Purely functional languages like sql do not prescribe an algorithm, and need
a code generator that has to make guesses about what a good algorithm would
be, as exemplified by sqlite's likelihood, likely, and unlikely no-ops.
And with a proof system we will, at least at first, have the human
make the algorithm but if he changes the algorithm,
while leaving the proof system language unchanged, will still work.
The open source nature of Plonky is ... complicated.
The repository on Github has been frozen for two years, so likely
Their [distributed hash table] (kad-dht) seems to be very much a work in progress. Checking their [implementations] page for the status of various libp2p components, *everything*
seems to be very much a work in progress. Some things are implemented
in some languages which are not implemented in other languages. Rust has
hole punching, C++ does not. But C++ has a whole lot of stuff that Rust
does not. And their documentation on kad-dht has its page blank. On the other hand,
ipfs works, and polkadot works. So, something is usable. libp2p-peer not implemented
in C++, but is implemented in rust, and is implemented in browser javascript.
Their rendevous protocol presupposes a single central and known server
which simply records everone's ID and network address. Unacceptable.
Anyone using this is fake and an enemy. Should be disabled in our fork.
libp2p is a pile of odds and ends and a framework for gluing them together.
But everything they do is crippled by the fact that you don't know the
likely uptime, downtime, or IP stability of an entity. Which cannot
in fact be known, but you can form probability of a probability estimates.
What is needed is that everyone forms their own probability of a probability.
And they compare what they know
(they have a high probability of a probability)
with other party's estimates, and rate the other party's reliability accordingly
If we add to that probability of probability estimate of IP and port stability
and use it to govern ping time and keep around time, that goes a large way
to solving the the problems with Kademlia
We can adapt it to the problem
by having them preferentially keep around the data for peers
that have stable ip and a stable port, and,
somewhat less preferentially, keep around peers that have a
stable nat penetration or tracking relationship with a peer that has a
stable ip and stable port. Selective pinging. You rarely ping peers
that have been around for a very long time with stable IP, and you
ping a peer that has a nat penetration relationship by not pinging it,
and instead asking the gateway peer how the relationship is going,
at infrequent intervals. Thus, a peer with stable IP
or stable relationship becomes very widely known.
Well, becomes widely known assuming shills do not register
one billion addresses that happen to be near him.
libp2p is something between actual code, and set of standards -
which standards you comply with so that you can use other people's code.
Someone writes something ad hoc for his use case, stuffs it into libp2p
somewhere somehow so that he can use other people's code.
Then other people use his code.
It is a pile of standards (many of them irritatingly stupid, incomplete, ad hoc,
or workarounds for using defective tools) that enable a
whole lot of people writing this sort of thing to copy a whole lot of
each other's code.
Their [NAT discovery algorithm](https://github.com/libp2p/specs/tree/master/autonat){target="_blank"}
Is particularly idiotic and broken. It is not a nat discovery algorithm,
but a closed port discovery algorithm, and a ludicrously laborious,
costly, indirect, error prone, and inefficient closed port discover algorithm.
NAT means the other guy sees your network address different from what you see.
In which case your port is probably closed, but could well be open.
If he sees the same network address as you, your port might be open,
but you don't know that,
and talking to the other guy might well temporarily open your port,
with the result that he might tell you that you are not behind a NAT,
when in fact you are, and your ports are normally closed.
The guys writing this stuff are dumb as posts,
and a whole lot of what they write is garbage.
But, nonetheless, a whole lot of people are using libp2p,
and a whole lot of people are doing a whole lot of work on it --
If looking for something to try out that begins to have the right shape, see [Manyverse] that uses the [Scuttlebutt] protocol. Jim is fond of Bitmessage and it is quite secure, but it has a big weakness in that it needs to flood every message to all nodes in the world so everyone can try their private key to see if it works for decryption. That won't scale. (Can't avoid passing every message on to all callers even if you know it's yours, as you don't want to let someone snooping see that you absorbed a message which is an information disclosure.)
Instead [Manyverse] and [Scuttlebutt] allow for publishing and reputation for a public key. The world can see what a key (its author really) publishes. Can publish public posts like a blog, signed by the private key for authenticity and verifiability. Also can publish private messages (DMs), visible to the world but whose contents are encrypted. Weakness in private messages is that the recipient public key is visible on the message - good for routing and avoiding testing every message, bad for privacy. Would be better to have a 3rd mode, private for "someone" but you have to test your private key to see if it's for you. That should not be hard to add to [Scuttlebutt] and [Manyverse].
Reputation is similar to what Jim has proposed: You can specify a list of primary keys/people (friends) you want to listen to and watch. You can also in the interface specify how many degrees of separation you want to see outward from your friends - the public messages from their friends and friends' friends. Presumably specifying 6 degrees gets you Kevin Bacon and the rest of the [Manyverse]. You can also block friends' friends so their connections are not visible to you - so if you don't like a friend's friend you at least don't have to listen any more.
Another advantage is that [Manyverse] works in a sometimes-connected universe: Turn off your computer or phone for days, turn back on, catch up to messages. You realy don't even have to be on the public Internet, you could sneakernet or local/private-net messages which is nice for, say, messaging in a disaster or SHTF scenario where you have a local wifi network while the main network connections are down. Bitmessage has a decay/lifetime for messages that means you need to be connected at least every 2-3 days.
Biggest weakness is hosting. Your service can be hosted by 3rd parties like any service, and you can host your own. Given the legal landscape as well as susceptibility to censorship via DDoS and hack attacks, you want to have your own server. There are some public servers but sensibly they don't want a rando or glowie from the net jumping on there to drop dank memes. But hosting is nontrivial to carve out your own network bubble that can see the Internet (at least periodically) while being fully patched and DDoS resistant.
Of course missing from this from Jim's long list of plans are DDoS protection, a name service that provides name mapping to key hierarchies for messaging and direct communications, and a coin tie-in. But [Manyverse] at least has the right shape for passing someone a message with a payment inside, while using a distributed network and sometimes connection with store-and-forward to let you avoid censorship-as-network-damage. A sovereign corporation can also message publicly or privately using its own sovereign name and key hierarchy and private ledger-coin.
The net is vast and deep. Maybe we need to start cobbling these pieces together. The era of centralized censorship needs to end. Musk will likely lose either way, and he's only one man against the might of so many paper tigers that happen to be winning the information war.
You will import a submodule from someone else's project, but eventually you and your team are going to make minor changes to it to customize it to your project. In which case you will need your own remote team repo, in place of the original other team's repo.
Construct the copied remote repositories so that their default branch is your tracking branch, not the upstream branch.
``` bash
git init --bare -b «our_branch»
```
Then, in your local repository where you created the new branch, reset
the remote in your local repository to the new remote by editing `.gitconfig`
and push «our_branch» from your local in which you created your team's new
If you fail to set your remote to your team's default branch, then your
local repositories will keep getting reset back to their team's branch, and
chaos ensues.
When moving the remote of a submodule in your local repository (usually from their team's remote to your team's remote) you update `.gitmodules` in your superproject, and in each submodule that has submodules of its own, then
Your submodule remotes propagate from `.gitmodules` file of your superproject, and their branch propagates from the superproject *and* from the remote default branch.
And the branch will *also* propagate from `.gitmodules` if `branch = ...` is set.
Because git is a distributed archive, it is perfectly possible, and often
necessary, to work with all these set to different values, *provided* that
everyone is mindful that they are set to different values, and the
consequences and implications of them being set to different values. Which
consequences and implications get complicated, unobvious, difficult to
predict, and surprising when you are working with submodules.
That library does not contain a collection of words organized by part of speech. Instead it calls a python library (word.net) of english, which has the information you actually need.
But no end of people say that sucky though it is, it is the standard
way to create install files.
[Hello World for Wix]:https://stackoverflow.com/questions/47970743/wix-installer-msi-not-installing-the-winform-app-created-with-visual-studio-2017/47972615#47972615
{target="_blank"}
[Hello World for Wix] is startling nontrivial. It does not default create
a minimal useful install for you. So even if you get it working, still
Choco, Chocolatey, is the Windows Package manager system. Does not use `*.msi` as its packaging system. A chocolatey package consists of an `*.nuget`, `chocolateyInstall.ps1`, `chocolateyUninstall.ps1`, and `chocolateyBeforeModify.ps1` (the latter script is run before upgrade or uninstall, and is to reverse stuff done by is accompanying
Interaction with stuff installed by `*.msi` is apt to be bad.
The community distribution redirects requests to particular servers,
which have to be maintained by particular people - which requires
an 8GB ram, 50GB disk Windows server. I could have `nginx` in the
cloud reverse proxying that to a physically local server over
wireguard, which solves the certificate problem, or I could use a
commercial service, which is cheap, but leaks identity all over the
place and is likely to be subject to hostile interdiction and state sponsored identity theft.
Getting on the `choco` list is largely automatic. Your package has to
install on their standard image, which is a deliberately obsolete
2012 windows server - and your install script may have to install
windows update packages. Your package is unlikely to successfully
install until you have first tested it on an imitation of their test
environment, which is a great deal of work and skill to set up.
Human curation exists, but is normally routine and superficial.
Installs, has license, done.
[whole lot more checks]:https://docs.chocolatey.org/en-us/information/security#chocolatey.org-packages
{target="_blank"}
[whole lot more rules]:https://docs.chocolatey.org/en-us/community-repository/moderation/package-validator/rules/
{target="_blank"}
Well, actually there are a [whole lot more checks], which enforce a [whole lot more rules], sixty eight rules and growing, but they are robotically checked and the outcome reported to human. If the robot OKs it, it normally goes through automatically into the community distribution.
A Choco package is immutable. Can be superseded, but cannot
change. Could have the program check for a Zooko signature of its package file against a list, and look for indications of broad
approval, thus solving the identity problem and eating my own dogfood.
Choco packages would be very handy to automatically install my build environment.
### Cmake
`cmake` has a pipeline for building choco files.
[wxWidgets has instructions for building with Cmake]:https://docs.wxwidgets.org/trunk/overview_cmake.html
{target="_blank"}
[wxWidgets has instructions for building with Cmake]. My other
libraries do not, and require their own idiosyncratic build scripts,
and I doubt that I can do what the authors were disinclined to do.
Presumably I could fix this with `add_custom_target` and
`add_custom_command`, where the custom command is bash script
that just invokes the author's scripts, but I just do not understand
the documentation for these commands, which documentation
resupposes knowledge of the incomprehensible domain specific language.
`Cmake` runs on both Windows and Linux, and is a replacement for autotools, that runs only on Linux.
Going with `cmake` means you have a defined standard cross platform development environment, `vscode` which is wholly open source, and a defined standard cross platform packaging system, or rather four somewhat equivalent standard packaging systems, two for each platform.
`cmake --install` installs from source, and has a pipeline (`cpack`)
to generate `*.msi` through [NSIS]. Notice it does *not* have a pipeline
through Wix and Wax. It also has a pipeline to Choco, and, on linux,
to `*.deb` and `*.rpm`.
No uninstall, which has to be hand written for your distribution.
`cmake` has the huge advantage that with certain compilers, far from
all of them, it integrates with the vscode ide, including a graphical
debugger that runs on both windows and linux. Which otherwise
you really do not have on linux.
It thus provides maximum cross platform portability. On the other
hand, all of my libraries rely on `.configure && make && make install`
on linux, and on visual studio on Windows. In my previous
encounter with `cmake`, I found mighty good reason for doing it that
way. The domain specific language of `CMakeLists.txt` is arcane,
unreadable, unwriteable, and subject to frequent, arbitrary,
inconsistent, and illogical ad hoc change. It inexplicably does
remarkably complicated things without obvious reason or purpose,
which strange complexity usually does things you do not want.
Glancing through their development blog, I keep seeing major
breaking changes being corrected by further major breaking
changes. Internals are undocumented, subject to surprising change,
and likely to change further, and you have to keep editing them,
without any clearly knowable boundary between what is internal
stuff that you should not need to look at and edit, and what is the
external language that you are supposed to use to define what
`cmake` is supposed to accomplish. It is not obvious how to tell `cmake` to do a certain thing, and looking at a `CmakeLists.txt` file, not at all obvious what `cmake` is going to do. And when the next
version comes out, probably going to do something different.
But allegedly the domain specific language of `./configure` has
grown a multitude of idiosyncrasies, making it even worse.
`ccmake` is a graphical tool that will do some editing of
`CMakeLists.txt` with respect for the mysterious undocumented
arcane syntax of the nowhere explained or documented domain
specific language.
# Library Package managers
Lately, however, library package managers have appeared: Conan and [vcPkg](https://blog.kitware.com/vcpkg-a-tool-to-build-open-source-libraries-on-windows/). Conan lacks wxWidgets, and has far fewer packages than [vcpkg](https://libraries.io/github/Microsoft/vcpkg).
A bitmessage client written in C. Designed to run on a linux mail server
and interface bitmessage to mail. Has no UI, intended to be used with the linux mail UI.
Unfortunately, setting up a linux mail server is a pain in the ass. Needs the Zooko UI.
But its library contains everything you need to share data around a group of people, many of them behind NATs.
Does not implement NAT penetration. Participants behind a NAT are second class unless they implement port forwarding, but participants with unstable IPs are not second class.
A reliable udp library with congestion control which has vastly more development work done on it than any other reliable udp networking library, but which is largely used to work with Steam gaming, and Steam's closed source code. Has no end of hooks to closed source built into it, but works fine without those hooks.
Written in C++. Architecture overly specific and married to Steam. Would
have to be married to Tokio to have massive concurrency. But you don't
need to support hundreds of clients right away.
Well, perhaps I do, because in the face of DDOS attack, you need to keep
a lot of long lived inactive connections around for a long time, any of
which could receive a packet at any time. I need to look at the
GameNetworkingSockets code and see how it listens on lots and lots of
sockets. If it uses [overlapped IO], then it is golden. Get it up first, and it put inside a service later.
on the other hand, is elegant, short, and self explanatory.
The code project has [example code written in C++](https://www.codeproject.com/Articles/13071/Programming-Windows-TCP-Sockets-in-C-for-the-Begin), but it is still mighty intimidating compared to the QT client server example. I have yet to look at the wxWidgets client server examples – but looking for wxWidgets networking code has me worried that it is a casual afterthought, not adequately supported or adequately used.
ZeroMQ is Linux, C, and Cish C++.
Boost Asio is highly praised, but I tried it, and concluded its architecture
is broken, trying to make simplicity and elegance where it cannot be made,
resulting in leaky abstractions which leak incomprehensible complexity the
moment you stray off the beaten path – I feel they have lost control of their
design, and are just throwing crap at it trying to make something that
cannot work, work. I similarly found the Boost time libraries failed, leaking
complexity that they tried to hide, with the hiding merely adding complexity.
[cpp-httplib](https://github.com/yhirose/cpp-httplib) is wonderful in its
elegance, simplicity, and ease of integration. You just include a single
header. Unfortunately, it is strictly http/https, and we need something that
can deal with the inherently messy lower levels.
[Poco](http://pocoproject.org/) does everything, and is C++, but hey, let us first see how far we can get with wxWidgets.
Further, the main reason for doing https integration with the existing
browser web ecosystem, whose security is fundamentally broken, due the
state’s capacity to seize names, and the capacity of lots of entities to
intercept ssl. It might well be easier to fork opera or embed chromium. I
notice that Chromium has features supporting payment built into it, a bunch
of “PaymentMethod\*\*\*\*\*Event”
The best open source browser, and best privacy browser, is Opera, in that it comes from an entity less evil than Google.
[Opera](https://bit.ly/2UpSTFy) needs to be configured with [a bunch of privacy add ons](https://gab.com/PatriotKracker80/posts/c3kvL3pBbE54NEFaRGVhK1ZiWCsxZz09) [HTTPS Everywhere Add-on](https://bit.ly/2ODbPeE),
[uBlock](https://bit.ly/2nUJLqd), [DisconnectMe](https://bit.ly/2HXEEks), [Privacy-Badger](https://bit.ly/2K5d7R1), [AdBlock Plus](https://bit.ly/2U81ddo), [AdBlock for YouTube](https://bit.ly/2YBzqRh), two tracker blockers, and three ad blockers.
It would be great if we could make our software another addon, possibly chatting by websocket to the wallet.
The way it would work be to add another protocol to the browser:
ro://name1.name2.name3/directory/directory/endpoint. When you connect to such
an endpoint, your wallet, possibly a wallet with no global name, connects to
the named wallet, and gets IP, a port, a virtual server name, a cookie
unique for your wallet, and the hash of the valid ssl certificate for that
name, and then the browser makes a connection to the that server, ignoring
the CA system and the DNS system. The name could be a DNS name and the
certificate a CA certificate, in which case the connection looks to the
server like any other, except for the cookie which enables it to send
messages, typically a payment request, to the wallet.
We could implement transaction outputs and inputs as a fixed amount of
fungible tokens, limited to $2^{64}-1$ tokens, using [Safeint] That will be
future proof for a long time, but not forever.
Indeed, anything that does not use Zksnarks is not future proof for the
indefinite future.
Or we could implement decimal floating point with unlimited exponents
and mantissa implemented on top of [MPIR]
Or we could go ahead with the canonical representation being unlimited
decimal exponent and unlimited mantissa, but the wallet initially only
generates, and only can handle, transactions that can be represented by[Safeint], and always converts the mantissa plus decimal exponent to and
from a safeint.
if we rely on safeint, and our smallest unit is the microrho, that is room for
eighteen trillion rho. We can start actually using the unlimited precision of
the exponent and the mantissa in times to come - not urgent, merely
architect it into the canonical format.
From the point of view of the end user, this will merely be an upgrade that
allows nanorho, picorho, femptorho, attorho, zeptorho, yoctorho, and allows a decimal point in yoctorho quantities. And then we go to a new unit, the jim, with one thousand yottajim equals one yoctorho, a billion yoctojim equals one attorho, a trillion exajim equals one attorho.
To go all the way around to two byte exponents, for testing purposes, will
need some additional new units after the jim. (And we should impose a
minimum unit size of $10^{-195}$ rho or $10{-6} rho, thereby ensuring
that transaction size is bounded while allowing compatibility for future expansion.)
Except in test and development code, any attempt to form a transaction
involving quantities with exponents less than $1000^{-2}$ will cause a
gracefully handled exception, and in all code any attempt to display
or perform calculations on transaction inputs and outputs for which no
display units exist will cause an ungracefully handled exception.
In the first release configuration parameters, the lowest allowed exponent
will be $1000^{-2}$, corresponding to microrho, and the highest allowed
exponent $1000^4$, corresponding to terarho, and machines will be
programmed to vote "incapable" and "no" on any proposal to change those
parameters. However they will correctly handle transactions beyond those
limits provided that when quantities are expressed in the smallest unit of
any of the inputs and outputs, the sum of all the inputs and of all the
outputs remains below $2^{64}$. To ensure that all releases are future
compatible, the blockchain should have some exajim transactions, and
unspent transaction outputs but the peers should refuse to form any more
of them. The documentation will say that arbitrarily small and large new
transaction outputs used to be allowed, but are currently not allowed, to
reduce the user interface attack surface that needs to be security checked
and to limit blockchain bloat, and since there is unlikely to be demand for
this, this will probably not be fixed for a very long time.
Or perhaps it would be less work to support humungous transactions from
the beginning, subject to some mighty large arbitrary limit to prevent
denial of service attack, and eventually implementing native integer
handling of normal sized transactions as an optimization, for transactions where all quantities fit within machine sized words, and rescaled intermediate outputs will be less than $64 - \lceil log_2($number of inputs and outputs$) \rceil$ bits.
Which leads me to digress how we are going to handle protocol updates:
## handling protocol updates
1. Distribute software capable of handling the update.
1. A proposed protocol update transaction is placed on the blockchain.
1. Peers indicate capability to handle the protocol update. Or ignore it,
My experience with Boost is that it is no damned good: They have an over
elaborate pile of stuff on top of the underlying abstractions, which pile has high runtime cost, and specializes the underlying stuff in ways that only
work with boost example programs and are not easily generalized to do what
one actually wishes done.
Their abstractions leak.
[Boost high precision arithmetic `gmp_int`]:https://gmplib.org/
[Boost high precision arithmetic `gmp_int`] A messy pile built on top of
GMP. Its primary benefit is that it makes `gmp` look like `mpir` Easier to use [MPIR] directly.
The major benefit of boost `gmp` is that it runs on some machines and
operating systems that `mpir` does not, and is for the most part source code
compatible with `mpir`.
A major difference is that boost `gmp` uses long integers, which are on sixty
four bit windows `int32_t`, where `mpir` uses `mpir_ui` and `mpir_si`, which are
on sixty four bit windows `uint64_t` and `int64_t`. This is apt to induce no
end of major porting issues between operating systems.
Boost `gmp` code running on windows is apt to produce radically different
results to the same boost `gmp` code running on linux. Long `int` is just not
portable, and should never be used. This kind of issue is absolutely typical
of boost.
In addition to the portability issue, it is also a typical example of boost
abstractions denying you access to the full capability of the thing being
abstracted away. It is silly to have a thirty two bit interface between sixty
four bit hardware and unlimited arithmetic precision software.
MySQL 5.7 supports [X Plugin / X Protocol, which allows asynchronous query execution and NoSQL But X devapi was created to support node.js and stuff. The basic idea is that you send text messages to mysql on a certain port, and asynchronously get text messages back, in google protobuffs, in php, JavaScript, or sql. No one has bothered to create a C++ wrapper for this, it being primarily designed for php or node.js](https://dev.mysql.com/doc/refman/5.7/en/document-store-setting-up.html)
SQLite nominally has synchronous access, and the use of one read/write
thread, many read threads is recommended. But under the hood, if you enable
WAL mode, access is asynchronous. The nominal synchrony sometimes leaks into
the underlying asynchrony.
By default, each `INSERT` is its own transaction, and transactions are
excruciatingly slow. Wal normal mode fixes this. All writes are writes to the
writeahead file, which gets cleaned up later.
The authors of SQLite recommend against multithreading writes, but we
do not want the network waiting on the disk, nor the disk waiting on the
network, therefore, one thread with asynch for the network, one purely
synchronous thread for the SQLite database, and a few number crunching
threads for encryption, decryption, and hashing. This implies shared
[Libpcap and Win10PCap](https://en.wikipedia.org/wiki/Pcap#Wrapper_libraries_for_libpcap) provide very low level, OS independent, access to packets, OS independent because they are below the OS, rather than above it. [Example code for visual studio.](https://www.csie.nuk.edu.tw/~wuch/course/csc521/lab/ex1-winpcap/)
[Simple sequential procedural socket programming for windows sockets.](https://www.binarytides.com/winsock-socket-programming-tutorial/)
If I program from the base upwards, the bottom most level would be a single
thread sitting on a select statement. Whenever the select fired, would
execute a corresponding functor transfering data between userspace and system
space.
One thread, and only one thread, responsible for timer events and
transferring network data between userspace and systemspace.
If further work required in userspace that could take significant time (disk
operations, database operations, cryptographic operations) that functor under
that thread would stuff another functor into a waitless stack, and a bunch
of threads would be waiting for that waitless stack to be signaled, and one
of those other threads would execute that functor.
The reason we have a single userpace thread handling the select and transfers
between userpace and systemspace is that that is a very fast and very common
operation, and we don’t want to have unnecessary thread switches, wherein
one thread does something, then immediately afterwards another thread does
almost the same thing. All quickie tasks should be handled sequentially by
one thread that works a state machine of functors.
The way to do asynch is to wrap sockets in classes that reflect the intended
use and function of the socket. Call each instance of such a class a
connection. Each connection has its own state machine state and its own
You will need to place all UTF‑8 string literals and string constants in a
resource file, which you will use for translated versions.
If you fail to set the compilation and run time environment to UTF‑8 then
for extra confusion, your debugger and compiler will *look* as if they are
handling UTF‑8 characters correctly as single byte characters, while at
least wxString alerts you that something bad is happening by run time
translating to the null string.
Automatic string conversion in wxWidgets is *not* UTF‑8, and if you have
any unusual symbols in your string, you get a run time error and the empty
string. So wxString automagic conversions will rape you in the ass at
runtime, and for double the confusion, your correctly translated UTF‑8
strings will look like errors. Hence the need to make sure that the whole
environment from source code to run time execution is consistently UTF‑8,
which has to be separately ensured in three separate place.
When wxWidgets is compiled using `#define wxUSE_UNICODE_UTF8 1`,
it provides UTF‑8 iterators and caches a character index, so that accessing
a character by index near a recently used character is fast. The usual
iterators `wx.begin()`, `wx.end()`, const and reverse iterators are available.
I assume something bad happens if you advance a reverse iterator after
writing to it.
wxWidgets compiled with `#define wxUSE_UNICODE_UTF8 1` is the
way of the future, but not the way of the present. Still a work in progress
Does not build under Windows. Windows now provide UTF8 entries to all
its system functions, which should make it easy.
# [UTF8-CPP](http://utfcpp.sourceforge.net/ "UTF-8 with C++ in a Portable Way")
A powerful library for handling UTF‑8. This somewhat duplicates the
facilities provided by wxWidgets with `wxUSE_UNICODE_UTF8==1`
For most purposes, wxString should suffice, when it actually works with
UTF8. Which it does not yet on windows. We shall see. wxWidgets
recommends not using wxString except to communicate with wxWidgets,
and not using it as general UTF‑8 system. Which is certainly the current
state of play with wxWidgets.
For regex to work correctly, probably need to do it on wxString's native
UTF‑16 (windows) or UTF‑32 (unix), but it supposedly works on `UTF8`,
assuming you can successfully compile it, which you cannot.
# Cap\'n Proto
[Designed for a download from github and run cmake install.](https://capnproto.org/install.html) As all software should be.
But for mere serialization to of data to a form invariant between machine
architectures and different compilers and different compilers on the same
machine, overkill for our purposes. Too much capability.
# Awesome C++
[Awesome C++] A curated list of awesome C/C++ frameworks, libraries, resources, and shiny things
[Awesome C++]:https://cpp.libhunt.com
"A curated list of awesome C/C++ frameworks, libraries, resources, and shiny things"
{target="_blank"}
I encountered this when looking at the Wt C++ Web framework, which seems to be mighty cool except I don't think I have any use for a web framework. But [Awesome C++] has a very pile of things that I might use.
Wt has the interesting design principle that every open web page maps to a
windows class, every widget on the web page, maps to a windows class,
every row in the sql table maps to a windows class. Cool design.
