Need to write up
	docs/lightning_layer.html
    docs/white_paper.html
    docs/white_paper_YarvinAppendix.html
	redo paxos from the point of view that peer knows that majority of stake holders know that  everyone knows that everyone knows that everyone knows.




UI design.

qBittorrent has tabs if there is more than one line, suppresses the tabs if there is only a single tab.

Steal the thunderbird design for messaging, and base it on the wxWidget aui app, which is similar.

Have a tree of wallet names, whose branches are messages, with subtrees of inbox, inbox read, inbox unread, sent, trashed, spammed, or dealt with messages, plus any other subnodes the user might create.

Have only zero or one wallet loaded in a single frame and single gui process, as multiple wallets in the same frame is a confusing exception with confusing UI required, which UI is a permanent tax on the user, albeit zero or one is no harder on the programmer than zero or many, if you are going to have to put up tabs anyway).

We have two pools of data on disk, the wallet, which contains private information, and the pool, which contains public information.  The wallet contains links to the pool.

A message view should show a message in the context of previous messages two or from that entity, as should a message composition view (steal from wordpress sms)

For wallet creation, we use a series of pages based on "https://www.monero.how/tutorial-how-to-use-the-monero-gui-wallet"

When you launch evolution, the debian email client, its frame is occuppied by a wizard, that guides you through initial email setup.  If you cancel out, you enter the entirely useless regular main window, from which you can add an email account through the regular interface.

Evolution and Visual studio allow you to cancel out of the wizard into a mainscreen that comes up, with most of its operations being useless and cheerfully displaying no data to be operated on, the only effective operations being to manually do what the wizard does.  This is a tax on the programmer, and I don't think I will allow that because every special case has to be unit tested, and that winds up being twice as many unit tests, albeit trivial unit tests.  If you close your wallet file, you get dropped back into the wizard, and if you load a new wallet file, the old wxPanel is displaying the old wallet file destroyed, thereby closing the database connection which is owned by wxPanel, and a new one created.  It is trivial to allow multiple wallet panels to be displayed, but more UI is more user confusion.

And twice as many ways of doing something is twice as many things to be written, debugged, and for the user to learn.

If you have no wallet open, you have no wallet manipulation UI on the screen, and if you have two wallets open, the UI and database connection for the previous wallet is immediately destroyed as soon as the UI for the new one is successfully constructed.

We have a permanent frame, but its main wxPanel object can be destroyed and replaced, and one of the wxPanel objects it might contain can itself contain multiple tabbed wxPanel objects.

Messaging protocol:  We don't want to reinvent the wheel - and we want to support video conferencing, which absolutely necessitates integration with someone else's messaging.  Basically we want to pillage someone else's messaging, but have email like interface for at least some messages (primarily the ones money related)

Serverless:  BitMessage, Briar, Echo, Jami, Tox

Echo supports media synchronization, which is handy

Ricochet is a tor based messaging system, to Tor addresses.

Jami and qTox support video

qtox uses qt, and more modern encryption than Jami.

Jami uses forty character hashes as addresses, which on reflection I probably need to take care of chain-of-signatures identities.

Secrets, encryption, and append only data structures are only meaningful and useful when different entities have different unshared secrets.

Hence, we are approaching the point where we cannot write anything useful, except for communicating entities.

Time for a step that requires communication, peers chatting with peers.

But before I can do that, have to implement some data for them to chat about.

So, names identified with a rowid integer, a public key, a set of nameserver rowids, and network address.  Initially we will not implement nameservice, nor any mechanism for ensuring that the list represents a global consensus.  The data will be stored on a machine local database, and the secret keys in wallet files on user local databases.

And the first thing we set up is chat about network addresses, in the clear, and not securely identified, and the only thing stored in a user wallet file is a set of master keys, possibly cold, and a set of derivation paths from those master keys, so that given a public key, you can check if you should be able to construct its secret key.

When we initially set up communication, we fail to make any use of the public key information.  Once we have a completely insecure and totally trust based system working, with secrets available, then we start using those secrets to harden it.  But we make the secrets available, though not the machinery for using them, before we implement communication.

So, order of implementation:

1.  Database containing human readable names, public keys, secret keys, network addresses

2.  Network Chat about network addresses, names, and public keys.

3.  Put those secret keys to work on those public keys.

We then set up serialization and deserialization of a view into this database that maps network addresses to rowids.

The first step is an entity that on launch, checks its database for who to talk to, and then updates network address database to reflect the append only updates of the entities listed, in the clear, and unauthenticated.

Then we institute encryption and ids, which requires each entity to know (and have hot) its secret key.  Then we introduce indirection on public keys, so that entities can have cold secrets, derived secrets, and use hot, time limited,  derived secrets.

But before we can do work on authority and secrets, have to have communication, for secrets, authority, and authentication have no meaning without communication.  And since we have to communicate something, we will start by communicating the map between rowids and network addresses.

When we finally get around to having the equivalent of bitcoin wallet files, they will be sql3 format files, and will have no serialization format.

But everything else the could potentially be attested to by the canonical total blockchain, will have a serialization and deserialization format, and will therefore be stored in serial files, which might well have a somewhat human intelligble ascii armor format, among their formats.

I would like to have software that makes something like a blog with comments and updates available through a bittorrent like mechanism.  But that can wait.  The first thing required is a nameserver like mechanism that distributes authoritative keys for names.

We start out with database entries listing public keys of names, and network addresses for names, and serialization and deserialization for these database entries.

We then add database entries for scalars (private keys) corresponding to the public keys - with different entities owning different names.  We make one hard coded name owner of the authoritative list of names,

And then we add networking, so that entities can get updates of the list, and updates of the network addresses (for which each name is its owning authority.)

We then make a blockchain for name ownership, with complete power the one hard coded name to revise it as it pleases.

We then replace the hard coded name with a set of names with ownership shares attested to, anc changeable in, the blockchain.

We then introduce the the paxos protocol for agreement on new ownership and new share ownership.  We then introduce the rule that no agreement will happen unless the updates are rightly derived from signed transfers by the previous owners, which updates are recorded in the blockchain.

We then have proof of stake crypto currency and namesystem, albeit one that is far from scalable.  Call that the alpha release - not yet ready for prime time, but ready for people to look at and play with.




We need unit test to interface with the gui, so that it pops up the unit test in progress modeless window, which then gets replaced by a modeless window displaying the result of the unit test.  This will save endless clicking around to view the results of unit test, and is a necessary platform to make unit test accessible from the gui, and capable of unit testing gui features.


Why do the constructors in my unit test that build points from naked data work?

I want to make the naked bytestream available, but only by explicit invocation, and I want to make it writeable, but only by explicit invocation.

So I was thinking I would have an explicit templated constructor that forwarded construction arguments, but to my surprise, seems to exist by default or implicitly, which is potentially dangerous.


Cannot write unit test for the sqlite code till I have secrets generation.

cannot finish the secrets code till I have serialize and deserialize

Redo the hash function to run off serialize.

Fix serialize integer so that it actually gets compiled and called.

Then finish the secrets code.

Introduce a special type deserialize, and have constructors that use it for each particular type.  thus point(deserialize(uint8_t*p, unsigned int size))
Or maybe we just require the class to support the method "deserialize"

But the method "serialize" has to return a span, or a class derived from span, so I guess "deserial" is a class that supports a method which that given a length, returns a span, or throws an exception

So, maybe we have a class that supports the methods "serialize" and "deserialize", and also a constructor called "serial", but which has to be invoked explicitly.

For feeding stuff into a hash, we just want an function "serialize" that returns a span.  And for composite objects, we are just going to copy each of these into a bigger span.  We are not going to type serialized objects - we type a request and a response, which type tells not only what thing this is, but what is to be done with it. (A request will say "this is request x of type X", and a response will say "this is a response of type Y to request x of type X", and then the entity replied to has to figure out how to parse it from that information.  It will parse the data with the schema appropriate for that particular x, X, and Y.  A patricia merkle tree, on the other hand, is a tree of blocks, and each block is a pile of changes to the total state, and there will probably be many possible kinds of change to the total state.  So for each such change, the change will say "this is a change of type Z", and then you parse the data with the schema appropriate for a change of type Z)  So maybe for every possible kind of item in the block, and every possible request, and every possible reply, we have a unique schema identifier.  We also have identifiers for groups of schemas, so that every entity announces which groups of schemas it can handle or is willing to handle.

But suppose we want to use the same function to feed stuff into a big stream for output,  OK, we need another type for big streams of output, and yet another type for big streams of input.  And you feed blobs into the stream, and copy them out of the stream.   Suppose we have an object composed of two scalars and a hash, such as a signature.  We would like to be able to serialize it also.  But that object serialized is just a blob.

How about an object derived from span, but with an explicit constructor, so that in order to convert an arbitrary c of type C, into an arbitrary b of type B, you would have to explicitly say B(deserialize(serialize(c))
And B is going to ask the deserialize object for a certain number of bytes, and if it does not have them (because it is actually merely a span), it is going to throw.
But we can template B's constructor to cheerfully accept any class that has the member deserialize(gsl::span<uint8_t), as the class deserialize does.  This member writes into the span, or throws if it has insufficient bytes, and the class with this member can be trivially constructed from a span, or from a "serialize" object, which is derived from a span, and is commonly returned by the method serialize(T)
It also has the method const gsl::span<const uint8_t>deserialize(unsigned int) which returns a span - difference being that if you tell it to deserialize a span, it writes into that span, an if you tell it to return a span, that is for you to read.

We want to be able to freely convert blobs into ristretto objects and back again, and we want the conversion to be automatic in the operator <<, but not automatic elsewhere in the code.  We don't want the compiler automaqically converting every distinct kind of ristretto255 object into every other.

We accomplish this with explicit constructors in class, not templated, which construct from an std::array<uint8_t, correct_size> and an explicit constructor which invoke a templated function std::array<uint8_t, correct_size>& deserialize<n>(T)  reinterpret_cast<std::array<T, n>&> works provided that std::is_pod<std::array<T, n> is true.  The templated function throws if the data is insufficient, it simply points if the data is the correct size, and later, if we have a deserial class with a method deserialize(int), it advances the the pointer in that class.

deserialize<n>(T) returns a reference to the array if T is an array of the correct size, and fails at compile time if the array is the wrong size or type, returns a reference to an array of the correct size if T is a gsl::span of the correct size and type, throws at runtime otherwise, and generates a call to the member deserialize(int) if T supports the method gsl::span<uint8_t>deserialize(int)

serialize<T>(T) returns an array or a reference to an array of specific size, or a gsl::span, depending on the type of T.

serialize<gsl::span<uint8_t> is a noop.

Explicitly invoking serialize(T) throws away our elaborate constructed type information, while T(deserialize<U, sizeof(T)>(U)) adds it back in again, which allows us to deal with everything as pointers to typeless streams of bytes.  But we normally keep type information around, and we can only discard it explicitly, and only add it back explicitly.

Serialize and deserialize are intended to be metacode that drops us back to simple pointers to raw bytes.  Maybe we will have a special explicit class of pointer to raw bytes and only construct typed things from this class of pointer.  class deserial_array<unsigned int n>:std::array<uint8_t, n>{}; and have a bunch of explicit constructors and type converters.  Or perhaps aggregate initialization will suffice.

The problem of the partial hash being finalized twice can be solved by making the right hand side a pure rvalue: operator <<=(multstage_hash&&)

Nah, too clever by half use of C++17 features will throw you into template code hell - better just to introduce a flag that throws if the same data structure is finalized twice.

When we hash a char *, we assume the data is zero delimited, and include the delimiter in the material to be hashed, so that
hash<256> x <<="quick brown "<<"fox";
hash<256> y <<="the quick" <<"brown fox";
will produce two different hashes.