From b091d38ede393fc2cf6f486daf1e7579eedd1328 Mon Sep 17 00:00:00 2001 From: "reaction.la" Date: Tue, 2 Jan 2024 00:23:04 +0000 Subject: [PATCH] Added the link to the investor pitch, so that search engines can find it. --- docs/manifesto/social_networking.md | 5 + docs/setup/core_lightning_in_debian.md | 218 ++++++++++++++++++++++++ docs/setup/set_up_build_environments.md | 28 ++- docs/setup/wireguard.md | 99 +++++++++-- 4 files changed, 329 insertions(+), 21 deletions(-) create mode 100644 docs/setup/core_lightning_in_debian.md diff --git a/docs/manifesto/social_networking.md b/docs/manifesto/social_networking.md index 97c1b2a..3270a52 100644 --- a/docs/manifesto/social_networking.md +++ b/docs/manifesto/social_networking.md @@ -1101,6 +1101,11 @@ Cold Start Problem: If no one is on the network, no one wants to be on it. The value of joining a network depends on the number of other people already using that network. So if there are big competing networks, no one wants to join the new network. + +To solve the cold start problem, you have to start by solving a +very specific problem for a very specific set of people, +and expand from there. + No one is going to want to be the first to start a Sovcorp. And until people do, programmers are not going to get paid except by angel investors expecting that people will start Sovcorps. So to get a foot in the door, we have to cherry pick diff --git a/docs/setup/core_lightning_in_debian.md b/docs/setup/core_lightning_in_debian.md new file mode 100644 index 0000000..b468d90 --- /dev/null +++ b/docs/setup/core_lightning_in_debian.md @@ -0,0 +1,218 @@ +--- +title: + Core lightning in Debian +sidebar: false +... + +Building lightning on Debian turned into a dead end. I just flat could not build core-lightning on Debian, due to python incompatibilities with +the managed python environment. + +Bottom line is that the great evil of python is that building installs +for python projects is a nightmare. It has its equivalent of dll hell. + +So nothing ever works on any system other than the exact system it was +built on. + +The great strength of lua is that every lua program runs its own lua +interpreter with its own lua libraries. + +Instead we have python version hell and pip hell. + +So, docker. + + +# Failed attempt to us Docker lightning + +The docker container as supplied instantly terminates with an error +because it expects to find /root/.lightning/bitcoin which of course +does not exist inside the docker container, and you cannot externally +muck with the files inside a docker container, except by running +commands that the container supports. + +So to run docked lightning, a whole lot of configuration information needs +to be supplied, which is nowhere explained. + +You cannot give it command line parameters, so you have to set them +in environment variables, which do not seem to be documented, +and you have to mount the directories external to your +docker container + +```bash +docker run -it --rm \ + --name clightning \ + -e LIGHTNINGD_NETWORK=bitcoin \ + -e LIGHTNINGD_RPC_PORT=10420 \ + -v $HOME/.lightning:/root/.lightning \ + -v $HOME/.bitcoin:/root/.bitcoin \ +elementsproject/lightningd:v23.11.2-amd64 \ +lightningd --network=bitcoin --log-level=debug +``` +docker run -it --rm \ + --name clightning \ + -e LIGHTNINGD_NETWORK=bitcoin \ + -e LIGHTNINGD_RPC_PORT=10420 \ + -v $HOME/.lightning:/root/.lightning \ + -v $HOME/.bitcoin:/root/.bitcoin \ +elementsproject/lightningd:v23.11.2-amd64 \ +lightningd --network=bitcoin --log-level=debug +``` + +docker run -it --rm \ + --name clightning \ + -e LIGHTNINGD_NETWORK=bitcoin \ + -e LIGHTNINGD_RPC_PORT=10420 \ + -v $HOME/.lightning:/root/.lightning \ + -v $HOME/.bitcoin:/root/.bitcoin \ +elementsproject/lightningd:v22.11.1 \ +lightningd --help + +docker run -it --rm \ + --name clightning \ + -e LIGHTNINGD_NETWORK=bitcoin \ + -e LIGHTNINGD_RPC_PORT=10420 \ +elementsproject/lightningd:v22.11.1 \ +lightningd --help + +The docker container can do one thing only: Run lightning with no +command arguments. + +Which is great if I have everything setup, but in order +to have everything setup, I need documentation and examples +which do not seem readily applicable to lightning inside +docker. I will want to interact with my lighning that is +inside docker with lightning-cli, which lives outside +docker, and because of python install hell, I will want +to use plugins that live inside docker, with which I will +interact using a lightning-cli that lives outside docker. + +But trouble is docker comes with a pile of scripts and plugins, and +the coupling between these and the docker image is going to need +a PhD in dockerology. + +The docker image just dies, because it expects no end of stuff that +just is not there. + + +## Install docker + +```bash +# remove any dead and broken obsolete dockers that might be hanging around +for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done + +# Add Docker's official GPG key: +sudo apt-get update +sudo apt-get install -y pinentry-gnome3 tor parcimonie xloadimage scdaemon pinentry-doc +sudo apt-get install -y ca-certificates curl gnupg +sudo install -m 0755 -d /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +sudo chmod a+r /etc/apt/keyrings/docker.gpg + +# Add the repository to Apt sources: +echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian bookworm stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +cat /etc/apt/sources.list.d/docker.list +sudo apt-get update +sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +sudo usermod -aG docker ${USER} +docker run hello-world +``` + + +## install lightning within docker + +Check available images in [docker hub](https://hub.docker.com/r/elementsproject/lightningd/tags){target="_blank"} + +```bash +docker pull elementsproject/lightningd:v23.11.2-amd64 +docker images +``` + +# Failed attempt to build core lightning in Debian + +## Set up a non Debian python + +Core lightning requires `python` and `pip` Debian does not like outsiders mucking with its fragile and complicated python. + +So you have to set up a virtual environment in which you are free +to do what you like without affecting the rest of Debian: + +The double braces `«»` mean that you do not copy the text inside +the curly braces, which is only there for example. +You have to substitute your own python version, mutas mutandis, +which you learned by typing `python3 -V` + +```bash +sudo apt update +sudo apt -y full-upgrade +sudo apt install -y python3-pip +sudo apt install -y build-essential libssl-dev libffi-dev +sudo apt install -y python3-dev python3-venv +python3 -V +mkdir mypython +cd mypython +python«3.11» -m venv my_env +``` + +You now have your own python environment, which you activate with the command + +```bash +source my_env/bin/activate +``` + +Within this environment, you no longer have to use python3 and pip3, nor should you use them. +You just use python and pip, which means that all those tutorials +on python projects out there on the web now work + +All your python stuff that is not part of Debian managed python should be inside your `mypython` directory, and when you leave it + +```bash +deactivate +``` + +## building core lightining + +Outside your python environment: + +```bash +sudo apt-get install -y \ + autoconf automake build-essential git libtool libsqlite3-dev \ + python3 python3-pip net-tools zlib1g-dev libsodium-dev gettext \ + python3-json5 python3-flask python3-gunicorn \ + cargo rustfmt protobuf-compiler +``` + +Then, inside your mypython directory, activate your python environment and install the python stuff + +```bash +source my_env/bin/activate +pip install --upgrade pip +pip install poetry +pip install flask-cors flask_restx pyln-client flask-socketio \ +gevent gevent-websocket mako +pip install -r plugins/clnrest/requirements.txt +``` +The instructions on the web for ubuntu say `--user` and `pip3`, but on Debian, we accomplish the equivalent through using a virtual environment. + +`--user` is Ubuntu’s way of keeping your custom python separate, +but Debian insists on a more total separation. + +Which means that anything that relies heavily on custom python has +to be inside this environment, so to be on the safe side, we are +going to launch core lighting with a bash script that first goes +into this environment. + +`poetry` is an enormous pile of python tools, which core lightning uses, +and outside this environment, probably will not work. + +Inside your environment: + +```bash +git clone https://github.com/ElementsProject/lightning.git +cd lightning +git tag +# find the most recent release version +# mutas mutandis +git checkout «v23.11.2» +./configure +make +# And then it dies +``` diff --git a/docs/setup/set_up_build_environments.md b/docs/setup/set_up_build_environments.md index b0979a8..0354ffc 100644 --- a/docs/setup/set_up_build_environments.md +++ b/docs/setup/set_up_build_environments.md @@ -30,6 +30,17 @@ And a gpt partition table for a linux system should look something like this To build a cross platform application, you need to build in a cross platform environment. +If you face grief launching an installer for your virtual box device +make sure the virtual network is bridged mode +and get into the live cd command line + +```bash +sudo -i +apt-get update +apt-get install debian-installer-launcher +debian-installer-launcher --text +``` + ## Setting up Ubuntu in VirtualBox Having a whole lot of different versions of different machines, with a @@ -1116,11 +1127,13 @@ All the other files don’t matter. The conf file gets you to the named server. The contents of /var/www/reaction.la are the html files, the important one being index.html. +[install certbot]:https://certbot.eff.org/instructions +"certbot install instructions" {target="_blank"} + To get free, automatically installed and configured, ssl certificates -and configuration +and configuration [install certbot], then ```bash -apt-get -qy install certbot python-certbot-apache certbot register --register-unsafely-without-email --agree-tos certbot --apache ``` @@ -1437,12 +1450,15 @@ your domain name is already publicly pointing to your new host, and your new host is working as desired, without, however, ssl/https that is great. + +To get free, automatically installed and configured, ssl certificates +and configuration [install certbot], then + ```bash # first make sure that your http only website is working as # expected on your domain name and each subdomain. # certbots many mysterious, confusing, and frequently # changing behaviors expect a working environment. -apt-get -qy install certbot python-certbot-nginx certbot register --register-unsafely-without-email --agree-tos certbot --nginx # This also, by default, sets up automatic renewal, @@ -1460,7 +1476,6 @@ server. Meanwhile, for the rest of the world, the domain name continues to map to the old server, until the new server works.) ```bash -apt-get -qy install certbot python-certbot-nginx certbot register --register-unsafely-without-email --agree-tos certbot run -a manual --preferred-challenges dns -i nginx \ -d reaction.la -d blog.reaction.la @@ -1480,10 +1495,7 @@ the big boys can play. But if you are doing this, not on your test server, but on your live server, the easy way, which will also setup automatic renewal and configure your webserver to be https only, is: ```bash -certbot --nginx -d \ -mail.reaction.la,blog.reaction.la,reaction.la,\ -www.reaction.la,www.blog.reaction.la,\ -gitea.reaction.la,git.reaction.la +certbot --nginx ``` If instead you already have a certificate, because you copied over your diff --git a/docs/setup/wireguard.md b/docs/setup/wireguard.md index 9f33e98..5dd23a8 100644 --- a/docs/setup/wireguard.md +++ b/docs/setup/wireguard.md @@ -174,25 +174,25 @@ The curly braces mean that you do not copy the text inside the curly braces, whi ```default [Interface] # public key = CHRh92zutofXTapxNRKxYEpxzwKhp3FfwUfRYzmGHR4= -Address = 10.10.10.1/24, 2405:4200:f001:13f6:7ae3:6c54:61ab:0001/112 +Address = 10.10.10.1/24, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0001/112 ListenPort = 115 PrivateKey = iOdkQoqm5oyFgnCbP5+6wMw99PxDb7pTs509BD6+AE8= [Peer] PublicKey = rtPdw1xDwYjJnDNM2eY2waANgBV4ejhHEwjP/BysljA= -AllowedIPs = 10.10.10.4/32, 2405:4200:f001:13f6:7ae3:6c54:61ab:0009/128 +AllowedIPs = 10.10.10.4/32, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0009/128 [Peer] PublicKey = YvBwFyAeL50uvRq05Lv6MSSEFGlxx+L6VlgZoWA/Ulo= -AllowedIPs = 10.10.10.8/32, 2405:4200:f001:13f6:7ae3:6c54:61ab:0019/128 +AllowedIPs = 10.10.10.8/32, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0019/128 [Peer] PublicKey = XpT68TnsSMFoZ3vy/fVvayvrQjTRQ3mrM7dmyjoWJgw= -AllowedIPs = 10.10.10.12/32, 2405:4200:f001:13f6:7ae3:6c54:61ab:0029/128 +AllowedIPs = 10.10.10.12/32, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0029/128 [Peer] PublicKey = f2m6KRH+GWAcCuPk/TChzD01fAr9fHFpOMbAcyo3t2U= -AllowedIPs = 10.10.10.16/32, 2405:4200:f001:13f6:7ae3:6c54:61ab:0039/128 +AllowedIPs = 10.10.10.16/32, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0039/128 ``` ```default @@ -212,6 +212,16 @@ which ought to be changed". In other words, watch out for those «...» . Or, as those that want to baffle you would say, metasyntactic variables are enclosed in «...» . +In the above example «AAAA:AAAA:AAAA:AAAA» is the 64 bits of the IPv6 +address range of your host and «BBBB:BBBB:BBBB» is a random 48 bit subnet +that you invented for your clients. + +This should be a random forty eight bit number to avoid collisions, +because who knows what other subnets have been reserved. + +This example supports IPv6 as well as IPv4, but getting IPv6 working +is likely to be hard so initially forget about IPv6, and just stick to IPv4 addresses. + Where: @@ -227,6 +237,24 @@ Change the file permission mode so that only root user can read the files. Priv sudo chmod 600 /etc/wireguard/ -R ``` +## IPv6 + +This just does not work on many hosts, depending on arcane +incomprehensible and always different and inaccessible +aspects of their networking setup. But when it works, it works. + +For IP6 to work, without network address translation, you just +give each client a subrange of the host IPv6 address +(which you may not know, or could be changed underneath you) + +When it works, no network address translation needed. +When IPv6 network address translation is needed, +you probably will not be able to get it working anyway, +because if it is needed, +it is needed because the host network is doing something +too clever by half with IPv6, and you don't know what they are doing, +and they probably do not know either. + ## Configure IP Masquerading on the Server We need to set up IP masquerading in the server firewall, so that the server becomes a virtual router for VPN clients. I will use UFW, which is a front end to the iptables firewall. Install UFW on Debian with: @@ -352,21 +380,27 @@ ufw route allow in on wg0 ufw route allow out on wg0 ufw allow in on wg0 ufw allow in from 10.10.10.0/24 -ufw allow in from 2405:4200:f001:13f6:7ae3:6c54:61ab:0001/112 +ufw allow in from «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB:0001»/112 ufw allow «51820»/udp ufw allow to 10.10.10.1/24 -ufw allow to 2405:4200:f001:13f6:7ae3:6c54:61ab:0001/112 +# Danger Will Robertson +ufw allow to «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0001/112 +# This las last line ileaves your clients naked on the IPv6 +# global internet with their own IPv6 addresses +# as if they were in the cloud with no firewall. ``` As always «...» means that this is an example value, and you need to substitute your actual value. "_Mutas mutandis_" means "changing that which should be changed", in other words, watch out for those «...» . Note that the last line is intended to leave your clients naked on the IPv6 global internet with their own IPv6 addresses, as if they were in the cloud -with no firewall. This is often desirable for linux systems, but dangerous +with no firewall.This is often desirable for linux systems, but dangerous for windows, android, and mac systems which always have loads of undocumented closed source mystery meat processes running that do who knows what. +It would be safer to only allow in specific ports. + You could open only part of the IPv6 subnet to incoming, and put windows, mac, and android clients in the part that is not open. @@ -484,7 +518,6 @@ And add allow recursion for your subnets. After which it should look something like this: - ```terminal_image :~# cat /etc/bind/named.conf.options | tail -n 9 acl bogusnets { @@ -497,7 +530,7 @@ acl my_net { ::1; 116.251.216.176; 10.10.10.0/24; - 2405:4200:f001:13f6::/64; + «AAAA:AAAA:AAAA:AAAA»::/64; }; options { @@ -605,13 +638,13 @@ for example, and has to be customized. Mutas mutandis. Metasyntactic variables ```default [Interface] -Address = 10.10.10.2/24 +Address =10.10.10.4/32, «AAAA:AAAA:AAAA:AAAA»:«BBBB:BBBB:BBBB»:0009/128 DNS = 10.10.10.1 PrivateKey = «cOFA+x5UvHF+a3xJ6enLatG+DoE3I5PhMgKrMKkUyXI=» [Peer] PublicKey = «kQvxOJI5Km4S1c7WXu2UZFpB8mHGuf3Gz8mmgTIF2U0=» -AllowedIPs = 0.0.0.0/0 +AllowedIPs = 0.0.0.0/0, ::/0 Endpoint = «123.45.67.89:51820» PersistentKeepalive = 25 ``` @@ -622,7 +655,7 @@ Where: - `DNS`: specify 10.10.10.1 (the VPN server) as the DNS server. It will be configured via the `resolvconf` command. You can also specify multiple DNS servers for redundancy like this: `DNS = 10.10.10.1 8.8.8.8` - `PrivateKey`: The client’s private key, which can be found in the `/etc/wireguard/private.key` file on the client computer. - `PublicKey`: The server’s public key, which can be found in the `/etc/wireguard/server_public.key` file on the server. -- `AllowedIPs`: 0.0.0.0/0 represents the whole Internet, which means all traffic to the Internet should be routed via the VPN. +- `AllowedIPs`: 0.0.0.0/0 represents the whole IPv4 Internet, which means all IPv4 traffic to the Internet should be routed via the VPN. ::/0 represents the whole IPv6 Internet. If you specify one but not the other, and your client has both IPv4 and IPv6 capability, only half your traffic will go through the vpn. If your client has both capabilities, but your vpn does not, this is bad, but things still work. - `Endpoint`: The public IP address and port number of VPN server. Replace 123.45.67.89 with your server’s real public IP address and the port number with your server’s real port number. - `PersistentKeepalive`: Send an authenticated empty packet to the peer every 25 seconds to keep the connection alive. If PersistentKeepalive isn’t enabled, the VPN server might not be able to ping the VPN client. @@ -636,6 +669,18 @@ chmod 600 /etc/wireguard/ -R Start WireGuard. +```bash +wg-quick up /etc/wireguard/wg-client0.conf +``` + +To stop it, run + +```bash +wg-quick down /etc/wireguard/wg-client0.conf +``` + +You can also use systemd service to start WireGuard. + ```bash systemctl start wg-quick@wg-client0.service ``` @@ -652,6 +697,34 @@ Check its status: systemctl status wg-quick@wg-client0.service ``` +The status should look something like this: + +```terminal_image +# systemctl status wg-quick@wg-client0.service +● wg-quick@wg-client0.service - WireGuard via wg-quick(8) for wg/client0 + Loaded: loaded (/lib/systemd/system/wg-quick@.service; disabled; vendor preset: enabled) + Active: active (exited) since Wed 2023-12-27 03:48:41 +08; 1min 11s ago + Docs: man:wg-quick(8) + man:wg(8) + https://www.wireguard.com/ + https://www.wireguard.com/quickstart/ + https://git.zx2c4.com/wireguard-tools/about/src/man/wg-quick.8 + https://git.zx2c4.com/wireguard-tools/about/src/man/wg.8 + Process: 2913 ExecStart=/usr/bin/wg-quick up wg-client0 (code=exited, status=0/SUCCESS) + Main PID: 2913 (code=exited, status=0/SUCCESS) + CPU: 109ms + +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -6 route add ::/0 dev wg-client0 table 51820 +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -6 rule add not fwmark 51820 table 51820 +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -6 rule add table main suppress_prefixlength 0 +Dec 27 03:48:41 backups wg-quick[2913]: [#] nft -f /dev/fd/63 +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -4 route add 0.0.0.0/0 dev wg-client0 table 51820 +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -4 rule add not fwmark 51820 table 51820 +Dec 27 03:48:41 backups wg-quick[2913]: [#] ip -4 rule add table main suppress_prefixlength 0 +Dec 27 03:48:41 backups wg-quick[2913]: [#] sysctl -q net.ipv4.conf.all.src_valid_mark=1 +Dec 27 03:48:41 backups wg-quick[2913]: [#] nft -f /dev/fd/63 +``` + Now go to this website: `http://icanhazip.com/` to check your public IP address. If everything went well, it should display your VPN server’s public IP address instead of your client computer’s public IP address. You can also run the following command to get the current public IP address.