Today is the first day I focus full time on building the Causevest Protocol Alpha. Laid out in the whitepaper, the Causevest protocol will be the bedrock of the Causevest Network The alpha will be a command line version of the basic protocol written in Go. It will be quite the adventure. Once I get into the swing of things, I expect to be able to begin streaming my progress. Regardless I will be recording with the intention to release the recordings at some point. Aside from the ethnomethodology[1], these recordings will be an excellent primer for anyone who wants to know how the protocol works and why I built it the way I did (and any mistakes I made along the way )
I’ll post updates in this thread as I move forward.
Ethnomethodology is the empirical study of everyday practical activity - in other words, how people actually do things day to day, instead of how things happen theoretically. ↩︎
First week is complete, and quite a lot of progress has been made. From a single go file that made a single sha3 hash to the ability to generate blocks, save them in leveldb, create dummy transactions and update the UTXO set.
Next week I should be able to complete the single node alpha: having a node that can generate blocks via PoS, update its UTXO set, and restart from storage. From there I will start working on the rpc, networking and validation portions for the full alpha protocol. With luck, by the end of Q1 there will be something you can download and run!
The second week has seen more progress. From not even implementing a UTXO set to beginning inter-node communication. Networking is going to be the name of the game for the next couple of weeks at least. Dealing with TCP/IP in a multi-threaded environment where all the connections are assumed to be hostile is not a simple system. Still, I am making good progress and am still on track for getting the alpha out by the end of Q1 2020. The aim is to having something a technical user could download and sync on testnet.
The third week is complete and networking has had major strides made on it. I now have a good map of how the P2P network will be constructed and how nodes will communicate with each other. Major progress has been made and a lot of code written since last week on networking and verification. I am hoping to give nodes the capacity to sync over the network by the end of this week. From there I will start on wallet work and being able to create transactions that only one person can spend. Along with building the RPC so you can use a command line app to interface with the node.
On the large scale, I’m not sure we will have the complete xcv-alpha done by the end of Q1 - there is more complexity here then I anticipated, but don’t worry, we will have something that you’ll be able to download and play with Soon.
Been so busy I almost forgot to post a weekly update! The P2P network is working so far and I have have one node stake while another receives the blocks and syncs with it. I am now working on allowing chain reorganizations so multiple nodes can stake and sync at the same time. Once this is complete, I’ll be able to start the alpha testnet and have nodes running and staking/syncing all the time. From there I will begin work on the wallet/keychain and allowing nodes to submit and share transactions and add them to blocks.
By the end of the month, I hope to have a swarm of nodes running and chatting with each other, and once this infrastructure is out of the way, I’ll be able to focus on Causevest specific features, such as voting.
Another week, another update! Progress is moving, I’ve written the code for handling chain reorganizations and adding blocks on the main chain. I’m testing it now. During testing I’ve run into some problems with the p2p code and needs to add some timeouts to keep it from hanging.
The networking and block handling infrastructure took more time then I would have hoped, but thinks are still on track for getting the alpha testnet up and running. I can’t wait till we have nodes constantly checking and talking with each other!
Next update! XD Great work so far really looking forward to seeing the next one. Hopefully we can get a tiny snippet of the video work once the Beta starts.
Just celebrated my birthday this last weekend, so apologies for forgetting last week’s update!
During testing at the beginning of last week, I realized that the mutex structure I had originally created to handle shared state was untenable for a system this size and complexity. I was getting repeated hangs where one thread would lock A and wait for B while another would lock B and wait for A. I decided I was going to have to use channels, handlers and internal message passing for shared state instead of trying to have mutexes for everything. As such, I began channelizing the main parts of the program last week. I was able to channelize the best header chain handler, and the peer handler and test them. I have begun work on have a handler deal with chain fragments, but this has proven to be more difficult more due to the complexity of dealing with chain fragments (and partial chain fragments) then anything to do with channels. Still, I am making progress and plan to have nearly everything channelized by the end of the week.
Update time! Over the last week I’ve completed channelization of chain fragments and implemented full reorganizations. It should now be possible to the node to accept any branch, verify it, and reorg to it if it turns out to be a longer chain. Testing is ongoing, and I’ve fixed a few issues already. Next I’m going to start work on the keychain (wallets). I’ve decided that the key storage and interface programs will be separate from the full node, only asking the node about the status of it’s UTXOs and passing keys over to allow staking. This will reduce the attack surface of the keychain significantly by keeping on-key related functions such as networking and validation in a separate program.
In order to have the keychain and node remain separate, I will have to build the node’s RPC server, which is my next task. It should be much easier then the node to node communication, so I expect it to be done before the end of the week. By next week, the wallet should be nearly done, and I can start working on transactions and the XCV economic features.
Forgot another update again, whoops. Work on keychain and rpc has been ongoing! I’ve been able to build the simple rpc server and client and a simple key passing interface. In order to get to the point where we can verify transactions though, the stack machine needs to be built, which involves getting cryptographic signatures working. Along that path, I have been looking at the code of the btcd project (https://github.com/btcsuite/btcd). I have been able to integrate the btcec, base58, and hdkeychain code into my own test program that can successfully validate a schnorr signature. For here, I will build out the stack machine and some test transactions. Once that is done and I integrate it into the code, and let nodes create their own PoS blocks with their own coins. After that is general transaction creation/propagation. Once that is done, the alpha will be nearly complete!
Trying to drag updates back on track. Lets try Monday updates for now.
Last week saw the completion of the rpc server, keychain and the construction of the basic stack machine to verify basic P2PKH transaction scripts. I imported cryptographic and key management code from the btcd bitcoin go project. This allows the creation of hierarchical deterministic keys (see here) and the creation and verification of schnorr signatures using this code. Obviously, for the full protocol release, this code will have to be audited, but this is an important step forward. From here, I will start working on adding basic transactions to blocks and having nodes send transactions to each other through the mempool. This will allow the creation of the first XCV transactions.
The alpha protocol has gotten to the testnet stage. Over the current week I will begin setting up a full testnet - if anyone wants to join, just contact me (or just download the code from the github. Although it is still going to be in a constant state of flux over the next few months).
I’ve attached a video below that shows off the major features of the alpha protocol - this may seem simple, but a lot of work has gone into the foundations for direct transactions, staking, p2p communications, reorgs and rpc access.
Much still has to be done, from vaults, multisig and voting to finality and archive nodes, along with a GUI that lets the average user access all of these features. Still, this is a major milestone for Causevest, and I am proud with how far we’ve come.
The next major updates will be some much needed refactoring and building unit tests. This won’t add any new features directly, but will be important for maintaining code stability and making sure that everything works like it’s supposed too. From there I think I need to rebuild the UTXO set interaction, as the current implementation doesn’t scale nearly well enough for our purposes. Once this is done, I will find the best way to implement voting vaults and multisig, and start planning out FFG finality and archive nodes.
This is the first peak on a long climb of this mountain range, but I’m happy to have shared it with you.
It’s been a bit more then a week since the last update, but things continue moving forward. These last few weeks have been mostly internal updates and refactoring. I have created a connections handler and changed how the node deals with peers in order to remove any residual race conditions with different connections, fixed an issue where the coinbase transction was not removing other txns that double spend the coinbase input from the mempool and re-wrote the signature code to makes sure that transactions and blocks were using the correct sighash for their signatures.
This coming week, I hope to start the alpha testnet in earnest and try and get 5-10 nodes running 24/7. This experience will probably uncover more issues that need to be taken care of. Once those are cleaned up, I’ll look at beginning to add Cause vest specific features such as voting, vaults, and multi signature schemes.
Another week another update! In making my first attempts to set up the alpha net, I have run into an issue with the way I was handling TCP/IP connections to other nodes. Previously, I was making a new TCP connection every time a node wanted to send any information to another, because I wasn’t quite sure how to have two nodes talk back and forth on a single connection, especially since I need other parts of the program to be able to send commands to the node network at any time.
However, I think I have figured out the correct pattern to keep long lived TCP connections between the nodes. I am in the process of rewriting the p2p code in the alpha node, and that should be done in a day or too. By the end of the week I think the alphanet should be up and running.
After that I’ll start working towards the Causevest specific features we need. Most likely by tacking a pass at the stack machine code and thinking about what we really need there.
Time for another update! The alpha testnet is tantalizingly close: I’ve completed and tested the code for long lived TCP connections and node communication has become much more stable and responsive. However, there does seem to be one major issue preventing the alphanet from working: it seems there is some kind of memory issue in the program that causes it too use far too much memory after a short time running. My next order of action is to understand why this is happening and fixing it. Once this is taken care of, I think the alphanet will be ready for full time running.
After the alphanet is up, I will move to adding the Causevest-specific features we need to complete the alpha and begin working on the GUI and beta.
My work on the nodes has made them much more stable, I left 3 running over the weekend and by Monday two were still up and responding normally (with 1300 blocks produced!). Next, I will begin automatic transaction generation so that there is load being placed on the network. I’m sure this will uncover new issues that will have to be dealt with, but by the end of the week I hope we can last next weekend with blocks full of transactions.
)
