Starting Fulltime Work on the Causevest Protocol!

Causevest XCV Development & Technical Discussion
#8

Just celebrated my birthday this last weekend, so apologies for forgetting last week’s update!

During testing at the beginning of last week, I realized that the mutex structure I had originally created to handle shared state was untenable for a system this size and complexity. I was getting repeated hangs where one thread would lock A and wait for B while another would lock B and wait for A. I decided I was going to have to use channels, handlers and internal message passing for shared state instead of trying to have mutexes for everything. As such, I began channelizing the main parts of the program last week. I was able to channelize the best header chain handler, and the peer handler and test them. I have begun work on have a handler deal with chain fragments, but this has proven to be more difficult more due to the complexity of dealing with chain fragments (and partial chain fragments) then anything to do with channels. Still, I am making progress and plan to have nearly everything channelized by the end of the week.

Let the channelization commence!
image

1 Like
#9

Update time! Over the last week I’ve completed channelization of chain fragments and implemented full reorganizations. It should now be possible to the node to accept any branch, verify it, and reorg to it if it turns out to be a longer chain. Testing is ongoing, and I’ve fixed a few issues already. Next I’m going to start work on the keychain (wallets). I’ve decided that the key storage and interface programs will be separate from the full node, only asking the node about the status of it’s UTXOs and passing keys over to allow staking. This will reduce the attack surface of the keychain significantly by keeping on-key related functions such as networking and validation in a separate program.

In order to have the keychain and node remain separate, I will have to build the node’s RPC server, which is my next task. It should be much easier then the node to node communication, so I expect it to be done before the end of the week. By next week, the wallet should be nearly done, and I can start working on transactions and the XCV economic features.

image

1 Like
#10

Forgot another update again, whoops. Work on keychain and rpc has been ongoing! I’ve been able to build the simple rpc server and client and a simple key passing interface. In order to get to the point where we can verify transactions though, the stack machine needs to be built, which involves getting cryptographic signatures working. Along that path, I have been looking at the code of the btcd project (https://github.com/btcsuite/btcd). I have been able to integrate the btcec, base58, and hdkeychain code into my own test program that can successfully validate a schnorr signature. For here, I will build out the stack machine and some test transactions. Once that is done and I integrate it into the code, and let nodes create their own PoS blocks with their own coins. After that is general transaction creation/propagation. Once that is done, the alpha will be nearly complete!

See you next week!

1 Like
#11

Hope you had a great birthday!

#12

Trying to drag updates back on track. Lets try Monday updates for now.

Last week saw the completion of the rpc server, keychain and the construction of the basic stack machine to verify basic P2PKH transaction scripts. I imported cryptographic and key management code from the btcd bitcoin go project. This allows the creation of hierarchical deterministic keys (see here) and the creation and verification of schnorr signatures using this code. Obviously, for the full protocol release, this code will have to be audited, but this is an important step forward. From here, I will start working on adding basic transactions to blocks and having nodes send transactions to each other through the mempool. This will allow the creation of the first XCV transactions.

1 Like
#13

Things have gone surprisingly well!

#14

Update for the quarter!

The alpha protocol has gotten to the testnet stage. Over the current week I will begin setting up a full testnet - if anyone wants to join, just contact me (or just download the code from the github. Although it is still going to be in a constant state of flux over the next few months).

I’ve attached a video below that shows off the major features of the alpha protocol - this may seem simple, but a lot of work has gone into the foundations for direct transactions, staking, p2p communications, reorgs and rpc access.

Much still has to be done, from vaults, multisig and voting to finality and archive nodes, along with a GUI that lets the average user access all of these features. Still, this is a major milestone for Causevest, and I am proud with how far we’ve come.

The next major updates will be some much needed refactoring and building unit tests. This won’t add any new features directly, but will be important for maintaining code stability and making sure that everything works like it’s supposed too. From there I think I need to rebuild the UTXO set interaction, as the current implementation doesn’t scale nearly well enough for our purposes. Once this is done, I will find the best way to implement voting vaults and multisig, and start planning out FFG finality and archive nodes.

This is the first peak on a long climb of this mountain range, but I’m happy to have shared it with you.

*Video link here, when it is uploaded to youtube.

1 Like
#15

Pride cometh before the fall. There is still much more work to be done ^^ but definetly a great milestone!

Well Done Technical Arch trooper team.

I’ll push for more community involvement in a few weeks.

Kr,

MO

#16

It’s been a bit more then a week since the last update, but things continue moving forward. These last few weeks have been mostly internal updates and refactoring. I have created a connections handler and changed how the node deals with peers in order to remove any residual race conditions with different connections, fixed an issue where the coinbase transction was not removing other txns that double spend the coinbase input from the mempool and re-wrote the signature code to makes sure that transactions and blocks were using the correct sighash for their signatures.

This coming week, I hope to start the alpha testnet in earnest and try and get 5-10 nodes running 24/7. This experience will probably uncover more issues that need to be taken care of. Once those are cleaned up, I’ll look at beginning to add Cause vest specific features such as voting, vaults, and multi signature schemes.

See you for the update next Monday.

1 Like
#17

Another week another update! In making my first attempts to set up the alpha net, I have run into an issue with the way I was handling TCP/IP connections to other nodes. Previously, I was making a new TCP connection every time a node wanted to send any information to another, because I wasn’t quite sure how to have two nodes talk back and forth on a single connection, especially since I need other parts of the program to be able to send commands to the node network at any time.

However, I think I have figured out the correct pattern to keep long lived TCP connections between the nodes. I am in the process of rewriting the p2p code in the alpha node, and that should be done in a day or too. By the end of the week I think the alphanet should be up and running.

After that I’ll start working towards the Causevest specific features we need. Most likely by tacking a pass at the stack machine code and thinking about what we really need there.

I’ll see you then.

1 Like
#18

Time for another update! The alpha testnet is tantalizingly close: I’ve completed and tested the code for long lived TCP connections and node communication has become much more stable and responsive. However, there does seem to be one major issue preventing the alphanet from working: it seems there is some kind of memory issue in the program that causes it too use far too much memory after a short time running. My next order of action is to understand why this is happening and fixing it. Once this is taken care of, I think the alphanet will be ready for full time running.

After the alphanet is up, I will move to adding the Causevest-specific features we need to complete the alpha and begin working on the GUI and beta.

See you next week!

1 Like
#19

Looking forward to progress next week

#20

This weeks update:

My work on the nodes has made them much more stable, I left 3 running over the weekend and by Monday two were still up and responding normally (with 1300 blocks produced!). Next, I will begin automatic transaction generation so that there is load being placed on the network. I’m sure this will uncover new issues that will have to be dealt with, but by the end of the week I hope we can last next weekend with blocks full of transactions.

1 Like
#21

Another weekly update:

This week I make syncing much faster and more efficient by upgrading the p2p network to allow multiple blocks and headers to be requested at once, allow the header handler to send multiple headers in one request, and allow the database to load multiple blocks in a single pass. I also created a database handler that interfaces with the database to replace the old sync.mutex locking mechanism I had before. All in all it syncs much faster now.

I have also created a random txn generator to begin populating the network with transactions. However, after building this, I found a couple of bugs in transaction processing that I have to fix. Once those are taken care of I will be able to begin testing the network under load.

Beyond that, I want to make a proper difficulty adjustment algorithm so I can control the average block time and begin figuring out what the optimal throughput for a node might be. I also need to think about how I will implement the Causevest specific features.

See you next week.

1 Like
#22

Weekly Update:

I’ve been working on debugging transaction generation and chain reorganizations this week. I’ve got the system accepting txns and able to reorg the chain with txns in it without crashing the node or having issues with the UTXO set. There are still some problems I’m running into, but as I move forward I will be making tools that audit the chain to squash any more bugs that I might run into. This next week will be cleaning up anything I missed from last week and starting dynamic difficulty adjustment and other tools necessary to bring the full chain into reality.

Talk to you next week,
Ben

1 Like
#23

Whoops I missed an update last week, but no worries, progress is still steaming along.

These last couple of weeks have led to the creation and implementation of the difficulty adjustment algorithm (DAA). After some testing, I’ve decided on using the ASERT DAA described here (with a similar RTT also shown). I ran the test by simulating 100,000 blocks and taking statistics of the difficulty and block time.

diff_vs_time

Once I had decided on the correct form of the DAA, I implemented it in the node using an integer only scheme to make sure the nodes are consistent and no errors arise due to numerical error of floating point math.

Once I finish final testing, I will be ready to move on and start building other parts of the protocol, such as multisig, vaults, voting and economics.

1 Like
#24

Trying to keep on track with weekly updates for now, lets talk about last week.

This last week I tested the difficulty adjustment algorithm with random transactions. I also built new functions that allow for even faster node synchronization and which should increase node stability. I have been able to run the nodes for a few days without major issues, although it does look like one crashed today.

For the next week, I want to begin on the infrastructure of the Causevest specific features. In particular, I want to look at how vaults, multisig, and voting transactions will be structured and implemented. I origonally was going to copy Bitcoin’s scripting language, but I now thing that is a mistake. It is both too complicated, and unable do important functions (vaults) simply. I will work on building a system that covers the main points without too much overhead.

See you next week!

1 Like
#25

I’ve been a bit slacking on updates, but not on work. I’ve got a new video out that showcases what I’ve accomplished over the last 6 months. I’ve also been working on the vault feature, and now have a complete technical plan on how to proceed with their implementation. That should be done by next week. Once vaults are implemented, I will likely begin work on finality or voting, and after that streamlining the UTXO set interaction.

So without further ado, the video!

1 Like
#26

but not on work >.> … is that sooo

#27

Hey Everyone,

For this update - I’ve gotten vaults implemented and working - you can send coins to a vault address and revert txns that unvault within the specified time period. Unvaulting txns cannot be used for staking or as inputs to any other txn until the unvaulting period is complete, at which point they become irreversible. Each vault address has a hierarchy of keys when created, such that keys can revert txns of an equal or lower level then themselves only. This means that if you have a transaction created by a level 2 key it cannot be reverted by a level 1 key, but a transaction by a level 3 key could revert it.

This feature allows for superior security and user experience. The user can keep a level 1 key (called a spend key) unencrypted for easy use, and if the key falls into the hands of an attacker, they can use a higher level key that was more strongly secured to reverse the transaction and save the coins.

Now, I am moving on to creating voting at the protocol layer - this with allow anyone with coins to vote on a hash that could represent everything form a cause to a delegator. This will allow people to signal with XCV.

Talk to you later