Technical: A Brief History of Payment Channels: from Satoshi to Lightning Network
Who cares about political tweets from some random country's president when payment channels are a much more interesting and are actually capable of carrying value? So let's have a short history of various payment channel techs!
Generation 0: Satoshi's Broken nSequence Channels
Because Satoshi's Vision included payment channels, except his implementation sucked so hard we had to go fix it and added RBF as a by-product. Originally, the plan for nSequence was that mempools would replace any transaction spending certain inputs with another transaction spending the same inputs, but only if the nSequence field of the replacement was larger. Since 0xFFFFFFFF was the highest value that nSequence could get, this would mark a transaction as "final" and not replaceable on the mempool anymore. In fact, this "nSequence channel" I will describe is the reason why we have this weird rule about nLockTime and nSequence. nLockTime actually only works if nSequence is not 0xFFFFFFFF i.e. final. If nSequence is 0xFFFFFFFF then nLockTime is ignored, because this if the "final" version of the transaction. So what you'd do would be something like this:
You go to a bar and promise the bartender to pay by the time the bar closes. Because this is the Bitcoin universe, time is measured in blockheight, so the closing time of the bar is indicated as some future blockheight.
For your first drink, you'd make a transaction paying to the bartender for that drink, paying from some coins you have. The transaction has an nLockTime equal to the closing time of the bar, and a starting nSequence of 0. You hand over the transaction and the bartender hands you your drink.
For your succeeding drink, you'd remake the same transaction, adding the payment for that drink to the transaction output that goes to the bartender (so that output keeps getting larger, by the amount of payment), and having an nSequence that is one higher than the previous one.
Eventually you have to stop drinking. It comes down to one of two possibilities:
You drink until the bar closes. Since it is now the nLockTime indicated in the transaction, the bartender is able to broadcast the latest transaction and tells the bouncers to kick you out of the bar.
You wisely consider the state of your liver. So you re-sign the last transaction with a "final" nSequence of 0xFFFFFFFF i.e. the maximum possible value it can have. This allows the bartender to get his or her funds immediately (nLockTime is ignored if nSequence is 0xFFFFFFFF), so he or she tells the bouncers to let you out of the bar.
Now that of course is a payment channel. Individual payments (purchases of alcohol, so I guess buying coffee is not in scope for payment channels). Closing is done by creating a "final" transaction that is the sum of the individual payments. Sure there's no routing and channels are unidirectional and channels have a maximum lifetime but give Satoshi a break, he was also busy inventing Bitcoin at the time. Now if you noticed I called this kind of payment channel "broken". This is because the mempool rules are not consensus rules, and cannot be validated (nothing about the mempool can be validated onchain: I sigh every time somebody proposes "let's make block size dependent on mempool size", mempool state cannot be validated by onchain data). Fullnodes can't see all of the transactions you signed, and then validate that the final one with the maximum nSequence is the one that actually is used onchain. So you can do the below:
Become friends with Jihan Wu, because he owns >51% of the mining hashrate (he totally reorged Bitcoin to reverse the Binance hack right?).
Slip Jihan Wu some of the more interesting drinks you're ordering as an incentive to cooperate with you. So say you end up ordering 100 drinks, you split it with Jihan Wu and give him 50 of the drinks.
When the bar closes, Jihan Wu quickly calls his mining rig and tells them to mine the version of your transaction with nSequence 0. You know, that first one where you pay for only one drink.
Because fullnodes cannot validate nSequence, they'll accept even the nSequence=0 version and confirm it, immutably adding you paying for a single alcoholic drink to the blockchain.
The bartender, pissed at being cheated, takes out a shotgun from under the bar and shoots at you and Jihan Wu.
Jihan Wu uses his mystical chi powers (actually the combined exhaust from all of his mining rigs) to slow down the shotgun pellets, making them hit you as softly as petals drifting in the wind.
The bartender mutters some words, clothes ripping apart as he or she (hard to believe it could be a she but hey) turns into a bear, ready to maul you for cheating him or her of the payment for all the 100 drinks you ordered from him or her.
Steely-eyed, you stand in front of the bartender-turned-bear, daring him to touch you. You've watched Revenant, you know Leonardo di Caprio could survive a bear mauling, and if some posh actor can survive that, you know you can too. You make a pose. "Drunken troll logic attack!"
I think I got sidetracked here.
Bears are bad news.
You can't reasonably invoke "Satoshi's Vision" and simultaneously reject the Lightning Network because it's not onchain. Satoshi's Vision included a half-assed implementation of payment channels with nSequence, where the onchain transaction represented multiple logical payments, exactly what modern offchain techniques do (except modern offchain techniques actually work). nSequence (the field, but not its modern meaning) has been in Bitcoin since BitCoin For Windows Alpha 0.1.0. And its original intent was payment channels. You can't get nearer to Satoshi's Vision than being a field that Satoshi personally added to transactions on the very first public release of the BitCoin software, like srsly.
Miners can totally bypass mempool rules. In fact, the reason why nSequence has been repurposed to indicate "optional" replace-by-fee is because miners are already incentivized by the nSequence system to always follow replace-by-fee anyway. I mean, what do you think those drinks you passed to Jihan Wu are, other than the fee you pay him to mine a specific version of your transaction?
Satoshi made mistakes. The original design for nSequence is one of them. Today, we no longer use nSequence in this way. So diverging from Satoshi's original design is part and parcel of Bitcoin development, because over time, we learn new lessons that Satoshi never knew about. Satoshi was an important landmark in this technology. He will not be the last, or most important, that we will remember in the future: he will only be the first.
Incentive-compatible time-limited unidirectional channel; or, Satoshi's Vision, Fixed (if transaction malleability hadn't been a problem, that is). Now, we know the bartender will turn into a bear and maul you if you try to cheat the payment channel, and now that we've revealed you're good friends with Jihan Wu, the bartender will no longer accept a payment channel scheme that lets one you cooperate with a miner to cheat the bartender. Fortunately, Jeremy Spilman proposed a better way that would not let you cheat the bartender. First, you and the bartender perform this ritual:
You get some funds and create a transaction that pays to a 2-of-2 multisig between you and the bartender. You don't broadcast this yet: you just sign it and get its txid.
You create another transaction that spends the above transaction. This transaction (the "backoff") has an nLockTime equal to the closing time of the bar, plus one block. You sign it and give this backoff transaction (but not the above transaction) to the bartender.
The bartender signs the backoff and gives it back to you. It is now valid since it's spending a 2-of-2 of you and the bartender, and both of you have signed the backoff transaction.
Now you broadcast the first transaction onchain. You and the bartender wait for it to be deeply confirmed, then you can start ordering.
The above is probably vaguely familiar to LN users. It's the funding process of payment channels! The first transaction, the one that pays to a 2-of-2 multisig, is the funding transaction that backs the payment channel funds. So now you start ordering in this way:
For your first drink, you create a transaction spending the funding transaction output and sending the price of the drink to the bartender, with the rest returning to you.
You sign the transaction and pass it to the bartender, who serves your first drink.
For your succeeding drinks, you recreate the same transaction, adding the price of the new drink to the sum that goes to the bartender and reducing the money returned to you. You sign the transaction and give it to the bartender, who serves you your next drink.
At the end:
If the bar closing time is reached, the bartender signs the latest transaction, completing the needed 2-of-2 signatures and broadcasting this to the Bitcoin network. Since the backoff transaction is the closing time + 1, it can't get used at closing time.
If you decide you want to leave early because your liver is crying, you just tell the bartender to go ahead and close the channel (which the bartender can do at any time by just signing and broadcasting the latest transaction: the bartender won't do that because he or she is hoping you'll stay and drink more).
If you ended up just hanging around the bar and never ordering, then at closing time + 1 you broadcast the backoff transaction and get your funds back in full.
Now, even if you pass 50 drinks to Jihan Wu, you can't give him the first transaction (the one which pays for only one drink) and ask him to mine it: it's spending a 2-of-2 and the copy you have only contains your own signature. You need the bartender's signature to make it valid, but he or she sure as hell isn't going to cooperate in something that would lose him or her money, so a signature from the bartender validating old state where he or she gets paid less isn't going to happen. So, problem solved, right? Right? Okay, let's try it. So you get your funds, put them in a funding tx, get the backoff tx, confirm the funding tx... Once the funding transaction confirms deeply, the bartender laughs uproariously. He or she summons the bouncers, who surround you menacingly. "I'm refusing service to you," the bartender says. "Fine," you say. "I was leaving anyway;" You smirk. "I'll get back my money with the backoff transaction, and posting about your poor service on reddit so you get negative karma, so there!" "Not so fast," the bartender says. His or her voice chills your bones. It looks like your exploitation of the Satoshi nSequence payment channel is still fresh in his or her mind. "Look at the txid of the funding transaction that got confirmed." "What about it?" you ask nonchalantly, as you flip open your desktop computer and open a reputable blockchain explorer. What you see shocks you. "What the --- the txid is different! You--- you changed my signature?? But how? I put the only copy of my private key in a sealed envelope in a cast-iron box inside a safe buried in the Gobi desert protected by a clan of nomads who have dedicated their lives and their childrens' lives to keeping my private key safe in perpetuity!" "Didn't you know?" the bartender asks. "The components of the signature are just very large numbers. The sign of one of the signature components can be changed, from positive to negative, or negative to positive, and the signature will remain valid. Anyone can do that, even if they don't know the private key. But because Bitcoin includes the signatures in the transaction when it's generating the txid, this little change also changes the txid." He or she chuckles. "They say they'll fix it by separating the signatures from the transaction body. They're saying that these kinds of signature malleability won't affect transaction ids anymore after they do this, but I bet I can get my good friend Jihan Wu to delay this 'SepSig' plan for a good while yet. Friendly guy, this Jihan Wu, it turns out all I had to do was slip him 51 drinks and he was willing to mine a tx with the signature signs flipped." His or her grin widens. "I'm afraid your backoff transaction won't work anymore, since it spends a txid that is not existent and will never be confirmed. So here's the deal. You pay me 99% of the funds in the funding transaction, in exchange for me signing the transaction that spends with the txid that you see onchain. Refuse, and you lose 100% of the funds and every other HODLer, including me, benefits from the reduction in coin supply. Accept, and you get to keep 1%. I lose nothing if you refuse, so I won't care if you do, but consider the difference of getting zilch vs. getting 1% of your funds." His or her eyes glow. "GENUFLECT RIGHT NOW." Lesson learned?
Payback's a bitch.
Transaction malleability is a bitchier bitch. It's why we needed to fix the bug in SegWit. Sure, MtGox claimed they were attacked this way because someone kept messing with their transaction signatures and thus they lost track of where their funds went, but really, the bigger impetus for fixing transaction malleability was to support payment channels.
Yes, including the signatures in the hash that ultimately defines the txid was a mistake. Satoshi made a lot of those. So we're just reiterating the lesson "Satoshi was not an infinite being of infinite wisdom" here. Satoshi just gets a pass because of how awesome Bitcoin is.
CLTV-protected Spilman Channels
Using CLTV for the backoff branch. This variation is simply Spilman channels, but with the backoff transaction replaced with a backoff branch in the SCRIPT you pay to. It only became possible after OP_CHECKLOCKTIMEVERIFY (CLTV) was enabled in 2015. Now as we saw in the Spilman Channels discussion, transaction malleability means that any pre-signed offchain transaction can easily be invalidated by flipping the sign of the signature of the funding transaction while the funding transaction is not yet confirmed. This can be avoided by simply putting any special requirements into an explicit branch of the Bitcoin SCRIPT. Now, the backoff branch is supposed to create a maximum lifetime for the payment channel, and prior to the introduction of OP_CHECKLOCKTIMEVERIFY this could only be done by having a pre-signed nLockTime transaction. With CLTV, however, we can now make the branches explicit in the SCRIPT that the funding transaction pays to. Instead of paying to a 2-of-2 in order to set up the funding transaction, you pay to a SCRIPT which is basically "2-of-2, OR this singlesig after a specified lock time". With this, there is no backoff transaction that is pre-signed and which refers to a specific txid. Instead, you can create the backoff transaction later, using whatever txid the funding transaction ends up being confirmed under. Since the funding transaction is immutable once confirmed, it is no longer possible to change the txid afterwards.
Todd Micropayment Networks
The old hub-spoke model (that isn't how LN today actually works). One of the more direct predecessors of the Lightning Network was the hub-spoke model discussed by Peter Todd. In this model, instead of payers directly having channels to payees, payers and payees connect to a central hub server. This allows any payer to pay any payee, using the same channel for every payee on the hub. Similarly, this allows any payee to receive from any payer, using the same channel. Remember from the above Spilman example? When you open a channel to the bartender, you have to wait around for the funding tx to confirm. This will take an hour at best. Now consider that you have to make channels for everyone you want to pay to. That's not very scalable. So the Todd hub-spoke model has a central "clearing house" that transport money from payers to payees. The "Moonbeam" project takes this model. Of course, this reveals to the hub who the payer and payee are, and thus the hub can potentially censor transactions. Generally, though, it was considered that a hub would more efficiently censor by just not maintaining a channel with the payer or payee that it wants to censor (since the money it owned in the channel would just be locked uselessly if the hub won't process payments to/from the censored user). In any case, the ability of the central hub to monitor payments means that it can surveill the payer and payee, and then sell this private transactional data to third parties. This loss of privacy would be intolerable today. Peter Todd also proposed that there might be multiple hubs that could transport funds to each other on behalf of their users, providing somewhat better privacy. Another point of note is that at the time such networks were proposed, only unidirectional (Spilman) channels were available. Thus, while one could be a payer, or payee, you would have to use separate channels for your income versus for your spending. Worse, if you wanted to transfer money from your income channel to your spending channel, you had to close both and reshuffle the money between them, both onchain activities.
Poon-Dryja Lightning Network
Bidirectional two-participant channels. The Poon-Dryja channel mechanism has two important properties:
No time limit.
Both the original Satoshi and the two Spilman variants are unidirectional: there is a payer and a payee, and if the payee wants to do a refund, or wants to pay for a different service or product the payer is providing, then they can't use the same unidirectional channel. The Poon-Dryjam mechanism allows channels, however, to be bidirectional instead: you are not a payer or a payee on the channel, you can receive or send at any time as long as both you and the channel counterparty are online. Further, unlike either of the Spilman variants, there is no time limit for the lifetime of a channel. Instead, you can keep the channel open for as long as you want. Both properties, together, form a very powerful scaling property that I believe most people have not appreciated. With unidirectional channels, as mentioned before, if you both earn and spend over the same network of payment channels, you would have separate channels for earning and spending. You would then need to perform onchain operations to "reverse" the directions of your channels periodically. Secondly, since Spilman channels have a fixed lifetime, even if you never used either channel, you would have to periodically "refresh" it by closing it and reopening. With bidirectional, indefinite-lifetime channels, you may instead open some channels when you first begin managing your own money, then close them only after your lawyers have executed your last will and testament on how the money in your channels get divided up to your heirs: that's just two onchain transactions in your entire lifetime. That is the potentially very powerful scaling property that bidirectional, indefinite-lifetime channels allow. I won't discuss the transaction structure needed for Poon-Dryja bidirectional channels --- it's complicated and you can easily get explanations with cute graphics elsewhere. There is a weakness of Poon-Dryja that people tend to gloss over (because it was fixed very well by RustyReddit):
You have to store all the revocation keys of a channel. This implies you are storing 1 revocation key for every channel update, so if you perform millions of updates over your entire lifetime, you'd be storing several megabytes of keys, for only a single channel. RustyReddit fixed this by requiring that the revocation keys be generated from a "Seed" revocation key, and every key is just the application of SHA256 on that key, repeatedly. For example, suppose I tell you that my first revocation key is SHA256(SHA256(seed)). You can store that in O(1) space. Then for the next revocation, I tell you SHA256(seed). From SHA256(key), you yourself can compute SHA256(SHA256(seed)) (i.e. the previous revocation key). So you can remember just the most recent revocation key, and from there you'd be able to compute every previous revocation key. When you start a channel, you perform SHA256 on your seed for several million times, then use the result as the first revocation key, removing one layer of SHA256 for every revocation key you need to generate. RustyReddit not only came up with this, but also suggested an efficient O(log n) storage structure, the shachain, so that you can quickly look up any revocation key in the past in case of a breach. People no longer really talk about this O(n) revocation storage problem anymore because it was solved very very well by this mechanism.
Another thing I want to emphasize is that while the Lightning Network paper and many of the earlier presentations developed from the old Peter Todd hub-and-spoke model, the modern Lightning Network takes the logical conclusion of removing a strict separation between "hubs" and "spokes". Any node on the Lightning Network can very well work as a hub for any other node. Thus, while you might operate as "mostly a payer", "mostly a forwarding node", "mostly a payee", you still end up being at least partially a forwarding node ("hub") on the network, at least part of the time. This greatly reduces the problems of privacy inherent in having only a few hub nodes: forwarding nodes cannot get significantly useful data from the payments passing through them, because the distance between the payer and the payee can be so large that it would be likely that the ultimate payer and the ultimate payee could be anyone on the Lightning Network. Lessons learned?
We can decentralize if we try hard enough!
"Hubs bad" can be made "hubs good" if everybody is a hub.
Smart people can solve problems. It's kinda why they're smart.
After LN, there's also the Decker-Wattenhofer Duplex Micropayment Channels (DMC). This post is long enough as-is, LOL. But for now, it uses a novel "decrementing nSequence channel", using the new relative-timelock semantics of nSequence (not the broken one originally by Satoshi). It actually uses multiple such "decrementing nSequence" constructs, terminating in a pair of Spilman channels, one in both directions (thus "duplex"). Maybe I'll discuss it some other time. The realization that channel constructions could actually hold more channel constructions inside them (the way the Decker-Wattenhofer puts a pair of Spilman channels inside a series of "decrementing nSequence channels") lead to the further thought behind Burchert-Decker-Wattenhofer channel factories. Basically, you could host multiple two-participant channel constructs inside a larger multiparticipant "channel" construct (i.e. host multiple channels inside a factory). Further, we have the Decker-Russell-Osuntokun or "eltoo" construction. I'd argue that this is "nSequence done right". I'll write more about this later, because this post is long enough. Lessons learned?
Bitcoin offchain scaling is more powerful than you ever thought.
Technical: Upcoming Improvements to Lightning Network
Price? Who gives a shit about price when Lightning Network development is a lot more interesting????? One thing about LN is that because there's no need for consensus before implementing things, figuring out the status of things is quite a bit more difficult than on Bitcoin. In one hand it lets larger groups of people work on improving LN faster without having to coordinate so much. On the other hand it leads to some fragmentation of the LN space, with compatibility problems occasionally coming up. The below is just a smattering sample of LN stuff I personally find interesting. There's a bunch of other stuff, like splice and dual-funding, that I won't cover --- post is long enough as-is, and besides, some of the below aren't as well-known. Anyway.....
Yeah the exciting new Lightning Network channel update protocol!
Solves "toxic waste" problem. In the current Poon-Dryja update protocol, old state ("waste") is dangerous ("toxic") because if your old state is acquired by your most hated enemy, they can use that old state to publish a stale unilateral close transaction, which your counterparty must treat as a theft attempt and punish you, causing you to lose funds. With Decker-Russell-Osuntokun old state is not revoked, but is instead gainsaid by later state: instead of actively punishing old state, it simply replaces the old state with a later state.
Allows multiple participants in the update protocol. This can be used as the update protocol for a channel factory with 3 or more participants, for example (channels are not practical for multiple participants since the loss of any one participants makes the channel completely unuseable; it's more sensible to have a multiple-participant factory that splits up into 2-participant channels). Poon-Dryja only supports two participants. Another update protocol, Decker-Wattenhofer, also supports multiple participants, but requires much larger locktimes in case of a unilateral close (measurable in weeks, whereas Poon-Dryja and Decker-Russell-Osuntokun can be measured in hours or days).
It uses nLockTime in a very clever way.
No, it does not solve the "watchtower needed" problem. Decker-Russell-Osuntokun still requires watchtowers if you're planning to be offline for a long time.
What might be confused is that it was initially thought that watchtowers under Decker-Russell-Osuntokun could be made more efficient by having the channel participant update a single "slot" in the watchtower, rather than having to consume one "slot" per update in Poon-Dryja. However, the existence of the "poisoned blob" attack by ZmnSCPxj means that having a replaceable "slot" is risky if the other participant of the channel can spoof you. And the safest way to prevent spoofing somebody is to identify that somebody --- but now that means the watchtower can surveill the activities of somebody it has identified, losing privacy.
Requires base layer change --- SIGHASH_NOINPUT / SIGHASH_ANYPREVOUT. This is still being worked out and may potentially not reach Bitcoin anytime soon.
Determining costs of routes is somewhat harder, and may complicate routefinding algorithms. In particular: every channel today has a "CLTV Delta", a number of blocks by which the total maximum delay of the payment is increased. This maximum delay is the maximum amount of time by which an outgoing payment can be locked, and needs to be reduced for UX purposes. Decker-Russell-Osuntokun will also add a "CSV minimum", a number of blocks, which must be smaller than the delay of an HTLC going through the channel. Current routefinding algos are good at minimizing a summed-up cost (like the "CLTV Delta") so the "CSV minimum" may require discovering / developing new routefinding algos.
Due to the "CSV minimum" above, existing nodes that don't understand Decker-Russell-Osuntokun cannot reliably route over Decker-Russell-Osuntokun channels, as they might not impose this minimum properly.
Multipart payments / AMP
Splitting up large payments into smaller parts!
There are at least three variants of multipart payments: Original, Base, and High.
Original is the original AMP proposed by Lightning Labs. It sacrifices proof-of-payment in order to allow each path to have a different payment hash. This is done by having the payer use a derivation scheme to generate each part's payment preimage from a seed, then having the split the seed (using secret sharing) to each part. The receiver can only reconstruct the seed if all parts reach it.
Base simply uses the same payment hash for all routes. This retains proof-of-payment (i.e. an invoice is undeniably signed by the receiver, including a payment hash in the invoice; public knowledge of the payment preimage is proof that the receiver has in fact received money, and any third party can be convinced of this by being shown the signed invoice and the preimage). The receiver could just take one part of the payment and then claim to be underpaid by the payer and then deny service, but claiming any one part is enough to publish the payment preimage, creating a proof-of-payment: so the receiver can provably be made liable, even if it took just one part, thus the incentive of the receiver is to only take in the payment once all parts have arrived to it.
High requires elliptic curve points / scalars. It combines both Original and Base, retaining proof-of-payment (sacrificed by Original) and ensuring cryptographically-secure waiting for all parts (rather than the mere economically-incentivized of Base). This is done by using elliptic curve homomorphism to addition of scalars to add together the payer-provided preimage (really scalar) of Original with the payee-provided preimage (really scalar) of Base.
Better expected reliability. Channels are limited by capacity. By splitting up into many smaller payments, you can fit into more channels and be more likely to successfully reach the payee.
Capacity on mutiple of your channels can be used to pay. Currently if you have 0.05BTC on one channel and 0.05BTC on another channel, you can't pay 0.06BTC without first rebalancing your channels (and paying fees for the rebalance first, whether the payment succeeds or not). With multipart you can now combine the capacities of multiple of your channels, and only pay fees for combining them if the payment pushes through.
Wumbo payments (oversized payments) come "for free" without having to be explicitly supported by the nodes of the network: you just split up wumbo payments into parts smaller than the wumbo limit.
Multipart will have higher fees. Part of the feerate of each channel is a flat-rate fee. Going through multiple paths means paying more of this flat-rate fee.
It's not clear how to split up payments. Heuristics for payment splitting have to be derived and developed and tested.
Payment points / scalars
Using the magic of elliptic curve homomorphism for fun and Lightning Network profits! Basically, currently on Lightning an invoice has a payment hash, and the receiver reveals a payment preimage which, when inputted to SHA256, returns the given payment hash. Instead of using payment hashes and preimages, just replace them with payment points and scalars. An invoice will now contain a payment point, and the receiver reveals a payment scalar (private key) which, when multiplied with the standard generator point G on secp256k1, returns the given payment point. This is basically Scriptless Script usage on Lightning, instead of HTLCs we have Scriptless Script Pointlocked Timelocked Contracts (PTLCs).
Enables a shit-ton of improvements: payment decorrelation, stuckless payments, noncustodial escrow over Lightning (the Hodl Hodl Lightning escrow is custodial, read the fine print), High multipart.
It's the same coolness that makes Schnorr Signatures cool. ECDSA, despite being based on elliptic curves, is not cool because the hash-the-nonce operation needed to prevent it from infringing Schnorr's fatherfucking patent also prevents ECDSA from using the cool elliptic curve homomorphism of addition over scalars.
Requires Schnorr on Bitcoin layer.
Actually, we can work with 2p-ECDSA without waiting for Schnorr. We get back the nice elliptic curve homomorphism by passing the ECDSA nonce through another cryptosystem, Paillier. This gets us the ability to do Scriptless Script. I think it has only 80-bits security because of going through Paillier though.
Basically the conundrum is: we could implement 2p-ECDSA now, hope we never have to test the 80-bit security anytime soon, then switch to Schnorr with 128-bit security later (which means reimplementing a bunch of things, because the calculations are different and the data that needs to be exchanged between channel participants is very different between the 2p-ECDSA and Schnorr). Reimplementing is painful and is more dev work. If we don't implement with 2p-ECDSA now, though, we will be delaying all the nice elliptic curve goodness (stuckless, noncustodial escrow, payment decorrelation) until Bitcoin gets Schnorr.
Elliptic curve discrete log problem is theoretically quantum-vulnerable. If we can't find a qunatum-resistant homomorphic construction, we'll have to give up the advantages (payment decorrelation, stuckless payments, noncustodial escrow over Lightning) we got from using elliptic curve points and go back to boring old hashes.
Ensuring that payers cannot access data or other digital goods without proof of having paid the provider. In a nutshell: the payment preimage used as a proof-of-payment is the decryption key of the data. The provider gives the encrypted data, and issues an invoice. The buyer of the data then has to pay over Lightning in order to learn the decryption key, with the decryption key being the payment preimage.
Enables data providers to sell data. This could be sensors, livestreams, blogs, articles, whatever.
There's no scheme to determine if the data provider is providing actually-useful data. The data-provider could just stream https://random.org for example. This is a potentially-impossible problem. Even if the data-provider provides a "sample" of the data, and is able to derive some proof that the sample is indeed a true snippet of the encrypted data, the rest of the data outside of the sample might just be random junk.
No more payments getting stuck somewhere in the Lightning network without knowing whether the payee will ever get paid! (that's actually a bit overmuch claim, payments still can get stuck, but what "stuckless" really enables is that we can now safely run another parallel payment attempt until any one of the payment attempts get through). Basically, by using the ability to add points together, the payer can enforce that the payee can only claim the funds if it knows two pieces of information:
The payment scalar corresponding to the payment point in the invoice signed by the payee.
An "acknowledgment" scalar provided by the payer to the payee via another communication path.
This allows the payer to make multiple payment attempts in parallel, unlike the current situation where we must wait for an attempt to fail before trying another route. The payer only needs to ensure it generates different acknowledgment scalars for each payment attempt. Then, if at least one of the payment attempts reaches the payee, the payee can then acquire the acknowledgment scalar from the payer. Then the payee can acquire the payment. If the payee attempts to acquire multiple acknowledgment scalars for the same payment, the payer just gives out one and then tells the payee "LOL don't try to scam me", so the payee can only acquire a single acknowledgment scalar, meaning it can only claim a payment once; it can't claim multiple parallel payments.
Can safely run multiple parallel payment attempts as long as you have the funds to do so.
Needs payment point + scalar
Non-custodial escrow over Lightning
The "acknowledgment" scalar used in stuckless can be reused here. The acknowledgment scalar is derived as an ECDH shared secret between the payer and the escrow service. On arrival of payment to the payee, the payee queries the escrow to determine if the acknowledgment point is from a scalar that the escrow can derive using ECDH with the payer, plus a hash of the contract terms of the trade (for example, to transfer some goods in exchange for Lightning payment). Once the payee gets confirmation from the escrow that the acknowledgment scalar is known by the escrow, the payee performs the trade, then asks the payer to provide the acknowledgment scalar once the trade completes. If the payer refuses to give the acknowledgment scalar even though the payee has given over the goods to be traded, then the payee contacts the escrow again, reveals the contract terms text, and requests to be paid. If the escrow finds in favor of the payee (i.e. it determines the goods have arrived at the payer as per the contract text) then it gives the acknowledgment scalar to the payee.
True non-custodial escrow: the escrow service never holds any funds.
Needs payment point + scalar.
Because elliptic curve points can be added (unlike hashes), for every forwarding node, we an add a "blinding" point / scalar. This prevents multiple forwarding nodes from discovering that they have been on the same payment route. This is unlike the current payment hash + preimage, where the same hash is used along the route. In fact, the acknowledgment scalar we use in stuckless and escrow can simply be the sum of each blinding scalar used at each forwarding node.
Privacy! Multiple forwarding nodes cannot coordinate to try to uncover the payer and payee of each payment.
Let's be honest, we **all** care about price. So why do we never discuss valuation?
Valuation, not price. Is it to keep others in the dark, so the cycle of shilling continues? It's confusing to me, because everyone keeps saying 'this coin is undervalued', or 'that coin is overvalued' and then, when asked to back up their claims, they go on listing items on the project's roadmap or talking about the team. I realize that many of you probably don't know what a valuation model is or why it works (to some extent, it's never perfect). Valuation models are especially useful in detecting whether a business (nearly all 2nd and 3rd gen cryptocurrencies are backed by business-like entities) is viable and using multiple different models will tend to yield some kind of 'reasonable range' for a project's market capitalization. If it's under, chances are that it's being underestimated (for cryptocurrencies, that chance is still very low, the entire industry is nascent and we are still in bubble territory). For example, let's take a SaaS (Software as a Service) company. Its most important metrics will likely be:
Customer Acquisition Cost (CAC): the dollar amount the company needs to spend to gain a new customer. This is usually done via various marketing efforts, reputation/word of mouth or direct sales efforts.
Customer Lifetime Value (CLTV): the total expected value our SaaS company can expect from an average customer over the course of their lifetime. By lifetime, we don't mean the client's actual lifespan, but rather the time they'll remain customers.
Market size: the total size (in dollars) of the market the company is in. The bigger it is, the more potential the company has in terms of growth.
Market penetration: the current share of the market the company currently captures.
If our company has a high CAC/CLTV ratio and poor market penetration, even if its market is ENORMOUS and their offering is good, it's unlikely to make a good investment. Growing would simply be too expensive. This is the same for cryptocurrencies. In my opinion, we need to think about valuation in this order:
Philosophy: What is 'x' coin's vision for the future? Is it achievable? Why or why not? We don't go into technical details here, but rather simply examine the leading group/company/foundation/community's plan moving forward. If there are none, you should instantly become extremely skeptical. Right away, this exposes a type of project I personally consider malicious: clones. Bitcoin, Ethereum and other original projects' clones essentially never have a standalone vision. They rely on the reputation and promises of the original.
Value distribution: Let's assume a team's grand vision is exciting, that it makes sense from a business and even tech perspective. How does that team intend to share value created? Most projects simply rely on having their token act as a currency, which is both lazy and dangerous, because it relies on a) high adoption levels, so a self-sustaining economy is created and b) lack of worthy competitors. Do you honestly think that, in the future, there'll be hundreds of economic micro-pockets that require their own currency?
Technology: In concrete scientific terms, how is your project attempting to achieve its grand vision? This is where comparisons like network structure, hashing algorithms, throughput and others are relevant and yes, it only comes third on my list. Technology is a means to an end, not an end itself.
Connections: What are the actors that will allow our technology to flourish? Are strategic partnerships needed? Will end users even be aware that they're using our technology? Partners are a sign of legitimacy, but that's just the beginning. Each and every cryptocurrency that exists today needs an ecosystem to surround it, for the sake of its liquidity, the realization of its use cases, etc.
Until we start creating and discussing valuation formulas, until we start extensively mapping our own vision of a post-DLT (distributed ledger technology) world as a community and calling out people who isolate facts to inflate the value of their own investment and shoot down competitors, we will not grow. We will certainly not help crypto become mainstream, either. The majority waits for safety and clarity before jumping on board, and we're here talking about sharding and audit trails in a vacuum, despite less than 10% of the space really understanding anything about it. We must work on the big picture, and it starts with trying to understand value.
Around summer 2015 when the scalability debate was heating up, two bitcoin conferences were organized. One in Montreal and the other in Hong Kong. After Hong Kong, an email was written to the bitcoin developer mailing list. It became the unofficial manifesto of the pro-Core side in the scalability debate. This "Core Scalability Roadmap" is well worth a read: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-Decembe011865.html 18 months on, it's interesting to see how much of it has happened. Libsecp256k1 has been added to bitcoin and provided a 7x speed up for initial blockchain synchronization. Pruning has been added, which allows a full node to be used without storing the entire blockchain. A number of options for limiting traffic have been added which makes it easier to use a full node on a bandwidth-constrained computer. OP_CLTV and OP_CSV have been added to bitcoin as soft forks. Lightning exists now on the testnet in alpha, it allows instant bitcoin transactions that are much cheaper and more private. The major feature missing is segregated witness, which increases the block size as a soft fork along with several other features. The way soft forks work right now is that miners have a veto on them, and it seems many miners don't want to take either side in the scalability debate. So nothing happens. Which is understandable in a sense that miners didn't ask to be political entities, their job was only ever to set the history and ordering of bitcoin transactions. There are some new thoughts about user-activated-soft-fork which could activate soft forks without all this politics that miners have to keep up with, although the idea is still in the early stages. Back when the scalability debate started the bitcoin price was about $250, as I write today it's nearing $1280; higher than it's ever been. So despite the holdup with segwit it's fair to say things are going pretty well. edit: the roadmap of features in less dense form: https://bitcoincore.org/en/2015/12/23/capacity-increases-faq/
Expose CLTV protocol feature at the GUI level. Using bitcoin-qt it is possible to create and send transactions not spendable until a certain block or time in the future
Expose OP_RETURN protocol feature at the GUI level. Using bitcoin-qt it is possible to create and send transaction with a "public label" -- that is, a string that is embedded in your transaction
Fix sig validation bug related to pre-fork transaction
Improve reindexing performances
Adapt the qa tools (functional and unit tests) to work in a post-fork scenario
Introduce new net magic set. For a period of time the client will accept both set of net magic bits (old and new). The mid term plan is to deprecate the old sets, in the mean time leverage the NODE_CASH service bit (1 << 5) to do preferential peering (already included in 1.1.0)
Avoid forwarding non replay protected transactions and signing new transaction only with the new SIGHASH_FORKID scheme.
Many fixes and small enhancements: orphan pool handling, extend unit tests coverage, improve dbcache performances.
This is a statement I back up 100%. Unfortunatelly I do not remember the autor, so please forgive my pirate deed. I just want to share an opinion I couldn't put better myself: Value. Valuation, not price. Is it to keep others in the dark, so the cycle of shilling continues? It's confusing to me, because everyone keeps saying 'this coin is undervalued', or 'that coin is overvalued' and then, when asked to back up their claims, they go on listing items on the project's roadmap or talking about the team. I realize that many of you probably don't know what a valuation model is or why it works (to some extent, it's never perfect). Valuation models are especially useful in detecting whether a business (nearly all 2nd and 3rd gen cryptocurrencies are backed by business-like entities) is viable and using multiple different models will tend to yield some kind of 'reasonable range' for a project's market capitalization. If it's under, chances are that it's being underestimated (for cryptocurrencies, that chance is still very low, the entire industry is nascent and we are still in bubble territory). For example, let's take a SaaS (Software as a Service) company. Its most important metrics will likely be: Customer Acquisition Cost (CAC): the dollar amount the company needs to spend to gain a new customer. This is usually done via various marketing efforts, reputation/word of mouth or direct sales efforts. Customer Lifetime Value (CLTV): the total expected value our SaaS company can expect from an average customer over the course of their lifetime. By lifetime, we don't mean the client's actual lifespan, but rather the time they'll remain customers. Market size: the total size (in dollars) of the market the company is in. The bigger it is, the more potential the company has in terms of growth. Market penetration: the current share of the market the company currently captures. If our company has a high CAC/CLTV ratio and poor market penetration, even if its market is ENORMOUS and their offering is good, it's unlikely to make a good investment. Growing would simply be too expensive. This is the same for cryptocurrencies. In my opinion, we need to think about valuation in this order: Philosophy: What is 'x' coin's vision for the future? Is it achievable? Why or why not? We don't go into technical details here, but rather simply examine the leading group/company/foundation/community's plan moving forward. If there are none, you should instantly become extremely skeptical. Right away, this exposes a type of project I personally consider malicious: clones. Bitcoin, Ethereum and other original projects' clones essentially never have a standalone vision. They rely on the reputation and promises of the original. Value distribution: Let's assume a team's grand vision is exciting, that it makes sense from a business and even tech perspective. How does that team intend to share value created? Most projects simply rely on having their token act as a currency, which is both lazy and dangerous, because it relies on a) high adoption levels, so a self-sustaining economy is created and b) lack of worthy competitors. Do you honestly think that, in the future, there'll be hundreds of economic micro-pockets that require their own currency? Technology: In concrete scientific terms, how is your project attempting to achieve its grand vision? This is where comparisons like network structure, hashing algorithms, throughput and others are relevant and yes, it only comes third on my list. Technology is a means to an end, not an end itself. Connections: What are the actors that will allow our technology to flourish? Are strategic partnerships needed? Will end users even be aware that they're using our technology? Partners are a sign of legitimacy, but that's just the beginning. Each and every cryptocurrency that exists today needs an ecosystem to surround it, for the sake of its liquidity, the realization of its use cases, etc. Until we start creating and discussing valuation formulas, until we start extensively mapping our own vision of a post-DLT (distributed ledger technology) world as a community and calling out people who isolate facts to inflate the value of their own investment and shoot down competitors, we will not grow. We will certainly not help crypto become mainstream, either. The majority waits for safety and clarity before jumping on board, and we're here talking about sharding and audit trails in a vacuum, despite less than 10% of the space really understanding anything about it. We must work on the big picture, and it starts with trying to understand value
Some thoughts about the activation mechanism for soft forks. In the past we used IsSuperMajority and currently use BIP9 as soft fork activation methods, where a supermajority of hashrate triggers nodes to begin enforcing new rules. Hashrate based activation is convenient because it is the simplest and most straightforward process. While convenient there are a number limitations with this method. Firstly, it requires trusting the hash power will validate after activation. The BIP66 soft fork was a case where 95% of the hashrate was signaling readiness but in reality about half was not actually validating the upgraded rules and mined upon an invalid block by mistake. Secondly, miner signalling has a natural veto which allows a small percentage of hashrate to veto node activation of the upgrade for everyone. To date, soft forks have taken advantage of the relatively centralised mining landscape where there are relatively few mining pools building valid blocks; as we move towards more hashrate decentralization, it's likely that we will suffer more and more from "upgrade inertia" which will veto most upgrades. Upgrade inertia in inevitable for widely deployed software and can be seen for example, with Microsoft Windows. At the time of writing 5.72% of all Microsoft Windows installations are still running Windows XP, despite mainstream support ending in 2009 and being superseded by 4 software generations, Vista, 7, 8 and 10. Thirdly, the signaling methodology is widely misinterpreted to mean the hash power is voting on a proposal and it seems difficult to correct this misunderstanding in the wider community. The hash powers' role is to select valid transactions, and to extend the blockchain with valid blocks. Fully validating economic nodes ensure that blocks are valid. Nodes therefore define validity according to the software they run, but miners decide what already valid transactions gets included in the block chain. As such, soft forks rules are actually always enforced by the nodes, not the miners. Miners of course can opt-out by simply not including transactions that use the new soft fork feature, but they cannot produce blocks that are invalid to the soft fork. The P2SH soft fork is a good example of this, where non-upgraded miners would see P2SH as spendable without a signature and consider them valid. If such an transaction were to be included in a block, the block would be invalid and the miner would lose the block reward and fees. So-called "censorship" soft forks do not require nodes to opt in, because >51% of the hash power already have the ability to orphan blocks that contain transactions they have blacklisted. Since this is not a change in validity, nodes will accept the censored block chain automatically. The fourth problem with supermajority hash power signaling is it draws unnecessary attention to miners which can become unnecessarily political. Already misunderstood as a vote, miners may feel pressure to "make a decision" on behalf of the community: who is and isn't signalling becomes a huge public focus and may put pressures onto miners they are unprepared for. Some miners may not be in a position to upgrade, or may prefer not to participate in the soft fork which is their right. However, that miner may now become a lone reason that vetoes activation for everyone, where the soft fork is an opt-in feature! This situation seems to be against the voluntary nature of the Bitcoin system where participation at all levels is voluntary and kept honest by well balanced incentives. Since miners already have the protocol level right to select whatever transaction they prefer (and not mine those they don't), it would be better if a miner could chose to not participate in triggering activation of something they won't use, but, without being a veto to the process (and all the ire they may have to experience as a consequence). The alternative discussed here is "flag day activation" where nodes begin enforcement at a predetermined time in the future. This method needs a longer lead time than a hash power based activation trigger, but offers a number of advantages and perhaps provides a better tradeoff. Soft forks are still entirely optional to use post activation. For example, with P2SH, many participants in the Bitcoin ecosystem still do not use P2SH. Only 11% of bitcoins are stored in P2SH addresses at the time of writing. Miners are free to not mine P2SH transactions, however, the incentives are such that miners should still validate transactions so they don't accidentally include invalid transactions and cause their block to be rejected. As an additional safety measure for well designed soft forks, relay policy rules prevent non-standard and invalid transactions from being relayed and mined by default; a miner would have to purposefully mine an invalid transaction, which is against their own economic interest. Since the incentives of the Bitcoin system rely on self validation, economic nodes (miners and users) should always remain safe by ensuring their nodes either validate the current rules, or, they can place their network behind a full node that will filter out invalid transactions and blocks at the edge of their network (so called firewall or border nodes). A user activated soft fork is permissive. Miners do not have to produce new version blocks and non-upgraded miners' blocks will not be orphaned as was the case with IsSuperMajority soft forks (e.g. BIP34, BIP66, BIP65-CLTV) which made it a compulsory upgrade for miners. BIP9 "versionbits" soft fork activation method is also permissive in so far as non-upgraded miners are not forced to upgrade after activation because their blocks wont be orphaned. A recent case was the "CSV" soft fork that activated BIP68, BIP112 and BIP113. As such, the CSV soft fork allows non-upgraded miners to continue mining so long as they didn't produce invalid blocks. Miners always retain discretion on which transactions to mine. However, regardless of whether they actively include transactions using the new soft fork feature, or not, the incentive for hash power to upgrade in order to validate is strong: if they do not, they could be vulnerable to a rogue miner willing to waste 12.5BTC to create an invalid block, which may cause non-validating miners to build on an invalid chain similar to the BIP66 incident. Validation has always had a strong requirement. A user activated soft fork is win-win because it adds an option that some people want that does not detract from other peoples' enjoyment. Even if only 10% of users ever wanted a feature, so long as the benefit outweighed the technical risks, it would not be rational to deny others the ability to opt-in. My suggestion is to have the best of both worlds. Since a user activated soft fork needs a relatively long lead time before activation, we can combine with BIP9 to give the option of a faster hash power coordinated activation or activation by flag day, whichever is the sooner. In both cases, we can leverage the warning systems in BIP9. The change is relatively simple, adding an activation-time parameter which will transition the BIP9 state to LOCKED_IN before the end of the BIP9 deployment timeout. You can find the proposal here https://gist.github.com/shaolinfry/0f7d1fd22743bb966da0c0b1682ea2ab References: : https://bitcoin.org/en/alert/2015-07-04-spv-mining : http://p2sh.info/dashboard/db/p2sh-statistics?from=1472043312917&to=1488030912918
Bitcoin Unlimtited version available now with public labels (aka voting) and coin freezing!
Recently I've been working on a pull request for Bitcoin Unlimited. https://github.com/BitcoinUnlimited/BitcoinUnlimited/pull/521 This is in the coin_freeze_cltv branch based on dev. It contains features for the user to freeze coins, to associate public labels with a transaction amount (aka voting) as well as fixes for transaction searching. These features can be merged into Bitcoin Unlimited but we need more user feedback and testing. If you can help test and provide feedback that would be great. If you can help create a deb package for the source that would also be a great help. Otherwise if you could just provide general feedback and let the other devs know if you want these features. If there is demand I will code more .... right now I am thinking about a Bitcoin Supply Chart in the Overview tab that incorporates the mining scchedule, frozen coins and compares your wallet balance versus current and upcoming supply. Freezing coins could be a powerful way for users to indicate which side of a fork they prefer. I also have prepared a draft design of an onchain exchange(ONX) between pre&post fork coins. In this way the community can bet on forking before it happens ONCHAIN! Regarding associating public labels with a transaction amount other than zero that is not native as amounts in the same output as the OP_RETURN are NOT spendable. So this feature was achieved by associating OP_RETURN public label output with the following UTXO. In this way public labels are associated with transaction amounts and so the community can use the same public label to vote on the same topic! The user spends to themselves using a public label and in so doing commits the amount toward that publc label. When multiple users use the exact same public label their unspent amounts are aggregated. The "Top Public Labels" are therefore available as a list on their own tab sorted by the aggregate of UNSPENT amounts currently associated with the public label. When a user wants to unvote their coins they just spend them again with a new public label or none. https://github.com/marlengit/BitcoinUnlimited/tree/gui_top_public_labels
Bitcoin dev IRC meeting in layman's terms (or an attempt to)
As you may or may not know, since scaling bitcoin in Montreal there's a weekly dev meeting on IRC. While very interesting to read, as a non-technical person such as myself it really takes some time to understand what they're all talking about, but I do like to know what they are working on. Since I'm doing the work to find out anyway, I might as well share it with the community. Please bare in mind I'm not a developer and I'd have problems coding "hello world!", so some things might be incorrect or plain wrong. Like any other write-up it likely contains personal biases, although I try to stay as neutral as I can. The full IRC-logs can be found here. There are no decisions being made in these meetings, so if I say "everyone agrees" this means everyone present in the meeting, that's not consensus, but since a fair amount of devs are present it's a good representation. Main topics discussed where: Mempool limiting BIP68 + CHECKSEQUENCEVERIFY CLTV soft fork deployment libconsensus merge time window Mempool limiting
devs are leaning towards 6722 (throwing away the cheapest txn and setting min relay fee to it) because it's the more simpler approach and possibly less edge-cases. The idea behind it is to have a mem-pool that gives a good approximation on what'll be included in the next blocks, meaning higher fee transactions. This approach also helps to build a fee-estimator. Some devs propose to include a time-based eviction as well.
6722 should be completed and 6722, 6557 and 6673 should be attacked by the others to try and find edge-cases. The default mempool size should be 300Mb. Chain limits
Related to mempool limiting. Chain in this context means connected transactions. When you send a transaction that depends on another transaction that has yet to be confirmed we talk about a chain of transactions. Miners ideally take the whole chain into account instead of just every single transaction (although that's not widely implemented afaik). So while a single transaction might not have a sufficient fee, a depending transaction could have a high enough fee to make it worthwhile to mine both. This is commonly known as child-pays-for-parent. Since you can make these chains very big it's possible to clog up the mempool this way. The first unconfirmed transaction is called the ancestor and the transactions depending on it the descendants. The total amount of transactions is referred to as "packages".
All of the mempool limiting approaches are way easier to attack if you have bigger chain limits. the reason to have larger descendant packages is you can't control that yourself, somebody pays you and bob, and bob chains off a million descendants and he ends up screwing you. if you have a say 900kb ancestor package limit, then even if the ancestor fee rate is reasonably high, default mining code is likely going to find 100kb of very high fee txs to include first, and then there won't be room for your ancestor package. Morcos proposes 25/250kb for ancestors and 50/500kb for descendants, meaning max. either 25 transactions or 250kb in size for ancestors. Most seem to be fine with those limits and even smaller. -meeting conclusion morcos writes a chain-limit proposal to post on the mailing list in order to find possible usecases for large chain transactions. CHECKLOCKTIMEVERIFY softfork
Commonly referred to as: How you thought nLockTime worked before you actually tried to use it. There's a fair amount of demand for this and the code is reviewed and has been running on sidechains alpha for 6 months. The only real issue is how and when it's merged. Currently softforks have been done by the isSuperMajority mechanism, meaning when 95% of the last X blocks has a version number higher than X the fork is deployed. A new way of doing this is currently being worked on and that uses all bits of the version number, appropriately being called versionbits. So instead of a fork happening when the version is larger than (for example) 00000000011 (3), a fork happens when (for example) the 3rd bit is up (so 00100000011). This way softforks can be deployed simultaneous and independent of each other.
Questions are being posed whether we wait for other time-related BIP's and/or versionbits, or do it now using isSuperMajority. If versionbits is deployed later it needs to wait for all supermajority softforks to be over. Vladimir van der Laan doesn't want to deploy any soft forks in major releases (0.12 in this case) so that people explicitly upgrade for the softfork not for other things. You could roll out multiple supermajority forks as long as they are cumulative. Talks seem to converge to using supermajority to deploy checkLockTimeVerify and checkSequenceVerify if it's ready by the end of October.
checkLockTimeVerify backports (deployment in older versions) needs to be reviewed as well as BIP68, 112 and 113 (all the time-related BIP's). Libconsensus
Satoshi wasn't the best programmer out there, which leaves a pretty messy code. Ideally you'd have the part of the code that influences the network consensus separately, but in bitcoin it's all intertwined. Libconsensus is what eventually should become this part. This way people can more easily make changes in the non-consensus part without fear of causing a network fork. This however is a slow and dangerous project of moving lot's of code around.
Lot's of discussion on when existing changes should be merged, when the code should be frozen for next release etc. In linux changes are merged right after a major release. jtimon notices this was planned for after 0.10 and 0.11 too, but nothing happened. There seems to be a lack of planning and overview as to what where has to go.
jtimon will provide a high level rationale for what and where things should move so people can make comments and review according to this rationale. Participants dstadulis Daniel Stadulis wumpus Wladimir J. van der Laan morcos Alex Morcos gmaxwell Gregory Maxwell btcdrak btcdrak jonasshnelli Jonas Schnelli maaku Mark Friedenbach sdaftuar Suhas Daftuar sipa Pieter Wuille BlueMatt Matt Corallo CodeShark Eric Lombrozo Luke-Jr Luke Dashjr bsm117532 Bob McElrath jgarzik Jeff Garzik
The cryptocurrency craze has been nothing short of a phenom, seeing billions and billions of dollars created out of thin air, seemingly what felt like overnight has been nothing short of a marvel to watch transpire. The headlines, blogs, forums all speak of new found riches, what seem like buried treasures found yet, as new high’s are made for Bitcoin and other currencies, new lows of self worth are discovered daily by the countless gambling addictions that have popped up with the rise and adoption of cryptocurrencies like Bitcoin. Available for iPod, iPhone, iPad, Mac, Windows, Linux, Android, Samsung, etc, 24/7, 365 days a week at lighting fast speeds. Deposit bonus’, VIP raffles for “free bitcoins”, holiday bonus’, VIP trips, personal hosts - yup, it’s all in there and at times - it is magnificent. However, that is the single most toxic thing about gambling addiction - if even for the briefest of moments, that time it is “magnificent’ takes a grasp so powerful, so determined and so unlikely to be found again until the hooks are set, play play play, reward, play play play, reward are deeply embedded into our cognitive behavior and we have conditioned ourselves to only derive happiness from the briefest of moments that the brain thinks are “magnificent” I am one of those soul’s who lost millions of dollars, friends, close relationships, values, moral compass and most importantly - my ability to derive self worth, self love and empathy until I was almost too far down the rabbit hole to turn back. Some statistics suggest nearly 80% of all Bitcoin’s in circulation are used for online gambling. They are the ideal currency for such gaming activities - anonymous, instant and can be broken down into payments both big and small. I am writing this not to shun any operators nor the industry, I am writing this to let the thousands of people across the world who are in pain that no words can truly describe from the ruin and shame that go hand in hand with gambling addiction see that they are not alone. The cycle of destruction that comes with gambling addiction is on par with the greatest of natural disasters. I have had the fortune of experiencing almost every type of addiction one could experience, nothing even comes close to the grip, power and intensity of online gambling addiction. Now, I can understand those reading have little room for sympathy, after all, gambling addiction is and has been portrayed as one of the most shameful addictions one can develop that are stigmatized by both the media and in real life. The goal of this is to not earn anyone’s sympathy, I do not feel like a victim - these are choices I made however, the US Surgeon General (whom I have the greatest respect for going against the grain) has coined (no pun intended) any addiction as a “Cognitive Brain Disease”. This means that the chemical makeup of one’s brain is altered and is not a choice. The brain gets rewired to release the chemical dopamine only when a jackpot or “near jackpot” is obtained. Perhaps a brief explanation of the “gamblers high” will help put things into context. If I am playing cards, and let’s say I had a good run and am up $1 million. Each win experiences diminishing marginal returns, this means that the pleasure or “high” I receive from each win decreases with each bet. In fact, even if I won another $1 million jackpot while up $1 million, it will offer significantly less of a “high” than if i were to be down $1 million. The gamblers high, as little sense as it may make to some, is not experienced when someone is winning money. The gamblers high truly kicks in when someone is down big and comes out of nowhere with a big win. The speed and velocity of such a swing, down $1 million to up $1 million (we call that a $2m swing) is the true “high” and the rush of that high is so great we chase it leaving our lives, pride and self worth behind. The force of that rush is so intense, gamblers will intentionally bet irrationally to lose their winnings simply so they can experience that rush. Like I said, it’s the most vicious addiction I have experienced New technologies that show the game is “Provably Fair” allow players to instantly verify the “die roll” if you will to show that the outcome was already decided by the software before the person decided how much they would bet or what decisions they would make during the game. (Ex: the makeup of the shoe of cards in a game of blackjack) I commend the industry for providing such transparency and unlike fiat currency gaming sites, offer the “fairest” chance of any gambler to “take down the house” however, it takes a bit more understanding of the psyche behind gambling addiction to understand the predatory nature of such sites and the bells and whistles used to retain those addicts over long periods of time to boost their CLTV (Customer Lifetime Value) Technology has led to great advancements and power, with that comes responsibility. I am very much a believer in capitalism and feel very strongly a business should have the ability to take advantage of inefficiencies in a market to their advantage. I would simply ask the operators of such site to do this with caution, offer a sense of humanity and empathy and help educate players around loss limits, self exclusion and even offer help if a player is playing on “edge”. While taking advantage of a market opportunity is one thing, taking advantage of someone’s emotional and financial vulnerability goes against every facet of our own humanity. For those who are struggling with online gambling addiction or any addiction to that matter, I know all too well just how dark times can get, I feel your pain and understand that hope is a hard thing to come by when such a spiral takes hold. Turn to those who you love for help, be brutally honest in whatever you are experiencing no matter how much shame you think it may bring. Shame is a funny thing, everyone has it, nobody likes to talk about it and the less we talk about it the more we have. The sense of relief and hope that offers almost instant gratification when you are honest with yourself and your families about your addiction is the biggest catalyst to overcoming the darkest of days. Find yourself, no matter how hard or dark it may be, the rewards are greater than any jackpot can offer
Once again my attempt to summarize and explain the weekly bitcoin developer meeting in layman's terms. Link to last weeks summarization Disclaimer Please bare in mind I'm not a developer and I'd have problems coding "hello world!", so some things might be incorrect or plain wrong. Like any other write-up it likely contains personal biases, although I try to stay as neutral as I can. There are no decisions being made in these meetings, so if I say "everyone agrees" this means everyone present in the meeting, that's not consensus, but since a fair amount of devs are present it's a good representation. The dev IRC and mailinglist are for bitcoin development purposes. If you have not contributed actual code to a bitcoin-implementation, this is probably not the place you want to reach out to. There are many places to discuss things that the developers read, including this sub-reddit. link to this week logslink to meeting minutes Main topics discussed this week where: Mempool limiting: chain limits Low-S change CLTV & CSV review Creation of bitcoin discuss mailing list off-topic but important notice This issue has made most JS bitcoin software vulnerable to generating incorrect public keys. "This is an ecosystem threat with the potential to cause millions of dollars in losses that needs higher visibility; though it's not a bitcoin core / bitcoin network issue. Common, critical, JS code is broken that may cause the generation of incorrect pubkeys (among other issues). Anyone who cares for a JS implementation should read that PR." Mempool limiting: chain limits
(c/p from last week) Chain in this context means connected transactions. When you send a transaction that depends on another transaction that has yet to be confirmed we talk about a chain of transactions. Miners ideally take the whole chain into account instead of just every single transaction (although that's not widely implemented afaik). So while a single transaction might not have a sufficient fee, a depending transaction could have a high enough fee to make it worthwhile to mine both. This is commonly known as child-pays-for-parent. Since you can make these chains very big it's possible to clog up the mempool this way. The first unconfirmed transaction is called the ancestor and the transactions depending on it the descendants. The total amount of transactions is reffered to as "packages".
since last week
As said in "Chain limits" last week Morcos did write a proposal about lowering the default limits for transaction-chains. 2 use cases came up which are currently in use or happened before: As example: someone buys bitcoin from a website and can spend those bitcoin in the marketplace of the same website without waiting for confirmation in order to improve the bitcoin user-experience. This leaves a sequential transaction chain. They don't need to chain more than 5 transactions deep for this, and it falls within the proposed limits. What's not within the proposed limits is the chain of +/- 100 transactions a company had during the spam-attacks. These where simply increased activities by end-users while not enough UTXO's where available (3 to be precise)(UTXO: unspent transaction output, an output that can be used as input for a new transaction). Notably this is with the best practices of using confirmed transactions first. Ways this can be solved from the company's end is to have more UTXO's available before hand, bundling transactions (which requires delaying customer's request) or using replace-by-fee to add payees (which saves blockchain space, is cheaper in fees and gets transactions through quicker, but is not widely deployed by miners atm). Bare in mind these proposals are for default values for the memorypool, not in any way hard limits.
Sense of urgency. Quoting sipa: "my mempool is 2.5G... we better get some solution!" Current attack analysis assumes child-pays-for-parent mining, it should probably be done again without. Higher limits on number of transactions increase attack-vectors. Proposed number of transactions gets some push-back, total size limit not. Mixing default values (for example having a 50% of a 10/10 limit and 50% of a 100/100 limit) wastes bandwidth while there are too many factors that limit utility of long chains as well. 25 transaction limit ought to be enough for everyone (for now).
This is in regards to the recent malleability attack. Which is caused by a value 'S' in the ECDSA signature which can be 2 values, a high and low value and still be valid. Resulting in different transaction id's. more info A solution for this is to require nodes to have the "low-s" encoding for signatures. Downside is that it will block most transactions made by sufficiently out of date software (+/- pre-march 2014) This does not replace the need for BIP62, it only eliminates the cheap DOS attack.
95% of transactions already confirm to this, and more fixes have been applied since. BlueMatt has a node which several people are running that auto-malleates to low-s transactions. Questions whether we release it ASAP or wait for the next release and get it to a couple of miners in the meantime (possibly with auto-lowS-malleating)
CLTV: checkLockTimeVerify CSV: checkSequenceVerify Both new time-related OP-codes. Been discussed heavily last week.
Concerns whether CSV will be ready enough for release later this month. There's no clarity on how things look when all 3 time related pull-requests are merged. There's a number of people still reviewing the pull-requests. Uncertainty and confusion about whether the semantics are final or not (in regards to using bits from nSequence). nSequence are 4 bytes intended for sequencing time-locked transactions, but this never got used. Now these bytes are being repurposed for a mixture of things. Currently the plan is: " bits 0..15 are the relative locktime, bit 30 determines units (0: height, 1: time w/ 512s granularity), and bit 31 toggles BIP 68 (0: on, 1: off). bits 16..29 are masked off and can take any value."
Clarification from maaku regarding nSequence for BIP68. (after the meeting he explained he was waiting for opinions, but not enough people seemed to know the issue at hand) Continue review of pull requests 6312, 6564 and 6566 Creation of bitcoin discuss mailing list
The bitcoin-dev mailing list is intented for technical discussions only. There's things that don't belong there but need to be discussed anyway. Now this is done in bitcoin-dev, but the volume of this is getting too big. There's recently also an influx of really inappropriate posts, level kindergarden.
No clarity about who are the moderators. Next week there'll be a bitcoin-discuss list created. Decisions are needed as to who'll become the moderators for that and bitcoin-dev. Decisions are needed as to what will be the list and moderation policies.
The bitcoin-discuss list will be created as well as a simple website listing all the lists and corresponding policies. A meeting is scheduled on monday to discuss the moderation and policies of said lists. Participants morcos Alex Morcos gmaxwell Gregory Maxwell wumpus Wladimir J. van der Laan sipa Pieter Wuille BlueMatt Matt Corallo btcdrak btcdrak petertodd Peter Todd warren Warren Togami phantomcircuit Patrick Strateman dstadulis Daniel Stadulis GreenIsMyPepper Joseph Poon bsm117532 Bob McElrath
Expose CLTV protocol feature at the GUI level. Using bitcoin-qt it is possible to create and send transactions not spendable until a certain block or time in the future
Expose OP_RETURN protocol feature at the GUI level. Using bitcoin-qt it is possible to create and send transaction with a "public label" -- that is, a string that is embedded in your transaction
Fix sig validation bug related to pre-fork transaction
Improve reindexing performances
Adapt the qa tools (functional and unit tests) to work in a post-fork scenario
Introduce new net magic set. For a period of time the client will accept both set of net magic bits (old and new). The mid term plan is to deprecate the old sets, in the mean time leverage the NODE_CASH service bit (1 << 5) to do preferential peering (already included in 1.1.0)
Avoid forwarding non replay protected transactions and signing new transaction only with the new SIGHASH_FORKID scheme.
Many fixes and small enhancements: orphan pool handling, extend unit tests coverage, improve dbcache performances.
Bitcoin dev IRC meeting in layman's terms (2015-10-22)
Once again my attempt to summarize and explain the weekly bitcoin developer meeting in layman's terms. Link to last weeks summarization Disclaimer Please bear in mind I'm not a developer and I'd have problems coding "hello world!", so some things might be incorrect or plain wrong. Like any other write-up it likely contains personal biases, although I try to stay as neutral as I can. There are no decisions being made in these meetings, so if I say "everyone agrees" this means everyone present in the meeting, that's not consensus, but since a fair amount of devs are present it's a good representation. The dev IRC and mailinglist are for bitcoin development purposes. If you have not contributed actual code to a bitcoin-implementation, this is probably not the place you want to reach out to. There are many places to discuss things that the developers read, including this sub-reddit. link to this week logsMeeting minutes by meetbot Main topics discussed where: Mempool Memory Usage LevelDB replacement Median Past locktime & CLTV Short topics/notes BIP 9 Versionbits PR #6816 is ready for implementation and needs more reviews. A 3 month moderation period on the bitcoin-dev mailinglist has started, as well as a new list bitcoin-discuss. more details: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-Octobe011591.html "bitcoin.org had incorrect release notes for 0.11.1. It's corrected now. They had posted the release notes for the initial RC and not updated them. Process wise it would be good to watch out for that in the future." Mempool Memory Usage
When a transaction is relayed across the network it is held by the nodes in memory, until it gets into a block. All these transactions that sit in memory are called the memorypool or mempool for short. Like we could see during the spam-attack if there's a big back-log of transactions that couldn't make it in the blockchain this mempool can get pretty big resulting in nodes crashing. To stop this from happening devs created a mechanism to reject and/or remove transactions from the mempool. This mempool limiting got merged this week. Also relevant: There is an already existing limit on the database cache size called "dbCache". The default value for that is 100MB.
Testing shows there's a discrepancy between the configured mempool limit and the actual memory usage. This is caused by the amount of UTXO data when processing transactions. This data is only flushed after a block is processed (so temporarily exceeding the cache limit set in dbCache). There are 2 "obvious" solutions for this:
Always enforce the UTXO cache limit, just like the mempool limit is always enforced. Downside for that is if you misconfigure your mempool limit an attack can blow away your UTXO cache, which significantly slows down validation and propagation.
Take the UTXO cache into account when limiting the mempool. Downside for that is that you could construct transactions which require way more cache space and thereby more easily kick out other transactions.
A more optimal solution would be to give priority in the cache to things in the mempool. Ways to achieve that are to kick UTXO's from transaction that are evicted from the mempool out of the cache and from transactions that never made it into the mempool. Something TheBlueMatt is working on
Continue to research and optimize. LevelDB replacement
LevelDB is the database system currently used in bitcoin. Since this is not being maintained for some time devs are looking for replacements.
jgarzik worked on a patch for SQLite Some people express concerns whether the performance will be good enough with SQLite, but there are no benchmark results yet.
Do research into other options Do lots of benchmarks and report results Median Past locktime & CLTV
When a block is created miners include a timestamp. This timestamp has to be between the median of the previous 11 blocks and the network-adjusted time +2 hours. So this timestamp can vary a decent amount from the real time. With the introduction of lock-time transactions, that are only valid after a certain time, miners are incentivised to lie about the time in order to include time-locked transactions (and their fees) that wouldn't otherwise be valid. BIP 113 enables the usage of GetMedianTimePast (the median of the previous 11 blocks) from the prior block in lock-time transactions to combat this behaviour. Users can compensate for this by adding 1 hour (6 blocks) to their lock times. CLTV stands for CheckLockTimeVerify, BIP65 Commonly reffered to as: How you thought nLockTime worked before you actually tried to use it.
CLTV is ready to be merged (and has been merged at time of writing) Questions of whether to add median past locktime as mempool only or as softfork Overall questions as to what to include in the CLTV deployment, what to include as mem-pool only and what as softfork. Median past locktime violates current 'standard' behavior, so we would prefer to have that violation dead in the network before the median past locktime softfork moves forward.
review BIP-113: Mempool-only median time-past as endpoint for lock-time calculations review the CLTV backports (done and merged at time of writing) Backport median past locktime to 0.10 and 0.11 Participants btcdrak btcdrak sipa Pieter Wuille gmaxwell Gregory Maxwell BlueMatt Matt Corallo morcos Alex Morcos petertodd Peter Todd CodeShark Eric Lombrozo jgarzik Jeff Garzik maaku Mark Friedenbach kanzure Bryan Bishop jcorgan Johnathan Corgan Luke-Jr Luke Dashjr jonasschnelli Jonas Schnelli sdaftuar Suhas Daftuar
Today Dalilcoin 0.1.6 is being released: https://github.com/aliibrahim80/dalilcoin/releases/tag/v0.1.6 Yesterday 0.1.5 was released, but there was an off by one staking bug. Please use 0.1.6 instead. The rest of this post is a copy and update of yesterday's deleted post. The consensus code has been rewritten (since 0.1.4) in a way that helps nodes sync more easily. Another important change is that the communication with the ltc node can be done with remote ltc nodes (including ltc nodes run behind a tor hidden service). Dalilcoin 0.1.6 will stop staking and will not accept blocks after May 1, 2019. The plan is to hard fork Dalilcoin on May 1, 2019, to make the following changes:
Change the retarget algorithm.
Add CLTV and CSV to scripts.
Allow publication of theories and signatures.
The code with support for the hard fork should be released as Dalilcoin 0.2.0 in a few weeks (by April 1, 2019 April 21, 2019), so be sure to check for the update in late April. Here are some more details about the planned hard fork. The current retarget algorithm oscillates too much. It has not been uncommon for the difficulty to go high enough that there would be several days with no blocks. If seven days were to go by with no blocks staked, the chain would die (without a hard fork) and seven days without a block seems like it would eventually happen due to the oscillation with the current retarget algorithm (retarget_orig in block.ml). There is a new retarget algorithm (retarget_dampened in block.ml) which makes a much smaller change in the difficulty each block. Dalilcoin scripts corresponded to Bitcoin scripts as they were at the time of the snapshot, before Bitcoin added CLTV and CSV. Adding CLTV and CSV would allow for atomic swaps between Dalilcoin and coins like Bitcoin and Litecoin. The ocaml code for this has not been written yet, but should be easy. Technically there are no "sequence numbers" or "nlocktime" values in Dalilcoin txs, so CLTV could be called "absolute locktime" and CSV "relative locktime," but calling them CLTV and CSV makes it easier to see the analogy with the corresponding bitcoin opcodes. The support for the theories and signatures has been in the code from the beginning, but has been disabled due to a lack of testing. Someone pointed me to the qeditas-egal github repo that contains examples of theories and signatures. If the testing goes well, theories and signatures will no longer be disabled.
Bitcoin dev IRC meeting in layman's terms (2015-10-15)
Once again my attempt to summarize and explain the weekly bitcoin developer meeting in layman's terms. Link to last weeks summarization Disclaimer Please bear in mind I'm not a developer and I'd have problems coding "hello world!", so some things might be incorrect or plain wrong. Like any other write-up it likely contains personal biases, although I try to stay as neutral as I can. There are no decisions being made in these meetings, so if I say "everyone agrees" this means everyone present in the meeting, that's not consensus, but since a fair amount of devs are present it's a good representation. The dev IRC and mailinglist are for bitcoin development purposes. If you have not contributed actual code to a bitcoin-implementation, this is probably not the place you want to reach out to. There are many places to discuss things that the developers read, including this sub-reddit. link to this week logsMeeting minutes by meetbot Main topics discussed where: Mempool limiting sendheaders BIP versionbits dev/discuss list policy CHECKSEQUENCEVERIFY Mempool limiting
When a transaction is relayed across the network it is held by the nodes in memory, until it gets into a block. All these transactions that sit in memory are called the memorypool or mempool for short. Like we could see during the spam-attack if there's a big back-log of transactions that couldn't make it in the blockchain this mempool can get pretty big resulting in nodes crashing. To stop this from happening devs are trying to find a way to limit this mempool, so a mechanism to reject and/or remove transactions from the mempool. The hard part here is to make it so nodes can't be attacked by abusing this mechanism. So far the devs are going with TheBlueMatt's proposal of throwing away the cheapest txn and setting the min relay fee to it
While testing, sipa encountered transactions that took 200ms to be accepted into the mempool. As it's the first time he has benchmarked this and the pull-request shouldn't make an impact on these times it likely doesn't have anything to do with this. However, such times are bad either way. The average time in sipa's tests is 4ms. (After the meeting Morcos did some benchmarking and confirmed it was not specific to this PR, and pointed out the outliers come from CheckInputs and HaveInputs (as you might guess, having to do with checking the inputs) Question on why we should revert the minrelay (minimum fee for nodes to relay a transaction) back to 1000 (it has been set to 5000 to quick-fix the mempool issues), sipa thinks it should be floating as well or the dust limit becomes ineffective.
send headers BIP Copy/paste from the BIP: Since the introduction of "headers-first" downloading of blocks in 0.10, blocks will not be processed unless they are able to connect to a (valid) headers chain. Consequently, block relay generally works as follows:
A node (N) announces the new tip with an "inv" message, containing the block hash
A peer (P) responds to the "inv" with a "getheaders" message (to request headers up to the new tip) and a "getdata" message for the new tip itself
N responds with a "headers" message (with the header for the new block along with any preceding headers unknown to P) and a "block" message containing the new block However, in the case where a new block is being announced that builds on the tip, it would be generally more efficient if the node N just announced the block header for the new block, rather than just the block hash, and saved the peer from generating and transmitting the getheaders message (and the required block locator).
Question on how to move forward. How to let the nodes know you want the blockheader instead of the blockhash. Options:
Send a "sendheaders" message early when connecting so the way peers want their block announcement is immediately known.
Send a "sendheaders" message at any time, changing the way peers want their block announcement from hashes to headers.
No one likes to extend the version message further. There's no strong advantage to have an "options" message over a "sendheaders" message. Having the message being sent early on might be too constraining. Possible usecase from morcos: "its entirely possible some future optimization may say, i want to send sendheaders to these peers b/c they announce a lot of new stuff to me and not these others b/c they don't". Most people like this to be enable-only, so no message to get back to receiving blockhashes. Which is how the BIP was drafted.
sdaftuar does a pull-request for the BIP to get a number assigned and proceeds with the BIP as drafted. versionbits
BIP 9 Currently softforks have been done by the isSuperMajority mechanism, meaning when 95% of the last X blocks has a version number higher than Y the fork is deployed. A new way of doing this is currently being worked on and that uses all bits of the version number, appropriately being called versionbits. So instead of a fork happening when the version is larger than (for example) 00000000011 (3), a fork happens when (for example) the 3rd bit is up (so 00100000011). This way softforks can be deployed simultaneous and independent of each other.
copy/paste from IRC, since I don't know what this specifically means: CodeShark: so right now it's just a unit that implements the versionbits logic but does not demonstrate its usage I thought it would be better to actually integrate in a separate PR, but I can add a demonstration sipa: separate commit, same PR - i think we need something that's mergable as a whole, to be able to see whether the whole thing easily backports Codeshark (who's implementing versionbits) had some more remarks but no one present had seemed to reviewed it, so not much use in discussing things further.
The bitcoin-dev mailing list is intended for technical discussions only. There's things that don't belong there but need to be discussed anyway. Now this is done in bitcoin-dev, but the volume of this is getting too big. There's recently also an influx of really inappropriate posts, level kindergarden. For the things that don't belong on bitcoin-dev, but need to be discussed anyway there's a new list being created namely bitcoin-discuss as well as clear policies and moderation for both.
Bitcoin-discuss was created, but the admin password wasn't distributed to jgarzik who's willing to guide the moderation. Seperate moderation-proposals have been done meanwhile. People just want it to move on.
Since none of the people who proposed a moderation-scheme are present we'll let them discuss it among each other and post their decisions publicly. CHECKSEQUENCEVERIFY
CheckLockTimeVerify (CLTV) repurposes the nSequence field (nSequence are 4 bytes intended for sequencing time-locked transactions, but this never got used). However, there's no way use these values in a bitcoin script. CheckSequenceVerify (CSV) makes this field accessible to bitcoin scripts. EDIT: Turns out this is not entirely correct as it is relative locktime that repurposes the nSequence field.
CLTV is pretty much done. Check to see maaku moving one of the bits to allow for other implementations to have better granularity has any objections. As long as we're using as few bits as possible the exact semantics are less important for most people. sipa points out a possible bug that influences the wallet. CSV is not on target for the end of of the month, although a lot of work and progress has been made.
Review and ACK/NACK of 6312 BIP-68: Mempool-only sequence number constraint verification Review and ACK/NACK of 6566 BIP-113: Mempool-only median time-past as endpoint for lock-time calculations Participants wumpus Wladimir J. van der Laan sipa Pieter Wuille btcdrak btcdrak gmaxwell Gregory Maxwell morcos Alex Morcos maaku Mark Friedenbach CodeShark Eric Lombrozo BlueMatt Matt Corallo sdaftuar Suhas Daftuar warren Warren Togami GreenIsMyPepper Joseph Poon davec Dave Collins cfields Cory Fields jonasschnelli Jonas Schnelli Comic relief 19:21 sdaftuar it sounds like everyone is ok with the BIP as drafted then? 19:21 wumpus yes 19:21 gmaxwell I think so. 19:22 davec yes 19:22 sipa well, the only person with concerns was cfields, who doesn't seem to be here :) 19:22 gmaxwell sipa: he can raise concerns later too! 19:22 cfields dammit! 19:22 sipa cfields: too late! 19:22 gmaxwell ha 19:23 cfields did i really miss my third one of these in a row?
Bitcoin dev IRC meeting in layman's terms (2015-11-05)
Once again my attempt to summarize and explain the weekly bitcoin developer meeting in layman's terms. Link to last weeks summarization Note that I crosspost this to Voat, bitcoin.com and the bitcoin-discuss mailing list every week. I can't control what's being talking about in the meeting, if certain things come up I might not be able to post here because of "guidelines". Disclaimer Please bear in mind I'm not a developer and I'd have problems coding "hello world!", so some things might be incorrect or plain wrong. Like any other write-up it likely contains personal biases, although I try to stay as neutral as I can. There are no decisions being made in these meetings, so if I say "everyone agrees" this means everyone present in the meeting, that's not consensus, but since a fair amount of devs are present it's a good representation. The dev IRC and mailinglist are for bitcoin development purposes. If you have not contributed actual code to a bitcoin-implementation, this is probably not the place you want to reach out to. There are many places to discuss things that the developers read, including this sub-reddit. link to this week logsMeeting minutes by meetbot Main topics discussed where: Sigcache performance Performance goals for 0.12 transaction priority sigops flooding attack chain limits Short topics/notes Note: cfields, mcelrath and BlueMatt (and maybe more) missed the meeting because of daylight saving time. Closing date for proposals for the scaling bitcoin workshop is the 9th. Check to see if there are any other commits for the 0.11.2 RC. As soon as 6948 and 6825 are merged it seems good to go. We need to move fairly quick as there are already miners voting for CLTV (F2Pool). Also testnet is CLTV locked already and is constantly forking. 0.11.2 RC1 has been released as of today: https://bitcoin.org/bin/bitcoin-core-0.11.2/test/ Most of the mempool-limiting analysis assumed child-pays-for-parent, however that isn't ready for 0.12 yet, so we should think about possible abuses in context of the existing mining algorithm. Because of time-constrains opt-in replace-by-fee has been deferred to next weeks meeting, but most people seem to want it in 0.12. sdaftuar makes a note that we need to make clear to users what they need to do if they don't want to accept opt-in transactions. Sigcache performance
The signature cache, which is in place to increase performance (by not having to check the signature multiple times), and to mitigate some attacks currently has a default limit of 50 000 signatures. Sipa has a pull-request which proposes to: Change the limit from number of entries to megabytes Change the default to 40MB, which corresponds to 500 000 signatures Store salted hashes instead of full entries Remove entries that have been validated in a block
Sipa did benchmarks for various signature cache sizes on hitrate in blocks (how many of the cached signatures are in the block). The maximum sigcache size was 68MB, resulting in a 3% miss-rate. Some blocks though have extremely high miss rates (60%) while others have none. Likely caused by miners running different policies. Gmaxwell proposed to always run script verification for mempool transactions, even if these transactions get rejected into the mempool by the clients policy. The result of that is that even a 300MB sigcache size only gets down to 15% misses. So there's too much crap being relayed to keep any reasonable sized cache. Gmaxwell points out downsides to not checking any rejected transactions, namely: there are some DOS attacks possible, and you increase your misrate if you set a policy which is more restrictive than the typical network, which might result in a race to the bottom.
Sipa continues his work and seeks out other strategies Performance goals for 0.12
Bitcoin-core 0.12 is scheduled for release December 1st.
Mentioned pull-requests should be reviewed, prioritizing CreateNewBlock transaction priority
Each transaction is assigned a priority, determined by the age, size, and number of inputs. Which makes some transactions free.
Sipa thinks we should get rid of the current priority completely and replace it with a function that modifies fee or size of a transaction. There's a pull-request available that optimizes the current transaction priority, thereby avoiding the political debate that goes with changing the definition of transaction priority. Luke-jr thinks the old policy should remain possible.
Check to see if PR #6357 is safe and efficient enough. sigops flooding attack
The number of ECDSA signature-checking operations or sigops is currently limited to 20 000 per block. This in order to prevent miners creating blocks that take ages to verify as those operations are time-consuming. You could however construct transactions that have a very high sigops count and since most miners don't take into account the sigops count they end up with very small blocks because the sigop limit is reached. This attack is described here.
Suggestion to take the number of sigops relative to the maximum blocksize into account with the total size. Meaning a 10k sigops transaction would currently be viewed as 500kB in size (for that single transaction, not towards the block). That suggestion would be easy to change in the mining code, but more invasive to try and plug that into everything that looks at feerate. This would also open up attacks on the mempool if these transactions are not evicted by mempool limiting. Luke-jr has a bytes-per-sigop limit, that filters out these attack transactions.
More analysis should be done, people seem fine with the general direction of fixing it. chain limits
Chain in this context means connected transactions. When you send a transaction that depends on another transaction that has yet to be confirmed we talk about a chain of transactions. Miners ideally take the whole chain into account instead of just every single transaction (although that's not widely implemented afaik). So while a single transaction might not have a sufficient fee, a depending transaction could have a high enough fee to make it worthwhile to mine both. This is commonly known as child-pays-for-parent. Since you can make these chains very big it's possible to clog up the mempool this way. With the recent malleability attacks, anyone who made transactions going multiple layers deep would've already encountered huge problems doing this (beautifully explained in let's talk bitcoin #258 from 13:50 onwards) Proposal and github link.
sdaftuar's analysis shows that 40% of blocks contain a chain that exceeds the proposed limits. Even a small bump doesn't make the problem go away. Possible sources of these chains: a service paying the fees on other transactions (child-pays-for-parent), an iOS wallet that gladly spends unconfirmed change. A business confirms they use child-pays-for-parent when they receive bitcoins from an unspent chain. It is possible that these long chains are delivered to miners directly, in which case they wouldn't be affected by the proposed relay limits (and by malleability). Since this is a problem that needs to be addressed, people seem fine with merging it anyway, communicating in advance to let businesses think about how this affects them.
morcos Alex Morcos gmaxwell Gregory Maxwell wumpus Wladimir J. van der Laan sipa Pieter Wuille jgarzik Jeff Garzik Luke-Jr Luke Dashjr phantomcircuit Patrick Strateman sdaftuar Suhas Daftuar btcdrak btcdrak jouke ??Jouke Hofman?? jtimon Jorge Timón jonasschnelli Jonas Schnelli
20:01 wumpus #meetingend 20:01 wumpus #meetingstop 20:01 gmaxwell Thanks all. 20:01 btcdrak #exitmeeting 20:01 gmaxwell #nomeetingnonono 20:01 btcdrak #meedingexit 20:01 wumpus #endmeeting 20:01 lightningbot Meeting ended Thu Nov 5 20:01:29 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . 20:01 btcdrak #rekt
Understanding the Lightning Network, Part 3: Completing the Puzzle and Closing the Channel
https://preview.redd.it/jxhm8fypd0t11.png?width=1341&format=png&auto=webp&s=007ddefb2d762a908d0644bee5aba637d668faa1 Understanding the Lightning Network, Part 3: Completing the Puzzle and Closing the Channel The Lightning Network is probably the most highly anticipated technological innovation that will be deployed on top of Bitcoin. The payment layer, first proposed by Joseph Poon and Tadge Dryja about a year ago, promises to support a virtually unlimited number of off-chain transactions among users, at nearly no cost – while leveraging the security offered by Bitcoin. At least three companies – Poon and Dryja's Lightning, Blockstream and Blockchain – are currently working on implementations of the technology. But few outside this small technological frontline fully grasp how the “future of micropayments” is set to boost Bitcoin’s capabilities. In this three-part series, Bitcoin Magazine lays out the basic building blocks of the Lightning Network, and shows how they fit together to realize this upcoming protocol layer. The first part of this series covered basic building blocks, and explained how these are used to establish bidirectional payment channels. The second part explained how a network is formed, and how Hash Timelock Contracts (HTLCs) link different channels in the network together. This third and final part of the series explains how HTLCs are placed inside bidirectional payment channels to ensure transactions can occur fully off-chain. The Lightning Network So far, Alice and Bob opened a bidirectional payment channel, which they both funded with five bitcoins. They've made two transactions back and forth, and at the current channel state, both Alice and Bob can claim five bitcoins for themselves by “dropping the channel” on the blockchain. Now, they want to include an HTLC in the channel. This is to ensure that if Carol claims a bitcoin from Bob in return for her value, Bob is guaranteed a bitcoin from Alice in return. Like the previous step, Alice and Bob start by creating a new commitment transaction each. In many ways, these commitment transactions are very similar to previous commitment transactions. They include a normal output, and an output to a funky multisig-address with a CSV (CheckSequenceVerify)-timelock and a special hash-lock. Likewise, as in the previous step, Alice and Bob exchange their old secrets, to effectively invalidate the old channel. And, once exchanged, both Alice and Bob can sign their halves of the commitment transactions and potentially drop them on the blockchain at any time. All familiar territory. Except for one change. Both Alice’s and Bob's commitment transactions now include one new output, worth one bitcoin. (This makes the balance 4-5-1; four for Alice, five for Bob, one for the new output.) This new output is essentially the HTLC. And it's even funkier than all other outputs so far, because there are three ways to unlock it. First, the new output (in both Alice’s and Bob's commitment transactions) releases the bitcoin on condition that Bob's signature and the value is included in the subsequent transaction. As such, regardless of whether Alice or Bob signs and broadcasts the commitment transaction, only Bob can unlock this output – if he includes the value. But there is one small difference between the two commitment transactions: if Bob drops the channel, there is a CSV-timelock involved. He will need to wait 1,000 blocks. (If Alice drops the channel he can claim this bitcoin immediately.) The reason Bob has to wait 1,000 blocks if he drops the channel is very similar to what we've seen before: It allows Alice to take this bitcoin in case Bob ever tries to sign and broadcast an old channel state. That's where the second way to unlock the output comes in. Alice can “steal” the funds if she provides Bob's (newest) secret. Two can play this game: If Alice ever tries to cheat and broadcast this channel when it's already outdated, Bob can claim this bitcoin using Alice’s secret. (He wouldn't even need to provide the value.) And third, as with any other HTLC, both commitment transactions also include the usual CLTV time-out fall-back for Alice. If Bob does not include the value in - say - two weeks (for instance because he didn't get it from Carol), Alice can claim her bitcoin back. Again, whether Alice or Bob drops the channel doesn't matter for this option. So where did all this get us? Both Alice and Bob hold a half-valid commitment transaction. If Alice drops her commitment transaction on the blockchain, she immediately sends five bitcoins to Bob. Additionally, she can wait for 1,000 blocks, and claim four bitcoins for herself. Plus, Bob has two weeks to provide the value, and claim the bitcoin in “HTLC output.” (If he doesn't provide the value in two weeks, Alice can claim this bitcoin back.) Bob, meanwhile, can drop his commitment transaction at any time as well, and immediately send four bitcoins to Alice. Then, he'd have wait 1,000 blocks to claim five more bitcoins from one address, and another bitcoin from the HTLC output if he provides the value. (If he doesn't provide the value in two weeks, Alice can reclaim it.) And of course, if either Alice or Bob tries to cheat at any point in the future, and sign and broadcast this channel when it’s outdated, both can completely block the other, and steal all bitcoins in the channel. https://preview.redd.it/essveiasd0t11.jpg?width=1289&format=pjpg&auto=webp&s=6871dbd14d4a9ea974ed2b46ec30ca46ad09fc36 Settling the Status At this point, Bob is guaranteed to receive a bitcoin in exchange for the value (assuming he has it). All he has to do is sign and broadcast the commitment transaction he got from Alice, include the value in a subsequent transaction, and sign and broadcast that as well. Alice knows this. There is no way she can cheat Bob out of his bitcoin – not even if she found out what the value is through some other means. As such, the two might as well just “settle” outside of the channel. Bob can simply give the value to Alice, and Alice can agree to update the channel status to the more normal state without the HTLC and the time-out deadline. Assuming both parties want to keep the channel open, that's what they would naturally do: it's less of a hassle than having to drop the channel on the blockchain.Payment Channel https://preview.redd.it/k1mjw3itd0t11.jpg?width=1203&format=pjpg&auto=webp&s=2ab576fe1fc63a6fa55cc9efabfc0d4020931d9b Closing the Channel And finally, here's the real power of the Lightning Network: Almost everything described in these three articles will typically never need to hit the Bitcoin blockchain at all. If both Alice and Bob want to close the channel “peacefully” they can simply create a transaction from the original opening transaction to override everything that happened since the opening transaction. From this closing transaction, they send themselves their fair share of the channel, as represented by the most recent channel state. Concretely, this means that if Alice wants to close the channel, she can at this point simply create a transaction paying herself four bitcoins and Bob six, and ask Bob to sign and broadcast the transaction. Since there is no reason for him not to, he will probably cooperate and close the channel. In the end, only two transactions will have been broadcast over the Bitcoin network and included in a block: the opening and the closing transactions. That will hold true even if Alice and Bob transact a million times in between, therefore unloading a huge burden away from the blockchain. https://preview.redd.it/d12qwarud0t11.jpg?width=722&format=pjpg&auto=webp&s=3ca17affcfe233e67397abe29ed1472b9bf06510 https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-completing-the-puzzle-and-closing-the-channel-1466178980/
Understanding the Lightning Network, Part 2: Creating the Network
https://preview.redd.it/0xnfa0daj3s11.png?width=990&format=png&auto=webp&s=eebff071633b7c8f93721d60bd68d2e43a2beccd The Lightning Network is the probably most highly anticipated technological innovation to be deployed on top of Bitcoin. The payment layer, first proposed by Joseph Poon and Tadge Dryja about a year ago, promises to support a virtually unlimited number of off-chain transactions among users, at nearly no cost – while leveraging the security offered by Bitcoin. At least three companies – Poon and Dryja's Lightning, Blockstream and Blockchain – are currently working on implementations of the technology. But few outside this small technological frontline fully grasp how the “future of micropayments” is set to boost Bitcoin’s capabilities. In this three-part series, Bitcoin Magazine lays out the basic building blocks of the Lightning Network, and shows how they fit together to realize this upcoming protocol layer. The first part of this series covered basic building blocks, and explained how these are used to establish bidirectional payment channels. This second part explains how bidirectional payment channels are turned into a network. The Network In the previous article, Alice and Bob established a bidirectional payment channel. Now, Alice wants to pay one bitcoin to a third person, Carol. To do so, Alice and Carol could open up a payment channel between them. But they don't actually need to. As it turns out, Bob and Carol already have a mutual channel, so Alice can simply pay Carol through Bob. Specifically, Alice can pay Bob one bitcoin, and Bob can pay Carol one bitcoin. However, Alice doesn't really trust Bob – or Carol for that matter. She's afraid that if she pays Bob, Bob will never actually pay Carol. Or perhaps Bob will pay Carol, but Carol will claim she never received the money, and Alice wouldn't know whom to blame. Alice, therefore, wants to ensure that she only pays Bob one bitcoin, if he also pays Carol one bitcoin. This is accomplished (in part) with a simple cryptographic trick. When Alice wants to send Carol a bitcoin, she tells Carol to create a value (a random string of numbers) and send her the hash. Alice also tells Carol to exchange the original value with Bob for a bitcoin. Alice, meanwhile, takes the hash from Carol, turns to Bob, and tells Bob she will give him a bitcoin if he provides her the corresponding value (which only Carol has). So, Bob turns to Carol, and gives Carol one bitcoin in return for the value. Then, Bob turns back to Alice with the value. Alice knows Bob must have gotten the value from Carol in exchange for a bitcoin, and therefore concludes Carol got her bitcoin. So Alice can confidently give Bob a bitcoin. Everybody is happy. https://preview.redd.it/7f17n7bfj3s11.jpg?width=1203&format=pjpg&auto=webp&s=8598cfddf9173d690e5ba194731ebe405cf0533b Well. .. almost everybody is happy. In this “naive” scenario, middleman Bob still has to trust Alice and Carol. Bob has to trust Carol to really give him the value after he sent her a bitcoin, and Bob has to trust Alice to really give him a bitcoin once he presents her the value. The bitcoin-for-value trades must therefore be absolutely guaranteed along the network. More specifically: if Bob gives a bitcoin to Carol, he must be guaranteed to get a bitcoin back from Alice. That's where Hash Time-Locked Contracts (HTLCs) come in. Hash Time-Locked Contracts So Alice and Bob want to exchange a bitcoin for the value through an HTLC. (And Bob and Carol also want to a bitcoin exchange for that same value - but never mind that for now.) To do so, rather than sending Bob a bitcoin straight up, Alice sends a bitcoin to a new (and, again: funky) multisig address. The bitcoins locked up on this address can be unlocked in two different ways. The first option is for Bob to include his signature and the value. The second option is for Alice to include her own signature. However, this option has a CLTV-timelock on it: Alice can sign and broadcast the transaction only after – say – two weeks have gone by. This means that Bob has two weeks to create a subsequent transaction in which he includes his signature and the value, and broadcast it to send the bitcoin from the funky multisig address to himself. As such, this trade is guaranteed. Bob can only claim Alice's bitcoin if he provides the value: broadcasting it over the Bitcoin network makes it publicly visible for Alice to see. And if Bob doesn’t provide the value in time, there is a “time-out alternative” for Alice to get her bitcoin back. Simple. https://preview.redd.it/sfimnxfgj3s11.jpg?width=1203&format=pjpg&auto=webp&s=c4ff868439076097a35f6b9848762ca05944b585 Back to the network, as that’s really why this HTLC setup is needed. As mentioned, not only Alice and Bob, but also Bob and Carol established an HTLC. So, if Carol claims her bitcoin from Bob, Bob will get the value in return; it will be visible on the blockchain. Therefore, if that happens, Bob is guaranteed to get a bitcoin from Alice as well. Bob can take the value that Carol made publicly visible on the blockchain, include it in his HTLC with Alice, and claim a bitcoin for himself, too. The two channels are effectively linked. As a final detail, it is important that Bob gets the value from Carol before Alice can reclaim her bitcoin from Bob. If Bob gets the value from Carol only after Alice already reclaimed hers back, Bob is stuck in the middle after all. The time-out in Bob and Carol’s HTLC must therefore expire before the time-out in Alice and Bob’s HTLC expires. (For example after exactly ten days, instead of two weeks. This is also why HTLCs need CheckLockTimeVerify (CLTV)--and not CheckSequenceVerify (CSV).) https://preview.redd.it/2axendhhj3s11.jpg?width=1203&format=pjpg&auto=webp&s=d62e7ffb675781880402c3edd665cf5741acd4a1 Lastly, there's one more problem to solve: for the Lightning Network to be useful, all this must be accomplished off-chain. How this is done, is covered in the third and final article of this series. https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-creating-the-network-1465326903/ #InziderX #Exchange #ico https://inziderx.io/
With the current min_cltv_expiry_delta which is around 40 to 144 blocks, nodes set the locktime_max to 2016 or two weeks, ... It is good news for the Bitcoin network, however, that these lapses were not discovered by some ruthless hackers, but great scientific minds happy to work for the good of humanity, and the blockchain. ... With the latest release of the BitcoinPlus 2.7.0, BitcoinPlus (XBC) has been dragged well and truly to the forefront of blockchain technology. This release includes the Bitcoin Improvement Proposal 65 (BIP65). BIP65 relates to CHECKLOCKTIMEVERIFY (CLTV). BIP65 is a Soft Fork to the BitcoinPlus Network. Bitcoin Core (formerly Bitcoin-Qt) is the third Bitcoin client, developed by Wladimir J. van der Laan based on the original reference code by Satoshi Nakamoto.It has been bundled with bitcoind since version 0.5. Bitcoin-Qt has been rebranded to Bitcoin Core since version 0.9.0.. Bitcoin Core is a full-fledged client that forms the basis of the network. . It is characterized by a high level of ... On 25/10/2018 the latest BitcoinPlus Client 2.7.0 was released. This release contained several soft forks. The first soft fork was Bitcoin Improvement Proposal 65 (BIP65) which relates to CHECKLOCKTIMEVERIFY (CLTV). $ watch -n 0.1 ./src/bitcoin-cli -netinfo 3 Bitcoin Core v0.20.99.0-bf1f913c44 - 70016/Satoshi:0.20.99/ Peer connections sorted by direction and min ping <-> relay net mping ping send recv txn blk uptime asmap id version in full onion 1 1 0 10206 70015/Satoshi:0.20.1/ in full ipv6 246 246 1 9 0 1 16509 10202 70015/Satoshi:0.19.1/ in block onion 37686 425955 42 42 25 10143 70015/Satoshi:0.20.1 ...
BITCOIN NEXT MAJOR MOVE WILL BE HORRIFYING (btc crypto live price news prediction analysis today ta
Bitcoin: The End Of Money As We Know It traces the history of money from the bartering societies of the ancient world to the trading floors of Wall St. The d... Bitcoin is one of the most important inventions in all of human history. For the first time ever, anyone can send or receive any amount of money with anyone ... News clips from CLTV. Sign in to like videos, comment, and subscribe. 🙋♂️Bybit $90 Bonus: https://www.bybit.com/app/register?ref=nOnj5 🙋♂️ Bitcoin Technical Analysis & Bitcoin News Today: Ethereum is looking bullish ... Bitcoin Price Analysis & Crypto News! 👍 THUMBS UP & SUBSCRIBE NOW + 🔔! ***** 🚨 VIP ELITE PRIVATE TRADE ALERTS- https://t.me/joinchat/AAAAAEts9GFT3RV_6wLjOQ...