Proof-of-work difficulty increasing

We had our first automatic adjustment of the proof-of-work difficulty on 30 Dec 2009.  

The minimum difficulty is 32 zero bits, so even if only one person was running a node, the difficulty doesn't get any easier than that.  For most of last year, we were hovering below the minimum.  On 30 Dec we broke above it and the algorithm adjusted to more difficulty.  It's been getting more difficult at each adjustment since then.

The adjustment on 04 Feb took it up from 1.34 times last year's difficulty to 1.82 times more difficult than last year.  That means you generate only 55% as many coins for the same amount of work.

The difficulty adjusts proportionally to the total effort across the network.  If the number of nodes doubles, the difficulty will also double, returning the total generated to the target rate.

For those technically inclined, the proof-of-work difficulty can be seen by searching on "target:" in debug.log.  It's a 256-bit unsigned hex number, which the SHA-256 value has to be less than to successfully generate a block.  It gets adjusted every 2016 blocks, typically two weeks.  That's when it prints "GetNextWorkRequired RETARGET" in debug.log.

minimum    00000000ffff0000000000000000000000000000000000000000000000000000
30/12/2009 00000000d86a0000000000000000000000000000000000000000000000000000
11/01/2010 00000000c4280000000000000000000000000000000000000000000000000000
25/01/2010 00000000be710000000000000000000000000000000000000000000000000000
04/02/2010 000000008cc30000000000000000000000000000000000000000000000000000
14/02/2010 0000000065465700000000000000000000000000000000000000000000000000
24/02/2010 0000000043b3e500000000000000000000000000000000000000000000000000
08/03/2010 00000000387f6f00000000000000000000000000000000000000000000000000
21/03/2010 0000000038137500000000000000000000000000000000000000000000000000
01/04/2010 000000002a111500000000000000000000000000000000000000000000000000
12/04/2010 0000000020bca700000000000000000000000000000000000000000000000000
21/04/2010 0000000016546f00000000000000000000000000000000000000000000000000
04/05/2010 0000000013ec5300000000000000000000000000000000000000000000000000
19/05/2010 00000000159c2400000000000000000000000000000000000000000000000000
29/05/2010 000000000f675c00000000000000000000000000000000000000000000000000
11/06/2010 000000000eba6400000000000000000000000000000000000000000000000000
24/06/2010 000000000d314200000000000000000000000000000000000000000000000000
06/07/2010 000000000ae49300000000000000000000000000000000000000000000000000
13/07/2010 0000000005a3f400000000000000000000000000000000000000000000000000
16/07/2010 000000000168fd00000000000000000000000000000000000000000000000000
27/07/2010 00000000010c5a00000000000000000000000000000000000000000000000000
05/08/2010 0000000000ba1800000000000000000000000000000000000000000000000000
15/08/2010 0000000000800e00000000000000000000000000000000000000000000000000
26/08/2010 0000000000692000000000000000000000000000000000000000000000000000

date, difficulty factor, % change
2009           1.00
30/12/2009     1.18   +18%
11/01/2010     1.31   +11%
25/01/2010     1.34    +2%
04/02/2010     1.82   +36%
14/02/2010     2.53   +39%
24/02/2010     3.78   +49%
08/03/2010     4.53   +20%
21/03/2010     4.57    +9%
01/04/2010     6.09   +33%
12/04/2010     7.82   +28%
21/04/2010    11.46   +47%
04/05/2010    12.85   +12%
19/05/2010    11.85    -8%
29/05/2010    16.62   +40%
11/06/2010    17.38    +5%
24/06/2010    19.41   +12%
06/07/2010    23.50   +21%
13/07/2010    45.38   +93%
16/07/2010   181.54  +300%
27/07/2010   244.21   +35%
05/08/2010   352.17   +44%
15/08/2010   511.77   +45%
26/08/2010   623.39   +22%

[Edit: I later found that I was generating quite a bit more than that, just didn't realize it because of the "matures in xx more blocks" concept. I still think it will be a major headache when the difficulty significantly increases though. I apologize for my silliness ]

Satoshi, I figured it will take my modern core 2 duo about 20 hours of nonstop work to create ฿50.00! With older PCs it will take forever. People like to feel that they "own" something as soon as possible, is there a way to make the generation more divisible? So say, instead of making ฿50 every 20 hours, make ฿5 every 2 hours?

I don't know if that means reducing the block size or reducing the 120-block threshold to say 12-block only or what, but because the difficulty is increasing I can imagine that a year from now the situation will be even worse (3+ weeks until you see the first spendable coins!) and we better find a solution for this ASAP.

I generated 5 blocks today on my Pentium processor. Two of them were within 3 minutes of each other.

I have noticed some slowdown since the adjustment, but I still generate a lot of coins. My computer is off while I'm sleeping, and BitCoin bootstraps quickly when I turn it back on. Do you guys-who-are-having-trouble have the BitCoin port open?

I thought about that but there wasn't a practical way to do smaller increments.  The frequency of block generation is balanced between confirming transactions as fast as possible and the latency of the network.

The algorithm aims for an average of 6 blocks per hour.  If it was 5 bc and 60 per hour, there would be 10 times as many blocks and the initial block download would take 10 times as long.  It wouldn't work anyway because that would be only 1 minute average between blocks, too close to the broadcast latency when the network gets larger.

2 seconds of latency in both directions should reduce your generation success by less than 1%.

Probably OK, but I'm not sure.  The protocol is designed to resync to the next message, and messages get re-requested from all the other nodes you're connected to until received.  If you miss a block, it'll also keep requesting it every time another blocks comes in and it sees there's a gap.  Before the original release I did a test dropping 1 out of 4 random messages under heavy load until I could run it overnight without any nodes getting stuck.

My understanding is that every Bitcoin client has the same algorithm (formula) built into it to automatically adjust the difficulty every so many blocks. Not only that, but I think that Bitcoin will not accept blocks generated at a different difficulty, so if a modified Bitcoin client tried to send out more easily generated blocks, all the authentic clients would reject the fake blocks.

Then how is it dependent on how many CPU's are connected to the whole network?

We need Satoshi to confirm that because clients accept blocks generated at easier difficulties all the time whenever the PoW's difficulty increases.

We were essentially discussing Sabunir's question about what prevents someone from messing with the program's source code to adjust block-generating difficulty to be very easy, then make a network on his own and create a, say, 50,000-block proof-of-work within seconds then finally propagate it across the real network to steal "votes" towards his new fake blocks as technically, his proof would be "the longest". So is there a way to verify how much work was actually put into a given PoW (for eg. how many zero's are at the beginning of each hash or something)?

Since we're at it, what's the approximate time for proof-of-work propagation across a network of about 100,000 machines? Is there a way to optimize connections so that broadcasting is done via a pyramid-form to minimize the needed time? For example, the block creator sends it to 10 nodes, then those 10 send it to a 100 provided that none of those 100 were among the original 11, then those 100 tell a 1000 provided that none of those 1000 were among the original 111, etc to save time.

It may take longer than 8 hours to generate a block.

Have you previously generated bitcoins? Are the number of blocks listed at the bottom of Bitcoin greater than 42650? Those need to download before it can start generating coins. How many connections are listed at the bottom of Bitcoin? Did you click Options > Generate Coins? How much CPU does your process viewer show that Bitcoin is using? Is your Internet connection steady? I had problems when I tried sharing Internet from my smartphone to my computer.

I think that no bitcoins generated in 8 hours from within a VM utilizing two modern cores is probably not unusual. Keep it running for a few days and I expect that you'll generate more than a few packs of bitcoins.

I am also wondering about Suggester's question.  It seems like modifying the code to give a node an advantage in generating coins might be possible.

I am confused as to why each node on the network is actually doing when set to generate coins.  What problem are they solving that takes 100% CPU?

Interestingly, using laszlo's mac os version of bitcoin, one can see how many hashes per second the computer is performing.  I'm currently getting about 1 million hashes per second.  Given the current difficulty 0000000013ec53, I'll have to perform about 2^35~3x10^10 hashes before I have a decent chance of getting one below the target, and at 10^6/s, that should take about 30000 sec, or about two per day.  The actual interval varies a lot - it's a random process, but that seems to be more-or-less the correct amount.

Satoshi, could you update the first post in this thread, with the complete history of difficulty-of-work increases please?  I'd try, but for some reason, I've lost my logfiles.  Fortunately the wallet is safe.

Yep, your understanding here is correct. It does not matter what exactly gets hashed, and no, you can't cheat without first breaking SHA-256, which is considered difficult.

The salient property of cryptographic hash functions is that they are as random as is possible while still being deterministic. That's what their strength depends on -- after all if they weren't random, if there were obvious patterns, they could be broken that way. So the ideal hash function behaves just like a random number generator. It does not matter what you feed in, timestamp or not, whatever's put in there, the hash should still behave randomly (i.e. every possible outcome has the same a-priori probability of occuring). Incrementing by one works just as well as completely changing everything every step (this follows from the avalanche property). However, the initial value, before you start incrementing, must be (pseudo-)randomly chosen, or every computer will start at the same point, and the fastest one always wins, which is not what is wanted here.

That's a good idea.  I'm not sure where exactly to fit that in, but it could certainly calculate the expected average time between blocks generated, and then people would know what to expect.

Every node and each processor has a different public key in its block, so they're guaranteed to be scanning different territory.

Whenever the 32-bit nonce starts over at 1, bnExtraNonce gets incremented, which is an arbitrary precision integer.

Can someone share the math required to compute the estimated amount of time to generate a pack of bitcoins?

Basically, a formula that integrates the difficulty (either as a hex number (currently 000000000f675c00000000000000000000000000000000000000000000000000) or as the difficulty factor (currently 16.62)) and number of hashes checked per second (currently like 1M for me) which would return the average time between blocks found?

Sounds completely reasonable. I just started generating today, so I'll let you know once I get more blocks. Thanks!

Cool, I was just playing around with trying to make it adjust the rate without taking much of a performance hit.. that's why it tries to adjust that variable, to keep it to a few per second or so.. maybe you can figure out a better formula, I just kind of did that by experimenting around with it.

The main part of the formula that I'm uneasy about is the "target probability" of 0.5. 0.5 is used for calculations involving brute-forcing passwords, but maybe this is different. If your blocks consistently take double the amount of time that the formula predicts, use 1 instead.

BitCoin does say when it has solved a block. Search for "proof-of-work found".


No. Each node's block is unique because it contains their unique public key. Functionally, every hash gives you a totally random number between 0 and the maximum value of a 256-bit number. If the random number is equal to or below the target, you win. So the probability of winning with one hash is target in max.

Starting on another block because somebody else won requires almost no work.

See the BitCoin Wiki "block" article and the BitCoin paper for more info.

Thanks for clearing that up, fergalish!

I say keep it simple; more choices isn't always better, it just makes it overwhelming and confusing for most users.

"difficulty" : 45.38582234101263

It jumped from 23 in a couple days.  I think this pretty much puts an end to generating a block a day with a personal computer.. but you can still get lucky.  Now you'll need to build a cluster or hijack a college computer lab for it to be worth doing   I expect the trading value will increase significantly over the next few weeks as the supply slows down; should be interesting.

I'd be interested in seeing something like "expected bitcoins generated/day" next to (or in place of) the khash/s number. I'd rarely need to see the khash/s number since that won't change unless I make changes to the software or hardware.

13/07/2010 0000000005a3f437d4a7f529fd4a7f529fd4a7f529fd4a7f529fd4a7f529fd4a

Holy....

Satoshi, what happens if the rush dries up for a bit; some of the slashdotters or whoever get tired? Does the difficulty ever go back down?

It adjusted to 181.54 a few minutes ago.  Typical time to get a block is about a week now.

The difficulty can adjust down as well as up.

The network should be generating close to 6 blocks per hour now.

Yeah, I've noticed the "10 second blocks" are gone, replaced with 419 and 741 second block generation with no more in the last 20 minutes. That should keep those server farms on hold for a while  

Now, correct me if I'm wrong, but now that block generation is taking a lot longer, doesn't that mean that the lucky person who got the block is going to take a lot longer to be verified by the network that he/she was the winner before they could ever spend it?

So if the difficulty was increased so high that it took a day to find a winning block, that means the lucky winner would have to wait 120 day before they could spend it or about 4 months if everyone else was averaging about the same speed? Seems like at the high end of the difficulty, there is an issue with coin generation vs. being able to put it into circulation by spending. Wouldn't the long delay cause a lot of generated coin to be lost because anything could happen to the PC that won in a long amount of time if the winner had to really wait that long? They might un-install the program or the computer get eaten by a virus or power surge well before then.

Right, the difficulty adjustment is trying to keep it so the network as a whole generates an average of 6 blocks per hour.  The time for your block to mature will always be around 20 hours.

The recent adjustment put us back to close to 6 blocks per hour again.

There's a site where you can see the time between blocks, and since block 68545, it's been more like 10 minutes per block:
http://nullvoid.org/bitcoin/statistix.php

No, this isn't correct.  Finding the hash requires a whole bunch of effort, but the process of verification that the "winner" has found a matching block is by comparison a trivial exercise and doesn't take all that long.  Once they have found the block, they can spend those coins right away.

What this does mean is that the coin allocation system has now become a lottery, where new winners are receiving a block of coins worth a whole lot more (due to exchange rates and scarcity) than earlier blocks which were in comparison relatively trivial in value.  My question is, how valuable will this "lottery" become in the long run (in terms of Euros or Dollars per block generated)?

Essentially what is happening is that the computer is picking a random sequence of numbers (like a lottery) and if that computer happens to pick the correct sequence of bits, you "win".  The change in difficulty is something akin to playing a lottery that has you only picking six numbers vs. one with fourteen or a hundred numbers to win.  The odds are actually worse for a bitcoin block than even the worst of any normal lottery that has ever been conceived at the moment.

I value Bitcoin as an anonymous digital currency.  Although I'm not expecting to get rich, I'd like the ability to continuously generate enough Bitcoin to purchase desired services.

Is there any expectation that economic value per khash/sec (or client) per day will be at least somewhat stable?  Difficulty just increased 300%, and USD/Bitcoin just increased about 500% (although that may turn out to be a spike).  I do get that there's no necessary relationship.  However, perhaps there's an economic basis for one (however approximate it might be).

New difficulty factor 244.213223092
+35%

I updated the first post.

date, difficulty factor, % change
2009          1.00
30/12/2009    1.18   +18%
11/01/2010    1.31   +11%
25/01/2010    1.34    +2%
04/02/2010    1.82   +36%
14/02/2010    2.53   +39%
24/02/2010    3.78   +49%
08/03/2010    4.53   +20%
21/03/2010    4.57    +9%
01/04/2010    6.09   +33%
12/04/2010    7.82   +28%
21/04/2010   11.46   +47%
04/05/2010   12.85   +12%
19/05/2010   11.85    -8%
29/05/2010   16.62   +40%
11/06/2010   17.38    +5%
24/06/2010   19.41   +12%
06/07/2010   23.50   +21%
13/07/2010   45.38   +93%
16/07/2010  181.54  +300%
27/07/2010  244.21   +35%