No-one has ever said raw computational power will spontaneously create a conscious being without appropriate programming. That’s several times more ridiculous than saying a fast enough computer will spontaneously model astrophysics. The computational power is a necessary but not sufficient condition.
The more power you have to work with, the less brilliant you have to be to write the software. Sometime soon the necessary intelligence level will drop below that of the smartest AI researchers, and if we still don’t have intelligent machines a few decades after that, THEN we can start arguing about whether AI has failed.

Put simply, humans have never created or been in full control of any system as complex as the human brain, including the brain itself. We absolutely cannot trust our instincts when it comes to predicting how these kinds of systems will behave.

Are you sure? Have you tested that scenario? No, you haven’t. Nobody has. So you don’t know that to be true. And anyway, who ever said AI would be “human”? Most transhumanists hope AI will be much, much more than human.

“If you are making the claim that consciousness is ‘just a computer’, then it is up to you to prove it.”

I am making the claim that everything we have learned about the brain so far indicates that it is a kind of computer. It’s primary purpose seems to be to store and retrieve information (and it also governs the production of certain hormones). And it is conscious, so yes, it would seem that sufficiently powerful computers are conscious. If you have evidence to suggest that the brain is something more than a data processing computer, please share it.

And you haven’t responded to my question about what evidence you have that the “promises” of transhumanism (and you don’t specify which ones) are “false and dangerous”?

The same number of artificial neurons and their connections does not a human make.

It’s all well and good to quote Thompson’s cynical comments about the forecasting of Kurzweil et al but their work is based on hard evidence. What science is your/Thompson’s cynicism based on?

What should be important are the facts. And the simplest facts are that the human brain is an organic data processing system. It’s processing capability has been estimated by Moravec and Kurzweil to be in the order to 10 quadrillion instructions per second (10 QIPS). Another fact is that IBm have recently built Roadrunner, a supercomputer with a processing capability of 1 QIPS. With the typical doubling of the capability of supercomputers we see every year, by 2012 we should have a supercomputer able to process around 16 QIPS, which would make it 50% faster than the theoretical speed of the human brain.

Who knows that man-made machine with that power will ‘experience’?

To suggest that consciousness is anything other than a really, really fast computation of data, you must explain what else it involves. What is your theory?