Just like Vernor Vinge said, we are on the edge of change comparable to the rise of human life on Earth (). The time we are right now may be a technological singularity, and the key factor of this singularity is artificial intelligence, the things that would change the world. Most parts of people think that artificial intelligence would make our life better, make the world better, but this view may be too one-sided and strongly subjective. Everything is a double-edged sword, artificial intelligence is not exception, we must take it seriously, otherwise what could artificial intelligence brought us, might be a disaster.
Why are people so keen on developing artificial intelligence? Eternal life they want to be immortal. Look back at history, we can see that most of the species would experience such a process: species emerged, existed for a period of time and then inevitably fell off the balance beam of life and fell into the abyss of extinction. Just like every creature would die, every specie would extinct, including human beings.
What if we develop a AI that is smart enough to lead us to immortal? We might face two possible choices eternal or extinct (Nick Bostrom). Unlike perpetual motion machines, which our knowledge of physics could prove impossible, it is not impossible to be immortal, at least we did not found any evidence to proof that the death is inevitable, it is just a kind of disease for which no cure has been found. For now, the death is bounded with aging, but scientists think aging may not be bounded with time.
Essentially speaking, death is caused by the decay and destruction of the tissues that make up our body, all we need is a technology that could perfectly produce tissues we need and replace ours with it. The technology we need, we could not develop it by ourselves, but AI could. If we could continuously produce tissues we need, which means we stop aging, we could be immortal, could we? The answer is yes, but how could we be sure about what would happen if we reach that level super Artificial Intelligence. If it could produce parts of body easily, why cant it destroy us easily. Why we hope for eternal life on a far beyond our intelligence on the super-AI? Because it is beyond our imagination, it could do anything, it could solve all problem, it could make us eternal, it could rejuvenate us, it can also make a specie extinct, unleash unimaginable diseases, but there has nothing we can do, only thing we could do is watching. This expectation is more like a gamble, it is not a proper scientific attitude, we cant grasp the consequences of this line. Counting on an unpredictable, temperamental super-intelligence, this mentality is as passive and helpless as a primitive man praying to the gods without being able to change whether it is a gift or an anger. Never try to manipulate what you do not know.
We could not guarantee that the super-intelligence would born after we know how to control it.