Not the first to say it but considering who is saying this, it deserves consideration. http://en.wikipedia.org/wiki/Elon_Musk <blockquote>Musk: I think we should be very careful about artificial intelligence. If I had to guess at what our biggest existential threat is, it's probably that. So we need to be very careful with artificial intelligence. I'm increasingly inclined to think that there should be some regulatory oversight, maybe at the national and international level, just to make sure that we don't do something very foolish. With artificial intelligence we're summoning the demon. You know those stories where there's the guy with the pentagram, and the holy water, and he's like - Yeah, he's sure he can control the demon? Doesn't work out. [audience laughs] Q: So I'll take it there'll be no HAL9000 going to mars? Musk: Heh. HAL 9000 would be easy [to deal with in comparison to the AI he's talking about]. It's way more complex… it'd put HAL9000 to shame. That's like a puppy dog. </blockquote>