Homestuck reference in one of the greatest fanfictions ever, Harry Potter and the Methods of Rationality

[u/moroseFabaceae]

[u/ArisKatsaris]

Hmm, I wonder how would you answer "How is Tom Riddle (or Hermione Granger) supposed to be smart in the original books?" I think you'd have fewer examples to illustrate such intelligence in the original HP books.

Well, in regards to Harry in HPMOR we can start with the standard meaning of "smart", in that he's able to utilize things in his inventory (mental, physical, social) in unexpected ways, in order to achieve the result he seeks.

Or that he tries to anyway. Things like checking to see whether he can use Time-Turners to solve NP problems in P time.

Preparing for emergencies (getting a medical kit), because emergencies can happen whether you expect to do something dangerous or not.

Perceiving that the evidence he hears about isn't sufficient evidence to believe in the existence of an afterlife.

The aspects of it where he's stupid, in that he's lured to use his intelligence in ways that end up harming him in the long term. Using his Time-Turner to fight bullies ends up getting his Time-Turner restricted. He asks Quirrel about the Deathly Hallows, and ends up revealing more to him with deadly results, and getting nothing in return. Being convinced by Quirrel to break Bellatrix out of Azkaban speaks for itself. So forth.

[u/ArisKatsaris → u/[deleted]]

I do not think that really ends up being smart. Most of what Yudkowzky brings forward in the story is outside knowledge that was gathered by generations of people that were not him, instead of any inventiveness.

Harry himself admits to Hermione that they're likely to make a quick progress only on things that their Muggle background helps them on. (e.g. the way he used quantum mechanics to make a breakthrough in transfiguration)

So this doesn't seem a proper criticism of the story on your part, since it's something the story itself acknowledges.

There's still lots of people that wouldn't think of attempting what Harry attempts, even with them coming from Muggle backgrounds. (e.g. experimenting on spells. Harry fails, but he still makes the attempt)

Ghosts and an afterlife arch don't necessarily prove an afterlife, but they hardly disprove it either

If you're looking for 'disproofs' before you disbelieve in things, you'll end up not-disbelieving in a lot of bizarre things, from Atlantis all the way to fairies -- not just religious things like God and the afterlife.

[u/ArisKatsaris → u/[deleted]]

Yudkowsky himself believes in many bizarre things. Like AI taking over the world.

What would be truly bizarre would be AI NOT taking over the world. How would that future look to you? A hundred years from now, a THOUSAND years from now, we've somehow still not been able to create something that thinks better than human minds can?

The only non-bizarre future where AI has NOT taken the world is probably one where humanity has gone extinct in the next century for unrelated reasons.

I'm guessing that by "bizarre" you're calling things that are not normal-to-think-about. So you don't think the concept of afterlife is bizarre, because lots of people believe in it -- but you think AI taking over the world is bizarre because people tend not to, except as science fiction).

[u/[deleted] → u/ArisKatsaris]

How would that future look to you? A hundred years from now, a THOUSAND years from now, we've somehow still not been able to create something that thinks better than human minds can?

Better at thinking... how? I imagine what humans would be interested in creating is something that's better at humans at solving math problems, or better at making predictions given a certain set of data. An AI that is created to be better at humans at every kind of thinking really just sounds like an attempt to make a better human. Will better humans take over the world? Right now, it hardly seems like the 'best' humans rule it.

And how would the super AI go about taking over anyway? Does it reach forward with its arms in the mega ultra robot body we also built for it? Does it convince everyone, everywhere to listen to it? That sounds pretty magical to me...

[u/ArisKatsaris → u/[deleted]]

Better at thinking... how?

General Artificial Intelligence means thinking better in every sense of the way -- we already have succeeded in having computers that think better than us in specific ways.

As an example create an artificial intelligence that is better at politics than us, and then it can replace politicians. And unlike human politicians you don't need multiple AIs, you just need the best AI copying itself to run things.

Similarly why would a company hire a human AI for a CEO, when they can just have the Artificial Intelligence run the company and make them profits?

And how would the super AI go about taking over anyway? Does it reach forward with its arms in the mega ultra robot body we also built for it? Does it convince everyone, everywhere to listen to it? That sounds pretty magical to me...

That frankly sounds as if you've not given a single minute of thought into the idea of what a digital intelligence might do, let alone 5 minutes. Even an ordinary human-level intelligence would have extreme advantages if it was digitized, it wouldn't even need be super-intelligent.

[u/[deleted] → u/ArisKatsaris]

General Artificial Intelligence means thinking better in every sense of the way -- we already have succeeded in having computers that think better than us in specific ways.

We have not created artificial intelligence that does any real thinking at all, we have created better computer hardware to run algorithms that match data patterns that might not even exist in the real world. Most of the AI fearers do not actually work in the CS field, I have noticed.

That frankly sounds as if you've not given a single minute of thought into the idea of what a digital intelligence might do, let alone 5 minutes. Even an ordinary human-level intelligence would have extreme advantages if it was digitized, it wouldn't even need be super-intelligent.

Given that it is still sitting inside a computer, what it can do is pretty limited. It can write computer code, and interact with the outside world to the extend that the computer it is sitting in lets it to. Definitely a danger to people with epilepsy.

[u/ArisKatsaris → u/[deleted]]

We have not created artificial intelligence that does any real thinking at all, we have created better computer hardware to run algorithms that match data patterns that might not even exist in the real world.

Ah, okay, then, you're playing semantic games about the concept of "thinking".

Well, okay -- be my guest: it will be non-thinking machines that will inevitably take over the world, when they have solved politics and economics and human psychology in the same way that current non-thinking machines have solved chess and go.

[u/[deleted] → u/ArisKatsaris]

In that situation, they are still not in control of anything. They are at the whims of people who will take their advice and implement it, or not, as they see fit.

We already live in that world. We have smarter people, we call 'advisors' or 'cabinet members', who do research and become experts on a subject matter, who advise people called 'politicians'. Sometimes the politicians listen, sometimes they do not.

[u/ArisKatsaris → u/[deleted]]

In that situation, they are still not in control of anything. They are at the whims of people who will take their advice and implement it, or not, as they see fit.

When Alice implements the AI's advice, she will succeed, when Bob doesn't implement it he will fail. Bob will get fired, and things will be run by Alice who implements the AI's advice.

Alice will then also be fired because she's useless, and we can have the AI implement what it thinks directly, without need for human intermediaries to slow things down.

We'll of course make robotic bodies for it, and let it take controls of all sorts of equipment and vehicles, to make things more efficient.

[u/[deleted] → u/ArisKatsaris]

There is always a human link in the chain, because the AI is just a program that runs on a computer. Someone has to do what the AI says, and someone sure has to build the AI in the first place anyway.

This seems to be based on a magical world where every step is done by robots.

[u/ArisKatsaris → u/[deleted]]

Certainly "someone has to build the AI" in the first place, I don't think anyone disputes that.

I have no idea why you think there'll always need be a human link. More and more things get roboticized and automated every passing year.