Just a couple of points that I feel need to be said:
Top Time Trial strategies are the opposite, where top human strategies do approach perfection, so even though the complexity of Time Trial is lower, that of its top strategies is far higher, so AIs don't discover unassisted the long-range strategies people do, like creative shortcuts and lap skips.
If a script is iterating on a top strategy then it may under certain conditions outperform the top strategy, but I wouldn't call that an artificial intelligence competing with a top player, I'd call it a top player competing with a top player where one side of this self-competition is assisted by a tool.
People don't discover the long range strategies unassisted either. Once one person comes up with a breakthrough that information is available to everyone else. Everyone else copies and optimises it. I think you're suggesting pitting one AI against all players, rather than one AI system versus the realistic performance of one human. I wouldn't be surprised if an AI could outperform the entirety of humanity starting from scratch with zero help anyway, but that is a different test.
Also, every person or institution has limited resources and motivation. If that in some way discredits the Mario Kart AI, it does the same for the fighter pilot AI developed under similar conditions, and thus discredits the stated basis for which you think MK64 Time Trials will soon be within reach of AI.
One would reasonably expect significantly more resources and motivation to apply to a military application like fighter pilot training (and making fighter pilots necessarily redundant) than to playing Mario Kart. When I say 'limited resources and motivation' I mean relative to a research goal that's actually useful.
Just a couple of points that I feel need to be said:
> Top Time Trial strategies are the opposite, where top human strategies do approach perfection, so even though the complexity of Time Trial is lower, that of its top strategies is far higher, so AIs don't discover unassisted the long-range strategies people do, like creative shortcuts and lap skips.
> If a script is iterating on a top strategy then it may under certain conditions outperform the top strategy, but I wouldn't call that an artificial intelligence competing with a top player, I'd call it a top player competing with a top player where one side of this self-competition is assisted by a tool.
People don't discover the long range strategies unassisted either. Once one person comes up with a breakthrough that information is available to everyone else. Everyone else copies and optimises it. I think you're suggesting pitting one AI against all players, rather than one AI system versus the realistic performance of one human. I wouldn't be surprised if an AI could outperform the entirety of humanity starting from scratch with zero help anyway, but that is a different test.
> Also, every person or institution has limited resources and motivation. If that in some way discredits the Mario Kart AI, it does the same for the fighter pilot AI developed under similar conditions, and thus discredits the stated basis for which you think MK64 Time Trials will soon be within reach of AI.
One would reasonably expect significantly more resources and motivation to apply to a military application like fighter pilot training (and making fighter pilots necessarily redundant) than to playing Mario Kart. When I say 'limited resources and motivation' I mean relative to a research goal that's actually useful.
I'd also add that the control system and general environment in terms of what is possible within Mario Kart favours humans. If the objective was to create an AI that could fly a plane and shoot targets, or simply drive around a track and finish a race in Mario Kart, I'd say Mario Kart would be easier.
An AI is much more capable when it comes to reaction times and decision making. I do not know what kind of controls were used in the flight simulation, but I'm willing to bet it was more advanced than an N64 controller. The flight sim probably did not run at 30FPS either. Whilst both of these things suggest that it would be harder to create a basic flying AI than an MK64 one, the opposite is true when the question is about high level performance. There is not a large amount of tactical decision making in Mario Kart, and the inputs are fairly simple. In other words, the two largest advantages an AI may have over a human are barely applicable in Mario Kart speedrunning.
I'd also add that the control system and general environment in terms of what is possible within Mario Kart favours humans. If the objective was to create an AI that could fly a plane and shoot targets, or simply drive around a track and finish a race in Mario Kart, I'd say Mario Kart would be easier.
An AI is much more capable when it comes to reaction times and decision making. I do not know what kind of controls were used in the flight simulation, but I'm willing to bet it was more advanced than an N64 controller. The flight sim probably did not run at 30FPS either. Whilst both of these things suggest that it would be harder to create a basic flying AI than an MK64 one, the opposite is true when the question is about high level performance. There is not a large amount of tactical decision making in Mario Kart, and the inputs are fairly simple. In other words, the two largest advantages an AI may have over a human are barely applicable in Mario Kart speedrunning.
I'm going to do some more research and write up a post on this.
One thing I would like to mention for the moment is that people do find these strategies by themselves. Yes, there is also a collaborative effort across different games to pool together the best strategies, but there are some people who find a lot of stuff by themselves and then share that information. I think everyone who is capable of playing Mario Kart is also capable of finding a skip by themselves if they keep attempting different strategies and learning how the game mechanics work. AI does not glitch hunt, at all, but any human can.
Edit: I would also point a couple of other things out. The strategies used by fighter pilots do not come from the individual either. These techniques have been pooled together over decades. If you consider this, you could argue that the individual is actually more important in MK64 in relation to finding skips and whatnot.
I think another point to consider is that AI has a natural advantage over humans in any kind of combat situation. AI has a completely different idea of what risk management is when it comes to something that is life threatening. This exercise was a simulation, but the experience and training of a fighter pilot is always going to be based around staying alive as the most important priority, and that is still a factor even in this exercise.
I'm going to do some more research and write up a post on this.
One thing I would like to mention for the moment is that people do find these strategies by themselves. Yes, there is also a collaborative effort across different games to pool together the best strategies, but there are some people who find a lot of stuff by themselves and then share that information. I think everyone who is capable of playing Mario Kart is also capable of finding a skip by themselves if they keep attempting different strategies and learning how the game mechanics work. AI does not glitch hunt, at all, but any human can.
Edit: I would also point a couple of other things out. The strategies used by fighter pilots do not come from the individual either. These techniques have been pooled together over decades. If you consider this, you could argue that the individual is actually more important in MK64 in relation to finding skips and whatnot.
I think another point to consider is that AI has a natural advantage over humans in any kind of combat situation. AI has a completely different idea of what risk management is when it comes to something that is life threatening. This exercise was a simulation, but the experience and training of a fighter pilot is always going to be based around staying alive as the most important priority, and that is still a factor even in this exercise.
17 comments
Just a couple of points that I feel need to be said:
People don't discover the long range strategies unassisted either. Once one person comes up with a breakthrough that information is available to everyone else. Everyone else copies and optimises it. I think you're suggesting pitting one AI against all players, rather than one AI system versus the realistic performance of one human. I wouldn't be surprised if an AI could outperform the entirety of humanity starting from scratch with zero help anyway, but that is a different test.
One would reasonably expect significantly more resources and motivation to apply to a military application like fighter pilot training (and making fighter pilots necessarily redundant) than to playing Mario Kart. When I say 'limited resources and motivation' I mean relative to a research goal that's actually useful.
I'd also add that the control system and general environment in terms of what is possible within Mario Kart favours humans. If the objective was to create an AI that could fly a plane and shoot targets, or simply drive around a track and finish a race in Mario Kart, I'd say Mario Kart would be easier.
An AI is much more capable when it comes to reaction times and decision making. I do not know what kind of controls were used in the flight simulation, but I'm willing to bet it was more advanced than an N64 controller. The flight sim probably did not run at 30FPS either. Whilst both of these things suggest that it would be harder to create a basic flying AI than an MK64 one, the opposite is true when the question is about high level performance. There is not a large amount of tactical decision making in Mario Kart, and the inputs are fairly simple. In other words, the two largest advantages an AI may have over a human are barely applicable in Mario Kart speedrunning.
I'm going to do some more research and write up a post on this.
One thing I would like to mention for the moment is that people do find these strategies by themselves. Yes, there is also a collaborative effort across different games to pool together the best strategies, but there are some people who find a lot of stuff by themselves and then share that information. I think everyone who is capable of playing Mario Kart is also capable of finding a skip by themselves if they keep attempting different strategies and learning how the game mechanics work. AI does not glitch hunt, at all, but any human can.
Edit: I would also point a couple of other things out. The strategies used by fighter pilots do not come from the individual either. These techniques have been pooled together over decades. If you consider this, you could argue that the individual is actually more important in MK64 in relation to finding skips and whatnot.
I think another point to consider is that AI has a natural advantage over humans in any kind of combat situation. AI has a completely different idea of what risk management is when it comes to something that is life threatening. This exercise was a simulation, but the experience and training of a fighter pilot is always going to be based around staying alive as the most important priority, and that is still a factor even in this exercise.