YesNoOk
avatar

IKEMEN and Deep Learning? (Read 12075 times)

Started by NEONSHADOWF13, June 18, 2018, 05:24:02 am
Share this topic:
IKEMEN and Deep Learning?
#1  June 18, 2018, 05:24:02 am
  • avatar
    • USA
One of the projects I found interesting for DOOM 1993 is ViZDOOM. (http://vizdoom.cs.put.edu.pl/)

It's a platform plugin for DOOM that allows people to create deep learning computer intelligence. There is even a ViZDOOM tournament by the same name where people send the bot they programmed and trained to fight each other in death match to see who built and trained the best bot.

Now with Ikemen we have a very powerful engine that can run characters created for Mugen. I wonder what it would be like if someone created a plugin or script or something where people could do the same for Ikemen that they do for ViZDOOM.

You'd essentially then program a deep learning bot into being and then train it match after match. Then there could be a competition where people would send the bots against each other.

Currently, computer intelligence is script based. But, deep learning could allow for more naturally evolving intelligence that could compete in ways that are less predictable and more fluid, given proper programming and enough quality training.

Some agreed upon characters would have to be chosen so that it's a fair fight for everyone. But, I could imagine a lot of fun potential here.

The plugin or script or whatever would need to give access to the computer intelligence to read some key elements of the game. It will need to be able to see its own and its opponents health bars, special bars, the round timer, hit boxes, hurt boxes, collision boxes, likely the floor and walls, and maybe match outcomes. Win state would be set as winning the round as well as trying to do so more efficiently. It could thus look at what it did and what methods got it to the win state quickest in relation to the round timer. It would need access, obviously, to all input controls as well (movement, A, B, C, X, Y, Z, and maybe Start.)

The program would start out knowing nothing, and they build off of each experience it tried as it learns to try random button inputs and figures out what works against opponents. It might be wise to run the first many, many matches against current bots to train against until its ready to go against humans. Or, it could be interesting to run it against humans for the entire training. One idea that might be interesting is if there is some way to allow people to connect to it online and play against it for an endless stream of human opponents to train against. But, that would require setting up with the same Ikemen set up and people being willing to fight the deep learning machine. That said, it would not very for interesting to fight against it for the first several thousands of matches, I would think. So, getting humans to fight it online for the start may yield few results.

Maybe possible positive outcomes. It might be possible once the bot is trained well enough to create bot intelligent packs to replace the standard scripting with. This could allow for more dynamic single player matches. So, maybe strong computers could train the bots until they're ready, and maybe there is some way to output the evolution tree to run in a non-learning mode with what it did learn on lower spec computers.

This could be a ground breaking idea. So, to me, a lot of the old school fighters I played as a kid in the 90's got to the point where I could defeat the computer because I could predict its scripting. You can learn how these bots react to stimuli and how to use that to goat them into positioning into bad spots. Some scripting took this into account and scripted deeper to give the bots a way to handle some of this. But, I realized if I pushed forward to break its root, it usually didn't have any scripted responses for countering two or three moves in. I always thought, "Why doesn't it just try something new instead of just doing the same predictable thing every single time I throw a fire ball at it?"

Difficulty can be increased through giving the computer more health / "shield" and or the player less health / "shield." But, that's not making the computer any better at playing the game. That's only making the computer more difficult by stacking the deck in its favor. The same is true for creating boss characters that have a special move for every occasion. They are better suited to counter any type of attack, which makes them more difficult, but they are not learning how to play the game any better. Again, they're just getting the deck stacked more in their favor. Another thing is adding randomness. Perhaps the characters has several different sets of combos and such to link together. Several "roots" that would need to be broken. If one root fails to produce results in defeating its opponent or defending itself, it randomly switched to another set of roots. While this varies combat more, and does give the computer more options in battle, it's still not really learning how to play the game better. It's playing better based on randomly picking a different technique. But, it's not learning why it should pick one root over another. It's  just lucking out that switching up throws the opponent off, unless the opponent has learned all the techniques it has; and then it makes little difference for when it switches.

A true deep learning machine should eventually start to build its own roots of combos and tricks to try against its opponents. It should also start to learn when to switch between different roots based on elements such as how time it has left in the round, how much health it has left, how much super bar it has, how much health its opponent has left, and how much super its opponent has. It might learn, for example, how to build five different defensive roots and to try to pick one of those when it's low on health. Or, as another example, it might learn that if it has a lot of health and the timer is low to keep using zoning moves like projectiles to run down the clock to achieve win condition.

If its opponent keeps throwing the same moves at it, it should eventually learn how to try new things until it develops a root system of commands to use to get over the hurdle. For instance, maybe after getting hit in the face with a fire ball for the 1004th time it might realize it should try jumping towards the opponent and then kick. Once it realizes that this both spared it the health loss, while also lowering the opponents health, it could then bookmark that element of what just happened as a root (set of actions) that are good for handling the situation of a fire ball. If this root is countered enough, it could try to develop a different root, maybe jump in place and land in a punch. Maybe then try randomizing between the two roots to see which one yields the statistically best results over time. Maybe it learns that it's most optimal for it to jump towards the opponent and kick 25% of the time but do a standing jump and fall down into a punch 75% of the time. These are just thoughts and examples.

This would provide players with more interesting fight in single player. I'm decent at fighting games. I'm no expert, but I'm decent. What I have noticed over the years is that most fighting games don't have very smart bots. Rather, they just ramp up how many multiple hit combos they're allowed to throw against you again and again on higher difficulties. This would be an opportunity to develop a fighting game intelligence that could grow organically to actually learn how to play better, rather thank simply be given a deck that's stacked higher in its favor.

Concerns. The person who wrote the Super Mario World deep learning bot noted that this type of deep learning performed poorly on his test at an old fighting game. Fighting games may be too complex for this type of deep learning to pick up. I am thinking that if this type of deep learning was allowed to train long enough, it would eventually pick up the game.

Here are some links to show you what deep learning looks like and with very good explanations for how it works. The first is the best for understanding what I'm talking about.





The other option is to have the computer learn from other people playing and try to mimic their play-styles.



Here is what ViZDOOM looks like.

Last Edit: June 18, 2018, 05:43:41 am by NEONSHADOWF13
Re: IKEMEN and Deep Learning?
#2  June 18, 2018, 07:35:04 am
  • avatar
  • **
  • If you rip sprites i´ll rip your code
    • Mexico
    • www.m3xweb.260mb.com
Please, stop; I am happy with the enthusiasm you show with all this, what's you propose with the licenses and this theory of auto learning is fine.

There is no magic button that makes everything run out of nothing, nor people who will do it for you, everything takes time and effort and this for many is a hobby because we have things to do in our daily lives.

If you really want all that, then put your batteries on and why don't you better help implement the things you want by learning how the engine works?

Or even learn something that you feel comfortable with, according to your needs, like anything from sdl, opengl, assembler ,c ,c++ ,c # ,python ,java ,lua , golang or some game oriented libraries like allegro, bennugd; see the codes that people have made for some engines and do yours.

What's more, you could even make it libre licenced as you dreamed it and sell or give away all your knowledge.



There is no knowledge that is not power
Last Edit: June 18, 2018, 07:54:33 am by MangeX
Re: IKEMEN and Deep Learning?
#3  June 18, 2018, 09:01:28 am
  • **
Interesting videos. Technically it’s possible (source is open, duh), but realistically I can't think about more complicated task. Unless those DooM geniuses will suddenly show interest in ikemen I wouldn't count on machine learning methods feature for this engine.
Re: IKEMEN and Deep Learning?
#4  June 18, 2018, 06:48:12 pm
  • avatar
    • USA
Interesting videos. Technically it’s possible (source is open, duh), but realistically I can't think about more complicated task. Unless those DooM geniuses will suddenly show interest in ikemen I wouldn't count on machine learning methods feature for this engine.

True. It's not a feature request. It's more of a thought I had that would be interesting and didn't think anyone has thrown it out there before.

As for a basic platform (and not being a programmer so probably naive lol) I wouldn't imagine having so the bot could plug into the game itself would be very difficult to do. It would essentially be some sort of plugin or script that gives the bot access to input keys, and telling it where in a live match to find values for things such as health, round timer, hit boxes, hurt boxes, and collision boxes, and maybe super meter. That way any bot developed could see what it's doing in-game.

Actually developing a bot would take a lot of time; depending on how complex it would be. Even setting up such a thing could take a while. I looked into setting up deep dream myself and there are a lot of programs that need to be installed and directed to work with each other. Once a setup was created, it would be too hard to make the bot do like Mari I/O and just press random input buttons until it actually succeeded in doing damage to an opponent. Then if it's set to look at opponent damage and remaining round time. That would be the simplest way to set it up.

But, I imagine that wouldn't produce very interesting results. It would take a very large amount of training time and would need a lot more fine tuning in looking at other aspects of the game to truly have a chance at creating a bot that could do anything competently.

All that said, the idea is something interesting that I don't think has really been done before. Teaching a bot to win at Dota, Super Mario World, and Super Mario Kart is a far more straight forward task than teaching the nuance of a fighting game. There are both far more button inputs and far more facets to consider to achieve victory. Such as when to play defensively, when to play offensively, when to go for a combo, when to stand back and zone with projectiles, that I'm not even sure how to go about priming a bot to learn.

Please, stop; I am happy with the enthusiasm you show with all this, what's you propose with the licenses and this theory of auto learning is fine.

There is no magic button that makes everything run out of nothing, nor people who will do it for you, everything takes time and effort and this for many is a hobby because we have things to do in our daily lives.

If you really want all that, then put your batteries on and why don't you better help implement the things you want by learning how the engine works?

Or even learn something that you feel comfortable with, according to your needs, like anything from sdl, opengl, assembler ,c ,c++ ,c # ,python ,java ,lua , golang or some game oriented libraries like allegro, bennugd; see the codes that people have made for some engines and do yours.

What's more, you could even make it libre licenced as you dreamed it and sell or give away all your knowledge.

Not sure why I should stop. The point of a forum is to discuss ideas and the point of having open source code is that we can do things now that we couldn't before.

I understand that these things don't pop out of nowhere and that it takes time to code. But if no one talks about them then who's going to think of doing them? To my knowledge, no one has even floated the idea before.

I don't know how to program myself. Have tried to teach myself online a few times but havn't grasped it yet. I can do basic game design in engines like Game Maker and have made many demo projects such as platform games, top down driving games, a simple DDR game and a simple 2D fighting game. I have also created some maps and done a very small amount of scripting in GZDOOM.

The key reason I keep bringing up the license is I'd like to know what exactly the engine is under before I invest the time in finally learning how to program and try to do anything interesting, or hire someone freelance to develop a simple script to allow a bot access to the information that it would need so that maybe I can hook up one of the freely available bots already in existence to it to experiment, only to find out later that the code is not released in the context that I originally assumed.

I'm trying to think of the possibilities and how much time I want to dedicate to it. Otherwise, it might just be more useful to focus on GZDOOM because I have at least two project ideas I'd like to implement over there and its scripting isn't too difficult to grasp and I already learned close to enough from the tutorials and already available resources that I think I could mock up a demo of my projects in a month or two and then try expanding from there. So, it's really more of trying to figure out where to focus my time and what I want to dedicate myself to learning; GZDOOM scripting or lua.

I'd like to try to implement Friday the 13Th the game or Dead By Daylight as a sort off-brand mods in GZDOOM, maybe get them working in Zandronum considering it's based on the former. DBD is basically power five generators and the exit door opens. I already found a Youtube tutorial that teaches me how to create variables and then have the script check that all are true. So, placing five generators and not having the door to the exit button open until all go from off (false) to on (true) wouldn't be too hard to figure out from that. For Friday the 13Th, there are already mods out there for drivable cars and such and so the idea shouldn't be too hard to plop some mods in and make work.

My interest here is that I finally found a fighting engine that can support Mugen content that works well, has online play, and is presumed to be BSD license; though no license is actually included in source anywhere. At least in GO and GO Plus. I talked with someone over at GO Plus who says he'll try to contact the original developer for clarification. If the intent is that it's BSD, it really should be plainly listed for everyone to see so that there are no questions or problems later on. Otherwise, technically, at least in the US (so I don't know how that works with international copyright) the copyright of projects without a license listed are per-assumed to reside with their original author, regardless of whether or not they specifically copyrighted it or not. It does not simply fall into being public domain just because the source code is openly visible.

So, on a technical level, these engines are currently proprietary software that belong to Suehiro, that happen to have the source code visible, unless or until Suehiro and those maintaining derivative works decide to actually place a license in the source code or on the repo listing page somewhere that explicitly states otherwise.

That's not a project that I care to dedicate any real time learning to do anything with. What license would I even legally be able to place derivative works under? Usually it's the same license as the original project... but there is none. It's not really public domain, because for that he'd have to explicitly say it is public domain. He doesn't explicitly say anything.

It becomes a potential rats nest of problems if Ikemen ever gets popular and people ever actually start making a large amount of mods/new features/derivative works for it. Even now, there are several derivative works people have made. Are those all expected to fall in line and be under the same license as the original two engines (Ikemen and Ikemen GO) should Suehiro pick a specific license for them? I would assume so, but who's going to go knocking on the door of each project and make them do so? They might say, "Hey, I got this code without a license before I put my work into it. Why should I update to the newer code and fall in line if I can just keep using the assumed public domain code I already worked off of and pick whatever license I want?" What happens if one of these derivatives adds a feature and becomes more popular than Ikemen or Ikemen GO before a license is put in place and everyone decides to develop that project instead and that project refuses to fall in line?

This is why a project should have a clear license from day one, so that this situation doesn't spin totally out of control. It regulates the engine to being able to only be used for a hobby project, as it becomes a nightmare to figure out what can and can't be done with it.

I'm going to give it a week and then contact the person from Ikemen Go Plus again to see if any progress has been made. If none has been made, I'm most likely just going to abandon Ikemen and Mugen again and just return focus back to GZDOOM. There is no point, for me at least, in even trying to learn this if there is never going to be a clear answer to the licensing situation. As the person already stated, there is no license listed in Ikemen Go Plus because there is no license listed in Ikemen Go. They probably can't legally place a license there themselves without running into potential legal problems later on if Suehiro didn't intend it to be licensed that way and ever comes back and says anything.

I can't just ask Suehiro myself because my Japanese from high school is very broken and I'm not sure I could even articulate my question clearly or clearly understand the response he might send back.

So, I will likely just stop in a week or so and go away for another three to five years because I don't have high hopes for getting this resolved and what I'm receiving from most people here is that they see this engine as nothing more than a hobby anyway. So much so, that development is so unprofessional that there isn't even a license attached. The basic thing that 99% of people who take their project seriously in any way at all attach day one because that's the proper thing to do. It's a duh.

To me, it would be the same as sinking my time into Mugen. The reason I left Mugen for years at a time is because it's proprietary and, on a technical level, Ikemen is equivalently the same. In fact, it might be more damaging because it's perceived as 'open source' even though it's technically just 'code available, no real license.' As such, the people who might have gone off to make an 'open source' Mugen might just waste their time on Ikemen instead and we may never see a properly licensed Free Software / Open Source Mugen Clone because of Ikemen.
Last Edit: June 18, 2018, 07:20:16 pm by NEONSHADOWF13
Re: IKEMEN and Deep Learning?
#5  June 18, 2018, 08:38:09 pm
  • avatar
  • **
  • If you rip sprites i´ll rip your code
    • Mexico
    • www.m3xweb.260mb.com
The problem with Ikemen and others is´nt the license in which they are made cause you can make a fight engine using some development tool;
every mugen clone ,even "libre" if you have the possibility to open and use the files originally created by elecbyte, obviously will be in conflict with the same license they proposed, unless the license only refers to the executable program itself and not to do what you want with the packed files of characters and others.

Ikemen is not the only current engine that can read mugen files natively for implementation, it is also as commented in the other tread the dolmexica engine, with the one you can contact directly ,ask and even help to program.

As I commented, the last instance is that YOU make a motor BASED also or inclusive with mugen files; the original made by elecbyte used the allegro libraries that still exist and are far used by many games.

Game maker is a collection of proprietary libraries to make games and is paid, most programs and games are made using the programming language C, if you really want to dedicate time to something that can benefit you economically,and mentally then why you do not intent to make a derivative of mugen, based on the already made and with exactly the bases what do you want.

All mugen clones were made by investigating how that closed source engine worked, so you could still investigate how derivatives that are already open source work; practically the difficult part is already done, which is the investigation.

I do not want to seem rude, but sometimes you give me to understand that you want something made to your measure in which you can contribute little and get some kind of profit, in this case it would seem that monetary at the cost of something free and libre.

It would be a bit unfair and even hypocritical that if you are not answered about the license you leave and after some time it would be uncomfortable, that you will return, when everything is complete and well programmed so that you only take it and use it at your whim without have contributed nothing.

If you were really interested, you would take what is there and adapt it in whatever way possible to what you are looking for; the good thing is that you have already left ideas that we can use, so when you retire you can do it quietly, knowing that you left something interesting.

For my part I thank you for the license and the neuronal intelligence, I learned a lot from that and I will try to implement it while I learn to program and use ikemen as a base, because like you I got into this project recently because I'm interested and it's something I would have wanted to do; there are things that are missing and if there is no one to program or implement them, there is no other option but to learn.

With everything you said about the licenses and others, if there really is a problem, then the least is to make another derivative; The good thing about learning something is that you can apply it; in fact what I'm doing from implementing sdl to ikemen go is practically writing a new engine since the current one is purely opengl and the code changes a lot.

So anything that happens we will get a compatible engine, the same but different and if there is a problem with licenses, I'll do a libre version in C++ with mandatory libre opening manipulation of source code in any derivative copy, whether free or paid.
There is no knowledge that is not power
Last Edit: June 18, 2018, 09:03:56 pm by MangeX
Re: IKEMEN and Deep Learning?
#6  June 19, 2018, 02:35:59 am
  • ****
  • BAD VIBES
  • FOREVER
    • Puerto Rico
    • Skype - TheFclass97
Please keep this sort of thing in IKEMEN discussion and not releases. Thank you and best of luck.
Re: IKEMEN and Deep Learning?
#7  June 19, 2018, 04:02:28 pm
  • ******
  • [E]
    • Mexico

  • Online
This is a fun thread.
I implemented genetic algorythm in my engine. It still needs lot of fine tunning, not only so it can "learn better", but also, so ti can run properly at 60 fps.