YesNoOk
avatar

Anyone interested in ai upscaling? (Read 591 times)

Started by LyonHrt, January 29, 2019, 04:43:24 am
Share this topic:
Anyone interested in ai upscaling?
#1  January 29, 2019, 04:43:24 am
  • **
  • Mugen curse strikes again
Long time no post, but has anyone tried enlarging stages/sprites with ai upscaling? Have to admit not done anything mugen related for years (yes the curse!), but as i've been working on upscaling stages in emulation, and i thought id share it with you guys.
imgur
This is actually an upscale using esrgan an ai powered upscaler.
The screenshots are actually from an older version of the model i've trained (current one below handles dithering better), it was trained using fan art by falcoon, hence the name, the manga109 model is a popular choice, but was hoping to improve on that for stages.
google drive

Hopefully with this, it would be possible to get HD/4K sprites and stages, or at least help.

Thoughts?
Re: Anyone interested in ai upscaling?
#2  January 29, 2019, 05:11:26 am
  • ******
  • 90's Kawaii
  • :thinking:
    • Guatemala
The SFA stage looks pretty good. Iori doesn't look the part imo.

People have tried more rudimentary filters in the past (usually preset filters for image edition software) and while your upscaling method looks way ahead of them, the loss of detail on character sprites is still painful. It works so much better on backgrounds since they are, well, backgrounds. They're meant to be out of focus, less detailed and less animated, so the drawbacks of filtering are severely mitigated and the benefits are so much more obvious.

Do you have any animated sample?
Re: Anyone interested in ai upscaling?
#3  January 29, 2019, 05:35:07 am
  • **
  • Mugen curse strikes again
I don't have an animated sample right now, i did generate a complete animation, but it looked a bit iffy! And was hoping to give it a few more hours/days to train to hopefully improve, before animating, some artefacts i can deal with, if editing doesn't take too long. But i might just to see what it looks like, even if it ends up all over the place.

But yeah, stages look great, some have next to no artefacts example

recently rendered upscales
Re: Anyone interested in ai upscaling?
#4  January 29, 2019, 05:49:51 am
  • ******
  • 90's Kawaii
  • :thinking:
    • Guatemala
Sagat's stage looks perfect, this method really works wonders on the cartoony SFA backgrounds.

Shame for the detail loss on the characters' faces on the kof stage (is that '94 Italy?), but the way the water blends is pretty impressive. Even if your ai doesn't quite get faces right this would be an excellent base for anyone willing to manually edit those details back.

btw did you pick Falcoon out of personal preference, or is it because he's made so much fan art of capcom/snk characters it's easy to get a large sample out of him?
Re: Anyone interested in ai upscaling?
#5  January 29, 2019, 06:04:10 am
  • **
  • Mugen curse strikes again
With falcoon it was a little of both, had a collection from years ago before he joined snk over 1200 images, so had that to hand (also it was a good first choice in training the ai for the snk sprites in particular), if you look you can see the influence in the shading and line work that the ai has picked up on, plus before i did this one, the other choice to download was a dataset based on manga which just lost some details for me and the rest focused on photos, so did one myself.

In an a ideal world, there would be all the in development scans to tell the ai to make this sprite look like this!

And yes you can train on the back of this, use this as a pretrain model, then train manga faces which should then add to it, this is still early days, but hopefully having blury enlarged HD stages will at least be a thing of the past, in the worse case should help as a tool.
Re: Anyone interested in ai upscaling?
#6  January 29, 2019, 06:09:32 am
  • ******
  • 90's Kawaii
  • :thinking:
    • Guatemala
@Bastard Mami: if you're still lurking, check this thread.

(he mentioned being interested in ai based sprite filtering on discord, and might have some feedback for this)
Re: Anyone interested in ai upscaling?
#7  January 29, 2019, 08:59:22 am
  • ***
    • Brazil
    • deathscythe.co.vu
Idk if it's just me, I never liked those "filtered" upscaled images. In stages it looks cool, but certain elements (small ones) might look weird. In characters it's even worse because the image is even smaller. What I like when it comes to HD/hires/upscale is a total redraw of whatever you're upscaling. Of course that's a lot of work, but the result is beautiful. If I had to choose between pixelated sprites and upscaled filtered ones, I'd pick pixelated. Don't mind me, it's just my worthless opinion, some people seem to enjoy it, so... :P
Re: Anyone interested in ai upscaling?
#8  January 29, 2019, 05:13:39 pm
  • **
  • Mugen curse strikes again
No i totally agree, i brought the train wreck that was street fighter 2 HD but choose the original over it any day lol
But this is mugen, i was around when cam rips of stages and sprites from dreamcast/ps2 games were a thing  :lipsrsealed2: there is always a market for this.
Although this isnt a filter, it will actually add detail based on what its learned, so its redrawn by the ai, but its got to be taught and that might be off putting for some.