The best example I can think of for this is Arakune's red effect. (BlazeBlue)
This is my attempt: https://imgur.com/a/Q5JlsYG (Shin Gouki's legs)
Mirror match support: https://imgur.com/a/PKV7YfY
The blinking is caused by changing SprPriority. This wouldn't be a problem in a character made for this effect.
The character I was testing it on.
https://www.dropbox.com/s/verwi6u3ftj0w8c/ShinGouki.zip?dl=0
I got the idea from an old post and it's done similar to how Byakko described.
Same way as the stage stuff. One sprite with additive transparency on top of the character, one sprite with negative transparency behind the character to cancel it out everywhere outside of the character's sprite.
The only issue with the "one sprite on top, one sprite behind" technique is that anyone can walk up to your character and have the flame appear on top of them. You might put your flames + character on one extremity of the sprpriority range and hope that no one uses the same range, or grabs you and forces you into another sprpriority range (a variation of the classic "attachment coded as helper, in a custom gethit animation" issue). You also shouldn't use him in a mirror match. ... That's the part that's going to be a problem.
There are some problems with this. As Byakko said P2 is also effected, though you can't see it very much here. The only solution to fix this is to have the character's SprPriority set to be further back than other characters or use a smaller animation that's bound to the character.
NOTE: The effect must cover the whole screen. Even if its just a solid black background. No effects should be applied in the air file. There are two animations needed, the effect and one that's solid black.
Statedef -2
;=================================================================================
;Inktrebuchet's Mask / Window effect.
;=================================================================================
[state mask inverted]
type = helper
trigger1 = numhelper(6546476) = 0
ID = 6546476
stateno = 6546476
name = "window/mask"
pos = 0,-10
ownpal = 1
[state mask inverted]
type = helper
trigger1 = numhelper(6546478) = 0
ID = 6546478
stateno = 6546478
name = "window/mask"
pos = 0,-10
ownpal = 1
[state mask color correction]
type = helper
trigger1 = numhelper(6546479) = 0
ID = 6546479
stateno = 6546479
name = "window/mask"
pos = 0,-10
ownpal = 1
[State 6546479, 3] ; lighten character
type = PalFX
trigger1 = 1
time = 1
add = helper(6546476),var(0) ,helper(6546476),var(0) ,helper(6546476),var(0)
;Used in testing to set character back.
[State charPriority]
type = SprPriority
trigger1 = 1
value = helper(6546476),var(1) - 3
;=================================================================================
CNS
;=================================================================================
;Inktrebuchet's Mask / Window effect.
;=================================================================================
[statedef 6546476] ;regular
type = A
physics = N
movetype = H
anim = 45245 ;effect full screen (black background)
;======================================================================================================
; config
;======================================================================================================
[State n]
type = VarSet
trigger1 = 1
v = 0
;#####################################################################################################################
;## change the value below to adjust effect.
;## character palettes will need to be adjusted (lightened) to work with this effect.
;## This value is saved to a var to ensure all values match throughout Mask/Window code.
;##
value = 64
;======================================================================================================
[State n] ; used vs other mask / window characters.
type = VarSet
trigger1 = 1
v = 1
value = cond(teamside = 1, 0, 5)
[State 6546476, 0]
type = AssertSpecial
trigger1 = 1
flag = noshadow
[State 6546476, 1] ;effect in front of character
type = SprPriority
trigger1 = 1
value = var(1) - 2
[State 6546476, 2]
type = Trans
trigger1 = 1
trans = addalpha
alpha = var(0), 256
;--------------------------------------
[statedef 6546478] ; inverted
type = S
physics = N
anim = 45245 ;effect full screen (black background)
[State 6546478, 0]
type = AssertSpecial
trigger1 = 1
flag = noshadow
[State 6546478, 1] ;effect behind character
type = SprPriority
trigger1 = numhelper(6546476) = 1
value = helper(6546476),var(1) - 4
[State 6546478, 2]
type = Trans
trigger1 = numhelper(6546476) = 1
trans = addalpha
alpha = helper(6546476),var(0) , 255
[State 6546478, 3] ; invert effect
type = PalFX
trigger1 = 1
time = 1
invertall = 1
;--------------------------------------
[statedef 6546479] ; correct background color
type = S
physics = N
anim = 45246 ; solid black(full screen)
[State 6546479, 0]
type = AssertSpecial
trigger1 = 1
flag = noshadow
[State 6546479, 1] ;in front of effect and character
type = SprPriority
trigger1 = 1
value = helper(6546476),var(1) - 1
[State 6546479, 2]
type = Trans
trigger1 = 1
trans = sub
[State 6546479, 3] ; lighten
type = PalFX
trigger1 = numhelper(6546476) = 1
time = 1
add = helper(6546476),var(0) ,helper(6546476),var(0) ,helper(6546476),var(0)
;================================================================================
