Shader scale offset problem?



  • I've been trying to write a shader to scale up pixels by a given factor. I've seen examples of this where it's as simple as doing the following, where the scale value means the resulting image is 4 times the size.

    float scale = 4.0;
    gl_FragColor = flixel_texture2D(bitmap, vec2(openfl_TextureCoordv.x/scale, openfl_TextureCoordv.y/scale));
    

    The following is a screen shot of 4 squares, each 4 pixels wide and high drawn to a state and then the filter applied to the camera. The fragment code almost works as expected in that the resulting squares are enlarged by a factor of 4 although the edges are blurred.

    fuzzy scale

    I've theorized that the blur is caused by the coordinates in the above fragment code being calculated as sub pixel positions. For example, imagining each gl_FragColor pixel on the x axis for 4, 5, 6, 7 it's enlarging the pixels from at positions that would be at 1.0, 1.25, 1.5, 1.75

    If I quantize the positions using the following code

    float scale = 4.0;
    
    // get the distance between each pixel in terms of gl co-ordinates
    float incrementX = 1.0 / openfl_TextureSize.x;
    float incrementY = 1.0 / openfl_TextureSize.y;
    
    // scale the co-ordinates 
    float scaledX = openfl_TextureCoordv.x / scale;
    float scaledY = openfl_TextureCoordv.y / scale;
    
    // round scaled co-ordinates down to nearest increment
    float quantizedX = scaledX - mod(scaledX, incrementX);
    float quantizedY = scaledY - mod(scaledY, incrementY);
    
    gl_FragColor = flixel_texture2D(bitmap, vec2(quantizedX, quantizedY));
    

    Then the resulting image is closer to what I'm looking for

    quantized scale

    The result is a 16 pixel square for each of the original 4 pixel squares as expected, but each one has an extra 4 pixels added to the edge.

    Furthermore, if I put a background that consisted of single pixel stripes, as shown here magnified (in gimp) by a factor of 4

    bg stripes

    Then run the shader on it, it looks like this, also magnified by a factor of 4 so we can see what's going on.

    Only the first stripe is scaled properly.

    bg stripes scaled

    It does show that the extra edge pixels have an alpha less than 1. So I thought I could test for that in the shader and not render the pixel unless the referenced pixel has an alpha of 1. This isn't ideal in practice because one might want to keep the transparency but I just wanted to test it out the theory.

    The following code adds a check to see if the reference pixel does not have alpha 1 then draw a white pixel with alpha 1, otherwise use the reference pixel.

    float scale = 4.0;
    		
    // get the distance between each pixel in terms of gl co-ordinates
    float incrementX = 1.0 / openfl_TextureSize.x;
    float incrementY = 1.0 / openfl_TextureSize.y;
    
    // scale the co-ordinates 
    float scaledX = openfl_TextureCoordv.x / scale;
    float scaledY = openfl_TextureCoordv.y / scale;
    
    // round scaled co-ordinates down to nearest increment
    float quantizedX = scaledX - mod(scaledX, incrementX);
    float quantizedY = scaledY - mod(scaledY, incrementY);
    
    // sample the reference pixel
    vec4 sample =  flixel_texture2D(bitmap, vec2(quantizedX, quantizedY));
    
    // does the pixel have apha less than 1?
    if(sample.a < 1.0){
        // then render a white pixel with alpha 1
        gl_FragColor = vec4(1,1,1,1);
    }
    else{
        // or use the reference pixel
        gl_FragColor = sample;
    }
    

    Unfortunately it still draws the the 'edge' pixels. Does anyone know why that is happening?



  • Man I wish I knew enough about shaders to help you out, I'm learning but I'm not quite there yet. Have you tried asking your question on the Haxe Discord server?

    https://discord.gg/Tw4fDv

    Maybe you can add a link to this question on there, it's much more active than the forums.

    Also, I feel like there are easier ways to scale a sprite than to use a shader. Well as in if you had a large sprite you could scale it down easily but I'm not sure about scaping up a small sprite 🤔



  • I will share a link in discord, thanks.

    It certainly is possible to scale up using built in haxeflixel functionality but unfortunately there are some compromises to that approach. I added a couple of bits to the scale modes demo from flixel-demos to test things out. It's published here - https://jobf.github.io/haxeflixel-html5-pixel-perfect/

    Using pixel perfect position or pixel perfect render (or both) it is possible to get close to the desired effect such that sub pixels are avoided when moving sprites on the horizontal and vertical axis. Notice the jagged movement of the sprite moving diagonally when either of those options are set.

    However when rotating a scaled sprite there is still some extra interpolation going on. It is possible to have the scaling work the way I would like which does work perfectly when targetting windows desktop and probably other desktop targets too though I've not tested. The set up is shown here - https://github.com/AustinEast/haxeflixel-pixel-perfect - unfortunately this 'hack' does not behave the same on HTML5.

    Also, apparently the pixel perfect settings are not always perfect. There's an example here - http://forum.haxeflixel.com/topic/953/pixelperfectrender-is-not-perfect-at-all/

    So I was hoping webgl would be the way forwards. Except something funny is going on, either in the calculations I've used or somewhere in the base shader code.



  • I've put a repo up with the code I was using to test the shader out here - https://github.com/jobf/haxeflixel-pixel-scaling-shader



  • Think I've solved it. Demo here, press f to toggle the shader off and on - https://jobf.github.io/haxeflixel-pixel-scaling-shader/

    I ended up using some code from https://csantosbh.wordpress.com/2014/01/25/manual-texture-filtering-for-pixelated-games-in-webgl/

    I'll do a bit more testing and change this so that the scale amount can be passed in.



  • Nice, glad you figured it out. I'm curious as to why to used texture2D() and not flixel_texture2D() on line 20?

    https://github.com/jobf/haxeflixel-pixel-scaling-shader/blob/master/source/PixelScaleShader.hx

    Also for line 16 in your shader shouldn't this work as well because of swizzling?

    ec2 alpha = vec2(0.07);
    


  • Choosing texture2D over flixel_texture2D wasn't something I made a specific call on previously. While experimenting I used them both and didn't notice any change. It's a good question so I had a quick look around flixel source to see what the difference is. Couldn't find much but there is mention of it in this blog post, as follows https://haxeflixel.com/documentation/upgrade-guide-4-0-0/

    using flixel_texture2D() in per-sprite shaders, the alpha and color transforms of a FlxSprite are already applied on the returned color

    I'm only interested in applying the shader to the entire scene with this technique so may well leave it using the openfl texture2d. It might be more efficient even, though probably not to a perceptible level.

    As for the swizzling, vec2(0.7) works too as you suggest. There's no particular reason I've not done that here but I do find you sacrifice readability doing it, without much gain. In this case it's just a leftover from rounds of experimentation, breaking and fixing the code. It's hard to debug this stuff and in general I've found being explicit is generally more useful as there's less to potentially go wrong.

    I think there is swizzling in the code I got from the blog post at csantosbh.wordpress.com but in my opinion it does not help with understanding what is going on. I actually started rewriting it to be more explicit but that was more effort than worth since the copy pasta got me where I want to be for now.

    I have since changed to use a smaller value for the alpha variable because I found that if you scale to extreme sizes 0.7 isn't enough.


Log in to reply
 

Looks like your connection to HaxeFlixel was lost, please wait while we try to reconnect.