GLSL with medium precision on large screens

A pixel shader is a function that is evaluated over every pixel that a piece of geometry covers. It outputs the color that that pixel should be colored. Let's consider someone drawing a single textured quad that is the size of the entire screen, using mediump precision (more on this later). In a pixel shader, you can ask a sampler what color a texture is at a given location, using the texture2D() GLSL function, and it will return the color. Let's consider a pixel shader that just outputs whatever this function returns.

When writing GLSL shaders, you have the option to specify the default precision for floating point math. There are three options: lowp, mediump, and highp.

The OpenGL ES Shading language spec version 1.0.17 says that mediump precision requires floats to have a relative precision of 2^(-10) (Section 4.5.2). The manual page for glGetShaderPrecisionFormat describes the meaning of that number: It is the size of a ULP in the range between 1 and 2. Because the exponent in the float is constant between 1 and 2 (set at 0), this means that the mantissa of the float has 10 bits in it, and can therefore take on 1024 values.

Samplers, such as the one described above, have to query the texture within the bounds of 0.0 and 1.0, with 0.0 being the top/left of the texture and 1.0 being the bottom/right of the texture. Let's just pay attention to the range between 0.5 and 1.0 (meaning: the bottom half of the texture). Inside the range of 0.5 to 1.0, the exponent in the float is set at (-1), which means that there are only 1024 different representable values within this range, because the mantissa is 10 bits.

However, when drawing a texture over the entire screen, the sampler is invoked on every pixel. Tablets with high resolutions (such as 1080p) are becoming increasingly common. In a 1080p screen, one dimension of the screen may be around 1920 pixels high. If we pay attention to the bottom half of the screen, it is 1920/2=960 pixels high.

Because 960 is close to 1024, there isn't a good mapping from pixels on the bottom half of the screen to texture coordinates between 0.5 and 1.0. What's more, the sampler has to map these 1024 texture coordinates back to the 960 texels in the texture. It's possible that, if you're drawing a fullscreen texture with mediump precision, you'll have a fair amount of repeated texels being outputted in the bottom/right half of your image.

There's actually a little more to the story here. Many drivers recognize these "pass-through" shaders and optimize them into blitting calls from textures to the framebuffer. Because of that, this kind of bug is only visible on sufficiently complicated shaders, so that the driver doesn't recognize that it's just a pass-through.