TsundereStation/data/shaders/vulkan/batch_textured_nearest.frag.glsl
LibretroAdmin 9dcfea435a common/vulkan, core: pre-bake batch FS textured Nearest slice (48 blobs)
Patch 14 in the glslang-elimination series. Second slice of the
batch FS conversion: the textured + Nearest-filter path
(texture_mode != Disabled AND m_texture_filter == Nearest). The
runtime decision in GetBatchFragmentShader now routes both the
untextured slice (patch 13) and this slice through the embedded
blob set; only textured non-Nearest still goes through
shadergen + glslang.

Five SPIR-V-structural axes (48 blobs total = 3 x 2 x 2 x 2 x 2):

  - Input interpolation qualifier (none / centroid / sample). 3.
  - Color input perspective (standard / noperspective). 2.
  - Dual-source output (1 vs 2 outputs). 2.
  - PGXP depth output (writes gl_FragDepth or omits it). 2.
  - UV limits (without / with v_uv_limits flat input). 2.

The UV_LIMITS axis is new vs the untextured slice - it controls
whether v_uv_limits is declared as a per-vertex flat input on the
FS, matching the per-session m_using_uv_limits choice that already
drives the matching batch VS structural pick. Without UV limits the
FS skips the per-sample clamp.

Three new per-call specialisation constants relative to the
untextured slice cover the textured texture_mode encoding:

  id = 107 PALETTE_4_BIT  (bool, actual_texture_mode == 0)
  id = 108 PALETTE_8_BIT  (bool, actual_texture_mode == 1)
  id = 109 RAW_TEXTURE    (bool, RawTextureBit set in texture_mode)

PALETTE = PALETTE_4_BIT || PALETTE_8_BIT is derived inside the
shader. The texture_mode bit layout is described in
src/core/gpu_types.h; the C++ side derives the three boolean spec
const values from the per-call texture_mode in GetBatchPipeline.
Untextured blobs do not declare 107-109; passing them is harmless
per the Vulkan spec (unrecognised spec const IDs in the supplied
VkSpecializationInfo are silently dropped).

The shader body covers the helpers the shadergen path inlines:
ApplyTextureWindow, ApplyUpscaledTextureWindow, FloatToIntegerCoords,
RGBA8ToRGBA5551, and SampleFromVRAM. SampleFromVRAM splits on the
derived PALETTE flag and, within the palette branch, on
PALETTE_4_BIT vs PALETTE_8_BIT to do the correct sub-texel address
arithmetic. fixYCoord on Vulkan is the identity function so we use
the coord directly. The Vulkan-only API_VULKAN-y-flip in the VS
already accounts for sample-space orientation.

Output and discard logic mirrors the shadergen TEXTURED branches:
the no-TRANSPARENCY case writes o_col0 (and o_col1 when DUAL_SOURCE)
unconditionally; the TRANSPARENCY case branches on the texel's
semitransparency bit and applies the TRANSPARENCY_ONLY_OPAQUE /
TRANSPARENCY_ONLY_TRANSPARENT per-texel discard the way the
shadergen does.

Helper-side index encoding for the 48-blob array, matching
GetBatchTexturedNearestFragmentShaderBlob in embedded_shaders.cpp:

  interp = per_sample_shading ? 2 : (msaa ? 1 : 0)        (0..2)
  persp  = noperspective_color ? 1 : 0                    (0..1)
  dual   = dual_source ? 1 : 0                            (0..1)
  pgxp   = pgxp_depth ? 1 : 0                             (0..1)
  uv     = uv_limits ? 1 : 0                              (0..1)
  index  = interp*16 + persp*8 + dual*4 + pgxp*2 + uv     (0..47)

The use_dual_source derivation in GetBatchFragmentShader matches
the shadergen-side check exactly: supports_dual_source_blend &&
((render_mode is transparent) || filter != Nearest). For the two
embedded branches (untextured and textured Nearest) the filter-half
of the OR evaluates false, so both branches fall through to the
render_mode test.

Smoke tests:
  - g++ -fsyntax-only on the touched common TU.
  - spirv-val --target-env vulkan1.0 on all 48 blobs - 48/48 valid.
  - Total .inc size: ~2.2 MiB across all 48 blobs (~46 KiB each in
    text form, which is the 0xNNNNNNNNu, packed layout from the regen
    tool; binary contribution to the linked output is roughly 1/4 of
    that, so ~550 KiB. Acceptable for a renderer that previously
    pulled in glslang at runtime).
  - --target-env vulkan1.0 vs un-flagged glslangValidator: byte-
    identical for every variant.

Test priority:
  - Routine smoke under Enabled / Lazy / Disabled with Nearest
    filter (the default). Any textured polygon in any game now
    exercises the embedded path: dialogue portraits, sprite layers,
    background textures - the vast majority of pixels in most
    games.
  - Toggle PGXP UV correction (drives m_using_uv_limits) to flip
    the UV_LIMITS structural axis. Games that benefit most from
    PGXP corrections - Final Fantasy series, Tekken 3, Soul Blade -
    are good exercise targets.
  - Antialiasing toggle round-trip under Disabled mode per the
    patch-5-fix protocol (flips interpolation axis).
  - Each texture mode is exercised by different draws: palette
    textures dominate (most game art is 4-bit indexed), with
    direct-16bpp showing up for render-to-texture effects. Raw
    textures appear in framebuffer effects and some UI passes.
  - Cross-check with shadergen path by switching texture filter to
    Bilinear (still on the shadergen path) and back to Nearest -
    rendering should remain consistent across the switch.

After this lands, the only remaining glslang consumers in the
Vulkan backend are the texture-filter slices (Bilinear /
BilinearBinAlpha, JINC2 / JINC2BinAlpha, xBR / xBRBinAlpha). Each
of those filter families has its own FilteredSampleFromVRAM body
and will get its own template in subsequent patches.
2026-05-18 01:43:15 +02:00

327 lines
11 KiB
GLSL

// Pre-baked Vulkan source template for the TEXTURED + NEAREST-FILTER
// slice of GPU_HW_ShaderGen::GenerateBatchFragmentShader (texture_mode
// != GPUTextureMode::Disabled AND m_texture_filter ==
// GPUTextureFilter::Nearest). Subsequent patches add equivalent
// templates for Bilinear / BilinearBinAlpha, JINC2 / JINC2BinAlpha,
// and xBR / xBRBinAlpha; the remaining "is m_texture_filter Nearest"
// gate in GetBatchFragmentShader on the C++ side picks this template
// for nearest-only sessions.
//
// Five SPIR-V-structural axes for this template:
//
// - Input interpolation qualifier (none / centroid / sample). 3.
// - Color input perspective (standard / noperspective). 2.
// - Dual-source output (1 vs 2 outputs). 2.
// - PGXP depth output (writes gl_FragDepth or omits it). 2.
// - UV limits (without / with v_uv_limits flat input). 2.
//
// 3 x 2 x 2 x 2 x 2 = 48 blobs.
//
// Per-call specialisation constants (collapse onto each blob):
//
// id = 0 RESOLUTION_SCALE (uint, per-session)
// id = 100 TRANSPARENCY (bool)
// id = 101 TRANSPARENCY_ONLY_OPAQUE (bool)
// id = 102 TRANSPARENCY_ONLY_TRANSPARENT (bool)
// id = 103 DITHERING (bool)
// id = 104 INTERLACING (bool)
// id = 105 DITHERING_SCALED (bool, per-session)
// id = 106 TRUE_COLOR (bool, per-session)
// id = 107 PALETTE_4_BIT (bool, derived from
// texture_mode actual mode)
// id = 108 PALETTE_8_BIT (bool, derived from
// texture_mode actual mode)
// id = 109 RAW_TEXTURE (bool, RawTextureBit set
// in texture_mode)
//
// PALETTE is derived inside the shader as PALETTE_4_BIT ||
// PALETTE_8_BIT; only at most one of the two is true for any draw.
// The texture_mode encoding is described in src/core/gpu_types.h.
//
// Bindings: pipeline layout m_batch_pipeline_layout. UBO at set=0
// binding=0 supplies BatchUBOData (texture window + transparency +
// interlacing). Combined-image-sampler at set=0 binding=1 is the
// upscaled m_vram_read_texture, single-sample (texture sampling
// always reads the non-multisampled shadow VRAM regardless of
// m_multisamples).
#version 450 core
// ---- Specialisation constants --------------------------------------
layout(constant_id = 0) const uint RESOLUTION_SCALE = 1u;
layout(constant_id = 100) const bool TRANSPARENCY = false;
layout(constant_id = 101) const bool TRANSPARENCY_ONLY_OPAQUE = false;
layout(constant_id = 102) const bool TRANSPARENCY_ONLY_TRANSPARENT = false;
layout(constant_id = 103) const bool DITHERING = false;
layout(constant_id = 104) const bool INTERLACING = false;
layout(constant_id = 105) const bool DITHERING_SCALED = false;
layout(constant_id = 106) const bool TRUE_COLOR = false;
layout(constant_id = 107) const bool PALETTE_4_BIT = false;
layout(constant_id = 108) const bool PALETTE_8_BIT = false;
layout(constant_id = 109) const bool RAW_TEXTURE = false;
// ---- Interpolation qualifier macros --------------------------------
#if defined(INTERP_SAMPLE)
# define INTERP sample
#elif defined(INTERP_CENTROID)
# define INTERP centroid
#else
# define INTERP
#endif
#if defined(NOPERSP)
# define COLOR_INTERP noperspective INTERP
#else
# define COLOR_INTERP INTERP
#endif
// ---- Batch UBO -----------------------------------------------------
layout(std140, set = 0, binding = 0) uniform BatchUBOData {
uvec2 u_texture_window_and;
uvec2 u_texture_window_or;
float u_src_alpha_factor;
float u_dst_alpha_factor;
uint u_interlaced_displayed_field;
bool u_set_mask_while_drawing;
};
// ---- VRAM atlas sampler --------------------------------------------
layout(set = 0, binding = 1) uniform sampler2D samp0;
// ---- Inputs from the batch VS --------------------------------------
layout(location = 0) in VertexData {
COLOR_INTERP vec4 v_col0;
INTERP vec2 v_tex0;
flat uvec4 v_texpage;
#if defined(UV_LIMITS)
flat vec4 v_uv_limits;
#endif
};
// ---- Outputs -------------------------------------------------------
layout(location = 0, index = 0) out vec4 o_col0;
#if defined(DUAL_SOURCE)
layout(location = 0, index = 1) out vec4 o_col1;
#endif
// ---- Dithering matrix ----------------------------------------------
const int s_dither_values[16] = int[16](
-4, 0, -3, 1,
2, -2, 3, -1,
-3, 1, -4, 0,
3, -1, 2, -2
);
uvec3 ApplyDithering(uvec2 coord, uvec3 icol)
{
uvec2 fc = DITHERING_SCALED
? (coord & uvec2(3u, 3u))
: ((coord / RESOLUTION_SCALE) & uvec2(3u, 3u));
int offset = s_dither_values[fc.y * 4u + fc.x];
if (TRUE_COLOR)
return uvec3(clamp(ivec3(icol) + ivec3(offset, offset, offset), 0, 255));
else
return uvec3(clamp((ivec3(icol) + ivec3(offset, offset, offset)) >> 3, 0, 31));
}
// ---- Texture-window helpers ----------------------------------------
uvec2 ApplyTextureWindow(uvec2 coords)
{
uint x = (coords.x & u_texture_window_and.x) | u_texture_window_or.x;
uint y = (coords.y & u_texture_window_and.y) | u_texture_window_or.y;
return uvec2(x, y);
}
uvec2 ApplyUpscaledTextureWindow(uvec2 coords)
{
uvec2 native_coords = coords / RESOLUTION_SCALE;
uvec2 coords_offset = coords % RESOLUTION_SCALE;
return (ApplyTextureWindow(native_coords) * RESOLUTION_SCALE) + coords_offset;
}
uvec2 FloatToIntegerCoords(vec2 coords)
{
// 1x-resolution-scale: round to nearest texel (the +0.5 vertex offset
// from the batch VS aligned us on a texel centre). >1x: floor, since
// the vertex offset is pre-applied CPU-side and we want the bottom-
// left of the upscaled cell.
return uvec2((RESOLUTION_SCALE == 1u) ? roundEven(coords) : floor(coords));
}
uint RGBA8ToRGBA5551(vec4 v)
{
uint r = uint(roundEven(v.r * 31.0));
uint g = uint(roundEven(v.g * 31.0));
uint b = uint(roundEven(v.b * 31.0));
uint a = (v.a != 0.0) ? 1u : 0u;
return r | (g << 5) | (b << 10) | (a << 15);
}
// ---- VRAM atlas sample ---------------------------------------------
vec4 SampleFromVRAM(uvec4 texpage, vec2 coords)
{
const vec2 rcp_vram_size = vec2(1.0) / vec2(uvec2(1024u, 512u) * RESOLUTION_SCALE);
bool palette = PALETTE_4_BIT || PALETTE_8_BIT;
if (palette)
{
// 4-bit and 8-bit paletted: address is 4 or 2 native-texel-wide
// packed cells in the VRAM atlas; the per-subpixel index pulls the
// 4- or 8-bit palette entry from that cell.
uvec2 icoord = ApplyTextureWindow(FloatToIntegerCoords(coords));
uvec2 index_coord = icoord;
if (PALETTE_4_BIT)
index_coord.x /= 4u;
else // PALETTE_8_BIT
index_coord.x /= 2u;
uvec2 vicoord = uvec2(texpage.x + index_coord.x * RESOLUTION_SCALE,
texpage.y + index_coord.y * RESOLUTION_SCALE);
vec4 texel = texture(samp0, vec2(vicoord) * rcp_vram_size);
uint vram_value = RGBA8ToRGBA5551(texel);
uint palette_index;
if (PALETTE_4_BIT)
{
uint subpixel = icoord.x & 3u;
palette_index = (vram_value >> (subpixel * 4u)) & 0x0Fu;
}
else // PALETTE_8_BIT
{
uint subpixel = icoord.x & 1u;
palette_index = (vram_value >> (subpixel * 8u)) & 0xFFu;
}
uvec2 palette_icoord = uvec2(texpage.z + (palette_index * RESOLUTION_SCALE),
texpage.w);
return texture(samp0, vec2(palette_icoord) * rcp_vram_size);
}
else
{
// Direct 16bpp - render-to-texture effects, upscaled VRAM coords.
uvec2 icoord = ApplyUpscaledTextureWindow(FloatToIntegerCoords(coords));
uvec2 direct_icoord = uvec2(texpage.x + icoord.x, texpage.y + icoord.y);
return texture(samp0, vec2(direct_icoord) * rcp_vram_size);
}
}
// --------------------------------------------------------------------
void main()
{
uvec3 vertcol = uvec3(v_col0.rgb * vec3(255.0, 255.0, 255.0));
// INTERLACING field discard (fixYCoord is identity on Vulkan).
if (INTERLACING)
{
if ((uint(gl_FragCoord.y) & 1u) == u_interlaced_displayed_field)
discard;
}
// Texture coords. Palette textures index the VRAM atlas with native-
// resolution coords; direct-16bpp textures use upscaled coords.
bool palette = PALETTE_4_BIT || PALETTE_8_BIT;
vec2 coords = v_tex0;
if (palette)
coords /= float(RESOLUTION_SCALE);
#if defined(UV_LIMITS)
vec4 uv_limits = v_uv_limits;
if (!palette)
{
// Direct-16bpp: extend the UV range to all upscaled pixels so 1-
// pixel-high polygon-based framebuffer effects (e.g. MegaMan Legends
// 2 haze) are not downsampled.
uv_limits = uv_limits * float(RESOLUTION_SCALE);
uv_limits.zw += float(RESOLUTION_SCALE - 1u);
}
vec4 texcol = SampleFromVRAM(v_texpage, clamp(coords, uv_limits.xy, uv_limits.zw));
#else
vec4 texcol = SampleFromVRAM(v_texpage, coords);
#endif
// Transparent-texel discard: VRAM rgba 0000h is the "no draw" sentinel.
if (all(equal(texcol, vec4(0.0, 0.0, 0.0, 0.0))))
discard;
float ialpha = 1.0;
bool semitransparent = (texcol.a >= 0.5);
uvec3 icolor;
if (!TRUE_COLOR)
{
icolor = uvec3(texcol.rgb * vec3(255.0, 255.0, 255.0)) >> 3;
if (!RAW_TEXTURE)
{
icolor = (icolor * vertcol) >> 4;
if (DITHERING)
icolor = ApplyDithering(uvec2(gl_FragCoord.xy), icolor);
else
icolor = min(icolor >> 3, uvec3(31u, 31u, 31u));
}
}
else
{
icolor = uvec3(texcol.rgb * vec3(255.0, 255.0, 255.0));
if (!RAW_TEXTURE)
{
icolor = (icolor * vertcol) >> 7;
if (DITHERING)
icolor = ApplyDithering(uvec2(gl_FragCoord.xy), icolor);
else
icolor = min(icolor, uvec3(255u, 255u, 255u));
}
}
// Mask bit: textured polygons mask if either u_set_mask_while_drawing
// is set or the texel was semitransparent.
float oalpha = u_set_mask_while_drawing ? 1.0 : (semitransparent ? 1.0 : 0.0);
// Premultiplied colour.
float premultiply_alpha = TRANSPARENCY
? (ialpha * (semitransparent ? u_src_alpha_factor : 1.0))
: ialpha;
vec3 color = TRUE_COLOR
? ((vec3(icolor) * premultiply_alpha) / vec3(255.0, 255.0, 255.0))
: (floor(vec3(icolor) * premultiply_alpha) / vec3(31.0, 31.0, 31.0));
// Output. Textured path differs from untextured in that the
// TRANSPARENCY branch is the "TEXTURED && TRANSPARENCY" branch in
// the shadergen, which also drives the per-texel discard logic for
// TRANSPARENCY_ONLY_OPAQUE / TRANSPARENCY_ONLY_TRANSPARENT.
if (TRANSPARENCY)
{
if (semitransparent)
{
o_col0 = vec4(color, oalpha);
#if defined(DUAL_SOURCE)
o_col1 = vec4(0.0, 0.0, 0.0, u_dst_alpha_factor / ialpha);
#endif
if (TRANSPARENCY_ONLY_OPAQUE)
discard;
}
else
{
o_col0 = vec4(color, oalpha);
#if defined(DUAL_SOURCE)
o_col1 = vec4(0.0, 0.0, 0.0, 1.0 - ialpha);
#endif
if (TRANSPARENCY_ONLY_TRANSPARENT)
discard;
}
}
else
{
o_col0 = vec4(color, oalpha);
#if defined(DUAL_SOURCE)
o_col1 = vec4(0.0, 0.0, 0.0, 1.0 - ialpha);
#endif
}
#if !defined(PGXP_DEPTH)
gl_FragDepth = oalpha * gl_FragCoord.z;
#endif
}