TsundereStation/data/shaders/vulkan/batch_untextured.frag.glsl
LibretroAdmin c532a34871 core: route TRANSPARENCY through the batch UBO
Cross-backend cbuffer-routing for the 4-value BatchRenderMode
enum that drove the batch FS body's transparency-mode branches.
Previously TRANSPARENCY / TRANSPARENCY_ONLY_OPAQUE /
TRANSPARENCY_ONLY_TRANSPARENT lived as 3 compile-time DefineMacro
calls on the C++ shadergen path and 3 specialisation constants
(constant_id 100 / 101 / 102) on the Vulkan pre-baked SPIR-V
path. Both encodings collapsed the 4 enum values across 3
booleans:

  TransparencyDisabled (0): TRANS=0, OO=0, OT=0
  TransparentAndOpaque (1): TRANS=1, OO=0, OT=0
  OnlyOpaque           (2): TRANS=1, OO=1, OT=0
  OnlyTransparent      (3): TRANS=1, OO=0, OT=1

Post-routing the 3 macros / spec consts are gone. The FS body
branches on u_render_mode (4-state uint at cbuffer offset 60 -
the former u_pad2 reserved slot) and derives the three former
booleans inline as (u_render_mode != 0u), (u_render_mode == 2u),
(u_render_mode == 3u). PSO blend state still varies per
BatchRenderMode at the GraphicsPipelineBuilder level
(m_batch_pipelines's [render_mode] matrix dim stays); the win
is purely on the FS bytecode side - 4x fewer distinct DXBC /
SPIR-V variants per (filter, render, texture) cell.

What changes:

  src/core/gpu_hw.h
    BatchUBOData: rename u_pad2 -> u_render_mode at offset 60.
    Same 64-byte total cbuffer size, same alignment, same offset
    for every other field. No external ABI break - the binding
    layout is opaque-blob-shaped from the host's perspective.

  src/core/gpu_hw_shadergen.cpp
    Drop the 3 DefineMacro calls. WriteBatchUniformBuffer's
    field list trades u_pad2 for u_render_mode. The FS body's
    `#if TRANSPARENCY ... #elif TRANSPARENCY && TEXTURED ...
    #else` pre-routing structure (with inner
    `#if TRANSPARENCY_ONLY_OPAQUE / #if TRANSPARENCY_ONLY_TRANSPARENT`
    discards) collapses to:

      if (u_render_mode != 0u) {
        if (semitransparent) {
          // emit colour + dual-source aux + optional OO discard
          if (u_render_mode == 2u) discard;
        } else {
          // emit colour + dual-source aux + optional OT discard
          if (u_render_mode == 3u) discard;
        }
      } else {
        // emit colour + dual-source aux without transparency premultiply
      }

    The TEXTURED gate stays compile-time - it's a structural
    shader-source dimension (texture_mode-dependent shadergen
    branch that decides whether to emit SampleFromVRAM /
    palette / texture-window helpers at all).

  src/core/gpu_hw.cpp
    FlushRender now stamps u_render_mode in m_batch_ubo_data
    before each draw:
      - Single-pass: set u_render_mode = m_batch.GetRenderMode(),
        mark dirty if it changed, then the existing
        m_batch_ubo_dirty check uploads. Same upload count as
        pre-routing for the common case.
      - Two-pass NeedsTwoPassRendering(): set u_render_mode =
        OnlyOpaque, upload, draw; set u_render_mode =
        OnlyTransparent, upload, draw. +1 ~64-byte UBO upload
        per two-pass FlushRender. Worth it - the FS variant
        matrix collapses by 4x on every backend that pre-bakes.

  src/core/gpu_hw_shadergen.cpp (premultiply branch)
    `#if TRANSPARENCY` -> `if (u_render_mode != 0u)` for the
    src_alpha_factor premultiply.

  src/core/gpu_hw_vulkan.cpp
    Drop the 3 fs_spec.AddBool calls for id=100/101/102 from
    the batch FS pipeline build. RESOLUTION_SCALE / DITHERING /
    INTERLACING / DITHERING_SCALED / TRUE_COLOR / PALETTE_*BIT /
    RAW_TEXTURE / BINALPHA stay (their spec const semantics
    are unchanged). The local `brm` BatchRenderMode local
    drops too; render_mode reaches the FS via u_render_mode
    in the cbuffer, populated by gpu_hw.cpp before each draw.

  data/shaders/vulkan/batch_untextured.frag.glsl
  data/shaders/vulkan/batch_textured_nearest.frag.glsl
  data/shaders/vulkan/batch_textured_bilinear.frag.glsl
  data/shaders/vulkan/batch_textured_jinc2.frag.glsl
  data/shaders/vulkan/batch_textured_xbr.frag.glsl
    5 Vulkan FS templates drop the 3 spec const declarations
    (constant_id = 100/101/102), add u_render_mode to the
    UBO declaration (layout(offset = 60) uint u_render_mode),
    and replace TRANSPARENCY / _ONLY_OPAQUE / _ONLY_TRANSPARENT
    body references with (u_render_mode != 0u) / (==2u) /
    (==3u). Same shape changes as the prior cbuffer-routing
    arc commits (DITHERING, INTERLACING, UV_LIMITS, PGXP_DEPTH).

  src/common/vulkan/embedded_spirv/batch_*_fs_*.inc (60 files)
    All 60 batch FS SPIR-V blobs regenerated. The SPIR-V
    instruction count grows by ~30-50 words per blob (one UBO
    field load + 1-3 compare + branch instructions per former
    spec-const reference site - 5 sites per untextured / 4-7
    sites per textured filter family). Net SPIR-V footprint
    grows by ~50-100 KiB across the 60 blobs.

  src/common/vulkan/embedded_spirv/batch_vs_*.inc (12 files)
    Untouched - the VS doesn't reference TRANSPARENCY.

What does NOT change in this commit:

  - D3D12 pre-baked untextured DXBC (24 blobs at
    batch_untextured_ps_*.inc) stays as-is. The blobs were
    built from data/shaders/d3d12/batch_untextured.ps.hlsl
    pre-this-commit with TRANSPARENCY as a `-D` macro; they
    still produce correct output because each baked variant
    has the right transparency behavior baked in, and the
    PickBatchUntexturedFSBytecode picker still selects the
    right variant from render_mode. The new u_render_mode
    cbuffer field is set in the UBO but the pre-baked DXBC
    doesn't read it (it doesn't reference the field).
    A follow-up commit can re-bake those 24 blobs with the
    routed HLSL, dropping the transparency axis to take the
    untextured slice from 24 -> 12 variants (or 6 if we also
    drop the dual_source axis for untextured-where-it-doesn't-
    matter cases - the d=true case for untextured non-trans
    is reachable only when filter != Nearest which is rare).
    Not blocking on this commit.

  - PSO matrix shape stays <5, 9, 4, 2, 7> = 2520 PSOs (filter
    x texture x render x depth_test x transparency). The
    [render_mode] dim stays because PSO blend state still
    varies per BatchRenderMode value. m_batch_fragment_shader_blobs
    stays <9, 4, 7> = 252 slots, though now the [render_mode]
    dim's 4 slots can store identical FS bytecode if every
    backend's runtime compile path produces the same DXBC for
    all 4 render_modes (which it will, post-routing). A
    follow-up could shrink the dim from 4 to 1 once we trust
    the C++ shadergen output to be invariant under render_mode.

  - The C++ shadergen TEXTURED branch is still compile-time.
    Routing TEXTURED to cbuffer would require declaring all
    textured-only helpers (SampleFromVRAM, ApplyTextureWindow,
    FloatToIntegerCoords) and the texture binding even for
    untextured shaders - bigger architecture change without
    proportional payoff.

  - render_mode dependency at the call site GetBatchFragmentShader
    on the D3D12 side. The function still takes render_mode as
    a parameter so it can hash into the right slot in
    m_batch_fragment_shader_blobs. The shader source it
    generates is now invariant under render_mode (the shadergen
    call's transparency parameter is unused for the FS body),
    so technically the 4 [render_mode] slots end up holding
    identical ID3DBlob refs. Cleaning that up is the follow-up
    "shrink [render_mode] dim" work mentioned above.

Test:
  - glslangValidator on all 5 Vulkan FS templates: passes.
  - python3 tools/regen_vulkan_spirv.py: produces 91 SPIR-V
    blobs (12 VS + 60 FS + 19 non-batch), all valid.
  - Boot any PSX title on Vulkan: render_mode flips per-batch
    pick up the new u_render_mode value via cbuffer. Two-pass
    semitransparent draws (e.g. Final Fantasy menus, MGS
    smoke) see the matching enum value on each pass via the
    +1 UBO re-upload between OnlyOpaque / OnlyTransparent
    DrawIndexed calls. Visual diff: zero.
  - Boot any PSX title on D3D11 / D3D12 / OpenGL: the
    runtime-compiled FS bytecode loses the 3 TRANSPARENCY
    macros and gains a u_render_mode runtime branch. Pre-
    baked D3D12 untextured slots still work via the picker.
    Visual diff: zero.
  - PGXP-flip mid-session: independent of this commit;
    behaviour unchanged.

Next:
  - Re-bake D3D12 untextured DXBC with the routed HLSL
    template (24 -> 12 variants, drop the transparency axis).
    Small follow-up.
  - Pre-bake the 4 textured filter slices on the routed
    matrix: 72 variants per template (3 interp x 2 persp x 2
    dual x 6 texture_mode) instead of 288 with TRANSPARENCY
    as a `-D` axis. Each slice gets a foundation commit
    (HLSL + regen + .inc) and an activation commit
    (extern decls + picker + GetBatchPipeline wiring),
    mirroring the untextured arc.
2026-05-19 08:13:03 +02:00

202 lines
8.1 KiB
GLSL

// Pre-baked Vulkan source template for the UNTEXTURED slice of
// GPU_HW_ShaderGen::GenerateBatchFragmentShader (texture_mode ==
// GPUTextureMode::Disabled). The textured slice is several times
// larger and will land in subsequent patches, one or two texture
// filter values at a time; this template only emits SPIR-V valid for
// pipelines whose lookup_mode is Disabled (the renderer's untextured
// path - solid-shaded polygons, no VRAM sampling).
//
// Structural axes for the untextured batch FS:
//
// - Input interpolation qualifier on v_col0. None / centroid /
// sample, matching the batch VS pair this FS is bound with. (3
// variants.)
// - Color input perspective. Standard / noperspective. (2 variants.)
// - Dual-source color output. With dual-source-blend hardware the
// transparency=TransparentAndOpaque or =OnlyTransparent render
// modes also write a second colour value at location 0 index 1
// that drives SRC1_COLOR / SRC1_ALPHA in the blend equation. The
// output declaration count is structural - the pipeline blend
// state references location-0-index-1 or it does not. (2 variants.)
//
// Total untextured structural blobs: 3 x 2 x 2 = 12.
//
// PGXP_DEPTH used to be a fourth axis (writes gl_FragDepth or omits
// it) but has been collapsed to a runtime branch on the u_pgxp_depth
// cbuffer scalar - gl_FragDepth is always written now, with the
// expression chosen at runtime. Brings the untextured cube into
// parity with the other 4 FS templates after their parallel
// collapses.
//
// Per-call and per-session knobs that DO NOT affect SPIR-V structure
// collapse into specialisation constants on every blob:
//
// constant_id = 0 RESOLUTION_SCALE (uint)
// constant_id = 103 DITHERING (bool)
// constant_id = 104 INTERLACING (bool)
// constant_id = 105 DITHERING_SCALED (bool)
// constant_id = 106 TRUE_COLOR (bool)
//
// TRANSPARENCY used to live as 3 spec consts at constant_id =
// 100/101/102 (TRANSPARENCY, TRANSPARENCY_ONLY_OPAQUE,
// TRANSPARENCY_ONLY_TRANSPARENT) encoding the 4-value
// BatchRenderMode enum. They have been collapsed to a runtime
// branch on the u_render_mode cbuffer scalar (offset 60 - the
// former u_pad2 slot). The C++ side re-uploads the UBO between
// two-pass DrawIndexed calls so each draw sees its matching
// u_render_mode value (see gpu_hw.cpp FlushRender). Mirrors the
// prior cbuffer-routing arc (DITHERING / INTERLACING / UV_LIMITS
// / PGXP_DEPTH).
//
// Bindings: pipeline layout m_batch_pipeline_layout. The dynamic UBO
// at set=0 binding=0 supplies the BatchUBOData fields read here
// (u_src_alpha_factor, u_dst_alpha_factor, u_interlaced_displayed_field,
// u_set_mask_while_drawing, u_render_mode). u_texture_window_* are
// present in the UBO but unused in the untextured path.
#version 450 core
// ---- Specialisation constants --------------------------------------
layout(constant_id = 0) const uint RESOLUTION_SCALE = 1u;
layout(constant_id = 103) const bool DITHERING = false;
layout(constant_id = 104) const bool INTERLACING = false;
layout(constant_id = 105) const bool DITHERING_SCALED = false;
layout(constant_id = 106) const bool TRUE_COLOR = false;
// ---- Interpolation qualifier macros (structural) -------------------
#if defined(INTERP_SAMPLE)
# define INTERP sample
#elif defined(INTERP_CENTROID)
# define INTERP centroid
#else
# define INTERP
#endif
#if defined(NOPERSP)
# define COLOR_INTERP noperspective INTERP
#else
# define COLOR_INTERP INTERP
#endif
// ---- Batch UBO -----------------------------------------------------
// Layout matches WriteBatchUniformBuffer in the shadergen. Field order
// is significant: the C++ side packs in this exact order. u_pgxp_depth
// at offset 52 is read at runtime to decide the gl_FragDepth write
// expression; the explicit layout(offset=52) skips past the spec-
// const-handled fields between offset 32 and 51 (u_resolution_scale,
// u_true_color, u_scaled_dithering, u_dithering, u_interlacing).
layout(std140, set = 0, binding = 0) uniform BatchUBOData {
uvec2 u_texture_window_and;
uvec2 u_texture_window_or;
float u_src_alpha_factor;
float u_dst_alpha_factor;
uint u_interlaced_displayed_field;
bool u_set_mask_while_drawing;
layout(offset = 52) uint u_pgxp_depth;
layout(offset = 60) uint u_render_mode;
};
// ---- Inputs from the batch VS --------------------------------------
layout(location = 0) in VertexData {
COLOR_INTERP vec4 v_col0;
};
// ---- Outputs (DUAL_SOURCE axis) ------------------------------------
layout(location = 0, index = 0) out vec4 o_col0;
#if defined(DUAL_SOURCE)
layout(location = 0, index = 1) out vec4 o_col1;
#endif
// PGXP_DEPTH axis collapsed: gl_FragDepth is now always written, with
// a runtime branch on u_pgxp_depth at the write site below choosing
// between v_pos.z pass-through (PGXP mode) and oalpha * v_pos.z (mask-
// bit encoding for legacy non-PGXP depth use).
// ---- Dithering matrix (CONSTANT in shadergen) ----------------------
// Same 4x4 Bayer values as DITHER_MATRIX in core/types.h.
const int s_dither_values[16] = int[16](
-4, 0, -3, 1,
2, -2, 3, -1,
-3, 1, -4, 0,
3, -1, 2, -2
);
uvec3 ApplyDithering(uvec2 coord, uvec3 icol)
{
uvec2 fc = DITHERING_SCALED
? (coord & uvec2(3u, 3u))
: ((coord / RESOLUTION_SCALE) & uvec2(3u, 3u));
int offset = s_dither_values[fc.y * 4u + fc.x];
if (TRUE_COLOR)
return uvec3(clamp(ivec3(icol) + ivec3(offset, offset, offset), 0, 255));
else
return uvec3(clamp((ivec3(icol) + ivec3(offset, offset, offset)) >> 3, 0, 31));
}
void main()
{
uvec3 vertcol = uvec3(v_col0.rgb * vec3(255.0, 255.0, 255.0));
// INTERLACING discard (untextured can still hit a field-strobed draw).
// fixYCoord on Vulkan is the identity function so we use gl_FragCoord.y
// directly.
if (INTERLACING)
{
if ((uint(gl_FragCoord.y) & 1u) == u_interlaced_displayed_field)
discard;
}
// Untextured fragment: all pixels are semitransparent by convention,
// colour comes from the interpolated vertex colour, ialpha is unity.
uvec3 icolor = vertcol;
float ialpha = 1.0;
if (DITHERING)
{
icolor = ApplyDithering(uvec2(gl_FragCoord.xy), icolor);
}
else
{
if (!TRUE_COLOR)
icolor >>= 3;
}
// Mask bit: untextured polygons clear the mask bit unless the
// 'set mask while drawing' command bit is set.
float oalpha = float(u_set_mask_while_drawing);
// Premultiply alpha so the colour output can carry the mask in alpha.
float premultiply_alpha = (u_render_mode != 0u) ? (ialpha * u_src_alpha_factor) : ialpha;
vec3 color = TRUE_COLOR
? ((vec3(icolor) * premultiply_alpha) / vec3(255.0, 255.0, 255.0))
: (floor(vec3(icolor) * premultiply_alpha) / vec3(31.0, 31.0, 31.0));
// Output. With dual-source blending o_col1 carries the destination
// alpha factor; without, only o_col0 is written. The matching
// pipeline blend state references SRC1_* or it does not, set by
// GetBatchPipeline on the C++ side.
o_col0 = vec4(color, oalpha);
#if defined(DUAL_SOURCE)
if ((u_render_mode != 0u))
o_col1 = vec4(0.0, 0.0, 0.0, u_dst_alpha_factor / ialpha);
else
o_col1 = vec4(0.0, 0.0, 0.0, 1.0 - ialpha);
#endif
// Untextured + transparency-only-opaque should never produce
// visible pixels - the original discards them after the colour
// output. This branch should be unreachable for untextured (the
// shadergen documents it as "We shouldn't be rendering opaque
// geometry only when untextured"), but the original code did not
// assert this so we mirror its silence rather than redirect.
// gl_FragDepth: always written (PGXP_DEPTH was a compile-time
// axis pre-routing). u_pgxp_depth != 0 picks gl_FragCoord.z
// pass-through (the rasterizer-interpolated VS depth IS the
// PGXP-replayed depth in this mode); u_pgxp_depth == 0 picks
// oalpha * gl_FragCoord.z to encode the PSX mask bit into the
// depth buffer for the legacy non-PGXP path.
gl_FragDepth = (u_pgxp_depth != 0u) ? gl_FragCoord.z : (oalpha * gl_FragCoord.z);
}