TsundereStation/tools/regen_vulkan_spirv.py
LibretroAdmin c6e9f87be0 core/vulkan: route the Nearest FS UV_LIMITS axis to a cbuffer scalar
Follow-up to 078783e (Vulkan VS UV_LIMITS collapse) and the
earlier 8b43518 (C++ shadergen UV_LIMITS routing). Brings the
Vulkan Nearest FS into the same shape as the C++ shadergen
routing - UV_LIMITS is no longer a SPIR-V-structural axis on
the Vulkan side; the Nearest FS source has a single shape and
runtime-branches on u_uv_limits.

With this commit m_using_uv_limits is no longer a shader-
source-changed trigger for any backend, so it leaves the
rebuild-trigger predicates entirely. It remains as a member
field that drives the per-vertex ComputePolygonUVLimits CPU
optimisation (gpu_hw.cpp:740) - that's an orthogonal concern
from shader / pipeline routing.

What changes:

  data/shaders/vulkan/batch_textured_nearest.frag.glsl
    - Adds u_uv_limits to the BatchUBOData declaration with
      an explicit layout(offset = 56) (skips past the spec-
      const-handled fields between offset 32 and 55, which
      Vulkan does not redeclare in its UBO because per-
      session settings are spec consts on this path -
      RESOLUTION_SCALE, TRUE_COLOR, SCALED_DITHERING,
      DITHERING, INTERLACING, PGXP_DEPTH). Explicit offset
      via GL_ARB_enhanced_layouts is core in GLSL 4.40+ and
      supported by Vulkan GLSL.
    - v_uv_limits input declaration drops its
      #if defined(UV_LIMITS) guard - always declared,
      matching the always-emitted output from the
      post-collapse batch VS.
    - Body sample/clamp block: the two-branch
      #if UV_LIMITS / #else compile-time fork collapses to
      a single shape with a runtime `if (u_uv_limits != 0u)`
      branch. The "scale uv_limits up by RESOLUTION_SCALE
      for non-palette modes" computation moves inside the
      branch so it only runs when the values are actually
      used.
    - Header comment: 5 axes / 48 blobs -> 4 axes / 24 blobs.

  tools/regen_vulkan_spirv.py
    - _batch_fs_textured_nearest_variants() drops the
      uv_axes dimension. The Nearest cube is now in parity
      shape with the Bilinear / JINC2 / xBR templates
      (UV_LIMITS implicit on those by ShouldUseUVLimits()
      coupling - TEXTURE_FILTERING != Nearest forces
      m_using_uv_limits true, so they never needed a UV
      variant axis).

  src/common/vulkan/embedded_spirv/
    - 24 new batch_textured_nearest_fs_*.inc files (no
      _nouv / _uv suffix) with the collapsed-shape SPIR-V.
    - 48 deleted batch_textured_nearest_fs_*_nouv.inc and
      _uv.inc files (the obsolete two-shape variants).
    - 151 .inc files in the directory total (was 175;
      the VS collapse already took it from 181).

  src/common/vulkan/embedded_shaders.{h,cpp}
    - k_batch_textured_nearest_fs_blobs[48] ->
      k_batch_textured_nearest_fs_blobs[24]. The table loses
      the alternating _nouv / _uv pairs; the surviving
      entries match the new filename layout.
    - GetBatchTexturedNearestFragmentShaderBlob drops the
      bool uv_limits parameter. Index encoding becomes
        index = interp*8 + persp*4 + dual*2 + pgxp  (0..23)
      (was: interp*16 + persp*8 + dual*4 + pgxp*2 + uv).
    - 48 #include lines collapse to 24, matching the
      regenerated filename set.
    - Header docstring updated to match (5 axes -> 4, blob
      array size, UV_LIMITS note added).

  src/core/gpu_hw_vulkan.cpp
    - The single GetBatchTexturedNearestFragmentShaderBlob
      call site (the m_texture_filter == Nearest branch of
      the per-pipeline FS lookup) drops its sixth argument
      (m_using_uv_limits). Bilinear / JINC2 / xBR call sites
      are unchanged - those helpers never had uv_limits as
      an argument because their templates already had
      UV_LIMITS implicit.

  src/core/gpu_hw.cpp
    - non_dim_diff in UpdateHWSettings drops the
      m_using_uv_limits != use_uv_limits clause.
    - shader_source_changed in UpdateHWSettings drops the
      same clause.
    - display_only_source_changed's other_source_diff
      drops it too.
    All three trigger predicates are now uv_limits-blind.
    m_using_uv_limits flips no longer invalidate the shader
    cache or PSO matrix on any backend.
    m_using_uv_limits itself stays as a member field -
    UpdateHWSettings still computes use_uv_limits via
    ShouldUseUVLimits() and assigns it for the runtime
    ComputePolygonUVLimits CPU-side decision and for the
    u_uv_limits cbuffer push - but it no longer triggers
    rebuilds.

What this completes:

  The UV_LIMITS axis is now fully routed across all four
  backends.
    D3D11 / D3D12 / OpenGL (C++ shadergen path):
      - Shader source invariant under m_uv_limits since
        8b43518.
      - Input layout binds ATTR4 unconditionally when
        textured (D3D11 / D3D12 since 8b43518; OpenGL was
        already unconditional).
    Vulkan (pre-baked SPIR-V path):
      - VS shape collapsed since 078783e (12 VS blobs).
      - Nearest FS shape collapsed in this commit (24 FS
        blobs).
      - Bilinear / JINC2 / xBR templates unchanged - they
        never had UV_LIMITS as an axis (settings-coupling
        guarantee).
      - Vulkan FS UBO grows by one field (u_uv_limits at
        offset 56); other Vulkan FS templates that don't
        read UV limits don't need to redeclare it - they
        just ignore the extra bytes in the backing buffer.
    All backends:
      - m_using_uv_limits is no longer a rebuild trigger.

Cost matrix delta:

  Per-session flip that touches m_using_uv_limits (e.g.
  PGXP on / off under filter Nearest, or filter cross-
  Nearest transition):
    Pre-commit (Vulkan only): 24 Nearest FS pipelines per
      cell rebuilt to swap from _nouv to _uv variants.
      Tens of milliseconds cold, instant with disk
      pipeline cache.
    Post-commit: 0 FS rebuilds across all backends. The
      u_uv_limits cbuffer write picks up at next
      FlushRender.

  Cold launch precompile pass on Vulkan: 24 fewer Nearest
  FS pipelines to enumerate. Tens of ms on cold cache.

  Steady-state runtime cost (Vulkan Nearest FS):
    + 4-byte UBO read on every fragment.
    + One branch on u_uv_limits (sub-coherent across the
      warp because u_uv_limits is uniform).
    + The "scale uv_limits up by RESOLUTION_SCALE" math
      moved inside the branch, so when u_uv_limits == 0
      that work is also skipped - net cost reduction on
      the non-UV path.

Disk footprint:
  Before:  175 .inc files, ~12.9 MiB.
  After:   151 .inc files, ~11.3 MiB. (-24 files, ~1.6
           MiB. The 24 new files are larger than the 24
           old _nouv files they implicitly replace because
           they always emit the v_uv_limits flat input
           declaration and the body branch.)

Test:
  - Build: tools/regen_vulkan_spirv.py runs cleanly,
    producing 151 blobs. Embedded_shaders.cpp compiles
    against the smaller table. gpu_hw_vulkan.cpp link is
    clean.
  - Boot any PSX title with PGXP off + filter Nearest:
    u_uv_limits = 0, the FS branch takes the no-clamp
    path. Output identical to pre-commit.
  - Toggle PGXP on with filter Nearest: u_uv_limits flips
    to 1 via the existing UpdateHWSettings cbuffer push;
    the FS picks the clamp path on next FlushRender. No
    pipeline rebuild fired.
  - Toggle filter to Bilinear: m_texture_filtering changes,
    which still fires shader_source_changed (filter is a
    distinct rebuild trigger). The Bilinear FS picks up
    its template normally, which always uses UV limits
    (settings coupling guarantee).
  - Cross-test on non-Vulkan backends: D3D11 / D3D12 /
    OpenGL also no longer rebuild on UV_LIMITS flip alone
    (the trigger drop in gpu_hw.cpp affects them too).
    Their shader sources were already invariant since
    8b43518, so the only behaviour change is the
    rebuild-trigger removal.

Next:
  - PGXP_DEPTH for the FS (always-emit SV_Depth / FragDepth
    perf measurement first - the FS keeps captured
    m_pgxp_depth because the !m_pgxp_depth value gates
    DeclareFragmentEntryPoint's depth_output parameter,
    which controls whether SV_Depth is declared in the FS
    entry signature; routing that requires either always-
    emitting SV_Depth or hoisting that decision through
    the DeclareFragmentEntryPoint surface).
  - per_sample / dcp collapse decisions (forced-always
    via settings or keep as variant axes).
  - Pre-bake the resulting reduced D3D12 FS matrix.
2026-05-19 03:07:27 +02:00

308 lines
11 KiB
Python
Executable file

#!/usr/bin/env python3
# Regenerate embedded Vulkan SPIR-V blobs from GLSL sources.
#
# This script is NOT invoked by the core build. It is run by contributors who
# edit shader sources under data/shaders/vulkan/. The build itself only sees
# the resulting .inc files (checked into src/common/vulkan/embedded_spirv/),
# so the libretro core has zero glslangValidator dependency.
#
# Usage:
# python3 tools/regen_vulkan_spirv.py [--glslang PATH]
#
# Requires glslangValidator from the Vulkan SDK or the `glslang-tools`
# package on Debian / Ubuntu.
import argparse
import os
import re
import struct
import subprocess
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
GLSL_DIR = REPO_ROOT / "data" / "shaders" / "vulkan"
INC_DIR = REPO_ROOT / "src" / "common" / "vulkan" / "embedded_spirv"
# Map glsl file suffix to glslangValidator stage flag. Anything not in this
# table is skipped with a warning.
STAGE_FROM_SUFFIX = {
".vert.glsl": ("vert", "vs"),
".frag.glsl": ("frag", "fs"),
".comp.glsl": ("comp", "cs"),
".geom.glsl": ("geom", "gs"),
}
# Templates: shader sources that produce more than one .inc output, each
# with a different combination of -D preprocessor defines. Index ordering
# below MUST match the helper in src/common/vulkan/embedded_shaders.cpp
# that picks the right blob at runtime - if you add a variant here, add
# the matching entry there too.
#
# For each entry: filename -> [(variant_suffix, [-D defines]), ...]
def _batch_vs_variants():
out = []
# Attribute layout: untextured / textured (the latter always with
# UV limits since the UV_LIMITS-routing commit lifted the axis to
# the FS-side u_uv_limits cbuffer scalar). 2 layouts now.
attr_axes = [
("untextured", []),
("textured", ["TEXTURED"]),
]
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
interp_axes = [
("none", []),
("centroid", ["INTERP_CENTROID"]),
("sample", ["INTERP_SAMPLE"]),
]
# Color perspective: standard / noperspective.
persp_axes = [
("persp", []),
("noperp", ["NOPERSP"]),
]
for a_name, a_defs in attr_axes:
for i_name, i_defs in interp_axes:
for p_name, p_defs in persp_axes:
suffix = f"{a_name}_{i_name}_{p_name}"
out.append((suffix, a_defs + i_defs + p_defs))
return out
def _batch_fs_untextured_variants():
out = []
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
# MUST match the batch VS chosen by the C++ side.
interp_axes = [
("none", []),
("centroid", ["INTERP_CENTROID"]),
("sample", ["INTERP_SAMPLE"]),
]
# Color perspective: standard / noperspective. MUST match batch VS.
persp_axes = [
("persp", []),
("noperp", ["NOPERSP"]),
]
# Dual-source colour output: 1 attachment vs 2 (location 0
# index 0/1). Pipeline blend state references SRC1_* or it does
# not - per-call decision driven by transparency_mode AND
# m_supports_dual_source_blend.
dual_axes = [
("nodual", []),
("dual", ["DUAL_SOURCE"]),
]
# PGXP depth axis: when set, the FS omits gl_FragDepth so the
# rasterizer-interpolated VS depth (a_pos.w under PGXP) is used
# directly and early-Z stays available. Per-session.
pgxp_axes = [
("pgxpoff", []),
("pgxpon", ["PGXP_DEPTH"]),
]
for i_name, i_defs in interp_axes:
for p_name, p_defs in persp_axes:
for d_name, d_defs in dual_axes:
for x_name, x_defs in pgxp_axes:
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
return out
def _batch_fs_textured_nearest_variants():
out = []
interp_axes = [
("none", []),
("centroid", ["INTERP_CENTROID"]),
("sample", ["INTERP_SAMPLE"]),
]
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
# UV_LIMITS used to be a fifth axis here. The collapse commit lifted
# it to the u_uv_limits cbuffer scalar so the Nearest FS source
# always declares v_uv_limits and runtime-branches on the cbuffer
# value. Brings the Nearest FS cube into parity shape with the
# Bilinear / JINC2 / xBR families (UV_LIMITS implicit on those by
# settings-layer coupling).
for i_name, i_defs in interp_axes:
for p_name, p_defs in persp_axes:
for d_name, d_defs in dual_axes:
for x_name, x_defs in pgxp_axes:
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
return out
def _batch_fs_textured_filter_variants():
"""Shared structural cube for non-Nearest filter templates.
UV_LIMITS is implicit (ShouldUseUVLimits() forces it true for any
non-Nearest filter), so the cube collapses to 4 axes vs the 5
Nearest template uses. 3 x 2 x 2 x 2 = 24 blobs.
"""
out = []
interp_axes = [
("none", []),
("centroid", ["INTERP_CENTROID"]),
("sample", ["INTERP_SAMPLE"]),
]
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
for i_name, i_defs in interp_axes:
for p_name, p_defs in persp_axes:
for d_name, d_defs in dual_axes:
for x_name, x_defs in pgxp_axes:
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
return out
TEMPLATE_VARIANTS = {
"batch.vert.glsl": _batch_vs_variants(),
"batch_untextured.frag.glsl": _batch_fs_untextured_variants(),
"batch_textured_nearest.frag.glsl": _batch_fs_textured_nearest_variants(),
"batch_textured_bilinear.frag.glsl": _batch_fs_textured_filter_variants(),
"batch_textured_jinc2.frag.glsl": _batch_fs_textured_filter_variants(),
"batch_textured_xbr.frag.glsl": _batch_fs_textured_filter_variants(),
}
def find_glslang(explicit):
if explicit:
return explicit
for name in ("glslangValidator", "glslangValidator.exe"):
from shutil import which
path = which(name)
if path:
return path
sys.stderr.write(
"error: glslangValidator not found on PATH. Install the Vulkan SDK\n"
" or the glslang-tools package, or pass --glslang PATH.\n")
sys.exit(1)
def sanitize_identifier(stem):
# Map "screen_quad.vert" -> "screen_quad_vs"; only [A-Za-z0-9_] survive.
name = re.sub(r"[^A-Za-z0-9_]", "_", stem)
return name
def compile_one(glslang, glsl_path, stage_flag, defines=None):
# Write SPIR-V to a temp file under INC_DIR so we don't pollute /tmp on
# restricted-filesystem hosts.
spv_path = INC_DIR / (glsl_path.stem + ".spv.tmp")
cmd = [
glslang,
"--target-env", "vulkan1.0",
"-S", stage_flag,
"-V", str(glsl_path),
"-o", str(spv_path),
]
for d in defines or ():
cmd.append(f"-D{d}")
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
sys.stderr.write(f"glslangValidator failed for {glsl_path.name}:\n")
sys.stderr.write(result.stdout)
sys.stderr.write(result.stderr)
if spv_path.exists():
spv_path.unlink()
sys.exit(1)
data = spv_path.read_bytes()
spv_path.unlink()
if len(data) % 4 != 0:
sys.stderr.write(
f"error: {glsl_path.name} produced {len(data)} bytes "
f"of SPIR-V (not a multiple of 4)\n")
sys.exit(1)
words = struct.unpack(f"<{len(data) // 4}I", data)
if not words or words[0] != 0x07230203:
sys.stderr.write(
f"error: {glsl_path.name} produced invalid SPIR-V (bad magic)\n")
sys.exit(1)
return words
def emit_inc(identifier, glsl_name, words, out_path):
# 8 words per line keeps the file diff-friendly without being too tall.
PER_LINE = 8
lines = []
lines.append(
f"// Autogenerated from data/shaders/vulkan/{glsl_name}.\n"
f"// DO NOT EDIT. Regenerate with tools/regen_vulkan_spirv.py.\n"
f"// glslangValidator --target-env vulkan1.0\n"
f"//\n"
f"// Included inside namespace Vulkan::EmbeddedShaders in\n"
f"// embedded_shaders.cpp. The matching extern declaration lives in\n"
f"// embedded_shaders.h.\n"
f"\n"
f"const uint32_t k_{identifier}[] = {{\n")
for i in range(0, len(words), PER_LINE):
chunk = ", ".join(f"0x{w:08x}u" for w in words[i:i + PER_LINE])
lines.append(f" {chunk},\n")
lines.append(f"}};\n"
f"const size_t k_{identifier}_size_bytes = "
f"sizeof(k_{identifier});\n")
out_path.write_text("".join(lines))
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--glslang", help="path to glslangValidator binary")
args = ap.parse_args()
glslang = find_glslang(args.glslang)
INC_DIR.mkdir(parents=True, exist_ok=True)
glsl_files = sorted(GLSL_DIR.glob("*.glsl"))
if not glsl_files:
sys.stderr.write(f"warning: no .glsl files under {GLSL_DIR}\n")
return
total = 0
for glsl_path in glsl_files:
# glsl_path.name looks like "screen_quad.vert.glsl".
suffix = None
for k in STAGE_FROM_SUFFIX:
if glsl_path.name.endswith(k):
suffix = k
break
if suffix is None:
sys.stderr.write(
f"warning: {glsl_path.name} has no recognised stage suffix; "
f"skipping\n")
continue
stage_flag, ident_suffix = STAGE_FROM_SUFFIX[suffix]
# "screen_quad.vert.glsl" -> stem "screen_quad.vert" -> "screen_quad"
base = glsl_path.name[:-len(suffix)]
variants = TEMPLATE_VARIANTS.get(glsl_path.name)
if variants is None:
# Single-variant shader: one .inc, no -D flags.
identifier = sanitize_identifier(base) + "_" + ident_suffix
out_path = INC_DIR / (identifier + ".inc")
words = compile_one(glslang, glsl_path, stage_flag)
emit_inc(identifier, glsl_path.name, words, out_path)
size_kb = (len(words) * 4) / 1024.0
print(f" {glsl_path.name:<40} -> {out_path.name} "
f"({len(words)} words, {size_kb:.1f} KiB)")
total += 1
else:
# Template: emit one .inc per variant, named with a suffix
# appended after the stage identifier.
for variant_suffix, defines in variants:
identifier = (sanitize_identifier(base) + "_" + ident_suffix +
"_" + variant_suffix)
out_path = INC_DIR / (identifier + ".inc")
words = compile_one(glslang, glsl_path, stage_flag, defines)
emit_inc(identifier, glsl_path.name, words, out_path)
size_kb = (len(words) * 4) / 1024.0
print(f" {glsl_path.name:<40} -> {out_path.name} "
f"({len(words)} words, {size_kb:.1f} KiB)")
total += 1
print(f"regenerated {total} SPIR-V blob(s) in {INC_DIR}")
if __name__ == "__main__":
main()