Follow-up to078783e(Vulkan VS UV_LIMITS collapse) and the earlier8b43518(C++ shadergen UV_LIMITS routing). Brings the Vulkan Nearest FS into the same shape as the C++ shadergen routing - UV_LIMITS is no longer a SPIR-V-structural axis on the Vulkan side; the Nearest FS source has a single shape and runtime-branches on u_uv_limits. With this commit m_using_uv_limits is no longer a shader- source-changed trigger for any backend, so it leaves the rebuild-trigger predicates entirely. It remains as a member field that drives the per-vertex ComputePolygonUVLimits CPU optimisation (gpu_hw.cpp:740) - that's an orthogonal concern from shader / pipeline routing. What changes: data/shaders/vulkan/batch_textured_nearest.frag.glsl - Adds u_uv_limits to the BatchUBOData declaration with an explicit layout(offset = 56) (skips past the spec- const-handled fields between offset 32 and 55, which Vulkan does not redeclare in its UBO because per- session settings are spec consts on this path - RESOLUTION_SCALE, TRUE_COLOR, SCALED_DITHERING, DITHERING, INTERLACING, PGXP_DEPTH). Explicit offset via GL_ARB_enhanced_layouts is core in GLSL 4.40+ and supported by Vulkan GLSL. - v_uv_limits input declaration drops its #if defined(UV_LIMITS) guard - always declared, matching the always-emitted output from the post-collapse batch VS. - Body sample/clamp block: the two-branch #if UV_LIMITS / #else compile-time fork collapses to a single shape with a runtime `if (u_uv_limits != 0u)` branch. The "scale uv_limits up by RESOLUTION_SCALE for non-palette modes" computation moves inside the branch so it only runs when the values are actually used. - Header comment: 5 axes / 48 blobs -> 4 axes / 24 blobs. tools/regen_vulkan_spirv.py - _batch_fs_textured_nearest_variants() drops the uv_axes dimension. The Nearest cube is now in parity shape with the Bilinear / JINC2 / xBR templates (UV_LIMITS implicit on those by ShouldUseUVLimits() coupling - TEXTURE_FILTERING != Nearest forces m_using_uv_limits true, so they never needed a UV variant axis). src/common/vulkan/embedded_spirv/ - 24 new batch_textured_nearest_fs_*.inc files (no _nouv / _uv suffix) with the collapsed-shape SPIR-V. - 48 deleted batch_textured_nearest_fs_*_nouv.inc and _uv.inc files (the obsolete two-shape variants). - 151 .inc files in the directory total (was 175; the VS collapse already took it from 181). src/common/vulkan/embedded_shaders.{h,cpp} - k_batch_textured_nearest_fs_blobs[48] -> k_batch_textured_nearest_fs_blobs[24]. The table loses the alternating _nouv / _uv pairs; the surviving entries match the new filename layout. - GetBatchTexturedNearestFragmentShaderBlob drops the bool uv_limits parameter. Index encoding becomes index = interp*8 + persp*4 + dual*2 + pgxp (0..23) (was: interp*16 + persp*8 + dual*4 + pgxp*2 + uv). - 48 #include lines collapse to 24, matching the regenerated filename set. - Header docstring updated to match (5 axes -> 4, blob array size, UV_LIMITS note added). src/core/gpu_hw_vulkan.cpp - The single GetBatchTexturedNearestFragmentShaderBlob call site (the m_texture_filter == Nearest branch of the per-pipeline FS lookup) drops its sixth argument (m_using_uv_limits). Bilinear / JINC2 / xBR call sites are unchanged - those helpers never had uv_limits as an argument because their templates already had UV_LIMITS implicit. src/core/gpu_hw.cpp - non_dim_diff in UpdateHWSettings drops the m_using_uv_limits != use_uv_limits clause. - shader_source_changed in UpdateHWSettings drops the same clause. - display_only_source_changed's other_source_diff drops it too. All three trigger predicates are now uv_limits-blind. m_using_uv_limits flips no longer invalidate the shader cache or PSO matrix on any backend. m_using_uv_limits itself stays as a member field - UpdateHWSettings still computes use_uv_limits via ShouldUseUVLimits() and assigns it for the runtime ComputePolygonUVLimits CPU-side decision and for the u_uv_limits cbuffer push - but it no longer triggers rebuilds. What this completes: The UV_LIMITS axis is now fully routed across all four backends. D3D11 / D3D12 / OpenGL (C++ shadergen path): - Shader source invariant under m_uv_limits since8b43518. - Input layout binds ATTR4 unconditionally when textured (D3D11 / D3D12 since8b43518; OpenGL was already unconditional). Vulkan (pre-baked SPIR-V path): - VS shape collapsed since078783e(12 VS blobs). - Nearest FS shape collapsed in this commit (24 FS blobs). - Bilinear / JINC2 / xBR templates unchanged - they never had UV_LIMITS as an axis (settings-coupling guarantee). - Vulkan FS UBO grows by one field (u_uv_limits at offset 56); other Vulkan FS templates that don't read UV limits don't need to redeclare it - they just ignore the extra bytes in the backing buffer. All backends: - m_using_uv_limits is no longer a rebuild trigger. Cost matrix delta: Per-session flip that touches m_using_uv_limits (e.g. PGXP on / off under filter Nearest, or filter cross- Nearest transition): Pre-commit (Vulkan only): 24 Nearest FS pipelines per cell rebuilt to swap from _nouv to _uv variants. Tens of milliseconds cold, instant with disk pipeline cache. Post-commit: 0 FS rebuilds across all backends. The u_uv_limits cbuffer write picks up at next FlushRender. Cold launch precompile pass on Vulkan: 24 fewer Nearest FS pipelines to enumerate. Tens of ms on cold cache. Steady-state runtime cost (Vulkan Nearest FS): + 4-byte UBO read on every fragment. + One branch on u_uv_limits (sub-coherent across the warp because u_uv_limits is uniform). + The "scale uv_limits up by RESOLUTION_SCALE" math moved inside the branch, so when u_uv_limits == 0 that work is also skipped - net cost reduction on the non-UV path. Disk footprint: Before: 175 .inc files, ~12.9 MiB. After: 151 .inc files, ~11.3 MiB. (-24 files, ~1.6 MiB. The 24 new files are larger than the 24 old _nouv files they implicitly replace because they always emit the v_uv_limits flat input declaration and the body branch.) Test: - Build: tools/regen_vulkan_spirv.py runs cleanly, producing 151 blobs. Embedded_shaders.cpp compiles against the smaller table. gpu_hw_vulkan.cpp link is clean. - Boot any PSX title with PGXP off + filter Nearest: u_uv_limits = 0, the FS branch takes the no-clamp path. Output identical to pre-commit. - Toggle PGXP on with filter Nearest: u_uv_limits flips to 1 via the existing UpdateHWSettings cbuffer push; the FS picks the clamp path on next FlushRender. No pipeline rebuild fired. - Toggle filter to Bilinear: m_texture_filtering changes, which still fires shader_source_changed (filter is a distinct rebuild trigger). The Bilinear FS picks up its template normally, which always uses UV limits (settings coupling guarantee). - Cross-test on non-Vulkan backends: D3D11 / D3D12 / OpenGL also no longer rebuild on UV_LIMITS flip alone (the trigger drop in gpu_hw.cpp affects them too). Their shader sources were already invariant since8b43518, so the only behaviour change is the rebuild-trigger removal. Next: - PGXP_DEPTH for the FS (always-emit SV_Depth / FragDepth perf measurement first - the FS keeps captured m_pgxp_depth because the !m_pgxp_depth value gates DeclareFragmentEntryPoint's depth_output parameter, which controls whether SV_Depth is declared in the FS entry signature; routing that requires either always- emitting SV_Depth or hoisting that decision through the DeclareFragmentEntryPoint surface). - per_sample / dcp collapse decisions (forced-always via settings or keep as variant axes). - Pre-bake the resulting reduced D3D12 FS matrix.
308 lines
11 KiB
Python
Executable file
308 lines
11 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
# Regenerate embedded Vulkan SPIR-V blobs from GLSL sources.
|
|
#
|
|
# This script is NOT invoked by the core build. It is run by contributors who
|
|
# edit shader sources under data/shaders/vulkan/. The build itself only sees
|
|
# the resulting .inc files (checked into src/common/vulkan/embedded_spirv/),
|
|
# so the libretro core has zero glslangValidator dependency.
|
|
#
|
|
# Usage:
|
|
# python3 tools/regen_vulkan_spirv.py [--glslang PATH]
|
|
#
|
|
# Requires glslangValidator from the Vulkan SDK or the `glslang-tools`
|
|
# package on Debian / Ubuntu.
|
|
|
|
import argparse
|
|
import os
|
|
import re
|
|
import struct
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
|
GLSL_DIR = REPO_ROOT / "data" / "shaders" / "vulkan"
|
|
INC_DIR = REPO_ROOT / "src" / "common" / "vulkan" / "embedded_spirv"
|
|
|
|
# Map glsl file suffix to glslangValidator stage flag. Anything not in this
|
|
# table is skipped with a warning.
|
|
STAGE_FROM_SUFFIX = {
|
|
".vert.glsl": ("vert", "vs"),
|
|
".frag.glsl": ("frag", "fs"),
|
|
".comp.glsl": ("comp", "cs"),
|
|
".geom.glsl": ("geom", "gs"),
|
|
}
|
|
|
|
|
|
# Templates: shader sources that produce more than one .inc output, each
|
|
# with a different combination of -D preprocessor defines. Index ordering
|
|
# below MUST match the helper in src/common/vulkan/embedded_shaders.cpp
|
|
# that picks the right blob at runtime - if you add a variant here, add
|
|
# the matching entry there too.
|
|
#
|
|
# For each entry: filename -> [(variant_suffix, [-D defines]), ...]
|
|
def _batch_vs_variants():
|
|
out = []
|
|
# Attribute layout: untextured / textured (the latter always with
|
|
# UV limits since the UV_LIMITS-routing commit lifted the axis to
|
|
# the FS-side u_uv_limits cbuffer scalar). 2 layouts now.
|
|
attr_axes = [
|
|
("untextured", []),
|
|
("textured", ["TEXTURED"]),
|
|
]
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
for a_name, a_defs in attr_axes:
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
suffix = f"{a_name}_{i_name}_{p_name}"
|
|
out.append((suffix, a_defs + i_defs + p_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_untextured_variants():
|
|
out = []
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
# MUST match the batch VS chosen by the C++ side.
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective. MUST match batch VS.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
# Dual-source colour output: 1 attachment vs 2 (location 0
|
|
# index 0/1). Pipeline blend state references SRC1_* or it does
|
|
# not - per-call decision driven by transparency_mode AND
|
|
# m_supports_dual_source_blend.
|
|
dual_axes = [
|
|
("nodual", []),
|
|
("dual", ["DUAL_SOURCE"]),
|
|
]
|
|
# PGXP depth axis: when set, the FS omits gl_FragDepth so the
|
|
# rasterizer-interpolated VS depth (a_pos.w under PGXP) is used
|
|
# directly and early-Z stays available. Per-session.
|
|
pgxp_axes = [
|
|
("pgxpoff", []),
|
|
("pgxpon", ["PGXP_DEPTH"]),
|
|
]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_textured_nearest_variants():
|
|
out = []
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
|
|
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
|
|
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
|
|
# UV_LIMITS used to be a fifth axis here. The collapse commit lifted
|
|
# it to the u_uv_limits cbuffer scalar so the Nearest FS source
|
|
# always declares v_uv_limits and runtime-branches on the cbuffer
|
|
# value. Brings the Nearest FS cube into parity shape with the
|
|
# Bilinear / JINC2 / xBR families (UV_LIMITS implicit on those by
|
|
# settings-layer coupling).
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_textured_filter_variants():
|
|
"""Shared structural cube for non-Nearest filter templates.
|
|
|
|
UV_LIMITS is implicit (ShouldUseUVLimits() forces it true for any
|
|
non-Nearest filter), so the cube collapses to 4 axes vs the 5
|
|
Nearest template uses. 3 x 2 x 2 x 2 = 24 blobs.
|
|
"""
|
|
out = []
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
|
|
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
|
|
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
TEMPLATE_VARIANTS = {
|
|
"batch.vert.glsl": _batch_vs_variants(),
|
|
"batch_untextured.frag.glsl": _batch_fs_untextured_variants(),
|
|
"batch_textured_nearest.frag.glsl": _batch_fs_textured_nearest_variants(),
|
|
"batch_textured_bilinear.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
"batch_textured_jinc2.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
"batch_textured_xbr.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
}
|
|
|
|
|
|
def find_glslang(explicit):
|
|
if explicit:
|
|
return explicit
|
|
for name in ("glslangValidator", "glslangValidator.exe"):
|
|
from shutil import which
|
|
path = which(name)
|
|
if path:
|
|
return path
|
|
sys.stderr.write(
|
|
"error: glslangValidator not found on PATH. Install the Vulkan SDK\n"
|
|
" or the glslang-tools package, or pass --glslang PATH.\n")
|
|
sys.exit(1)
|
|
|
|
|
|
def sanitize_identifier(stem):
|
|
# Map "screen_quad.vert" -> "screen_quad_vs"; only [A-Za-z0-9_] survive.
|
|
name = re.sub(r"[^A-Za-z0-9_]", "_", stem)
|
|
return name
|
|
|
|
|
|
def compile_one(glslang, glsl_path, stage_flag, defines=None):
|
|
# Write SPIR-V to a temp file under INC_DIR so we don't pollute /tmp on
|
|
# restricted-filesystem hosts.
|
|
spv_path = INC_DIR / (glsl_path.stem + ".spv.tmp")
|
|
cmd = [
|
|
glslang,
|
|
"--target-env", "vulkan1.0",
|
|
"-S", stage_flag,
|
|
"-V", str(glsl_path),
|
|
"-o", str(spv_path),
|
|
]
|
|
for d in defines or ():
|
|
cmd.append(f"-D{d}")
|
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
|
if result.returncode != 0:
|
|
sys.stderr.write(f"glslangValidator failed for {glsl_path.name}:\n")
|
|
sys.stderr.write(result.stdout)
|
|
sys.stderr.write(result.stderr)
|
|
if spv_path.exists():
|
|
spv_path.unlink()
|
|
sys.exit(1)
|
|
data = spv_path.read_bytes()
|
|
spv_path.unlink()
|
|
if len(data) % 4 != 0:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced {len(data)} bytes "
|
|
f"of SPIR-V (not a multiple of 4)\n")
|
|
sys.exit(1)
|
|
words = struct.unpack(f"<{len(data) // 4}I", data)
|
|
if not words or words[0] != 0x07230203:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced invalid SPIR-V (bad magic)\n")
|
|
sys.exit(1)
|
|
return words
|
|
|
|
|
|
def emit_inc(identifier, glsl_name, words, out_path):
|
|
# 8 words per line keeps the file diff-friendly without being too tall.
|
|
PER_LINE = 8
|
|
lines = []
|
|
lines.append(
|
|
f"// Autogenerated from data/shaders/vulkan/{glsl_name}.\n"
|
|
f"// DO NOT EDIT. Regenerate with tools/regen_vulkan_spirv.py.\n"
|
|
f"// glslangValidator --target-env vulkan1.0\n"
|
|
f"//\n"
|
|
f"// Included inside namespace Vulkan::EmbeddedShaders in\n"
|
|
f"// embedded_shaders.cpp. The matching extern declaration lives in\n"
|
|
f"// embedded_shaders.h.\n"
|
|
f"\n"
|
|
f"const uint32_t k_{identifier}[] = {{\n")
|
|
for i in range(0, len(words), PER_LINE):
|
|
chunk = ", ".join(f"0x{w:08x}u" for w in words[i:i + PER_LINE])
|
|
lines.append(f" {chunk},\n")
|
|
lines.append(f"}};\n"
|
|
f"const size_t k_{identifier}_size_bytes = "
|
|
f"sizeof(k_{identifier});\n")
|
|
out_path.write_text("".join(lines))
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("--glslang", help="path to glslangValidator binary")
|
|
args = ap.parse_args()
|
|
glslang = find_glslang(args.glslang)
|
|
INC_DIR.mkdir(parents=True, exist_ok=True)
|
|
|
|
glsl_files = sorted(GLSL_DIR.glob("*.glsl"))
|
|
if not glsl_files:
|
|
sys.stderr.write(f"warning: no .glsl files under {GLSL_DIR}\n")
|
|
return
|
|
|
|
total = 0
|
|
for glsl_path in glsl_files:
|
|
# glsl_path.name looks like "screen_quad.vert.glsl".
|
|
suffix = None
|
|
for k in STAGE_FROM_SUFFIX:
|
|
if glsl_path.name.endswith(k):
|
|
suffix = k
|
|
break
|
|
if suffix is None:
|
|
sys.stderr.write(
|
|
f"warning: {glsl_path.name} has no recognised stage suffix; "
|
|
f"skipping\n")
|
|
continue
|
|
stage_flag, ident_suffix = STAGE_FROM_SUFFIX[suffix]
|
|
|
|
# "screen_quad.vert.glsl" -> stem "screen_quad.vert" -> "screen_quad"
|
|
base = glsl_path.name[:-len(suffix)]
|
|
|
|
variants = TEMPLATE_VARIANTS.get(glsl_path.name)
|
|
if variants is None:
|
|
# Single-variant shader: one .inc, no -D flags.
|
|
identifier = sanitize_identifier(base) + "_" + ident_suffix
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
else:
|
|
# Template: emit one .inc per variant, named with a suffix
|
|
# appended after the stage identifier.
|
|
for variant_suffix, defines in variants:
|
|
identifier = (sanitize_identifier(base) + "_" + ident_suffix +
|
|
"_" + variant_suffix)
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag, defines)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
|
|
print(f"regenerated {total} SPIR-V blob(s) in {INC_DIR}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|