Patch 15 in the glslang-elimination series. Final batch FS conversion
covering all three remaining texture filter families:
- Bilinear / BilinearBinAlpha (24 blobs)
- JINC2 / JINC2BinAlpha (24 blobs)
- xBR / xBRBinAlpha (24 blobs)
After this lands, every batch FS slot in m_batch_fragment_shaders is
filled from a pre-baked SPIR-V blob; the runtime shadergen path for
batch shaders is gone. The m_shadergen unique_ptr and its
construction in CompilePipelines are removed since they have no
consumers left.
Each filter family shares the same 4-axis structural cube as the
patch-15-Bilinear-prep work that immediately preceded this commit:
- Input interpolation qualifier (none / centroid / sample). 3.
- Color input perspective (standard / noperspective). 2.
- Dual-source output (1 vs 2 outputs). 2.
- PGXP depth output (writes gl_FragDepth or omits it). 2.
3 x 2 x 2 x 2 = 24 blobs per filter family. UV_LIMITS is implicit
for all non-Nearest filter families because ShouldUseUVLimits() in
gpu_hw.cpp forces m_using_uv_limits = true whenever
m_texture_filtering != Nearest. The *BinAlpha variant of each
family collapses into the BINALPHA per-call specialisation constant
(id=110) rather than a structural axis - the only SPIR-V difference
is a single ialpha-quantisation step at the end of
FilteredSampleFromVRAM.
Filter bodies are lifted from gpu_hw_shadergen.cpp WriteBatchTexture-
Filter:
- Bilinear lines ~188-224 (4-tap weighted average)
- JINC2 lines ~229-374 (4x4-tap sinc-windowed with anti-ring)
- xBR lines ~381-661 (5x5 neighbourhood, YCbCr edge
detection with BLEND_NONE / NORMAL /
DOMINANT directions per quadrant)
The xBR template is the largest of the three (~17 KiB per blob in
binary, the body is ~280 lines of GLSL). The translation from the
shadergen DSL is mechanical: float4 -> vec4, lerp -> mix, frac ->
fract, sign -> sign, VECTOR_EQ -> all(equal(...)), VECTOR_NEQ ->
any(notEqual(...)). The P() helper macro on xBR closes over coords
and uv_limits the same way the shadergen original does.
One xBR quirk is preserved verbatim: the Iw flag computation in
shadergen uses VECTOR_NEQ(H, ...) instead of VECTOR_NEQ(I, ...)
which is almost certainly a bug, but the alternative output would
differ visually from the released shadergen path. Preserving it
keeps the embedded-vs-shadergen output byte-identical for compat;
fixing it can be a follow-up patch with its own visual-test cycle.
GetBatchFragmentShader is now a switch on m_texture_filtering with
no fallback: every filter resolves to one of the embedded helpers.
The 'dual_source = ... || filter != Nearest' branch is preserved
verbatim from the shadergen-side derivation.
GenerateBatchFragmentShader and the m_shadergen unique_ptr are no
longer referenced. The construction at CompilePipelines is removed
along with the reset in DestroyPipelines and the declaration in
the header. The 'gpu_hw_shadergen.h' include is dropped from both
gpu_hw_vulkan.cpp and gpu_hw_vulkan.h. shader_compiler.{h,cpp},
dep/glslang/, and the Makefile / CMake glslang references are NOT
removed in this patch - that final cleanup will land separately
once this patch has been confirmed working in the wild, on the
theory that backing it out is easier with the runtime path's
support files still present.
Smoke tests:
- g++ -fsyntax-only on the touched common TU.
- spirv-val --target-env vulkan1.0 on all 72 new blobs: 72/72 valid.
- Total .inc text size for the three filter families: ~9.2 MiB
(Bilinear ~1.4 MiB, JINC2 ~2.6 MiB, xBR ~4.7 MiB). Binary
contribution is ~25% of those numbers, so adding ~2.3 MiB to
the .so. xBR alone is ~1.2 MiB binary contribution; this is the
largest single chunk of pre-baked SPIR-V in the project.
Test priority - this patch covers the actively-rendered path for
users who enable any non-Nearest filter, so:
- **Bilinear / BilinearBinAlpha**: smooth edges on textured
surfaces, slight softening of palette textures. Test by setting
the texture filter core option to Bilinear / Bilinear (BinAlpha).
- **JINC2 / JINC2BinAlpha**: sharper sampling than Bilinear with
visible anti-ringing on textured polygons. Heavier per-pixel
cost; testing on FF8/9 backgrounds, MGS textured screens, etc.
- **xBR / xBRBinAlpha**: pixel-art upscaler, very visible smooth
edges on 2D sprites. Best exercise targets are 2D-heavy games -
Castlevania SOTN, Symphony of the Night, Mega Man X4-6, the
Suikoden series. xBR is the most complex shader in the project
and produces visually-distinctive output that immediately
reveals miscompilation.
- All three families x every precompile mode (Disabled / Lazy /
Enabled) x antialiasing toggle round-trips.
- Cross-check: switch m_texture_filtering at runtime between all
six filter values plus Nearest. Each transition should produce
visibly-correct rendering without intermediate corruption.
After this lands and is confirmed working, the final patch in the
series will remove dep/glslang/, src/common/vulkan/shader_compiler
.{h,cpp}, and the Makefile.libretro / CMakeLists.txt entries that
pull glslang into the link.
303 lines
11 KiB
Python
Executable file
303 lines
11 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
# Regenerate embedded Vulkan SPIR-V blobs from GLSL sources.
|
|
#
|
|
# This script is NOT invoked by the core build. It is run by contributors who
|
|
# edit shader sources under data/shaders/vulkan/. The build itself only sees
|
|
# the resulting .inc files (checked into src/common/vulkan/embedded_spirv/),
|
|
# so the libretro core has zero glslangValidator dependency.
|
|
#
|
|
# Usage:
|
|
# python3 tools/regen_vulkan_spirv.py [--glslang PATH]
|
|
#
|
|
# Requires glslangValidator from the Vulkan SDK or the `glslang-tools`
|
|
# package on Debian / Ubuntu.
|
|
|
|
import argparse
|
|
import os
|
|
import re
|
|
import struct
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
|
GLSL_DIR = REPO_ROOT / "data" / "shaders" / "vulkan"
|
|
INC_DIR = REPO_ROOT / "src" / "common" / "vulkan" / "embedded_spirv"
|
|
|
|
# Map glsl file suffix to glslangValidator stage flag. Anything not in this
|
|
# table is skipped with a warning.
|
|
STAGE_FROM_SUFFIX = {
|
|
".vert.glsl": ("vert", "vs"),
|
|
".frag.glsl": ("frag", "fs"),
|
|
".comp.glsl": ("comp", "cs"),
|
|
".geom.glsl": ("geom", "gs"),
|
|
}
|
|
|
|
|
|
# Templates: shader sources that produce more than one .inc output, each
|
|
# with a different combination of -D preprocessor defines. Index ordering
|
|
# below MUST match the helper in src/common/vulkan/embedded_shaders.cpp
|
|
# that picks the right blob at runtime - if you add a variant here, add
|
|
# the matching entry there too.
|
|
#
|
|
# For each entry: filename -> [(variant_suffix, [-D defines]), ...]
|
|
def _batch_vs_variants():
|
|
out = []
|
|
# Attribute layout: untextured / textured / textured + UV limits.
|
|
attr_axes = [
|
|
("untextured", []),
|
|
("textured", ["TEXTURED"]),
|
|
("textured_uvlim", ["TEXTURED", "UV_LIMITS"]),
|
|
]
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
for a_name, a_defs in attr_axes:
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
suffix = f"{a_name}_{i_name}_{p_name}"
|
|
out.append((suffix, a_defs + i_defs + p_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_untextured_variants():
|
|
out = []
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
# MUST match the batch VS chosen by the C++ side.
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective. MUST match batch VS.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
# Dual-source colour output: 1 attachment vs 2 (location 0
|
|
# index 0/1). Pipeline blend state references SRC1_* or it does
|
|
# not - per-call decision driven by transparency_mode AND
|
|
# m_supports_dual_source_blend.
|
|
dual_axes = [
|
|
("nodual", []),
|
|
("dual", ["DUAL_SOURCE"]),
|
|
]
|
|
# PGXP depth axis: when set, the FS omits gl_FragDepth so the
|
|
# rasterizer-interpolated VS depth (a_pos.w under PGXP) is used
|
|
# directly and early-Z stays available. Per-session.
|
|
pgxp_axes = [
|
|
("pgxpoff", []),
|
|
("pgxpon", ["PGXP_DEPTH"]),
|
|
]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_textured_nearest_variants():
|
|
out = []
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
|
|
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
|
|
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
|
|
uv_axes = [("nouv", []), ("uv", ["UV_LIMITS"])]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
for u_name, u_defs in uv_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}_{u_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs + u_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_textured_filter_variants():
|
|
"""Shared structural cube for non-Nearest filter templates.
|
|
|
|
UV_LIMITS is implicit (ShouldUseUVLimits() forces it true for any
|
|
non-Nearest filter), so the cube collapses to 4 axes vs the 5
|
|
Nearest template uses. 3 x 2 x 2 x 2 = 24 blobs.
|
|
"""
|
|
out = []
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
|
|
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
|
|
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
TEMPLATE_VARIANTS = {
|
|
"batch.vert.glsl": _batch_vs_variants(),
|
|
"batch_untextured.frag.glsl": _batch_fs_untextured_variants(),
|
|
"batch_textured_nearest.frag.glsl": _batch_fs_textured_nearest_variants(),
|
|
"batch_textured_bilinear.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
"batch_textured_jinc2.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
"batch_textured_xbr.frag.glsl": _batch_fs_textured_filter_variants(),
|
|
}
|
|
|
|
|
|
def find_glslang(explicit):
|
|
if explicit:
|
|
return explicit
|
|
for name in ("glslangValidator", "glslangValidator.exe"):
|
|
from shutil import which
|
|
path = which(name)
|
|
if path:
|
|
return path
|
|
sys.stderr.write(
|
|
"error: glslangValidator not found on PATH. Install the Vulkan SDK\n"
|
|
" or the glslang-tools package, or pass --glslang PATH.\n")
|
|
sys.exit(1)
|
|
|
|
|
|
def sanitize_identifier(stem):
|
|
# Map "screen_quad.vert" -> "screen_quad_vs"; only [A-Za-z0-9_] survive.
|
|
name = re.sub(r"[^A-Za-z0-9_]", "_", stem)
|
|
return name
|
|
|
|
|
|
def compile_one(glslang, glsl_path, stage_flag, defines=None):
|
|
# Write SPIR-V to a temp file under INC_DIR so we don't pollute /tmp on
|
|
# restricted-filesystem hosts.
|
|
spv_path = INC_DIR / (glsl_path.stem + ".spv.tmp")
|
|
cmd = [
|
|
glslang,
|
|
"--target-env", "vulkan1.0",
|
|
"-S", stage_flag,
|
|
"-V", str(glsl_path),
|
|
"-o", str(spv_path),
|
|
]
|
|
for d in defines or ():
|
|
cmd.append(f"-D{d}")
|
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
|
if result.returncode != 0:
|
|
sys.stderr.write(f"glslangValidator failed for {glsl_path.name}:\n")
|
|
sys.stderr.write(result.stdout)
|
|
sys.stderr.write(result.stderr)
|
|
if spv_path.exists():
|
|
spv_path.unlink()
|
|
sys.exit(1)
|
|
data = spv_path.read_bytes()
|
|
spv_path.unlink()
|
|
if len(data) % 4 != 0:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced {len(data)} bytes "
|
|
f"of SPIR-V (not a multiple of 4)\n")
|
|
sys.exit(1)
|
|
words = struct.unpack(f"<{len(data) // 4}I", data)
|
|
if not words or words[0] != 0x07230203:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced invalid SPIR-V (bad magic)\n")
|
|
sys.exit(1)
|
|
return words
|
|
|
|
|
|
def emit_inc(identifier, glsl_name, words, out_path):
|
|
# 8 words per line keeps the file diff-friendly without being too tall.
|
|
PER_LINE = 8
|
|
lines = []
|
|
lines.append(
|
|
f"// Autogenerated from data/shaders/vulkan/{glsl_name}.\n"
|
|
f"// DO NOT EDIT. Regenerate with tools/regen_vulkan_spirv.py.\n"
|
|
f"// glslangValidator --target-env vulkan1.0\n"
|
|
f"//\n"
|
|
f"// Included inside namespace Vulkan::EmbeddedShaders in\n"
|
|
f"// embedded_shaders.cpp. The matching extern declaration lives in\n"
|
|
f"// embedded_shaders.h.\n"
|
|
f"\n"
|
|
f"const uint32_t k_{identifier}[] = {{\n")
|
|
for i in range(0, len(words), PER_LINE):
|
|
chunk = ", ".join(f"0x{w:08x}u" for w in words[i:i + PER_LINE])
|
|
lines.append(f" {chunk},\n")
|
|
lines.append(f"}};\n"
|
|
f"const size_t k_{identifier}_size_bytes = "
|
|
f"sizeof(k_{identifier});\n")
|
|
out_path.write_text("".join(lines))
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("--glslang", help="path to glslangValidator binary")
|
|
args = ap.parse_args()
|
|
glslang = find_glslang(args.glslang)
|
|
INC_DIR.mkdir(parents=True, exist_ok=True)
|
|
|
|
glsl_files = sorted(GLSL_DIR.glob("*.glsl"))
|
|
if not glsl_files:
|
|
sys.stderr.write(f"warning: no .glsl files under {GLSL_DIR}\n")
|
|
return
|
|
|
|
total = 0
|
|
for glsl_path in glsl_files:
|
|
# glsl_path.name looks like "screen_quad.vert.glsl".
|
|
suffix = None
|
|
for k in STAGE_FROM_SUFFIX:
|
|
if glsl_path.name.endswith(k):
|
|
suffix = k
|
|
break
|
|
if suffix is None:
|
|
sys.stderr.write(
|
|
f"warning: {glsl_path.name} has no recognised stage suffix; "
|
|
f"skipping\n")
|
|
continue
|
|
stage_flag, ident_suffix = STAGE_FROM_SUFFIX[suffix]
|
|
|
|
# "screen_quad.vert.glsl" -> stem "screen_quad.vert" -> "screen_quad"
|
|
base = glsl_path.name[:-len(suffix)]
|
|
|
|
variants = TEMPLATE_VARIANTS.get(glsl_path.name)
|
|
if variants is None:
|
|
# Single-variant shader: one .inc, no -D flags.
|
|
identifier = sanitize_identifier(base) + "_" + ident_suffix
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
else:
|
|
# Template: emit one .inc per variant, named with a suffix
|
|
# appended after the stage identifier.
|
|
for variant_suffix, defines in variants:
|
|
identifier = (sanitize_identifier(base) + "_" + ident_suffix +
|
|
"_" + variant_suffix)
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag, defines)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
|
|
print(f"regenerated {total} SPIR-V blob(s) in {INC_DIR}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|