Patch 14 in the glslang-elimination series. Second slice of the
batch FS conversion: the textured + Nearest-filter path
(texture_mode != Disabled AND m_texture_filter == Nearest). The
runtime decision in GetBatchFragmentShader now routes both the
untextured slice (patch 13) and this slice through the embedded
blob set; only textured non-Nearest still goes through
shadergen + glslang.
Five SPIR-V-structural axes (48 blobs total = 3 x 2 x 2 x 2 x 2):
- Input interpolation qualifier (none / centroid / sample). 3.
- Color input perspective (standard / noperspective). 2.
- Dual-source output (1 vs 2 outputs). 2.
- PGXP depth output (writes gl_FragDepth or omits it). 2.
- UV limits (without / with v_uv_limits flat input). 2.
The UV_LIMITS axis is new vs the untextured slice - it controls
whether v_uv_limits is declared as a per-vertex flat input on the
FS, matching the per-session m_using_uv_limits choice that already
drives the matching batch VS structural pick. Without UV limits the
FS skips the per-sample clamp.
Three new per-call specialisation constants relative to the
untextured slice cover the textured texture_mode encoding:
id = 107 PALETTE_4_BIT (bool, actual_texture_mode == 0)
id = 108 PALETTE_8_BIT (bool, actual_texture_mode == 1)
id = 109 RAW_TEXTURE (bool, RawTextureBit set in texture_mode)
PALETTE = PALETTE_4_BIT || PALETTE_8_BIT is derived inside the
shader. The texture_mode bit layout is described in
src/core/gpu_types.h; the C++ side derives the three boolean spec
const values from the per-call texture_mode in GetBatchPipeline.
Untextured blobs do not declare 107-109; passing them is harmless
per the Vulkan spec (unrecognised spec const IDs in the supplied
VkSpecializationInfo are silently dropped).
The shader body covers the helpers the shadergen path inlines:
ApplyTextureWindow, ApplyUpscaledTextureWindow, FloatToIntegerCoords,
RGBA8ToRGBA5551, and SampleFromVRAM. SampleFromVRAM splits on the
derived PALETTE flag and, within the palette branch, on
PALETTE_4_BIT vs PALETTE_8_BIT to do the correct sub-texel address
arithmetic. fixYCoord on Vulkan is the identity function so we use
the coord directly. The Vulkan-only API_VULKAN-y-flip in the VS
already accounts for sample-space orientation.
Output and discard logic mirrors the shadergen TEXTURED branches:
the no-TRANSPARENCY case writes o_col0 (and o_col1 when DUAL_SOURCE)
unconditionally; the TRANSPARENCY case branches on the texel's
semitransparency bit and applies the TRANSPARENCY_ONLY_OPAQUE /
TRANSPARENCY_ONLY_TRANSPARENT per-texel discard the way the
shadergen does.
Helper-side index encoding for the 48-blob array, matching
GetBatchTexturedNearestFragmentShaderBlob in embedded_shaders.cpp:
interp = per_sample_shading ? 2 : (msaa ? 1 : 0) (0..2)
persp = noperspective_color ? 1 : 0 (0..1)
dual = dual_source ? 1 : 0 (0..1)
pgxp = pgxp_depth ? 1 : 0 (0..1)
uv = uv_limits ? 1 : 0 (0..1)
index = interp*16 + persp*8 + dual*4 + pgxp*2 + uv (0..47)
The use_dual_source derivation in GetBatchFragmentShader matches
the shadergen-side check exactly: supports_dual_source_blend &&
((render_mode is transparent) || filter != Nearest). For the two
embedded branches (untextured and textured Nearest) the filter-half
of the OR evaluates false, so both branches fall through to the
render_mode test.
Smoke tests:
- g++ -fsyntax-only on the touched common TU.
- spirv-val --target-env vulkan1.0 on all 48 blobs - 48/48 valid.
- Total .inc size: ~2.2 MiB across all 48 blobs (~46 KiB each in
text form, which is the 0xNNNNNNNNu, packed layout from the regen
tool; binary contribution to the linked output is roughly 1/4 of
that, so ~550 KiB. Acceptable for a renderer that previously
pulled in glslang at runtime).
- --target-env vulkan1.0 vs un-flagged glslangValidator: byte-
identical for every variant.
Test priority:
- Routine smoke under Enabled / Lazy / Disabled with Nearest
filter (the default). Any textured polygon in any game now
exercises the embedded path: dialogue portraits, sprite layers,
background textures - the vast majority of pixels in most
games.
- Toggle PGXP UV correction (drives m_using_uv_limits) to flip
the UV_LIMITS structural axis. Games that benefit most from
PGXP corrections - Final Fantasy series, Tekken 3, Soul Blade -
are good exercise targets.
- Antialiasing toggle round-trip under Disabled mode per the
patch-5-fix protocol (flips interpolation axis).
- Each texture mode is exercised by different draws: palette
textures dominate (most game art is 4-bit indexed), with
direct-16bpp showing up for render-to-texture effects. Raw
textures appear in framebuffer effects and some UI passes.
- Cross-check with shadergen path by switching texture filter to
Bilinear (still on the shadergen path) and back to Nearest -
rendering should remain consistent across the switch.
After this lands, the only remaining glslang consumers in the
Vulkan backend are the texture-filter slices (Bilinear /
BilinearBinAlpha, JINC2 / JINC2BinAlpha, xBR / xBRBinAlpha). Each
of those filter families has its own FilteredSampleFromVRAM body
and will get its own template in subsequent patches.
275 lines
9.9 KiB
Python
Executable file
275 lines
9.9 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
# Regenerate embedded Vulkan SPIR-V blobs from GLSL sources.
|
|
#
|
|
# This script is NOT invoked by the core build. It is run by contributors who
|
|
# edit shader sources under data/shaders/vulkan/. The build itself only sees
|
|
# the resulting .inc files (checked into src/common/vulkan/embedded_spirv/),
|
|
# so the libretro core has zero glslangValidator dependency.
|
|
#
|
|
# Usage:
|
|
# python3 tools/regen_vulkan_spirv.py [--glslang PATH]
|
|
#
|
|
# Requires glslangValidator from the Vulkan SDK or the `glslang-tools`
|
|
# package on Debian / Ubuntu.
|
|
|
|
import argparse
|
|
import os
|
|
import re
|
|
import struct
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
|
GLSL_DIR = REPO_ROOT / "data" / "shaders" / "vulkan"
|
|
INC_DIR = REPO_ROOT / "src" / "common" / "vulkan" / "embedded_spirv"
|
|
|
|
# Map glsl file suffix to glslangValidator stage flag. Anything not in this
|
|
# table is skipped with a warning.
|
|
STAGE_FROM_SUFFIX = {
|
|
".vert.glsl": ("vert", "vs"),
|
|
".frag.glsl": ("frag", "fs"),
|
|
".comp.glsl": ("comp", "cs"),
|
|
".geom.glsl": ("geom", "gs"),
|
|
}
|
|
|
|
|
|
# Templates: shader sources that produce more than one .inc output, each
|
|
# with a different combination of -D preprocessor defines. Index ordering
|
|
# below MUST match the helper in src/common/vulkan/embedded_shaders.cpp
|
|
# that picks the right blob at runtime - if you add a variant here, add
|
|
# the matching entry there too.
|
|
#
|
|
# For each entry: filename -> [(variant_suffix, [-D defines]), ...]
|
|
def _batch_vs_variants():
|
|
out = []
|
|
# Attribute layout: untextured / textured / textured + UV limits.
|
|
attr_axes = [
|
|
("untextured", []),
|
|
("textured", ["TEXTURED"]),
|
|
("textured_uvlim", ["TEXTURED", "UV_LIMITS"]),
|
|
]
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
for a_name, a_defs in attr_axes:
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
suffix = f"{a_name}_{i_name}_{p_name}"
|
|
out.append((suffix, a_defs + i_defs + p_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_untextured_variants():
|
|
out = []
|
|
# Output interpolation: standard / centroid (MSAA) / sample (SSAA).
|
|
# MUST match the batch VS chosen by the C++ side.
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
# Color perspective: standard / noperspective. MUST match batch VS.
|
|
persp_axes = [
|
|
("persp", []),
|
|
("noperp", ["NOPERSP"]),
|
|
]
|
|
# Dual-source colour output: 1 attachment vs 2 (location 0
|
|
# index 0/1). Pipeline blend state references SRC1_* or it does
|
|
# not - per-call decision driven by transparency_mode AND
|
|
# m_supports_dual_source_blend.
|
|
dual_axes = [
|
|
("nodual", []),
|
|
("dual", ["DUAL_SOURCE"]),
|
|
]
|
|
# PGXP depth axis: when set, the FS omits gl_FragDepth so the
|
|
# rasterizer-interpolated VS depth (a_pos.w under PGXP) is used
|
|
# directly and early-Z stays available. Per-session.
|
|
pgxp_axes = [
|
|
("pgxpoff", []),
|
|
("pgxpon", ["PGXP_DEPTH"]),
|
|
]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs))
|
|
return out
|
|
|
|
|
|
def _batch_fs_textured_nearest_variants():
|
|
out = []
|
|
interp_axes = [
|
|
("none", []),
|
|
("centroid", ["INTERP_CENTROID"]),
|
|
("sample", ["INTERP_SAMPLE"]),
|
|
]
|
|
persp_axes = [("persp", []), ("noperp", ["NOPERSP"])]
|
|
dual_axes = [("nodual", []), ("dual", ["DUAL_SOURCE"])]
|
|
pgxp_axes = [("pgxpoff", []), ("pgxpon", ["PGXP_DEPTH"])]
|
|
uv_axes = [("nouv", []), ("uv", ["UV_LIMITS"])]
|
|
for i_name, i_defs in interp_axes:
|
|
for p_name, p_defs in persp_axes:
|
|
for d_name, d_defs in dual_axes:
|
|
for x_name, x_defs in pgxp_axes:
|
|
for u_name, u_defs in uv_axes:
|
|
suffix = f"{i_name}_{p_name}_{d_name}_{x_name}_{u_name}"
|
|
out.append((suffix, i_defs + p_defs + d_defs + x_defs + u_defs))
|
|
return out
|
|
|
|
|
|
TEMPLATE_VARIANTS = {
|
|
"batch.vert.glsl": _batch_vs_variants(),
|
|
"batch_untextured.frag.glsl": _batch_fs_untextured_variants(),
|
|
"batch_textured_nearest.frag.glsl": _batch_fs_textured_nearest_variants(),
|
|
}
|
|
|
|
|
|
def find_glslang(explicit):
|
|
if explicit:
|
|
return explicit
|
|
for name in ("glslangValidator", "glslangValidator.exe"):
|
|
from shutil import which
|
|
path = which(name)
|
|
if path:
|
|
return path
|
|
sys.stderr.write(
|
|
"error: glslangValidator not found on PATH. Install the Vulkan SDK\n"
|
|
" or the glslang-tools package, or pass --glslang PATH.\n")
|
|
sys.exit(1)
|
|
|
|
|
|
def sanitize_identifier(stem):
|
|
# Map "screen_quad.vert" -> "screen_quad_vs"; only [A-Za-z0-9_] survive.
|
|
name = re.sub(r"[^A-Za-z0-9_]", "_", stem)
|
|
return name
|
|
|
|
|
|
def compile_one(glslang, glsl_path, stage_flag, defines=None):
|
|
# Write SPIR-V to a temp file under INC_DIR so we don't pollute /tmp on
|
|
# restricted-filesystem hosts.
|
|
spv_path = INC_DIR / (glsl_path.stem + ".spv.tmp")
|
|
cmd = [
|
|
glslang,
|
|
"--target-env", "vulkan1.0",
|
|
"-S", stage_flag,
|
|
"-V", str(glsl_path),
|
|
"-o", str(spv_path),
|
|
]
|
|
for d in defines or ():
|
|
cmd.append(f"-D{d}")
|
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
|
if result.returncode != 0:
|
|
sys.stderr.write(f"glslangValidator failed for {glsl_path.name}:\n")
|
|
sys.stderr.write(result.stdout)
|
|
sys.stderr.write(result.stderr)
|
|
if spv_path.exists():
|
|
spv_path.unlink()
|
|
sys.exit(1)
|
|
data = spv_path.read_bytes()
|
|
spv_path.unlink()
|
|
if len(data) % 4 != 0:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced {len(data)} bytes "
|
|
f"of SPIR-V (not a multiple of 4)\n")
|
|
sys.exit(1)
|
|
words = struct.unpack(f"<{len(data) // 4}I", data)
|
|
if not words or words[0] != 0x07230203:
|
|
sys.stderr.write(
|
|
f"error: {glsl_path.name} produced invalid SPIR-V (bad magic)\n")
|
|
sys.exit(1)
|
|
return words
|
|
|
|
|
|
def emit_inc(identifier, glsl_name, words, out_path):
|
|
# 8 words per line keeps the file diff-friendly without being too tall.
|
|
PER_LINE = 8
|
|
lines = []
|
|
lines.append(
|
|
f"// Autogenerated from data/shaders/vulkan/{glsl_name}.\n"
|
|
f"// DO NOT EDIT. Regenerate with tools/regen_vulkan_spirv.py.\n"
|
|
f"// glslangValidator --target-env vulkan1.0\n"
|
|
f"//\n"
|
|
f"// Included inside namespace Vulkan::EmbeddedShaders in\n"
|
|
f"// embedded_shaders.cpp. The matching extern declaration lives in\n"
|
|
f"// embedded_shaders.h.\n"
|
|
f"\n"
|
|
f"const uint32_t k_{identifier}[] = {{\n")
|
|
for i in range(0, len(words), PER_LINE):
|
|
chunk = ", ".join(f"0x{w:08x}u" for w in words[i:i + PER_LINE])
|
|
lines.append(f" {chunk},\n")
|
|
lines.append(f"}};\n"
|
|
f"const size_t k_{identifier}_size_bytes = "
|
|
f"sizeof(k_{identifier});\n")
|
|
out_path.write_text("".join(lines))
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("--glslang", help="path to glslangValidator binary")
|
|
args = ap.parse_args()
|
|
glslang = find_glslang(args.glslang)
|
|
INC_DIR.mkdir(parents=True, exist_ok=True)
|
|
|
|
glsl_files = sorted(GLSL_DIR.glob("*.glsl"))
|
|
if not glsl_files:
|
|
sys.stderr.write(f"warning: no .glsl files under {GLSL_DIR}\n")
|
|
return
|
|
|
|
total = 0
|
|
for glsl_path in glsl_files:
|
|
# glsl_path.name looks like "screen_quad.vert.glsl".
|
|
suffix = None
|
|
for k in STAGE_FROM_SUFFIX:
|
|
if glsl_path.name.endswith(k):
|
|
suffix = k
|
|
break
|
|
if suffix is None:
|
|
sys.stderr.write(
|
|
f"warning: {glsl_path.name} has no recognised stage suffix; "
|
|
f"skipping\n")
|
|
continue
|
|
stage_flag, ident_suffix = STAGE_FROM_SUFFIX[suffix]
|
|
|
|
# "screen_quad.vert.glsl" -> stem "screen_quad.vert" -> "screen_quad"
|
|
base = glsl_path.name[:-len(suffix)]
|
|
|
|
variants = TEMPLATE_VARIANTS.get(glsl_path.name)
|
|
if variants is None:
|
|
# Single-variant shader: one .inc, no -D flags.
|
|
identifier = sanitize_identifier(base) + "_" + ident_suffix
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
else:
|
|
# Template: emit one .inc per variant, named with a suffix
|
|
# appended after the stage identifier.
|
|
for variant_suffix, defines in variants:
|
|
identifier = (sanitize_identifier(base) + "_" + ident_suffix +
|
|
"_" + variant_suffix)
|
|
out_path = INC_DIR / (identifier + ".inc")
|
|
words = compile_one(glslang, glsl_path, stage_flag, defines)
|
|
emit_inc(identifier, glsl_path.name, words, out_path)
|
|
size_kb = (len(words) * 4) / 1024.0
|
|
print(f" {glsl_path.name:<40} -> {out_path.name} "
|
|
f"({len(words)} words, {size_kb:.1f} KiB)")
|
|
total += 1
|
|
|
|
print(f"regenerated {total} SPIR-V blob(s) in {INC_DIR}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|