TsundereStation/data/shaders/d3d12
LibretroAdmin cd971cd1d2 core/d3d12: pre-bake the VRAM copy pixel shader
First multi-variant pre-bake, exercising the TEMPLATE_VARIANTS
infrastructure in tools/regen_d3d12_dxbc.py for the first time.
vram_copy_ps is the smallest cleanly-pre-bakeable non-batch shader
after the e56d4d4 RESOLUTION_SCALE refactor took it from
PGXP x N variants down to just PGXP (2 blobs). Pure body-branch
variant axis - no texture binding changes, no MSAA split, no
input-layout differences. Body is small (~30 lines) and the
runtime call site is one function (GetVRAMCopyPipeline) in one
backend.

Why pre-bake vram_copy_ps:

  * It's hit constantly during gameplay. Every PSX game uses
    VRAM-to-VRAM copies for sprite caching (Final Fantasy 7's
    pre-rendered backgrounds streaming sprites in, Tekken 3's
    text rendering between rounds, every fighting game's HUD
    composite, every JRPG's menu system).
  * After e56d4d4 the only variant axis is PGXP_DEPTH on/off, so
    2 blobs cover every (m_pgxp_depth_buffer, m_resolution_scale)
    combination - cleanest possible variant count.
  * It's the natural exercise for TEMPLATE_VARIANTS (which has
    sat empty since the regen tool was first scaffolded at
    ca8c9b0). Subsequent multi-variant pre-bakes (vram_write 4
    variants, vram_read M MSAA variants, display 12 x MSAA
    variants) reuse the same pattern.

Files added:

* data/shaders/d3d12/vram_copy.ps.hlsl: standalone HLSL source
  matching the GenerateVRAMCopyFragmentShader output for D3D12
  post-e56d4d4. The cbuffer layout matches VRAMCopyUBOData
  member-for-member (u_src_coords, u_dst_coords, u_end_coords,
  u_size, u_set_mask_bit, u_depth_value, u_resolution_scale,
  u_pad0 = 48 bytes). VRAM_SIZE alias inlined since the helper
  emission isn't accessible to standalone HLSL.
  PGXP_DEPTH variant is a single #if in the body controlling
  the o_depth assignment. MSAA path omitted entirely - vram_copy
  never runs in MSAA mode because the source texture is bound
  as both shader-resource AND framebuffer-output, which D3D12
  doesn't permit with an MSAA resource; the runtime uses the
  non-MSAA m_vram_read_texture shadow.

* src/common/d3d12/embedded_dxbc/vram_copy_ps_pgxp0.inc: DXBC
  blob for /DPGXP_DEPTH=0 (PGXP off, conditional o_depth).
  ~3.1 KB of bytecode.

* src/common/d3d12/embedded_dxbc/vram_copy_ps_pgxp1.inc: DXBC
  blob for /DPGXP_DEPTH=1 (PGXP on, o_depth = 1.0). ~3.0 KB.

Files modified:

* tools/regen_d3d12_dxbc.py: TEMPLATE_VARIANTS dict populated
  with the vram_copy.ps.hlsl entry. Two variant rows:
    ("pgxp0", ["PGXP_DEPTH=0"])
    ("pgxp1", ["PGXP_DEPTH=1"])
  The dispatch loop at line 232-238 (untouched, pre-existed
  from ca8c9b0) does the right thing: emit one .inc per row,
  identifier suffix from the variant_suffix string.

* src/common/d3d12/embedded_shaders.h: extern declarations
  added for the two new blobs (k_vram_copy_ps_pgxp0,
  k_vram_copy_ps_pgxp0_size_bytes, and pgxp1 pair). Each pair
  matches the regen tool's naming convention.

* src/common/d3d12/embedded_shaders.cpp: two #include lines
  added inside namespace D3D12::EmbeddedShaders, mirroring
  what was done for fullscreen_quad_vs + copy_ps. The .inc
  files define the storage; the prior extern declarations in
  the header give them external linkage.

* src/core/gpu_hw_d3d12.cpp GetVRAMCopyPipeline: dropped the
  shadergen-call + m_shader_cache.GetPixelShader path; replaced
  with an inline D3D12_SHADER_BYTECODE constructed from the
  embedded blob selected via m_pgxp_depth_buffer. Same PSO-
  creation flow downstream (gpbuilder, MSAA state, depth state,
  shader-cache PSO disk-cache via gpbuilder.Create). The
  m_shadergen-not-yet-constructed check is dropped for this
  path since the embedded blobs are statically linked - no
  ordering dependency on shadergen lifetime.

Toolchain confirmation:
  Built with Wine's d3dcompiler_47.dll via /tmp/fxc_lite/fxc_lite.exe
  on the contributor side (same tooling as 77b255f / 443a2a1).
  DXBC magic verified at regen time (the 'DXBC' signature in the
  first 4 bytes; the regen tool aborts the build if that doesn't
  match).

Cost matrix delta (D3D12 specifically):
  vram_copy_ps D3DCompile cost on cold-cache flip: ~50 ms each
  for the two PGXP variants -> 0 ms after this commit. The PSO
  compile itself still runs through gpbuilder.Create (~10 ms),
  so total cold-flip-of-PGXP cost drops by ~50 ms per variant.
  Disk-cache-warm cost was already in the noise (~1 ms cache
  lookup); now it's literally zero.

  Progress tracker: non_batch_progress_units stays at 17 - we
  still tick once per (depth_test) GetVRAMCopyPipeline call
  inside the precompile_sync loop. PSO compile is still the
  unit; we just stopped doing the D3DCompile half of that
  work.

Variant count after this commit, for the cold-flip cost matrix
columns we care about, for vram_copy specifically:
  * resolution_scale  : 0 ms (cbuffer-routed since e56d4d4)
  * pgxp_depth        : 0 ms D3DCompile + ~10 ms PSO compile
  * multisamples      : 0 (not a vram_copy variant axis)
  * filter / chroma   : 0 (no relationship to vram_copy)

The shadergen GenerateVRAMCopyFragmentShader function is left
intact in gpu_hw_shadergen.cpp - the other three backends
(D3D11, OpenGL, Vulkan) still call it through their runtime
shader-cache paths. Only the D3D12 backend bypasses it now.
Future per-backend pre-bake work could extend the pattern
to those backends too (Vulkan via SPIR-V pre-bake, OpenGL via
something else - GLSL doesn't have a comparable offline-
compiled bytecode format; SPIR-V cross would be the route).

Test:
  - Build: D3D12 backend should link without unresolved symbols.
    The shadergen->GenerateVRAMCopyFragmentShader path is no
    longer reachable through GetVRAMCopyPipeline on D3D12 but
    the function is still defined and used by other backends.
  - Runtime: every VRAM copy that previously worked must still
    work. Concrete test - FF7 menu navigation (uses VRAM copy
    for menu sprite composition heavily), Tekken 3 round-
    transition text rendering (VRAM copy each frame to splat
    the round number / TOTAL DMG sprites). If the embedded
    blob were wrong, sprites and menu text would garble within
    a second of entering a menu or starting a round.
  - PGXP toggle: flip the "PGXP CPU mode" option on and off
    in RetroArch's core options. The first flip recompiles
    PSOs; the embedded blob path picks blob[1] vs blob[0] and
    builds fresh PSOs. Subsequent same-direction flips hit the
    PSO cache.
  - Resolution scale toggle: 1x <-> 4x. No PSO recompile (since
    the blob is scale-independent now); CopyVRAM next frame
    just pushes a different u_resolution_scale value through
    the cbuffer.

This is the first non-batch FRAGMENT pre-bake on top of the
RESOLUTION_SCALE refactor. Following the same template:
vram_write_ps (4 variants: PGXP x use_ssbo), vram_fill_ps (8
variants: PGXP x wrapped x interlaced, no refactor needed
since the body doesn't reference RESOLUTION_SCALE), and
vram_update_depth_ps (2 MSAA variants, exercises MSAA
texture-binding variant for the first time).
2026-05-18 21:48:20 +02:00
..
copy.ps.hlsl core/d3d12: pre-bake the copy/blit pixel shader 2026-05-18 20:22:17 +02:00
fullscreen_quad.vs.hlsl core/d3d12: pre-bake the fullscreen-quad vertex shader 2026-05-18 19:40:13 +02:00
README.md core/d3d12: scaffolding for pre-baked DXBC blobs 2026-05-18 09:14:52 +02:00
vram_copy.ps.hlsl core/d3d12: pre-bake the VRAM copy pixel shader 2026-05-18 21:48:20 +02:00

D3D12 shader sources (pre-baked DXBC)

This directory holds the HLSL sources for shaders used by the D3D12 backend that have been moved off the runtime ShaderGen path. Each .hlsl file here has a matching pre-compiled DXBC blob checked in to src/common/d3d12/embedded_dxbc/.

Why this exists

Historically the D3D12 backend generated HLSL at runtime via src/core/shadergen.cpp and src/core/gpu_hw_shadergen.cpp, then handed the text to D3DCompile (the in-process HLSL -> DXBC compiler from d3dcompiler_47.dll) to produce a DXBC blob for each PSO. On a cold shader cache, walking the 1164-cell batch pipeline matrix in precompile_sync mode takes on the order of one minute, which is the worst entry in the post-dim-cache cost matrix (see the cost-matrix comments in 10c53b8, 57ac62e, and the patches that followed). The runtime cache hides this on warm boots, but cold boots and shader cache invalidations re-pay the full per-cell D3DCompile cost every time the user changes a setting that hits the full-flush branch.

Pre-baking the DXBC removes:

  • per-launch HLSL compilation stall on shader cache misses,
  • a class of "stale cache after d3dcompiler_47.dll version drift" failures,
  • the implicit dependency on D3DCompile being available at runtime for the cells that have been pre-baked.

The OpenGL / OpenGL ES / D3D11 / Vulkan backends still use the runtime ShaderGen path; only the D3D12 backend reads from the embedded DXBC. (The Vulkan equivalent of this infrastructure lives at data/shaders/vulkan/ / src/common/vulkan/embedded_spirv/ / tools/regen_vulkan_spirv.py; this directory is the mirror.)

Build-time constraint

The core build (Makefile.libretro and src/common/CMakeLists.txt) must never invoke fxc.exe or any other external shader compiler. The .inc files under src/common/d3d12/embedded_dxbc/ are checked into the repository and compiled like any other C++ source. The tools/regen_d3d12_dxbc.py helper exists for contributors who edit the .hlsl sources in this directory; it is not part of the build.

Regenerating the DXBC blobs

After editing a .hlsl file here, run:

python3 tools/regen_d3d12_dxbc.py

This invokes fxc.exe (Shader Model 5.0, /O3, /E main) on each .hlsl file and rewrites the matching .inc under src/common/d3d12/embedded_dxbc/. Both the .hlsl and the regenerated .inc must be committed together so the two stay in sync.

fxc.exe ships with the Windows 10 SDK and is not invoked by the core build. On Linux, run the script under Wine - it will look for fxc.exe in the standard SDK install path under $WINEPREFIX/drive_c/Program Files (x86)/Windows Kits/10/bin/<version>/x64/fxc.exe, or you can pass --fxc PATH explicitly. The --wine PATH flag lets you point at a specific wine binary if wine isn't on PATH.

Filename convention

Stage is encoded in the suffix:

  • .vs.hlsl -> vertex shader (compiled as vs_5_0)
  • .ps.hlsl -> pixel shader (compiled as ps_5_0)
  • .cs.hlsl -> compute shader (compiled as cs_5_0)
  • .gs.hlsl -> geometry shader (compiled as gs_5_0)

The _5_0 shader model targets D3D_FEATURE_LEVEL_11_0, matching the runtime baseline of the D3D12 backend (see the shared compile path in src/common/d3d11/shader_compiler.cpp which both D3D11 and D3D12 used prior to the pre-bake migration).

Template variants

Shaders that need multiple pre-baked variants (e.g. one per BatchRenderMode or per GPUTextureFilter) are listed in the TEMPLATE_VARIANTS dict at the top of tools/regen_d3d12_dxbc.py. Each entry maps an .hlsl filename to the list of (variant_suffix, [/D defines]) tuples that the regen script should produce.

If you add a template variant to the dict, add the matching extern declaration to src/common/d3d12/embedded_shaders.h AND the matching slot lookup to src/common/d3d12/embedded_shaders.cpp - the index ordering at the runtime call site must match the order in the dict so the correct blob gets bound at PSO-creation time.