First end-to-end pre-bake migration off the runtime D3DCompile
path, on top of the scaffolding from ca8c9b0. Picks the smallest
possible target - a single-variant vertex shader with zero state
dependency, used by every non-batch pipeline (vram_fill /
vram_copy / vram_write / vram_update_depth / vram_readback /
display / copy/blit) - to validate the full pipeline before
larger migrations land.
Reasoning for starting here: the screen-quad VS emits a fullscreen
triangle from SV_VertexID via the standard bit-shift trick and
has no #defines or compile-time-baked uniforms, so a single
pre-baked variant covers all seven call sites. Compared to vram_read
(which I evaluated as the first target earlier in this arc):
RESOLUTION_SCALE bakes in compile-time on non-batch shaders via
WriteCommonFunctions, so vram_read would need either one variant
per scale value or a refactor of the runtime to bind the batch
cbuffer to non-batch shaders. That's a real refactor, not a
validation patch. The screen-quad VS sidesteps it entirely.
Files in this patch:
* data/shaders/d3d12/fullscreen_quad.vs.hlsl - standalone HLSL
equivalent to what ShaderGen::GenerateScreenQuadVertexShader()
produces in D3D12 mode after preprocessor expansion. Stripped
the shadergen prelude (#define HLSL 1, #define vec2 float2,
the SAMPLE_TEXTURE / LOAD_TEXTURE / FOR_UNROLL macros, the API_*
defines) because the shader uses none of them - native HLSL
is what fxc wants. Also dropped the dead
#if API_OPENGL || API_OPENGL_ES || API_VULKAN branch since
API_OPENGL=0 / API_OPENGL_ES=0 / API_VULKAN=0 for D3D12.
* src/common/d3d12/embedded_shaders.h - replaces the placeholder
comment block from ca8c9b0 with the first concrete extern
declaration:
extern const uint8_t k_fullscreen_quad_vs[];
extern const size_t k_fullscreen_quad_vs_size_bytes;
Pattern matches the regen_d3d12_dxbc.py emission convention
documented in ca8c9b0's commit message.
* src/common/d3d12/embedded_shaders.cpp - replaces its placeholder
with the first #include of a generated .inc. The .inc itself
(src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc) is
generated by tools/regen_d3d12_dxbc.py from the .hlsl source
above. **The .inc is NOT included in this patch** because the
submission environment doesn't have fxc.exe available. See the
applying instructions at the bottom of this commit message.
* src/core/gpu_hw_d3d12.h:
- GetFullscreenQuadVertexShader() return type changed from
ComPtr<ID3DBlob> to D3D12_SHADER_BYTECODE. The function is no
longer a lazy-fault helper - the embedded blob is statically
linked, always available, no allocation. Function comment
refreshed accordingly.
- m_fullscreen_quad_vertex_shader member dropped. It was the
cache slot for the lazy-faulted blob; with pre-bake there's
no fault-in, no caching state, no mutex.
* src/core/gpu_hw_d3d12.cpp:
- #include "common/d3d12/embedded_shaders.h" added next to the
other common/d3d12/ includes.
- GetFullscreenQuadVertexShader body rewritten to return a
D3D12_SHADER_BYTECODE aggregate over the embedded blob. No
mutex, no shader_cache.GetVertexShader, no D3DCompile.
- Two precompile_sync if-checks removed. The block at line ~956
used to call GetFullscreenQuadVertexShader() and increment
progress; the block at line ~1023 used to fault it in for
Lazy mode. Both meaningless now - the blob is statically
linked, there is no fault to drive. Left explanatory comments
at both sites so the absence is documented.
- Progress unit count updated 18 -> 17. The fullscreen-quad
VS was one of the 18 non-batch compilation units the
progress bar ticked through; with pre-bake it has no compile
step to tick. Inline comment refreshed at both the count
declaration site (line ~825) and the "1 + 4 + 2 + 2 + 1 + 1
+ 6 + 1 = 18" comment that explained the breakdown.
- Seven call sites in the GetXxxPipeline helpers converted
from:
ComPtr<ID3DBlob> vs = GetFullscreenQuadVertexShader();
if (!vs)
return {};
...
gpbuilder.SetVertexShader(vs.Get());
to:
const D3D12_SHADER_BYTECODE vs = GetFullscreenQuadVertexShader();
...
gpbuilder.SetVertexShader(vs.pShaderBytecode,
static_cast<uint32_t>(vs.BytecodeLength));
The null check is dropped because the embedded blob is never
null. The cast to uint32_t matches the existing
SetVertexShader(const void*, uint32_t) overload signature in
src/common/d3d12/util.h. (A future cleanup could add a
SetVertexShader(const D3D12_SHADER_BYTECODE&) overload to
elide the verbose unpacking at call sites; left for the next
patch in the arc where we'll have several more embedded
blobs and the savings justify the overload.)
- m_fullscreen_quad_vertex_shader.Reset() in DestroyPipelines
dropped. Comment block trimmed accordingly.
- Stale comment block above GetFullscreenQuadVertexShader
refreshed to describe pre-bake semantics instead of the
fault-in pattern that no longer applies.
Applying instructions (the .inc file isn't in this patch, so
build will fail until it lands):
$ git am 0001-core-d3d12-pre-bake-the-fullscreen-quad-vertex-shader.patch
$ python3 tools/regen_d3d12_dxbc.py
# produces src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc
$ git add src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc
$ git commit --amend --no-edit
$ git push
The amend folds the generated .inc into this commit so upstream
sees a single self-contained patch. (Alternative: commit the .inc
as a separate "regenerate fullscreen_quad_vs.inc" follow-up;
either works, but amending keeps the history one-commit-per-
migration which matches the Vulkan-side convention.)
Cost / behaviour:
- Cold boot, shader_precompile_mode = Enabled: one fewer
D3DCompile call, one fewer ID3DBlob allocation. Microseconds
saved per boot, not user-visible at this scale.
- Lazy mode pre-fill: same delta.
- Disabled mode first FillVRAM / etc.: same delta.
- Runtime: no behaviour change. PSOs that bind the embedded
blob produce byte-identical bytecode to what D3DCompile would
have produced from GenerateScreenQuadVertexShader() (modulo
fxc version compiler-output differences, which the runtime
cache normalises against anyway).
- Failure mode: if fxc produces a different shader (e.g. a
fxc version drift bug), every non-batch pipeline using the
embedded VS would render incorrectly - blank framebuffer,
wrong UVs, etc. The visual regression is immediate on first
frame. Easy to spot.
Test:
- Cold boot under shader_precompile_mode = Enabled. Progress
bar reaches 100% (now sized to 17 non-batch units). FillVRAM
/ display / readback / etc. all render correctly.
- Cold boot under Lazy. Pre-fill happens in the foreground
before runloop, no nullptr at draw time on the first frame.
- Cold boot under Disabled. First FillVRAM lazy-faults the
fill pipeline; the embedded VS is bound, render is correct.
- Toggle a non_dim setting (multisamples) to force full-flush
of pipelines. CompilePipelines walks the matrix again,
rebinds the embedded VS on every non-batch PSO. No leak,
no use-after-free.
- Toggle chroma_smoothing to fire the partial-flush branch
(display pipelines only). Embedded VS is bound on the new
display PSOs. Display still renders.
- DestroyPipelines on shutdown / UpdateSettings: no
m_fullscreen_quad_vertex_shader to Reset; nothing to leak.
Next patch in the arc: extend to non-batch pixel shaders that
don't bake state. vram_copy_ps and vram_update_depth_ps are
the next candidates - both are single-variant and have no
RESOLUTION_SCALE dependency (they sample VRAM coordinates
directly, no box-filter loop). After those, vram_fill_ps (two
variants: wrapped/unwrapped, interlaced/not - 4 total - so
TEMPLATE_VARIANTS gets its first non-empty entry). vram_read
and the batch matrix come later, after we have a cbuffer-
routing migration that lets RESOLUTION_SCALE be runtime-set
in non-batch shaders too.
First step of the D3D12 pre-baked DXBC arc that was originally
flagged in 7b575a3's commit message as the natural follow-up to
the cbuffer / dim cache / chroma-split / non-batch-skip
optimisation series. Sets up the directory layout, regen tool,
and namespace scaffolding that subsequent patches will populate
with individual shader migrations.
Mirror of the Vulkan-side infrastructure that lives at
data/shaders/vulkan/ + src/common/vulkan/embedded_shaders.{h,cpp}
+ tools/regen_vulkan_spirv.py + src/common/vulkan/embedded_spirv/
(181 .inc files). The Vulkan side replaces glslang -> SPIR-V at
runtime with pre-compiled SPIR-V blobs; the D3D12 side will
replace D3DCompile -> DXBC at runtime with pre-compiled DXBC
blobs from fxc.exe.
Motivation: after the dim cache (10c53b8) and the chroma-split
(57ac62e), the D3D12 backend's UpdateSettings cost matrix has
one remaining 'still pays the full rebuild cost' class of
toggles - multisamples / per_sample_shading / uv_limits /
pgxp_depth / colour perspective - that genuinely invalidate
every HLSL variant in the 1164-cell batch pipeline matrix. On
a cold shader cache, walking that matrix synchronously costs on
the order of one minute under shader_precompile_mode = Enabled,
because every cell pays a fresh D3DCompile. Pre-baking the DXBC
eliminates the D3DCompile pass on those cells, dropping the
cold-flip cost to whatever the PSO-creation pass alone takes
(ID3D12Device::CreateGraphicsPipelineState, much faster than
D3DCompile + CreateGraphicsPipelineState).
The pre-bake also forecloses a class of 'd3dcompiler_47.dll
version drift invalidates the cached DXBC' failures the
runtime shader cache sometimes hits on driver updates.
Files in this scaffold patch:
* tools/regen_d3d12_dxbc.py - mirror of regen_vulkan_spirv.py
for the fxc.exe / DXBC path. Walks data/shaders/d3d12/*.hlsl,
determines stage from filename suffix (.vs.hlsl / .ps.hlsl /
.cs.hlsl / .gs.hlsl), compiles each via fxc /O3 /T <stage>_5_0
/E main, validates the output DXBC magic, and emits a .inc
with const uint8_t array + size_t size_bytes. Honours a
TEMPLATE_VARIANTS dict (initially empty) for shaders that need
multiple pre-baked variants via /D defines. Auto-detects
fxc.exe in the Windows 10 SDK install path; supports Wine on
Linux via --wine and explicit --fxc PATH.
In its empty-input case (the state immediately after this
patch lands) the script exits zero with a 'nothing to do'
message - no fxc lookup needed when there are no shaders to
compile, so the script is runnable on systems without the
Windows SDK / Wine until the first shader source actually
shows up under data/shaders/d3d12/.
* data/shaders/d3d12/README.md - workflow documentation mirror
of data/shaders/vulkan/README.md. Documents the
edit-.hlsl -> rerun-script -> commit-both convention, the
filename suffix -> shader stage mapping, the
TEMPLATE_VARIANTS partition for multi-variant blobs, and the
build-time constraint that the core build must never invoke
fxc.exe.
* src/common/d3d12/embedded_shaders.h - namespace
D3D12::EmbeddedShaders header. No extern declarations yet -
this patch lays down the namespace scaffold. Subsequent
patches add pairs of:
extern const uint8_t k_<stem>_<stage>[];
extern const size_t k_<stem>_<stage>_size_bytes;
per migrated shader. Comment block documents the (data, size)
-> D3D12_SHADER_BYTECODE aggregate-initialiser convention -
no wrapper helper needed (unlike Vulkan's CreateShaderModule
helper, because D3D12_SHADER_BYTECODE is a plain struct, not
an API call).
* src/common/d3d12/embedded_shaders.cpp - storage TU. Currently
empty inside the namespace block. As .inc files get added
under src/common/d3d12/embedded_dxbc/, each is #included
here from inside the namespace block so the arrays resolve
against the header's extern declarations.
* Makefile.libretro and src/common/CMakeLists.txt - register
embedded_shaders.cpp as a TU under the existing USE_D3D12
gate. Compiles as part of the libretro build whenever D3D12
support is enabled.
Subsequent patches in this arc will each:
1. Add data/shaders/d3d12/<name>.<stage>.hlsl from the runtime
ShaderGen output (cleaned up - no preprocessor noise that
fxc handles at compile time anyway).
2. Run tools/regen_d3d12_dxbc.py to produce the .inc(s).
3. Add the matching extern declarations to embedded_shaders.h
and the #include line to embedded_shaders.cpp.
4. Switch the D3D12 backend call site (typically in
gpu_hw_d3d12.cpp's GetXxxPipeline or CompilePipelines) to
bind the embedded blob via D3D12_SHADER_BYTECODE instead
of calling D3DCompile through the shared D3D11::ShaderCompiler
path.
Ordering preference: start with non-batch, single-variant
shaders (vram_read / vram_write / vram_update_depth /
vram_copy / vram_fill / display) to validate the pipeline
end-to-end with minimal blast radius. Then the multi-variant
display / chroma family. Then the batch matrix - which is the
bulk of the work and where the cold-flip cost actually lives.
Note: the .inc files don't exist yet because fxc.exe isn't
runnable in this submission environment (Linux without Wine /
Windows SDK). The script is functional and will produce
.inc files as soon as it gets run on a host with fxc.exe.
This patch is intentionally a foundation: build succeeds with
no behaviour change, runtime is untouched, and the next patch
in the arc adds the first shader source + its .inc + the
matching call-site flip.
Test:
- python3 tools/regen_d3d12_dxbc.py with no shader sources
exits 0 with 'nothing to do' message. No fxc.exe required.
- Build under USE_D3D12=1: embedded_shaders.cpp compiles
cleanly as an empty namespace TU. No new symbols emitted.
- Build under USE_D3D12=0: embedded_shaders.cpp is excluded
from the build per the existing if(USE_D3D12) gate in
src/common/CMakeLists.txt and the ifeq guard in
Makefile.libretro.
- Runtime: no behavior change. D3D12 backend continues to
compile every shader through the existing D3DCompile path.