First step of the D3D12 pre-baked DXBC arc that was originally
flagged in 7b575a3's commit message as the natural follow-up to
the cbuffer / dim cache / chroma-split / non-batch-skip
optimisation series. Sets up the directory layout, regen tool,
and namespace scaffolding that subsequent patches will populate
with individual shader migrations.
Mirror of the Vulkan-side infrastructure that lives at
data/shaders/vulkan/ + src/common/vulkan/embedded_shaders.{h,cpp}
+ tools/regen_vulkan_spirv.py + src/common/vulkan/embedded_spirv/
(181 .inc files). The Vulkan side replaces glslang -> SPIR-V at
runtime with pre-compiled SPIR-V blobs; the D3D12 side will
replace D3DCompile -> DXBC at runtime with pre-compiled DXBC
blobs from fxc.exe.
Motivation: after the dim cache (10c53b8) and the chroma-split
(57ac62e), the D3D12 backend's UpdateSettings cost matrix has
one remaining 'still pays the full rebuild cost' class of
toggles - multisamples / per_sample_shading / uv_limits /
pgxp_depth / colour perspective - that genuinely invalidate
every HLSL variant in the 1164-cell batch pipeline matrix. On
a cold shader cache, walking that matrix synchronously costs on
the order of one minute under shader_precompile_mode = Enabled,
because every cell pays a fresh D3DCompile. Pre-baking the DXBC
eliminates the D3DCompile pass on those cells, dropping the
cold-flip cost to whatever the PSO-creation pass alone takes
(ID3D12Device::CreateGraphicsPipelineState, much faster than
D3DCompile + CreateGraphicsPipelineState).
The pre-bake also forecloses a class of 'd3dcompiler_47.dll
version drift invalidates the cached DXBC' failures the
runtime shader cache sometimes hits on driver updates.
Files in this scaffold patch:
* tools/regen_d3d12_dxbc.py - mirror of regen_vulkan_spirv.py
for the fxc.exe / DXBC path. Walks data/shaders/d3d12/*.hlsl,
determines stage from filename suffix (.vs.hlsl / .ps.hlsl /
.cs.hlsl / .gs.hlsl), compiles each via fxc /O3 /T <stage>_5_0
/E main, validates the output DXBC magic, and emits a .inc
with const uint8_t array + size_t size_bytes. Honours a
TEMPLATE_VARIANTS dict (initially empty) for shaders that need
multiple pre-baked variants via /D defines. Auto-detects
fxc.exe in the Windows 10 SDK install path; supports Wine on
Linux via --wine and explicit --fxc PATH.
In its empty-input case (the state immediately after this
patch lands) the script exits zero with a 'nothing to do'
message - no fxc lookup needed when there are no shaders to
compile, so the script is runnable on systems without the
Windows SDK / Wine until the first shader source actually
shows up under data/shaders/d3d12/.
* data/shaders/d3d12/README.md - workflow documentation mirror
of data/shaders/vulkan/README.md. Documents the
edit-.hlsl -> rerun-script -> commit-both convention, the
filename suffix -> shader stage mapping, the
TEMPLATE_VARIANTS partition for multi-variant blobs, and the
build-time constraint that the core build must never invoke
fxc.exe.
* src/common/d3d12/embedded_shaders.h - namespace
D3D12::EmbeddedShaders header. No extern declarations yet -
this patch lays down the namespace scaffold. Subsequent
patches add pairs of:
extern const uint8_t k_<stem>_<stage>[];
extern const size_t k_<stem>_<stage>_size_bytes;
per migrated shader. Comment block documents the (data, size)
-> D3D12_SHADER_BYTECODE aggregate-initialiser convention -
no wrapper helper needed (unlike Vulkan's CreateShaderModule
helper, because D3D12_SHADER_BYTECODE is a plain struct, not
an API call).
* src/common/d3d12/embedded_shaders.cpp - storage TU. Currently
empty inside the namespace block. As .inc files get added
under src/common/d3d12/embedded_dxbc/, each is #included
here from inside the namespace block so the arrays resolve
against the header's extern declarations.
* Makefile.libretro and src/common/CMakeLists.txt - register
embedded_shaders.cpp as a TU under the existing USE_D3D12
gate. Compiles as part of the libretro build whenever D3D12
support is enabled.
Subsequent patches in this arc will each:
1. Add data/shaders/d3d12/<name>.<stage>.hlsl from the runtime
ShaderGen output (cleaned up - no preprocessor noise that
fxc handles at compile time anyway).
2. Run tools/regen_d3d12_dxbc.py to produce the .inc(s).
3. Add the matching extern declarations to embedded_shaders.h
and the #include line to embedded_shaders.cpp.
4. Switch the D3D12 backend call site (typically in
gpu_hw_d3d12.cpp's GetXxxPipeline or CompilePipelines) to
bind the embedded blob via D3D12_SHADER_BYTECODE instead
of calling D3DCompile through the shared D3D11::ShaderCompiler
path.
Ordering preference: start with non-batch, single-variant
shaders (vram_read / vram_write / vram_update_depth /
vram_copy / vram_fill / display) to validate the pipeline
end-to-end with minimal blast radius. Then the multi-variant
display / chroma family. Then the batch matrix - which is the
bulk of the work and where the cold-flip cost actually lives.
Note: the .inc files don't exist yet because fxc.exe isn't
runnable in this submission environment (Linux without Wine /
Windows SDK). The script is functional and will produce
.inc files as soon as it gets run on a host with fxc.exe.
This patch is intentionally a foundation: build succeeds with
no behaviour change, runtime is untouched, and the next patch
in the arc adds the first shader source + its .inc + the
matching call-site flip.
Test:
- python3 tools/regen_d3d12_dxbc.py with no shader sources
exits 0 with 'nothing to do' message. No fxc.exe required.
- Build under USE_D3D12=1: embedded_shaders.cpp compiles
cleanly as an empty namespace TU. No new symbols emitted.
- Build under USE_D3D12=0: embedded_shaders.cpp is excluded
from the build per the existing if(USE_D3D12) gate in
src/common/CMakeLists.txt and the ifeq guard in
Makefile.libretro.
- Runtime: no behavior change. D3D12 backend continues to
compile every shader through the existing D3DCompile path.