Commit graph

2 commits

Author SHA1 Message Date
LibretroAdmin
dd54a73809 core/d3d12: pre-bake the fullscreen-quad vertex shader
First end-to-end pre-bake migration off the runtime D3DCompile
path, on top of the scaffolding from ca8c9b0. Picks the smallest
possible target - a single-variant vertex shader with zero state
dependency, used by every non-batch pipeline (vram_fill /
vram_copy / vram_write / vram_update_depth / vram_readback /
display / copy/blit) - to validate the full pipeline before
larger migrations land.

Reasoning for starting here: the screen-quad VS emits a fullscreen
triangle from SV_VertexID via the standard bit-shift trick and
has no #defines or compile-time-baked uniforms, so a single
pre-baked variant covers all seven call sites. Compared to vram_read
(which I evaluated as the first target earlier in this arc):
RESOLUTION_SCALE bakes in compile-time on non-batch shaders via
WriteCommonFunctions, so vram_read would need either one variant
per scale value or a refactor of the runtime to bind the batch
cbuffer to non-batch shaders. That's a real refactor, not a
validation patch. The screen-quad VS sidesteps it entirely.

Files in this patch:

* data/shaders/d3d12/fullscreen_quad.vs.hlsl - standalone HLSL
  equivalent to what ShaderGen::GenerateScreenQuadVertexShader()
  produces in D3D12 mode after preprocessor expansion. Stripped
  the shadergen prelude (#define HLSL 1, #define vec2 float2,
  the SAMPLE_TEXTURE / LOAD_TEXTURE / FOR_UNROLL macros, the API_*
  defines) because the shader uses none of them - native HLSL
  is what fxc wants. Also dropped the dead
  #if API_OPENGL || API_OPENGL_ES || API_VULKAN branch since
  API_OPENGL=0 / API_OPENGL_ES=0 / API_VULKAN=0 for D3D12.

* src/common/d3d12/embedded_shaders.h - replaces the placeholder
  comment block from ca8c9b0 with the first concrete extern
  declaration:

    extern const uint8_t k_fullscreen_quad_vs[];
    extern const size_t k_fullscreen_quad_vs_size_bytes;

  Pattern matches the regen_d3d12_dxbc.py emission convention
  documented in ca8c9b0's commit message.

* src/common/d3d12/embedded_shaders.cpp - replaces its placeholder
  with the first #include of a generated .inc. The .inc itself
  (src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc) is
  generated by tools/regen_d3d12_dxbc.py from the .hlsl source
  above. **The .inc is NOT included in this patch** because the
  submission environment doesn't have fxc.exe available. See the
  applying instructions at the bottom of this commit message.

* src/core/gpu_hw_d3d12.h:
  - GetFullscreenQuadVertexShader() return type changed from
    ComPtr<ID3DBlob> to D3D12_SHADER_BYTECODE. The function is no
    longer a lazy-fault helper - the embedded blob is statically
    linked, always available, no allocation. Function comment
    refreshed accordingly.
  - m_fullscreen_quad_vertex_shader member dropped. It was the
    cache slot for the lazy-faulted blob; with pre-bake there's
    no fault-in, no caching state, no mutex.

* src/core/gpu_hw_d3d12.cpp:
  - #include "common/d3d12/embedded_shaders.h" added next to the
    other common/d3d12/ includes.
  - GetFullscreenQuadVertexShader body rewritten to return a
    D3D12_SHADER_BYTECODE aggregate over the embedded blob. No
    mutex, no shader_cache.GetVertexShader, no D3DCompile.
  - Two precompile_sync if-checks removed. The block at line ~956
    used to call GetFullscreenQuadVertexShader() and increment
    progress; the block at line ~1023 used to fault it in for
    Lazy mode. Both meaningless now - the blob is statically
    linked, there is no fault to drive. Left explanatory comments
    at both sites so the absence is documented.
  - Progress unit count updated 18 -> 17. The fullscreen-quad
    VS was one of the 18 non-batch compilation units the
    progress bar ticked through; with pre-bake it has no compile
    step to tick. Inline comment refreshed at both the count
    declaration site (line ~825) and the "1 + 4 + 2 + 2 + 1 + 1
    + 6 + 1 = 18" comment that explained the breakdown.
  - Seven call sites in the GetXxxPipeline helpers converted
    from:
        ComPtr<ID3DBlob> vs = GetFullscreenQuadVertexShader();
        if (!vs)
          return {};
        ...
        gpbuilder.SetVertexShader(vs.Get());
    to:
        const D3D12_SHADER_BYTECODE vs = GetFullscreenQuadVertexShader();
        ...
        gpbuilder.SetVertexShader(vs.pShaderBytecode,
                                  static_cast<uint32_t>(vs.BytecodeLength));
    The null check is dropped because the embedded blob is never
    null. The cast to uint32_t matches the existing
    SetVertexShader(const void*, uint32_t) overload signature in
    src/common/d3d12/util.h. (A future cleanup could add a
    SetVertexShader(const D3D12_SHADER_BYTECODE&) overload to
    elide the verbose unpacking at call sites; left for the next
    patch in the arc where we'll have several more embedded
    blobs and the savings justify the overload.)
  - m_fullscreen_quad_vertex_shader.Reset() in DestroyPipelines
    dropped. Comment block trimmed accordingly.
  - Stale comment block above GetFullscreenQuadVertexShader
    refreshed to describe pre-bake semantics instead of the
    fault-in pattern that no longer applies.

Applying instructions (the .inc file isn't in this patch, so
build will fail until it lands):

  $ git am 0001-core-d3d12-pre-bake-the-fullscreen-quad-vertex-shader.patch
  $ python3 tools/regen_d3d12_dxbc.py
    # produces src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc
  $ git add src/common/d3d12/embedded_dxbc/fullscreen_quad_vs.inc
  $ git commit --amend --no-edit
  $ git push

The amend folds the generated .inc into this commit so upstream
sees a single self-contained patch. (Alternative: commit the .inc
as a separate "regenerate fullscreen_quad_vs.inc" follow-up;
either works, but amending keeps the history one-commit-per-
migration which matches the Vulkan-side convention.)

Cost / behaviour:
  - Cold boot, shader_precompile_mode = Enabled: one fewer
    D3DCompile call, one fewer ID3DBlob allocation. Microseconds
    saved per boot, not user-visible at this scale.
  - Lazy mode pre-fill: same delta.
  - Disabled mode first FillVRAM / etc.: same delta.
  - Runtime: no behaviour change. PSOs that bind the embedded
    blob produce byte-identical bytecode to what D3DCompile would
    have produced from GenerateScreenQuadVertexShader() (modulo
    fxc version compiler-output differences, which the runtime
    cache normalises against anyway).
  - Failure mode: if fxc produces a different shader (e.g. a
    fxc version drift bug), every non-batch pipeline using the
    embedded VS would render incorrectly - blank framebuffer,
    wrong UVs, etc. The visual regression is immediate on first
    frame. Easy to spot.

Test:
  - Cold boot under shader_precompile_mode = Enabled. Progress
    bar reaches 100% (now sized to 17 non-batch units). FillVRAM
    / display / readback / etc. all render correctly.
  - Cold boot under Lazy. Pre-fill happens in the foreground
    before runloop, no nullptr at draw time on the first frame.
  - Cold boot under Disabled. First FillVRAM lazy-faults the
    fill pipeline; the embedded VS is bound, render is correct.
  - Toggle a non_dim setting (multisamples) to force full-flush
    of pipelines. CompilePipelines walks the matrix again,
    rebinds the embedded VS on every non-batch PSO. No leak,
    no use-after-free.
  - Toggle chroma_smoothing to fire the partial-flush branch
    (display pipelines only). Embedded VS is bound on the new
    display PSOs. Display still renders.
  - DestroyPipelines on shutdown / UpdateSettings: no
    m_fullscreen_quad_vertex_shader to Reset; nothing to leak.

Next patch in the arc: extend to non-batch pixel shaders that
don't bake state. vram_copy_ps and vram_update_depth_ps are
the next candidates - both are single-variant and have no
RESOLUTION_SCALE dependency (they sample VRAM coordinates
directly, no box-filter loop). After those, vram_fill_ps (two
variants: wrapped/unwrapped, interlaced/not - 4 total - so
TEMPLATE_VARIANTS gets its first non-empty entry). vram_read
and the batch matrix come later, after we have a cbuffer-
routing migration that lets RESOLUTION_SCALE be runtime-set
in non-batch shaders too.
2026-05-18 19:40:13 +02:00
LibretroAdmin
ca8c9b05e8 core/d3d12: scaffolding for pre-baked DXBC blobs
First step of the D3D12 pre-baked DXBC arc that was originally
flagged in 7b575a3's commit message as the natural follow-up to
the cbuffer / dim cache / chroma-split / non-batch-skip
optimisation series. Sets up the directory layout, regen tool,
and namespace scaffolding that subsequent patches will populate
with individual shader migrations.

Mirror of the Vulkan-side infrastructure that lives at
data/shaders/vulkan/ + src/common/vulkan/embedded_shaders.{h,cpp}
+ tools/regen_vulkan_spirv.py + src/common/vulkan/embedded_spirv/
(181 .inc files). The Vulkan side replaces glslang -> SPIR-V at
runtime with pre-compiled SPIR-V blobs; the D3D12 side will
replace D3DCompile -> DXBC at runtime with pre-compiled DXBC
blobs from fxc.exe.

Motivation: after the dim cache (10c53b8) and the chroma-split
(57ac62e), the D3D12 backend's UpdateSettings cost matrix has
one remaining 'still pays the full rebuild cost' class of
toggles - multisamples / per_sample_shading / uv_limits /
pgxp_depth / colour perspective - that genuinely invalidate
every HLSL variant in the 1164-cell batch pipeline matrix. On
a cold shader cache, walking that matrix synchronously costs on
the order of one minute under shader_precompile_mode = Enabled,
because every cell pays a fresh D3DCompile. Pre-baking the DXBC
eliminates the D3DCompile pass on those cells, dropping the
cold-flip cost to whatever the PSO-creation pass alone takes
(ID3D12Device::CreateGraphicsPipelineState, much faster than
D3DCompile + CreateGraphicsPipelineState).

The pre-bake also forecloses a class of 'd3dcompiler_47.dll
version drift invalidates the cached DXBC' failures the
runtime shader cache sometimes hits on driver updates.

Files in this scaffold patch:

* tools/regen_d3d12_dxbc.py - mirror of regen_vulkan_spirv.py
  for the fxc.exe / DXBC path. Walks data/shaders/d3d12/*.hlsl,
  determines stage from filename suffix (.vs.hlsl / .ps.hlsl /
  .cs.hlsl / .gs.hlsl), compiles each via fxc /O3 /T <stage>_5_0
  /E main, validates the output DXBC magic, and emits a .inc
  with const uint8_t array + size_t size_bytes. Honours a
  TEMPLATE_VARIANTS dict (initially empty) for shaders that need
  multiple pre-baked variants via /D defines. Auto-detects
  fxc.exe in the Windows 10 SDK install path; supports Wine on
  Linux via --wine and explicit --fxc PATH.

  In its empty-input case (the state immediately after this
  patch lands) the script exits zero with a 'nothing to do'
  message - no fxc lookup needed when there are no shaders to
  compile, so the script is runnable on systems without the
  Windows SDK / Wine until the first shader source actually
  shows up under data/shaders/d3d12/.

* data/shaders/d3d12/README.md - workflow documentation mirror
  of data/shaders/vulkan/README.md. Documents the
  edit-.hlsl -> rerun-script -> commit-both convention, the
  filename suffix -> shader stage mapping, the
  TEMPLATE_VARIANTS partition for multi-variant blobs, and the
  build-time constraint that the core build must never invoke
  fxc.exe.

* src/common/d3d12/embedded_shaders.h - namespace
  D3D12::EmbeddedShaders header. No extern declarations yet -
  this patch lays down the namespace scaffold. Subsequent
  patches add pairs of:

    extern const uint8_t k_<stem>_<stage>[];
    extern const size_t k_<stem>_<stage>_size_bytes;

  per migrated shader. Comment block documents the (data, size)
  -> D3D12_SHADER_BYTECODE aggregate-initialiser convention -
  no wrapper helper needed (unlike Vulkan's CreateShaderModule
  helper, because D3D12_SHADER_BYTECODE is a plain struct, not
  an API call).

* src/common/d3d12/embedded_shaders.cpp - storage TU. Currently
  empty inside the namespace block. As .inc files get added
  under src/common/d3d12/embedded_dxbc/, each is #included
  here from inside the namespace block so the arrays resolve
  against the header's extern declarations.

* Makefile.libretro and src/common/CMakeLists.txt - register
  embedded_shaders.cpp as a TU under the existing USE_D3D12
  gate. Compiles as part of the libretro build whenever D3D12
  support is enabled.

Subsequent patches in this arc will each:

  1. Add data/shaders/d3d12/<name>.<stage>.hlsl from the runtime
     ShaderGen output (cleaned up - no preprocessor noise that
     fxc handles at compile time anyway).
  2. Run tools/regen_d3d12_dxbc.py to produce the .inc(s).
  3. Add the matching extern declarations to embedded_shaders.h
     and the #include line to embedded_shaders.cpp.
  4. Switch the D3D12 backend call site (typically in
     gpu_hw_d3d12.cpp's GetXxxPipeline or CompilePipelines) to
     bind the embedded blob via D3D12_SHADER_BYTECODE instead
     of calling D3DCompile through the shared D3D11::ShaderCompiler
     path.

Ordering preference: start with non-batch, single-variant
shaders (vram_read / vram_write / vram_update_depth /
vram_copy / vram_fill / display) to validate the pipeline
end-to-end with minimal blast radius. Then the multi-variant
display / chroma family. Then the batch matrix - which is the
bulk of the work and where the cold-flip cost actually lives.

Note: the .inc files don't exist yet because fxc.exe isn't
runnable in this submission environment (Linux without Wine /
Windows SDK). The script is functional and will produce
.inc files as soon as it gets run on a host with fxc.exe.
This patch is intentionally a foundation: build succeeds with
no behaviour change, runtime is untouched, and the next patch
in the arc adds the first shader source + its .inc + the
matching call-site flip.

Test:
  - python3 tools/regen_d3d12_dxbc.py with no shader sources
    exits 0 with 'nothing to do' message. No fxc.exe required.
  - Build under USE_D3D12=1: embedded_shaders.cpp compiles
    cleanly as an empty namespace TU. No new symbols emitted.
  - Build under USE_D3D12=0: embedded_shaders.cpp is excluded
    from the build per the existing if(USE_D3D12) gate in
    src/common/CMakeLists.txt and the ifeq guard in
    Makefile.libretro.
  - Runtime: no behavior change. D3D12 backend continues to
    compile every shader through the existing D3DCompile path.
2026-05-18 09:14:52 +02:00