TsundereStation/data
LibretroAdmin ebd2300cde tools/d3d12: refuse non-Microsoft fxc output in the regen script
Post-mortem follow-up to cd971cd / da20f5b. The pre-bake attempt
failed because Wine's bundled d3dcompiler_47.dll (which is the
vkd3d-shader reimplementation, currently at version 1.10) produces
DXBC that's syntactically valid as a container but trips Microsoft
D3D12's CreateGraphicsPipelineState with E_FAIL for non-trivial
shaders. The vram_copy.ps.hlsl case crossed that threshold; the
simpler fullscreen_quad.vs.hlsl + copy.ps.hlsl cases happen to
work and are still in the repo, but they were also vkd3d-shader-
compiled and should be regenerated with Microsoft fxc when the
opportunity arises.

The fix is twofold: improve the SDK auto-detection so contributors
who have Microsoft fxc installed don't have to fight the script,
and add an output validator that refuses to write .inc files
produced by any compiler that isn't Microsoft fxc.

Auto-detection improvements (find_fxc):

* Wine prefix discovery widened. Previously the script only
  checked $WINEPREFIX/drive_c if WINEPREFIX was set. Now it also
  checks ~/.wine (the Wine default when WINEPREFIX is unset),
  and any prefix directory under ~/.local/share/wineprefixes/
  that has a drive_c subtree (PlayOnLinux / lutris-style layouts).
  Both /Program Files/ and /Program Files (x86)/ are searched in
  each prefix.
* Error message now points at the README's installation steps
  rather than telling the user to just install the SDK.
* Error message explicitly calls out that Wine's bundled
  d3dcompiler_47.dll is NOT a substitute and references
  da20f5b for context.

Output validator (extract_rdef_creator + check_microsoft_creator):

* extract_rdef_creator walks the DXBC container chunk index, finds
  the RDEF chunk, follows the creator_offset field at chunk body
  offset 24, and reads back the null-terminated ASCII creator
  string. The DXBC container layout (16-byte hash + version +
  total size + chunk count + chunk offsets) and the RDEF body
  layout (constant_buffer_count, constant_buffer_offset,
  resource_binding_count, resource_binding_offset, shader_version,
  compiler_flags, creator_offset) are documented in MSDN's old
  D3D11 reflection notes and in the wikitalk pages around DXBC;
  the implementation here uses fixed offsets that match what
  fxc.exe and vkd3d-shader both produce.
* check_microsoft_creator hooks into compile_one after the magic
  check. It refuses with a clear error message if the creator
  string doesn't contain "Microsoft". Microsoft fxc stamps
  something like "Microsoft (R) HLSL Shader Compiler 10.1";
  vkd3d-shader stamps "vkd3d-shader 1.10 (Wine bundled)". One
  substring check disambiguates.

What the validator does NOT do:

* It does not retroactively reject existing .inc files in the
  repo. The validator only fires on freshly-produced DXBC during
  a regen run. The vkd3d-shader-built fullscreen_quad_vs.inc and
  copy_ps.inc are unchanged by this commit; they'll get refreshed
  the next time a contributor with Microsoft fxc runs the regen
  script. Until then, they keep working because they're already
  in the repo and the build just compiles them as plain C++
  byte arrays - they never go through PSO validation (they're
  already valid DXBC).
* It does not lock the script to a specific fxc version. Any
  Microsoft fxc.exe (10.1, 10.5, the version that ships with the
  Windows 11 SDK, etc.) is accepted - they all stamp "Microsoft"
  in the creator string.
* It does not validate the binding layout vs the runtime root
  signature. That's a future enhancement - the runtime side
  expects specific (b0, t0, s0) layouts and the regen script
  could cross-check, but that's a much bigger feature.

README changes:

* New "Microsoft fxc.exe is required" section spells out why and
  cites the cd971cd / da20f5b failure as the worked example.
* "Installing Microsoft fxc.exe" subsection with separate
  Windows / Linux instructions. Linux gets two paths: install
  the SDK under a Wine prefix (auto-detected), or extract
  fxc.exe + d3dcompiler_47.dll directly from the MSI with
  msitools and pass --fxc. Both must live in the same dir so
  Wine's loader prefers Microsoft's d3dcompiler_47.dll over its
  bundled vkd3d-shader version.
* The auto-detection scope (WINEPREFIX, ~/.wine, ~/.local/share/
  wineprefixes/*) is now documented so contributors know where
  the script is looking.

This commit does NOT re-add the vram_copy.ps.hlsl source removed
by da20f5b. That comes back in a follow-up patch once a Microsoft
fxc-built .inc is available - same patch lands the .hlsl, the
.inc, and the call-site flip together (the dd54a73 lesson).

Test:
  - python3 -c 'import ast; ast.parse(open("tools/regen_d3d12_dxbc.py").read())'
    passes.
  - Loading the script as a module and feeding the EXISTING
    vkd3d-built copy_ps.inc into extract_rdef_creator returns
    'vkd3d-shader 1.10 (Wine bundled)'; the Microsoft substring
    check returns False. Confirms the validator would catch the
    cd971cd mistake if anyone tries to reproduce it.
  - find_fxc's broader Wine prefix scan is exercised by the
    standard test of running the script in an environment without
    Microsoft fxc; it now fails with the new pointed-at-README
    message instead of the old generic SDK message.
2026-05-18 22:16:18 +02:00
..
shaders tools/d3d12: refuse non-Microsoft fxc output in the regen script 2026-05-18 22:16:18 +02:00