Post-mortem follow-up to cd971cd / da20f5b. The pre-bake attempt
failed because Wine's bundled d3dcompiler_47.dll (which is the
vkd3d-shader reimplementation, currently at version 1.10) produces
DXBC that's syntactically valid as a container but trips Microsoft
D3D12's CreateGraphicsPipelineState with E_FAIL for non-trivial
shaders. The vram_copy.ps.hlsl case crossed that threshold; the
simpler fullscreen_quad.vs.hlsl + copy.ps.hlsl cases happen to
work and are still in the repo, but they were also vkd3d-shader-
compiled and should be regenerated with Microsoft fxc when the
opportunity arises.
The fix is twofold: improve the SDK auto-detection so contributors
who have Microsoft fxc installed don't have to fight the script,
and add an output validator that refuses to write .inc files
produced by any compiler that isn't Microsoft fxc.
Auto-detection improvements (find_fxc):
* Wine prefix discovery widened. Previously the script only
checked $WINEPREFIX/drive_c if WINEPREFIX was set. Now it also
checks ~/.wine (the Wine default when WINEPREFIX is unset),
and any prefix directory under ~/.local/share/wineprefixes/
that has a drive_c subtree (PlayOnLinux / lutris-style layouts).
Both /Program Files/ and /Program Files (x86)/ are searched in
each prefix.
* Error message now points at the README's installation steps
rather than telling the user to just install the SDK.
* Error message explicitly calls out that Wine's bundled
d3dcompiler_47.dll is NOT a substitute and references
da20f5b for context.
Output validator (extract_rdef_creator + check_microsoft_creator):
* extract_rdef_creator walks the DXBC container chunk index, finds
the RDEF chunk, follows the creator_offset field at chunk body
offset 24, and reads back the null-terminated ASCII creator
string. The DXBC container layout (16-byte hash + version +
total size + chunk count + chunk offsets) and the RDEF body
layout (constant_buffer_count, constant_buffer_offset,
resource_binding_count, resource_binding_offset, shader_version,
compiler_flags, creator_offset) are documented in MSDN's old
D3D11 reflection notes and in the wikitalk pages around DXBC;
the implementation here uses fixed offsets that match what
fxc.exe and vkd3d-shader both produce.
* check_microsoft_creator hooks into compile_one after the magic
check. It refuses with a clear error message if the creator
string doesn't contain "Microsoft". Microsoft fxc stamps
something like "Microsoft (R) HLSL Shader Compiler 10.1";
vkd3d-shader stamps "vkd3d-shader 1.10 (Wine bundled)". One
substring check disambiguates.
What the validator does NOT do:
* It does not retroactively reject existing .inc files in the
repo. The validator only fires on freshly-produced DXBC during
a regen run. The vkd3d-shader-built fullscreen_quad_vs.inc and
copy_ps.inc are unchanged by this commit; they'll get refreshed
the next time a contributor with Microsoft fxc runs the regen
script. Until then, they keep working because they're already
in the repo and the build just compiles them as plain C++
byte arrays - they never go through PSO validation (they're
already valid DXBC).
* It does not lock the script to a specific fxc version. Any
Microsoft fxc.exe (10.1, 10.5, the version that ships with the
Windows 11 SDK, etc.) is accepted - they all stamp "Microsoft"
in the creator string.
* It does not validate the binding layout vs the runtime root
signature. That's a future enhancement - the runtime side
expects specific (b0, t0, s0) layouts and the regen script
could cross-check, but that's a much bigger feature.
README changes:
* New "Microsoft fxc.exe is required" section spells out why and
cites the cd971cd / da20f5b failure as the worked example.
* "Installing Microsoft fxc.exe" subsection with separate
Windows / Linux instructions. Linux gets two paths: install
the SDK under a Wine prefix (auto-detected), or extract
fxc.exe + d3dcompiler_47.dll directly from the MSI with
msitools and pass --fxc. Both must live in the same dir so
Wine's loader prefers Microsoft's d3dcompiler_47.dll over its
bundled vkd3d-shader version.
* The auto-detection scope (WINEPREFIX, ~/.wine, ~/.local/share/
wineprefixes/*) is now documented so contributors know where
the script is looking.
This commit does NOT re-add the vram_copy.ps.hlsl source removed
by da20f5b. That comes back in a follow-up patch once a Microsoft
fxc-built .inc is available - same patch lands the .hlsl, the
.inc, and the call-site flip together (the dd54a73 lesson).
Test:
- python3 -c 'import ast; ast.parse(open("tools/regen_d3d12_dxbc.py").read())'
passes.
- Loading the script as a module and feeding the EXISTING
vkd3d-built copy_ps.inc into extract_rdef_creator returns
'vkd3d-shader 1.10 (Wine bundled)'; the Microsoft substring
check returns False. Confirms the validator would catch the
cd971cd mistake if anyone tries to reproduce it.
- find_fxc's broader Wine prefix scan is exercised by the
standard test of running the script in an environment without
Microsoft fxc; it now fails with the new pointed-at-README
message instead of the old generic SDK message.