tools/d3d12: refuse non-Microsoft fxc output in the regen script
Post-mortem follow-up tocd971cd/da20f5b. The pre-bake attempt failed because Wine's bundled d3dcompiler_47.dll (which is the vkd3d-shader reimplementation, currently at version 1.10) produces DXBC that's syntactically valid as a container but trips Microsoft D3D12's CreateGraphicsPipelineState with E_FAIL for non-trivial shaders. The vram_copy.ps.hlsl case crossed that threshold; the simpler fullscreen_quad.vs.hlsl + copy.ps.hlsl cases happen to work and are still in the repo, but they were also vkd3d-shader- compiled and should be regenerated with Microsoft fxc when the opportunity arises. The fix is twofold: improve the SDK auto-detection so contributors who have Microsoft fxc installed don't have to fight the script, and add an output validator that refuses to write .inc files produced by any compiler that isn't Microsoft fxc. Auto-detection improvements (find_fxc): * Wine prefix discovery widened. Previously the script only checked $WINEPREFIX/drive_c if WINEPREFIX was set. Now it also checks ~/.wine (the Wine default when WINEPREFIX is unset), and any prefix directory under ~/.local/share/wineprefixes/ that has a drive_c subtree (PlayOnLinux / lutris-style layouts). Both /Program Files/ and /Program Files (x86)/ are searched in each prefix. * Error message now points at the README's installation steps rather than telling the user to just install the SDK. * Error message explicitly calls out that Wine's bundled d3dcompiler_47.dll is NOT a substitute and referencesda20f5bfor context. Output validator (extract_rdef_creator + check_microsoft_creator): * extract_rdef_creator walks the DXBC container chunk index, finds the RDEF chunk, follows the creator_offset field at chunk body offset 24, and reads back the null-terminated ASCII creator string. The DXBC container layout (16-byte hash + version + total size + chunk count + chunk offsets) and the RDEF body layout (constant_buffer_count, constant_buffer_offset, resource_binding_count, resource_binding_offset, shader_version, compiler_flags, creator_offset) are documented in MSDN's old D3D11 reflection notes and in the wikitalk pages around DXBC; the implementation here uses fixed offsets that match what fxc.exe and vkd3d-shader both produce. * check_microsoft_creator hooks into compile_one after the magic check. It refuses with a clear error message if the creator string doesn't contain "Microsoft". Microsoft fxc stamps something like "Microsoft (R) HLSL Shader Compiler 10.1"; vkd3d-shader stamps "vkd3d-shader 1.10 (Wine bundled)". One substring check disambiguates. What the validator does NOT do: * It does not retroactively reject existing .inc files in the repo. The validator only fires on freshly-produced DXBC during a regen run. The vkd3d-shader-built fullscreen_quad_vs.inc and copy_ps.inc are unchanged by this commit; they'll get refreshed the next time a contributor with Microsoft fxc runs the regen script. Until then, they keep working because they're already in the repo and the build just compiles them as plain C++ byte arrays - they never go through PSO validation (they're already valid DXBC). * It does not lock the script to a specific fxc version. Any Microsoft fxc.exe (10.1, 10.5, the version that ships with the Windows 11 SDK, etc.) is accepted - they all stamp "Microsoft" in the creator string. * It does not validate the binding layout vs the runtime root signature. That's a future enhancement - the runtime side expects specific (b0, t0, s0) layouts and the regen script could cross-check, but that's a much bigger feature. README changes: * New "Microsoft fxc.exe is required" section spells out why and cites thecd971cd/da20f5bfailure as the worked example. * "Installing Microsoft fxc.exe" subsection with separate Windows / Linux instructions. Linux gets two paths: install the SDK under a Wine prefix (auto-detected), or extract fxc.exe + d3dcompiler_47.dll directly from the MSI with msitools and pass --fxc. Both must live in the same dir so Wine's loader prefers Microsoft's d3dcompiler_47.dll over its bundled vkd3d-shader version. * The auto-detection scope (WINEPREFIX, ~/.wine, ~/.local/share/ wineprefixes/*) is now documented so contributors know where the script is looking. This commit does NOT re-add the vram_copy.ps.hlsl source removed byda20f5b. That comes back in a follow-up patch once a Microsoft fxc-built .inc is available - same patch lands the .hlsl, the .inc, and the call-site flip together (thedd54a73lesson). Test: - python3 -c 'import ast; ast.parse(open("tools/regen_d3d12_dxbc.py").read())' passes. - Loading the script as a module and feeding the EXISTING vkd3d-built copy_ps.inc into extract_rdef_creator returns 'vkd3d-shader 1.10 (Wine bundled)'; the Microsoft substring check returns False. Confirms the validator would catch thecd971cdmistake if anyone tries to reproduce it. - find_fxc's broader Wine prefix scan is exercised by the standard test of running the script in an environment without Microsoft fxc; it now fails with the new pointed-at-README message instead of the old generic SDK message.
This commit is contained in:
parent
da20f5b80e
commit
ebd2300cde
2 changed files with 180 additions and 19 deletions
|
|
@ -51,11 +51,53 @@ file and rewrites the matching `.inc` under `src/common/d3d12/embedded_dxbc/`.
|
|||
Both the `.hlsl` and the regenerated `.inc` must be committed together so the
|
||||
two stay in sync.
|
||||
|
||||
`fxc.exe` ships with the Windows 10 SDK and is **not** invoked by the core
|
||||
build. On Linux, run the script under Wine - it will look for `fxc.exe` in
|
||||
the standard SDK install path under `$WINEPREFIX/drive_c/Program Files (x86)/Windows Kits/10/bin/<version>/x64/fxc.exe`,
|
||||
or you can pass `--fxc PATH` explicitly. The `--wine PATH` flag lets you
|
||||
point at a specific wine binary if `wine` isn't on `PATH`.
|
||||
### Microsoft `fxc.exe` is required
|
||||
|
||||
The regen script will refuse to write `.inc` files produced by any compiler
|
||||
other than Microsoft's `fxc.exe`. The check parses the DXBC `RDEF` chunk and
|
||||
matches the creator string against a Microsoft-prefix allowlist.
|
||||
|
||||
The reason is empirical: Wine ships its own `d3dcompiler_47.dll`
|
||||
implementation (vkd3d-shader), and that compiler's output, while a valid
|
||||
DXBC container, doesn't pass the Microsoft D3D12 runtime's
|
||||
`CreateGraphicsPipelineState` validation for non-trivial shaders. The
|
||||
failure mode is `E_FAIL` (0x80004005) at PSO-creation time with no
|
||||
useful diagnostic from `Log_ErrorPrintf`. Commit cd971cd shipped a
|
||||
vkd3d-shader-compiled `vram_copy_ps.inc` and broke the entire D3D12
|
||||
backend on launch; da20f5b reverts it. The script's creator-string
|
||||
validator exists to prevent that mistake from recurring.
|
||||
|
||||
### Installing Microsoft `fxc.exe`
|
||||
|
||||
**Windows**: Install the Windows 10 or 11 SDK from Microsoft. `fxc.exe`
|
||||
ends up at
|
||||
`C:\Program Files (x86)\Windows Kits\10\bin\<version>\x64\fxc.exe` (or
|
||||
the same path under `\Program Files\` if the 64-bit SDK is installed).
|
||||
The regen script auto-detects it there.
|
||||
|
||||
**Linux**: Microsoft's `fxc.exe` is a Windows x86 PE binary and runs
|
||||
fine under Wine - the trick is just to get the SDK installed inside a
|
||||
Wine prefix. Two options:
|
||||
|
||||
* Run the Windows SDK installer (`winsdksetup.exe` from Microsoft) under
|
||||
`wine` and select the Windows SDK Signing Tools / SDK component that
|
||||
ships `fxc.exe`. The default install puts it at
|
||||
`<prefix>/drive_c/Program Files (x86)/Windows Kits/10/bin/<version>/x64/fxc.exe`,
|
||||
which the regen script picks up automatically.
|
||||
* Or extract `fxc.exe` + `d3dcompiler_47.dll` directly from the SDK MSI
|
||||
using `msiextract` from `msitools`, drop both in any directory, and
|
||||
pass `--fxc PATH/TO/fxc.exe` to the regen script. Both files must
|
||||
live in the same directory so Wine's loader finds Microsoft's
|
||||
`d3dcompiler_47.dll` before falling back to its bundled
|
||||
vkd3d-shader version.
|
||||
|
||||
The script searches for `fxc.exe` in: `$WINEPREFIX` (if set), `~/.wine`,
|
||||
and any subdirectory under `~/.local/share/wineprefixes/` that has a
|
||||
`drive_c` tree. Set `WINEPREFIX` to your prefix path if it lives
|
||||
elsewhere.
|
||||
|
||||
The `--wine PATH` flag lets you point at a specific wine binary if
|
||||
`wine` isn't on `$PATH`.
|
||||
|
||||
## Filename convention
|
||||
|
||||
|
|
|
|||
|
|
@ -60,9 +60,21 @@ TEMPLATE_VARIANTS = {}
|
|||
|
||||
|
||||
def find_fxc(explicit_fxc, explicit_wine):
|
||||
# On Windows, look for fxc.exe in the Windows 10 SDK install location
|
||||
# first, then PATH. On Linux, require Wine + explicit --fxc pointing at
|
||||
# the Windows fxc.exe install.
|
||||
# On Windows, look for fxc.exe in the Windows 10 / 11 SDK install
|
||||
# location first, then PATH. On Linux, look in any Wine prefix
|
||||
# we can find that has the SDK installed; only fall back to the
|
||||
# --fxc override when the user explicitly points us at one.
|
||||
#
|
||||
# Microsoft's actual fxc.exe is required - Wine's bundled
|
||||
# d3dcompiler_47.dll (which is the vkd3d-shader reimplementation)
|
||||
# produces DXBC that doesn't pass Microsoft D3D12 PSO validation
|
||||
# for non-trivial shaders. cd971cd / da20f5b chronicles a
|
||||
# vram_copy_ps pre-bake attempt that failed with
|
||||
# CreateGraphicsPipelineState E_FAIL because the .inc was built
|
||||
# with vkd3d-shader. The check_microsoft_creator() validator
|
||||
# below rejects vkd3d-compiled output post-hoc, but it's better
|
||||
# to find Microsoft's fxc.exe up-front and skip the wasted
|
||||
# compile time entirely. See data/shaders/d3d12/README.md.
|
||||
if explicit_fxc:
|
||||
fxc = Path(explicit_fxc)
|
||||
if not fxc.exists():
|
||||
|
|
@ -77,17 +89,32 @@ def find_fxc(explicit_fxc, explicit_wine):
|
|||
if path:
|
||||
return [path]
|
||||
|
||||
# Try the common Windows 10 SDK install location even when not on
|
||||
# Windows (in case the user has the SDK mounted under Wine prefix).
|
||||
# SDK auto-detection. The Windows 10 / 11 SDK installs fxc.exe at
|
||||
# <SDK_root>/<version>/x64/fxc.exe. Try the standard install
|
||||
# locations on Windows AND under every plausible Wine prefix on
|
||||
# Linux. The /Program Files/ and /Program Files (x86)/ split
|
||||
# depends on whether the SDK was installed as 32-bit or 64-bit;
|
||||
# check both. Wine prefix discovery covers $WINEPREFIX (if set),
|
||||
# the default ~/.wine, and any prefixes under ~/.local/share/
|
||||
# wineprefixes/ that PlayOnLinux and lutris-style installs use.
|
||||
sdk_roots = [
|
||||
Path("C:/Program Files (x86)/Windows Kits/10/bin"),
|
||||
Path("C:/Program Files/Windows Kits/10/bin"),
|
||||
]
|
||||
# Add Wine-prefix style path if we have one.
|
||||
wine_prefix = os.environ.get("WINEPREFIX")
|
||||
if wine_prefix:
|
||||
sdk_roots.append(Path(wine_prefix) / "drive_c" / "Program Files (x86)" / "Windows Kits" / "10" / "bin")
|
||||
sdk_roots.append(Path(wine_prefix) / "drive_c" / "Program Files" / "Windows Kits" / "10" / "bin")
|
||||
wine_prefixes = []
|
||||
if "WINEPREFIX" in os.environ:
|
||||
wine_prefixes.append(Path(os.environ["WINEPREFIX"]))
|
||||
default_prefix = Path.home() / ".wine"
|
||||
if default_prefix.exists() and default_prefix not in wine_prefixes:
|
||||
wine_prefixes.append(default_prefix)
|
||||
extra_prefix_dir = Path.home() / ".local" / "share" / "wineprefixes"
|
||||
if extra_prefix_dir.exists():
|
||||
for child in extra_prefix_dir.iterdir():
|
||||
if child.is_dir() and (child / "drive_c").exists():
|
||||
wine_prefixes.append(child)
|
||||
for prefix in wine_prefixes:
|
||||
sdk_roots.append(prefix / "drive_c" / "Program Files (x86)" / "Windows Kits" / "10" / "bin")
|
||||
sdk_roots.append(prefix / "drive_c" / "Program Files" / "Windows Kits" / "10" / "bin")
|
||||
|
||||
for root in sdk_roots:
|
||||
if not root.exists():
|
||||
|
|
@ -102,13 +129,101 @@ def find_fxc(explicit_fxc, explicit_wine):
|
|||
[explicit_wine or "wine", str(candidate)])
|
||||
|
||||
sys.stderr.write(
|
||||
"error: fxc.exe not found. Install the Windows 10 SDK (or a newer\n"
|
||||
" Windows SDK that still ships fxc.exe), or pass --fxc PATH\n"
|
||||
" pointing at a working fxc.exe binary. On Linux pass --wine\n"
|
||||
" PATH if `wine` isn't on PATH.\n")
|
||||
"error: fxc.exe not found. The Microsoft Windows 10/11 SDK ships\n"
|
||||
" it under <SDK>/bin/<version>/x64/fxc.exe. See\n"
|
||||
" data/shaders/d3d12/README.md for installation steps,\n"
|
||||
" including how to set up the SDK under a Wine prefix\n"
|
||||
" for Linux contributors. If you have fxc.exe elsewhere,\n"
|
||||
" pass --fxc PATH. On Linux pass --wine PATH if `wine`\n"
|
||||
" isn't on PATH.\n"
|
||||
"\n"
|
||||
" Wine's bundled d3dcompiler_47.dll (vkd3d-shader\n"
|
||||
" reimplementation) is NOT a substitute - its DXBC output\n"
|
||||
" fails Microsoft D3D12 PSO validation. See da20f5b for\n"
|
||||
" a worked example.\n")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def extract_rdef_creator(data):
|
||||
# Inspect the DXBC RDEF chunk and pull out the compiler-creator
|
||||
# string. The DXBC container is:
|
||||
# [0:4] magic "DXBC"
|
||||
# [4:20] 16-byte content hash
|
||||
# [20:24] container version
|
||||
# [24:28] total size
|
||||
# [28:32] chunk count
|
||||
# [32:..] uint32 offsets, one per chunk
|
||||
# Each chunk header is 8 bytes (4-byte magic + 4-byte size) and
|
||||
# is followed by chunk-type-specific payload. The RDEF chunk
|
||||
# body lays out as:
|
||||
# [0:4] constant_buffer_count
|
||||
# [4:8] constant_buffer_offset
|
||||
# [8:12] resource_binding_count
|
||||
# [12:16] resource_binding_offset
|
||||
# [16:20] shader_version (program_type | major | minor)
|
||||
# [20:24] compiler_flags
|
||||
# [24:28] creator_offset (relative to body_start)
|
||||
# Offsets in the RDEF body are relative to the START of the
|
||||
# chunk body, NOT the start of the file - that's the same
|
||||
# convention DXBC uses internally for SGN tables.
|
||||
chunk_count_bytes = data[28:32]
|
||||
if len(chunk_count_bytes) != 4:
|
||||
return None
|
||||
chunk_count = struct.unpack("<I", chunk_count_bytes)[0]
|
||||
chunk_offsets_bytes = data[32:32 + 4 * chunk_count]
|
||||
if len(chunk_offsets_bytes) != 4 * chunk_count:
|
||||
return None
|
||||
chunk_offsets = struct.unpack(f"<{chunk_count}I", chunk_offsets_bytes)
|
||||
for off in chunk_offsets:
|
||||
if off + 8 > len(data):
|
||||
continue
|
||||
magic = data[off:off + 4]
|
||||
if magic != b"RDEF":
|
||||
continue
|
||||
body_start = off + 8
|
||||
if body_start + 28 > len(data):
|
||||
return None
|
||||
creator_offset = struct.unpack("<I", data[body_start + 24:body_start + 28])[0]
|
||||
creator_addr = body_start + creator_offset
|
||||
# Creator string is null-terminated ASCII.
|
||||
end = data.find(b"\0", creator_addr)
|
||||
if end == -1:
|
||||
return None
|
||||
return data[creator_addr:end].decode("ascii", errors="replace")
|
||||
return None
|
||||
|
||||
|
||||
def check_microsoft_creator(data, hlsl_path):
|
||||
# Reject anything that isn't from Microsoft's fxc.exe. The
|
||||
# creator string lives in the RDEF chunk and is what the
|
||||
# compiler stamps as its identifier:
|
||||
#
|
||||
# Microsoft fxc: "Microsoft (R) HLSL Shader Compiler 10.1"
|
||||
# (or similar - version trails the prefix)
|
||||
# Wine vkd3d-shader: "vkd3d-shader 1.10 (Wine bundled)"
|
||||
#
|
||||
# Microsoft fxc.exe is the only compiler whose DXBC output is
|
||||
# known to pass D3D12 PSO validation for the shaders this
|
||||
# backend uses. Other compilers may produce bytecode that's
|
||||
# technically valid DXBC but trips PSO creation with E_FAIL.
|
||||
# cd971cd / da20f5b is the worked example.
|
||||
creator = extract_rdef_creator(data)
|
||||
if creator is None:
|
||||
sys.stderr.write(
|
||||
f"error: could not parse RDEF / creator string from output\n"
|
||||
f" for {hlsl_path.name}. The DXBC structure may be\n"
|
||||
f" malformed; refusing to write .inc.\n")
|
||||
sys.exit(1)
|
||||
if "Microsoft" not in creator:
|
||||
sys.stderr.write(
|
||||
f"error: {hlsl_path.name} compiled with non-Microsoft fxc.\n"
|
||||
f" Creator string: {creator!r}\n"
|
||||
f" Microsoft's fxc.exe is required - see\n"
|
||||
f" data/shaders/d3d12/README.md. da20f5b chronicles\n"
|
||||
f" the failure mode when other compilers are used.\n")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def sanitize_identifier(stem):
|
||||
# Map "vram_read.ps" -> "vram_read_ps"; only [A-Za-z0-9_] survive.
|
||||
name = re.sub(r"[^A-Za-z0-9_]", "_", stem)
|
||||
|
|
@ -152,6 +267,10 @@ def compile_one(fxc_cmd, hlsl_path, target, defines=None):
|
|||
f"error: {hlsl_path.name} produced output with bad magic "
|
||||
f"(0x{magic:08x}, expected 0x43425844)\n")
|
||||
sys.exit(1)
|
||||
# Reject non-Microsoft compilers. The check parses the RDEF chunk
|
||||
# for the creator string; Wine's bundled d3dcompiler_47.dll
|
||||
# (vkd3d-shader) gets refused here.
|
||||
check_microsoft_creator(data, hlsl_path)
|
||||
return data
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue