-
Notifications
You must be signed in to change notification settings - Fork 722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[OMM] Implement front end diagnostics for OMM, including on TraceRayInline #7156
base: main
Are you sure you want to change the base?
Conversation
✅ With the latest revision this PR passed the C/C++ code formatter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couple of questions, but otherwise looks good to my untrained eye.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good - a few style nits and I think I spotted a few places where there's some unwanted copies happening.
tools/clang/lib/Sema/SemaHLSL.cpp
Outdated
@@ -14161,6 +14161,97 @@ static bool IsUsageAttributeCompatible(AttributeList::Kind kind, bool &usageIn, | |||
return true; | |||
} | |||
|
|||
bool DiagnoseRayQueryObject(QualType qt, Declarator &D, Sema &S) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bool DiagnoseRayQueryObject(QualType qt, Declarator &D, Sema &S) { | |
static bool DiagnoseRayQueryObject(QualType qt, Declarator &D, Sema &S) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Partial review, see comments so far.
lib/DxcSupport/FileIOHelper.cpp
Outdated
@@ -1369,6 +1369,8 @@ class ReadOnlyBlobStream : public IStream { | |||
return E_POINTER; | |||
ULONG cbLeft = m_size - m_offset; | |||
*pcbRead = std::min(cb, cbLeft); | |||
if (m_pMemory + m_offset == 0) | |||
return S_FALSE; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain why this change is made at this time? Was m_pMemory
uninitialized (nullptr
), when attempting to read? If so, what code path hit this?
Can this just check m_pMemory
for nullptr
and abort with E_FAIL
or something like that earlier, since we shouldn't find ourselves reading from an uninitialized stream object in the first place?
If we do want to return S_FALSE
for this case, we could still check m_pMemory
for nullpltr
earlier and just explicitly set *pcbRead = 0
, since it must be set that way for this code path. I realize that if other assumptions hold for an uninitialized object, m_offset
should be zero (hopefully), and m_size
should be zero, so cbLeft
should end up being zero, which will set the *pcbRead
to zero, but why bother with all those assumptive dependencies when m_pMemory
is nullpltr
?
In any case, we should have a unit test for this change. Perhaps this change can be a separate bugfix PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe there was some CompileTest that was failing, because the addition resulted in 0, so it was being treated as a nullptr when passed to memcpy. But I am no longer seeing this. I'll undo these 2 lines, and if it pops back up again I'll put up a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, it appeared again. Here's the output for the NIX Linux Clang release pipeline:
FAIL: Clang-Unit :: HLSL/ClangHLSLTests/CompilerTest.CodeGenHashStabilityHLSL (1674 of 4075)
******************** TEST 'Clang-Unit :: HLSL/ClangHLSLTests/CompilerTest.CodeGenHashStabilityHLSL' FAILED ********************
Note: Google Test filter = CompilerTest.CodeGenHashStabilityHLSL
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from CompilerTest
/home/vsts/work/1/s/lib/DxcSupport/FileIOHelper.cpp:1372:16: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/vsts/work/1/s/lib/DxcSupport/FileIOHelper.cpp:1372:16
********************
This points to the fact that the sum ends up as 0, and is being treated as a nullptr.
I don't see a straightforward way to test this, as this is deep deep into the call stack, and deals with a private member variable of a stream object type.
Also, I don't think setting pcbRead to 0 will solve the issue, since m_pMemory + m_offset still sums to 0, and the test is failing because the function specified that it should never receive a nullptr as the second argument.
I'll reintroduce the original addition check, and return S_FALSE, though switching to an E_FAIL may be the right move later on. I can do some more investigating as to why this specific change serves as a unit test for this if condition.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Further detail, here is the call stack:
hlsl::ReadOnlyBlobStream::Read(void * pv, unsigned long cb, unsigned long * pcbRead) Line 1373 C++
ReadAllBytes(IStream * pStream, void * pDst, unsigned __int64 uSize) Line 82 C++
PDBReader::ReadSuperblock(MSF_SuperBlock * pSB) Line 375 C++
PDBReader::PDBReader(IMalloc * pMalloc, IStream * pStream) Line 365 C++
hlsl::pdb::LoadDataFromStream(IMalloc * pMalloc, IStream * pIStream, IDxcBlob * * ppHash, IDxcBlob * * ppContainer) Line 494 C++
hlsl::pdb::LoadDataFromStream(IMalloc * pMalloc, IStream * pIStream, IDxcBlob * * ppContainer) Line 534 C++
DxilContainerReflection::Load(IDxcBlob * pContainer) Line 409 C++
CompileForHash(hlsl::options::DxcOpts & opts, const wchar_t * CommandFileName, dxc::DxcDllSupport & DllSupport, std::vector<wchar_t const *,std::allocator<wchar_t const *>> & flags, IDxcBlob * * ppHashBlob, std::string & output) Line 345 C++
FileRunCommandPart::RunDxcHashTest(dxc::DxcDllSupport & DllSupport) Line 410 C++
FileRunCommandPart::RunHashTests(dxc::DxcDllSupport & DllSupport) Line 64 C++
FileRunTestResultImpl::RunHashTestFromCommands(const char * commands, const wchar_t * fileName) Line 1280 C++
FileRunTestResultImpl::RunHashTestFromFileCommands(const wchar_t * fileName) Line 1375 C++
FileRunTestResult::RunHashTestFromFileCommands(const wchar_t * fileName) Line 1385 C++
> CompilerTest::CodeGenTestHashFullPath(const wchar_t * fullPath) Line 403 C++
The tools\clang\test\HLSLFileCheck\hlsl\objects\RayQuery\rayquery-trace-ray-inline-errors.hlsl
is causing the error. Seems like something related to not initializing the PDBReader object correctly, which should stem from the incorrect pStream initialiation on IFR(hlsl::CreateReadOnlyBlobStream(pContainer, &pStream));
, located at
D:\DXC\lib\HLSL\DxilContainerReflection.cpp::408
Turns out it was initialized incorrectly because in FileCheckerTest.cpp
, on line 329, IFT(pCompiler2->CompileWithDebug(pSource, CommandFileName, entry.c_str(), profile.c_str(), flags.data(), flags.size(), nullptr, 0, pIncludeHandler, &pResult, &pDebugName, &pPDBBlob));
Didn't produce any output (because the -verify flag was given, no output was produced by the compiler)
So, with no output, pCompiledBlob
is empty / has no memory. This emptiness is the nullptr that gets passed along to the point where the nullptr is detected.
TLDR: the -verify file rayquery-trace-ray-inline-errors.hlsl
doesn't belong in the file-check directory, it needs to go somewhere where the CompilerTest doesn't check, not in a directory where all files are expected to have compiler output. Oops!
@@ -7652,8 +7652,17 @@ def err_payload_fields_is_payload_and_overqualified : Error< | |||
"payload field '%0' is a payload struct. Payload access qualifiers are not allowed on payload types.">; | |||
def warn_hlsl_payload_qualifer_dropped : Warning< | |||
"payload access qualifiers ignored. These are only supported for lib_6_7+ targets and lib_6_6 with with the -enable-payload-qualifiers flag.">, InGroup<HLSLPayloadAccessQualifer>; | |||
def warn_hlsl_unexpected_value_for_ray_query_flags_template_arg : Warning< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The warning isn't what I expected from the name. How about this?
def warn_hlsl_unexpected_value_for_ray_query_flags_template_arg : Warning< | |
def warn_hlsl_rayquery_flags_disallowed: Warning< |
I also joined rayquery
for consistency with object and other warning names below.
@@ -7652,8 +7652,17 @@ def err_payload_fields_is_payload_and_overqualified : Error< | |||
"payload field '%0' is a payload struct. Payload access qualifiers are not allowed on payload types.">; | |||
def warn_hlsl_payload_qualifer_dropped : Warning< | |||
"payload access qualifiers ignored. These are only supported for lib_6_7+ targets and lib_6_6 with with the -enable-payload-qualifiers flag.">, InGroup<HLSLPayloadAccessQualifer>; | |||
def warn_hlsl_unexpected_value_for_ray_query_flags_template_arg : Warning< | |||
"A non-zero value for the RayQueryFlags template argument requires shader model 6.9 or above.">, DefaultError; | |||
def warn_hlsl_unexpected_rayquery_flag : Warning< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about:
def warn_hlsl_unexpected_rayquery_flag : Warning< | |
def warn_hlsl_rayquery_flags_conflict : Warning< |
@@ -7652,8 +7652,17 @@ def err_payload_fields_is_payload_and_overqualified : Error< | |||
"payload field '%0' is a payload struct. Payload access qualifiers are not allowed on payload types.">; | |||
def warn_hlsl_payload_qualifer_dropped : Warning< | |||
"payload access qualifiers ignored. These are only supported for lib_6_7+ targets and lib_6_6 with with the -enable-payload-qualifiers flag.">, InGroup<HLSLPayloadAccessQualifer>; | |||
def warn_hlsl_unexpected_value_for_ray_query_flags_template_arg : Warning< | |||
"A non-zero value for the RayQueryFlags template argument requires shader model 6.9 or above.">, DefaultError; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These may be DefaultError, but are kind of pointless as warnings unless you add them to a warning group to allow overriding of the setting with -W*
options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I've put the two warnings of interest into the HLSLAvailabilityConstant
group
def warn_hlsl_unexpected_rayquery_flag : Warning< | ||
"When using 'RAY_FLAG_FORCE_OMM_2_STATE' in RayFlags, RayQueryFlags must have RAYQUERY_FLAG_ALLOW_OPACITY_MICROMAPS set.">, DefaultError; | ||
def note_hlsl_unexpected_rayquery_flag : Note< | ||
"RayQueryFlags must have RAYQUERY_FLAG_ALLOW_OPACITY_MICROMAPS set here.">; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's sufficient to use the existing note_previous_decl
for this instead.
def note_previous_decl : Note<"%0 declared here">;
bool IsRayFlagForceOMM2StateSet(const CallExpr *CE) { | ||
const ImplicitCastExpr *RayFlagArg = | ||
dyn_cast<ImplicitCastExpr>(CE->getArg(1)); | ||
if (!RayFlagArg) | ||
return false; | ||
|
||
const DeclRefExpr *DR = dyn_cast<DeclRefExpr>(RayFlagArg->getSubExpr()); | ||
if (DR) { | ||
const ValueDecl *valueDecl = DR->getDecl(); | ||
const VarDecl *varDecl = dyn_cast<VarDecl>(valueDecl); | ||
|
||
if (varDecl) { | ||
const Expr *initializer = varDecl->getInit(); | ||
if (!initializer) | ||
return false; | ||
|
||
const IntegerLiteral *intLit = dyn_cast<IntegerLiteral>(initializer); | ||
if (!intLit) | ||
return false; | ||
|
||
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | ||
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | ||
return true; | ||
} | ||
} | ||
} else if (const IntegerLiteral *intLit = | ||
dyn_cast<IntegerLiteral>(RayFlagArg->getSubExpr())) { | ||
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | ||
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) | ||
return true; | ||
} | ||
return false; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should simply be trying to check a constant expression, and evaluating the expression to look for the flag value, not drilling specifically into a DeclRefExpr
. Something like this?
bool IsRayFlagForceOMM2StateSet(const CallExpr *CE) { | |
const ImplicitCastExpr *RayFlagArg = | |
dyn_cast<ImplicitCastExpr>(CE->getArg(1)); | |
if (!RayFlagArg) | |
return false; | |
const DeclRefExpr *DR = dyn_cast<DeclRefExpr>(RayFlagArg->getSubExpr()); | |
if (DR) { | |
const ValueDecl *valueDecl = DR->getDecl(); | |
const VarDecl *varDecl = dyn_cast<VarDecl>(valueDecl); | |
if (varDecl) { | |
const Expr *initializer = varDecl->getInit(); | |
if (!initializer) | |
return false; | |
const IntegerLiteral *intLit = dyn_cast<IntegerLiteral>(initializer); | |
if (!intLit) | |
return false; | |
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | |
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | |
return true; | |
} | |
} | |
} else if (const IntegerLiteral *intLit = | |
dyn_cast<IntegerLiteral>(RayFlagArg->getSubExpr())) { | |
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | |
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) | |
return true; | |
} | |
return false; | |
} | |
bool IsRayFlagForceOMM2StateSet(const CallExpr *CE) { | |
Expr *E = CE->getArg(1); | |
llvm::APSInt constantResult; | |
return E->isIntegerConstantExpr(constantResult, sema->getASTContext()) && | |
(constantResult.getLimitedValue() & | |
(uint64_t)DXIL::RayFlag::ForceOMM2State != 0); | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, after writing this, I don't think this function is even used. It should be deleted then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Made use of the function and your suggestion 👍
// validate that if the ray flag parameter is | ||
// RAY_FLAG_FORCE_OMM_2_STATE, the ray query object | ||
// has the RAYQUERY_FLAG_ALLOW_OPACITY_MICROMAPS | ||
// flag set, otherwise emit a diagnostic |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not "if the ray flag parameter is RAY_FLAG_FORCE_OMM_2_STATE", but "if the ray flag parameter has the RAY_FLAG_FORCE_OMM_2_STATE flag set".
// validate that if the ray flag parameter is | |
// RAY_FLAG_FORCE_OMM_2_STATE, the ray query object | |
// has the RAYQUERY_FLAG_ALLOW_OPACITY_MICROMAPS | |
// flag set, otherwise emit a diagnostic | |
// If the RayFlag parameter has RAY_FLAG_FORCE_OMM_2_STATE set, | |
// the RayQuery decl must have RAYQUERY_FLAG_ALLOW_OPACITY_MICROMAPS set, | |
// otherwise emit a diagnostic. |
bool IsRayFlagForceOMM2State = false; | ||
if (ImplicitCastExpr *RayFlagArg = | ||
dyn_cast<ImplicitCastExpr>(callExpr->getArg(1))) { | ||
if (DeclRefExpr *DR = dyn_cast<DeclRefExpr>(RayFlagArg->getSubExpr())) { | ||
ValueDecl *valueDecl = DR->getDecl(); | ||
VarDecl *varDecl = dyn_cast<VarDecl>(valueDecl); | ||
|
||
if (varDecl) { | ||
Expr *initializer = varDecl->getInit(); | ||
|
||
if (initializer) { | ||
if (const IntegerLiteral *intLit = | ||
dyn_cast<IntegerLiteral>(initializer)) { | ||
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | ||
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | ||
IsRayFlagForceOMM2State = true; | ||
} | ||
} | ||
} | ||
} | ||
} else if (const IntegerLiteral *intLit = | ||
dyn_cast<IntegerLiteral>(RayFlagArg->getSubExpr())) { | ||
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | ||
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | ||
IsRayFlagForceOMM2State = true; | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should just use isIntegerConstantExpr
to get the value from the ICE if possible.
bool IsRayFlagForceOMM2State = false; | |
if (ImplicitCastExpr *RayFlagArg = | |
dyn_cast<ImplicitCastExpr>(callExpr->getArg(1))) { | |
if (DeclRefExpr *DR = dyn_cast<DeclRefExpr>(RayFlagArg->getSubExpr())) { | |
ValueDecl *valueDecl = DR->getDecl(); | |
VarDecl *varDecl = dyn_cast<VarDecl>(valueDecl); | |
if (varDecl) { | |
Expr *initializer = varDecl->getInit(); | |
if (initializer) { | |
if (const IntegerLiteral *intLit = | |
dyn_cast<IntegerLiteral>(initializer)) { | |
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | |
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | |
IsRayFlagForceOMM2State = true; | |
} | |
} | |
} | |
} | |
} else if (const IntegerLiteral *intLit = | |
dyn_cast<IntegerLiteral>(RayFlagArg->getSubExpr())) { | |
unsigned int rayFlagValue = intLit->getValue().getZExtValue(); | |
if (rayFlagValue & (unsigned int)DXIL::RayFlag::ForceOMM2State) { | |
IsRayFlagForceOMM2State = true; | |
} | |
} | |
} | |
Expr *Expr1 = CE->getArg(1); | |
llvm::APSInt constantResult; | |
bool IsRayFlagForceOMM2State = | |
Expr1->isIntegerConstantExpr(constantResult, sema->getASTContext()) && | |
(constantResult.getLimitedValue() & | |
(uint64_t)DXIL::RayFlag::ForceOMM2State) != 0; |
auto *attr = methodDecl->getAttr<HLSLIntrinsicAttr>(); | ||
if (!attr) | ||
return true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to check group before assuming you can cast and compare the opcode.
auto *attr = methodDecl->getAttr<HLSLIntrinsicAttr>(); | |
if (!attr) | |
return true; | |
auto *attr = methodDecl->getAttr<HLSLIntrinsicAttr>(); | |
if (!attr) | |
return true; | |
StringRef OpcodeGroup = GetHLOpcodeGroupName(HLOpcodeGroup::HLIntrinsic); | |
if (attr->getGroup() != OpcodeGroup) | |
return true; |
if (IsRayFlagForceOMM2State) { | ||
const DeclRefExpr *DRE = dyn_cast<DeclRefExpr>(ME->getBase()); | ||
assert(DRE); | ||
const VarDecl *VD = cast<VarDecl>(DRE->getDecl()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is the right approach. I think you want to call getImplicitObjectArgument()
on the CXXMemberCallExpr
instead. Then use the expression's type to drill into the template argument.
dyn_cast<DeclRefExpr>(callExpr->getImplicitObjectArgument()); | ||
assert(DRE); | ||
QualType QT = DRE->getType(); | ||
auto typeRecordDecl = QT->getAsCXXRecordDecl(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
auto typeRecordDecl = QT->getAsCXXRecordDecl(); | |
auto *typeRecordDecl = QT->getAsCXXRecordDecl(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved, assuming the test failures are fixed and Tex is happy you've addressed his feedback.
This PR addresses the front end part of OMM, defining the new flags defined in the spec, and implementing the relevant diagnostics should the flags be incompatible. It also adds the second template argument to the RayQuery object, which is set to have a default value of 0 if no explicit template argument is provided.
Fixes #7145