Data flow: Fix a bad join order #18457
Merged
+7
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
Files not reviewed (1)
- shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qll: Language not supported
Tip: If you use Visual Studio Code, you can request a review from Copilot before you push from the "Source Control" tab. Learn more
aschackmull
reviewed
Jan 10, 2025
| if allowParameterReturnInSelfEx(p) | ||
| then result.isNone() | ||
| else p.isParameterOf(_, result.asSome().(ParamUpdateReturnKind).getPosition()) | ||
| } | ||
|
|
||
| bindingset[p] | ||
| pragma[inline_late] | ||
| private ReturnKindExtOption getDisallowedReturnKind(ParamNodeEx p) { |
There was a problem hiding this comment.
What's going wrong? This predicate is binary, nomagic'ed and only used in two (nomagic'ed) contexts where the result isn't otherwise bound, so I fail to see what this transformation achieves?
There was a problem hiding this comment.
It is needed because fwdFlowIn is inlined inside fwdFlowIsEntered. Without this change, each recursive iteration of fwdFlowIsEntered starts at getDisallowedReturnKind, instead of the delta. The DIL before is
noinline
nomagic
incremental
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowIsEntered/3#137febc2`(
/* DataFlowDispatch::DataFlowCall */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCall call,
/* Option::Option<DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::ReturnKindExt>::Option */ `Option#8eb11f23::Option<DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::ReturnKindExt>::TOption` disallowReturnKind,
boolean cc
)
{
[base_case] false()
[recursive_case]
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::ParamNodeEx */ `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` p
|
(
(
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` dummyField
|
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlow/2#65dc322b`(dummyField,
cc) and
`DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::viableParamArgEx/3`(call,
p, dummyField)
) and
(
cc = false
or
(
cc = true and
not(
`project#DataFlowImplCommon::Cached::CachedCallContextSensitivity::reducedViableImplInCallContext/3#f0f91b5d`(call)
)
)
) and
not(
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::fullBarrier/1#e25deefa`(p)
)
)
or
exists(
/* DataFlowDispatch::DataFlowCallable */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCallable target
|
cc = true and
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::viableImplInSomeFwdFlowCallContextExt/1#23a13296`(call,
target) and
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ dontcare `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` _
|
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowInReducedViableImplInSomeCallContext/4#ccb02962`(call,
_, p, target)
)
)
or
exists(
/* DataFlowDispatch::DataFlowCallable */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCallable target
|
cc = true and
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ dontcare `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` _
|
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowInReducedViableImplInSomeCallContext/4#ccb02962`(call,
_, p, target)
) and
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::viableImplInSomeFwdFlowCallContextExt/1#23a13296`(call,
target)
)
) and
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::getDisallowedReturnKind0/1#0f59df6c`(p,
disallowReturnKind)
) and
not(
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowIsEntered/3#137febc2`(call,
disallowReturnKind, cc)
)
}
and afterwards
noinline
nomagic
incremental
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowIsEntered/3#137febc2`(
/* DataFlowDispatch::DataFlowCall */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCall call,
/* Option::Option<DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::ReturnKindExt>::Option */ `Option#8eb11f23::Option<DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::ReturnKindExt>::TOption` disallowReturnKind,
boolean cc
)
{
[base_case] false()
[recursive_case]
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::ParamNodeEx */ `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` p
|
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::getDisallowedReturnKind/1#a2c82816`(p,
disallowReturnKind) and
(
(
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` dummyField
|
not(
`DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::fullBarrier/1#e25deefa`(p)
) and
`DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::viableParamArgEx/3`(call,
p, dummyField) and
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlow/2#65dc322b`(dummyField,
cc)
) and
(
cc = false
or
(
cc = true and
not(
`project#DataFlowImplCommon::Cached::CachedCallContextSensitivity::reducedViableImplInCallContext/3#f0f91b5d`(call)
)
)
)
)
or
(
exists(
/* DataFlowDispatch::DataFlowCallable */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCallable target
|
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ dontcare `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` _
|
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowInReducedViableImplInSomeCallContext/4#ccb02962`(call,
_, p, target)
) and
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::viableImplInSomeFwdFlowCallContextExt/1#23a13296`(call,
target)
) and
cc = true
)
or
(
exists(
/* DataFlowDispatch::DataFlowCallable */ DataFlowDispatch#2409fd0c::Cached::TDataFlowCallable target
|
exists(
/* DataFlowImplCommon::MakeImplCommon<Location::Location, DataFlowImplSpecific::CsharpDataFlow>::NodeEx */ dontcare `DataFlowImplCommon#f7de413b::MakeImplCommon<Location#a178a2aa::Location,DataFlowImplSpecific#b13302e2::CsharpDataFlow>::Cached::TNodeEx` _
|
delta previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowInReducedViableImplInSomeCallContext/4#ccb02962`(call,
_, p, target)
) and
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::viableImplInSomeFwdFlowCallContextExt/1#23a13296`(call,
target)
) and
cc = true
)
)
) and
not(
previous rec `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::CsharpDataFlow>::Impl<XSSQuery::XssTracking::C>::Stage1::fwdFlowIsEntered/3#137febc2`(call,
disallowReturnKind, cc)
)
}
There was a problem hiding this comment.
That's a pretty clear optimiser bug, I think: putting prev before delta like that.
There was a problem hiding this comment.
I actually hadn't spotted that, only that getDisallowedReturnKind was pushed to the top, good spot.
There was a problem hiding this comment.
That's the actual reason why this join-order turns out poorly. The bindingset that we add in this PR isn't actually restricting the join-orderer in any way for the reason I stated, and that's what confused me - it's only giving a slight nudge that happens to cause the opimiser to dodge its bug.
aschackmull
approved these changes
Jan 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#18333 follow-up.
While I did run DCA on the original PR, I did not rerun it on the last commit, because "what can possibly go wrong, right?". Turns out things did go wrong...