Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Forward" cross-crate PDG construction #153

Closed
wants to merge 98 commits into from
Closed
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
9725139
Untangling the child constructor from recursive loading
JustusAdam May 10, 2024
a187aae
Independence from calling context
JustusAdam May 11, 2024
ff439f6
Use the manufactured dyns again
JustusAdam May 11, 2024
cad9596
Fixing basic test cases
JustusAdam May 13, 2024
eb466f8
Fix async call strings
JustusAdam May 13, 2024
26ed7c7
Decoder and Encoder instances
JustusAdam May 14, 2024
89bacce
Output side
JustusAdam May 15, 2024
d10230b
DefId is actually encodable
JustusAdam May 15, 2024
dee9ee6
Decoder
JustusAdam May 15, 2024
aba0f6d
Wiring for laoder
JustusAdam May 15, 2024
e69046c
Reuse memo cosntructor
JustusAdam May 15, 2024
c540163
Pass the judge to PDG construction
JustusAdam May 15, 2024
431fd2d
Loading
JustusAdam May 15, 2024
4c249a2
Unused imports
JustusAdam May 15, 2024
f8beb0e
Create cross crate test case
JustusAdam May 15, 2024
8dba09a
Local crate fixes
JustusAdam May 15, 2024
c3337de
Make the instruction info creation more robust
JustusAdam May 15, 2024
c98687e
Hack filenames
JustusAdam May 15, 2024
1195be4
Misc fixes
JustusAdam May 15, 2024
2ea3965
Basic test case works 🎉
JustusAdam May 15, 2024
fbf01f5
Fix generics tracking
JustusAdam May 16, 2024
bc90d79
Found a bug I wasn't looking for
JustusAdam May 16, 2024
1671c93
WIP fixing generics recording
JustusAdam May 16, 2024
721d5fe
Fix return place detection
JustusAdam May 16, 2024
ce5d30b
Schedule analysis targets for construction, use manufactured type arg…
JustusAdam May 16, 2024
4144f71
Cross crate marker test case
JustusAdam May 16, 2024
143ff0f
WIP default method test case
JustusAdam May 16, 2024
f6889fb
Trialing elimination of FnResolutions
JustusAdam May 16, 2024
5fbd6d6
Simplifying in search of a bug
JustusAdam May 17, 2024
704e713
Deal with auto traits and `Sized`
JustusAdam May 22, 2024
9ba8a7a
Refactoring
JustusAdam May 23, 2024
9da9167
Fix entrypoint generics by resolving functions late
JustusAdam May 23, 2024
d767766
Fixing warnings
JustusAdam May 23, 2024
859c499
Export mIssing definitions
JustusAdam May 24, 2024
1d0d6a5
Factor out metadata
JustusAdam May 24, 2024
3bbd843
WIP debugging virtual instance resolution
JustusAdam May 24, 2024
7c5c8b4
Fix freedit test case
JustusAdam May 25, 2024
30e9c0b
Make sure not to check asyncness for closures
JustusAdam May 25, 2024
b486f2c
All test cases fixed or reported
JustusAdam May 25, 2024
0d94bd9
Documentation and renaming
JustusAdam May 25, 2024
71e876f
Remove SubgraphDescriptor type
JustusAdam May 26, 2024
88a9d00
Encapsulation for PartialGraph
JustusAdam May 26, 2024
b99aaf5
Unexport
JustusAdam May 26, 2024
f434160
Misc
JustusAdam May 26, 2024
7a7df79
Additional test case for cross crate
JustusAdam May 28, 2024
cb68508
Test case for generic cross crate
JustusAdam May 28, 2024
89f0083
Simple generics suppport
JustusAdam May 28, 2024
7085558
Enable cross crate tests in CI
JustusAdam May 28, 2024
c5a03fc
Clippy fixes
JustusAdam May 28, 2024
6c1c167
Fix warnings
JustusAdam May 28, 2024
5d46a1d
Clippy wants me to rename this
JustusAdam May 28, 2024
88e6db0
Extra docs and dead code removal
JustusAdam May 28, 2024
2c31d83
Splitting `construct`
JustusAdam May 31, 2024
d88d98e
Unused trait import
JustusAdam May 31, 2024
0f56d7f
Fmt
JustusAdam May 31, 2024
5237007
Delaying emitting errors
JustusAdam Jun 5, 2024
514d8f7
Accumulate and propagate errors
JustusAdam Jun 5, 2024
95e4cea
Make instance resolution that fails on dyn a warning
JustusAdam Jun 5, 2024
a85ffd1
Serializable async errors
JustusAdam Jun 6, 2024
22cc707
Remove unused code
JustusAdam Jun 7, 2024
9b8a3ff
Clippy
JustusAdam Jun 8, 2024
c3624cf
Merge branch 'main' into cross-crate
JustusAdam Jun 10, 2024
38c4888
Clippy
JustusAdam Jun 10, 2024
a0d1027
Fmt
JustusAdam Jun 10, 2024
be95519
Reenable MIR dumping
JustusAdam Jun 10, 2024
7eac289
Make calling convention matching more explicit
JustusAdam Jun 10, 2024
7b4468a
New test case for calling async_trait
JustusAdam Jun 10, 2024
ad5ed28
Test old version and latest
JustusAdam Jun 10, 2024
6c6cc75
Rename error types
JustusAdam Jun 11, 2024
3c6f7ed
Handle top-level async workarounds on the paralegal side
JustusAdam Jun 11, 2024
872d307
Defer normalization errors
JustusAdam Jun 11, 2024
7ca7d6d
Alternate test harness + clone test case
JustusAdam Jun 13, 2024
a2fd805
Perhaps the dumbest error ever
JustusAdam Jun 13, 2024
2080a43
Equiv check doesn't actually work
JustusAdam Jun 14, 2024
0a92416
Enable profiling env var
JustusAdam Jun 14, 2024
ce66f3a
Use commit ref, update tests
JustusAdam Jun 18, 2024
9129b56
Single source for rustc flags
JustusAdam Jun 18, 2024
2f668d7
Add stats back to make griswold compile
JustusAdam Jun 19, 2024
1cbb490
Merge branch 'main' into cross-crate
JustusAdam Jun 27, 2024
59fca35
Whoops
JustusAdam Jun 27, 2024
2d51cc6
Fix parentable_{srcs|dsts}
JustusAdam Jun 28, 2024
a0131da
Allow dyn Any as wildcard
JustusAdam Jun 28, 2024
5a9912b
Formatting
JustusAdam Jun 28, 2024
fa05757
Allow custom output for context
JustusAdam Jun 30, 2024
5b3999b
Adjust output mechanism
JustusAdam Jun 30, 2024
6433d43
Clippy fixes
JustusAdam Jun 30, 2024
1a58054
Test case for markers on return nodes when control flow is present
JustusAdam Jul 1, 2024
a9c4ed2
Enable all policy framework tests
JustusAdam Jul 1, 2024
e21d3b9
Use a clean version
JustusAdam Jul 2, 2024
1d0a0df
Merge branch 'cross-crate' of github.com:brownsys/paralegal into cros…
JustusAdam Jul 2, 2024
4385ce9
Mimic rustc version
JustusAdam Jul 11, 2024
7df232e
Format and clippy
JustusAdam Jul 11, 2024
8791426
Misc small changes
JustusAdam Jul 11, 2024
0d404e2
Test cases for async + generics interactions
JustusAdam Jul 17, 2024
88583b5
Rename test case
JustusAdam Jul 17, 2024
f8565cd
Move it to other async test cases
JustusAdam Jul 17, 2024
5ce9ea7
Another test case for async handling issues
JustusAdam Jul 17, 2024
504cb58
Indirect async test case
JustusAdam Jul 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Makefile.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ args = [
"--test",
"async_tests",
"--no-fail-fast",
"--test",
"cross-crate",
]

[tasks.policy-framework-tests]
Expand Down
5 changes: 5 additions & 0 deletions crates/flowistry_pdg/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ pub(crate) mod rustc {
pub use middle::mir;
}

#[cfg(feature = "rustc")]
extern crate rustc_macros;
#[cfg(feature = "rustc")]
extern crate rustc_serialize;

mod pdg;
#[cfg(feature = "rustc")]
mod rustc_impls;
Expand Down
61 changes: 58 additions & 3 deletions crates/flowistry_pdg/src/pdg.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ use serde::{Deserialize, Serialize};
use crate::rustc_portable::*;
#[cfg(feature = "rustc")]
use crate::rustc_proxies;
#[cfg(feature = "rustc")]
use rustc_macros::{Decodable, Encodable};
#[cfg(feature = "rustc")]
use rustc_serialize::{Decodable, Decoder, Encodable, Encoder};

/// Extends a MIR body's `Location` with `Start` (before the first instruction) and `End` (after all returns).
#[derive(PartialEq, Eq, Hash, Clone, Copy, Debug, Serialize, Deserialize)]
Expand All @@ -26,6 +30,35 @@ pub enum RichLocation {
End,
}

#[cfg(feature = "rustc")]
impl<E: Encoder> Encodable<E> for RichLocation {
fn encode(&self, s: &mut E) {
match self {
Self::Location(loc) => s.emit_enum_variant(0, |s| {
s.emit_u32(loc.block.as_u32());
s.emit_usize(loc.statement_index);
}),
Self::Start => s.emit_enum_variant(1, |_| ()),
Self::End => s.emit_enum_variant(2, |_| ()),
}
}
}

#[cfg(feature = "rustc")]
impl<D: Decoder> Decodable<D> for RichLocation {
fn decode(d: &mut D) -> Self {
match d.read_usize() {
0 => Self::Location(Location {
block: d.read_u32().into(),
statement_index: d.read_usize(),
}),
1 => Self::Start,
2 => Self::End,
v => panic!("Unknown variant index: {v}"),
}
}
}

impl RichLocation {
/// Returns true if this is a `Start` location.
pub fn is_start(self) -> bool {
Expand Down Expand Up @@ -74,17 +107,18 @@ impl From<Location> for RichLocation {

/// A [`RichLocation`] within a specific point in a codebase.
#[derive(PartialEq, Eq, Hash, Clone, Copy, Debug, Serialize, Deserialize)]
#[cfg_attr(feature = "rustc", derive(Encodable, Decodable))]
pub struct GlobalLocation {
// TODO Change to `DefId`
/// The function containing the location.
#[cfg_attr(feature = "rustc", serde(with = "rustc_proxies::LocalDefId"))]
pub function: LocalDefId,
#[cfg_attr(feature = "rustc", serde(with = "rustc_proxies::DefId"))]
pub function: DefId,

/// The location of an instruction in the function, or the function's start.
pub location: RichLocation,
}

#[cfg(not(feature = "rustc"))]

impl fmt::Display for GlobalLocation {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{:?}::{}", self.function, self.location)
Expand All @@ -103,6 +137,21 @@ impl fmt::Display for GlobalLocation {
#[derive(PartialEq, Eq, Hash, Copy, Clone, Debug, Serialize, Deserialize)]
pub struct CallString(Intern<CallStringInner>);

#[cfg(feature = "rustc")]
impl<S: Encoder> Encodable<S> for CallString {
fn encode(&self, s: &mut S) {
let inner: &CallStringInner = &self.0;
inner.encode(s);
}
}

#[cfg(feature = "rustc")]
impl<D: Decoder> Decodable<D> for CallString {
fn decode(d: &mut D) -> Self {
Self(Intern::new(CallStringInner::decode(d)))
}
}

type CallStringInner = Box<[GlobalLocation]>;

impl CallString {
Expand Down Expand Up @@ -153,6 +202,10 @@ impl CallString {
CallString::new(string)
}

pub fn push_front(self, loc: GlobalLocation) -> Self {
CallString::new([loc].into_iter().chain(self.0.iter().copied()).collect())
}

pub fn is_at_root(self) -> bool {
self.0.len() == 1
}
Expand Down Expand Up @@ -199,13 +252,15 @@ impl fmt::Display for CallString {
#[derive(
PartialEq, Eq, PartialOrd, Ord, Hash, Clone, Copy, Debug, Serialize, Deserialize, strum::EnumIs,
)]
#[cfg_attr(feature = "rustc", derive(Decodable, Encodable))]
pub enum SourceUse {
Operand,
Argument(u8),
}

/// Additional information about this mutation.
#[derive(PartialEq, Eq, Hash, Clone, Copy, Debug, Serialize, Deserialize, strum::EnumIs)]
#[cfg_attr(feature = "rustc", derive(Decodable, Encodable))]
pub enum TargetUse {
/// A function returned, assigning to it's return destination
Return,
Expand Down
2 changes: 1 addition & 1 deletion crates/flowistry_pdg/src/rustc_impls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ impl From<DefIndex> for def_id::DefIndex {
impl fmt::Display for GlobalLocation {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
tls::with_opt(|opt_tcx| match opt_tcx {
Some(tcx) => match tcx.opt_item_name(self.function.to_def_id()) {
Some(tcx) => match tcx.opt_item_name(self.function) {
Some(name) => name.fmt(f),
None => write!(f, "<closure>"),
},
Expand Down
2 changes: 2 additions & 0 deletions crates/flowistry_pdg_construction/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ flowistry_pdg = { version = "0.1.0", path = "../flowistry_pdg", features = [
] }
#flowistry = { path = "../../../flowistry/crates/flowistry", default-features = false }
flowistry = { workspace = true }
serde = { workspace = true, features = ["derive"] }
strum = { workspace = true }

[dev-dependencies]
rustc_utils = { workspace = true, features = ["indexical", "test"] }
Expand Down
98 changes: 63 additions & 35 deletions crates/flowistry_pdg_construction/src/async_support.rs
Original file line number Diff line number Diff line change
@@ -1,21 +1,40 @@
use std::rc::Rc;

use either::Either;
use flowistry_pdg::{CallString, GlobalLocation};
use itertools::Itertools;
use rustc_abi::{FieldIdx, VariantIdx};
use rustc_hir::def_id::{DefId, LocalDefId};
use rustc_macros::{Decodable, Encodable};
use rustc_middle::{
mir::{
AggregateKind, BasicBlock, Body, Location, Operand, Place, Rvalue, Statement,
StatementKind, Terminator, TerminatorKind,
},
ty::{GenericArgsRef, TyCtxt},
ty::{GenericArgsRef, Instance, TyCtxt},
};

use crate::construct::{CallKind, PartialGraph};
use crate::{
construct::{push_call_string_root, CallKind, LocalAnalysis},
utils, PartialGraph,
};

/// Describe in which way a function is `async`.
///
/// Critically distinguishes between a normal `async fn` and an
/// `#[async_trait]`.
#[derive(Debug, Clone, Copy, Decodable, Encodable)]
pub enum Asyncness {
No,
AsyncFn,
AsyncTrait,
}

use super::construct::GraphConstructor;
use super::utils::{self, FnResolution};
impl Asyncness {
pub fn is_async(self) -> bool {
!matches!(self, Asyncness::No)
}
}

/// Stores ids that are needed to construct projections around async functions.
pub(crate) struct AsyncInfo {
Expand Down Expand Up @@ -143,20 +162,28 @@ fn get_async_generator<'tcx>(body: &Body<'tcx>) -> (LocalDefId, GenericArgsRef<'
(def_id.expect_local(), generic_args, location)
}

/// Try to interpret this function as an async function.
///
/// If this is an async function it returns the [`Instance`] of the generator,
/// the location where the generator is bound and the type of [`Asyncness`]
/// which in this case is guaranteed to satisfy [`Asyncness::is_async`].
pub fn determine_async<'tcx>(
tcx: TyCtxt<'tcx>,
def_id: LocalDefId,
body: &Body<'tcx>,
) -> Option<(FnResolution<'tcx>, Location)> {
let (generator_def_id, args, loc) = if tcx.asyncness(def_id).is_async() {
get_async_generator(body)
) -> Option<(Instance<'tcx>, Location, Asyncness)> {
let ((generator_def_id, args, loc), asyncness) = if tcx.asyncness(def_id).is_async() {
(get_async_generator(body), Asyncness::AsyncFn)
} else {
try_as_async_trait_function(tcx, def_id.to_def_id(), body)?
(
try_as_async_trait_function(tcx, def_id.to_def_id(), body)?,
Asyncness::AsyncTrait,
)
};
let param_env = tcx.param_env_reveal_all_normalized(def_id);
let generator_fn =
utils::try_resolve_function(tcx, generator_def_id.to_def_id(), param_env, args);
Some((generator_fn, loc))
utils::try_resolve_function(tcx, generator_def_id.to_def_id(), param_env, args)?;
Some((generator_fn, loc, asyncness))
}

#[derive(Debug, Clone, PartialEq, Eq)]
Expand All @@ -166,29 +193,29 @@ pub enum AsyncDeterminationResult<T> {
NotAsync,
}

impl<'tcx> GraphConstructor<'tcx> {
impl<'tcx, 'a> LocalAnalysis<'tcx, 'a> {
pub(crate) fn try_handle_as_async(&self) -> Option<PartialGraph<'tcx>> {
let (generator_fn, location) = determine_async(self.tcx, self.def_id, &self.body)?;
let (generator_fn, location, asyncness) =
determine_async(self.tcx(), self.def_id, &self.body)?;

let calling_context = self.calling_context_for(generator_fn.def_id(), location);
let params = self.pdg_params_for_call(generator_fn);
Some(
GraphConstructor::new(
params,
Some(calling_context),
self.async_info.clone(),
&self.pdg_cache,
)
.construct_partial(),
)
let g = self.memo.construct_for(generator_fn)?;
let gloc = GlobalLocation {
function: self.def_id.to_def_id(),
location: flowistry_pdg::RichLocation::Location(location),
};
let mut new_g = push_call_string_root(g, gloc);
//let g_generics = std::mem::replace(&mut new_g.graph.generics, self.generic_args());
new_g.asyncness = asyncness;
new_g.monos.insert(CallString::single(gloc), new_g.generics);
Some(new_g)
}

pub(crate) fn try_poll_call_kind<'a>(
&'a self,
pub(crate) fn try_poll_call_kind<'b>(
&'b self,
def_id: DefId,
original_args: &'a [Operand<'tcx>],
original_args: &'b [Operand<'tcx>],
) -> AsyncDeterminationResult<CallKind<'tcx>> {
let lang_items = self.tcx.lang_items();
let lang_items = self.tcx().lang_items();
if lang_items.future_poll_fn() == Some(def_id) {
match self.find_async_args(original_args) {
Ok((fun, loc, args)) => {
Expand All @@ -202,10 +229,10 @@ impl<'tcx> GraphConstructor<'tcx> {
}
/// Given the arguments to a `Future::poll` call, walk back through the
/// body to find the original future being polled, and get the arguments to the future.
fn find_async_args<'a>(
&'a self,
args: &'a [Operand<'tcx>],
) -> Result<(FnResolution<'tcx>, Location, Place<'tcx>), String> {
fn find_async_args<'b>(
&'b self,
args: &'b [Operand<'tcx>],
) -> Result<(Instance<'tcx>, Location, Place<'tcx>), String> {
macro_rules! let_assert {
($p:pat = $e:expr, $($arg:tt)*) => {
let $p = $e else {
Expand Down Expand Up @@ -241,7 +268,7 @@ impl<'tcx> GraphConstructor<'tcx> {
debug_assert!(new_pin_args.len() == 1);

let future_aliases = self
.aliases(self.tcx.mk_place_deref(new_pin_args[0].place().unwrap()))
.aliases(self.tcx().mk_place_deref(new_pin_args[0].place().unwrap()))
.collect_vec();
debug_assert!(future_aliases.len() == 1);
let future = *future_aliases.first().unwrap();
Expand Down Expand Up @@ -308,11 +335,12 @@ impl<'tcx> GraphConstructor<'tcx> {
let (op, generics, calling_convention, async_fn_call_loc) = chase_target.unwrap();

let resolution = utils::try_resolve_function(
self.tcx,
self.tcx(),
op,
self.tcx.param_env_reveal_all_normalized(self.def_id),
self.tcx().param_env_reveal_all_normalized(self.def_id),
generics,
);
)
.ok_or("Resolving function failed")?;

Ok((resolution, async_fn_call_loc, calling_convention))
}
Expand Down
12 changes: 5 additions & 7 deletions crates/flowistry_pdg_construction/src/callback.rs
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
//! CAllbacks to influence graph construction and their supporting types.

use flowistry_pdg::{rustc_portable::Location, CallString};

use crate::FnResolution;
use rustc_middle::ty::Instance;

pub trait CallChangeCallback<'tcx> {
fn on_inline(&self, info: CallInfo<'tcx>) -> CallChanges;

fn on_inline_miss(
&self,
_resolution: FnResolution<'tcx>,
_resolution: Instance<'tcx>,
_loc: Location,
_under_analysis: FnResolution<'tcx>,
_call_string: Option<CallString>,
_under_analysis: Instance<'tcx>,
_reason: InlineMissReason,
) {
}
Expand Down Expand Up @@ -50,11 +48,11 @@ impl Default for CallChanges {
/// Information about the function being called.
pub struct CallInfo<'tcx> {
/// The potentially-monomorphized resolution of the callee.
pub callee: FnResolution<'tcx>,
pub callee: Instance<'tcx>,

/// If the callee is an async closure created by an `async fn`, this is the
/// `async fn` item.
pub async_parent: Option<FnResolution<'tcx>>,
pub async_parent: Option<Instance<'tcx>>,

/// The call-stack up to the current call site.
pub call_string: CallString,
Expand Down
Loading
Loading