Skip to content

Commit

Permalink
Relaced UnknownDetails with MiscDetails
Browse files Browse the repository at this point in the history
  • Loading branch information
NimishMishra committed Nov 13, 2021
1 parent 3aa4ab2 commit 780cea1
Show file tree
Hide file tree
Showing 149 changed files with 8,460 additions and 5,297 deletions.
2 changes: 1 addition & 1 deletion clang-tools-extra/docs/doxygen.cfg.in
Original file line number Diff line number Diff line change
Expand Up @@ -1230,7 +1230,7 @@ CHM_FILE =
HHC_LOCATION =

# The GENERATE_CHI flag controls if a separate .chi index file is generated (
# YES) or that it should be included in the master .chm file ( NO).
# YES) or that it should be included in the main .chm file ( NO).
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.

Expand Down
2 changes: 1 addition & 1 deletion clang/docs/doxygen.cfg.in
Original file line number Diff line number Diff line change
Expand Up @@ -1219,7 +1219,7 @@ CHM_FILE =
HHC_LOCATION =

# The GENERATE_CHI flag controls if a separate .chi index file is generated (
# YES) or that it should be included in the master .chm file ( NO).
# YES) or that it should be included in the main .chm file ( NO).
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.

Expand Down
111 changes: 97 additions & 14 deletions flang/docs/FlangDriver.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,107 @@
:local:
```


> **_NOTE:_** This document assumes that Flang's drivers can already generate code and
> produce executables. However, this is still work-in-progress. By making this
> assumption, we are able to prepare this document ahead-of-time and to provide
> an overview of the design that we are working towards.
There are two main drivers in Flang:
* the compiler driver, `flang-new`
* the frontend driver, `flang-new -fc1`

The compiler driver will allow you to control all compilation phases (i.e.
preprocessing, frontend code-generation, middlend/backend code-optimisation and
lowering, linking). For frontend specific tasks, the compiler driver creates a
Fortran compilation job and delegates it to `flang-new -fc1`, the frontend driver.
> **_NOTE:_** The diagrams in this document refer to `flang` as opposed to
> `flang-new`. This is because the diagrams reflect the final design that we
> are still working towards. See the note on [the flang script](https://github.com/llvm/llvm-project/blob/main/flang/docs/FlangDriver.md#the-flang-script)
> below for more context.
The **compiler driver** will allow you to control all compilation phases (e.g.
preprocessing, semantic checks, code-generation, code-optimisation, lowering
and linking). For frontend specific tasks, the compiler driver creates a
Fortran compilation job and delegates it to `flang-new -fc1`, the frontend
driver. For linking, it creates a linker job and calls an external linker (e.g.
LLVM's [`lld`](https://lld.llvm.org/)). It can also call other tools such as
external assemblers (e.g. [`as`](https://www.gnu.org/software/binutils/)). In
Clang, the compiler driver can also link the generated binaries with LLVM's
static analysis/sanitizer libraries (e.g.
[MemorySanitizer](https://clang.llvm.org/docs/MemorySanitizer.html)). This is
not yet available in Flang, but will be relatively easy to support once such
libraries become available. Flang's compiler driver is intended for Flang's
end-users - its interface needs to remain stable. Otherwise, Flang's users will
have to adjust their build scripts every time a compiler flag is changed.

| ![Compiler Driver](compiler_driver.png) |
|:--:|
| *Flang’s compiler driver and the **tools** that it runs* |

The **frontend driver** glues together and drives all of the Flang's frontend
libraries. As such, it provides an easy-to-use and intuitive interface to the
frontend. It uses MLIR and LLVM for code-generation and can be viewed as a
driver for Flang, LLVM and MLIR libraries. Contrary to the compiler driver, it
is not capable of calling any external tools (including linkers). It is aware
of all the frontend internals that are "hidden" from the compiler driver. It
accepts many frontend-specific options not available in `flang-new` and as such
it provides a finer control over the frontend. Note that this tool is mostly
intended for Flang developers. In particular, there are no guarantees about the
stability of its interface and compiler developers can use it to experiment
with new flags.

| ![Frontend Driver](frontend_driver.png) |
|:-:|
| *Flang's frontend driver and the **libraries** that it drives* |

Note that similarly to `-Xclang` in `clang`, you can use `-Xflang` to forward a
frontend specific flag from the _compiler_ directly to the _frontend_ driver,
e.g.:

```lang=bash
flang-new -Xflang -fdebug-dump-parse-tree input.f95
```

The frontend driver glues all of the frontend libraries together and provides
an easy-to-use and intuitive interface to the frontend. It accepts many
frontend-specific options not available in `flang-new` and as such it provides a
finer control over the frontend. Similarly to `-Xclang` in `clang`, you can use
`-Xflang` to forward the frontend specific flags from the compiler directly to
the frontend driver.
In the invocation above, `-fdebug-dump-parse-tree` is forwarded to `flang-new
-fc1`. Without the forwarding flag, `-Xflang`, you would see the following
warning:

## Compiler Driver
```lang=bash
flang-new: warning: argument unused during compilation:
```

As `-fdebug-dump-parse-tree` is only supported by `flang-new -fc1`, `flang-new`
will ignore it when used without `Xflang`.

## Why Do We Need Two Drivers?
As hinted above, `flang-new` and `flang-new -fc1` are two separate tools. The
fact that these tools are accessed through one binary, `flang-new`, is just an
implementation detail. Each tool has a separate list of options, albeit defined
in the same file: `clang/include/clang/Driver/Options.td`.

The separation helps us split various tasks and allows us to implement more
specialised tools. In particular, `flang-new` is not aware of various
compilation phases within the frontend (e.g. scanning, parsing or semantic
checks). It does not have to be. Conversely, the frontend driver, `flang-new
-fc1`, needs not to be concerned with linkers or other external tools like
assemblers. Nor does it need to know where to look for various systems
libraries, which is usually OS and platform specific.

One helpful way of differentiating these tools is to keep in mind that:

* the compiler driver is an end-user tool
* frontend driver is a compiler developer tool with many additional options,

Also, Since the compiler driver can call external tools, e.g. linkers, it can
be used to generate **executables**. The frontend driver cannot call external
tools and hence can only generate **object files**. A similar model is
implemented in Clang (`clang` vs `clang -cc1` vs `clang -cc1as`), which is
based on the [architecture of
GCC](https://en.wikibooks.org/wiki/GNU_C_Compiler_Internals/GNU_C_Compiler_Architecture).
In fact, Flang needs to adhere to this model in order to be able to re-use
Clang's driver library. If you are more familiar with the [architecture of
GFortran](https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gfortran/About-GNU-Fortran.html)
than Clang, then `flang-new` corresponds to `gfortran` and `flang-new -fc1` to
`f951`.

## Compiler Driver
The main entry point for Flang's compiler driver is implemented in
`flang/tools/flang-driver/driver.cpp`. Flang's compiler driver is implemented
in terms of Clang's driver library, `clangDriver`. This approach allows us to:
Expand Down Expand Up @@ -92,9 +175,9 @@ You can read more on the design of `clangDriver` in Clang's [Driver Design &
Internals](https://clang.llvm.org/docs/DriverInternals.html).

## Frontend Driver
Flang's frontend driver is the main interface between end-users and the Flang
frontend. The high-level design is similar to Clang's frontend driver, `clang
-cc1` and consists of the following classes:
Flang's frontend driver is the main interface between compiler developers and
the Flang frontend. The high-level design is similar to Clang's frontend
driver, `clang -cc1` and consists of the following classes:
* `CompilerInstance`, which is a helper class that encapsulates and manages
various objects that are always required by the frontend (e.g. `AllSources`,
`AllCookedSources, `Parsing`, `CompilerInvocation`, etc.). In most cases
Expand Down
Binary file added flang/docs/compiler_driver.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion flang/docs/doxygen.cfg.in
Original file line number Diff line number Diff line change
Expand Up @@ -1222,7 +1222,7 @@ CHM_FILE =
HHC_LOCATION =

# The GENERATE_CHI flag controls if a separate .chi index file is generated (
# YES) or that it should be included in the master .chm file ( NO).
# YES) or that it should be included in the main .chm file ( NO).
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.

Expand Down
Binary file added flang/docs/frontend_driver.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,6 @@ bool OpenMPCounterVisitor::Pre(const OpenMPConstruct &c) {
ompWrapperStack.push_back(ow);
return true;
}
bool OpenMPCounterVisitor::Pre(const OmpEndLoopDirective &c) { return true; }
bool OpenMPCounterVisitor::Pre(const DoConstruct &) {
loopLogRecordStack.push_back(curLoopLogRecord);
return true;
}

void OpenMPCounterVisitor::Post(const OpenMPDeclarativeConstruct &) {
PostConstructsCommon();
Expand All @@ -178,27 +173,11 @@ void OpenMPCounterVisitor::PostConstructsCommon() {
clauseStrings[curConstruct]};
constructClauses.push_back(r);

// Keep track of loop log records if it can potentially have the
// nowait clause added on later.
if (const auto *oc = std::get_if<const OpenMPConstruct *>(curConstruct)) {
if (const auto *olc = std::get_if<OpenMPLoopConstruct>(&(*oc)->u)) {
const auto &beginLoopDir{
std::get<Fortran::parser::OmpBeginLoopDirective>(olc->t)};
const auto &beginDir{
std::get<Fortran::parser::OmpLoopDirective>(beginLoopDir.t)};
if (beginDir.v == llvm::omp::Directive::OMPD_do ||
beginDir.v == llvm::omp::Directive::OMPD_do_simd) {
curLoopLogRecord = &constructClauses.back();
}
}
}

auto it = clauseStrings.find(curConstruct);
clauseStrings.erase(it);
ompWrapperStack.pop_back();
delete curConstruct;
}
void OpenMPCounterVisitor::Post(const OmpEndLoopDirective &c) {}

void OpenMPCounterVisitor::Post(const OmpProcBindClause::Type &c) {
clauseDetails += "type=" + OmpProcBindClause::EnumToString(c) + ";";
Expand Down Expand Up @@ -242,26 +221,9 @@ void OpenMPCounterVisitor::Post(const OmpClause &c) {
clauseDetails.clear();
}
void OpenMPCounterVisitor::PostClauseCommon(const ClauseInfo &ci) {
// The end loop construct (!$omp end do) can contain a nowait clause.
// The flang parser does not parse the end loop construct as part of
// the OpenMP construct for the loop construct. So the end loop is left
// hanging as a separate executable construct. If a nowait clause is seen in
// an end loop construct we have to find the associated loop construct and
// add nowait to its list of clauses. Note: This is not a bug in flang, the
// parse tree is corrected during semantic analysis.
if (ci.clause == "nowait") {
assert(curLoopLogRecord &&
"loop Construct should be visited before a nowait clause");
curLoopLogRecord->clauses.push_back(ci);
} else {
assert(!ompWrapperStack.empty() &&
"Construct should be visited before clause");
clauseStrings[ompWrapperStack.back()].push_back(ci);
}
}
void OpenMPCounterVisitor::Post(const DoConstruct &) {
curLoopLogRecord = loopLogRecordStack.back();
loopLogRecordStack.pop_back();
assert(
!ompWrapperStack.empty() && "Construct should be visited before clause");
clauseStrings[ompWrapperStack.back()].push_back(ci);
}
} // namespace parser
} // namespace Fortran
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringRef.h"

#include <deque>
#include <string>

namespace Fortran {
Expand Down Expand Up @@ -62,13 +61,10 @@ struct OpenMPCounterVisitor {
template <typename A> void Post(const A &) {}
bool Pre(const OpenMPDeclarativeConstruct &c);
bool Pre(const OpenMPConstruct &c);
bool Pre(const OmpEndLoopDirective &c);
bool Pre(const DoConstruct &);

void Post(const OpenMPDeclarativeConstruct &);
void Post(const OpenMPConstruct &);
void PostConstructsCommon();
void Post(const OmpEndLoopDirective &c);

void Post(const OmpProcBindClause::Type &c);
void Post(const OmpDefaultClause::Type &c);
Expand All @@ -83,20 +79,9 @@ struct OpenMPCounterVisitor {
void Post(const OmpCancelType::Type &c);
void Post(const OmpClause &c);
void PostClauseCommon(const ClauseInfo &ci);
void Post(const DoConstruct &);

std::string clauseDetails{""};

// curLoopLogRecord and loopLogRecordStack store
// pointers to this datastructure's entries. Hence a
// vector cannot be used since pointers are invalidated
// on resize. Next best option seems to be deque. Also a
// list cannot be used since YAML gen requires a
// datastructure which can be accessed through indices.
std::deque<LogRecord> constructClauses;

LogRecord *curLoopLogRecord{nullptr};
llvm::SmallVector<LogRecord *> loopLogRecordStack;
llvm::SmallVector<LogRecord> constructClauses;
llvm::SmallVector<OmpWrapperType *> ompWrapperStack;
llvm::DenseMap<OmpWrapperType *, llvm::SmallVector<ClauseInfo>> clauseStrings;
Parsing *parsing{nullptr};
Expand Down
4 changes: 0 additions & 4 deletions flang/examples/flang-omp-report-plugin/flang-omp-report.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,6 @@ namespace llvm {
namespace yaml {
using llvm::yaml::IO;
using llvm::yaml::MappingTraits;
template <typename T>
struct SequenceTraits<std::deque<T>,
std::enable_if_t<CheckIsBool<SequenceElementTraits<T>::flow>::value>>
: SequenceTraitsImpl<std::deque<T>, SequenceElementTraits<T>::flow> {};
template <> struct MappingTraits<ClauseInfo> {
static void mapping(IO &io, ClauseInfo &info) {
io.mapRequired("clause", info.clause);
Expand Down
2 changes: 2 additions & 0 deletions flang/examples/flang-omp-report-plugin/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ruamel.yaml==0.17.16
ruamel.yaml.clib==0.2.6
Loading

0 comments on commit 780cea1

Please sign in to comment.