NativeAOT build support and doc improvements (#68038)

- Add IlcInstructionSet property that can be used to set the target instruction set from the command line.
- Moved link to the documentation of reflection-free mode to be in less prime location, so that people trying NativeAOT for the first time are not steared towards it.
- Copied help text improvements from crossgen2

Author: jkotas

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

@@ -231,6 +231,7 @@ The .NET Foundation licenses this file to you under the MIT license.
       <IlcArg Condition="$(IlcFoldIdenticalMethodBodies) == 'true'" Include="--methodbodyfolding" />
       <IlcArg Condition="$(Optimize) == 'true' and $(IlcOptimizationPreference) == 'Size'" Include="--Os" />
       <IlcArg Condition="$(Optimize) == 'true' and $(IlcOptimizationPreference) == 'Speed'" Include="--Ot" />
+      <IlcArg Condition="$(IlcInstructionSet) != ''" Include="--instructionset:$(IlcInstructionSet)" />
       <IlcArg Condition="$(IlcDisableReflection) == 'true'" Include="--disablereflection" />
       <IlcArg Condition="$(IlcDisableReflection) == 'true'" Include="--feature:System.Collections.Generic.DefaultComparers=false" />
       <IlcArg Condition="$(IlcSingleThreaded) == 'true'" Include="--singlethreaded" />

src/coreclr/nativeaot/docs/README.md

@@ -4,7 +4,6 @@
 - [Compiling applications](compiling.md)
 - [Debugging applications](debugging.md)
 - [Optimizing applications](optimizing.md)
-- [Reflection Free Mode](reflection-free-mode.md)
 - [Reflection In AOT](reflection-in-aot-mode.md)
 - [Managed/Native Interop in AOT](interop.md)
 - [Troubleshooting](troubleshooting.md)

src/coreclr/nativeaot/docs/optimizing.md

@@ -36,15 +36,14 @@ To aid in troubleshooting some of the most common problems related to trimming a
 
 * `<IlcGenerateStackTraceData>false</IlcGenerateStackTraceData>`: this disables generation of stack trace metadata that provides textual names in stack traces. This is for example the text string one gets by calling `Exception.ToString()` on a caught exception. With this option disabled, stack traces will still be generated, but will be based on reflection metadata alone (they might be less complete).
 * `<IlcTrimMetadata>true</IlcTrimMetadata>`: allows the compiler to remove reflection metadata from things that were not visible targets of reflection. By default, the compiler keeps metadata for everything that was compiled. With this option turned on, reflection metadata (and therefore reflection) will only be available for visible targets of reflection. Visible targets of reflection are things like assemblies rooted from the project file, RD.XML, ILLinkTrim descriptors, DynamicallyAccessedMembers annotations or DynamicDependency annotations.
-* `<IlcDisableReflection>true</IlcDisableReflection>`: this completely disables the reflection metadata generation. Very basic reflection will still work (you can still use `typeof`, call `Object.GetType()`, compare the results, and query for basic properties such as `Type.IsValueType` or `Type.BaseType`), but most of the reflection stack will no longer work (no way to query/access methods and fields on types, or get names of types). This mode is experimental - more details in the [Reflection free mode](reflection-free-mode.md) document.
 
 ## Options related to code generation
 * `<IlcOptimizationPreference>Speed</IlcOptimizationPreference>`: when generating optimized code, favor code execution speed.
 * `<IlcOptimizationPreference>Size</IlcOptimizationPreference>`: when generating optimized code, favor smaller code size.
-* `<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>`: folds method bodies with identical bytes (method body deduplication). This makes your app smaller, but the stack traces might sometimes look nonsensical (unexpected methods might show up in the stack trace because the expected method had the same bytes as the unexpected method). Note: the current implementation of deduplication doesn't attempt to make the folding unobservable to managed code: delegates pointing to two logically different methods that ended up being folded together will compare equal.
+* `<IlcInstructionSet>`: By default, the compiler targets the minimum instruction set supported by the target OS and architecture. This option allows targeting newer instruction sets for better performance. The native binary will require the instruction sets to be supported by the hardware in order to run. For example, `<IlcInstructionSet>avx2,bmi2,fma,pclmul,popcnt,aes</IlcInstructionSet>` will produce binary that takes advantage of instruction sets that are typically present on current Intel and AMD processors. Run `ilc.exe --help` for the full list of available instruction sets.
 
 ## Special considerations for Linux/macOS
 
-Debugging symbols (data about your program required for debugging) is by default part of native executable files on Unix-like operating systems. To minimize the size of your CoreRT-compiled executable, you can run the `strip` tool to remove the debugging symbols.
+Debugging symbols (data about your program required for debugging) is by default part of native executable files on Unix-like operating systems. To minimize the size of your native executable, you can run the `strip` tool to remove the debugging symbols.
 
 No action is needed on Windows since the platform convention is to generate debug information into a separate file (`*.pdb`).

src/coreclr/nativeaot/docs/reflection-free-mode.md

@@ -6,7 +6,7 @@ Reflection-free mode is a mode of the NativeAOT compiler and runtime that greatl
 * Reduced working set and better code locality - parts of the program are more tightly packed together.
 * Less metadata for people to reverese engineer - apps compiled in reflection-free mode are as hard to reverse engineer as apps written in e.g. C++.
 
-Of course the benefits come with a drawback: not all .NET code can work in such environment. In fact, most of the existing code probably won't. Use this mode with caution.
+Of course the benefits come with a drawback: not all .NET code can work in such environment. In fact, most of the existing code probably won't. Use this mode with caution. https://github.com/dotnet/runtime/issues/67193 tracks potential improvements of this mode.
 
 To enable reflection-free mode in a project that is already using CoreRT, add the following property to a `PropertyGroup` in your project file:
 

src/coreclr/nativeaot/docs/reflection-in-aot-mode.md

@@ -93,3 +93,7 @@ To enable simulated `Assembly.GetCallingAssembly`, you will need:
   <ItemGroup>
     <RuntimeHostConfigurationOption Include="Switch.System.Reflection.Assembly.SimulatedCallingAssembly" Value="true" />
   </ItemGroup>
+
+## Experimental Reflection Free Mode
+
+Reflection-free mode is a an experimental mode of the NativeAOT compiler and runtime that greatly reduces the functionality of the reflection APIs and demonstrates how far can reflection trimming can get. See [Reflection Free Mode](reflection-free-mode.md) for mode details.

src/coreclr/tools/aot/ILCompiler/Program.cs

@@ -6,6 +6,7 @@
 using System.IO;
 using System.Reflection;
 using System.Runtime.InteropServices;
+using System.Text;
 
 using Internal.IL;
 using Internal.TypeSystem;
@@ -118,41 +119,42 @@ private void Help(string helpText)
             Console.WriteLine(helpText);
         }
 
-        private void InitializeDefaultOptions()
+        public static void ComputeDefaultOptions(out TargetOS os, out TargetArchitecture arch)
         {
-            // We could offer this as a command line option, but then we also need to
-            // load a different RyuJIT, so this is a future nice to have...
             if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
-                _targetOS = TargetOS.Windows;
+                os = TargetOS.Windows;
             else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
-                _targetOS = TargetOS.Linux;
+                os = TargetOS.Linux;
             else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
-                _targetOS = TargetOS.OSX;
+                os = TargetOS.OSX;
+            else if (RuntimeInformation.IsOSPlatform(OSPlatform.FreeBSD))
+                os = TargetOS.FreeBSD;
             else
                 throw new NotImplementedException();
 
             switch (RuntimeInformation.ProcessArchitecture)
             {
                 case Architecture.X86:
-                    _targetArchitecture = TargetArchitecture.X86;
+                    arch = TargetArchitecture.X86;
                     break;
                 case Architecture.X64:
-                    _targetArchitecture = TargetArchitecture.X64;
+                    arch = TargetArchitecture.X64;
                     break;
                 case Architecture.Arm:
-                    _targetArchitecture = TargetArchitecture.ARM;
+                    arch = TargetArchitecture.ARM;
                     break;
                 case Architecture.Arm64:
-                    _targetArchitecture = TargetArchitecture.ARM64;
+                    arch = TargetArchitecture.ARM64;
                     break;
                 default:
                     throw new NotImplementedException();
             }
 
-            // Workaround for https://github.com/dotnet/corefx/issues/25267
-            // If pointer size is 8, we're obviously not an X86 process...
-            if (_targetArchitecture == TargetArchitecture.X86 && IntPtr.Size == 8)
-                _targetArchitecture = TargetArchitecture.X64;
+        }
+
+        private void InitializeDefaultOptions()
+        {
+            ComputeDefaultOptions(out _targetOS, out _targetArchitecture);
         }
 
         private ArgumentSyntax ParseCommandLine(string[] args)
@@ -241,6 +243,71 @@ private ArgumentSyntax ParseCommandLine(string[] args)
 
                 syntax.DefineParameterList("in", ref inputFiles, "Input file(s) to compile");
             });
+
+            if (_help)
+            {
+                List<string> extraHelp = new List<string>();
+
+                extraHelp.Add("Options may be passed on the command line, or via response file. On the command line switch values may be specified by passing " +
+                    "the option followed by a space followed by the value of the option, or by specifying a : between option and switch value. A response file " +
+                    "is specified by passing the @ symbol before the response file name. In a response file all options must be specified on their own lines, and " +
+                    "only the : syntax for switches is supported.");
+
+                extraHelp.Add("");
+
+                extraHelp.Add("Use the '--' option to disambiguate between input files that have begin with -- and options. After a '--' option, all arguments are " +
+                    "considered to be input files. If no input files begin with '--' then this option is not necessary.");
+
+                extraHelp.Add("");
+
+                string[] ValidArchitectures = new string[] { "arm", "arm64", "x86", "x64" };
+                string[] ValidOS = new string[] { "windows", "linux", "osx" };
+
+                Program.ComputeDefaultOptions(out TargetOS defaultOs, out TargetArchitecture defaultArch);
+
+                extraHelp.Add(String.Format("Valid switches for {0} are: '{1}'. The default value is '{2}'", "--targetos", String.Join("', '", ValidOS), defaultOs.ToString().ToLowerInvariant()));
+
+                extraHelp.Add("");
+
+                extraHelp.Add(String.Format("Valid switches for {0} are: '{1}'. The default value is '{2}'", "--targetarch", String.Join("', '", ValidArchitectures), defaultArch.ToString().ToLowerInvariant()));
+
+                extraHelp.Add("");
+
+                extraHelp.Add("The allowable values for the --instruction-set option are described in the table below. Each architecture has a different set of valid " +
+                    "instruction sets, and multiple instruction sets may be specified by separating the instructions sets by a ','. For example 'avx2,bmi,lzcnt'");
+
+                foreach (string arch in ValidArchitectures)
+                {
+                    StringBuilder archString = new StringBuilder();
+
+                    archString.Append(arch);
+                    archString.Append(": ");
+
+                    TargetArchitecture targetArch = Program.GetTargetArchitectureFromArg(arch);
+                    bool first = true;
+                    foreach (var instructionSet in Internal.JitInterface.InstructionSetFlags.ArchitectureToValidInstructionSets(targetArch))
+                    {
+                        // Only instruction sets with are specifiable should be printed to the help text
+                        if (instructionSet.Specifiable)
+                        {
+                            if (first)
+                            {
+                                first = false;
+                            }
+                            else
+                            {
+                                archString.Append(", ");
+                            }
+                            archString.Append(instructionSet.Name);
+                        }
+                    }
+
+                    extraHelp.Add(archString.ToString());
+                }
+
+                argSyntax.ExtraHelpParagraphs = extraHelp;
+            }
+
             if (waitForDebugger)
             {
                 Console.WriteLine("Waiting for debugger to attach. Press ENTER to continue");
@@ -305,6 +372,32 @@ private IReadOnlyCollection<MethodDesc> CreateInitializerList(CompilerTypeSystem
             return initializerList;
         }
 
+        private static TargetArchitecture GetTargetArchitectureFromArg(string archArg)
+        {
+            if (archArg.Equals("x86", StringComparison.OrdinalIgnoreCase))
+                return TargetArchitecture.X86;
+            else if (archArg.Equals("x64", StringComparison.OrdinalIgnoreCase))
+                return TargetArchitecture.X64;
+            else if (archArg.Equals("arm", StringComparison.OrdinalIgnoreCase))
+                return TargetArchitecture.ARM;
+            else if (archArg.Equals("arm64", StringComparison.OrdinalIgnoreCase))
+                return TargetArchitecture.ARM64;
+            else
+                throw new CommandLineException("Target architecture is not supported");
+        }
+
+        private static TargetOS GetTargetOSFromArg(string osArg)
+        {
+            if (osArg.Equals("windows", StringComparison.OrdinalIgnoreCase))
+                return TargetOS.Windows;
+            else if (osArg.Equals("linux", StringComparison.OrdinalIgnoreCase))
+                return TargetOS.Linux;
+            else if (osArg.Equals("osx", StringComparison.OrdinalIgnoreCase))
+                return TargetOS.OSX;
+            else
+                throw new CommandLineException("Target OS is not supported");
+        }
+
         private int Run(string[] args)
         {
             InitializeDefaultOptions();
@@ -324,29 +417,11 @@ private int Run(string[] args)
             //
             if (_targetArchitectureStr != null)
             {
-                if (_targetArchitectureStr.Equals("x86", StringComparison.OrdinalIgnoreCase))
-                    _targetArchitecture = TargetArchitecture.X86;
-                else if (_targetArchitectureStr.Equals("x64", StringComparison.OrdinalIgnoreCase))
-                    _targetArchitecture = TargetArchitecture.X64;
-                else if (_targetArchitectureStr.Equals("arm", StringComparison.OrdinalIgnoreCase))
-                    _targetArchitecture = TargetArchitecture.ARM;
-                else if (_targetArchitectureStr.Equals("armel", StringComparison.OrdinalIgnoreCase))
-                    _targetArchitecture = TargetArchitecture.ARM;
-                else if (_targetArchitectureStr.Equals("arm64", StringComparison.OrdinalIgnoreCase))
-                    _targetArchitecture = TargetArchitecture.ARM64;
-                else
-                    throw new CommandLineException("Target architecture is not supported");
+                _targetArchitecture = GetTargetArchitectureFromArg(_targetArchitectureStr);
             }
             if (_targetOSStr != null)
             {
-                if (_targetOSStr.Equals("windows", StringComparison.OrdinalIgnoreCase))
-                    _targetOS = TargetOS.Windows;
-                else if (_targetOSStr.Equals("linux", StringComparison.OrdinalIgnoreCase))
-                    _targetOS = TargetOS.Linux;
-                else if (_targetOSStr.Equals("osx", StringComparison.OrdinalIgnoreCase))
-                    _targetOS = TargetOS.OSX;
-                else
-                    throw new CommandLineException("Target OS is not supported");
+                _targetOS = GetTargetOSFromArg(_targetOSStr);
             }
 
             InstructionSetSupportBuilder instructionSetSupportBuilder = new InstructionSetSupportBuilder(_targetArchitecture);