Skip to content

Windows recipe patterns

Patrick Snape edited this page Apr 8, 2016 · 12 revisions

Building blocks for Windows recipes

Building things on Windows can be tricky because there are 3 different compilers to contend with (VS 2008 for Py 2.7-Py3.2, VS 2010 for Py 3.3-Py3.4, and VS 2015 for Py 3.5+). Hopefully the patterns below will help you get started faster.

Checking for 32-bit or 64-bit

This is often used to feed the correct platform to msbuild, or to adjust configuration steps.

if "%ARCH%" == "64" (
  set ARCH=x64
) else (
  set ARCH=Win32
)

Using CMake with Visual Studio

There are at least two options using CMake with Visual Studio: NMake makefiles, and version-specific solution files.

NMake makefiles

Visual Studio includes a tool called NMake that is much like Make on Linux and OS X. CMake can output NMake makefiles. These makefiles are compiled with whatever compiler is active for conda-build. This means no additional configuration is necessary for you to support multiple Python platforms.

cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%LIBRARY_PREFIX% ..\
cmake --build . --target INSTALL --config Release

Version-specific solution files

The downside of using NMake is that it does not build in parallel, and there are occasionally strange corner cases that prevent it from working.

If you have problems using NMake, try generating version-specific solution files instead.

REM write a temporary batch file to map cl.exe version to visual studio version
echo @echo 15=9 2008> msvc_versions.bat
echo @echo 16=10 2010>> msvc_versions.bat
echo @echo 19=14 2015>> msvc_versions.bat

REM Run cl.exe to find which version our compiler is
for /f "delims=" %%A in ('cl /? 2^>^&1 ^| findstr /C:"Version"') do set "CL_TEXT=%%A"
FOR /F "tokens=1,2 delims==" %%i IN ('msvc_versions.bat') DO echo %CL_TEXT% | findstr /C:"Version %%i" > nul && set VSTRING=%%j && goto FOUND
EXIT 1
:FOUND

if "%ARCH%" == "64" (
   set "VSTRING=%VSTRING%Win64"
)

REM Trim trailing whitespace that may prevent CMake from finding which generator to use
call :TRIM VSTRING %VSTRING%

cd build

cmake -G "Visual Studio %VSTRING%" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%LIBRARY_PREFIX% ..
cmake --build . --target INSTALL --config Release
 
:TRIM
  SetLocal EnableDelayedExpansion
  set Params=%*
  for /f "tokens=1*" %%a in ("!Params!") do EndLocal & set %1=%%b
  exit /B

Note: If you are using the 'Microsoft Visual C++ Compiler for Python 2.7', you will not be able to use the Visual Studio generator in CMake. This is due to the fact that CMake expects either msbuild or devenv to exist, and the Python 2.7 Compiler tools do not contain either of these. If this is the case, we suggest you try NMake.

Updating older .sln files

It is not necessary to create solution files for each different VS version you want. Instead, you can take a VS 2008 solution and update it as necessary.

devenv.exe FreeImage.2008.sln /Upgrade
msbuild FreeImage.2008.sln /property:Configuration=Release /property:Platform=%ARCH%

Passing data from Python to your build script

It can be helpful to pass characteristics about your Python environment that Conda-build does not make available as an environment variable. Obtain these values by executing print statements:

for /f "delims=" %%A in ('%PREFIX%\python -c "import sys; print(sys.version_info.major)"') DO SET PY_MAJOR=%%A

for /f "delims=" %%A in ('%PREFIX%\python -c "import sys; print(sys.version_info.minor)"') DO SET PY_MINOR=%%A

Downloading files in addition to any source download

Conda build takes care of downloading the source you specify in meta.yaml, but what if you need something more? The answer is to include curl as a build-time requirement, then use it in bld.bat.

In meta.yaml:

requirements:
    build:
        - curl

In bld.bat:

curl -O http://www.example.com/test_file.h

This saves that file with its name into the local folder (the work folder). More examples of downloading files with curl are at: http://www.compciv.org/recipes/cli/downloading-with-curl/

Missing stdint.h

Visual Studio 2008 did not include stdint.h. You can download them using:

curl -O http://msinttypes.googlecode.com/svn/trunk/stdint.h

Note that you need curl as a build-time requirement for this to work.

Finally, copy this downloaded file wherever your recipe looks for include files. For example, if your source code working folder has an include folder:

copy stdint.h %SRC_DIR%\include\stdint.h

Duplicate stdint.h symbols

Very similar to missing stdint.h above. Many recipes compensate for the lack of stdint.h by including their own definitions. If a recipe does not limit the scope correctly, these symbols conflict with those that Microsoft provides (starting with VS 2010). To avoid duplicate symbols, fix the includes.

The preprocessor lines where stdint.h is should be:

#if defined(_MSC_VER) && _MSC_VER >= 1600
#include "stdint.h"
#else
#include "<internal library stdint definitions>"
#endif

Elevation problems

As of Windows Vista or 7, Windows employs name heuristics and requires elevation, even when programs shouldn't normally require elevation. There are more details at http://superuser.com/questions/102975/how-to-tell-windows-7-that-an-application-does-not-need-to-run-with-admin-rights

To make your program not require elevation, you can either create a manifest file with the following contents:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
        <security>
            <requestedPrivileges>
                <requestedExecutionLevel level="asInvoker" uiAccess="false"/>
            </requestedPrivileges>
        </security>
    </trustInfo>
</assembly>

named <your_program>.exe.manifest - and place it alongside your exe.

You can also embed this manifest into your exe, so that you don't have to ship the manifest file with your application:

mt.exe -nologo -manifest "%RECIPE_DIR%\<your_manifest_file.manifest>" -updateresource:"<your_program>.exe;#1"

If, after all this, your program still requires elevation, it is because your program is performing operations that require elevation. There is no way around it. There are sometimes flags you can pass to your program to disable those functions. For example, cygwin's installer takes the -B flag to disable these kinds of operations: https://cygwin.com/ml/cygwin-apps/2013-11/msg00019.html

Checksum validation in bat scripts

The main download of a conda recipe may not include everything you need. If you need to download additional files in the recipe, you should add curl to your build requirements, and download the file(s) you need. However, it is still good to calculate a signature of any file you download, to ensure that it is what you think it is.

set HASH_TYPE=<hash_type (supported types: MD2 MD4 MD5 SHA1 SHA256 SHA384 SHA512)>
set HASH=<your_hash>
set FILENAME=<your_filename>

curl -L <your_url -o %FILENAME%
set command="if ( $($(CertUtil -hashfile %FILENAME% %HASH_TYPE%)[1] -replace ' ','') -eq '%HASH%' ) { echo '%FILENAME% download ok' } else {echo '%FILENAME% checksum bad.  Has it changed/been updated?' -and exit 1}"
powershell -Command %command%
if errorlevel 1 exit 1

This example is largely derived from http://superuser.com/a/898377/184799