unable to create database if there is a space in the command argument

[sandilyabhagi]

From, https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/static-tools-and-codeql

C:\codeql-home>C:\codeql-home\codeql\codeql.cmd database create -l=cpp -s=C:\codeql-home\drivers\kmdf -c "msbuild /t:rebuild "C:\codeql-home\drivers\kmdf\kmdfecho.sln" /p:UseSharedCompilation=false" "C:\codeql-home\databases\kmdf" -j 0

If the above command is modified to incorporate configuration the database creation fails,
C:\codeql-home>C:\codeql-home\codeql\codeql.cmd database create -l=cpp -s=C:\codeql-home\drivers\kmdf -c "msbuild /t:rebuild "C:\codeql-home\drivers\kmdf\kmdfecho.sln" /p:Configuration="win10 Debug" /p:UseSharedCompilation=false" "C:\codeql-home\databases\kmdf" -j 0

Error:
The system cannot find the file specified.
Invalid value for positional parameter at index 0 (): cannot convert 'Debug /p:UseSharedCompilation=false' to Path (java.nio.file.InvalidPathException: Trailing char < > at index 5: Debug /p:UseSharedCompilation=false)
Try codeql database create --help for usage help.

However the same command using msbuild succeeds. Also the same command without space works,
C:\codeql-home>C:\codeql-home\codeql\codeql.cmd database create -l=cpp -s=C:\codeql-home\drivers\kmdf -c "msbuild /t:rebuild "C:\codeql-home\drivers\kmdf\kmdfecho.sln" /p:Configuration="win10Debug" /p:UseSharedCompilation=false" "C:\codeql-home\databases\kmdf" -j 0

Creates database successfully.
Initializing database at C:\codeql-home\databases\kmdf.
Running build command: [msbuild, /t:rebuild, kmdfecho.sln, /p:Configuration=win10Debug, /p:UseSharedCompilation=false]

[adityasharad]

Thanks for reporting this! We have a fix in progress that will make it into a future version of CodeQL, most likely 2.6.0.

[sandilyabhagi]

Thank you Aditya.

[m4sachi]

@adityasharad Is the beta version have the fix for this above issue?

[sandilyabhagi]

@adityasharad Is this issue fixed in the latest v2.6.0 ? Can you please confirm. Thank you.
I have tried and looks like either it is not fixed or I have tried it incorrectly.

[adityasharad]

To handle spaces in your command arguments, I believe the safest approach is to create a separate batch script that contains your build command, and invoke that from codeql database create. (An alternative is to escape the quotes within the argument to -c "...", but I recommend the separate script as a first approach.)

For example, create a script build.bat containing

msbuild /t:rebuild "C:\codeql-home\drivers\kmdf\kmdfecho.sln" /p:Configuration="win10 Debug" /p:UseSharedCompilation=false

and then run

C:\codeql-home\codeql\codeql.cmd database create -l=cpp -s="C:\codeql-home\drivers\kmdf" -c "build.bat" "C:\codeql-home\databases\kmdf" -j 0

Could you please try that with CodeQL 2.6.0?

[sandilyabhagi]

@adityasharad Thank you for your response.

To handle spaces in your command arguments, I believe the safest approach is to create a separate batch script that contains your build command, and invoke that from codeql database create. (An alternative is to escape the quotes within the argument to -c "...", but I recommend the separate script as a first approach.)

We are currently using this approach and it is working well from v2.5.8(we started from this version). Only issue is we are adding additional bat file in the process and want to know if it can be avoided. If this is the preferred solution we will continue as suggested.

Thanks for reporting this! We have a fix in progress that will make it into a future version of CodeQL, most likely 2.6.0.

Just confirming since it was mentioned. Please keep us posted in case if it can be expected in upcoming releases.

[sandilyabhagi]

(An alternative is to escape the quotes within the argument to -c "...", but I recommend the separate script as a first approach.)

Hi @adityasharad tried escaping the quotes and didn't work. Tried the following and none of them worked. Do you have anything to suggest here to escape the quotes,

1. Without any quotes for msbuild command argument
   call D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\codeql.cmd database create -l=cpp -s=%CURDIR%\Src\kernel\mydrv -c "msbuild myproj.VcxProj /t:Rebuild /p:Configuration=Windows 10 Release" "D:\work\CodeQL\codeql-home\databases\myproj" -j 0
   Running build command: [msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration=Windows, 10, Release]
   [2021-08-30 12:08:43] [build-stdout] Switch: 10
   [2021-08-30 12:08:43] [ERROR] Spawned process exited abnormally (code 1; tried to run: [D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\tracer.exe, D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration=Windows, 10, Release])
   A fatal error occurred: Exit status 1 from command: [D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration=Windows, 10, Release]

2. With Single quote for msbuild command argument
   call D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\codeql.cmd database create -l=cpp -s=%CURDIR%\Src\kernel\mydrv -c "msbuild myproj.VcxProj /t:Rebuild /p:Configuration='Windows 10 Release'" "D:\work\CodeQL\codeql-home\databases\myproj" -j 0
   Running build command: [msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration='Windows, 10, Release']
   [2021-08-30 12:09:19] [build-stdout] Switch: 10
   [2021-08-30 12:09:19] [ERROR] Spawned process exited abnormally (code 1; tried to run: [D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\tracer.exe, D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration='Windows, 10, Release'])
   A fatal error occurred: Exit status 1 from command: [D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, msbuild, myproj.VcxProj, /t:Rebuild, /p:Configuration='Windows, 10, Release']

3. With Double quotes for msbuild command argument
   call D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\codeql.cmd database create -l=cpp -s=%CURDIR%\Src\kernel\mydrv -c "msbuild myproj.VcxProj /t:Rebuild /p:Configuration="Windows 10 Release"" "D:\work\CodeQL\codeql-home\databases\myproj" -j 0
   The system cannot find the file specified.
   Unmatched arguments from index 5: 'Release', 'D:\work\CodeQL\codeql-home\databases\myproj'

4. With Double quotes escaped with \ for msbuild command argument
   call D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\codeql.cmd database create -l=cpp -s=%CURDIR%\Src\kernel\mydrv -c "msbuild myproj.VcxProj /t:Rebuild /p:Configuration=\"Windows 10 Release\"" "D:\work\CodeQL\codeql-home\databases\myproj" -j 0
   Initializing database at D:\work\CodeQL\codeql-home\databases\myproj.
   A fatal error occurred: Failed to expand 'msbuild myproj.VcxProj /t:Rebuild /p:Configuration="Windows 10 Release"' as an argument list.
   (eventual cause: UserError "At position 54, the quote should either be preceded by a space (if it is intende...")

5. With Double quotes only for msbuild command argument
   call D:\work\CodeQL\codeql-home\codeql-win64-2.6.0\codeql\codeql.cmd database create -l=cpp -s=%CURDIR%\Src\kernel\mydrv -c msbuild myproj.VcxProj /t:Rebuild /p:Configuration="Windows 10 Release" "D:\work\CodeQL\codeql-home\databases\myproj" -j 0
   Unmatched arguments from index 5: '/t:Rebuild', '/p:Configuration=Windows 10 Release', 'D:\work\CodeQL\codeql-home\databases\myproj'
   Try codeql database create --help for usage help.

[sam-cha-tfs]

Has this been resolved? running into the same issue

[qianfei11]

Running into the same issue in release 2.15.1, solved by putting the commands into a shell script and replace the command with the script.

[sam-cha-tfs]

@qianfei11 I think this is a ongoing issue. I had to do the same thing in order to get the commands to work.