'utf-8' codec can't decode byte 0xa2 in position 27: invalid start byte

[adrobkov]

Hello! I do everything according to the description. Drains 1.4 GB and falls on this error:
`2020-02-13 23:47:24,718 finished project "GTest_32"

2020-02-13 23:47:24,719 run task "msbuild_32 usvfs"
C:\modorganizer-umbrella\build\usvfs\vsbuild> msbuild usvfs.sln /maxcpucount /property:Configuration=Release /verbosity:minimal /consoleloggerparameters:Summary /fileLogger /fileloggerparameters:Summary;Verbosity=normal /property:Platform=x86 /target:Clean;Build /p:WindowsTargetPlatformVersion=10.0.18362.0
Compiling [ ] 0%
2020-02-13 23:47:24,828 'utf-8' codec can't decode byte 0xa2 in position 27: invalid start byte
Traceback (most recent call last):
File "C:\modorganizer-umbrella\unibuild\modules\msbuild.py", line 128, in process
line = proc.stdout.readline().decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 27: invalid start byte
2020-02-13 23:47:24,829 task msbuild_32 usvfs failed
2020-02-13 23:47:24,879 failed to run "C:\Users\ADrobkov\AppData\Local\Programs\Python\Python38\python.exe" unimake.py --set architecture="x86" -b "build" -p "progress" -i "install" -d "C:\modorganizer-umbrella" "usvfs" (returncode 1)
2020-02-13 23:47:24,880 task run unimake_x86_usvfs failed`
What could be the reason? Everything was installed as it is written. And if you need any data, I will provide it...

[Al12rs]

Do you have a system language that isn't english? Another user reported a similar problem and he had german as system language when installing visual studio, reinstalling it using english seemed to fix it for him.

[AnyOldName3]

Can you please run this in PowerShell and tell us what it prints?

Get-WinSystemLocale | Select-Object Name, DisplayName, @{ n='OEMCP'; e={ $_.TextInfo.OemCodePage } }, @{ n='ACP';   e={ $_.TextInfo.AnsiCodePage } }
[cultureinfo]::CurrentCulture.TextInfo.ANSICodePage

Hopefully we can figure out which of these is weird and make whichever bit of Python is failing when it gets a weird one query what's actually being used.

[adrobkov]

Name DisplayName OEMCP ACP

---

ru-RU Русский (Россия) 866 1251
1251

This response is for execution in PS. But I don't have paths with Russian names. Although the system is Russian speaking

[adrobkov]

VS must be in English?

[adrobkov]

Switched to the English interface. In Visual Studio, the Installer disabled the Russian language pack and enabled English. In PS, the answer remained the same:

Get-WinSystemLocale | Select-Object Name, DisplayName, @{ n='OEMCP'; e={ $.TextInfo.OemCodePage } }, @{ n='ACP'; e={ $.TextInfo.AnsiCodePage } }
Name DisplayName OEMCP ACP

---

ru-RU Russian (Russia) 866 1251

After that, I was able to install the project via python.exe unimake.py

Open organizer.sln. But when building in debug mode, mismatch errors are cooked and the project is not assembled. And in the release only the last successful version is collected for the selected INSTALL project.
What am I doing wrong? What am I missing?

[adrobkov]

[adrobkov]

[adrobkov]

1>------ Build started: Project: ModOrganizer, Configuration: Release x64 ------
1>Automatic MOC and UIC for target ModOrganizer
1>Generating ../../src/organizer_en.ts
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/loot.cpp:58: Unterminated C++ character
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/modinfodialogconflicts.cpp:299: Class 'OverwriteConflictListModel' lacks Q_OBJECT macro
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/modinfodialogconflicts.cpp:312: Class 'OverwrittenConflictListModel' lacks Q_OBJECT macro
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/modinfodialogconflicts.cpp:325: Class 'NoConflictListModel' lacks Q_OBJECT macro
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/modinfodialogconflicts.cpp:337: Class 'AdvancedConflictListModel' lacks Q_OBJECT macro
1>C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/src/overwriteinfodialog.cpp:48: Class 'MyFileSystemModel' lacks Q_OBJECT macro
1>Generating organizer_en.qm
1>Updating 'C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/vsbuild/src/organizer_en.qm'...
1> Generated 0 translation(s) (0 finished and 0 unfinished)
1> Ignored 1614 untranslated source text(s)
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5): warning MSB8064: Custom build for item "C:\modorganizer-umbrella\build\modorganizer_super\modorganizer\vsbuild\CMakeFiles\67e9185306bd440bd6e0658fb15e676f\organizer_en.ts.rule" succeeded, but specified dependency "c:\modorganizer-umbrella\build\modorganizer_super\modorganizer\src" does not exist. This may cause incremental build to work incorrectly.
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5): warning MSB8064: Custom build for item "C:\modorganizer-umbrella\build\modorganizer_super\modorganizer\vsbuild\CMakeFiles\67e9185306bd440bd6e0658fb15e676f\organizer_en.ts.rule" succeeded, but specified dependency "c:\modorganizer-umbrella\build\modorganizer_super\uibase\src" does not exist. This may cause incremental build to work incorrectly.
1>ModOrganizer.vcxproj -> C:\modorganizer-umbrella\build\modorganizer_super\modorganizer\vsbuild\src\Release\ModOrganizer.exe
1>Done building project "ModOrganizer.vcxproj".
2>------ Build started: Project: INSTALL, Configuration: Release x64 ------
2>-- Install configuration: "Release"
2>CMake Error at src/cmake_install.cmake:51 (file):
2> file INSTALL cannot find
2> "C:/modorganizer-umbrella/build/modorganizer_super/modorganizer/vsbuild/src/Release/ModOrganizer.pdb":
2> No error.
2>Call Stack (most recent call first):
2> cmake_install.cmake:41 (include)
2>
2>
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: The command "setlocal
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: "C:\Program Files\CMake\bin\cmake.exe" -DBUILD_TYPE=Release -P cmake_install.cmake
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: :cmEnd
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: :cmErrorLevel
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: exit /b %1
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: :cmDone
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd
2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(149,5): error MSB3073: :VCEnd" exited with code 1.
2>Done building project "INSTALL.vcxproj" -- FAILED.
========== Build: 1 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========

[Al12rs]

You need to use RelWithDebugInfo as configuration.
Drop by our Development discord https://discord.gg/JM7sxT so we can help you building more directly

[adrobkov]

Thanks!!!

[AnyOldName3]

This issue shouldn't be closed yet as it's not intentional that you need to set the VS language to English. Thankfully, the stack trace in the first post says exactly where we're decoding something as UTF-8 that isn't UTF-8, but if you can change the VS language separately from the system's code page, that could make it difficult to detect the right one from Python.

Hopefully it's just relying on there being Latin characters in cp1251 and it's still using the system default encoding.

[AnyOldName3]

Apparently we can request MSBuild uses a specific encoding (like UTF8 which would work everywhere) when logging to a file, but not to stdout, which is irritating.

[adrobkov]

OK! I don't understand much about it, to be honest. My goal was to add several fields. I did it. If you need my help to catch an error or something else-please contact me-I will help!

[AnyOldName3]

I think I know what needs changing. Can you try setting the language back to Russian, and then changing this line https://github.com/ModOrganizer2/modorganizer-umbrella/blob/master/unibuild/modules/msbuild.py#L128 to use mbcs instead of utf-8 on your machine and trying the build again?

[adrobkov]

Yes, it worked!

[adrobkov]

2020-02-15 14:44:35,826 finished project "spdlog"

2020-02-15 14:44:35,828 run task "cmake zlib"
C:\modorganizer\build\zlib-1.2.11\build> C:\Program Files\CMake\bin\cmake.exe -G NMake Makefiles .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX:PATH=C:\modorganizer\build\zlib-1.2.11
C:\modorganizer\build\zlib-1.2.11\build> nmake verbose=1
Compiling [========= ] 18%
2020-02-15 14:44:44,107 'utf-8' codec can't decode byte 0xaf in position 67: invalid start byte
Traceback (most recent call last):
File "C:\modorganizer\unibuild\modules\cmake.py", line 110, in process
line = proc.stdout.readline().decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 67: invalid start byte
2020-02-15 14:44:44,111 task cmake zlib failed

[adrobkov]

cmake.py
line = proc.stdout.readline().decode("utf-8") -> line = proc.stdout.readline().decode("mbcs")

[AnyOldName3]

Yeah, there's another place in the same commit where Silarn defaulted to UTF-8: 0697ec7

Does everything work if you sort out the CMake one, too?

[adrobkov]

2020-02-15 16:56:42,481 run task "msbuild ncc"
C:\modorganizer\build\NexusClientCli> msbuild C:\modorganizer\build\Nexus-Mod-Manager\NexusClientCli.sln /maxcpucount /property:Configuration=Release /verbosity:minimal /consoleloggerparameters:Summary /fileLogger /fileloggerparameters:Summary;Verbosity=normal /property:Platform=Any CPU /p:WindowsTargetPlatformVersion=10.0.18362.0
Compiling [ ] 0%
2020-02-15 16:57:01,072 failed to build (returncode 1), see C:\modorganizer\build\NexusClientCli\stdout.log and C:\modorganizer\build\NexusClientCli\stderr.log
Traceback (most recent call last):
File "C:\modorganizer\unibuild\modules\msbuild.py", line 138, in process
raise Exception("failed to build (returncode %s), see %s and %s" % (proc.returncode, soutpath, serrpath))
Exception: failed to build (returncode 1), see C:\modorganizer\build\NexusClientCli\stdout.log and C:\modorganizer\build\NexusClientCli\stderr.log
2020-02-15 16:57:01,077 task msbuild ncc failed

[adrobkov]

I did not encounter such an error during the last installation in the Russian version of the system and VS

[AnyOldName3]

Can you put the two log files in two github gists and link them here? I don't think that's likely to be the same encoding problem, but obviously things shouldn't break.

[adrobkov]

I don't know what to do? What are the files? Yes, and the gists somehow did not have to use.

[AnyOldName3]

The error you got says

Compiling [ ] 0%
2020-02-15 16:57:01,072 failed to build (returncode 1), see C:\modorganizer\build\NexusClientCli\stdout.log and C:\modorganizer\build\NexusClientCli\stderr.log

If you look in those files and upload them as github gists, it will say what broke.

[adrobkov]

https://gist.github.com/adrobkov/2468163dc6bca1a805e5a050ccaf710b

[AnyOldName3]

I have a suspicion that the Nexus have done something dumb. What happens if you download the Nexus Mod Manager source on its own and try to build it with your language still set to Russian?

I think they've made it build using a file specific to the system language, and there's not one for Russian. The English and German ones are identical, so there's probably no reason why you couldn't make another copy and give it the right name for Russian and submit it in a pull request to the Nexus.

[adrobkov]

I have now encountered another error! I don't even know what the reason is. Very strange. I made a lot of attempts, but it feels like something broke. Despite the fact that I tried it in the English-language system.

C:\modorganizer>python.exe unimake.py Traceback (most recent call last): File "unimake.py", line 18, in <module> import eggs File "C:\modorganizer\eggs\__init__.py", line 20, in <module> import urllib.request, urllib.error, urllib.parse ImportError: No module named request

All software was reinstalled, changed paths, disks, tried on the current repository and on the new one.

[AnyOldName3]

Is it possible you're using Python 2 instead of 3 or have otherwise buggered your Python install? urllib.request is part of the standard Python library, so it should definitely be available if you've got a working new enough Python version.

[adrobkov]

YES, it's related to Python. Why is it in the PATH would have ordered 2 and not 3.
But "2020-02-20 14: 22:13,752 task msbuild ncc failed" remains so. Regardless of the system language

[AnyOldName3]

If you put something on the front of the system path, it's going to take priority over anything that's already there. Most installers just put their entry at the front as that's how they can guarantee it can be seen. I guess you installed 2.7 after 3.

If you want to run with a specific version when both are installed, instead of python , use py -2 for 2.x and py -3 your 3.x.

As for the NCC stuff, have you tried what I suggested, i.e.trying to build Nexus Mod Manager from source from the official repo. That'll help us narrow down whether it's or cider on top of NMM that's breaking our if it's their code.

[adrobkov]

Today, I finally got to a new attempt to build a project on the Russian- language system. And have encountered an unexpected error:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.7-zip.org', port=80): Max retries exceeded with url: /a/7z1900-src.7z (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001FACF4C99D0>: Failed to establish a new connection: [WinError 10060] Попытка установить соединение была безуспешной, т.к. от другого компьютера за требуемое время не получен нужный отклик, или было разорвано уже установленное соединение из-за неверного отклика уже подключенного компьютера'))

And in fact , "http://www.7-zip.org" unavailable.
I of course copied it to a folder from the previous installation , but I think you just need to know if it will not be available again!

[adrobkov]

https://github.com/Nexus-Mods/Nexus-Mod-Manager
I don 't understand what kind of client should I put from nexus? I found this one , but in the build folder , there are 2 folders: NexusClientCli and Nexus-Mod-Manager. Error on the first one, but I can 't find its repository.