Fix restore causing unicorn cpu exception

[crass]

When restoring a snapshot created with reg=True a unicorn cpu exception can
be thrown when restoring the GS register. It looks like for at least 32bit
mode on X86 processors unicorn will validate memory structures pointed to by
loaded GS reg. If the memory is not loaded the validation will surely fail,
and thus the exception. The fix is then to load the memory maps first,
before loading the registers. Note, this validation does not happen when
restoring via cpu_context.

Included is a test case which will have errors if this fix is not applied.

Checklist

Which kind of PR do you create?

• This PR only contains minor fixes.
• This PR contains major feature update.
• This PR introduces a new function/api for Qiling Framework.

Coding convention?

• The new code conforms to Qiling Framework naming convention.
• The imports are arranged properly.
• Essential comments are added.
• The reference of the new code is pointed out.

Extra tests?

• No extra tests are needed for this PR.
• I have added enough tests for this PR.
• Tests will be added after some discussion and review.

Changelog?

• This PR doesn't need to update Changelog.
• Changelog will be updated after some proper review, if needed
• Changelog has been updated in my PR.

Target branch?

• The target branch is dev branch.

One last thing

• I have read the contribution guide

---

[crass]

Also, I've added the new test_snapshot.py file to the linux script for running all its tests. I think the test should be capable of running on windows, but can't test that right now. The only reason why it might not run on windows is the hardcoded path separators which are different between linux and windows (but then again, python might be smart enough to convert them).

[xwings]

The test should be in test_elf. but again, we already have the same test? any reason why we need a new code and there are no changes in the code ?

[crass]

There is a change to the code, see changes to file qiling/core.py . Or am I misunderstanding you? What is the "same test" you are referring to? Can you point me to the specific test? Try running the tests without the change in qiling/core.py and you'll see the tests fail.

The reason that I created a separate test file instead of putting the test in test_elf.py is because I was thinking this test should be run on both linux and windows. Actually, all linux tests should be run on windows too (that's the point of this framework). And all windows tests should run on linux, altough this is trickier because the windows tests need windows system libraries. But that's a discussion for a different issue.

[xwings]

besides moving the location for mem restore. i did not see any changes in core.py

[xwings]

There are few similar snapshot test eg,

qiling/tests/test_elf.py

Line 33 in 35db51a

def test_elf_partial_linux_x8664(self):

will you be able to move it to your new snapshot test ?

[crass]

besides moving the location for mem restore. i did not see any changes in core.py

Moving the location of mem restore is the fix. That way memory is mapped before the GS register is loaded.

will you be able to move it to your new snapshot test ?

Yes, I can move that to the test_snapshot.py file. I will also add test_snapshot.py to windows tests and make sure the path issue I mentioned above is not a problem.

[xwings]

Yes, I can move that to the test_snapshot.py file. I will also add test_snapshot.py to windows tests and make sure the path issue I mentioned above is not a problem.

linux and windows test always seperate. due to dll restriction. So, you can either

1. move linux snapshot test to test_elf, and Windows snapshottest to test_pe
2. make two different snapshot test, test_snapshot_elf and test snapshot_pe

method 1 is cleaner

[crass]

linux and windows test always seperate. due to dll restriction. So, you can either

1. move linux snapshot test to test_elf, and Windows snapshottest to test_pe

2. make two different snapshot test, test_snapshot_elf and test snapshot_pe

method 1 is cleaner

Now I'm thinking about this again. and I think option 1 is better, to put my test in test_elf.py. And in a separate change we should get all linux tests running in windows. I don't think I should put the test in test_pe.py` because its not technically a PE test.

I agree that the linux and windows tests should be separate, but I think that test_pe.bat should be renamed to test_onwindows.bat and test_onwindows.bat should include all windows tests. As you say, we can not put all the windows tests in test_onlinux.sh because of the dll restriction.

[xwings]

There are some issue with elf test runs on windows. that is the reason why we did not run linux test on windows.

[crass]

What are some of the issues?

[xwings]

checkout issue #333 or just run the test in windows.

[xwings]

merge for now, thanks!