Not all plugins work with MSVC or Windows

[nemequ]

Several plugins don't work on Windows. The following plugins fail to compile with MSVC:

• Density
  • MSFT build ? g1mv/density#49
• Gipfeli
  • Fix memory leak when sinks throw an exception google/gipfeli#3
  • Fix use of pointer into string after assignment google/gipfeli#4
  • Fix read of uninitialized memory google/gipfeli#5
  • Don't hardcode HAVE_BUILTIN_* macros in config.h google/gipfeli#7
• Snappy
• Zling
  • zling plugin issue #153
  • Add squash MSVC support richox/libzling#9
• LZFSE
  • MSVC support lzfse/lzfse#8
• Brotli

Assuming the issues are with the code not Squash's build system, upstreams need to be notified and, ideally, provided with a patch.

There may also be some plugins which compile but are buggy. Unfortunately, since Squash's Windows support is so new (and buggy) it's hard to tell right now which bugs are in Squash and which are in the libraries.

[nemequ]

@jibsen, brotli seems to work. Do you remember why you disabled it?

[jibsen]

You are right, it was the squash plugin code that didn't compile, and 8ced0b1 fixed it.

[jibsen]

I have submitted a couple of PR to Gipfeli that appear to fix the plugin on Windows.

It should probably be compiled as C++11 also.

[nemequ]

Thanks. I added a config.h file for Snappy which will typedef ptrdiff_t ssize_t if the target doesn't have ssize_t, so it should work now. IMHO your solution is much nicer, but this way we don't need to wait on the snappy devs.

[jibsen]

Zling was updated and builds here with MSVC and mingw-w64.

[nemequ]

Nice, thanks. Updated (b5593e4).

[elliotwoods]

Side note:
Congratulations on all the great work with Squash.
It's everything i'd hoped a compression library would be one day. Especially this page : https://quixdb.github.io/squash-benchmark/
me + @satoruhiga are looking to implement ofxSquash (a squash wrapper for openFrameworks)

The main course:
The first platform develop on is MSVC2015/x64 here, and we're especially interested in the Density algorithm. From what I understand, we can't use that with MSVC.

The question:

• Are there any caveats (is it even possible) to build under MinGW and then use under MSVC?
• This seems under active development. Is there anything I should watch out for? Is this an active priority?

Thanks!

[elliotwoods]

perhaps this is an option for plugins which specifically want clang/gcc:
https://blogs.msdn.microsoft.com/vcblog/2015/12/04/clang-with-microsoft-codegen-in-vs-2015-update-1/

[jibsen]

Thank you for the kind words about squash! (mostly on behalf of @nemequ, my contribution has just been to add a few #ifdef _MSC_VER here and there).

Are there any caveats (is it even possible) to build under MinGW and then use under MSVC?

Squash provides a plain C interface, so in theory there should be no problem calling the dll and plugins compiled with MinGW from MSVC. However in practice, unless you restrict yourself to interfaces that only pass pointers to memory or filenames, you will get issues. Things like passing a file handle created in one to the other will crash. There are also some practical issues you will have to handle, like differences in file names and how the runtime is linked to the dll files.

So it's possible, but it might be easier to port DENSITY to MSVC.

This seems under active development. Is there anything I should watch out for? Is this an active priority?

@nemequ is the one to answer that. I think the changes to squash itself to compile with MinGW and MSVC are more or less done, but we are waiting for a few upstream fixes for MSVC compatibility. We were looking at clang support on Windows, but ran into some bugs.

perhaps this is an option for plugins which specifically want clang/gcc

Thank you for the link. A clang toolset supported by MS sounds interesting. CMake assumes clang running on MSVC uses the cl wrapper, but this clang toolset uses gcc switches. This currently results in some issues.

That being said, I did manage to build the DENSITY squash plugin using it, and it passed the unit tests. I also had an ICE and a couple of other bugs from within the compiler, so I would say it's promising, but perhaps not quite ready.

[nemequ]

me + @satoruhiga are looking to implement ofxSquash (a squash wrapper for openFrameworks)

Nice. I hadn't heard of openFrameworks, it looks like a very interesting project.

Thank you for the kind words about squash! (mostly on behalf of @nemequ, my contribution has just been to add a few #ifdef _MSC_VER here and there).

Hugely understated. Not only did you get Squash working on Windows (which was much more than adding an ifdef here and there), you've submitted lots of fixes for upstream codecs to get them working, too. Your contributions to Squash have been very significant, but your contributions to the larger community have been amazing.

This seems under active development. Is there anything I should watch out for? Is this an active priority?

I'm not sure what you mean by "this". If you're talking about Squash, yes, it's under active development, and not quite API/ABI stable yet. I have a few minor changes planned for plugins, but that should be transparent to you. The user-facing API may see some minor changes (for example, I'm considering changing the index arguments of squash_options_get_*_at from size_t to int), but for the most part it should be pretty stable now.

If you're talking about Squash on Windows, I think whether it is an active area of development is largely up to others (@jibsen has done most of the work so far). I'm basically useless on Windows, so I mostly have to rely on others to test, report issues, and often even to fix the issues. If you are interested in helping, probably the best thing you could do right now would be to add configurations to our AppVeyor configuration to help shed light on any issues. Once we know about issues and have a way to reproduce them it's a lot easier to figure out fixes.

If you're talking about getting density working with MSVC, I'm not sure. Guillaume (who wrote density) has been busy with other stuff for a while now, I'm not sure when (or if) he'll get back to density. That said, I really don't think it would be too much work for someone to get density working on Windows; it just requires someone who uses Windows to put in some effort. It doesn't really have anything to do with the underlying compression code, AFAIK the issues are with things like restricted pointers and function inlining.

That said, for density I'm more concerned with the fact that there are known security issues with the decompressor (see g1mv/density#60) which haven't been fixed, making it unsuitable for untrusted data. If that isn't fixed by the time Squash 0.8 is ready I'll have to disable the plugin by default, which I really don't want to do; density is a great codec. TBH for most other codecs I probably would have done it a long time ago.

[elliotwoods]

hey @jibsen and @nemequ
thanks for the warm welcome!
Sorry for not being clear, I was referring to whether the MSVC implementation was under active development.

So far I'm just trying to get a cleanly working setup here which works 'out of the box' for other openFrameworks users. I'm not familiar with AppVeyor, I might not be of much help here but i'll do what i can.

@jibsen - great to see you had some progress with CodeGen
but yes. it would be great if MSVC support was added to density! (and that nasty security issue was fixed)

we're using density plugin dll built with MinGW + squash dll built in MSVC
this is hosted in an MSVC project
all builds fine so far.