Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak on OS X #35

Open
sethfowler opened this issue Apr 17, 2013 · 11 comments
Open

Memory leak on OS X #35

sethfowler opened this issue Apr 17, 2013 · 11 comments

Comments

@sethfowler
Copy link

There seems to be a memory leak when using this library on OS X. I am not sure if it comes from code in System.FSNotify or from the underlying System.OSX.FSEvents, though.

@gregwebs
Copy link
Member

Thanks for the report. Can you give a minimal test program that creates the memory leak?

@sethfowler
Copy link
Author

Sure. This program is enough to do it:

{-# LANGUAGE OverloadedStrings #-}

import System.FSNotify
import System.IO

main = withManager doWatch

doWatch m = do
  putStrLn "doWatch is starting. Do something that will cause large amounts of"
  putStrLn "disk activity in the current directory; my test case is to compile"
  putStrLn "Firefox. Press Enter when things settle down to terminate. This"
  putStrLn "program should easily reach hundreds of megabytes of memory"
  putStrLn "when compiling a large program like Firefox."
  watchTree m "." (const True) (\e -> putStrLn $ "Got event " ++ (show e))
  getLine >> return ()

Instructions are included. =) If you want to try with my exact test case, you can grab a copy of the Firefox source using hg clone http://hg.mozilla.org/mozilla-central and build it with make -f client.mk.

I just ran this twice and checked the final memory usage as reported by Activity Monitor. I got 442.9 MB on the first run and 650.7 MB on the second.

@gregwebs
Copy link
Member

One thing that might be related to this: I had slow performance with const True on a large code base, but once I filtered down to the files I actually needed it worked fine.

@sethfowler
Copy link
Author

It's very possible that they're related, yeah. In my real application I filter out everything but C++ source files and the leak does seem slower. If filtered out events don't contribute to the leak, that is quite interesting. I'm not 100% sure there's actually an effect, though, because there's so much variance in the leak size.

@sethfowler
Copy link
Author

The hfsevents package comes with a trace program in the tests directory that does something very similar to what my test program above does. The difference is that trace does not leak memory. I'm afraid that the leak does not appear to be in System.OSX.FSEvents. It must be in this package.

@sethfowler
Copy link
Author

@gregwebs I'm thinking of trying to get this fixed sometime in the near future. Which repo and branch should I work from?

@gregwebs
Copy link
Member

the rc branch. Thanks!

@thomasjm
Copy link
Contributor

@sethfowler I realize this was 2 years ago, but did you ever make any progress on memory leak issues? I'm running into it on Ubuntu linux as well. Setting up a watch with a broad predicate (essentially just const True) causes big memory usage.

In addition, I noticed that if I trigger a large number of files changes by invoking chown -R on the watched folder, then the memory usage becomes even bigger (like hundreds of megabytes in my case)

@sethfowler
Copy link
Author

@thomasjm I didn't, unfortunately. I just ended up periodically killing and restarting the thread on a timer.

@thomasjm
Copy link
Contributor

Note to self: try replacing Chans with strict-concurrency versions and see if it helps

@simonmichael
Copy link

simonmichael commented Nov 8, 2019

We see this with hledger-ui too. [Using watchDir, with a restrictive predicate matching only a few files.]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants