Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error while backup large collection #20

Open
bushev opened this issue Dec 17, 2016 · 5 comments · May be fixed by #22
Open

Error while backup large collection #20

bushev opened this issue Dec 17, 2016 · 5 comments · May be fixed by #22
Assignees
@hex7c0
Labels
bug

Comments

@bushev
Copy link

bushev commented Dec 17, 2016

Hello!

It seems this issue still not resolved: #16

Node: v7.2.1
mongodb-backup: v1.6.8

{ Error: Cannot find module 'tar'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at /Users/bushev/IdeaProjects/proj/node_modules/mongodb-backup/index.min.js:195:42
    at /Users/bushev/IdeaProjects/proj/node_modules/mongodb-backup/index.min.js:22:20
    at /Users/bushev/IdeaProjects/proj/node_modules/mongodb-backup/node_modules/graceful-fs/polyfills.js:287:18
    at FSReqWrap.oncomplete (fs.js:111:15) code: 'MODULE_NOT_FOUND' }
fs.js:557
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open '/Users/bushev/IdeaProjects/proj/logs/error.log'
    at Object.fs.openSync (fs.js:557:18)
    at Object.fs.writeFileSync (fs.js:1222:33)
    at Object.fs.appendFileSync (fs.js:1268:6)
    at process.on.err (/Users/bushev/IdeaProjects/proj/queue-server.js:50:19)
    at emitOne (events.js:96:13)
    at process.emit (events.js:188:7)
    at process._fatalException (bootstrap_node.js:292:26)

Also, Q: Why we need to create a separate file for each collection item?
@hex7c0
Copy link
Owner

hex7c0 commented Jan 13, 2017

hi @bushev,
sorry for delay!

err is about Cannot find module 'tar', where is the relationship between large collection?

@bushev
Copy link
Author

bushev commented Jan 13, 2017

Hi @hex7c0!

The root cause of "Cannot find module 'tar'" is a require call fail. It fails due to EMFILE, which originated by writing a lot of separated files (instances on large collection).

@hex7c0
Copy link
Owner

hex7c0 commented Jan 30, 2017

sure, sorry for this issue

I'm working on it

@hex7c0 hex7c0 added the bug label Jan 30, 2017
@hex7c0 hex7c0 self-assigned this May 14, 2017
@spmiller spmiller linked a pull request Jun 8, 2017 that will close this issue
Open
@crh3675
Copy link

crh3675 commented Apr 2, 2018 •
edited
Loading

We had the same EMFILE issue but even more so, this module completely crashed our server. We have 6 CPUs and they pegged to 299% and then completely obliterated all other PM2 processes and corrupted the PM2 processes file.

We had to empty out the PM2 processes file and reboot the server - pretty awful experience. We definitely are going to look for an alternative unless this can be looked at.

@Zwimber
Copy link

Zwimber commented May 30, 2018

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hex7c0 hex7c0

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Write tar files to a stream instead of to the filesystem first spmiller/mongodb-backup
4 participants
@crh3675 @hex7c0 @Zwimber @bushev