jest unit test : mongodb connection fails on docker (CI) pipeline, works well on local

[sagargadekar601]

Versions

• NodeJS: 12.20.1
• mongodb-memory-server-*: 6.9.3
• mongodb: 3.6.2
• mongoose: 5.11.13
• system: Windows 10

package: mongo-memory-server

What is your question?

"mongodb" connection fails on docker (CI) pipeline, however it works well on local. Below is the error it shows when pipeline fails. It should create in-memory mongodb server and execute the unit tests on that. I tried searching similar issue, but I didn't find anything apart from this. I did the suggestions provided in that issue but it didn't help. Can someone please shed some light on this? I am baddly stuck due to this.

vue-cli-service test:unit --expand --no-cache --coverage
  console.warn node_modules/mongodb-memory-server-core/lib/MongoMemoryServer.js:143
    Starting the instance failed, please enable debug for more infomation
  console.error __tests__/routes/ser-data.spec.js:40
    MongoDB connection FAIL
2021-02-14T18:06:15.049Z bff-ser:ser-data:error Insert ser info MongooseError: Operation `ser_collection.insertOne()` buffering timed out after 10000ms
2021-02-14T18:06:15.056Z bff-ser:ser-data:error Get SER data MongooseError: Operation `ser_collection.find()` buffering timed out after 10000ms
  ●  Cannot log after tests are done. Did you forget to wait for something async in your test?
    Attempted to log "response.statusCode : 500".
      45 |     var bodyData = mockData.ser_test_SingleData;
      46 |     var response = await request.post(`/ser/ser-data/`).send(bodyData);
    > 47 |     console.log("response.statusCode : " + response.statusCode);
         |             ^
      48 |     expect(response.statusCode).toBe(200);
      49 |     expect(response.body.name).toBe("SERProject1");
      50 |   });
      at console.log (node_modules/@vue/cli-plugin-unit-jest/node_modules/@jest/console/build/CustomConsole.js:183:10)
      at Object.<anonymous> (__tests__/routes/ser-data.spec.js:47:13)
FAIL __tests__/routes/ser-data.spec.js (12.462s)
  bff-ser
    ✕ bff-ser /ser/ser-data/ POST query (11ms)
    ✕ get SERs (10052ms)
  ● bff-ser › bff-ser /ser/ser-data/ POST query
    spawn /builds/mainstream-eng-cloud/bff-ser/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.14/mongod ENOENT
  ● bff-ser › get SERs
    expected "Content-Type" matching /json/, got "text/plain; charset=utf-8"
      at Test.Object.<anonymous>.Test._assertHeader (node_modules/supertest/lib/test.js:245:14)

Ran all test suites.
  console.warn node_modules/@sentry/node/dist/integrations/onunhandledrejection.js:63
    This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
  console.error node_modules/@sentry/node/dist/integrations/onunhandledrejection.js:64
    Error: Caught error after test environment was torn down
    
    expect(received).toBe(expected) // Object.is equality
    
    Expected: 200
    Received: 500

Below is my Code :

process.env.NODE_ENV = "test";
const app = require("../../app"); // Link to your server file
const supertest = require("supertest");
const request = supertest(app);
const mongoose = require("mongoose");
const mockData = require("../helpers/data.json");
const { MongoMemoryServer } = require("mongodb-memory-server");

const mongod = new MongoMemoryServer();
//jasmine.DEFAULT_TIMEOUT_INTERVAL = 600000;

describe("cff-ser", () => {
  beforeAll(async () => {
    try {
      jest.setTimeout(30 * 1000);
      const uri = await mongod.getUri();

      const mongooseOpts = {
        useNewUrlParser: true,
        useUnifiedTopology: true
      };

      if (mongoose.connection.name != "") await mongoose.connection.close();

      mongoose.connect(uri, mongooseOpts);

      mongoose.Promise = global.Promise;
      var db = mongoose.connection;

      db.on(
        "error",
        console.error.bind(console, "DocumentDB connection error:")
      );

      db.once("open", function () {
        // we're connected!
        console.log("db object" + db);
      });
    } catch (error) {
      console.error("MongoDB connection FAIL");
    }
  });

  it("cff-ser /ser/ser-data/ POST query", async () => {
    var bodyData = mockData.ser_test_SingleData;
    var response = await request.post(`/ser/ser-data/`).send(bodyData);
    console.log("response.statusCode : " + response.statusCode);
    expect(response.statusCode).toBe(200);
    expect(response.body.name).toBe("serProject1");
  });

  
  afterAll(async () => {
    await mongoose.connection.dropDatabase();
    await mongoose.connection.close();
    await mongoose.disconnect();
    await mongod.stop();
  });
});

[hasezoey]

console.warn node_modules/mongodb-memory-server-core/lib/MongoMemoryServer.js:143
Starting the instance failed, please enable debug for more infomation

as this says, please enable & post the debug output, read here how to enable it

---

if this is an docker problem, please provide what image is used

---

did you already try mongodb 4.0+? and did you already try the beta of mongodb-memory-server?

[sagargadekar601]

@hasezoey Appreciate your quick response. I have enabled the debug and below is the output. [Error I can see is : MongoMS:MongoInstance Mongo[37559]: MongodbInstance: Instance has failed]

[email protected] test:ci /builds/mainstream-eng-cloud/cff-ser
> vue-cli-service test:unit --expand --no-cache --coverage
2021-02-15T04:32:16.910Z MongoMS:MongoMemoryServer Called MongoMemoryServer.ensureInstance() method
2021-02-15T04:32:16.911Z MongoMS:MongoMemoryServer  - no running instance, call `start()` command
2021-02-15T04:32:16.911Z MongoMS:MongoMemoryServer Called MongoMemoryServer.start() method
2021-02-15T04:32:16.914Z MongoMS:MongoMemoryServer starting with port 37559, since undefined was locked: 37559
2021-02-15T04:32:16.915Z MongoMS:MongoMemoryServer Starting MongoDB instance with following options: {"port":37559,"dbName":"ad7172ca-e707-466d-861f-80f53093f141","ip":"127.0.0.1","storageEngine":"ephemeralForTest","dbPath":"/tmp/mongo-mem--486-iCKzVzXkJKoB","tmpDir":{"name":"/tmp/mongo-mem--486-iCKzVzXkJKoB"},"uri":"mongodb://127.0.0.1:37559/ad7172ca-e707-466d-861f-80f53093f141?"}
2021-02-15T04:32:16.917Z MongoMS:MongoBinary MongoBinary options: {
  "downloadDir": "/builds/mainstream-eng-cloud/cff-ser/node_modules/.cache/mongodb-memory-server/mongodb-binaries",
  "platform": "linux",
  "arch": "x64",
  "version": "4.0.14",
  "checkMD5": false
}
2021-02-15T04:32:16.921Z MongoMS:MongoBinary MongoBinary: Download lock removed
2021-02-15T04:32:16.921Z MongoMS:MongoBinary MongoBinary: Mongod binary path: "/builds/mainstream-eng-cloud/cff-ser/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.14/mongod"
2021-02-15T04:32:16.951Z MongoMS:MongoInstance Mongo[37559]: MongodbInstance: Instance has failed: Error: spawn /builds/mainstream-eng-cloud/cff-ser/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.14/mongod ENOENT
2021-02-15T04:32:16.957Z MongoMS:MongoInstance Mongo[37559]: Mongod instance closed with an non-0 code!
2021-02-15T04:32:16.957Z MongoMS:MongoInstance Mongo[37559]: CLOSE: -2
2021-02-15T04:32:16.957Z MongoMS:MongoInstance Mongo[37559]: MongodbInstance: Instance has failed: Mongod instance closed with code "-2"
  console.error __tests__/routes/ser-data.spec.js:40
    MongoDB connection FAIL

[sagargadekar601]

Not sure if this is a docker issue. I will get and provide the details on the image used.

I haven't tried with mongodb 4.0+, neither with mongo-memory-server beta.

[hasezoey]

did you already try mongodb 4.0+?

I haven't tried with mongodb 4.0+

sorry if it wasnt clear from the template & me, i meant the binary used, from the log i could see the "used" binary is 4.0.14 (package default)

Error I can see is : MongoMS:MongoInstance Mongo[37559]: MongodbInstance: Instance has failed]

no, this is not the error, this is just to know that it failed not what failed, from the log i could see that the error is 2021-02-15T04:32:16.951Z MongoMS:MongoInstance Mongo[37559]: MongodbInstance: Instance has failed: Error: spawn /builds/mainstream-eng-cloud/cff-ser/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.14/mongod ENOENT, meaning the binary that it wants to use does not exist, for this issue, please provide the environment (docker image) used, and try the beta (has some fixes & re-writes of the download code)

[sagargadekar601]

my docker file :

FROM node:12-alpine

RUN apk add --no-cache tini
# Tini is now available at /sbin/tini
ENTRYPOINT ["/sbin/tini", "--"]

USER node

ENV NODE_ENV=production

# update npm
RUN npm install -g npm

# copy source files into the 'service' directory.
RUN mkdir -p /home/node/service
WORKDIR /home/node/service
COPY audit-ci.json ./

RUN ls /home/node/service/

# temporarily install global dependencies and run audit-ci on them.
RUN mkdir -p /home/node/global-npm-check
WORKDIR /home/node/global-npm-check
RUN npm init -y && \
  npm install --no-audit forever && \
  npx audit-ci --config /home/node/service/audit-ci.json
RUN rm -rf global-npm-check

# install global dependencies
RUN npm install -g forever

WORKDIR /home/node/service

# copy the package file and run `npm ci`
COPY package.json package-lock.json ./
RUN npm ci --only=production && npx audit-ci --config /home/node/service/audit-ci.json

# copy the rest of the source files
COPY .env.example app.js swaggerDef.js rds-combined-ca-bundle.certs ./
COPY bin/ ./bin
COPY public/ ./public
COPY lib/ ./lib
COPY routes/ ./routes
COPY views/ ./views
COPY dist/ ./dist
COPY model/ ./model
COPY services/ ./services

ENV PORT=3000
EXPOSE 3000

RUN ls

CMD ["forever", "--fifo", "--minUptime=1000", "--spinSleepTime=15000", "./bin/www"]


# build with:
# docker build -t cff-ser .
# run with:
# docker run -it --rm -p 3000:3000 --env-file .env cff-ser

[hasezoey]

instead of +++ please use code blocks, read at Mastering-Markdown on how to do it

---

i can see from this file that your are using alpine, this system dosnt have an native build by mongodb, as stated in the readme (or the new documentation of supported systems)

tracking issue is #347

[sagargadekar601]

Thanks @hasezoey .. Do you have any idea of any node:apline docker image (>12) which has mongodb in it?
Or do you see any other way out (i.e. adding "apk" run command in docker file etc.)?

[hasezoey]

there are some possibilities:

• find an image with node & alpine & mongodb binary
• create one yourself (like testing some binaries that work, and include them in your image)
• use an non-alpine node image (like the debian stretch or buster versions)

if you should find an working image, please report back

PS: you can try #347 (comment) , i have not verified this myself yet, but it seems to be an working binary
(or this may not be related to alpine at all)

[sagargadekar601]

Thanks @hasezoey . I will check on this and revert back..

[hasezoey]

Duplicate of #347