Skip to content

peteralbert/prisma-bulkcreate-test

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is an SSCEE for (this issue)[prisma/prisma#3835]. It is based one Prisma's example

Problem

In our production schema, we have multiple nested entries (think of spreadsheet with cells, comments, etc.). When a new spreadsheet is created, for multiple reasons we need to initialize/create a lot of cells, comments, etc. As a consequence, prisma.sheet.create() leads to >1,000 (sometimes 10,000!) records that need to be created in different tables. This single statement takes 15-20seconds!

Suggested solution

Under the hood the Prisma Engine creates one INSERT statement for each record, i.e. a lot of statements must be executed - and as the records are related, this must be done sequentially as the parentIDs are required for the next statement. So if for instance 1 parent with 2 children and 3 child-childs respectively need to be created, 9 INSERTs must be executed:

  1. parent
  2. child1
  3. child-child 1.1
  4. child-child 1.2
  5. child-child 1.3
  6. child2
  7. child-child 2.1
  8. child-child 2.2
  9. child-child 2.3

Instead, running one INSERT with multiple VALUES will reduce the number of INSERTstatements and improve performance a lot when dealing with multiple entries. In the above example, there would be 3 inserts:

  1. parent
  2. child1, child2
  3. child-child 1.1, child-child 1.2, child-child 1.3, child-child 2.1, child-child 2.2, child-child 2.3

Demo

I create a small sample repo that reproduces this and measures the execution time for both approaches based on the number of records. While the Prisma approach in this specific example is faster for about 500 records, the manual approach is relatively faster the more records there are:

  • 10 records: Prisma: 21ms, Bulk: 87ms
  • 100 records: Prisma: 33ms, Bulk: 85ms
  • 1,000 records: Prisma: 132ms, Bulk: 87ms
  • 10,000 records: Prisma: 1,041ms, Bulk: 173ms
  • 100,000 records: Prisma: 10,313ms, Bulk: 893ms

The demo uses SQLLite, but we have the same issue with PostgreSQL in production.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published