Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split migrations into separte bundle #29

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

echocrow
Copy link

Fixes #26

src/migrations.ts Outdated Show resolved Hide resolved
Copy link
Author

@echocrow echocrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some code comments for point-of-interest & suggestions

additionally:
I believe it's somewhat common practice to have the src folder resemble the exports structure. could achieve this by moving all files except index.ts and migrations.ts into a new subdirectory, e.g. src/lib. optional and skipped for now, but figured worth calling out, now that there's more than one export.
(edit: above note was made redundant with the switch to the src/migrations subdir)

package.json Outdated
},
"./migrations": {
"import": "./dist/esm/migrations.js",
"require": "./dist/cjs/migrations.js"
Copy link
Author

@echocrow echocrow Apr 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternative suggestion:
other than file paths, "import" and "require" can also be objects, containing "types" and "default".
could leverage those declarations here, which might make dist-fix.js obsolete?

(edit - forgot to link to) docs:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! 🤔

I think we'd still want to alter how tsup outputs and have a folder for each still. What are the downsides?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have a folder for each still

intuitively I'd agree. (read: me wanting to organize and put different things into different boxes)

at the same time, that requirement may be obsolete?

  • these days .mjs file extensions seem widely supported.
  • the specs allow for arbitrary paths via "exports" & co-located type def files with matching extensions. wanting an "esm"/"cjs" folder seems like custom build decision that doesn't really impact lib users(?)
  • I'd trust tools like tsup to know better than my own intuition; there's likely a reason outputting separate folders is non-default and moved behind the aptly named --legacy-output flag. ¯\_(ツ)_/¯

What are the downsides?

omitting --legacy-output would generate ESM files using .mjs file extensions. I haven't worked with any bundler/environment in some time that does not understand that extension. theoretically some (archaic?) environment might not understand .mjs, but for that to be relevant they'd also need to somehow explicitely want to load the .mjs file over the co-located CJS .js file.

(Hypothesis: Could be that this was more relevant at a time when ESM was still competing with CJS and IIFE. Back then, I don't think we had extensions to differentiate between those, or they weren't as well established yet; and we certainly did not have "exports" in package.json back then.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Personally I'd even be in favor of leaving CJS in the dust. But I understand that, for now, it may still be controversial forcing that decision as lib author.)

Copy link
Member

@igalklebanov igalklebanov Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's ditch legacy output flag. See if anyone complains. Worst case, we'll revert back.

Copy link
Author

@echocrow echocrow Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See if anyone complains.

nice - Test-In-Prod is my middle name 👋 changes pushed! (edit: re-pushed; missed to update the cjs imports in the knexfile)

I've made sure to keep the /// <reference .../> injection (now also widened to the CSJ files).
not fully sure how to handle the chunk files (or they need any handling) for deno, so I've omitted them for now. if problematic, can also config tsup to not output chunks I believe.

while I was at it, I've also moved the dist-fix script to ESM in a separate comit - hope that's cool? no real reason other than cLeAnUp. happy to revert if CJS is preferred.

tests/nodejs/migrations.test.ts Outdated Show resolved Hide resolved
@@ -12,7 +13,7 @@ import {
for (const dialect of SUPPORTED_DIALECTS) {
describe(`KyselyFsMigrationSource: ${dialect}`, () => {
let ctx: TestContext
let migrationSource: KyselyFsMigrationSource
let migrationSource: FsMigrations
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this change necessary?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary per-se, and independent of the core of this PR. it's a separate commit before the actual one that splits the bundle; figured if I'm already here, might as well clean up sth on my way out.


I noticed that TS was not happy with the KyselyFsMigrationSource type assignment here, complaining that:

let migrationSource: KyselyFsMigrationSource
                     ~~~~~~~~~~~~~~~~~~~~~~~
'KyselyFsMigrationSource' refers to a value, but is being used as a type here. Did you mean 'typeof KyselyFsMigrationSource'?

However, switching to typeof KyselyFsMigrationSource causes even more TS problems.

Using InstanceType<typeof KyselyFsMigrationSource> is more accurate with the later assignment, but then TS still complains about this type not perfectly matching what knex.migrate.rollback expects.

Falling back to knex's own FsMigrations type aligns with the variable assignment, and with with knex itself later expects.

...just one of the cases were I do sympathize with those who hate TS and its type juggling. 😅

Copy link
Member

@igalklebanov igalklebanov Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Let's go with InstanceType<typeof KyselyFsMigrationSource>. The build step turned the exported class declaration into var x = class.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Let's go with InstanceType.

roger, pushing this. as per my earlier post, this still causes TS complain in two places:

await ctx.knex.migrate.latest({migrationSource})
                               ~~~~~~~~~~~~~~~
Type 'KyselyFsMigrationSource' is missing the following properties from type 'MigrationSource<unknown>': getMigrations, getMigrationNamets(2739)

but these type errors should be safe to ignore.

The build step turned the exported class declaration into var x = class.

can you clarify (if worth it)? the ESM output should not be relevant to the TS warning, and this TS type declaration here should have no impact on any build or test output.

scripts/dist-fix.js Outdated Show resolved Hide resolved
echocrow added a commit to echocrow/kysely-knex-split-migrations that referenced this pull request Apr 22, 2024
@echocrow echocrow changed the title Split migrations in separte bundle Split migrations into separte bundle Apr 22, 2024
echocrow added a commit to echocrow/kysely-knex-split-migrations that referenced this pull request Apr 22, 2024
@echocrow
Copy link
Author

echocrow commented Apr 26, 2024

looks like node 18 is failing while node 20 is fine. import.meta.dirname is a node 20+ thing, which is likely what's causing the node 18 tests to fail during build. (somewhat of a false positive for tests; tests should still pass on node 18, but build requring 20+ is now getting in the way.)
probably not worth making the test pipeline more complex, i'll just revert my last ESM-switch commit and keep the dist-fix build script in CJS.

edit: last commit removed; dist-fix is back to CSJ. also rebased onto main while I was at it.

scripts/dist-fix.js Outdated Show resolved Hide resolved
scripts/dist-fix.js Outdated Show resolved Hide resolved
await unlink(distEsmFilePath)
const dtsFilePath = filename.replace(/\.(m?)js$/, '.d.$1ts')
const denoFriendlyJsFileContents =
`/// <reference types="${dtsFilePath}" />\n` +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a minimal deno test @ tests/deno to verify we're building this stuff correctly..
see https://github.com/kysely-org/kysely/tree/master/test/deno for reference (just local is fine).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

disclaimer: never used deno, so very new to both it and this kind of deno testing. 😅 tried to roughly replicate the setup from the kysely repo with a new bare-bones test. not sure if this aligns with what you had in mind though(?)

I also don't think this setup actually tests our TS-only /// <reference ...> injection at all. If I alter the reference, the deno script still completes fine. So this might only somewhat tests if the bundled runtime code is deno-compatible.

@igalklebanov
Copy link
Member

igalklebanov commented Apr 29, 2024

Let's add https://www.npmjs.com/package/@arethetypeswrong/cli to package.json#scripts (attw --pack . this works for me in a different project) and CI. It verifies we're exporting CJS/ESM correctly.

@echocrow
Copy link
Author

echocrow commented May 2, 2024

Let's add https://www.npmjs.com/package/@arethetypeswrong/cli to package.json#scripts (attw --pack . this works for me in a different project) and CI. It verifies we're exporting CJS/ESM correctly.

neato! heard of the package a while ago; looking forward to seeing it in action myself.
I think integrating it for this package might a bit overkill (small package & exports; we're mostly testing tsup's output?), but it's also only one more package/script/command.

I should have some time this weekend to get back to this 🤞

@igalklebanov
Copy link
Member

igalklebanov commented May 2, 2024

Let's add npmjs.com/package/@arethetypeswrong/cli to package.json#scripts (attw --pack . this works for me in a different project) and CI. It verifies we're exporting CJS/ESM correctly.

neato! heard of the package a while ago; looking forward to seeing it in action myself. I think integrating it for this package might a bit overkill (small package & exports; we're mostly testing tsup's output?), but it's also only one more package/script/command.

I should have some time this weekend to get back to this 🤞

we're testing the package.json configuration is aligned with tsup, and nothing's missing.

@echocrow
Copy link
Author

echocrow commented May 2, 2024

we're testing the package.json configuration is aligned with tsup, and nothing's missing.

hence the "mostly" in my post 😅 had I feeling I'd get called out on this

echocrow added 2 commits May 4, 2024 19:24
subpath export currently fails for node10
@echocrow
Copy link
Author

echocrow commented May 4, 2024

@igalklebanov attw added to scripts! available via test:types.

I was hoping I could add this to the combined test script. however, attw validation currently fails for node10 (but passes everywhere else), and thus errors with exit code 1.
node10 resolution fails because package.json#exports support was added later. The attw docs call out this issue
Unsure if you care about node10 support. I skimmed other examples that illustrate setups for subpath export, but couldn't get them to pass attw's node10 check either. (I also could not find a way to ignore node10 without blanked-ignoring resolution failures.)

thoughts?

@igalklebanov
Copy link
Member

igalklebanov commented Jun 12, 2024

Sorry for delay. Life and other efforts.

Check out this project for reference. non-legacy tsup with 2 entry points, a single export condition (I don't think it matters), attw is passing in prepublishOnly and a GitHub workflow.

@echocrow
Copy link
Author

Sorry for delay. Life and other efforts.

no need to apologize! but same here ATM. not sure when I'll get back to this; hopefully in the next few weeks.

cheers for the reference! I'll check check out that project first thing when looping back to this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Minor bundle regression: Un-treeshakable "FsMigrations" import from knex
2 participants