Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add script to get npm package data #499

Merged
merged 38 commits into from
Dec 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9fff772
Add zod schemas for CI validation
lachlancollins Nov 5, 2023
14f7624
Require npm field for components.json
lachlancollins Nov 6, 2023
1e2edc5
Remove svelte-layout-resizable
lachlancollins Nov 6, 2023
f97d23f
Stricter Zod validation
lachlancollins Nov 6, 2023
9c5242a
Stricter regex
lachlancollins Nov 7, 2023
5d4c9f3
Stricter repository field validation
lachlancollins Nov 12, 2023
c668c8f
Merge branch 'main' into require-npm-field
lachlancollins Nov 12, 2023
54cee2a
Implement requested changes
lachlancollins Nov 12, 2023
8386ee2
Add back accidentally removed field
lachlancollins Nov 12, 2023
6cf2e8b
Move SvelteStore to templates.json
lachlancollins Nov 12, 2023
b593067
Update category and tags
lachlancollins Nov 12, 2023
77d7f68
Merge branch 'main' into require-npm-field
lachlancollins Nov 13, 2023
0c9a44b
Merge branch 'main' into require-npm-field
lachlancollins Nov 14, 2023
ac4f44a
Add script to get npm data
lachlancollins Nov 14, 2023
30880ac
Merge branch 'main' into update-npm-data
lachlancollins Nov 22, 2023
96116de
Merge branch 'main' into update-npm-data
lachlancollins Nov 26, 2023
20591ca
Re-run updateNpm.js
lachlancollins Nov 26, 2023
9c8a894
Merge branch 'main' into update-npm-data
lachlancollins Nov 27, 2023
c349b57
Implement initial feedback
lachlancollins Nov 27, 2023
64ab391
Update npm.js
lachlancollins Nov 29, 2023
d6589a4
Switch async/await to then/catch
lachlancollins Dec 5, 2023
50630f3
Fix prettier
lachlancollins Dec 5, 2023
0c00771
Run script
lachlancollins Dec 5, 2023
1620e73
Merge branch 'main' into update-npm-data
lachlancollins Dec 10, 2023
1bdfd41
Merge branch 'main' into update-npm-data
lachlancollins Dec 17, 2023
2e46d12
Re-run updateNpm
lachlancollins Dec 17, 2023
77a078a
Also read tools.json
lachlancollins Dec 17, 2023
0aff9bb
Add @types/node
lachlancollins Dec 17, 2023
2379a37
Restructure npm.json output
lachlancollins Dec 17, 2023
2f7d1bb
Display last update on components page
lachlancollins Dec 17, 2023
6934616
Add to weekly workflow
lachlancollins Dec 17, 2023
8972dd6
Clarify updating npm data
lachlancollins Dec 17, 2023
9094497
Merge branch 'main' into update-npm-data
lachlancollins Dec 17, 2023
0f4ea68
Update src/lib/utils/injectNpmData.ts
lachlancollins Dec 17, 2023
2813b31
Smaller date font, add version
lachlancollins Dec 17, 2023
0d6202f
Improve error text
lachlancollins Dec 17, 2023
37131b4
Fix property title
lachlancollins Dec 17, 2023
4853733
Merge branch 'main' into update-npm-data
lachlancollins Dec 17, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Update stars count
name: Fetch latest data

on:
schedule:
Expand All @@ -19,10 +19,12 @@ jobs:
cache: 'pnpm'
- name: Install dependencies
run: pnpm install
- name: Run script
- name: Update stars
uses: ./.github/actions/update-stars
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Update npm data
run: node scripts/updateNpm.js
- name: Run format
run: pnpm run format
- name: Create Pull Request
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"@types/node": "^20.10.4",
"@typescript-eslint/eslint-plugin": "^6.14.0",
"@typescript-eslint/parser": "^6.14.0",
"dayjs": "^1.11.10",
"eslint": "^8.55.0",
"eslint-config-prettier": "^9.1.0",
"eslint-plugin-svelte": "^2.35.1",
Expand Down
7 changes: 7 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions scripts/updateNpm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// @ts-check

import { writeFileSync } from 'node:fs';
import { promisify } from 'node:util';
import { exec } from 'node:child_process';
import { componentsSchema, toolsSchema } from '../src/lib/schemas.js';
import components from '../src/routes/components/components.json' assert { type: 'json' };
import tools from '../src/routes/tools/tools.json' assert { type: 'json' };

const execAsync = promisify(exec);

const data = [...componentsSchema.parse(components), ...toolsSchema.parse(tools)];

const npm = await Promise.all(
data.map((pkg) => processPackage(pkg).catch((error) => console.log(error.message)))
).then((values) => {
return values.reduce((result, value) => Object.assign(result, value), {});
});

writeFileSync('src/lib/npm.json', JSON.stringify(npm));

/** @param {ReturnType<typeof data>[0]} pkg */
async function processPackage(pkg) {
if (!pkg.npm) {
throw new Error(`npm field missing from ${pkg.title} (skipping)`);
}
const { stdout } = await execAsync(`npm view ${pkg.npm} --json`);
const data = JSON.parse(stdout.toString());
const version = data.version;
const date = data.time[version];
const support = data.peerDependencies?.svelte ? data.peerDependencies.svelte : 'Unknown';
return { [pkg.npm]: { version: version, date: date, support: support } };
}
65 changes: 41 additions & 24 deletions src/lib/components/ComponentIndex/Card.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import Tag from '../Tag.svelte';
import { copyToClipboard } from '$lib/utils/clipboard';
import { packageManager as manager } from '$stores/packageManager';
import { relativeDate } from '$utils/relativeDate';

export let active = false;
export let title = '';
Expand All @@ -11,6 +12,8 @@
export let url = '';
export let npm = '';
export let repository = undefined;
export let date = undefined;
export let version = undefined;

let clipboardCopy = false;

Expand All @@ -33,44 +36,51 @@
</script>

<div class="card" class:active id="component-{title}">
<h3>
<a href="#component-{title}">#</a>
{#if url || repository}<a href={url || repository}>{title}</a>{:else}<span>{title}</span>{/if}
{#if npm}<Tag
click={() => copy()}
variant="copy"
title={clipboardCopy ? 'copied!' : `${packageManagers[$manager]} ${cleanupNpm(npm)}`}
/>{/if}
</h3>
<p class="flex-grow">{description}</p>
{#if tags}
<div class="card__tags">
{#each tags as tag}
<Tag title={tag} variant="blue" />
{/each}
<div class="card__top">
<div>
<h3>
<a href="#component-{title}">#</a>
{#if url || repository}<a href={url || repository}>{title}</a>{:else}<span>{title}</span
>{/if}
</h3>
</div>
{/if}
<div class="card__bottom">
<div>
{#if (repository || url || '').includes('github')}
<a title="Go to the source code" href={repository || url}
<a class="repo" title="Go to the source code" href={repository || url}
><img style="display:inline" src="/images/github_logo.svg" alt="github logo" /></a
>
{:else if (repository || url || '').includes('gitlab')}
<a title="Go to the source code" href={repository || url}
<a class="repo" title="Go to the source code" href={repository || url}
><img style="display:inline" src="/images/gitlab_logo.svg" alt="gitlab logo" /></a
>
<!-- {:else} -->
{/if}
</div>
</div>

{#if npm}
<Tag
click={() => copy()}
variant="copy"
title={clipboardCopy ? 'copied!' : `${packageManagers[$manager]} ${cleanupNpm(npm)}`}
/>
{/if}
<p class="flex-grow">{description}</p>
{#if tags}
<div class="card__tags">
{#each tags as tag}
<Tag title={tag} variant="blue" />
{/each}
</div>
{/if}
<div class="card__bottom">
<div>
{#if typeof stars !== 'undefined'}
&#9733;
<code>{stars}</code>
{/if}
</div>
<!-- commenting out dates just cause it is not very updated yet - all the cards show same date. put back in when we have better scraping -->
<!-- <datetime value={addedOn}>{new Intl.DateTimeFormat('en-Us').format(Date.parse(addedOn))}</datetime> -->
{#if date && version}<span class="date">Updated {relativeDate(date)} ({version})</span>{/if}
</div>
</div>

Expand All @@ -97,6 +107,11 @@
flex-wrap: wrap;
margin-bottom: 1rem;
}
.card__top {
display: flex;
justify-content: space-between;
align-items: top;
}
.card__bottom {
display: flex;
justify-content: space-between;
Expand All @@ -105,7 +120,7 @@
.card__bottom > * {
white-space: nowrap;
}
.card__bottom a {
.repo {
border-bottom: none;
aspect-ratio: 1/1;
display: flex;
Expand All @@ -117,10 +132,12 @@
background-color: rgba(0, 0, 0, 0);
transition: background-color 200ms ease-out;
}
.card__bottom a:hover {
.repo:hover {
background-color: rgba(0, 0, 0, 0.25);
}

.date {
font-size: 14px;
}
.flex-grow {
flex-grow: 1;
}
Expand Down
Loading