Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev: check for env vars at startup #230

Merged
merged 37 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
e7d427c
Make sure env vars are set
kalilsn Nov 30, 2023
3c12feb
Prevent use of process.env
kalilsn Nov 30, 2023
59d19e5
Fix missing key lint errors
kalilsn Nov 30, 2023
02535d1
Use .env.development for non-secret values
kalilsn Nov 30, 2023
5d4af69
merge with main
tefkah Feb 14, 2024
9e3f261
feat: use @t3-oss/env for env validation instead of homegrown solution
tefkah Feb 14, 2024
b642b45
chore: remove @t3-oss/env-core
tefkah Feb 14, 2024
551f9d5
fix: use correct babel config for eslint
tefkah Feb 14, 2024
e60e397
refactor: use new env everywhere
tefkah Feb 14, 2024
b41ab27
Merge branch 'main' into tfk/config
tefkah Feb 15, 2024
220dbd4
chore: make build
tefkah Feb 15, 2024
42d66dd
chore: simplify .env.template for core
tefkah Feb 19, 2024
8c47a85
chore: remove .env.templates and replace with .env.developments
tefkah Feb 19, 2024
dc80bea
feat: use global config directory
tefkah Feb 19, 2024
dbeda89
chore: upgrade turbo to prevent weird issue with concurrency
tefkah Feb 19, 2024
4843747
chore: remove config/env, was not working with preconstruct
tefkah Feb 20, 2024
5d471f4
feat: add config directory for tsconfig
tefkah Feb 20, 2024
ec4afde
feat: use new config/tsconfig location settings
tefkah Feb 20, 2024
1f8c78e
feat: add lib/env.mjs for all nextjs apps
tefkah Feb 20, 2024
4f0461f
chore: update dependencies, remove unused plugins, and adjust dev env…
tefkah Feb 20, 2024
e07566b
refactor: use new env everywhere
tefkah Feb 20, 2024
5d65ca3
chore: update pnpm-lock
tefkah Feb 20, 2024
106dc03
chore: remove console.log
tefkah Feb 20, 2024
e9659c4
Merge branch 'main' into tfk/config
tefkah Feb 20, 2024
7b540ab
fix: don't check on build, incompatible with docker setup
tefkah Feb 20, 2024
44d526c
fix: do not check env vars during CI
tefkah Feb 21, 2024
5687898
fix: remove bad import and test ci var
tefkah Feb 21, 2024
83764df
fix: set CI true
tefkah Feb 21, 2024
7f98fc9
fix: set env var in packagejson instead
tefkah Feb 21, 2024
02d6ef3
fix: fail on startup, not on page load
tefkah Feb 21, 2024
1ee32fd
fix: import correct path for integrations
tefkah Feb 21, 2024
9b323e5
chore: merge with main
tefkah Feb 27, 2024
a8720aa
dev: use tsx to run scripts instead of ts-node
tefkah Feb 27, 2024
6dc1542
fix: make NODE_ENV optional in zod schema; correct import paths in sc…
tefkah Feb 27, 2024
51a8b73
fix: change dotenv loading order to actually work
tefkah Feb 27, 2024
89ec60a
Merge branch 'main' into tfk/config
tefkah Feb 29, 2024
d204a93
chore: merge with main
tefkah Mar 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion tsconfig/base.json → config/tsconfig/base.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
"noUnusedParameters": false,
"preserveWatchOutput": true,
"skipLibCheck": true,
"strict": true
"strict": true,
"incremental": true
},
"exclude": ["node_modules"]
}
3 changes: 2 additions & 1 deletion tsconfig/nextjs.json → config/tsconfig/nextjs.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"downlevelIteration": true
"downlevelIteration": true,
"moduleResolution": "Bundler"
},
"include": ["src", "next-env.d.ts"],
"exclude": ["node_modules"]
Expand Down
File renamed without changes.
File renamed without changes.
14 changes: 14 additions & 0 deletions core/.env.development
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
MAILGUN_SMTP_HOST=localhost
MAILGUN_SMTP_PORT=54325
API_KEY="super_secret_key"
DATABASE_URL="postgresql://postgres:postgres@localhost:54322/postgres"
NEXT_PUBLIC_SUPABASE_URL="http://localhost:54321"
NEXT_PUBLIC_PUBPUB_URL="http://localhost:3000"
ASSETS_BUCKET_NAME="assets.v7.pubpub.org"
ASSETS_REGION="us-east-1"

SUPABASE_WEBHOOKS_API_KEY="xxx"
ASSETS_UPLOAD_KEY="xxx"
ASSETS_UPLOAD_SECRET_KEY="xxx"
MAILGUN_SMTP_PASSWORD="xxx"
MAILGUN_SMTP_USERNAME="xxx"
12 changes: 0 additions & 12 deletions core/.env.template
Original file line number Diff line number Diff line change
@@ -1,16 +1,4 @@
## Dev
API_KEY="super_secret_key"
DATABASE_URL="postgresql://postgres:postgres@localhost:54322/postgres"
NEXT_PUBLIC_SUPABASE_URL="http://localhost:54321"
MAILGUN_SMTP_USERNAME=
MAILGUN_SMTP_PASSWORD=
ASSETS_UPLOAD_KEY=""
ASSETS_UPLOAD_SECRET_KEY=""
ASSETS_BUCKET_NAME="assets.v7.pubpub.org"
ASSETS_REGION="us-east-1"
## ------
## These values are copied from the output of `supabase start`
NEXT_PUBLIC_SUPABASE_PUBLIC_KEY=""
SUPABASE_SERVICE_ROLE_KEY=""
JWT_SECRET=""
## ------
1 change: 1 addition & 0 deletions core/app/(user)/settings/SettingsForm.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ export default function SettingsForm({
<Link
href={`/c/${community.slug}`}
className="cursor-pointer hover:bg-gray-50"
key={community.id}
>
<Button variant="outline">
<div className="flex items-center">
Expand Down
85 changes: 46 additions & 39 deletions core/app/api/supabase-webhooks/user/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { PrismaClientKnownRequestError } from "@prisma/client/runtime/library";
import { captureException } from "@sentry/nextjs";
import { NextRequest, NextResponse } from "next/server";
import { compareAPIKeys, getBearerToken } from "~/lib/auth/api";
import { env } from "~/lib/env/env.mjs";
import { BadRequestError, UnauthorizedError, handleErrors } from "~/lib/server/errors";
import prisma from "~/prisma/db";

Expand All @@ -10,46 +11,52 @@ import prisma from "~/prisma/db";
// user when they change and confirm their email.
// For debugging, the responses sent by this handler are stored in supabase under net._http_response
export async function POST(req: NextRequest) {
return await handleErrors(async () => {
const serverKey = process.env.SUPABASE_WEBHOOKS_API_KEY!
const authHeader = req.headers.get('authorization')
if (!authHeader) {
throw new UnauthorizedError("Authorization header missing")
}
compareAPIKeys(getBearerToken(authHeader), serverKey)
return await handleErrors(async () => {
const serverKey = env.SUPABASE_WEBHOOKS_API_KEY;
const authHeader = req.headers.get("authorization");
if (!authHeader) {
throw new UnauthorizedError("Authorization header missing");
}
compareAPIKeys(getBearerToken(authHeader), serverKey);

const body = await req.json();
if (!body.record || !body.old_record) {
console.error("unexpected webhook payload:", body)
throw new BadRequestError("Unexpected webhook payload")
}
const body = await req.json();
if (!body.record || !body.old_record) {
console.error("unexpected webhook payload:", body);
throw new BadRequestError("Unexpected webhook payload");
}

const oldEmail = body.old_record.email
const newEmail = body.record.email
const oldEmail = body.old_record.email;
const newEmail = body.record.email;

if (newEmail && newEmail !== oldEmail) {
try {
await prisma.user.update({
where: {
supabaseId: body.record.id
},
data: {
email: newEmail
}
});
} catch (error) {
if (error instanceof PrismaClientKnownRequestError) {
if (error.code === "P2002") { // Unique constraint violated (email already exists)
const newErr = new BadRequestError(`User changed supabase email from ${oldEmail} to ${newEmail} but another account exists for ${newEmail}`)
captureException(newErr)
throw newErr
}
}
throw error
}
return NextResponse.json({ message: `User ${body.record.id} updated email to ${body.record.email}`}, { status: 200 });
}
if (newEmail && newEmail !== oldEmail) {
try {
await prisma.user.update({
where: {
supabaseId: body.record.id,
},
data: {
email: newEmail,
},
});
} catch (error) {
if (error instanceof PrismaClientKnownRequestError) {
if (error.code === "P2002") {
// Unique constraint violated (email already exists)
const newErr = new BadRequestError(
`User changed supabase email from ${oldEmail} to ${newEmail} but another account exists for ${newEmail}`
);
captureException(newErr);
throw newErr;
}
}
throw error;
}
return NextResponse.json(
{ message: `User ${body.record.id} updated email to ${body.record.email}` },
{ status: 200 }
);
}

return NextResponse.json({ message: "No action taken" }, { status: 200 });
})
};
return NextResponse.json({ message: "No action taken" }, { status: 200 });
});
}
11 changes: 6 additions & 5 deletions core/app/api/user/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { generateHash, getSlugSuffix, slugifyString } from "lib/string";
import { getSupabaseId } from "lib/auth/loginId";
import { BadRequestError, ForbiddenError, UnauthorizedError, handleErrors } from "~/lib/server";
import { captureException } from "@sentry/nextjs";
import { env } from "~/lib/env/env.mjs";

export type UserPostBody = {
firstName: string;
Expand Down Expand Up @@ -40,12 +41,12 @@ export async function POST(req: NextRequest) {
email,
password,
options: {
emailRedirectTo: `${process.env.NEXT_PUBLIC_PUBPUB_URL}/login`,
emailRedirectTo: `${env.NEXT_PUBLIC_PUBPUB_URL}/login`,
data: {
firstName,
lastName,
canAdmin: true
}
canAdmin: true,
},
},
});
/* Supabase returns:
Expand Down Expand Up @@ -75,7 +76,7 @@ export async function POST(req: NextRequest) {
}

if (existingUser) {
// TODO: create community membership here, update name, slug etc.
// TODO: create community membership here, update name, slug etc.
// await prisma.user.update({
// where: {
// email,
Expand All @@ -97,7 +98,7 @@ export async function POST(req: NextRequest) {
email,
},
});
return NextResponse.json({message: "New user created"}, { status: 201 });
return NextResponse.json({ message: "New user created" }, { status: 201 });
}
});
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ const IntegrationList: React.FC<Props> = function ({ instances, token }) {
<div className="mt-4">
{instance.pubs.map((pub) => {
return (
<div className="text-sm">
<div key={pub.id} className="text-sm">
Attached to pub:{" "}
<span className="font-bold">{getTitle(pub)}</span>
</div>
Expand Down
2 changes: 1 addition & 1 deletion core/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export default async function Page() {
redirect("/login");
}
} else {
redirect("/login")
redirect("/login");
}
return <>Home...</>;
}
79 changes: 40 additions & 39 deletions core/lib/auth/loginId.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,14 @@ import jwt from "jsonwebtoken";
import { getRefreshCookie, getTokenCookie } from "~/lib/auth/cookies";
import { getServerSupabase } from "~/lib/supabaseServer";
import type { UserAppMetadata, UserMetadata } from "@supabase/supabase-js";

const JWT_SECRET: string = process.env.JWT_SECRET || "";
import { env } from "../env/env.mjs";

/* This is only called from API calls */
/* When rendering server components, use getLoginData from loginData.ts */
export async function getSupabaseId(req: NextRequest): Promise<string> {
const sessionJWT = getTokenCookie(req);
if (!sessionJWT) {
return ""
return "";
}
const refreshToken = getRefreshCookie(req);
return await getSupabaseIdFromJWT(sessionJWT, refreshToken);
Expand All @@ -23,51 +22,53 @@ export async function getSupabaseIdFromJWT(sessionJWT?: string, refreshToken?: s
return "";
}

const user = await getUserInfoFromJWT(sessionJWT, refreshToken)
const user = await getUserInfoFromJWT(sessionJWT, refreshToken);

if (!user?.id) {
return ""
return "";
}

return user.id;
}

type jwtUser = {
id: string,
email?: string,
aud: string,
app_metadata: UserAppMetadata,
user_metadata: UserMetadata,
role?: string,
}
id: string;
email?: string;
aud: string;
app_metadata: UserAppMetadata;
user_metadata: UserMetadata;
role?: string;
};

export async function getUserInfoFromJWT(sessionJWT: string, refreshToken?: string): Promise<jwtUser | null> {
try {
const decoded = await jwt.verify(sessionJWT, JWT_SECRET);
if (typeof decoded === 'string' || !decoded.sub) {
throw new Error('Invalid jwt payload')
}
// TODO: actually validate the JWT payload!
// Rename `sub` to `id` for a consistent return type with the User that the
// refreshSession method below returns
return { id: decoded.sub, ...decoded} as jwtUser
export async function getUserInfoFromJWT(
sessionJWT: string,
refreshToken?: string
): Promise<jwtUser | null> {
try {
const decoded = await jwt.verify(sessionJWT, env.JWT_SECRET);
if (typeof decoded === "string" || !decoded.sub) {
throw new Error("Invalid jwt payload");
}
// TODO: actually validate the JWT payload!
// Rename `sub` to `id` for a consistent return type with the User that the
// refreshSession method below returns
return { id: decoded.sub, ...decoded } as jwtUser;
} catch (jwtError) {
console.error("Error verifying jwt", jwtError);
/* We may get a jwtError if it has expired. In which case, */
/* we try to use the refreshToken to sign back in before */
/* waiting for the client to that after initial page load. */
const supabase = getServerSupabase();
const { data, error } = await supabase.auth.refreshSession({
refresh_token: refreshToken || "",
});
if (error) {
console.error("Error refreshing session:", error.message);
return null;
}
catch (jwtError) {
console.error("Error verifying jwt", jwtError);
/* We may get a jwtError if it has expired. In which case, */
/* we try to use the refreshToken to sign back in before */
/* waiting for the client to that after initial page load. */
const supabase = getServerSupabase();
const { data, error } = await supabase.auth.refreshSession({
refresh_token: refreshToken || "",
});
if (error) {
console.error("Error refreshing session:", error.message)
return null;
}
if (!data.user?.id) {
return null;
}
return data.user;
if (!data.user?.id) {
return null;
}
return data.user;
}
}
38 changes: 38 additions & 0 deletions core/lib/env/env.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// @ts-check

import { createEnv } from "@t3-oss/env-nextjs";
import { z } from "zod";

export const env = createEnv({
shared: {
NODE_ENV: z.enum(["development", "production", "test"]).optional(),
},
server: {
API_KEY: z.string(),
ASSETS_BUCKET_NAME: z.string(),
ASSETS_REGION: z.string(),
ASSETS_UPLOAD_KEY: z.string(),
ASSETS_UPLOAD_SECRET_KEY: z.string(),
DATABASE_URL: z.string().url(),
JWT_SECRET: z.string(),
MAILGUN_SMTP_PASSWORD: z.string(),
MAILGUN_SMTP_USERNAME: z.string(),
SUPABASE_SERVICE_ROLE_KEY: z.string(),
SUPABASE_WEBHOOKS_API_KEY: z.string(),
MAILGUN_SMTP_HOST: z.string(),
MAILGUN_SMTP_PORT: z.string(),
},
client: {
NEXT_PUBLIC_PUBPUB_URL: z.string().url(),
NEXT_PUBLIC_SUPABASE_PUBLIC_KEY: z.string(),
NEXT_PUBLIC_SUPABASE_URL: z.string().url(),
},
experimental__runtimeEnv: {
NODE_ENV: process.env.NODE_ENV,
NEXT_PUBLIC_PUBPUB_URL: process.env.NEXT_PUBLIC_PUBPUB_URL,
NEXT_PUBLIC_SUPABASE_PUBLIC_KEY: process.env.NEXT_PUBLIC_SUPABASE_PUBLIC_KEY,
NEXT_PUBLIC_SUPABASE_URL: process.env.NEXT_PUBLIC_SUPABASE_URL,
},
skipValidation: Boolean(process.env.SKIP_VALIDATION),
emptyStringAsUndefined: true,
});
13 changes: 6 additions & 7 deletions core/lib/server/assets.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { env } from "../env/env.mjs";

export const generateSignedAssetUploadUrl = async (pubId: string, fileName: string) => {
const region = process.env.ASSETS_REGION;
const key = process.env.ASSETS_UPLOAD_KEY;
const secret = process.env.ASSETS_UPLOAD_SECRET_KEY;
const bucket = process.env.ASSETS_BUCKET_NAME;
if (!region || !key || !secret || !bucket) {
throw new Error("Missing assets upload paramters");
}
const region = env.ASSETS_REGION;
const key = env.ASSETS_UPLOAD_KEY;
const secret = env.ASSETS_UPLOAD_SECRET_KEY;
const bucket = env.ASSETS_BUCKET_NAME;

const client = new S3Client({
region: region,
credentials: {
Expand Down
Loading