Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Option to register runners on organization instead of repo #451

Merged
merged 59 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
455beeb
refactor(app): add initial setup for runner level registration
Oct 16, 2023
f070815
refactor(app): append runnerLevel to registration request
Oct 16, 2023
50de4cc
refactor(infra): add new secret for runner registration level
Oct 16, 2023
7c6f65f
refactor(runner): add runner registration level secret to setup function
Oct 16, 2023
135650e
refactor(setup): add runnerLevel property for existingApp and when pa…
Oct 16, 2023
552c8f6
refactor(tokenRetriever): add condition to get a registration token f…
Oct 16, 2023
1d45fc6
refactor(statemachine): inject runnerLevel value into stepfunctions
Oct 16, 2023
222ff3c
refactor(ec2): inject runnerLevel into ec2 userdata
Oct 16, 2023
5006bf0
Merge branch 'main' into ref/442-org-runner-registration
Oct 17, 2023
347ed27
refactor(app): add condition to only show registration level for new …
Oct 17, 2023
2bab850
docs(setup_github): update instructions
Oct 17, 2023
b415192
refactor(app): set repositoryPermissions as default value in manifest
Oct 17, 2023
46d64d6
refactor(setup): update runnerLevel when newapp is created
pharindoko Oct 17, 2023
d1d0ed2
refactor(token-retriever): update lambda function
Oct 18, 2023
4227dbb
docs(API): add runnerLevel
Oct 18, 2023
3b43573
refactor(ec2): update parameter injection in linuxUserDataTemplate
Oct 18, 2023
424f1b4
refactor: create own secret for runner level configuration
Oct 18, 2023
7dc223b
refactor(token-retriever): restructure code for handler
Oct 18, 2023
0386317
Merge branch 'main' into ref/442-org-runner-registration
pharindoko Oct 19, 2023
6d8a1dd
refactor(ec2): update windows ec2 runner to support organization leve…
Oct 19, 2023
94e726f
refactor(codebuild): update config script to register runner base d o…
Oct 20, 2023
1b4c652
refactor(lambda-runner): update config script to register runner base…
Oct 20, 2023
aad9f73
refactor(fargate-runner): update config script to register runner bas…
Oct 20, 2023
0f6bc94
refactor(ecs-runner): update config script to register runner based o…
Oct 20, 2023
1752dc6
refactor(token-retriever): sort imports
Oct 20, 2023
2f56695
refactor(providers): update github agent registration scripts
Oct 20, 2023
74a6023
refactor(token-retriever): determine registration url for agent
Oct 24, 2023
130d3ec
refactor(ec2): update windows userdata script
Oct 24, 2023
502725d
refactor(fargate): update environment variable
Oct 24, 2023
41aa9ec
Merge branch 'main' into ref/442-org-runner-registration
pharindoko Oct 25, 2023
1f76b17
refactor(app): require to set runnerLevel for existing app
Oct 25, 2023
338363c
refactor(app): remove runnerLevel property from pat request
Oct 25, 2023
e43cd44
docs(api): update properties in api.md
Oct 25, 2023
8cdc31d
test(integration): update snapshots
Oct 25, 2023
d2613ca
eslint
kichik Oct 26, 2023
87cd04b
Undo some formatting for easier diff
kichik Oct 26, 2023
1f064b1
Merge branch 'main' into ref/442-org-runner-registration
pharindoko Oct 26, 2023
db2e5a8
Don't override default runner level value
kichik Oct 26, 2023
714bc9e
test(integration): update snapshot
Oct 27, 2023
f40110a
Cleanup
kichik Oct 27, 2023
e72ab4e
Prepare for making runner level optional
kichik Oct 27, 2023
2c657b4
Update docs
kichik Oct 28, 2023
531abb5
Set default value
kichik Oct 28, 2023
b65264a
Check for bad runner levels
kichik Oct 28, 2023
5415a63
Add some more permission details
kichik Oct 28, 2023
933f9ee
Update snapshot
kichik Oct 28, 2023
c9eeacd
Consolidate secrets
kichik Oct 28, 2023
27294e8
Mention runner level in secret
kichik Oct 28, 2023
c033e63
Update API.md
kichik Oct 29, 2023
dcc9662
runnerLevel can be undefined
kichik Oct 29, 2023
1539b3f
"resolve" exception todo
kichik Oct 29, 2023
6d17a6e
cleaner diff by removing prettier formatting
kichik Oct 29, 2023
777b05a
undo escaping that's not needed
kichik Oct 29, 2023
200b473
some more minor revisions
kichik Oct 29, 2023
e4fb882
all permissions
kichik Oct 29, 2023
0802ec3
don't override runnerLevel on callback
kichik Oct 29, 2023
25ecd61
handle deleting runners + mention in status
kichik Oct 29, 2023
8783c1b
be explicit about pat runner level
kichik Oct 30, 2023
26f605b
Merge remote-tracking branch 'origin/main' into ref/442-org-runner-re…
kichik Oct 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 60 additions & 6 deletions setup/src/App.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,34 @@
let domain = 'INSERT_DOMAIN_HERE';
let auth: undefined | 'newApp' | 'existingApp' | 'pat';
let appScope: 'user' | 'org' = 'user';
let runnerLevel: 'repo' | 'org';
let org = 'ORGANIZATION';
let existingAppId: string = '';
let existingAppPk: string = '';
let pat: string = '';
let success: boolean;
let result: string | undefined;

const repositoryPermissions = {
actions: 'write',
administration: 'write',
deployments: 'read',
};

const organizationPermissions = {
actions: 'write',
organization_self_hosted_runners: 'write',
deployments: 'read',
};

const manifest = {
url: 'https://github.com/CloudSnorkel/cdk-github-runners',
hook_attributes: {
url: 'INSERT_WEBHOOK_URL_HERE',
},
redirect_url: 'INSERT_BASE_URL_HERE/complete-new-app',
public: false,
default_permissions: {
actions: 'write',
administration: 'write',
deployments: 'read',
},
default_permissions: repositoryPermissions || organizationPermissions,
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
default_events: [
'workflow_job',
],
Expand Down Expand Up @@ -89,9 +98,10 @@

function promise(): Promise<string> {
const rightDomain = instance === 'ghes' ? domain : 'github.com';
manifest.default_permissions = runnerLevel === 'repo' ? repositoryPermissions : organizationPermissions;
switch (auth) {
case 'newApp':
return postJson('domain', { domain: rightDomain })
return postJson('domain', { domain: rightDomain, runnerLevel })
.then(_ => {
(document.getElementById('appform') as HTMLFormElement).submit();
return Promise.resolve('Redirecting to GitHub...');
Expand All @@ -101,11 +111,13 @@
appid: existingAppId,
pk: existingAppPk,
domain: rightDomain,
runnerLevel,
});
case 'pat':
return postJson('pat', {
pat: pat,
domain: rightDomain,
runnerLevel,
});
}
}
Expand Down Expand Up @@ -261,6 +273,48 @@
</div>
{/if}


<h3>Choose Registration Level</h3>
<div class="px-3 py-3">
<p>
You can configure the github app to register the runners at the
repository or organization level. <br>
The main difference are the
permissions assigned to the github app.<br>
</p>
<ul>
<li>
Repository level: the github app will
have full <b>administrative</b> access to the selected
repositories.
</li>
<li>
Organization level: the github app will have only the "self-hosted
github runner" permission to register runners in the organization. (Recommended for organizations)
</li>
</ul>
<div class="form-check">
<input
class="form-check-input"
type="radio"
bind:group={runnerLevel}
value="repo"
id="repo"
/>
<label class="form-check-label" for="repo">Repository</label>
</div>
<div class="form-check">
<input
class="form-check-input"
type="radio"
bind:group={runnerLevel}
value="org"
id="org"
/>
<label class="form-check-label" for="org">Organization</label>
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
</div>
</div>

<h2>Finish Setup</h2>
<div class="px-3 py-3">
{#if result === undefined}
Expand Down
1 change: 1 addition & 0 deletions src/lambda-github.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export interface GitHubSecrets {
domain: string;
appId: number;
personalAuthToken: string;
runnerLevel: string;
}

const octokitCache: {
Expand Down
6 changes: 6 additions & 0 deletions src/providers/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,12 @@ export interface RunnerRuntimeParameters {
* Path to repository name.
*/
readonly repoPath: string;

/**
* Level to register runner at. Can be either 'repo' or 'org'.
*/
readonly runnerLevel: string;

}

/**
Expand Down
55 changes: 48 additions & 7 deletions src/providers/ec2.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,51 @@ EOF
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/tmp/log.conf || exit 2
}
action () {
if [ "$(< RUNNER_VERSION)" = "latest" ]; then RUNNER_FLAGS=""; else RUNNER_FLAGS="--disableupdate"; fi
sudo -Hu runner /home/runner/config.sh --unattended --url "https://{}/{}/{}" --token "{}" --ephemeral --work _work --labels "{},cdkghr:started:\`date +%s\`" $RUNNER_FLAGS --name "{}" || exit 1
# Define variables for injected parameters
taskToken="$1"
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
logGroupName="$2"
runnerNamePath="$3"
githubDomainPath="$4"
ownerPath="$5"
repoPath="$6"
runnerTokenPath="$7"
labels="$8"
runnerNamePath2="$9"
labels2="${10}"
runnerLevel="${11}"

# Determine the value of RUNNER_FLAGS
if [ "$(< RUNNER_VERSION)" = "latest" ]; then
RUNNER_FLAGS=""
else
RUNNER_FLAGS="--disableupdate"
fi

# Construct the URL and labels templates
url="https/$githubDomainPath/$ownerPath/$repoPath"
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
labelsTemplate="$labels,cdkghr:started:$(date +%s)"

# Define the registration URL based on runnerLevel
if [ "$runnerLevel" = "org" ]; then
registrationURL="https://$githubDomainPath/$ownerPath"
elif [ "$runnerLevel" = "repo" ]; then
registrationURL="https://$githubDomainPath/$ownerPath/$repoPath"
else
echo "Invalid runnerLevel: $runnerLevel"
exit 1
fi

# Execute the configuration command for runner registration
sudo -Hu runner /home/runner/config.sh --unattended --url "$registrationURL" --token "$runnerTokenPath" --ephemeral --work _work --labels "$labelsTemplate" $RUNNER_FLAGS --name "$runnerNamePath" || exit 1

# Execute the run command
sudo --preserve-env=AWS_REGION -Hu runner /home/runner/run.sh || exit 2
STATUS=$(grep -Phors "finish job request for job [0-9a-f\\\\-]+ with result: \\\\K.*" /home/runner/_diag/ | tail -n1)
[ -n "$STATUS" ] && echo CDKGHA JOB DONE "{}" "$STATUS"

# Retrieve the status
STATUS=$(grep -Phors "finish job request for job [0-9a-f\\-]+ with result: \K.*" /home/runner/_diag/ | tail -n1)

# Check and print the job status
[ -n "$STATUS" ] && echo CDKGHA JOB DONE "$labels2" "$STATUS"
}
heartbeat &
if setup_logs && action | tee /var/log/runner.log 2>&1; then
Expand Down Expand Up @@ -108,7 +148,7 @@ function setup_logs () {
}
function action () {
cd /actions
$RunnerVersion = Get-Content RUNNER_VERSION -Raw
$RunnerVersion = Get-Content RUNNER_VERSION -Raw
if ($RunnerVersion -eq "latest") { $RunnerFlags = "" } else { $RunnerFlags = "--disableupdate" }
./config.cmd --unattended --url "https://{}/{}/{}" --token "{}" --ephemeral --work _work --labels "{},cdkghr:started:$(Get-Date -UFormat +%s)" $RunnerFlags --name "{}" 2>&1 | Out-File -Encoding ASCII -Append /actions/runner.log
if ($LASTEXITCODE -ne 0) { return 1 }
Expand Down Expand Up @@ -366,6 +406,7 @@ export class Ec2RunnerProvider extends BaseProvider implements IRunnerProvider {
this.labels.join(','),
parameters.runnerNamePath,
this.labels.join(','),
parameters.runnerLevel,
];

const passUserData = new stepfunctions.Pass(this, `${this.labels.join(', ')} data`, {
Expand All @@ -390,7 +431,7 @@ export class Ec2RunnerProvider extends BaseProvider implements IRunnerProvider {
const rootDeviceResource = amiRootDevice(this, this.ami.launchTemplate.launchTemplateId);
rootDeviceResource.node.addDependency(this.amiBuilder);
const subnetRunners = this.subnets.map((subnet, index) => {
return new stepfunctions_tasks.CallAwsService(this, `${this.labels.join(', ')} subnet${index+1}`, {
return new stepfunctions_tasks.CallAwsService(this, `${this.labels.join(', ')} subnet${index + 1}`, {
comment: subnet.subnetId,
integrationPattern: IntegrationPattern.WAIT_FOR_TASK_TOKEN,
service: 'ec2',
Expand Down Expand Up @@ -442,7 +483,7 @@ export class Ec2RunnerProvider extends BaseProvider implements IRunnerProvider {

// chain up the rest of the subnets
for (let i = 1; i < subnetRunners.length; i++) {
subnetRunners[i-1].addCatch(subnetRunners[i], {
subnetRunners[i - 1].addCatch(subnetRunners[i], {
errors: ['Ec2.Ec2Exception', 'States.Timeout'],
resultPath: stepfunctions.JsonPath.stringAt('$.lastSubnetError'),
});
Expand Down
3 changes: 2 additions & 1 deletion src/runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ export class GitHubRunners extends Construct implements ec2.IConnectable {
private readonly webhook: GithubWebhookHandler;
private readonly orchestrator: stepfunctions.StateMachine;
private readonly setupUrl: string;
private readonly extraLambdaEnv: {[p: string]: string} = {};
private readonly extraLambdaEnv: { [p: string]: string } = {};
private readonly extraLambdaProps: lambda.FunctionOptions;
private stateMachineLogGroup?: logs.LogGroup;
private jobsCompletedMetricFilters?: logs.MetricFilter[];
Expand Down Expand Up @@ -369,6 +369,7 @@ export class GitHubRunners extends Construct implements ec2.IConnectable {
githubDomainPath: stepfunctions.JsonPath.stringAt('$.runner.domain'),
ownerPath: stepfunctions.JsonPath.stringAt('$.owner'),
repoPath: stepfunctions.JsonPath.stringAt('$.repo'),
runnerLevel: stepfunctions.JsonPath.stringAt('$.runner.runnerLevel'),
},
);
providerChooser.when(
Expand Down
1 change: 1 addition & 0 deletions src/secrets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ export class Secrets extends Construct {
domain: 'github.com',
appId: '',
personalAuthToken: '',
runnerLevel: 'repo',
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
}),
generateStringKey: 'dummy',
includeSpace: false,
Expand Down
25 changes: 17 additions & 8 deletions src/setup.lambda.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,14 @@ async function handleDomain(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewa
if (!body.domain) {
return response(400, 'Invalid domain');
}
if (!body.runnerLevel) {
pharindoko marked this conversation as resolved.
Show resolved Hide resolved
return response(400, 'Invalid runner regisration level');
}

const githubSecrets: GitHubSecrets = await getSecretJsonValue(process.env.GITHUB_SECRET_ARN);
githubSecrets.domain = body.domain;
githubSecrets.runnerLevel = body.runnerLevel;
await updateSecretValue(process.env.GITHUB_SECRET_ARN, JSON.stringify(githubSecrets));

return response(200, 'Domain set');
}

Expand All @@ -68,26 +71,31 @@ async function handlePat(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewayPr
if (!body.pat || !body.domain) {
return response(400, 'Invalid personal access token');
}
if (!body.runnerLevel) {
return response(400, 'Invalid runner regisration level');
}

await updateSecretValue(process.env.GITHUB_SECRET_ARN, JSON.stringify(<GitHubSecrets>{
domain: body.domain,
appId: -1,
personalAuthToken: body.pat,
runnerLevel: body.runnerLevel,

}));
await updateSecretValue(process.env.SETUP_SECRET_ARN, JSON.stringify({ token: '' }));

return response( 200, 'Personal access token set');
return response(200, 'Personal access token set');
}

async function handleNewApp(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewayProxyResultV2> {
if (!event.queryStringParameters) {
return response( 400, 'Invalid code');
return response(400, 'Invalid code');
}

const code = event.queryStringParameters.code;

if (!code) {
return response( 400, 'Invalid code');
return response(400, 'Invalid code');
}

const githubSecrets: GitHubSecrets = await getSecretJsonValue(process.env.GITHUB_SECRET_ARN);
Expand All @@ -105,25 +113,26 @@ async function handleNewApp(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewa
}));
await updateSecretValue(process.env.SETUP_SECRET_ARN, JSON.stringify({ token: '' }));

return response( 200, `New app set. <a href="${newApp.data.html_url}/installations/new">Install it</a> for your repositories.`);
return response(200, `New app set. <a href="${newApp.data.html_url}/installations/new">Install it</a> for your repositories.`);
}

async function handleExistingApp(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewayProxyResultV2> {
const body = decodeBody(event);

if (!body.appid || !body.pk || !body.domain) {
return response( 400, 'Missing fields');
if (!body.appid || !body.pk || !body.domain || !body.runnerLevel) {
return response(400, 'Missing fields');
}

await updateSecretValue(process.env.GITHUB_SECRET_ARN, JSON.stringify(<GitHubSecrets>{
domain: body.domain,
appId: body.appid,
personalAuthToken: '',
runnerLevel: body.runnerLevel,
}));
await updateSecretValue(process.env.GITHUB_PRIVATE_KEY_SECRET_ARN, body.pk as string);
await updateSecretValue(process.env.SETUP_SECRET_ARN, JSON.stringify({ token: '' }));

return response( 200, 'Existing app set. Don\'t forget to set up the webhook.');
return response(200, 'Existing app set. Don\'t forget to set up the webhook.');
}

export async function handler(event: ApiGatewayEvent): Promise<AWSLambda.APIGatewayProxyResultV2> {
Expand Down
20 changes: 14 additions & 6 deletions src/token-retriever.lambda.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { RequestError } from '@octokit/request-error';
import { getOctokit } from './lambda-github';
import { GitHubSecrets, getOctokit } from './lambda-github';
import { StepFunctionLambdaInput } from './lambda-helpers';

class RunnerTokenError extends Error {
Expand All @@ -18,14 +18,22 @@ export async function handler(event: StepFunctionLambdaInput) {
octokit,
} = await getOctokit(event.installationId);

const response = await octokit.rest.actions.createRegistrationTokenForRepo({
owner: event.owner,
repo: event.repo,
});
let response;
if ((githubSecrets as GitHubSecrets).runnerLevel === 'repo') {
response = await octokit.rest.actions.createRegistrationTokenForRepo({
owner: event.owner,
repo: event.repo,
});
} else if ((githubSecrets as GitHubSecrets).runnerLevel === 'org') {
response = await octokit.rest.actions.createRegistrationTokenForOrg({
org: event.owner,
});
}

return {
domain: githubSecrets.domain,
token: response.data.token,
runnerLevel: githubSecrets.runnerLevel,
token: response!.data.token,
};
} catch (error) {
console.error(error);
Expand Down