Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/dev' into stable
Browse files Browse the repository at this point in the history
  • Loading branch information
@adrians5j
adrians5j committed Aug 31, 2023
2 parents 59d3771 + e631694 commit c454fc0
Show file tree
Hide file tree
Showing 28 changed files with 763 additions and 357 deletions.
9 changes: 9 additions & 0 deletions docs/DEPLOY_WEBINY_PROJECT_CF_TEMPLATE.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,15 @@ Resources:
- arn:aws:iam::*:role/wby-*
- arn:aws:iam::*:policy/wby-*

# AWS Identity and Access Management (IAM) - Service-Linked Roles
# Only needed for the "Amazon DynamoDB + Amazon Elasticsearch" database setup.
# https://www.webiny.com/docs/architecture/introduction#different-database-setups
- Effect: Allow
Action:
- iam:CreateServiceLinkedRole
Resource:
- arn:aws:iam::*:role/aws-service-role/es.amazonaws.com/AWSServiceRoleForAmazonElasticsearchService

# AWS Lambda
- Effect: Allow
Action:
Expand Down
60 changes: 26 additions & 34 deletions packages/cli-plugin-deploy-pulumi/commands/executeMigrations.js
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
const readline = require("readline");
const LambdaClient = require("aws-sdk/clients/lambda");
const { getStackOutput } = require("../utils");
const { runMigration, printReport, getDuration } = require("@webiny/data-migration/cli");

const clearLine = () => {
if (process.stdout.isTTY) {
readline.clearLine(process.stdout, 0);
readline.cursorTo(process.stdout, 0);
}
};
const {
MigrationRunner,
InteractiveCliStatusReporter,
NonInteractiveCliStatusReporter,
LogReporter,
CliMigrationRunReporter
} = require("@webiny/data-migration/cli");

module.exports = async (params, context) => {
const apiOutput = getStackOutput({ folder: "apps/api", env: params.env });
Expand All @@ -20,35 +18,29 @@ module.exports = async (params, context) => {
region: apiOutput.region
});

const response = await runMigration({
const functionName = apiOutput["migrationLambdaArn"];

const logReporter = new LogReporter(functionName);
const statusReporter =
!process.stdout.isTTY || "CI" in process.env
? new NonInteractiveCliStatusReporter(logReporter)
: new InteractiveCliStatusReporter(logReporter);

const runner = MigrationRunner.create({
lambdaClient,
functionName: apiOutput["migrationLambdaArn"],
payload: {
version: process.env.WEBINY_VERSION || context.version,
pattern: params.pattern
},
statusCallback: ({ status, migrations }) => {
clearLine();
if (status === "running") {
const currentMigration = migrations.find(mig => mig.status === "running");
if (currentMigration) {
const duration = getDuration(currentMigration.startedOn);
process.stdout.write(
`Running data migration ${currentMigration.id} (${duration})...`
);
}
return;
}

if (status === "init") {
process.stdout.write(`Checking data migrations...`);
}
}
functionName,
statusReporter
});

clearLine();
const result = await runner.runMigration({
version: process.env.WEBINY_VERSION || context.version,
pattern: params.pattern
});

printReport({ response, context, migrationLambdaArn: apiOutput["migrationLambdaArn"] });
if (result) {
const reporter = new CliMigrationRunReporter(logReporter, context);
await reporter.report(result);
}
} catch (e) {
context.error(`An error occurred while trying to execute data migration Lambda function!`);
console.log(e);
Expand Down
18 changes: 0 additions & 18 deletions packages/cli-plugin-deploy-pulumi/commands/index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -315,24 +315,6 @@ module.exports = [
process.exit(0);
}
);

yargs.command(
"get-migration-status",
`Get data migrations Lambda status.`,
() => {
yargs.example("$0 get-migration-status --env dev");

yargs.option("env", {
describe: `Environment`,
type: "string",
required: true
});
},
async argv => {
await require("./printMigrationStatus")(argv, context);
process.exit(0);
}
);
}
}
];
31 changes: 0 additions & 31 deletions packages/cli-plugin-deploy-pulumi/commands/printMigrationStatus.js
View file

This file was deleted.

80 changes: 49 additions & 31 deletions packages/create-webiny-project/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,27 +12,33 @@ A tool for setting up a new Webiny project.
#### Simple:

```
npx create-webiny-project@beta my-test-project --tag beta
npx create-webiny-project@local-npm my-test-project --tag local-npm
```

#### Advanced:

```
npx create-webiny-project@beta my-test-project
--tag beta --no-interactive
npx create-webiny-project@local-npm my-test-project
--tag local-npm --no-interactive
--assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}'
--template-options '{"region":"eu-central-1","vpc":false}'
```

This usage is more ideal for CI/CD environments, where interactivity is not available.

But do note that this is probably more useful to us, Webiny developers, than for actual Webiny projects. This is simply because in real project's CI/CD pipelines, users would simply start off by cloning the project from their private repository, and not create a new one with the above command.
But do note that this is probably more useful to us, Webiny developers, than for actual Webiny projects. This is simply
because in real project's CI/CD pipelines, users would simply start off by cloning the project from their private
repository, and not create a new one with the above command.

## Development Notes

Testing this, and related packages (like [cwp-template-aws](./../cwp-template-aws)) is a bit complicated, because in order to get the best results, it's recommended to test everything with packages published to a real NPM.
Testing this, and related packages (like [cwp-template-aws](./../cwp-template-aws)) is a bit complicated, because in
order to get the best results, it's recommended to test everything with packages published to a real NPM.

But of course, publishing to NPM just to test something is not ideal, and that's why, we use [Verdaccio](https://verdaccio.org/) instead, which is, basically, an NPM-like service you can run locally. So, instead of publishing packages to NPM, you publish them to Verdaccio, which is much cleaner, because everything stays on your laptop.
But of course, publishing to NPM just to test something is not ideal, and that's why, we
use [Verdaccio](https://verdaccio.org/) instead, which is, basically, an NPM-like service you can run locally. So,
instead of publishing packages to NPM, you publish them to Verdaccio, which is much cleaner, because everything stays on
your laptop.

#### Usage

Expand All @@ -42,35 +48,42 @@ The following steps show how to do it.

#### 1. Start Verdaccio

Start by running the `yarn verdaccio:start` command, which will, as the script name itself suggests, spin up Verdaccio locally.
Start by running the `yarn verdaccio:start` command, which will, as the script name itself suggests, spin up Verdaccio
locally.

> All of the files uploaded to Verdaccio service will be stored in the `.verdaccio` folder, located in your project root.
> All of the files uploaded to Verdaccio service will be stored in the `.verdaccio` folder, located in your project
> root.
#### 2. Set default NPM registry

Once you have Verdaccio up and running, you'll also need to change the default NPM registry. Meaning, when you run `npx create-webiny-project ...`, you want it to start fetching packages from Verdaccio, not real NPM. Verdaccio runs on localhost, on port 4873, so, in your terminal, run the following command:
Once you have Verdaccio up and running, you'll also need to change the default NPM registry. Meaning, when you
run `npx create-webiny-project ...`, you want it to start fetching packages from Verdaccio, not real NPM. Verdaccio runs
on localhost, on port 4873, so, in your terminal, run the following command:

```
npm config set registry http://localhost:4873
```

Note that this will only help you with `npx`, but won't help you when a new project foundation is created, and the dependencies start to get pulled. This is because we're using yarn2, which actually doesn't respect the values that were written by the `npm config set ...` command we just executed.
Note that this will only help you with `npx`, but won't help you when a new project foundation is created, and the
dependencies start to get pulled. This is because we're using yarn2, which actually doesn't respect the values that were
written by the `npm config set ...` command we just executed.

It's super important that, when you're testing your npx project, you also pass the following argument:

```
--assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}'
```

This will set the necessary values in yarn2 config file, which will be located in your newly created project. But don't worry about it right now, this will be revisited in step 4.
This will set the necessary values in yarn2 config file, which will be located in your newly created project. But don't
worry about it right now, this will be revisited in step 4.

> Yarn2 projects don't rely on global configurations and is not installed globally, but on per-project basis. This allows having multiple versions of yarn2, for different projects.
> Yarn2 projects don't rely on global configurations and is not installed globally, but on per-project basis. This
> allows having multiple versions of yarn2, for different projects.
#### 3. Release

Commit (no need to push it if you don't want to) all of the code changes, and execute the following command:


```bash
yarn release --type=verdaccio
```
Expand All @@ -80,7 +93,7 @@ yarn release --type=verdaccio
Test your changes with the following command:

```
npx create-webiny-project@next my-test-project --tag next --assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}'
npx create-webiny-project@local-npm my-test-project --tag local-npm --assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}'
```

This should create a project, with all of the packages pulled from Verdaccio.
Expand All @@ -94,36 +107,41 @@ Once you're done, do the following:

### Commands Cheat Sheet

| Description | Command |
|-----------------------------------------------| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Remove `.verdaccio` folder | `rm -rf .verdaccio` |
| List all v5\* tags | `git tag -l "v5*"` |
| Remove specific tag | `git tag -d "v5.0.0-next.5"` |
| Set Verdaccio as the NPM registry | `npm config set registry http://localhost:4873` |
| Reset NPM registry | `npm config set registry https://registry.npmjs.org/` |
| Start Verdaccio | `yarn verdaccio:start` |
| Release to Verdaccio | `yarn release --type=verdaccio` |
| Create a new Webiny project | `npx create-webiny-project@next my-test-project --tag next --assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}'` |
| Revert versioning commit | `git reset HEAD~ && git reset --hard HEAD` |
| Description | Command |
|-----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Remove `.verdaccio` folder | `rm -rf .verdaccio` |
| List all v5\* tags | `git tag -l "v5*"` |
| Remove specific tag | `git tag -d "v5.0.0-next.5"` |
| Set Verdaccio as the NPM registry | `npm config set registry http://localhost:4873` |
| Reset NPM registry | `npm config set registry https://registry.npmjs.org/` |
| Start Verdaccio | `yarn verdaccio:start` |
| Release to Verdaccio | `yarn release --type=verdaccio` | |
| Create a new Webiny project | `npx create-webiny-project@local-npm my-test-project --tag local-npm --assign-to-yarnrc '{"npmRegistryServer":"http://localhost:4873","unsafeHttpWhitelist":["localhost"]}` |
| Revert versioning commit | `git reset HEAD~ && git reset --hard HEAD` |

## Troubleshooting

#### I made a new release to Verdaccio, but I still receive old code.

This is probably because of one of the Yarn package caching mechanisms.

Yarn has two levels of cache - local and shared.
Yarn has two levels of cache - local and shared.

When you install a package, it gets cached in the local cache folder (located in your project), and in the shared cache folder. This makes it much faster when you're working on a couple of projects on your local machine, and you're pulling the same package in each. If the package doesn't exist in local cache, it will be pulled from shared cache.
When you install a package, it gets cached in the local cache folder (located in your project), and in the shared cache
folder. This makes it much faster when you're working on a couple of projects on your local machine, and you're pulling
the same package in each. If the package doesn't exist in local cache, it will be pulled from shared cache.

On Windows, the shared cache folder should be located in: `C:\Users\{USER-NAME}\AppData\Local\Yarn`.
On Windows, the shared cache folder should be located in: `C:\Users\{USER-NAME}\AppData\Local\Yarn`.
On Linux/Mac, the shared cache folder should be located in: `/Users/adrian/Library/Caches/Yarn`.

In these folders, most probably, you'll also have the `\Berry\cache` folder. But, there were also cases where this folder did not exist.
In these folders, most probably, you'll also have the `\Berry\cache` folder. But, there were also cases where this
folder did not exist.

Deleting the mentioned cache folders should help with the issue of still receiving old packages in your testing sessions.
Deleting the mentioned cache folders should help with the issue of still receiving old packages in your testing
sessions.

With all of this being said, you can also try the [following command](https://yarnpkg.com/features/offline-cache#cleaning-the-cache):
With all of this being said, you can also try
the [following command](https://yarnpkg.com/features/offline-cache#cleaning-the-cache):

```bash
yarn cache clean --mirror
Expand Down
Loading

0 comments on commit c454fc0

Please sign in to comment.