Added test plan for TypeSpec extension #5552
+701
−0
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| changeKind: internal | ||
| packages: | ||
| - typespec-vscode | ||
| --- | ||
|
|
||
| Added test plan for TypeSpec extension |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| # Create TypeSpec Project from A Template Test Scenario | ||
|
|
||
| New TypeSpec projects can be created using a variety of templates for specific purposes. TypeSpec supports the following non-branded templates: | ||
|
|
||
| * Empty Project | ||
| * Generic REST API | ||
| * TypeSpec Library (with TypeScript) | ||
| * TypeSpec Emitter (with TypeScript) | ||
|
|
||
| Selecting a template involves: | ||
|
|
||
| * Installing required libraries | ||
| * Initializing essential project files with a specified folder structure | ||
| * Configuring output settings for a designated emission purpose | ||
|
|
||
|
|
||
| ## Test Environment | ||
|
|
||
| * OS : Windows or Linux | ||
| * Template : Empty Project, Generic REST API, TypeSpec Library (with TypeScript), TypeSpec Emitter (with TypeScript) | ||
|
|
||
| > Note: The extension should support all test cases in VS Code for Windows and Linux. Mac support is a stretch goal for Selenium semester. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Install TypeSpec Compiler before starting to write TypeSpec. | ||
|
|
||
| * [Node.js 20+](https://nodejs.org/download/) | ||
| * Npm 7+ | ||
| * [Install TypeSpec Compiler CLI](https://typespec.io/docs/): `"npm install -g @typespec/compiler"` | ||
|
|
||
| ## Test Steps | ||
|
|
||
| ### Step 1: Install the typespec extension. | ||
|
|
||
| _Option 1_. Install using .vsix file: | ||
| `Extension` -> `…` -> `Install form VSIX...` | ||
|
|
||
|  | ||
|
|
||
| Find the .vsix file you want to install locally. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Install typespec with vscode extension marketplace: | ||
| `Extension` -> input `TypeSpec for VS Code` -> `Install` | ||
|
|
||
|  | ||
|
|
||
| ### Step 2: Trigger create TypeSpec Project | ||
|
|
||
| _Option 1_. Clicking “Create TypeSpec Project” button/link in the “No Folder Opened” View of Explore. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Typing `> TypeSpec: Create TypeSpec Project` in the _Command Palette_. | ||
|
|
||
|  | ||
|
|
||
| ### Step 3. Select an empty folder as the root folder for the new TypeSpec project. | ||
|
There was a problem hiding this comment. please also try an non-empty folder which will trigger a confirmation step to confirm with end user about the selected folder |
||
|
|
||
|  | ||
|
|
||
| ### Step 4. Check if TypeSpec Compiler CLI is install. (optional) | ||
|
There was a problem hiding this comment. There are three path here,
|
||
|
|
||
| If the TypeSpec Compiler is not installed, the Quick Pick will initiate the installation of the TypeSpec Compiler. If TypeSpec Compiler is installed, Skip to the next step. | ||
|
|
||
|  | ||
|
|
||
| ### Step 5. After successfully installing TypeSpec Compiler, will go through the questions of `tsp init`. | ||
|
|
||
| 1. If the specified folder is not empty, it will prompt `Folder C:\xxx\xxx\xxx is not empty. Are you sure you want to initialize a new project here?`. If the folder is empty, skip to the next step. _(optional)_ | ||
|
|
||
|   **Validate:** Will it appear: `Folder C:\xxx\xxx\xxx is not empty. Are you sure you want to initialize a new project here?` | ||
|
|
||
|    | ||
|
|
||
| 2. Select a template _(Single choice)_. | ||
|
|
||
|   **Validate:** There should be a prompt "Select a template", and should see four options: `Empty project`, `Generic REST API`, `TypeSpec Library (With TypeScript)`, `TypeSpec Emitter (With TypeScript)`. | ||
|
|
||
|    | ||
|
|
||
| 3. Input project name - _(Text input)_ by default, it is the project root folder name.. | ||
|
|
||
|  | ||
|
|
||
| 4. Choose whether to generate a .ignore file. _(Single choice)_ `Y/N` | ||
|
|
||
|  | ||
|
|
||
| 5. Select libraries to update. _(Multiple choice)_ | ||
|
|
||
| > Note: Select libraries to update is required only when you select `Generic REST API`. This step will not appear if you select other templates. | ||
|
|
||
|  | ||
|
|
||
| 6. Click `OK` and the project will be created, the folder structure will be set up, dependencies will be installed, and tspconfig.yaml will be updated.. | ||
|
|
||
| **Validate:** Verify that the project was created correctly and the dependencies are installed. | ||
|
|
||
| - Install Dependencies | ||
|
|
||
|  | ||
|
|
||
| - Folder structure | ||
|
|
||
| - For `Empty project`, the folder structure. | ||
|
|
||
|  | ||
|
|
||
| - For `Generic REST API`, the folder structure. | ||
|
|
||
|  | ||
|
|
||
| - For `TypeSpec Library (With TypeScript)`, the folder structure. | ||
|
|
||
|  | ||
|
|
||
| - For `TypeSpec Emitter (With TypeScript)`, the folder structure. | ||
|
|
||
|  | ||
|
There was a problem hiding this comment. More init template can be configured. Please add one test case for that
|
||
|
|
||
| ## Issue Report | ||
|
|
||
| When an error is detected, it’s necessary to document the findings by using the following form: | ||
|
|
||
| | No | Title | Template | Issue Description | Repro Steps | Expected Results | Actual Results | Comments | | ||
| | ---------| :--: | :--: | :--: | :--: | :--: | :--: | :--: | | ||
| | 1 | e.g. Create typespec project failed | Empty project / Generic REST API / TypeSpec Library (With TypeScript) / TypeSpec Emitter (With TypeScript) | Create project feature is not supported by the current TypeSpec Compiler (ver <= 0.63.0). Please upgrade TypeSpec Compiler and try again. | 1. Typing `>TypeSpec: Create TypeSpec Project` in the _Command Palette_. <br> 2. Select an empty folder as the root folder for the new TypeSpec project. <br> 3. Select a template. | There should be a prompt "Select a template", and should see four options: `Empty project`, `Generic REST API`, `TypeSpec Library (With TypeScript)`, `TypeSpec Emitter (With TypeScript)`. | Create project feature is not supported by the current TypeSpec Compiler (ver <= 0.63.0). Please upgrade TypeSpec Compiler and try again. | Issue link | | ||
|
|
||
| ## Test Results Summary | ||
|
|
||
| The test results will be presented in the following form: | ||
|
|
||
| | NO | Test Cases | Platform | Template | Result | Issues | Comments | | ||
| | --------------- | :-: |:-: | :--: | :--: | :--: | :--: | | ||
| | 1 | Create TypeSpec Project from A Template | Windows/Linux | Empty project | | | | | ||
| | 2 | Create TypeSpec Project from A Template | Windows/Linux | Generic REST API | | | | | ||
| | 3 | Create TypeSpec Project from A Template | Windows/Linux | TypeSpec Library (With TypeScript) | | | | | ||
| | 4 | Create TypeSpec Project from A Template | Windows/Linux | TypeSpec Emitter (With TypeScript) | | | | | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| # Generate Client Code from TypeSpec Test Scenario | ||
|
|
||
| Generate client code from TypeSpec. In VS Code extension, we can complete code generation with step-by-step guidance. | ||
| TypeSpec Extension support will be extended to client code generation for first-class languages: `.NET`, `Python`, `Java`, and `JavaScript`. | ||
|
|
||
| **Important: There must be at least one TypeSpec project in the project folder.** | ||
|
|
||
| ## Test Environment | ||
|
|
||
| * OS : Windows or Linux | ||
| * Language : Python, Java, JavaScript, .NET | ||
|
|
||
| > Note: The extension should support all test cases in VS Code for Windows and Linux. Mac support is a stretch goal for Selenium semester. | ||
| ## Prerequisites | ||
|
|
||
| Install TypeSpec Compiler before starting to write TypeSpec. | ||
|
|
||
| * [Node.js 20+](https://nodejs.org/download/) | ||
| * Npm 7+ | ||
| * [Install TypeSpec Compiler CLI](https://typespec.io/docs/): `"npm install -g @typespec/compiler"` | ||
|
|
||
| Install required SDK/runtime for executing the specified language: | ||
|
|
||
| * [.NET 8.0 SDK](https://dotnet.microsoft.com/en-us/download) | ||
| * [Java 11 or above](https://www.oracle.com/java/technologies/downloads/), and [Maven](https://maven.apache.org/download.cgi) | ||
| * [Python 3.8+](https://www.python.org/downloads/) | ||
| * [Node.js 20+](https://nodejs.org/download/) | ||
|
|
||
| ## Test Steps | ||
|
|
||
| ### Step 1: Install the typespec extension. | ||
|
|
||
| _Option 1_. Install using .vsix file: | ||
| `Extension` -> `…` -> `Install form VSIX...` | ||
|
|
||
|  | ||
|
|
||
| Find the .vsix file you want to install locally. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Install typespec with vscode extension marketplace: | ||
| `Extension` -> input `TypeSpec for VS Code` -> `Install` | ||
|
|
||
|  | ||
|
|
||
| ### Step 2: Trigger generate from TypeSpec | ||
|
|
||
| Generation from a TypeSpec can be triggered in two ways: | ||
|
|
||
| _Option 1_. Clicking `Generate from TypeSpec` in the _Context Menu_ for a .tsp file in the extended TypeSpec project. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Typing `>TypeSpec: Generate from TypeSpec` in the _Command Palette_ with at least a TypeSpec project folder extended in the _Side Bar_. | ||
|
|
||
|  | ||
|
|
||
| ### Step 3: Click the command `TypeSpec: Generate from TypeSpec`, and choose a project. | ||
|
|
||
| **Validate:** There should be a prompt "Select a Typespec Project". | ||
|
|
||
|  | ||
|
|
||
| ### Step 4: Select an Emitter Type. | ||
|
There was a problem hiding this comment. the UI will be a little different after Compiler can provide template for default emitters. Will keep you updated. |
||
|
|
||
| **Validate:** There should be a prompt "Select an Emitter Type", and should see three emitter types: `Client Code`, `<PREVIEW> Server Stub`, `Protocol Schema`. | ||
|
|
||
|  | ||
|
|
||
| ### Step 5: Click `Client Code`. | ||
|
|
||
| **Validate:** There should be a prompt "Select a Language", and should see four languages: `DotNet`, `Java`, `JavaScript`, `Python`. | ||
|
|
||
|  | ||
|
|
||
| ### Step 6: Select a Language, confirm the TypeSpec emitters you want to install and their minimum requirements. | ||
|
|
||
| **Validate:** There should be a prompt `Here are libraries to install or update`, and TypeSpec emitters to install and their minimum requirements. | ||
|
|
||
|  | ||
|
|
||
| ### Step 7: Initiate the generation of client code on the backend. | ||
|
|
||
| **Validate:** The emitter package is already installed and the client folder is generated. The result appears in the lower right corner as a notification. | ||
|
|
||
| - For `DotNet`: | ||
|
|
||
|  | ||
|  | ||
|  | ||
|
|
||
| - For `Java`: | ||
|
|
||
|  | ||
|  | ||
|  | ||
|
|
||
| - For `JavaScript`: | ||
|
|
||
|  | ||
|  | ||
|  | ||
|
|
||
| - For `Python`: | ||
|
|
||
|  | ||
|  | ||
|  | ||
|
|
||
| ## Issue Report | ||
|
|
||
| When an error is detected, it’s necessary to document the findings by using the following form: | ||
|
|
||
| | No | Title | Emitter Type | Language | Issue Description | Repro Steps | Expected Results | Actual Results | Comments | | ||
| | ---------| :--: | :-: | :--: | :--: | :--: | :--: | :--: | :--: | | ||
| | 1 | e.g. Generate Client Code failed | Client Code | DotNet/ Java / JavaScript / Python | Exception occurred when generating client code for JavaScript. | 1. Typing `>TypeSpec: Generate from TypeSpec` in the _Command Palette_. <br> 2. choose a project. <br> 3. Select a template. <br> 4. Select an Emitter Type. <br> 5. Select a Language.| Generate client code for JavaScript ...Succeeded. | Exception occurred when generating client code for JavaScript. | Issue link | | ||
|
|
||
| ## Test Results Summary | ||
|
|
||
| The test results will be presented in the following form: | ||
|
|
||
| | NO | Test Cases | Platform | Language | Result | Issues | Comments | | ||
| | --------------- | :-: |:-: | :--: | :--: | :--: | :--: | | ||
| | 1 | Generate Client Code from TypeSpec | Windows/Linux | Python | | | | | ||
| | 2 | Generate Client Code from TypeSpec | Windows/Linux | Java | | | | | ||
| | 3 | Generate Client Code from TypeSpec | Windows/Linux | .NET | | | | | ||
| | 4 | Generate Client Code from TypeSpec | Windows/Linux | JavaScript | | | | | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| # Generate OpenAPI 3.x from TypeSpec Test Scenario | ||
|
There was a problem hiding this comment. I think we can merge the client, server stub and openapi3 user scenario into one, they are basically the same except choosing different emitters |
||
|
|
||
| Emit OpenAPI3 from TypeSpec to automate API-related tasks: generate API documentation, test API, etc. | ||
| The TypeSpec file itself is not sufficient to generate OpenAPI 3. The conversion process will always reference the entry point (main.tsp) of the TypeSpec build, which includes the main definitions of models, services, and operations. | ||
|
|
||
| **Important: There must be at least one TypeSpec project in the project folder.** | ||
|
|
||
| ## Test Environment | ||
|
|
||
| * OS : Windows or Linux | ||
| * Language : OpenAPI3 | ||
|
|
||
| > Note: The extension should support all test cases in VS Code for Windows and Linux. Mac support is a stretch goal for Selenium semester. | ||
| ## Prerequisites | ||
|
|
||
| Install TypeSpec Compiler before starting to write TypeSpec. | ||
|
|
||
| * [Node.js 20+](https://nodejs.org/download/) | ||
| * Npm 7+ | ||
| * [Install TypeSpec Compiler CLI](https://typespec.io/docs/): `"npm install -g @typespec/compiler"` | ||
|
|
||
| ## Test Steps | ||
|
|
||
| ### Step 1: Install the typespec extension. | ||
|
|
||
| _Option 1_. Install using .vsix file: | ||
| `Extension` -> `…` -> `Install form VSIX...` | ||
|
|
||
|  | ||
|
|
||
| Find the .vsix file you want to install locally. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Install typespec with vscode extension marketplace: | ||
| `Extension` -> input `TypeSpec for VS Code` -> `Install` | ||
|
|
||
|  | ||
|
|
||
| ### Step 2: Trigger generate from TypeSpec | ||
|
|
||
| Generation from a TypeSpec can be triggered in two ways: | ||
|
|
||
| _Option 1_. Clicking `Generate from TypeSpec` in the _Context Menu_ for a .tsp file in the extended TypeSpec project. | ||
|
|
||
|  | ||
|
|
||
| _Option 2_. Typing `>TypeSpec: Generate from TypeSpec` in the _Command Palette_ with at least a TypeSpec project folder extended in the _Side Bar_. | ||
|
|
||
|  | ||
|
|
||
| ### Step 3: Click the command `TypeSpec: Generate from TypeSpec`, and choose a project. | ||
|
|
||
| **Validate:** There should be a prompt "Select a Typespec Project". | ||
|
|
||
|  | ||
|
|
||
| ### Step 4: Select an Emitter Type. | ||
|
|
||
| **Validate:** There should be a prompt "Select an Emitter Type", and should see three emitter types: `Client Code`, `<PREVIEW> Server Stub`, `Protocol Schema`. | ||
|
|
||
|  | ||
|
|
||
| ### Step 5: Click `Protocol Schema`. | ||
|
|
||
| **Validate:** There should be a prompt "Select a Language", and should see languages: `OpenAPI3`. | ||
|
|
||
|  | ||
|
|
||
| ### Step 6: Select a Language, confirming installation of the required TypeSpec library @typespec/openapi3. | ||
|
|
||
| **Validate**: There should be a prompt `Here are libraries to install or update`, and confirming installation of the required TypeSpec library @typespec/openapi3. | ||
|
|
||
|  | ||
|
|
||
| ### Step 7: Initiate the generation of OpenAPI on the backend. | ||
|
|
||
| **Validate:** A detailed trace log should be printed in the OUTPUT window. The result appears as a Notification in the bottom right corner, and generate the schema folder. | ||
|
|
||
|  | ||
|  | ||
|
|
||
| If the OpenAPI 3 generation fails, the error message should indicate the root cause and suggest possible resolutions to continue. | ||
|
|
||
|  | ||
|
|
||
| ## Issue Report | ||
|
|
||
| When an error is detected, it’s necessary to document the findings by using the following form: | ||
|
|
||
| | No | Title | Emitter Type | Language | Issue Description | Repro Steps | Expected Results | Actual Results | Comments | | ||
| | ---------| :--: | :-: | :--: | :--: | :--: | :--: | :--: | :--: | | ||
| | 1 | e.g. Generate schema code failed | Protocol Schema | OpenAPI3 | Exception occurred when generating schema code for OpenAPI3. | 1. Typing `>TypeSpec: Generate from TypeSpec` in the _Command Palette_. <br> 2. choose a project. <br> 3. Select a template. <br> 4. Select an Emitter Type. <br> 5. Select a Language.| Generate schema code for OpenAPI3 ...Succeeded. | Exception occurred when generating schema code for OpenAPI3. | Issue link | | ||
|
|
||
| ## Test Results Summary | ||
|
|
||
| The test results will be presented in the following form: | ||
|
|
||
| | NO | Test Cases | Platform | Language | Result | Issues | Comments | | ||
| | --------------- | :-: |:-: | :--: | :--: | :--: | :--: | | ||
| | 1 | Generate OpenAPI 3.x from TypeSpec Test Scenario | Windows/Linux | OpenAPI3 | | | | | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the templates are loaded from compiler which may change when compiler changes. Please run 'tsp init' to get the latest templates as baseline.