Skip to content

Commit

Permalink
Added per language scope support docs (#2451)
Browse files Browse the repository at this point in the history
## Checklist

- [/] I have added
[tests](https://www.cursorless.org/docs/contributing/test-case-recorder/)
- [x] I have updated the
[docs](https://github.com/cursorless-dev/cursorless/tree/main/docs) and
[cheatsheet](https://github.com/cursorless-dev/cursorless/tree/main/cursorless-talon/src/cheatsheet)
- [/] I have not broken the cheatsheet

---------

Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Pokey Rule <[email protected]>
  • Loading branch information
3 people authored Jul 30, 2024
1 parent f56004e commit bc82122
Show file tree
Hide file tree
Showing 92 changed files with 377 additions and 74 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import {
type ScopeType,
} from "../types/command/PartialTargetDescriptor.types";

const scopeSupportFacets = [
export const scopeSupportFacets = [
"command",

"element",
Expand Down
4 changes: 0 additions & 4 deletions packages/cursorless-org-docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,6 @@
.docusaurus
.cache-loader

# Copied from root of the repository
docs/
images/

# Misc
.DS_Store
.env.local
Expand Down
8 changes: 3 additions & 5 deletions packages/cursorless-org-docs/docusaurus.config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ function remarkPluginFixLinksToRepositoryArtifacts(): Transformer<Root> {
"../..",
);
const artifact = resolve(file.dirname!, url);
const artifactRelative = relative(repoRoot, artifact);
const artifactRelative = relative(repoRoot, artifact).replace(/\\/g, "/");

// We host all files under docs, will resolve as a relative link
if (artifactRelative.startsWith("docs/")) {
Expand Down Expand Up @@ -104,13 +104,11 @@ const config: Config = {
"classic",
{
docs: {
path: "../../docs",
path: "./src/docs",
// Followed https://ricard.dev/how-to-set-docs-as-homepage-for-docusaurus/
// to serve a markdown document on homepage
routeBasePath: "/",
// Note that we add dummy/dummy so that the `../..` above has something to strip
editUrl:
"https://github.com/cursorless-dev/cursorless/edit/main/dummy/dummy",
editUrl: "https://github.com/cursorless-dev/cursorless/edit/main",
sidebarPath: require.resolve("./sidebar.js"),
beforeDefaultRemarkPlugins: [
remarkPluginFixLinksToRepositoryArtifacts,
Expand Down
2 changes: 2 additions & 0 deletions packages/cursorless-org-docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"clean": "pnpm clear && rm -rf ./out tsconfig.tsbuildinfo ./dist ./build"
},
"dependencies": {
"@cursorless/common": "workspace:*",
"@algolia/client-search": "4.22.1",
"@docsearch/react": "3.6.0",
"@docusaurus/core": "3.1.1",
Expand Down Expand Up @@ -50,6 +51,7 @@
"@docusaurus/types": "3.1.1",
"@tsconfig/docusaurus": "2.0.2",
"@types/mdast": "4.0.3",
"@types/react": "18.2.71",
"typescript": "^5.5.3",
"unified": "11.0.4"
},
Expand Down
4 changes: 4 additions & 0 deletions packages/cursorless-org-docs/src/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,7 @@
[data-theme="light"] .light-mode-invert {
filter: invert(90%) hue-rotate(180deg);
}

.hidden {
display: none;
}
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ for how to add support for a new parser

## 2. Ensure file type is supported by VSCode

If you are adding support for a new language that isn't natively detected by VSCode, you will need to add the appropriate extension to the list of dependencies. The list of languages officially supported by VSCode is listed [in the VSCode docs](https://code.visualstudio.com/docs/languages/identifiers#_known-language-identifiers). If your language is in that list, you can skip this step and proceed to step 3. If your language is not in that list, you need to find a VSCode extension that adds support for your language, and add the id of the given extension to [`packages/common/src/extensionDependencies.ts`](../../packages/common/src/extensionDependencies.ts) and then re-run `pnpm init-vscode-sandbox` to ensure it is installed. If you do not do this you will encounter errors when attempting to execute cursorless commands in the next step. See [#1895](https://github.com/cursorless-dev/cursorless/issues/1895) for more info.
If you are adding support for a new language that isn't natively detected by VSCode, you will need to add the appropriate extension to the list of dependencies. The list of languages officially supported by VSCode is listed [in the VSCode docs](https://code.visualstudio.com/docs/languages/identifiers#_known-language-identifiers). If your language is in that list, you can skip this step and proceed to step 3. If your language is not in that list, you need to find a VSCode extension that adds support for your language, and add the id of the given extension to [`packages/common/src/extensionDependencies.ts`](../../../../../packages/common/src/extensionDependencies.ts) and then re-run `pnpm init-vscode-sandbox` to ensure it is installed. If you do not do this you will encounter errors when attempting to execute cursorless commands in the next step. See [#1895](https://github.com/cursorless-dev/cursorless/issues/1895) for more info.

## 3. Register your language with Cursorless

1. Create a file with your scope support map to indicate which scopes you support. See eg [`markdown.ts`](../../packages/common/src/scopeSupportFacets/markdown.ts). At the start, you can leave the actual scope support table empty, so it will look something like the following (keeping in mind it's best to look at [`markdown.ts`](../../packages/common/src/scopeSupportFacets/markdown.ts) or another support file in case the following snippet rots):
1. Create a file with your scope support map to indicate which scopes you support. See eg [`markdown.ts`](../../../../../packages/common/src/scopeSupportFacets/markdown.ts). At the start, you can leave the actual scope support table empty, so it will look something like the following (keeping in mind it's best to look at [`markdown.ts`](../../../../../packages/common/src/scopeSupportFacets/markdown.ts) or another support file in case the following snippet rots):

```ts
import {
Expand All @@ -29,9 +29,9 @@ If you are adding support for a new language that isn't natively detected by VSC
export const markdownScopeSupport: LanguageScopeSupportFacetMap = {};
```

2. Add an entry pointing to your support table to [the global scope support table](../../packages/common/src/scopeSupportFacets/languageScopeSupport.ts)
2. Add an entry pointing to your support table to [the global scope support table](../../../../../packages/common/src/scopeSupportFacets/languageScopeSupport.ts)

3. Create an empty `.scm` file in [`queries/`](../../queries) to hold your parse tree patterns. It should be named after your language, eg `java.scm`.
3. Create an empty `.scm` file in [`queries/`](../../../../../queries) to hold your parse tree patterns. It should be named after your language, eg `java.scm`.

You can file a PR with just these changes to get the ball rolling.

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ For example, `"funk"` (`namedFunction`) has the following facets:
- `namedfunction.method`, corresponding to a class method declaration, and
- `namedfunction.constructor`, corresponding to a class constructor declaration.

Have a look in [`scopeSupportFacetInfos`](../../packages/common/src/scopeSupportFacets/scopeSupportFacetInfos.ts) to see which facets the given scope has. The key is the id of the facet, and the value has information about the facet, including a description and a `scopeType` field indicating which scope type the facet corresponds to.
Have a look in [`scopeSupportFacetInfos`](../../../../../packages/common/src/scopeSupportFacets/scopeSupportFacetInfos.ts) to see which facets the given scope has. The key is the id of the facet, and the value has information about the facet, including a description and a `scopeType` field indicating which scope type the facet corresponds to.

These facet ids will be the keys in your language's scope support table below.

Expand Down Expand Up @@ -50,14 +50,14 @@ When you're done, say `"cursorless save scope"` to save the tests to the appropr

## 5. Add parse tree patterns for the given scope

Launch your extension in debug mode and open a file in your language. You can create one or more files in [`playground/`](../../data/playground) and feel free to include those in your PR.
Launch your extension in debug mode and open a file in your language. You can create one or more files in [`playground/`](../../../../../data/playground) and feel free to include those in your PR.

Then add parse tree patterns for the given scope to your language's `.scm` file in the [`queries` directory](../../queries). The parse tree patterns should match the syntactic constructs that should be considered to be the given scope. Tag the nodes in the parse tree that correspond to the given scope with the internal identifier you found in step 1 above, eg `@namedFunction`. Note that you use the scope identifier (`@namedFunction`), _**not**_ the facet identifier (`@namedFunction.class`).
Then add parse tree patterns for the given scope to your language's `.scm` file in the [`queries` directory](../../../../../queries). The parse tree patterns should match the syntactic constructs that should be considered to be the given scope. Tag the nodes in the parse tree that correspond to the given scope with the internal identifier you found in step 1 above, eg `@namedFunction`. Note that you use the scope identifier (`@namedFunction`), _**not**_ the facet identifier (`@namedFunction.class`).

### Notes / tips

- See our [Tree-sitter query syntax](tree-sitter-query-syntax.md) guide for more information on the syntax we support.
- Look at the existing language definitions in the [`queries` directory](../../queries) for examples.
- Look at the existing language definitions in the [`queries` directory](../../../../../queries) for examples.
- Use the [scope visualizer](../user/scope-visualizer.md) to see your scope highlighted in real time every time you save the `.scm` file.
- Use the command `"parse tree <target>"` to see the parse tree for a given target. For example `"parse tree line"` will show you the parse tree for the current line, as well as all of its ancestors. This will generate a markdown file with parse tree info, which you can then use to write your patterns. You might find it helpful to open a markdown preview of the file.
- You will likely want to look at `node-types.json` for your language, (eg [java](https://github.com/tree-sitter/tree-sitter-java/blob/master/src/node-types.json)). This file is generated from the language's `grammar.js`, which might also be helpful to look at (eg [java](https://github.com/tree-sitter/tree-sitter-java/blob/master/grammar.js)).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ The implementation of the local version of the cheatsheet is split between the T

## Adding a new spoken form

When you add a new scope type, action, modifier, etc, you'll need to ensure that it shows up both locally and on the website. It will usually automatically show up in the local cheatsheet. You can verify this by saying `"cursorless cheatsheet"` with your development version of `cursorless-talon` active in your Talon user directory, and inspecting the cheatsheet that appears. If it does not, you'll need to make fixes to [the Talon side of the cheatsheet](../../cursorless-talon/src/cheatsheet).
When you add a new scope type, action, modifier, etc, you'll need to ensure that it shows up both locally and on the website. It will usually automatically show up in the local cheatsheet. You can verify this by saying `"cursorless cheatsheet"` with your development version of `cursorless-talon` active in your Talon user directory, and inspecting the cheatsheet that appears. If it does not, you'll need to make fixes to [the Talon side of the cheatsheet](../../../../../cursorless-talon/src/cheatsheet).

In either case, to get your changes to appear on the website, you need to update the defaults in [`defaults.json`](../../packages/cheatsheet/src/lib/sampleSpokenFormInfos/defaults.json). First make sure you have the `cursorless-talon-dev` user file set in your Talon home directory, as indicated in the [initial contributor setup instructions](CONTRIBUTING.md#initial-setup). Then you can say `"cursorless update cheatsheet"` to update the default spoken forms. Note that this will use your custom spoken forms, so you may need to do some manual cleanup.
In either case, to get your changes to appear on the website, you need to update the defaults in [`defaults.json`](../../../../../packages/cheatsheet/src/lib/sampleSpokenFormInfos/defaults.json). First make sure you have the `cursorless-talon-dev` user file set in your Talon home directory, as indicated in the [initial contributor setup instructions](CONTRIBUTING.md#initial-setup). Then you can say `"cursorless update cheatsheet"` to update the default spoken forms. Note that this will use your custom spoken forms, so you may need to do some manual cleanup.

## Running the cheatsheet in development mode

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ The documentation search is powered by Algolia.

## Tweaking crawling / indexing / ranking

Unfortunately, today, the source of truth for the Algolia search configuration lives in the Algolia web console. Whenever we update the configuration, we update the copies that we keep in [source control](../../packages/cursorless-org-docs/config/algolia). In the future, we'd like to use the files in source control as the source of truth and deploy them to Algolia in CI. See #917.
Unfortunately, today, the source of truth for the Algolia search configuration lives in the Algolia web console. Whenever we update the configuration, we update the copies that we keep in [source control](../../../../../packages/cursorless-org-docs/config/algolia). In the future, we'd like to use the files in source control as the source of truth and deploy them to Algolia in CI. See #917.

To see what changes we've made to the default configuration, compare the contents of [this directory](../../packages/cursorless-org-docs/config/algolia) with https://github.com/cursorless-dev/cursorless/tree/e043ce4795ffcda5a3f5875d91887a09e0f9905b/website/config/algolia
To see what changes we've made to the default configuration, compare the contents of [this directory](../../../../../packages/cursorless-org-docs/config/algolia) with https://github.com/cursorless-dev/cursorless/tree/e043ce4795ffcda5a3f5875d91887a09e0f9905b/website/config/algolia

### Crawler config

1. Use the [crawler console](https://crawler.algolia.com/admin/crawlers/ff3ea576-b9e0-4e01-8a19-110106760e74/configuration/edit) to experiment with the config until it works as expected.
2. Copy the new crawler config and paste it to [`crawler-settings.js`](../../docs-site/config/algolia/crawler-settings.js)
2. Copy the new crawler config and paste it to [`crawler-settings.js`](../../../../../docs-site/config/algolia/crawler-settings.js)
3. **IMPORTANT** Replace the `apiKey` field with `"<REDACTED>"`
4. File a PR to get feedback on the new config
5. Press the Save button in the crawler console to persist the new config.
Expand All @@ -20,5 +20,5 @@ To see what changes we've made to the default configuration, compare the content
### Index settings

1. Use the [Algolia console](https://www.algolia.com/apps/YTJQ4I3GBJ/explorer/configuration/cursorless/searchable-attributes) to tweak the settings until you're happy.
2. Click on _Manage index > Export Configuration_ to export the configuration json, saving it to [`index-settings.json`](../../packages/cursorless-org-docs/config/algolia/index-settings.json)
2. Click on _Manage index > Export Configuration_ to export the configuration json, saving it to [`index-settings.json`](../../../../../packages/cursorless-org-docs/config/algolia/index-settings.json)
3. File a PR to get feedback on the new config.
File renamed without changes.
File renamed without changes.
Loading

0 comments on commit bc82122

Please sign in to comment.