Skip to content

Commit 429334a

Browse files
committed
Change snippet extraction to use regex, add support for inline snippet tokens, update readme
1 parent 53cceb9 commit 429334a

19 files changed

+594
-108
lines changed

README.md

Lines changed: 126 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
`doc-snippets` is a simple tool that allows you to extract and inject snippets from code into markdown files.
44

5-
## Installation
5+
# Installation
66

77
Using NPM:
88

@@ -16,9 +16,9 @@ Using Yarn:
1616
yarn add -D doc-snippets
1717
```
1818

19-
## Usage
19+
# Usage
2020

21-
### Marking and injecting
21+
## Marking and injecting
2222

2323
`doc-snippets` extracts and injects snippets by using tokens:
2424

@@ -28,7 +28,7 @@ yarn add -D doc-snippets
2828
- To inject a snippet:
2929
- `$snippet: snippet-name` - This gets replaced by the snippet with the given name.
3030

31-
### Configuration
31+
## Configuration
3232

3333
`doc-snippets` is, by defaut, configured using a JSON file that contains a `doc-snippets` object. By default, this file is `package.json`.
3434

@@ -46,14 +46,24 @@ The configuration object has the following structure (and default values):
4646
"ignore": [],
4747
"dir": "./src/docs"
4848
},
49-
"startToken": "$start: ",
50-
"endToken": "$end",
49+
"startTokens": [
50+
{
51+
"pattern": "$start: ",
52+
"inline": false
53+
}
54+
],
55+
"endTokens": [
56+
{
57+
"pattern": "$end",
58+
"inline": false
59+
}
60+
],
5161
"injectToken": "$snippet: ",
5262
"outputDir": "./docs"
5363
}
5464
```
5565

56-
#### `extract` and `inject`
66+
### `extract` and `inject`
5767

5868
The `extract` and `inject` objects both have the same structure:
5969

@@ -66,29 +76,107 @@ The `extract` object specifies which files will be parsed for snippets to extrac
6676
The `inject` object specifies which files will be copied into `outputDir` and have snippets injected into them.
6777

6878
**Example `extract` object:**
79+
6980
```JSON
7081
"extract": {
7182
"include": ["sample.sql", "./**/*.{js,ts}"],
7283
"ignore": "./**/node_modules/**",
7384
"dir": "./src"
7485
}
7586
```
87+
7688
In this example, `extract` will perform its search within the `./src` directory.
7789

7890
It will include:
91+
7992
- `./src/sample.sql`
8093
- All files within `./src` and its subdirectories ending in `.js` and `.ts`
8194

8295
It will ignore:
96+
8397
- All directories and subdirectories of any `node_modules` directory found within `./src` and its subdirectories.
8498

8599
The same principles apply for the `inject` object.
86100

87-
#### `outputDir`
101+
### Extraction tokens (`startTokens` and `endTokens`)
102+
103+
Extraction tokens mark the beginning and end of our snippets.
104+
105+
The tokens have two properties:
106+
- `pattern` - the **exact string** that denotes the token
107+
- `inline` - defines whether the token is a **regular** or **inline** extraction token
108+
109+
#### Regular and inline extraction tokens
110+
111+
By default, extraction tokens are designed to be written within their own lines. Anything else written within the token's line will be ignored when extracting snippets.
112+
113+
Let's take a look at an example:
114+
115+
```typescript
116+
// $start: hello-snippet we can add comments here
117+
const greeting = "Hello World!";
118+
console.log(greeting);
119+
// $end
120+
```
121+
122+
Using the default configuration, this code will yield a snippet called `hello-snippet` with the following contents:
123+
```
124+
const greeting = "Hello World!";
125+
console.log(greeting);
126+
```
127+
128+
To allow for more flexible snippet extraction, we can use **inline** extraction tokens.
129+
130+
For example, let's configure our extraction tokens as follows:
131+
132+
```json
133+
{
134+
//...
135+
"startTokens": [
136+
{
137+
"pattern": "/* $start: {SNIPPET_NAME} */",
138+
"inline": true
139+
}
140+
],
141+
"endTokens": [
142+
{
143+
"pattern": "/* $end */",
144+
"inline": true
145+
}
146+
]
147+
}
148+
```
149+
150+
Note that there is a special token within the start token's `pattern` property: **`{SNIPPET_NAME}`**. This denotes where in the inline token the snippet name will appear.
151+
152+
In addition, let's assume that we have a file with the following code:
153+
154+
```typescript
155+
const greeting = /* $start: hello-inline-snippet */"Hello World!";
156+
console.log(greeting);/* $end */
157+
```
158+
159+
Snippet extraction would now yield a snippet named `hello-inline-snippet` with the following contents:
160+
```
161+
"Hello World!";
162+
console.log(greeting);
163+
```
164+
165+
Inline extraction tokens can be useful when you need to extract only part of your line.
166+
167+
Regular and inline extraction tokens can be used at the same time, and snippet extraction will always search for **any** end token after it encounters **any** start token.
168+
169+
For reference:
170+
- When a **regular start token** is encoutered, snippet extraction starts at the beginning of the next line.
171+
- When an **inline start token** is encountered, snippet extraction starts immediately after the token ends.
172+
- When a **regular end token** is encountered, snippet extraction ends at the end of the previous line.
173+
- When an **inline end token** is encountered, snippet extraction ends at the beginning of the token.
174+
175+
### `outputDir`
88176

89177
The `outputDir` is the output directory for the documentation injected with snippets.
90178

91-
### Running `doc-snippets`
179+
## Running `doc-snippets`
92180

93181
`doc-snippets` comes with a CLI tool which is designed to handle most scenarios.
94182

@@ -100,7 +188,7 @@ doc-snippets combine
100188

101189
The `combine` command reads a `"doc-snippets"` section from a configuration file (by default this is `package.json`) and performs snippet extraction and injection, and outputs documentation with injected snippets into an `outputDir`.
102190

103-
#### 'combine' command options
191+
### 'combine' command options
104192

105193
- `-c --config <path>` - Path to configuration file (default: './package.json')
106194
- `-o --output-dir <path>` - Combined documentation output directory
@@ -110,12 +198,14 @@ The `combine` command reads a `"doc-snippets"` section from a configuration file
110198
- `--inject-dir <path>` - The base directory within which to search for injectable files
111199
- `--inject-include <paths...>` - Include specified paths or glob patterns in snippet injection
112200
- `--inject-ignore <paths...>` - Ignore specified paths or glob patterns in snippet injection
113-
- `--start-token <token>` - Token marking the start of the snippet
114-
- `--end-token <token>` - Token marking the end of the snippet
201+
- `--start-tokens <tokens...>` - Tokens marking the start of the snippet
202+
- `--end-tokens <tokens...>` - Tokens marking the end of the snippet
203+
- `--inline-start-tokens <tokens...>` - Inline Tokens marking the start of the snippet
204+
- `--inline-end-tokens <tokens...>` - Inline Tokens marking the end of the snippet
115205
- `--inject-token <token>` - Token marking the point of snippet injection
116206

117207

118-
### In your own code
208+
## In your own code
119209

120210
If you want to use `doc-snippets` programatically, it offers two exported functions:
121211

@@ -125,11 +215,31 @@ import { extractSnippets, injectSnippetsIntoFile } from "doc-snippets";
125215
const searchOptions = {
126216
include: ["sample.sql", "./**/*.{js,ts}"],
127217
ignore: "./**/node_modules/**",
128-
dir: "./src"
129-
}
218+
dir: "./src",
219+
};
130220

131221
//Returns snippets as `Record<string, string>`.
132-
const snippets = await extractSnippets(searchOptions, "$start: ", "$end");
222+
const startTokens = [
223+
{
224+
pattern: "$start: "
225+
},
226+
{
227+
pattern: "/* $start: {SNIPPET_NAME} */",
228+
inline: true
229+
}
230+
];
231+
232+
const endTokens = [
233+
{
234+
pattern: "$end"
235+
},
236+
{
237+
pattern: "/* $end */",
238+
inline: true
239+
}
240+
];
241+
242+
const snippets = await extractSnippets(searchOptions, startTokens, endTokens);
133243

134244
//Injects `snippets` into `./dest/readme.md` replacing all instances of `$snippet: snippet-name` with the appropriate snippet
135245
await injectSnippetsIntoFile(snippets, "./dest/readme.md", "$snippet: ");

0 commit comments

Comments
 (0)