-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(benchmark): Add benchmark for full language server #15
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# language-server-tests-benchmarks | ||
|
||
The tests and benchmarks in this folder aims to test and benchmark the language server in a close-to-reality scenario, using the same JavaScript client powered by `vscode-jsonrpc` used in VS Code. | ||
|
||
## Why are the benchmarks so slow? | ||
|
||
If you're used to the numbers from the benchmarks of the language services, the numbers here might be surprisingly slow. The reason for that is that the benchmark also includes the large overhead caused by the client-server communication. Even if the language server can sometimes answer in 50-100μs, just sending and waiting for the response can take 95-99% of the time the benchmark measures, leading to times closer to 1-2ms. | ||
|
||
Unfortunately, due to the multiple processes involved it's not possible to get accurate flamegraphs from CodSpeed for these benchmarks at the time of writing. Locally, [flamegraph](https://github.com/flamegraph-rs/flamegraph) can be used, but it requires a bit of setup, especially on non-Linux systems. |
17 changes: 17 additions & 0 deletions
17
packages/language-server-tests-benchmarks/fixture/colors_benchmark.css
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
.header { | ||
background-color: #333; | ||
color: white; | ||
padding: 15px 20px; | ||
text-align: center; | ||
} | ||
|
||
h1 { | ||
color: red; | ||
background-color: lab(50% 50% 50%); | ||
} | ||
|
||
.header .logo { | ||
font-size: 2rem; | ||
font-weight: bold; | ||
color: lch(50% 50% 50%); | ||
} |
74 changes: 74 additions & 0 deletions
74
packages/language-server-tests-benchmarks/fixture/folding_benchmark.css
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
/* General Layout */ | ||
.container { | ||
width: 100%; | ||
padding: 20px; | ||
margin: 0 auto; | ||
max-width: 1200px; | ||
} | ||
|
||
.header { | ||
background-color: #333; | ||
color: white; | ||
padding: 15px 20px; | ||
text-align: center; | ||
} | ||
|
||
.header .logo { | ||
font-size: 2rem; | ||
font-weight: bold; | ||
} | ||
|
||
.sidebar { | ||
flex: 1; | ||
padding: 20px; | ||
background-color: #fff; | ||
border-radius: 8px; | ||
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); | ||
} | ||
|
||
.footer { | ||
margin-top: 40px; | ||
padding: 20px; | ||
background-color: #222; | ||
color: white; | ||
text-align: center; | ||
font-size: 0.9rem; | ||
} | ||
|
||
/* Media Queries */ | ||
@media (max-width: 1024px) { | ||
.content { | ||
flex-direction: column; | ||
align-items: center; | ||
} | ||
.main, | ||
.sidebar { | ||
width: 80%; | ||
margin-bottom: 20px; | ||
} | ||
} | ||
|
||
@media (max-width: 768px) { | ||
.header .logo { | ||
font-size: 1.5rem; | ||
} | ||
.footer { | ||
font-size: 0.8rem; | ||
} | ||
} | ||
|
||
@media (max-width: 480px) { | ||
.container { | ||
padding: 10px; | ||
} | ||
.header { | ||
padding: 10px; | ||
} | ||
.content { | ||
flex-direction: column; | ||
} | ||
.main, | ||
.sidebar { | ||
padding: 10px; | ||
} | ||
} |
3 changes: 3 additions & 0 deletions
3
packages/language-server-tests-benchmarks/fixture/hover_benchmark.css
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
h1 { | ||
color: red; | ||
} |
8 changes: 6 additions & 2 deletions
8
packages/language-server-tests/package.json → ...uage-server-tests-benchmarks/package.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
54 changes: 54 additions & 0 deletions
54
packages/language-server-tests-benchmarks/src/benchmarks/css/colors.bench.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
import { afterAll, bench, describe } from "vitest"; | ||
import { startLanguageServer } from "../../server"; | ||
import { fileURLToPath } from "url"; | ||
|
||
const filePath = fileURLToPath( | ||
new URL("../../../fixture/colors_benchmark.css", import.meta.url) | ||
); | ||
|
||
const weblsp = await startLanguageServer(undefined, "weblsp"); | ||
const weblspUri = (await weblsp.openTextDocument(filePath, "css")).uri; | ||
const weblspColors = await weblsp.sendDocumentColorRequest(weblspUri); | ||
|
||
const vscodeLsp = await startLanguageServer(undefined, "vscode-css"); | ||
const vscodeLspUri = (await vscodeLsp.openTextDocument(filePath, "css")).uri; | ||
const vscodeColors = await vscodeLsp.sendDocumentColorRequest(vscodeLspUri); | ||
|
||
describe("Document Colors", async () => { | ||
bench("weblsp - Document Colors", async () => { | ||
await weblsp.sendDocumentColorRequest(weblspUri); | ||
}); | ||
|
||
if (!process.env.CODSPEED) { | ||
bench("vscode-css-languageserver - Document Colors", async () => { | ||
await vscodeLsp.sendDocumentColorRequest(vscodeLspUri); | ||
}); | ||
} | ||
}); | ||
|
||
describe("Color Presentations", async () => { | ||
bench("weblsp - Color Presentation", async () => { | ||
await weblsp.sendColorPresentationRequest( | ||
weblspUri, | ||
weblspColors[0].color, | ||
weblspColors[0].range | ||
); | ||
}); | ||
|
||
if (!process.env.CODSPEED) { | ||
bench("vscode-css-languageserver - Color Presentation", async () => { | ||
await vscodeLsp.sendColorPresentationRequest( | ||
vscodeLspUri, | ||
vscodeColors[0].color, | ||
vscodeColors[0].range | ||
); | ||
}); | ||
} | ||
|
||
afterAll(async () => { | ||
await weblsp.shutdown(); | ||
await vscodeLsp.shutdown(); | ||
await weblsp.exit(); | ||
await vscodeLsp.exit(); | ||
}); | ||
}); |
32 changes: 32 additions & 0 deletions
32
packages/language-server-tests-benchmarks/src/benchmarks/css/folding.bench.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
import { afterAll, bench, describe } from "vitest"; | ||
import { startLanguageServer } from "../../server"; | ||
import { fileURLToPath } from "url"; | ||
|
||
const filePath = fileURLToPath( | ||
new URL("../../../fixture/folding_benchmark.css", import.meta.url) | ||
); | ||
|
||
const weblsp = await startLanguageServer(undefined, "weblsp"); | ||
const weblspUri = (await weblsp.openTextDocument(filePath, "css")).uri; | ||
|
||
const vscodeLsp = await startLanguageServer(undefined, "vscode-css"); | ||
const vscodeLspUri = (await vscodeLsp.openTextDocument(filePath, "css")).uri; | ||
|
||
describe("Folding Ranges", async () => { | ||
bench("weblsp - Folding Ranges", async () => { | ||
await weblsp.sendFoldingRangesRequest(weblspUri); | ||
}); | ||
|
||
if (!process.env.CODSPEED) { | ||
bench("vscode-css-languageserver - Folding Ranges", async () => { | ||
await vscodeLsp.sendFoldingRangesRequest(vscodeLspUri); | ||
}); | ||
} | ||
|
||
afterAll(async () => { | ||
await weblsp.shutdown(); | ||
await vscodeLsp.shutdown(); | ||
await weblsp.exit(); | ||
await vscodeLsp.exit(); | ||
}); | ||
}); |
38 changes: 38 additions & 0 deletions
38
packages/language-server-tests-benchmarks/src/benchmarks/css/hover.bench.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
import { afterAll, bench, describe } from "vitest"; | ||
import { startLanguageServer } from "../../server"; | ||
import { fileURLToPath } from "url"; | ||
|
||
const filePath = fileURLToPath( | ||
new URL("../../../fixture/hover_benchmark.css", import.meta.url) | ||
); | ||
|
||
const weblsp = await startLanguageServer(undefined, "weblsp"); | ||
const weblspUri = (await weblsp.openTextDocument(filePath, "css")).uri; | ||
|
||
const vscodeLsp = await startLanguageServer(undefined, "vscode-css"); | ||
const vscodeLspUri = (await vscodeLsp.openTextDocument(filePath, "css")).uri; | ||
|
||
describe("Hover", async () => { | ||
bench("weblsp - Hover", async () => { | ||
await weblsp.sendHoverRequest(weblspUri, { | ||
line: 1, | ||
character: 6, | ||
}); | ||
}); | ||
|
||
if (!process.env.CODSPEED) { | ||
bench("vscode-css-languageserver - Hover", async () => { | ||
await vscodeLsp.sendHoverRequest(vscodeLspUri, { | ||
line: 1, | ||
character: 6, | ||
}); | ||
}); | ||
} | ||
|
||
afterAll(async () => { | ||
await weblsp.shutdown(); | ||
await vscodeLsp.shutdown(); | ||
await weblsp.exit(); | ||
await vscodeLsp.exit(); | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
import { startLanguageServer } from "../server"; | ||
|
||
declare global { | ||
var languageServer: import("../server").LanguageServerHandle; | ||
} | ||
|
||
if (!globalThis.languageServer) { | ||
globalThis.languageServer = await startLanguageServer(); | ||
} |
1 change: 0 additions & 1 deletion
1
...guage-server-tests/tests/shutdown.test.ts → ...sts-benchmarks/src/tests/shutdown.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our server currently rely on stderr working, otherwise it panics, oops.