Add support for external auth providers in code search #6919

pkukielka · 2025-02-03T16:14:20Z

Changes

This PR adds agent endpoint which allow us to fetch auth headers, which in turn we need to deliver to the sourcegraph search webview.

Webview changes are on a separate PR: pkukielka/sourcegraph-public-snapshot#1

Test plan

Setup:

Clone [email protected]:pkukielka/sourcegraph-public-snapshot.git and checkout pkukielka/add-external-auth-providers-support branch.
export CODE_SEARCH_DIR_OVERRIDE=/path/to/pkukielka/sourcegraph-public-snapshot
Checkout this PR (pkukielka/ext-auth-fix-search branch) in Cody project dir
cd jetbrains
Run ./gradlew customRunIde -PforceCodeSearchBuild=true -PforceAgentBuild=true
Add external auth provider configuration:

{
  "cody.override.serverEndpoint":  "https://piotr-kukielka.sgdev.dev",

  "cody.auth.externalProviders": [
    {
      "endpoint": "https://piotr-kukielka.sgdev.dev",
      "executable": {
        "commandLine": ["echo '{ \"headers\": { \"Authorization\": \"token sgp_local_YOUR_TOKEN\" } }'"],
        "shell": "/bin/bash"
      }
    },

Test

Hit Option + S to open sourcegraph search
Search for cody repo and click Open in Browser
Make sure https://piotr-kukielka.sgdev.dev instance is opened
Comment out "cody.override.serverEndpoint" line and then switch to other account, e.g. on https://sg02.sourcegraphcloud.com/ (this one will authenticate using token), save settings
1. Search for cody repo and click Open in Browser
Make sure https://sg02.sourcegraphcloud.com/ instance is opened

Reproducing failing fetches:

Edit config by changing commandLine line to:
"commandLine": ["echo '{ \"headers\": { \"X-Forwarded-User\": \"SomeUser\" } }'"],
Open Find Actions: (Ctrl+Shift+A / ⌘⇧A)
Search for "Registry..." and open it
Find option ide.browser.jcef.debug.port
Change the default value to an open port (we use 9222)
Restart IDE
Hit Option + S to open sourcegraph search
Notice sourcegraph search is not loading anymore
Go to http://localhost:9222/ and click on Sourcegraph link to inspect the failures in the devtools

mkondratek

code looks good. I am going to test it

mkondratek · 2025-02-07T12:11:50Z

jetbrains/build.gradle.kts

@@ -343,7 +343,9 @@ tasks {
      return destinationDir
    }

-    val sourcegraphDir = unzipCodeSearch()
+    val codeSearchDirOverride = System.getenv("CODE_SEARCH_DIR_OVERRIDE")


mkondratek · 2025-02-07T12:17:56Z

lib/shared/src/sourcegraph-api/utils.ts

        }
    }
+    return {}


previously we had a console.error. do we need it? or is {} a valid option?

mkondratek · 2025-02-07T12:21:32Z

jetbrains/src/main/kotlin/com/sourcegraph/cody/config/notification/CodySettingChangeListener.kt

@@ -19,8 +19,6 @@ class CodySettingChangeListener(project: Project) : ChangeListener(project) {
        CodySettingChangeActionNotifier.TOPIC,
        object : CodySettingChangeActionNotifier {
          override fun afterAction(context: CodySettingChangeContext) {
-            // Notify JCEF about the config changes


don't we need to refreshConfiguration now?

That config was already not containing any info useful for sourcegraph search.
We care about endpoint and headers.
Endpoint change is signalled in CodyAuthService.
Headers are fetched every time so we do not need to notify about any change in their case.

mkondratek · 2025-02-07T12:23:04Z

jetbrains/src/main/java/com/sourcegraph/find/browser/HttpSchemeHandler.java

+    } catch (Exception ignored) {
+      return false;
+    }


is it safe to ignore it and return false? does it make sense to log/warn here?

pkukielka · 2025-02-13T10:11:47Z

jetbrains/build.gradle.kts

@@ -319,10 +343,9 @@ val pnpmPath =
    }

 tasks {
-  val codeSearchCommit = "9d86a4f7d183e980acfe5d6b6468f06aaa0d8acf"
+  val codeSearchCommit = "048679a2e7f2a2d94a49c46bc972c954cb2b5bac"


This points to https://github.com/sourcegraph/sourcegraph/tree/release/jetbrains

mkondratek · 2025-02-13T11:12:20Z

.github/workflows/release-jetbrains-experimental.yml

@@ -41,3 +41,4 @@ jobs:
          popd > /dev/null
        env:
          PUBLISH_TOKEN: ${{ secrets.JETBRAINS_MARKETPLACE_PUBLISH_TOKEN }}
+          GITHUB_TOKEN: ${{ secrets.PRIVATE_SG_TOKEN }}


dominiccooney

Minor nit inline, looks great! What a load of work that turned out to be.

dominiccooney · 2025-02-13T12:55:53Z

jetbrains/build.gradle.kts

-  assert(output.parentFile.mkdirs()) { output.parentFile }
-  Files.copy(URL(url).openStream(), output.toPath())
+
+  // Optional: Retrieve the GitHub token if needed for authorization (require private repo access)


Suggested change

// Optional: Retrieve the GitHub token if needed for authorization (require private repo access)

// Optional: Retrieve the GitHub token if needed for authorization (requires sourcegraph/sourcegraph repo access)

dominiccooney · 2025-02-13T12:56:51Z

jetbrains/build.gradle.kts

-  assert(output.parentFile.mkdirs()) { output.parentFile }
-  Files.copy(URL(url).openStream(), output.toPath())
+
+  // Optional: Retrieve the GitHub token if needed for authorization (require private repo access)


Please document a bit about this wrinkle in CONTRIBUTING.md.

dominiccooney · 2025-02-13T13:00:58Z

lib/shared/src/sourcegraph-api/utils.ts

+export async function addAuthHeaders(auth: AuthCredentials, headers: Headers, url: URL): Promise<void> {
+    await getAuthHeaders(auth, url).then(authHeaders => {
+        for (const [key, value] of Object.entries(authHeaders)) {
+            headers.set(key, value)


You can do Object.assign for this and avoid the loop.

…network hiccups better

pkukielka marked this pull request as draft February 3, 2025 16:14

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from dfc3ef0 to aa904c1 Compare February 6, 2025 15:46

pkukielka changed the title ~~Pass custom headers to the search plugin~~ Add support for external auth providers in code search Feb 6, 2025

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from aa904c1 to 4a47767 Compare February 7, 2025 09:04

mkondratek approved these changes Feb 7, 2025

View reviewed changes

pkukielka commented Feb 13, 2025

View reviewed changes

pkukielka added 4 commits February 13, 2025 11:12

Pass custom headers to the search plugin

45d4765

Add support for external auth providers in code search

8063b96

Fixes for review

09c36d4

Fix crashes in 2023.2

97d7959

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from 2402bc9 to 33d8a88 Compare February 13, 2025 10:12

pkukielka marked this pull request as ready for review February 13, 2025 10:13

Switch to fetching sourcegraph search from sourcegraph/sourcegraph

3576de9

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from 33d8a88 to 3576de9 Compare February 13, 2025 10:17

mkondratek reviewed Feb 13, 2025

View reviewed changes

mkondratek approved these changes Feb 13, 2025

View reviewed changes

dominiccooney approved these changes Feb 13, 2025

View reviewed changes

Fixes for review

b40635d

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from 539846b to b52c2f4 Compare February 13, 2025 17:32

Fixes for tests

ca403b4

pkukielka force-pushed the pkukielka/ext-auth-fix-search branch from b52c2f4 to ca403b4 Compare February 14, 2025 08:31

Bump version of gradle wrapper validation action which should handle …

bc8ddaf

…network hiccups better

pkukielka merged commit 86d940a into main Feb 14, 2025
20 of 21 checks passed

pkukielka deleted the pkukielka/ext-auth-fix-search branch February 14, 2025 09:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for external auth providers in code search #6919

Add support for external auth providers in code search #6919

pkukielka commented Feb 3, 2025 •

edited by dominiccooney

Loading

mkondratek left a comment

mkondratek Feb 7, 2025

mkondratek Feb 7, 2025

mkondratek Feb 7, 2025

pkukielka Feb 7, 2025

mkondratek Feb 7, 2025

pkukielka Feb 13, 2025

mkondratek Feb 13, 2025

dominiccooney left a comment

dominiccooney Feb 13, 2025

dominiccooney Feb 13, 2025

dominiccooney Feb 13, 2025

	// Optional: Retrieve the GitHub token if needed for authorization (require private repo access)
	// Optional: Retrieve the GitHub token if needed for authorization (requires sourcegraph/sourcegraph repo access)

Add support for external auth providers in code search #6919

Add support for external auth providers in code search #6919

Conversation

pkukielka commented Feb 3, 2025 • edited by dominiccooney Loading

Changes

Test plan

mkondratek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dominiccooney left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkukielka commented Feb 3, 2025 •

edited by dominiccooney

Loading