Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Protected audience analysis in PSAT #811

Merged
merged 44 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
26caaa0
Add PA analysis in service worker.
amovar18 Aug 8, 2024
f99a184
Add storage.interestGroupAccessed method to service worker.
amovar18 Aug 8, 2024
c5644ff
Add bid value to the auction events.
amovar18 Aug 8, 2024
ace2317
Split syncCookieStore into dataStore, CookieStore and PAStore.
amovar18 Aug 8, 2024
836ac9c
Export interface from dataStore.
amovar18 Aug 8, 2024
b50d673
Change some more references.
amovar18 Aug 8, 2024
7a506b7
Add protected audience provider to the devtools.
amovar18 Aug 8, 2024
bb48995
Convert bid to null.
amovar18 Aug 8, 2024
97e9170
Add last mile connection to the devtools
amovar18 Aug 8, 2024
405d595
Add method to incorporate multiple auction ID.
amovar18 Aug 9, 2024
594af19
Handle multiSeller auctions.
amovar18 Aug 10, 2024
dddcf52
Descructure details in the extension.
amovar18 Aug 14, 2024
73760d3
Refactor service worker.
amovar18 Aug 14, 2024
5229801
Fix errors in the refactor.
amovar18 Aug 14, 2024
6e4d165
Fix lint errors
amovar18 Aug 20, 2024
2b10c1c
Fix tests.
amovar18 Aug 20, 2024
dd8b0d3
Merge branch 'develop' of github.com:GoogleChromeLabs/ps-analysis-too…
amovar18 Aug 21, 2024
64acdab
Move PA types to common package.
amovar18 Aug 22, 2024
fca8fc8
Update devtools protocol package to latest version
amovar18 Aug 22, 2024
bc16572
Move InterestGroup and auctionEvents to common package
amovar18 Aug 22, 2024
097d7e3
Add support to hear globalEvents.
amovar18 Aug 22, 2024
31c9107
Add comment to explain auctionEvents structure.
amovar18 Aug 22, 2024
f819660
Use globalEvents instead of interestGroupEvents.
amovar18 Aug 22, 2024
f1b99ab
compute received Bids and noBids.
amovar18 Aug 23, 2024
4a92868
Fix noBids in the protected audience provider.
amovar18 Aug 26, 2024
5b14df1
Fix nobids value.
amovar18 Aug 26, 2024
de69dea
Add ads and Bidders in protectedAudience API provider.
amovar18 Aug 26, 2024
2a62233
Add adUnitCode optional chaining.
amovar18 Aug 28, 2024
933503d
Fix data being displayed on the panel.
amovar18 Sep 3, 2024
ddd2f6c
Fix data refresh on tab reload.
amovar18 Sep 3, 2024
fbf4f74
Reduce state updates.
amovar18 Sep 4, 2024
786f8e5
Fix error of not updating.
amovar18 Sep 10, 2024
52efd2c
Fix undefined error.
amovar18 Sep 30, 2024
615fce7
Merge branch 'develop' of github.com:GoogleChromeLabs/ps-analysis-too…
amovar18 Sep 30, 2024
9fea26e
Remove logic to reset interest group data on page load or navigation.
amovar18 Sep 30, 2024
30e1bc5
Fix failing tests.
amovar18 Oct 1, 2024
a00a050
Merge branch 'develop' of github.com:GoogleChromeLabs/ps-analysis-too…
amovar18 Oct 8, 2024
5eedfbe
Fix merge conflicts.
amovar18 Oct 8, 2024
228174b
Fix failing build.
amovar18 Oct 8, 2024
75a06f3
Fix failing tests
amovar18 Oct 25, 2024
2100f6e
Fix initial sync override.
amovar18 Oct 29, 2024
b8ba385
Fix failing tests.
amovar18 Oct 29, 2024
649df03
Fix tab reload data refresh.
amovar18 Oct 29, 2024
61328d5
Merge branch 'develop' of github.com:GoogleChromeLabs/ps-analysis-too…
mohdsayed Oct 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions packages/cli/src/e2e-tests/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ describe('CLI E2E Test', () => {

it('Should run site analysis', () => {
return coffee
.fork(cli, ['-u https://httpstatus.us/200', '-w 1000'])
.includes('stdout', '/out/httpstatus-us-200/report_')
.fork(cli, ['-u https://domain-aaa.com/', '-w 1000'])
.includes('stdout', '/out/domain-aaa-com/report_')
.end();
}, 60000);
});
2 changes: 1 addition & 1 deletion packages/common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,6 @@
"tldts": "^6.0.14"
},
"devDependencies": {
"devtools-protocol": "^0.0.1333880"
"devtools-protocol": "^0.0.1345247"
}
}
1 change: 1 addition & 0 deletions packages/common/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,4 @@ export * from './worker/enums';
export * from './utils/generateReports';
export * from './cookies.types';
export * from './libraryDetection.types';
export * from './protectedAudience.types';
86 changes: 86 additions & 0 deletions packages/common/src/protectedAudience.types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* External dependencies
*/
import type { Protocol } from 'devtools-protocol';

export interface singleAuctionEvent {
bidCurrency?: string;
uniqueAuctionId?: Protocol.Storage.InterestGroupAuctionId;
bid?: number;
name?: string;
ownerOrigin?: string;
type: string;
formattedTime: string | Date;
componentSellerOrigin?: string;
time: number;
auctionConfig?: object;
interestGroupConfig?: Protocol.Storage.InterestGroupAccessedEvent;
parentAuctionId?: Protocol.Storage.InterestGroupAuctionId;
eventType:
| 'interestGroupAuctionEventOccurred'
| 'interestGroupAuctionNetworkRequestCompleted'
| 'interestGroupAuctionNetworkRequestCreated'
| 'interestGroupAccessed';
}

export interface auctionData {
[uniqueAuctionId: Protocol.Storage.InterestGroupAuctionId]: {
auctionTime: Protocol.Network.TimeSinceEpoch;
auctionConfig?: any;

Check warning on line 44 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 44 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 44 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 44 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 44 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type
parentAuctionId?: Protocol.Storage.InterestGroupAuctionId;
};
}

export type InterestGroups = singleAuctionEvent & {
details: any;

Check warning on line 50 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 50 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 50 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 50 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

Check warning on line 50 in packages/common/src/protectedAudience.types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type
};

export type MultiSellerAuction = {
[parentAuctionId: string]: {
[uniqueAuctionId: string]: singleAuctionEvent[];
};
};

export type SingleSellerAuction = {
[parentAuctionId: string]: singleAuctionEvent[];
};

export type NoBidsType = {
[auctionId: string]: {
ownerOrigin: string;
name: string;
uniqueAuctionId: string;
adUnitCode?: string;
mediaContainerSize?: number[][];
};
};

export type AdsAndBiddersType = {
[adUnitCode: string]: {
adUnitCode: string;
bidders: string[];
mediaContainerSize: number[][];
};
};

export type ReceivedBids = singleAuctionEvent & {
adUnitCode?: string;
mediaContainerSize?: number[];
};

export type AuctionEventsType = SingleSellerAuction | MultiSellerAuction | null;
2 changes: 1 addition & 1 deletion packages/extension/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
},
"devDependencies": {
"@types/react-copy-to-clipboard": "^5.0.4",
"devtools-protocol": "^0.0.1333880",
"devtools-protocol": "^0.0.1345247",
"html-inline-script-webpack-plugin": "^3.2.1"
}
}
56 changes: 37 additions & 19 deletions packages/extension/src/serviceWorker/attachCDP.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,23 +17,41 @@
* This function will attach the debugger to the given target.
* @param {{ [key: string]: number | string }} target The target where debugger needs to be attached.
*/
export default async function attachCDP(target: {
[key: string]: number | string;
}) {
try {
await chrome.debugger.attach(target, '1.3');
await chrome.debugger.sendCommand(target, 'Target.setAutoAttach', {
// If this is set to true, debugger will be attached to every new target that is added to the current target.
autoAttach: true,
waitForDebuggerOnStart: false,
//Enables "flat" access to the session via specifying sessionId attribute in the commands.
// If this is set to true the debugger is also attached to the child targets of that the target it has been attached to.
flatten: true,
});
await chrome.debugger.sendCommand(target, 'Network.enable');
await chrome.debugger.sendCommand(target, 'Audits.enable');
await chrome.debugger.sendCommand(target, 'Page.enable');
} catch (error) {
//Fail silently
}
export default function attachCDP(target: { [key: string]: number | string }) {
chrome.debugger.attach(target, '1.3', async () => {
if (chrome.runtime.lastError) {
// eslint-disable-next-line no-console
console.warn(chrome.runtime.lastError);
}
try {
await chrome.debugger.sendCommand(target, 'Target.setAutoAttach', {
autoAttach: true,
flatten: false,
waitForDebuggerOnStart: true,
});

await chrome.debugger.sendCommand(
target,
'Storage.setInterestGroupAuctionTracking',
{ enable: true }
);

await chrome.debugger.sendCommand(target, 'Audits.enable');

await chrome.debugger.sendCommand(
target,
'Storage.setInterestGroupTracking',
{
enable: true,
}
);

await chrome.debugger.sendCommand(target, 'Page.enable');

await chrome.debugger.sendCommand(target, 'Network.enable');
} catch (error) {
// eslint-disable-next-line no-console
console.warn(error);
}
});
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
* Internal dependencies
*/
import parseHeaders from '../../utils/parseHeaders';
import synchnorousCookieStore from '../../store/synchnorousCookieStore';
import dataStore from '../../store/dataStore';
import cookieStore from '../../store/cookieStore';
import { getTab } from '../../utils/getTab';

export const onBeforeSendHeadersListener = ({
Expand All @@ -27,26 +28,26 @@ export const onBeforeSendHeadersListener = ({
frameId,
requestId,
}: chrome.webRequest.WebRequestHeadersDetails) => {
if (synchnorousCookieStore.globalIsUsingCDP) {
if (dataStore.globalIsUsingCDP) {
return;
}

(async () => {
const tab = await getTab(tabId);
let tabUrl = synchnorousCookieStore?.getTabUrl(tabId);
let tabUrl = dataStore?.getTabUrl(tabId);

if (tab && tab.pendingUrl) {
tabUrl = tab.pendingUrl;
}

const cookies = await parseHeaders(
synchnorousCookieStore.globalIsUsingCDP,
dataStore.globalIsUsingCDP,
'request',
synchnorousCookieStore.tabToRead,
synchnorousCookieStore.tabMode,
dataStore.tabToRead,
dataStore.tabMode,
tabId,
url,
synchnorousCookieStore.cookieDB ?? {},
dataStore.cookieDB ?? {},
tabUrl,
frameId.toString(),
requestId,
Expand All @@ -58,6 +59,6 @@ export const onBeforeSendHeadersListener = ({
}

// Adds the cookies from the request headers to the cookies object.
synchnorousCookieStore?.update(tabId, cookies);
cookieStore?.update(tabId, cookies);
})();
};
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* Internal dependencies
*/
import { TABID_STORAGE } from '../../constants';
import synchnorousCookieStore from '../../store/synchnorousCookieStore';
import dataStore from '../../store/dataStore';
import getQueryParams from '../../utils/getQueryParams';
import attachCDP from '../attachCDP';

Expand All @@ -36,7 +36,7 @@ export const onCommittedNavigationListener = async ({
}

const targets = await chrome.debugger.getTargets();
const mainFrameId = synchnorousCookieStore?.globalIsUsingCDP
const mainFrameId = dataStore?.globalIsUsingCDP
? targets.filter((target) => target.tabId && target.tabId === tabId)[0]
?.id
: 0;
Expand All @@ -50,19 +50,19 @@ export const onCommittedNavigationListener = async ({
isUsingCDP: queryParams.psat_cdp === 'on',
});

synchnorousCookieStore.globalIsUsingCDP = queryParams.psat_cdp === 'on';
synchnorousCookieStore.tabMode =
dataStore.globalIsUsingCDP = queryParams.psat_cdp === 'on';
dataStore.tabMode =
queryParams.psat_multitab === 'on' ? 'unlimited' : 'single';
}

synchnorousCookieStore?.updateUrl(tabId, url);
dataStore?.updateUrl(tabId, url);

if (url && !url.startsWith('chrome://')) {
synchnorousCookieStore?.removeCookieData(tabId);
dataStore?.removeCookieData(tabId);

if (synchnorousCookieStore.globalIsUsingCDP) {
synchnorousCookieStore.deinitialiseVariablesForTab(tabId.toString());
synchnorousCookieStore.initialiseVariablesForNewTab(tabId.toString());
if (dataStore.globalIsUsingCDP) {
dataStore.deinitialiseVariablesForTab(tabId.toString());
dataStore.initialiseVariablesForNewTab(tabId.toString());

await attachCDP({ tabId });

Expand All @@ -76,11 +76,7 @@ export const onCommittedNavigationListener = async ({
}
);

synchnorousCookieStore.updateParentChildFrameAssociation(
tabId,
targetId,
'0'
);
dataStore.updateParentChildFrameAssociation(tabId, targetId, '0');
}
}
await chrome.tabs.sendMessage(tabId, {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
/**
* Internal dependencies
*/
import synchnorousCookieStore from '../../store/synchnorousCookieStore';
import dataStore from '../../store/dataStore';
import { updateGlobalVariableAndAttachCDP, setupIntervals } from './utils';

export const onEnabledListener = async (
Expand All @@ -26,7 +26,7 @@ export const onEnabledListener = async (
return;
}

synchnorousCookieStore?.clear();
dataStore?.clear();

setupIntervals();

Expand Down
Loading
Loading