Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MWPW-159924 [CAI] | Integrate SDK (PIN & Modal) into Homepage #3210

Open
wants to merge 19 commits into
base: stage
Choose a base branch
from

Conversation

sharmrj
Copy link
Contributor

@sharmrj sharmrj commented Nov 18, 2024

  • CAI-enabled images are identified by looking for the CAI prefix on the img alt text (delimited from the actual alt text by a |)
  • After each section loads, If any image in the section is CAI enabled, an event listener is registered on the img so that on hover the src of the img is stripped of all its query parameters. This makes it so that if someone right clicks on the image and saves it, the image they download has the correct C2PA metadata in it [0].
  • CAI is enabled only on viewports larger than 900px.
  • The cai block is loaded and initialized. An event listener is registered to start preloading metadata when the image is hovered over. This is required since we lazy load all the metadata on demand (to minimize impact to the user and their potentially limited bandwidth and computing power) and this may require network requests to remote manifests.
  • initCAI  is imported from another file, and initializes
    1. the c2pa sdk,
    2. the logic for extracting the required data from the img,
    3. the DOM for the popover,
    4. and event listeners for displaying the popover.
  • In order to initialize the c2pa sdk we host the cai-toolkit.wasm file and the cai-worker.min.js  files in the deps folder in milo. Since most of the computation takes place inside a worker spawned by cai-worker.min.js, we needn't worry about spawning our own worker thread.
  • After the sdk has extracted the metadata (if the extraction fails for any reason we will display an error state on the popover). We need five pieces of information to build the DOM for the popover:
    1. The issuer of the manifest certificate – obtained from the signatureInfo.issuer  field of the activeManifest.
    2. The time the certificate was issued – obtained from the signatureInfo.time  field of the activeManifest.
    3. Information about whether AI was used when making the image – this is obtained by parsing the output of the selectGenerativeInfo(activeManifest) function from the c2pa sdk.
    4. The app used to generate or update the manifest – obtained using the selectFormattedGenerator(activeManifest) function from the c2pa sdk.
    5. AI tools used – obtained by doing [...new Set(selectGenerativeInfo(activeManifest).map(({ softwareAgent }) => softwareAgent))].join(', ') || 'None'; 
  • Currently only en_US is supported

[0] - The query parameters tell the CDN which optimized version of the image needs to be served. Unfortunately the CDN strips away all metadata from the image when optimizing it, so the c2pa metadata is lost. However, without the query parameters the unoptimized image with the correct metadata is pointed to. In order to minimize LCP impact while still allowing the user to download the image and inspect its metadata, we do the first load with the optimized image and swap out the src of the img tag. The source tags are left untouched since only the src of the img tag matters when the user right clicks and downloads the image. This has been tested at different viewport sizes where different source sets should apply.

Analytics will be implemented in a future PR.

Resolves: MWPW-159924

Test URLs:

URL for QA (homepage clone): https://main--homepage--adobecom.hlx.page/homepage/drafts/raghavs/index-loggedout?milolibs=cai--milo--sharmrj
URL for QA (regular milo page): https://cai--milo--sharmrj.hlx.page/drafts/raghavs/cai

Copy link
Contributor

aem-code-sync bot commented Nov 18, 2024

Page Scores Audits Google
📱 /?martech=off PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI
🖥️ /?martech=off PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI
📱 /drafts/raghavs/cai PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI
🖥️ /drafts/raghavs/cai PERFORMANCE A11Y SEO BEST PRACTICES SI FCP LCP TBT CLS PSI

Copy link

codecov bot commented Nov 18, 2024

Codecov Report

Attention: Patch coverage is 89.82036% with 17 lines in your changes missing coverage. Please review.

Project coverage is 96.36%. Comparing base (6f4d98c) to head (09e91fe).

Files with missing lines Patch % Lines
libs/blocks/cai/cai.js 86.23% 15 Missing ⚠️
libs/utils/utils.js 96.55% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##            stage    #3210      +/-   ##
==========================================
- Coverage   96.37%   96.36%   -0.02%     
==========================================
  Files         245      246       +1     
  Lines       56716    56883     +167     
==========================================
+ Hits        54661    54815     +154     
- Misses       2055     2068      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@spadmasa spadmasa self-assigned this Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants