VC Profile: transparency

[SimoneZaza]

Description

To build trust in our AI offering, we must provide transparency into the technology powering each Virtual Contributor (VC). By offering users insights into each VC’s capabilities and data handling practices, we empower them to make informed decisions about which VCs to invite into their spaces.

Goal

Equip users with the information needed to make conscious, informed choices about inviting specific VCs into their environment. This transparency ensures users understand each VC’s functionality, limitations, and data handling, fostering confidence and trust in our AI solutions.

Acceptance Criteria

Setup: Answers below, are for now hardcoded in the client per engine

Client work

Left side of the page (see design)

Add block on the left side of the page, below Description with:

• Title: Body of Knowledge
• If BoK is in a Space, keep current functionality:
  • Text: Answers (vc name) gives are based on the body of knowledge coming from:
  • Space card for subspace that's being used
• If BoK is in the VC profile:
  • Text: input field for the user
  • Edit icon visible for people with the update vc privilege
  • Button to visit the BoK
    • If the BoK is private, disable button and show tooltip: "This Body of Knowledge is private"
• If the VC is based on an external engine and there is no BoK:
  • Text: This Virtual Contributor utilizes (engine name in bold), an external AI engine. As a result, the Body of Knowledge is not accessible.
  • No button

Right side of the page

• Functionality block with 3 blocks inside, see design
  • Block 1: Functional Capabilities
    • List of capabilities this VC either has or doesn't have. All are shown, with check or minus in front. Limited set for now, but will grow in the future
  • Block 2: Data access from the Space where it’s a member
    • List of data access this VC either has or doesn't have. All are shown, with check or minus in front. Limited set for now, but will grow in the future
  • Block 3: Role Requirements
    • Text with in bold what role is required for this VC to function correctly
• Transparency block with 6 blocks inside
  • Title: AI Engine: [engine name]
  • 4 blocks with icon, text, and yes/no "answer". This yes/no is combined with an icon, either a check or !. NB yes and no don't always get the same icon! See below when yes is a check (v) and when an exclamation mark (!)
    • Open Model Transparency: Yes (v) / No (!) / Unknown (!)
    • Data usage disclosure: Yes (!) / No (v) / Unknown (!)
    • Knowledge restriction: Yes (v) / No (!) / Unknown (!) / Yes, if present (!)
    • Web Access: Yes (!) / No (v) / Unknown (!)
  • 1 block for the location
  • 1 Block for the technical references with a button that opens these in a new tab
• Data usage by Alkemio block (static')
  • Title: Monitoring by Alkemio
  • Text: Since Alkemio facilitates the interaction with the external provider, it holds an operational responsibility to monitor the service. As with all data and interactions on the platform, these are governed by our Terms & Conditions (link to T&C in new tab).

3 Important things to note:

1. Giving transparency is the main point here.
2. All variables written above will be fixed per AI Engine for now. Functionality will probably be attached to the VC itself at some point
3. This is a learning journey, and many iterations will probably follow. Please take this into consideration when making judgement calls about the effort going into replicating the design, it doesn't have to be pixel-perfect.

Details per engine

Alkemio AI

• Open model: Yes
• Data usage disclosure: No
• Knowledge restriction: Yes
• Web access: No
• Physical location: Sweden, EU
• Technical References: https://huggingface.co/mistralai/Mistral-Small-Instruct-2409/tree/main
  NB also use this one for the Guidance engine!

OpenAI Generic

• Open model: No
• Data usage disclosure: Unknown (product team: check T&C openAi)
• Knowledge restriction: No
• Web access: Yes
• Physical location: Unknown
• Technical References: https://platform.openai.com/docs/overview

OpenAI Assistant

• Open model: No
• Data usage disclosure: Unknown (product team: check T&C openAi)
• Knowledge restriction: Yes, when provided
• Web access: No
• Physical location: Unknown
• Technical References: https://platform.openai.com/docs/assistants/overview

Stakeholders

Product Team, Legal Team

Design

Figma: https://www.figma.com/design/WFcbewbnhPqYpGBzBJaPq1/VC-Profile-Settings?node-id=209-718&t=7vyCWcBwsu8tmEwW-1
Original whiteboard: Derived from the whiteboard: https://alkem.io/building-alkemio-org/challenges/virtualcontributors-54/collaboration/vcprofile

[bobbykolev]

When you are implementing the profile, have in mind the Invite VC dialog, where we'll need to use the blocks in different order.

[reactoholic]

7654 closed as it's closely related to this task.
I will implement it with this ticket which makes the estimate higher but it doesn't make sense to keep both tickets.

CC: @SimoneZaza , @bobbykolev

[Comoque1]

Tested on DEV env.

Feedback

1. Left side, BoK Block
   If the VC is based on an external engine and there is no BoK:
   Text: This Virtual Contributor utilizes (engine name in bold), an external AI engine. As a result, the Body of Knowledge is not accessible. - missing the engine name
2. Left side, BoK Block
   For knowledge-based VC description is missing until the description is saved/updated

3. Right side of the page; Block 2: Data
   1. Data is hard-coded - should come from the type of the VC (refer to Details per engine)
   2. Review text color
4. Icons - @reactoholic , icons are not as per design.

[reactoholic]

@Comoque1 Thanks for the feedback! Bobi and I decided to use MUI icons. If an icon from MUI isn't the same as the one in the design - use similar from MUI. The only icon coming from the design is the cogwheel with globus. Everything else I am fixing right now.

PS: The Missing description in the Body of Knowledge block is not related to the current implementation. The description is there ones the dialog is opened and saved. The issue is coming on VC creation, e.g., the default description isn't saved along with the VC creation, just its name. I will fix this with this ticket since in theory it should be rather quick fix.

Another issue we currently have when opening the profile of an External VC is in the useVirtualContributorQuery() where the bodyOfKnowledgeID property comes empty which is not okay, because we need it for fetching the BOK Profile from useBodyOfKnowledgeProfileQuery(), thus the bokProfile comes undefined and we can't show anything to the user as a description to our users. I will coordinate this with Vladi.

All of these things are making the task bigger than 8 but all of them are mandatory to fix.

CC: @SimoneZaza @bobbykolev

[SimoneZaza]

@reactoholic Thanks for elaborating, indeed the idea is to use MUI icons. We've had small differences between Material Symbols and MUI icons (=Material icons) before, but not to this extent. Let's hope MUI soon starts supporting symbols, but until then, these icons are fine by me.

[Comoque1]

Overall looks good
Tested with the following setups:

• roles:

  • Unauthenticated - always redirected to sign up/in page
  • Authenticated no membership - OK
  • Member of Space based on which knowledge is based - OK
  • Admin - OK

• VCs:

  • Space based knowledge - combinations public/private space/subspace - OK
  • Own knowledge - public/private knowledge - OK
  • External - OK
  • Assistant - OK

Findings:
Left side, BoK Block
For knowledge-based VC description is missing until the description is saved/updated

The cases, when we have: Unknown (!) / Yes, if present (!) are not properly handled

1 Bug raised:
BUG: When user opens VC based on subspace of a private space to which the user is not member, get Space name as BoK, but url behind it is to the VC itself #7806

@SimoneZaza if the findings are not acceptable, I'll move the story back to backlog. If acceptable - I can raise tasks?

[Comoque1]

After discussion with @SimoneZaza the second point: The cases, when we have: Unknown (!) / Yes, if present (!) are not properly handled - must be addressed.

[Comoque1]

Tested OK on DEV env.