[bounty] $200 to make the onboarding more intuitive / smooth / less buggy

[louis030195]

please first suggest some improvements

also i know there are some issues on some OSes (windows) with the screenpipe setup command? check other issues

there are some improvements to make the onboarding more intuitive, smooth, useful, idk what, i'm not officially a designer

maybe show the data coming in idk

lets bounce around ideas

/bounty 200

[linear]

MED-363 [bounty] $100 to make the onboarding more intuitive / smooth / less buggy

[algora-pbc]

💎 $200 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #810 with your implementation plan
2. Submit work: Create a pull request including /claim #810 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @oliverqx		#852

[tribhuwan-kumar]

I'll show you some prototype or figma design later!

[oliverqx]

So I mapped out current flow:

Red notes contain things I would change. Blue shows new ideas. Green shows things I liked.

This is the redesign, really just ideation atm.

[oliverqx]

I wanted the design of onboarding to be cohesive with the entire user flow which I mapped out here:

https://www.figma.com/board/7ktNX4KGbn9cmQhOHAJI5n/Untitled?node-id=4-1680&node-type=section&t=0kp552FshjHVo8Ke-0

imo the video is a great short and sweet summary of the landing page journey. Needs a bit of improvement, background music and recording my voice + some distortion would go a long way. Also needs compression or something cause its lagging in macos desktop app.

Bugs would be reduced by updating state management through the implementation of a state machine. No external libraries but rather a simple custom one which travels through an object that represents this process.

A thing I did notice was that the setup feels a bit like a void. Would be nice to show status, as seen in my proposal. Status of the command would be shown next to a visual representation of how screenpipe works and tips.

I'd personally remove the tips panels, have them be a component on the main page. Kinda like this:

Thing is that Screenpipe's design is very text heavy, which i do enjoy, but at this point the user has been seeing a lot of text heavy screens from their journey which starts at the landing page. Feels like these tips would be ignored, and are way more useful and easily accesible this way. When clicked it would open a modal with the tips, while showing one a time on a marquee

[oliverqx]

Also, a finish line is missing. The modal just suddenly disappears, I would add a panel at the end witih MagicUIs confetti effect behind screenpipes logo and some message. The modal would fade away

[oliverqx]

@louis030195 let me know what you think, i can start working on it asap

[tribhuwan-kumar]

@oliverqx that's a great remodification, but i was thinking to add some more slides or a some text maybe, explaining about some new features of screenpipe (eg. timeline) there are also some major changes aren't covered in onboarding. it'd be great if you could cover those changes and i noticed that channel log dialog and onboarding is overlapping each other in the startup of app. anyway let me know if you need any help!

[oliverqx]

that sounds good.

I was thinking there could be a separate modal for new features. it'd pop up whenever a new feature is available instead of doing it in this one which is focused on setting things up. These new features could also be displayed in the tips component. Making it more akin to a news feed component that can show information from several sources tips/pipestore/features etc

[tribhuwan-kumar]

that sounds good.

I was thinking there could be a separate modal for new features. it'd pop up whenever a new feature is available instead of doing it in this one which is focused on setting things up. These new features could also be displayed in the tips component. Making it more akin to a news feed component that can show information from several sources tips/pipestore/features etc

that'd be great. also the sign in option should include in onboarding

right now, its in settings & its kinda hard to find.

[oliverqx]

aah that makes a lot of sense yeah

[louis030195]

great ideas, some thoughts:

• agree main page has too much text, too many ? things (the UI should be self explanatory, no need docs, just intuitive, eg apple style)
• regarding the video, honestly i dont even know which is this one atm, any recommendation? i can do the video
• status/setup button is using the screenpipe setup command - maybe this command could stream some checks like mic check, download models, etc - i like screenstudio onboarding (button clicks to enable permissions of screen, mic, etc.)
• i think in general something missing is people want to see the data flowing in, things recorded, but maybe logs is enough (could also show a sql/supabase like table) - maybe just for devs idk
• also just an idea for dev it would be nice to get them started quickly to build pipes using bunx @screenpipe/create-pipe@latest like a codeblock or maybe embedded terminal or auto open terminal if that's something possible
• should we suggest "add these pipes!" or something like this? atm most pipes are experimental except loom one (paid) and keyword which works well

also some more context:

we're going to move search UI, timeline, and meeting pages in the pipe store, so screenpipe UI will be only the pipe store

also we're likely going to give access to pipes the general app settings (so AI settings etc, only sat once)

and some things in the future that will need to be added to onboarding but it's fine for now:

• stripe integration for devs

that's a bit messy feedback, hope it's useful

[oliverqx]

okay i think i see a path here. These would be the steps:

1. Create a flow entity which represents the onboarding flow. This object would be easily extendable and editable. Reordering and creation of steps would be as simple as changing index in the array or extending the array.
2. Create state machine to traverse the onboarding flow
3. Update setting up step, making it akin to screenstudio onboarding
4. Create visual representation that explains how screenpipe works under the hood, this would replace the how it works section.
5. Add login step
6. Add install pipes step
7. Add create pipes step to devMode process

I'm done with step 1 and almost done with 2. Once 2 is complete, steps 5-7 are pretty much done as well because adding them wont be complicated.

I think i can be done sometime tomorrow although im not familiar with how screenstudio onboarding works under the hood. I was thinking of simply showing the stout that is already printed when running screenpipe setup

Implementing a different UX might require some work that id deem out of scope if its too complex, considering the bounty is only 100$. I will give it a try though, sounds like it shouldnt take that much time. im just worried about how time consuming cross-platform permissions can get.

The visual representation I was thinking could be something like this: https://magicui.design/docs/components/animated-beam

WIth input being devices being recorded (screen,keyboard,etc) and output would be either search or pipes, which themselves have another output.

Also havent been able to see the linkedin pipe video, but that could be a cool use case to show. I think the current one is great too, but I think having several short videos showing use cases would be useful

[oliverqx]

Essentially once im done the onboarding flow will have no bugs, be updated to include some new steps and reorganized for a better ux, but most importantly it would be really easy to extend it or reorganize it in the future

The flow entity and state machine could then be used to render a new features dialog, even remotely updating its content as the flow itself could be stored in a db and retrieved whenever the app is opened.

[algora-pbc]

💡 @oliverqx submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[louis030195]

@oliverqx nice!

i increased bounty to $200

also i was considering to expose specific subcommand or something to trigger specific permissions

i think screenpipe on macos need:

• screen + audio (main features)

(i always use screenpipe in CLI through alacritty terminal)

• accessibility (UI monitoring and keyboard/mouse control, atm under feature flag, and soon keyboard recording $200 [metadata] keyboard recording #10)

i don't understand this permission:

not sure we need it actually

so i could build

screenpipe setup screen

screenpipe setup mic

screenpipe setup accessibility

and maybe

screenpipe setup ai -> download models

essentially this is screen studio onboarding roughly

wdyt?

i think hardest part is this:

user click allow and it would restart the app - making sure it opens at the right time in the onboarding back

what if user deny? or click away

ofc permission is only for macos - windows is openbar and same for linux

on windows the equivalent issue is windows defender #321 but it's unrelated to this task imho (we just need to sign the build or beg microsoft to whitelist us)

[louis030195]

holy shit ... just discovered the long issue i had with microphone transcription was that my permission in audio was off

so this permission is important indeed

[oliverqx]

@louis030195 yeah i was imagining something like that in terms of wireframe. I can start prototyping the ui

the flow state machine atm is not persistent but saving progress would require only a store update. Kinda like settings but probably best if its another store. Making progress persistent would allow us to manage the reopening of the app. Once its open we could trigger a healthhcheck on whatever permission was just granted.

would be awesome if you could create the cli commands, i was thinking maybe tauri had an api for that but it does make more sense for the cli to manage that. Im not thaaat familiar with rust, but ive used it in the past. If you make one of the commands im pretty sure i can take care of the others

[oliverqx]

oh i didnt know other os's dont require permission thats wild. im guessing if its linux the user would be capable enough to troubleshoot any issues if it comes to that. I think windows has a microphone privacy setting since windows 10. According to some quick reddit reading i just did: The user must enable microphone access for apps in Settings > Privacy > Microphone. I imagine there must be a way to trigger that from cmd or if platform === windows we show that as an instruction

[louis030195]

update: going to make these permission button with rust code now in the status that you can reuse in onboarding

[oliverqx]

so im already working on the prototypes. had to do some work related stuff so its not done yet:

figma file

I have ideas for 2 different UXs but only one is mapped out rn. tomorrow morning i'll record a video showing the 2 different UXs.

the one mapped out rn is very similar to screenstudio. The other one would be the same but each of the 4 tasks (screen,audio,accessibility,ai models) would fade away once they're done. This way by the time the user downloads ai models theres enough space to show download status and the illustration

[tribhuwan-kumar]

@oliverqx
loved your design, the ai model download option would help me to fix this #682

[m13v]

looks good @oliverqx <https://github.com/oliverqx> !

…

On Fri, Dec 6, 2024 at 9:49 PM tribhuwan ***@***.***> wrote: @oliverqx <https://github.com/oliverqx> loved your design, the ai model download option would help me to fix this #682 <#682> — Reply to this email directly, view it on GitHub <#810 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY62CDDZNMVJSRZYBQJEH332EKD7VAVCNFSM6AAAAABSYVQMKSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRUHE2TCOJQGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[louis030195]

added permission buttons in status

[oliverqx]

hey guys i have an idea, recorded a short loom video explaining it. Imma make a prototype in figma but wanted to share the idea to get some feedback as I work on the prototype

gamified onboarding

video is 3min long, but tldr:

we can gamify the experience of user onboarding

instead of having a panel based onboarding, the onboarding would happen through this interactive illustration. There would be a text box at the top left corner were screen pipe is talking to the user walking them through the steps, kinda like this type of games:

complex steps like api key configuration or pipe installation would trigger dialogs that are dismissed once the step is successful or skipped.

The illustration is a react component so it could also be part of the status dialog, showing the user which component is not working by highlighting it with danger/warning colors and/or ! icons

[oliverqx]

cc: @louis030195 @m13v @tribhuwan-kumar

[oliverqx]

the prototype for a gamified onboarding flow is ready. i also recorded a short loom video going through it. i know not everyone likes gamified experiences but im pretty confident this has a lot of potential to make people understand whats going on in a very visual way. could easily be extended with more complex components to show data structures, db interactions ando other more technical details

figma file

loom video

im also slowly building a figma design system for screenpipe with components, variables and other things that ease design proposals. could come in handy in the future

[tribhuwan-kumar]

@oliverqx
a gamified onboarding would be a great implementation for onboarding and status. I don't see any drawbacks in it.

anyway, it depends upon @louis030195 what he wants

[louis030195]

@oliverqx nice!

we're going to move search page, meeting page, and timeline, to the pipe store / app store

for the AI settings just ask to set up this, bit like cursor does it maybe, basically first option should be screenpipe cloud (which is openai atm) - ideally we should have drop downs for models input actually, like cursor

for the schema i could see more something like this

we're going to change naming from "pipes" to "apps" because many people did not understand

and screenpipe's app store

not sure where the AI providers should be positioned exactly

also not sure you included step to make the user run the embedded AI?

give option to skip onboarding (i personally always skip onboardings, maybe more technical people do this)

other than that keep it simple