Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[iOS] Dictation #2710

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

mykhailonehelia
Copy link

fixes #2119

@mykhailonehelia mykhailonehelia changed the title [iOS] Dictation #2119 [iOS] Dictation Dec 18, 2024
Copy link
Contributor

@raineorshine raineorshine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks like a good start.

The only issue is that this will incorrectly trigger when the user types "new thought" on the keyboard. We don't want this triggering when the user types or edits a thought. It should only trigger during dictation.

@@ -346,6 +346,20 @@ const Editable = ({
),
)

if (newValue.toLowerCase().indexOf(' new thought') > 0) {
const [prev, next] = newValue.split('new thought')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The split should also be case insensitive.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

setTimeout(() => {
dispatch(
newThought({
at: rootedParentOf(state, path),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should omit at. It will create the new thought after the cursor thought by default. Creating the new thought at the end of the parent doesn't make sense.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

@@ -346,6 +347,19 @@ const Editable = ({
),
)

if (inputMethod === 'dictation' && newValue.toLowerCase().indexOf(' new thought') > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would result in a false positive when "new thought" is typed on the keyboard and then the user enters dictation mode. The intended behavior is for "new thought" and other commands to only be transformed when the user is dictating.

When you type input event usually fired every 1-4 characters, but when you dictate whole `new thought` text appears at once
Copy link
Contributor

@raineorshine raineorshine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! This is an improvement, as typed "new thought" no longer triggers the dictation command. However, I'm finding that it fails to detect "new thought" in the case where dictation inserts "new" and "thought" in separate events, which is quite common. When I practiced dictating I found that I got a lot of false negatives.

I think you will need to look at the full value of the thought to pick up on these cases where "new" and "thought" are input by the dictation feature as separate words. However, you also need to avoid false positives and ignore "new thought" that was previously typed. Therefore, a better approach might be to record the selection offset when dictation starts so that you can exclude existing text from detection.

e.g. If you have thought Hello| World (with the caret in between "Hello" and "World") and you start dictating, then value.replace(/^Hello/, '').replace(/ world$/, '') will give you the dictated text excluding text that was already there.

While the expected functionality is fairly intuitive when using the app, it can be helpful to spell it out more explicitly so that you can review each of these cases and ensure that your solution works as expected:

  • Trigger the New Thought command when "new thought" is dictated.
  • Trigger the New Thought command when "new [pause] thought" is dictated.
  • Do not trigger New Thought when "new thought" is typed.
  • Do not trigger New Thought when dictating new text on a thought that already contains the text "new thought".

Thanks and let me know if I can clarify anything else!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[iOS] Dictation
2 participants