Live2D with Lipsync (Outdated) #117
Conversation
Changes: 1. updated model.motion(group, index, priority) to model.motion(group, index, priority, sound, volume, expression); 2. added model.stopSpeaking() 3. updated readme.MD with demos 4. Workflow will save files to dist (won't be gitignored) 5. Praying this new change doesn't break
voice volume expressions are now optional arg {name: value, ....}
| #### Feel free to support the Maintainer: | ||
| <a href="https://www.buymeacoffee.com/RaSan147" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a> | ||
|
|
There was a problem hiding this comment.
It's okay to put this in your fork, but not into the main repository.
There was a problem hiding this comment.
Will make another fork 😅 copy pasted it in all my repo. Sorry.
| @@ -97,16 +100,24 @@ import { Live2DModel } from 'pixi-live2d-display/cubism2'; | |||
| import { Live2DModel } from 'pixi-live2d-display/cubism4'; | |||
| ``` | |||
|
|
|||
| #### Via CDN | |||
| #### Via CDN (lipsync patched) | |||
There was a problem hiding this comment.
This and the (not lipsync patch) above is unnecessary after this feature is merged.
README.md
Outdated
| <script src="https://cdn.jsdelivr.net/gh/RaSan147/[email protected]/dist/index.min.js"></script> | ||
|
|
||
| <!-- if only Cubism 2.1 --> | ||
| <script src="https://cdn.jsdelivr.net/npm/pixi-live2d-display/dist/cubism2.min.js"></script> | ||
| <script src="https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v0.4.0-ls-2/dist/cubism2.min.js"></script> | ||
|
|
||
| <!-- if only Cubism 4 --> | ||
| <script src="https://cdn.jsdelivr.net/npm/pixi-live2d-display/dist/cubism4.min.js"></script> | ||
| <script src="https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v0.4.0-ls-2/dist/cubism4.min.js"></script> |
There was a problem hiding this comment.
These links should point to the main repository.
src/cubism-common/SoundManager.ts
Outdated
| @@ -1,7 +1,7 @@ | |||
| import { logger, remove } from '@/utils'; | |||
|
|
|||
| const TAG = 'SoundManager'; | |||
| const VOLUME = 0.5; | |||
| const VOLUME = 0.9; | |||
There was a problem hiding this comment.
This should not be changed. 0.9 would be too loud as a default volume.
There was a problem hiding this comment.
However since the lip movement value is dependent on the audio volume, lowering it makes unusual movement (like mumbling)
There was a problem hiding this comment.
Is it possible to add a multiplier after getting the value? Some thing like amplitude *= 2
There was a problem hiding this comment.
nope, like if you set volume to 0, there will be no response, no matter how the audio or multiplier is (Browser JS API, cant change).... But yeah multiplier can be really great, will try to add one.
There was a problem hiding this comment.
tried, sending Amp variable to analizer is hard. Let it be...
src/cubism-common/SoundManager.ts
Outdated
| @@ -39,6 +41,8 @@ export class SoundManager { | |||
|
|
|||
| audio.volume = this._volume; | |||
| audio.preload = 'auto'; | |||
| audio.autoplay = true; | |||
There was a problem hiding this comment.
autoplay should not be true because we are not immediately playing the audio. When config.motionSync is on, we'll postpone it until the related motion/expression is ready.
src/cubism-common/SoundManager.ts
Outdated
| @@ -39,6 +41,8 @@ export class SoundManager { | |||
|
|
|||
| audio.volume = this._volume; | |||
| audio.preload = 'auto'; | |||
| audio.autoplay = true; | |||
| audio.crossOrigin = "anonymous"; | |||
There was a problem hiding this comment.
crossOrigin should not be set by default, otherwise if the audio's source is cross-origin but the server does not respond with a CORS header, the audio will be blocked by the browser.
It should be an option, for example, model.speak('audio.mp3', { crossOrigin: true })
src/cubism2/Cubism2InternalModel.ts
Outdated
| @@ -33,9 +34,12 @@ export class Cubism2InternalModel extends InternalModel { | |||
| angleZParamIndex: number; | |||
| bodyAngleXParamIndex: number; | |||
| breathParamIndex: number; | |||
| mouthFormIndex: number; | |||
There was a problem hiding this comment.
I do remember it using somewhere in cubism 2, cubism 4 has it by default. It takes the mouthForm id...
| updateFacialEmotion(mouthForm: number) { | ||
| this.coreModel.addParameterValueById(this.idParamMouthForm, mouthForm); // -1 ~ 1 | ||
| } |
There was a problem hiding this comment.
This is not related to lip sync, I suppose?
There was a problem hiding this comment.
Thats the main thinggggg. 😅 It makes the lips move.
|
Sorry for the late reply. |
RaSan147
changed the title
Co-authored-by: Guan <[email protected]>
Co-authored-by: Guan <[email protected]>
Co-authored-by: Guan <[email protected]>
also remove cache buster and autoplay
also updated package+version info fixed NodeEnv issue
Update Readme
Improve example code quality in update README ZH and EN
update README.zh
No description provided.