New (suspected) NAI model leak - AnythingV3.0 and VAE for it

[aliencaocao]

https://huggingface.co/Linaqruf/anything-v3.0/tree/main

Some comparisons: questianon/sdupdates#1 (comment)
For those that understands chinese, here are more comparisons: https://www.bilibili.com/read/cv19603218

I have also been trying it out and it indeed seems to be better than the animefull one, especially for hands holding stuff and backdrop. Note that it works with the animefull VAE and original SD VAE, just like the NAI model. In fact, the Anything VAE is exactly the same as NAI VAE.

This model should be a finetuned version of the original NAI animefull model. All embedding and hypetnetworks designed for original NAI model should work on this.

[NaughtDZ]

The test result of a user generated image shows that the effect of the generated image is much better than the previously leaked model.
A user claims that the file scanned by "microsoft defender" indicates that there is a Trojan virus in the ckpt file. The specific location is "archive data 846". After verification by virtotal, it is likely to be a false positive. However, please take your own risk. For example, I like adventure, XD.
However, according to the user "Relic-0772" of Bilibili, this model is not a leaked model, but a model trained by others.
And,however again, waiting for Waifu Diffusion 1.4 is also a good choice for generating animation pictures.
https://gist.github.com/harubaru/313eec09026bb4090f4939d01f79a7e7

[aliencaocao]

I have tried WD1.3 and all I can say is NAI >>>>>> WD1.3

[Kilvoctu]

I like NAI for its consistency in quality, but I feel WD1.3 is more flexible for applying different styles while staying anime. I find it more difficult for NAI to stray far from its (very good) style. Each have their merits. NAI is definitely much better about not cropping heads, though.

[ethanfel]

I have tried WD1.3 and all I can say is NAI >>>>>> WD1.3

All NAI image look very similar to each other. IMO it look great but lack variety. The skin in particular.
I've been playing a game by myself on unstable diffusion, Guessing if NAI was involved. 90% of the time, just by looking at the skin texture i win haha

[ghost]

No need for skin, just look at hair, whenever i browse pixiv and other sites i can almost 100% say if it's basic NAI or not, lol :D

[Manimap]

It's not another leak, just resumed training and finetuning from the original leak, from what I understood?

[aliencaocao]

It is unclear if novelAI resumed training themselves or someone else, because the quality improvement is massive and very unlikely that any single person can achieve. If novelAI resumed training then it is a leak

[Manimap]

Oh maybe then, I didn't try this yet, as the origin looks sketchy.

[starik222]

Did a few comparisons, the "Anything" model looks better than the NAI, at least on higher resolution images. But many of the images look like it's a completely different model.
sample 1
sample 2
sample 3
sample 4
Also compared the versions of Anything-V3.0-pruned-fp16, Anything-V3.0-pruned-fp32, Anything-V3.0. fp32 and full give identical results, but fp16 is very different from them, although perhaps it should be.
compare
Anything model, as well as NAI, works great with SD VAE

[aliencaocao]

Wow that is interesting....when I tried it produced very diff results. I will go reconfirm this tmr.

[tearxinnuan]

Anything3.0 VAE should actually be NAI's VAE, I have downloaded several Anything3.0 VAE to compare with NAI VAE, they are exactly the same

[aliencaocao]

I am getting slightly different results though, although the hashes indeed matches...
Anything V3.0

Animefullpruned

miku
Negative prompt: lowres, bad anatomy, bad hands, text, error, missing fingers, bad feet, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name
Steps: 28, Sampler: Euler, CFG scale: 8, Seed: 0, Size: 768x768, Model hash: 2700c435, Model: Anything-V3.0-pruned, Clip skip: 2, ENSD: 31337

I agree that the difference is very subtle, but if you switch around the image side by side, it is obvious. Does this not happen to you? Pay attention to the part where the girl's chest area overlaps - one is above while another is below. The skin colour also differs a bit, especially at the left hand sleeves.

Tried with seed=1 and everything else same - also same observation, very subtle but there is difference.

[starik222]

If you generate multiple images with the same settings and don't change anything, you will end up with slightly different images. I also get a slight difference when generating the same images. If I understood correctly, it has to do with xformers optimization. Also, the use of half floating-point numbers probably plays a role (--no-half disables their use).

[aliencaocao]

Yes you are right @starik222 , thank you and sorry for my ignorance on the VAE being the same previously.

[DuskXi]

Guys, I have a big confusion about Anything-v3. That is, when training with Textual-Inversion, there is a problem if you use Anything-v3. The problem is that the generated images appear strangely distorted, as if they were trained without uninstalling vae. Yes, I have confirmed that there is no vae in the model directory and no vae is loaded in the webui logs, but training Textual-Inversion with Anything-v3 still results in a similar situation to training Textual-Inversion with vae loaded.

I am not sure what is the reason for this. I have also tried adjusting the learning rate downwards, which even though it makes the "crashing of the image due to training after loading vae" disappear, it seems to be no change. I believe this should not be a problem with the training set either, because the same training set is very successful when using novelai as the main model for Textual-Inversion training, with character similarity seen at 3000step, approximation at 5000step, and perfect convergence above 8000step. So I am surprised because I am sure that when training Textual-Inversion with Anything-v3, there is no vae in the model directory and the settings show no vae loaded.

As for the hypernetwork, it is also useless, as I found out when using novelai that the hypernetwork can forcefully suppress vae to affect the training. So I think the hyper network training is not affected by vae when using Anything-v3 as the main model, so the result is the same output after a few thousand steps as when there is no training.

[DuskXi]

@andrei007999
Dude, have you made any progress?

[ghost]

@andrei007999 Dude, have you made any progress?

Im having a blast with my 60k+ steps hypernetworks, but i never went back to iversion, so, nope.
Hyp[ernetworks work as intended.

[DuskXi]

@andrei007999 Dude, have you made any progress?

Im having a blast with my 60k+ steps hypernetworks, but i never went back to iversion, so, nope. Hyp[ernetworks work as intended.

In fact, I also tried hypernetwork today, it is true that at about 10k position can accurately draw the character, but another problem came up, that is, hypernetwork complex seems to destroy the background, originally anything-v3 background painting ability is very strong, but after enabling hypernetwork, drawing the background quality is as bad as novelai

[ghost]

@andrei007999 Dude, have you made any progress?

Im having a blast with my 60k+ steps hypernetworks, but i never went back to iversion, so, nope. Hyp[ernetworks work as intended.

In fact, I also tried hypernetwork today, it is true that at about 10k position can accurately draw the character, but another problem came up, that is, hypernetwork complex seems to destroy the background, originally anything-v3 background painting ability is very strong, but after enabling hypernetwork, drawing the background quality is as bad as novelai

I have no issues with my hypernetworks. Check your dataset for bad reference and learning rate.

[DuskXi]

Obviously the most suitable is to use the dreambooth, TI training is a bit lacking in the character's demeanor and temperament.

[ClashSAN]

I wanted to make a post for this model because it's so bizarre.
I got almost 80-90% perfect aesthetic pictures with DPM++ 2M Karras with only 6 steps.
I do not know why this is happening.
I was testing the minimum amount of steps for a coherent picture, usually some embeddings will not form a good picture with DDIM 8 steps, so I have to use Euler A 20 steps.

But this model for some reason, produces the intended pictures well at an extremely low step count without the usual glitches.

Has anyone experienced these good results too?

[ClashSAN]

That reminds me, somebody pester @dfaker to add a trading card feature to embedding_to_png script
Trading our images would be so cool.

[dfaker]

That reminds me, somebody pester @dfaker to add a trading card feature to embedding_to_png script Trading our images would be so cool.

This what you were imagining?

[ClashSAN]

😁 I really like the format from https://bighugelabs.com/deck.php

But I'm sure there's better aesthetics..

[aliencaocao]

The DPM++ series is by itself good at very very low steps, the author mentioned it in its PR and their paper. 6 steps seems normal for this series of samplers.
Also, interesting that you have clip end=1. Being a animefull resume training, it should have been clip end=2 for best results.

[ClashSAN]

t should have been clip end=2 for best results

that lowered the amount of geometry/background duds I got.
I'm specifying bad_prompt in the negative, (I have no TI embedding) as it greatly raises the amount of good random gens I get (with no prompt) for some reason.
I didn't use webui exclusive enhancements as I'm curious what would be the speed results on TPU once Flax Pipe includes this sampler. (my 50step LMS=14.4s | 8step DDIM=3.2s | 6step DPM++ 2M Karras=2.6s) DDIM 8 steps also works but has a lot more geometry/background duds.
I tried converting the model to openvino format using scripts, but I found it's too difficult for now

[ReaLifecyborg]

from what i read, and the suspect original leak source, anything V3.0 is a result of mixing multiple hypernetwork and models including the novel ai leak model, it is done by someone who then post it out for others to share and use, the information is coming from bilibili(the very first place where i can find the anything V3.0 model appears on internet)

[ProfaneServitor]

Where is it possible to find lore behind NAI leaks and Anything?

[ghost]

Someone used 0-day on NovelAi and leaked everything. That's kinda it.

Anything3 is chinese continuation and fine-tune, as general lore so far.

[ProfaneServitor]

Also, is there any proof they are somehow connected? It kinda feels like Anything is more consistent, but a bit deaf when it comes to perspective tags (from above, from below, from side, profile, etc..). Here's comparison:

[Infinitay]

With the introduction of safetensors I'm confused on whether or not it is safe to still use Anything v3's vae.pt file? It's still a pickle and not a safetensor. Also, there is a model.safetensors file that was recently added among the safetensor variants of the various Anything v3 models. What is that file used for?

[therealbigdumb]

Is someone able to explain how I get the Model file? In huggingface there is no model file only a "safetensors"

[wmooring]

That is the model file. It's an alternative format to .ckpt, which is safer from containing viruses and such. AUTOMATIC1111 can handle either format.

[lastYoueven]

with the https://huggingface.co/Linaqruf/anything-v3.0 is 404