BBDM updated publishing

Pseudo-Lab · Nov 9, 2023 · e728d89 · e728d89
1 parent dc6dc36
commit e728d89
Show file tree

Hide file tree

Showing 23 changed files with 350 additions and 207 deletions.
diff --git a/_images/img_002.png b/_images/img_002.png
diff --git a/_images/img_012.png b/_images/img_012.png
diff --git a/_images/img_022.png b/_images/img_022.png
diff --git a/_images/img_032.png b/_images/img_032.png
diff --git a/_images/img_041.png b/_images/img_041.png
diff --git a/_images/img_051.png b/_images/img_051.png
diff --git a/_images/img_062.png b/_images/img_062.png
diff --git a/_images/img_071.png b/_images/img_071.png
diff --git a/_images/img_082.png b/_images/img_082.png
diff --git a/_images/img_091.png b/_images/img_091.png
diff --git a/_images/img_101.png b/_images/img_101.png
diff --git a/_images/img_111.png b/_images/img_111.png
diff --git a/_sources/docs/review/BBDM.md b/_sources/docs/review/BBDM.md
diff --git a/_sources/docs/review/Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.md b/_sources/docs/review/Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.md
diff --git a/docs/experiments/js_exp.html b/docs/experiments/js_exp.html
diff --git a/docs/experiments/swjo_exp.html b/docs/experiments/swjo_exp.html
@@ -225,6 +225,7 @@
 
 <li class="toctree-l1"><a class="reference internal" href="../review/GLIDE.html">GLIDE</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../review/BBDM.html">BBDM</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../review/Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.html"><strong>Your Diffusion Model is Secretly a Zero-Shot Classifier</strong></a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Experiments</span></p>
 <ul class="current nav bd-sidenav">
@@ -459,7 +460,7 @@ <h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to t
 <figure class="align-default" id="id1">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_01.png"><img alt="swjo_exp_01" class="bg-primary mb-1" src="../../_images/swjo_exp_01.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 320 </span><span class="caption-text">Training Data</span><a class="headerlink" href="#id1" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 332 </span><span class="caption-text">Training Data</span><a class="headerlink" href="#id1" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <p>DreamBooth 를 실험하면서 대표적으로 instance prompt, guidance scale, negative prompt, 그리고 마지막으로 prior preservation loss 를 반영하는 정도를 조절하는 prior_loss_weight 를 바꿔가면서 학습해보았습니다. 사전학습된 text-to-image 모델로 처음에는 <em>hakurei/waifu-diffusion</em> 모델을 시도해봤지만 결과가 만족스럽지 못해 <em>runwayml/stable-diffusion-v1-5</em> 모델로 fine-tuning 작업을 진행했습니다.</p>
@@ -534,7 +535,7 @@ <h3>Prior Preservation Loss<a class="headerlink" href="#prior-preservation-loss"
 <figure class="align-default" id="id2">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_02.png"><img alt="swjo_exp_02" class="bg-primary mb-1" src="../../_images/swjo_exp_02.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 321 </span><span class="caption-text">With Prior Preservation Loss</span><a class="headerlink" href="#id2" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 333 </span><span class="caption-text">With Prior Preservation Loss</span><a class="headerlink" href="#id2" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul class="simple">
@@ -543,7 +544,7 @@ <h3>Prior Preservation Loss<a class="headerlink" href="#prior-preservation-loss"
 <figure class="align-default" id="id3">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_03.png"><img alt="swjo_exp_03" class="bg-primary mb-1" src="../../_images/swjo_exp_03.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 322 </span><span class="caption-text">Without Prior Preservation Loss</span><a class="headerlink" href="#id3" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 334 </span><span class="caption-text">Without Prior Preservation Loss</span><a class="headerlink" href="#id3" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 </section>
@@ -556,7 +557,7 @@ <h3>Negative Prompt<a class="headerlink" href="#negative-prompt" title="Permalin
 <figure class="align-default" id="id4">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_03.png"><img alt="swjo_exp_03" class="bg-primary mb-1" src="../../_images/swjo_exp_03.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 323 </span><span class="caption-text">Without Negative Prompt</span><a class="headerlink" href="#id4" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 335 </span><span class="caption-text">Without Negative Prompt</span><a class="headerlink" href="#id4" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul>
@@ -567,7 +568,7 @@ <h3>Negative Prompt<a class="headerlink" href="#negative-prompt" title="Permalin
 <figure class="align-default" id="id5">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_04.png"><img alt="swjo_exp_04" class="bg-primary mb-1" src="../../_images/swjo_exp_04.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 324 </span><span class="caption-text">With Negative Prompt</span><a class="headerlink" href="#id5" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 336 </span><span class="caption-text">With Negative Prompt</span><a class="headerlink" href="#id5" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 </section>
@@ -641,7 +642,7 @@ <h3>Instance Prompt / Guidance Scale<a class="headerlink" href="#instance-prompt
 <figure class="align-default" id="id6">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_05.png"><img alt="swjo_exp_05" class="bg-primary mb-1" src="../../_images/swjo_exp_05.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 325 </span><span class="caption-text">Instance Prompt</span><a class="headerlink" href="#id6" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 337 </span><span class="caption-text">Instance Prompt</span><a class="headerlink" href="#id6" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <p>Inference step 을 늘려가면서 추론된 인물 이미지의 퀄리티가 상승하는 부분도 확인할 수 있었습니다. 또한, guidance scale 에 대한 실험도 진행했는데 guidance scale 이 작을수록 prompt 와 무관한 random 한 이미지들을 생성하게 됩니다. 최종적으로 num_inference steps 와 guidance scale 의 값은 각각 100 과 7.5 로 설정하였습니다.</p>
@@ -651,7 +652,7 @@ <h3>Instance Prompt / Guidance Scale<a class="headerlink" href="#instance-prompt
 <figure class="align-default" id="id7">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_06.png"><img alt="swjo_exp_06" class="bg-primary mb-1" src="../../_images/swjo_exp_06.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 326 </span><span class="caption-text">Increasing Number of Inference Steps</span><a class="headerlink" href="#id7" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 338 </span><span class="caption-text">Increasing Number of Inference Steps</span><a class="headerlink" href="#id7" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul>
@@ -662,7 +663,7 @@ <h3>Instance Prompt / Guidance Scale<a class="headerlink" href="#instance-prompt
 <figure class="align-default" id="id8">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_07.png"><img alt="swjo_exp_07" class="bg-primary mb-1" src="../../_images/swjo_exp_07.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 327 </span><span class="caption-text">Increasing Number of Inference Steps / Negative Prompt</span><a class="headerlink" href="#id8" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 339 </span><span class="caption-text">Increasing Number of Inference Steps / Negative Prompt</span><a class="headerlink" href="#id8" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul>
@@ -674,7 +675,7 @@ <h3>Instance Prompt / Guidance Scale<a class="headerlink" href="#instance-prompt
 <figure class="align-default" id="id9">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_08.png"><img alt="swjo_exp_08" class="bg-primary mb-1" src="../../_images/swjo_exp_08.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 328 </span><span class="caption-text">Guidance Scale</span><a class="headerlink" href="#id9" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 340 </span><span class="caption-text">Guidance Scale</span><a class="headerlink" href="#id9" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <p>동일한 inference prompt 로 prior-preservation loss 를 제외해본 결과, 생성된 남자의 머리카락이 더 길어지고 더 여성스러운 생김새를 가지는 놀라운 사실도 발견했습니다.</p>
@@ -686,7 +687,7 @@ <h3>Instance Prompt / Guidance Scale<a class="headerlink" href="#instance-prompt
 <figure class="align-default" id="id10">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_09.png"><img alt="swjo_exp_09" class="bg-primary mb-1" src="../../_images/swjo_exp_09.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 329 </span><span class="caption-text">Without Prior Preservation Loss</span><a class="headerlink" href="#id10" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 341 </span><span class="caption-text">Without Prior Preservation Loss</span><a class="headerlink" href="#id10" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 </section>
@@ -702,7 +703,7 @@ <h2>Appendix<a class="headerlink" href="#appendix" title="Permalink to this head
 <figure class="align-default" id="id11">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_10.png"><img alt="swjo_exp_10" class="bg-primary mb-1" src="../../_images/swjo_exp_10.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 330 </span><span class="caption-text">Appendix 1</span><a class="headerlink" href="#id11" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 342 </span><span class="caption-text">Appendix 1</span><a class="headerlink" href="#id11" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul>
@@ -713,7 +714,7 @@ <h2>Appendix<a class="headerlink" href="#appendix" title="Permalink to this head
 <figure class="align-default" id="id12">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_11.png"><img alt="swjo_exp_11" class="bg-primary mb-1" src="../../_images/swjo_exp_11.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 331 </span><span class="caption-text">Appendix 2</span><a class="headerlink" href="#id12" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 343 </span><span class="caption-text">Appendix 2</span><a class="headerlink" href="#id12" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <ul>
@@ -724,7 +725,7 @@ <h2>Appendix<a class="headerlink" href="#appendix" title="Permalink to this head
 <figure class="align-default" id="id13">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_12.png"><img alt="swjo_exp_12" class="bg-primary mb-1" src="../../_images/swjo_exp_12.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 332 </span><span class="caption-text">Appendix 3</span><a class="headerlink" href="#id13" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 344 </span><span class="caption-text">Appendix 3</span><a class="headerlink" href="#id13" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 <p>마지막으로 하단의 좌측과 우측 사진은 각각 “A photo of <em>sks</em> girl” 그리고 “A photo of a girl in the style of <em>sks</em>” 이라는 prompt 로 DreamBooth 모델을 각각 학습한 후, 나비를 생성하라는 동일한 prompt 로 추론해본 결과입니다. <em>sks</em> 가 수식하는 명사가 girl 이 아닌 style 이도록 prompt 를 수정함으로써, butterfly 사진을 생성할때 조금이나마 더 프리드로우 웹툰의 그림체를 반영할 수 있었던 부분도 확인할 수 있었습니다.</p>
@@ -734,7 +735,7 @@ <h2>Appendix<a class="headerlink" href="#appendix" title="Permalink to this head
 <figure class="align-default" id="id14">
 <a class="bg-primary mb-1 reference internal image-reference" href="../../_images/swjo_exp_13.png"><img alt="swjo_exp_13" class="bg-primary mb-1" src="../../_images/swjo_exp_13.png" style="width: 700px;" /></a>
 <figcaption>
-<p><span class="caption-number">Fig. 333 </span><span class="caption-text">Appendix 4</span><a class="headerlink" href="#id14" title="Permalink to this image">#</a></p>
+<p><span class="caption-number">Fig. 345 </span><span class="caption-text">Appendix 4</span><a class="headerlink" href="#id14" title="Permalink to this image">#</a></p>
 </figcaption>
 </figure>
 </section>

diff --git a/docs/review/BBDM.html b/docs/review/BBDM.html
@@ -69,7 +69,7 @@
     <link rel="shortcut icon" href="../../_static/PseudoLab_logo.png"/>
     <link rel="index" title="Index" href="../../genindex.html" />
     <link rel="search" title="Search" href="../../search.html" />
-    <link rel="next" title="YDMSZC 발표 자료" href="Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.html" />
+    <link rel="next" title="Your Diffusion Model is Secretly a Zero-Shot Classifier" href="Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.html" />
     <link rel="prev" title="GLIDE" href="GLIDE.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
@@ -228,7 +228,7 @@
 
 <li class="toctree-l1"><a class="reference internal" href="GLIDE.html">GLIDE</a></li>
 <li class="toctree-l1 current active"><a class="current reference internal" href="#">BBDM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.html">YDMSZC 발표 자료</a></li>
+<li class="toctree-l1"><a class="reference internal" href="Your_Diffusion_Model_is_Secretly_a_Zero_Shot_Classifier.html"><strong>Your Diffusion Model is Secretly a Zero-Shot Classifier</strong></a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Experiments</span></p>
 <ul class="nav bd-sidenav">
@@ -477,7 +477,7 @@ <h1>BBDM<a class="headerlink" href="#bbdm" title="Permalink to this heading">#</
 <li><p>유체의 미소입자가 불규칙하게 운동하는 현상</p></li>
 </ul>
 <figure class="align-default" id="id1">
-<a class="bg-primary mb-1 reference internal image-reference" href="../../_images/img_001.png"><img alt="img_00" class="bg-primary mb-1" src="../../_images/img_001.png" style="width: 700px;" /></a>
+<a class="bg-primary mb-1 reference internal image-reference" href="../../_images/img_001.png"><img alt="img_00" class="bg-primary mb-1" src="../../_images/img_001.png" style="width: 400px;" /></a>
 <figcaption>
 <p><span class="caption-number">Fig. 293 </span><span class="caption-text">굴뚝에서 퍼져나간 연기 사진을 오른쪽으로 90도 회전시킨 사진</span><a class="headerlink" href="#id1" title="Permalink to this image">#</a></p>
 </figcaption>
@@ -488,7 +488,7 @@ <h1>BBDM<a class="headerlink" href="#bbdm" title="Permalink to this heading">#</
 <li><p>Brownian Motion 을 연속 시간 확률 과정으로 모델링한 것</p></li>
 </ul>
 <figure class="align-default" id="id2">
-<a class="bg-primary mb-1 reference internal image-reference" href="../../_images/img_011.png"><img alt="img_01" class="bg-primary mb-1" src="../../_images/img_011.png" style="width: 1400px;" /></a>
+<a class="bg-primary mb-1 reference internal image-reference" href="../../_images/img_011.png"><img alt="img_01" class="bg-primary mb-1" src="../../_images/img_011.png" style="width: 2000px;" /></a>
 <figcaption>
 <p><span class="caption-number">Fig. 294 </span><span class="caption-text"><span class="math notranslate nohighlight">\(W_0\)</span> = 0 이고 max time T=1000 인 Wiener Process 를 100번 Sampling 한 결과</span><a class="headerlink" href="#id2" title="Permalink to this image">#</a></p>
 </figcaption>
@@ -684,7 +684,8 @@ <h1>BBDM<a class="headerlink" href="#bbdm" title="Permalink to this heading">#</
 <p><span class="math notranslate nohighlight">\(= t - 2{t\over T}\mathbb{E}(W(t)W(T)) + {t^2\over T}\)</span></p>
 <p><span class="math notranslate nohighlight">\(= t - 2 {t \over T}t + {t^2 \over T}\)</span></p>
 <p><span class="math notranslate nohighlight">\(= t - {t^2 \over T}\)</span></p>
-<p><span class="math notranslate nohighlight">\(\because\)</span> <span class="math notranslate nohighlight">\(W(t)\)</span> 와 <span class="math notranslate nohighlight">\(W(T) - W(t)\)</span> 는 독립이므로,</p>
+<p><span class="math notranslate nohighlight">\(\because\)</span> <span class="math notranslate nohighlight">\(W(t)\)</span> 와 <span class="math notranslate nohighlight">\(W(T) - W(t)\)</span> 는 독립이므로,
+$\mathbb{E}(W(t)W(T)) = \mathbb{E}(W(t)(W(T)-W(t))) + \mathbb{E}(W(t)^2) = \mathbb{E}(W(t))\mathbb{E}(W(T)-W(t)) + t = t</p>
 <p>따라서,</p>
 <p><span class="math notranslate nohighlight">\(Var(B(t)) = \mathbb{E}(B(t)^2) = {t^2\over T} + t - {t^2 \over T} = t\)</span></p>
 <p>이므로,  <span class="math notranslate nohighlight">\(B(t)\)</span> 는 Wiener Process 이다.</p>
@@ -752,7 +753,7 @@ <h1>BBDM<a class="headerlink" href="#bbdm" title="Permalink to this heading">#</
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">YDMSZC 발표 자료</p>
+        <p class="prev-next-title"><strong>Your Diffusion Model is Secretly a Zero-Shot Classifier</strong></p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>