-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
9aedd51
commit 761085b
Showing
43 changed files
with
18,066 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| <!DOCTYPE HTML> | ||
| <html> | ||
| <head> | ||
| <title>Blopost</title> | ||
| <meta charset="utf-8" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" /> | ||
| <link rel="stylesheet" href="assets/css/main.css" /> | ||
| <noscript><link rel="stylesheet" href="assets/css/noscript.css" /></noscript> | ||
| </head> | ||
| <body class="is-preload"> | ||
| <div id="page-wrapper"> | ||
|
|
||
| <!-- Header --> | ||
| <header id="header"> | ||
| <h1 id="logo"><a href="index.html">LLM360</a></h1> | ||
| <nav id="nav"> | ||
| <ul> | ||
| <li><a href="index.html">Home</a></li> | ||
| <li> | ||
| <a href="#">Research</a> | ||
| <ul> | ||
| <li><a href="">Paper</a></li> | ||
|
|
||
| </ul> | ||
| </li> | ||
| <li> | ||
| <a href="#">Models</a> | ||
| <ul> | ||
| <li><a href="">Learn More</a></li> | ||
| <li><a href="">Download</a></li> | ||
| <li><a href="">Reports</a></li> | ||
| </ul> | ||
| </li> | ||
| <li><a href="Blogpost.html">Blogpost</a></li> | ||
| <li><a href="#one">About</a></li> | ||
| <li><a href="#index.html" class="button primary">Contact</a></li> | ||
| </ul> | ||
| </nav> | ||
| </header> | ||
|
|
||
| <!-- Main --> | ||
| <div id="main" class="wrapper style1"> | ||
| <div class="container"> | ||
| <header class="major"> | ||
| <h2>Blogpost</h2> | ||
| <p>This is a blogpost page</p> | ||
| </header> | ||
|
|
||
| <!-- Content --> | ||
| <section id="content"> | ||
| <a href="#" class="image fit"><img src="images/pic07.jpg" alt="" /></a> | ||
| <h3>Introduction</h3> | ||
| <p>The LLM360 project goes beyond the existing practice of sharing only final weights and brief technical reports on new LLMs. We believe that true transparency lies in providing full access to the assets and the process of LLM training.</p> | ||
| <p>Our primary objective is to democratize LLM understanding by making the training process accessible to a broader audience. Due to the significant financial constraints required for pretraining, it is unrealistic for most people in the research community to have ability to work on a LLM pretraining project. LLM 360 seeks to level the playing field and accelerate LLM research. By releasing a series of models, we hope to stimulate further research and analysis of LLMs by providing the necessary resources and tools for in-depth exploration.</p> | ||
| <h3>Open Sourced Artifacts</h3> | ||
| <p>To achieve our goals of transparency and accessibility, we will open source the following project artifacts:</p> | ||
| <ul> | ||
| <li>360 Model Checkpoints: Checkpoints have been chosen from various stages of the training process. Up to 360 checkpoints will be released. The vast number of checkpoints was chosen to enable far reaching research efforts and to signify the openness of the project.</li> | ||
| <li>Data Buckets with Full Data Sequence: Unlike any project before, the data buckets and data sequences for each checkpoint are available.</li> | ||
| <li>Source code: The full suite of code used during the project including the preprocessing code, model training code, inference code, and analysis code. The full training code will ensure even the smallest detail of models will be disclosed.</li> | ||
| <li>Logs Book and Analysis: We unveil all the training details to provide a first hand view into LLM training. We aim to encourage digestible insights into the training process, ability to track performance metrics across training, and uncover potential areas for improvement.</li> | ||
| <li>Errors and Challenges: In our technical report, we will transparently document the errors and challenges encountered during the training process, providing valuable information for future research and improvement efforts.</li> | ||
| <li>Community Input: We welcome suggestions and feedback from the community and remain open to releasing additional details and resources based on the needs and interests of researchers and practitioners.</li> | ||
| </ul> | ||
| <h3>Potential Project Impacts</h3> | ||
| <p>To be continue...</p> | ||
| </section> | ||
|
|
||
| </div> | ||
| </div> | ||
|
|
||
| <!-- Footer --> | ||
| <footer id="footer"> | ||
| <ul class="icons"> | ||
| <li><a href="#" class="icon brands alt fa-twitter"><span class="label">Twitter</span></a></li> | ||
| <li><a href="#" class="icon brands alt fa-facebook-f"><span class="label">Facebook</span></a></li> | ||
| <li><a href="#" class="icon brands alt fa-linkedin-in"><span class="label">LinkedIn</span></a></li> | ||
| <li><a href="#" class="icon brands alt fa-instagram"><span class="label">Instagram</span></a></li> | ||
| <li><a href="#" class="icon brands alt fa-github"><span class="label">GitHub</span></a></li> | ||
| <li><a href="#" class="icon solid alt fa-envelope"><span class="label">Email</span></a></li> | ||
| </ul> | ||
| <!-- <ul class="copyright">--> | ||
| <!-- <li>© Untitled. All rights reserved.</li><li>Design: <a href="http://html5up.net">HTML5 UP</a></li>--> | ||
| <!-- </ul>--> | ||
| </footer> | ||
|
|
||
| </div> | ||
|
|
||
| <!-- Scripts --> | ||
| <script src="assets/js/jquery.min.js"></script> | ||
| <script src="assets/js/jquery.scrolly.min.js"></script> | ||
| <script src="assets/js/jquery.dropotron.min.js"></script> | ||
| <script src="assets/js/jquery.scrollex.min.js"></script> | ||
| <script src="assets/js/browser.min.js"></script> | ||
| <script src="assets/js/breakpoints.min.js"></script> | ||
| <script src="assets/js/util.js"></script> | ||
| <script src="assets/js/main.js"></script> | ||
|
|
||
| </body> | ||
| </html> |
Large diffs are not rendered by default.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.