diff --git a/README.md b/README.md index 1e751ff1..45f2850c 100644 --- a/README.md +++ b/README.md @@ -17,14 +17,14 @@ # ๐Ÿ“ฐ News | ๐Ÿ—ž๏ธNews | ๐Ÿ“Description | | -- | ------ | -| First release | RDAgent are release on Github | +| First release | RDAgent is release on Github | # ๐ŸŒŸ Introduction ![Our focused scenario](docs/_static/scen.jpg) -RDAgent aims to automate the most critical and valuable aspects of the industrial R&D process, and we begins with focusing on the data-driven scenarios to streamline the development of models and data. +RDAgent aims to automate the most critical and valuable aspects of the industrial R&D process, and we begin with focusing on the data-driven scenarios to streamline the development of models and data. Methodologically, we have identified a framework with two key components: 'R' for proposing new ideas and 'D' for implementing them. We believe that the automatic evolution of R&D will lead to solutions of significant industrial value. @@ -38,15 +38,21 @@ R&D is a very general scenario. The advent of RDAgent can be your You can click the [๐ŸŽฅlink](https://rdagent.azurewebsites.net) above to view the demo. More methods and scenarios are being added to the project to empower your R&D processes and boost productivity. + + # โšกQuick start -You can try our demo by running the following command: +You can try above demos by running the following command: + +### ๐ŸณDocker installation. +Users must ensure Docker is installed before attempting most scenarios. Please refer to the [official ๐ŸณDocker page](https://docs.docker.com/engine/install/) for installation instructions. ### ๐Ÿ Create a Conda Environment -- Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI): +- Create a new conda environment with Python (3.10 and 3.11 are well-tested in our CI): ```sh conda create -n rdagent python=3.10 ``` @@ -60,27 +66,20 @@ You can try our demo by running the following command: ```sh pip install rdagent ``` - - -### ๐Ÿ› ๏ธ Run Make Files - -- If you want to use our project in development mode. **Navigate to the directory containing the MakeFile** and set up the development environment: - ```sh - make dev - ``` +### โš™๏ธ Configuration +You have to config your GPT model in the `.env` +```bash +cat << EOF > .env +OPENAI_API_KEY= +# EMBEDDING_MODEL=text-embedding-3-small +CHAT_MODEL=gpt-4-turbo +EOF +``` -### โš™๏ธ Environment Configuration -- Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - please refer to [Configuration](docs/build/html/installation.html#azure-openai) for the detailed explanation of the `.env` ### ๐Ÿš€ Run the Application -- Start web app to show log traces: - ```sh - rdagent ui --port 80 --log_dir - ``` -The [๐ŸŽฅdemo](https://rdagent.azurewebsites.net) is implemented by the following commands: +The [๐ŸŽฅdemo](https://rdagent.azurewebsites.net) is implemented by the following commands(each item represents one demo, you can select the one you prefer): - Run the **Automated Quantitative Trading & Iterative Factors Evolution**: Qlib self-loop factor proposal and implementation application ```sh @@ -107,10 +106,10 @@ The [๐ŸŽฅdemo](https://rdagent.azurewebsites.net) is implemented by the followin rdagent general_model ``` -### ๐Ÿš€ Monitor the Application +### ๐Ÿš€ Monitor the Application Results - You can serve our demo app to monitor the RD loop by running the following command: ```sh - rdagent ui --port 80 --log_dir + rdagent ui --port 80 --log_dir ``` # Scenarios @@ -144,8 +143,12 @@ The supported scenarios are listed below: Different scenarios vary in entrance and configuration. Please check the detailed setup tutorial in the scenarios documents. -TODO: Scenario Gallary -- map(scenario) => knowledge list; +Here is a gallery of successful explorations. You can download the source code and view the execution trace using the command below: + +```bash +rdagent ui --port 80 --log_dir gallary/ +``` + # โš™๏ธFramework @@ -206,7 +209,7 @@ For more detail, please refer to our [Demos page](https://rdagent.azurewebsites. # Contributing -More documents can be found in the [๐Ÿ“šreadthedocs](). TODO: add link +More documents can be found in the [๐Ÿ“šreadthedocs](https://rdagent.readthedocs.io/). ## Guidance This project welcomes contributions and suggestions. diff --git a/docs/development.rst b/docs/development.rst index 837f5689..15c59f5d 100644 --- a/docs/development.rst +++ b/docs/development.rst @@ -73,4 +73,4 @@ File Naming Convention * - `conf.py` - The configuration for the module, app, and project. - +.. diff --git a/docs/installation_and_configuration.rst b/docs/installation_and_configuration.rst index c12b824e..23d50c1c 100644 --- a/docs/installation_and_configuration.rst +++ b/docs/installation_and_configuration.rst @@ -143,3 +143,25 @@ Configuration List | max_past_message_include | Maximum number of past messages to include | 10 | +------------------------------+--------------------------------------------------+-------------------------+ + + + +Loading Configuration +--------------------- + +For users' convenience, we provide a CLI interface called `rdagent`, which automatically runs `load_dotenv()` to load environment variables from the `.env` file. +However, this feature is not enabled by default for other scripts. We recommend users load the environment with the following steps: + + +- โš™๏ธ Environment Configuration + - Place the `.env` file in the same directory as the `.env.example` file. + - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) + + - Export each variable in the .env file: + + .. code-block:: sh + + export $(grep -v '^#' .env | xargs) + + - If you want to change the default environment variables, you can refer to the above configuration and edith the `.env` file. + diff --git a/docs/project_framework_introduction.rst b/docs/project_framework_introduction.rst index 482a9e93..bfe7052d 100644 --- a/docs/project_framework_introduction.rst +++ b/docs/project_framework_introduction.rst @@ -23,19 +23,5 @@ We have established a basic method framework that continuously proposes hypothes The figure above shows the main classes and how they fit into the workflow for those interested in the detailed code. -Detailed Design -========================= - - -Configuration -------------- - -You can manually source the `.env` file in your shell before running the Python script: -Most of the workflow are controlled by the environment variables. -```sh -# Export each variable in the .env file; Please note that it is different from `source .env` without export -export $(grep -v '^#' .env | xargs) -# Run the Python script -python your_script.py -``` - +.. Detailed Design +.. =============== diff --git a/docs/scens/data_agent_fin.rst b/docs/scens/data_agent_fin.rst index e0ef74d4..1c957c16 100644 --- a/docs/scens/data_agent_fin.rst +++ b/docs/scens/data_agent_fin.rst @@ -17,9 +17,17 @@ By leveraging these factors, quantitative traders can develop sophisticated stra The ability to systematically analyze and apply these factors is what separates ordinary trading from truly strategic market outmaneuvering. And this is where the **Finance Model Agent** comes into play. -๐ŸŽฅ Demo -~~~~~~~~~~ -TODO: Here should put a video of the demo. +๐ŸŽฅ `Demo `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. https://rdagent.azurewebsites.net:443/media/54d59c2bf85211de6a7cd2bd3c57d3b56c36cf9795a31111a9111ddb.mp4 + +.. raw:: html + + ๐ŸŒŸ Introduction @@ -75,6 +83,8 @@ Here's an enhanced outline of the steps: โšก Quick Start ~~~~~~~~~~~~~~~~~ +Please refer to the installation part in :doc:`../installation_and_configuration` to prepare your system dependency. + You can try our demo by running the following command: - ๐Ÿ Create a Conda Environment @@ -97,12 +107,6 @@ You can try our demo by running the following command: pip install rdagent -- โš™๏ธ Environment Configuration - - Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - - If you want to change the default environment variables, you can refer to `Env Config`_ below - - ๐Ÿš€ Run the Application .. code-block:: sh @@ -126,4 +130,4 @@ The following environment variables can be set in the `.env` file to customize t :settings-show-field-summary: False :members: coder_use_cache, data_folder, data_folder_debug, cache_location, enable_execution_cache, file_based_execution_timeout, select_method, select_threshold, max_loop, knowledge_base_path, new_knowledge_base_path :exclude-members: Config, fail_task_trial_limit, v1_query_former_trace_limit, v1_query_similar_success_limit, v2_query_component_limit, v2_query_error_limit, v2_query_former_trace_limit, v2_error_summary, v2_knowledge_sampler - :no-index: \ No newline at end of file + :no-index: diff --git a/docs/scens/data_copilot_fin.rst b/docs/scens/data_copilot_fin.rst index 6ea9e07d..88c4e852 100644 --- a/docs/scens/data_copilot_fin.rst +++ b/docs/scens/data_copilot_fin.rst @@ -20,9 +20,16 @@ Therefore, there is an urgent need for a systematic approach to design a framewo And this is where the **Finance Data Copilot** steps in. -๐ŸŽฅ Demo -~~~~~~~~~~ -TODO: Here should put a video of the demo. +๐ŸŽฅ `Demo `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. raw:: html + + + ๐ŸŒŸ Introduction @@ -76,6 +83,8 @@ Here's an enhanced outline of the steps: โšก Quick Start ~~~~~~~~~~~~~~~~~ +Please refer to the installation part in :doc:`../installation_and_configuration` to prepare your system dependency. + You can try our demo by running the following command: - ๐Ÿ Create a Conda Environment @@ -98,12 +107,6 @@ You can try our demo by running the following command: pip install rdagent -- โš™๏ธ Environment Configuration - - Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - - If you want to change the default environment variables, you can refer to `Env Config`_ below - - ๐Ÿš€ Run the Application - Store the factors you want to extract from the financial reports in your desired folder. Then, save the paths of the reports in the `report_result_json_file_path`. The format should be as follows: @@ -139,4 +142,4 @@ The following environment variables can be set in the `.env` file to customize t :settings-show-field-summary: False :members: coder_use_cache, data_folder, data_folder_debug, cache_location, enable_execution_cache, file_based_execution_timeout, select_method, select_threshold, max_loop, knowledge_base_path, new_knowledge_base_path :exclude-members: Config, python_bin, fail_task_trial_limit, v1_query_former_trace_limit, v1_query_similar_success_limit, v2_query_component_limit, v2_query_error_limit, v2_query_former_trace_limit, v2_error_summary, v2_knowledge_sampler - :no-index: \ No newline at end of file + :no-index: diff --git a/docs/scens/model_agent_fin.rst b/docs/scens/model_agent_fin.rst index 896f8996..43e9262b 100644 --- a/docs/scens/model_agent_fin.rst +++ b/docs/scens/model_agent_fin.rst @@ -16,9 +16,20 @@ The effectiveness of a quantitative strategy depends not only on the factors use However, the process of developing and optimizing these models can be labor-intensive and complex, requiring continuous refinement and adaptation to ever-changing market conditions. And this is where the **Finance Model Agent** steps in. -๐ŸŽฅ Demo -~~~~~~~~~~ -TODO: Here should put a video of the demo. + +๐ŸŽฅ `Demo `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. raw:: html + + + + + + ๐ŸŒŸ Introduction @@ -74,6 +85,8 @@ Here's an enhanced outline of the steps: โšก Quick Start ~~~~~~~~~~~~~~~~~ +Please refer to the installation part in :doc:`../installation_and_configuration` to prepare your system dependency. + You can try our demo by running the following command: - ๐Ÿ Create a Conda Environment @@ -96,18 +109,6 @@ You can try our demo by running the following command: pip install rdagent -- โš™๏ธ Environment Configuration - - Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - - Export each variable in the .env file: - - .. code-block:: sh - - export $(grep -v '^#' .env | xargs) - - - If you want to change the default environment variables, you can refer to `Env Config`_ below - - ๐Ÿš€ Run the Application .. code-block:: sh @@ -143,4 +144,4 @@ The following environment variables can be set in the `.env` file to customize t - **batch_size**: The batch size, set to `2000`. - **metric**: The evaluation metric, set to `loss`. - **loss**: The loss function, set to `mse`. - - **n_jobs**: The number of parallel jobs, set to `20`. \ No newline at end of file + - **n_jobs**: The number of parallel jobs, set to `20`. diff --git a/docs/scens/model_agent_med.rst b/docs/scens/model_agent_med.rst index 41071e91..25f000da 100644 --- a/docs/scens/model_agent_med.rst +++ b/docs/scens/model_agent_med.rst @@ -12,9 +12,15 @@ Medical Model Agent In this scenario, we consider the problem of risk prediction from patients' ICU monitoring data. We use the a public EHR dataset - MIMIC-III and extract a binary classification task for evaluating the framework. In this task, we aim at predicting the whether the patients will suffer from Acute Respiratory Failure (ARF) based their first 12 hours ICU monitoring data. -๐ŸŽฅ Demo -~~~~~~~~~~ -TODO: Here should put a video of the demo. +๐ŸŽฅ `Demo `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. raw:: html + + ๐ŸŒŸ Introduction @@ -58,6 +64,8 @@ Here's an enhanced outline of the steps: โšก Quick Start ~~~~~~~~~~~~~~~~~ +Please refer to the installation part in :doc:`../installation_and_configuration` to prepare your system dependency. + You can try our demo by running the following command: - ๐Ÿ Create a Conda Environment @@ -85,17 +93,6 @@ You can try our demo by running the following command: - Request access to FIDDLE preprocessed data: `FIDDLE Dataset `_. - Place your username and password in `.rdagent.app.data_mining.conf`. -- โš™๏ธ Environment Configuration - - Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - - Export each variable in the .env file: - - .. code-block:: sh - - export $(grep -v '^#' .env | xargs) - - - If you want to change the default environment variables, you can refer to `Env Config`_ below - ๐Ÿš€ Run the Application .. code-block:: sh @@ -113,4 +110,4 @@ The following environment variables can be set in the `.env` file to customize t .. autopydantic_settings:: rdagent.app.data_mining.conf.PropSetting :settings-show-field-summary: False - :exclude-members: Config \ No newline at end of file + :exclude-members: Config diff --git a/docs/scens/model_copilot_general.rst b/docs/scens/model_copilot_general.rst index e1a308dc..11160252 100644 --- a/docs/scens/model_copilot_general.rst +++ b/docs/scens/model_copilot_general.rst @@ -15,9 +15,15 @@ However, reproducing and implementing these models can be a daunting task, requi Researchers often face challenges in extracting the essential details from these papers and converting them into functional code. And this is where the **General Model Copilot** steps in. -๐ŸŽฅ Demo -~~~~~~~~~~ -TODO: Here should put a video of the demo. +๐ŸŽฅ `Demo `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. raw:: html + + ๐ŸŒŸ Introduction ~~~~~~~~~~~~~~~~ @@ -49,6 +55,8 @@ This demo automates the extraction and iterative development of models from acad โšก Quick Start ~~~~~~~~~~~~~~~~~ +Please refer to the installation part in :doc:`../installation_and_configuration` to prepare your system dependency. + You can try our demo by running the following command: - ๐Ÿ Create a Conda Environment @@ -72,10 +80,6 @@ You can try our demo by running the following command: pip install rdagent -- โš™๏ธ Environment Configuration - - Place the `.env` file in the same directory as the `.env.example` file. - - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) - - ๐Ÿš€ Run the Application - Prepare relevant files (in pdf format) by uploading papers to the directory below and copy the path as report_file_path. @@ -87,4 +91,4 @@ You can try our demo by running the following command: .. code-block:: sh - rdagent general_model --report_file_path= \ No newline at end of file + rdagent general_model --report_file_path=