Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding new from_github vehicle method #86

Merged
merged 28 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fc74da8
adding new from_github vehicle method
Dec 19, 2023
7f2d22f
merging in create_project_subdir function
robinsteuteville Dec 21, 2023
5d2c1d4
adding from_url and to_cache methods to SerdeAPI
robinsteuteville Dec 21, 2023
84754c3
fixing caching functions
robinsteuteville Dec 25, 2023
c81eca6
fixing from_url function
robinsteuteville Dec 26, 2023
f68d055
updating from_url and from_str
robinsteuteville Dec 26, 2023
8d556f3
adding from_github_or_url method
robinsteuteville Jan 2, 2024
f6896f6
updating error messages
robinsteuteville Jan 3, 2024
1ff4316
merge fastsim-2 into vehicle-fetching-caching
Jan 3, 2024
474df42
adding fetch_github_list function
robinsteuteville Jan 3, 2024
ea631fd
updating fetch_github_list
robinsteuteville Jan 3, 2024
5275708
adding caching of vehicle if not downloaded to from_github_or_url
robinsteuteville Jan 4, 2024
24a11ac
updating from_github_or_url doc string
robinsteuteville Jan 4, 2024
7e9a51c
updating doc strings
robinsteuteville Jan 4, 2024
05fea86
adding temporary tests/file to support the tests
robinsteuteville Jan 5, 2024
da1f8a4
updating test to produce test file for comparison
robinsteuteville Jan 5, 2024
1301482
troubleshooting fetch_github_list
robinsteuteville Jan 6, 2024
f6c607f
pulling fastsim-2 updates to get tests to pass
robinsteuteville Jan 10, 2024
d729966
updating fetch_github_list
robinsteuteville Jan 18, 2024
364395a
fixing looping problem with fetch_github_list
robinsteuteville Jan 18, 2024
49ba054
fixing fetch_github_list test so it runs
robinsteuteville Jan 18, 2024
1f9860e
adding path_to_cache and from_cache
robinsteuteville Jan 19, 2024
6086641
ading clear cache function
robinsteuteville Jan 23, 2024
4c4f1d9
updating tests so they pass
robinsteuteville Jan 23, 2024
512284a
adding 403 error to retry errors for fetch_github_list
robinsteuteville Jan 23, 2024
7960663
deleted deprecated comment
calbaker Feb 6, 2024
4fb10f3
minor tweaks
calbaker Feb 6, 2024
867f72f
reorganize reference vehicle for github caching
Feb 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions rust/fastsim-core/src/cycle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -726,6 +726,19 @@ impl SerdeAPI for RustCycle {
),
}
}

/// takes an object from a url and saves it in the fastsim data directory in a rust_objects folder
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved
/// WARNING: if there is a file already in the data subdirectory with the same name, it will be replaced by the new file
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved
fn to_cache<S: AsRef<str>>(url: S) {
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved
let url = url.as_ref();
let url_parts: Vec<&str> = url.split("/").collect();
let file_name = url_parts.last().unwrap_or_else(||panic!("Could not determine file name/type."));
let data_subdirectory = create_project_subdir("cycles").unwrap_or_else(|_|panic!("Could not find or create Fastsim data subdirectory."));
let file_path = data_subdirectory.join(file_name);
// I believe this will overwrite any existing files with the same name -- is this preferable, or should we add
// a bool argument so user can determine whether the file should overwrite an existing file or not
download_file_from_url(url, &file_path);
}
}

impl TryFrom<HashMap<String, Vec<f64>>> for RustCycle {
Expand Down
1 change: 1 addition & 0 deletions rust/fastsim-core/src/imports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ pub(crate) use std::path::PathBuf;

pub(crate) use crate::traits::*;
pub(crate) use crate::utils::*;
pub(crate) use crate::vehicle_utils::*;
1 change: 1 addition & 0 deletions rust/fastsim-core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
//! ```

extern crate ndarray;
extern crate tempfile;
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved

#[macro_use]
pub mod macros;
Expand Down
39 changes: 39 additions & 0 deletions rust/fastsim-core/src/traits.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use crate::imports::*;
use std::collections::HashMap;
use tempfile::tempdir;

pub trait SerdeAPI: Serialize + for<'a> Deserialize<'a> {
const ACCEPTED_BYTE_FORMATS: &'static [&'static str] = &["yaml", "json", "bin"];
Expand Down Expand Up @@ -154,6 +155,44 @@ pub trait SerdeAPI: Serialize + for<'a> Deserialize<'a> {
fn from_bincode(encoded: &[u8]) -> anyhow::Result<Self> {
Ok(bincode::deserialize(encoded)?)
}

/// instantiates an object from a url
fn from_url<S: AsRef<str>>(url: S) -> anyhow::Result<Self> {
let url = url.as_ref();
let temp_dir = tempdir()?;
let mut file_path = PathBuf::new();
// do these file types need to be specific to the object?
// TODO: either make funciton work for csv files, or remove from supported file list
if url.ends_with("yaml"){
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved
let file_path = temp_dir.path().join("temporary_object.yaml");
} else if url.ends_with("csv"){
let file_path = temp_dir.path().join("temporary_object.csv");
} else if url.ends_with("json"){
let file_path = temp_dir.path().join("temporary_object.json");
} else {
bail!("Unsupported file type, must be a yaml, json, or csv file.");
robinsteuteville marked this conversation as resolved.
Show resolved Hide resolved
}
download_file_from_url(url, &file_path);
// only works for json and yaml
// seems like I might also be able to use from_reader instead -- which one is preferable?
Self::from_file(file_path)
}

/// takes an object from a url and saves it in the fastsim data directory in a rust_objects folder
/// WARNING: if there is a file already in the data subdirectory with the same name, it will be replaced by the new file
/// to save to a folder other than rust_objects for a specific object type, override this default
/// implementation for the Rust object, and in the object-specific implementation, replace
/// "rust_objects" with your choice of folder name
fn to_cache<S: AsRef<str>>(url: S) {
let url = url.as_ref();
let url_parts: Vec<&str> = url.split("/").collect();
let file_name = url_parts.last().unwrap_or_else(||panic!("Could not determine file name/type."));
let data_subdirectory = create_project_subdir("rust_objects").unwrap_or_else(|_|panic!("Could not find or create Fastsim data subdirectory."));
let file_path = data_subdirectory.join(file_name);
// I believe this will overwrite any existing files with the same name -- is this preferable, or should we add
// a bool argument so user can determine whether the file should overwrite an existing file or not
download_file_from_url(url, &file_path);
}
}

pub trait ApproxEq<Rhs = Self> {
Expand Down
30 changes: 30 additions & 0 deletions rust/fastsim-core/src/vehicle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use crate::params::*;
use crate::proc_macros::{add_pyo3_api, doc_field, ApproxEq};
#[cfg(feature = "pyo3")]
use crate::pyo3imports::*;
use crate::utils::create_project_subdir;

use lazy_static::lazy_static;
use regex::Regex;
Expand Down Expand Up @@ -1033,6 +1034,23 @@ impl RustVehicle {
v.set_derived().unwrap();
v
}
/// Downloads specified vehicle from vehicle repo into fastsim/python/fastsim/resources/vehdb.
///
/// Arguments:
/// ----------
/// vehicle (string): name of the vehicle to be downloaded

// make separate function that gets list of yaml files off of github, then user can choose from that for input -- Rust function
// exposed to python, put in utilities
pub fn from_github(vehicle_file_name: String) {
let mut vehicle_url: String = "https://github.com/NREL/fastsim-vehicles/blob/main/public/".to_string();
vehicle_url.push_str(&vehicle_file_name);
// find or create fastsim data directory with subdirectory "vehicles" and return path
let data_directory: Option<PathBuf> = create_project_subdir("vehicles");
// see if you can find file_name in vehicle data
// if file doesn't already exist in directory, and once we have file path to directory:
// download_file_from_url(vehicle_url, data_directory)
}
}

impl Default for RustVehicle {
Expand Down Expand Up @@ -1064,6 +1082,18 @@ impl SerdeAPI for RustVehicle {
fn init(&mut self) -> anyhow::Result<()> {
self.set_derived()
}
/// takes an object from a url and saves it in the fastsim data directory in a rust_objects folder
/// WARNING: if there is a file already in the data subdirectory with the same name, it will be replaced by the new file
fn to_cache<S: AsRef<str>>(url: S) {
let url = url.as_ref();
let url_parts: Vec<&str> = url.split("/").collect();
let file_name = url_parts.last().unwrap_or_else(||panic!("Could not determine file name/type."));
let data_subdirectory = create_project_subdir("vehicles").unwrap_or_else(|_|panic!("Could not find or create Fastsim data subdirectory."));
let file_path = data_subdirectory.join(file_name);
// I believe this will overwrite any existing files with the same name -- is this preferable, or should we add
// a bool argument so user can determine whether the file should overwrite an existing file or not
download_file_from_url(url, &file_path);
}
}

#[cfg(test)]
Expand Down
Loading