Get best model parameters/coefficients

[AFilipaVEL]

Hi all,

I'm estimating an ETS model, using autoETS(). I would like to know if it's possible to obtain the coefficients from the final model. My code is:

import os
os.environ['NIXTLA_ID_AS_COL'] = '1'

from statsforecast import StatsForecast
from statsforecast.models import AutoETS

models = [AutoETS()] 

fcst = StatsForecast(models=models,
                     freq='M',
                     n_jobs=1)

mod_v = fcst.forecast(df=imputed_v, 
                      h=1, 
                      level=[90])

Thanks in advance!

[jmoralez]

Hey. The forecast method doesn't keep any model information, it's just to forecast. If you want to keep the trained models you can use the fit method instead, once you've done that you can access the models parameters. So something like the following should work:

fcst.fit(df=imputed_v)  # this saves the trained models in the fitted_ attribute, which is an (n_series, n_models) array
print(fcst.fitted_[0, 0].model_)  # access the first model

[AFilipaVEL]

Thank you, José

Unfortunately it doesn't work. Here's the error:

AttributeError                            Traceback (most recent call last)
File <command-3806312540048459>, line 1
----> 1 fcst.fit(df=imputed_v) 

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-30d55c40-b1b1-41df-948d-390d8a964034/lib/python3.10/site-packages/statsforecast/core.py:636, in _StatsForecast.fit(self, df, sort_df, prediction_intervals, id_col, time_col, target_col)
    607 def fit(
    608     self,
    609     df: Optional[DataFrame] = None,
   (...)
    614     target_col: str = "y",
    615 ):
    616     """Fit statistical models.
    617 
    618     Fit `models` to a large set of time series from DataFrame `df`
   (...)
    634         Returns with stored `StatsForecast` fitted `models`.
    635     """
--> 636     self._prepare_fit(
    637         df=df,
    638         sort_df=sort_df,
    639         id_col=id_col,
    640         time_col=time_col,
    641         target_col=target_col,
    642     )
    643     self._validate_sizes_for_prediction_intervals(prediction_intervals)
    644     self._set_prediction_intervals(prediction_intervals=prediction_intervals)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-30d55c40-b1b1-41df-948d-390d8a964034/lib/python3.10/site-packages/statsforecast/core.py:548, in _StatsForecast._prepare_fit(self, df, sort_df, id_col, time_col, target_col)
    546     _warn_df_constructor()
    547     return
--> 548 df = ensure_time_dtype(df, time_col)
    549 validate_freq(df[time_col], self.freq)
    550 if isinstance(df, pd.DataFrame) and df.index.name == id_col:

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-30d55c40-b1b1-41df-948d-390d8a964034/lib/python3.10/site-packages/utilsforecast/validation.py:51, in ensure_time_dtype(df, time_col)
     48 """Make sure that `time_col` contains timestamps or integers.
     49 If it contains strings, try to cast them as timestamps."""
     50 times = df[time_col]
---> 51 if _is_dt_or_int(times):
     52     return df
     53 parse_err_msg = (
     54     f"Failed to parse '{time_col}' from string to datetime. "
     55     "Please make sure that it contains valid timestamps or integers."
     56 )

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-30d55c40-b1b1-41df-948d-390d8a964034/lib/python3.10/site-packages/utilsforecast/validation.py:35, in _is_dt_or_int(s)
     34 def _is_dt_or_int(s: Series) -> bool:
---> 35     return _is_dt_dtype(s) or _is_int_dtype(s)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-30d55c40-b1b1-41df-948d-390d8a964034/lib/python3.10/site-packages/utilsforecast/validation.py:30, in _is_dt_dtype(s)
     28     out = pd.api.types.is_datetime64_any_dtype(s.dtype)
     29 else:
---> 30     out = s.dtype in (pl.Date, pl.Datetime)
     31 return out

AttributeError: 'NoneType' object has no attribute 'Date'

[jmoralez]

Is imputed_v a spark dataframe? If it is then we don't support the fit method for distributed dataframes at the moment.

[AFilipaVEL]

It is a spark dataframe :( are there any perspectives of new developments on this subject?

[jmoralez]

We have #642 tracking it, we'll implement it soon.

[AFilipaVEL]

Great! Thanks a lot!

[AFilipaVEL]

Hi!

I read somewhere that this feature was already available but i'm still getting an error while executing the following code:

models = [AutoETS()] 

fcst = StatsForecast(models=models,
                     freq='M',
                     n_jobs=1)

fcst.fit(df= d_train)

d_train is a spark dataframe.

AttributeError: 'NoneType' object has no attribute 'Date'
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File <command-1057362304605362>, line 1
----> 1 fcst.fit(df= d_train)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-e42996b9-9863-417f-ae31-0e825698751e/lib/python3.10/site-packages/statsforecast/core.py:636, in _StatsForecast.fit(self, df, sort_df, prediction_intervals, id_col, time_col, target_col)
    607 def fit(
    608     self,
    609     df: Optional[DataFrame] = None,
   (...)
    614     target_col: str = "y",
    615 ):
    616     """Fit statistical models.
    617 
    618     Fit `models` to a large set of time series from DataFrame `df`
   (...)
    634         Returns with stored `StatsForecast` fitted `models`.
    635     """
--> 636     self._prepare_fit(
    637         df=df,
    638         sort_df=sort_df,
    639         id_col=id_col,
    640         time_col=time_col,
    641         target_col=target_col,
    642     )
    643     self._validate_sizes_for_prediction_intervals(prediction_intervals)
    644     self._set_prediction_intervals(prediction_intervals=prediction_intervals)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-e42996b9-9863-417f-ae31-0e825698751e/lib/python3.10/site-packages/statsforecast/core.py:548, in _StatsForecast._prepare_fit(self, df, sort_df, id_col, time_col, target_col)
    546     _warn_df_constructor()
    547     return
--> 548 df = ensure_time_dtype(df, time_col)
    549 validate_freq(df[time_col], self.freq)
    550 if isinstance(df, pd.DataFrame) and df.index.name == id_col:

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-e42996b9-9863-417f-ae31-0e825698751e/lib/python3.10/site-packages/utilsforecast/validation.py:51, in ensure_time_dtype(df, time_col)
     48 """Make sure that `time_col` contains timestamps or integers.
     49 If it contains strings, try to cast them as timestamps."""
     50 times = df[time_col]
---> 51 if _is_dt_or_int(times):
     52     return df
     53 parse_err_msg = (
     54     f"Failed to parse '{time_col}' from string to datetime. "
     55     "Please make sure that it contains valid timestamps or integers."
     56 )

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-e42996b9-9863-417f-ae31-0e825698751e/lib/python3.10/site-packages/utilsforecast/validation.py:35, in _is_dt_or_int(s)
     34 def _is_dt_or_int(s: Series) -> bool:
---> 35     return _is_dt_dtype(s) or _is_int_dtype(s)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-e42996b9-9863-417f-ae31-0e825698751e/lib/python3.10/site-packages/utilsforecast/validation.py:30, in _is_dt_dtype(s)
     28     out = pd.api.types.is_datetime64_any_dtype(s.dtype)
     29 else:
---> 30     out = s.dtype in (pl.Date, pl.Datetime)
     31 return out

AttributeError: 'NoneType' object has no attribute 'Date'

Thanks a lot!