Statistic data is not renewed within graphs's live time

[marcelhoogantink]

Describe the bug
Having a bar-graph of statistic data, the graph is not renewed with an extra new bar when the set period of the graph is crossed by time.
In this link in point (2) example it is discussed in more detail.

Screenshots
Here is a screenshot of twice the same graph. The upper one was opened before the set period was crossed, the lower one after:

Yaml

Here's the latest code of the above screen-shot:

type: custom:plotly-graph
title: Plotly
hours_to_show: current_week
stack: false
entities:
  - entity: sensor.p1_meter_totale_energie_export
  - entity: sensor.p1_meter_totale_energie_import
defaults:
  entity:
    texttemplate: '%{y}'
    type: bar
    statistic: state
    period:
      0s: 5minute
      2h: hour
      6d: day
      360d: month
    filters:
      - delta
      - store_var: test
      - fn: |
          ({xs,ys,hass,statistics,get,meta,path}) =>   {   
            let statLast = statistics.length-1;
            let entityNr = path.split('.')[1];  
            let entity = get('entities['+entityNr+'].entity');  
            let lastStatisticsDateTime = new Date(statistics[statLast].end);   
            let lastStateDateTime = new Date(hass.states[entity].last_changed);   
            let stateLastStatic = statistics[statLast].state; 
            let stateLastState = hass.states[entity].state;
            let state = stateLastState-stateLastStatic; 
            if (state>0){   
              if (lastStateDateTime.getTime()>lastStatisticsDateTime.getTime()){ 
                xs.push(lastStatisticsDateTime);
                ys.push(state);
              }else{
                ys[statLast]+=state; 
              }
            }
            debugger;
            return ({ xs,ys }) 
          }
      - fn: console.log
refresh_interval: auto
fn: |
  $fn({getFromConfig, vars})=> {
    const range = getFromConfig("visible_range");
    const width = range[1] - range[0];
    vars.scroll = (label, p) => ({
      args: [
        {
          layout: {
            "xaxis.range": [range[0] + width*p, range[1] + width*p],
          }
        }, {
          transition: {
            duration: 150,
          }
        }
      ],
      label,
      method: "animate",
    })
    vars.zoom = (label, h) => ({
      args: [
        {
          layout: {
            "xaxis.range": [Date.now()-1000*60*60*h, Date.now()],
          }
        }
      ],
      label,
      method: "animate",
    })
  }
layout:
  barcornerradius: 15
  bargap: 0.1
  bargroupgap: 0.1
  height: 50%
  xaxis:
    tickangle: -45
  updatemenus:
    - buttons:
        - $fn({vars}) => vars.scroll( '<', -.5)
        - $fn({vars}) => vars.scroll( '>', .5)
      direction: right
      active: -1
      pad:
        r: 10
        t: 0
      type: buttons
      x: -0.1
      xanchor: left
      'y': -0.2
      yanchor: top
    - buttons:
        - $fn({vars}) => vars.zoom( '1y', 24*366)
        - $fn({vars}) => vars.zoom( '1m', 24*31)
        - $fn({vars}) => vars.zoom( '1w', 24*7)
        - $fn({vars}) => vars.zoom( '1d', 24)
        - $fn({vars}) => vars.zoom( '1h', 1)
      direction: right
      active: -1
      pad:
        r: 100
        t: 0
      type: buttons
      x: 0.1
      xanchor: left
      'y': -0.2
      yanchor: top

Additional context
I think the statistics of the entities in the graph should be updated right after a period is crossed by time. So the 5_minute-period every 5 minutes, the hour-period every hour, and so on.

[dbuezas]

Thanks.
Note to self: statistics don't auto update. Likely because the cache indicates that it already has requested data for the past period and statistics add data retroactively

[scinos]

In case it helps, I have another instance of the same bug.

First image was loaded around 10:00. Note how the curve "flattens" after that time. Second image is after reloading the dashboard.

.

This is the card config:

- type: custom:plotly-graph
  view_layout:
    grid-area: electricity
  style:
    .: |
      ha-card {
        height: 100px;
      }
      layout-card {
        height: 100%;
        display: block;
      }
  defaults:
    entity:
      extend_to_present: true
      period: auto
      line:
        width: 0
  layout:
    hovermode: "x unified"
  hours_to_show: 24
  entities:
    - entity: sensor.grid_import_energy
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
        - store_var: grid_import
      internal: true

    - entity: sensor.battery_drain_energy
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
        - store_var: battery_import
      internal: true

    - entity: sensor.house_consumption
      name: Usage from PV
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
        - fn: |-
            ({ ys, xs, vars }) => {
              return {
                ys: ys.map( (v,idx) => v - vars.grid_import.ys[idx] - vars.battery_import.ys[idx])
              }
            }
      stackgroup: 1
      fillcolor: "#e0a21f"

    - entity: sensor.battery_store_energy
      name: Battery store
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
      stackgroup: 1
      fillcolor: "#36b336"

    - entity: sensor.grid_export_energy
      name: Back to grid
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
      stackgroup: 1
      fillcolor: "#1d621d"

    - entity: sensor.battery_drain_energy
      name: Battery usage
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
      stackgroup: 1
      fillcolor: "#e06f1f"

    - entity: sensor.grid_import_energy
      name: Grid usage
      filters:
        - resample: 5m
        - map_y_numbers: Math.round(Number(y))
      stackgroup: 1
      fillcolor: "#e03e1f"

[marcelhoogantink]

@dbuezas
This issue is now 10 months old. Is there any change that it wil be solved some time?

[dbuezas]

I hope to get to the issues at some point, but no news for now

[scinos]

FWIW, this only affects statistics entities. I don't fully understand the difference between statistics and regular entities, but I think statistics is only useful if you are going to display lots of past data. That was not my case, so removing period: auto from my config "fixes" the problem (as in, I don't seem to be using statistics data sources anymore).