Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weather Condition generating locally #17

Open
GlennGoddard opened this issue Jun 11, 2021 · 91 comments
Open

Weather Condition generating locally #17

GlennGoddard opened this issue Jun 11, 2021 · 91 comments
Labels
enhancement New feature or request

Comments

@GlennGoddard
Copy link
Contributor

GlennGoddard commented Jun 11, 2021

The below are the only inputs allowed for HA weather condition. Currently this is pulled from the web API. I was toying with the idea of expanding my weather condition template to pull this locally. I understand I can't get the forecast local only; but I like as much self-hosted as possible. I think the only things I can't verify is clear-night, due to light conditions at night. Thoughts below:
‘clear-night’ can't check if fog at night since no light level to compare to (open to ideas)
‘cloudy’ can get a base line lux amount and compare and if not any other precipitation
‘fog’ same as cloudy but with window below a determined temperature
‘hail’ sensor for this
‘lightning’ sensor for this (maybe any number in last 3 hours)
‘lightning-rainy’ combine sensors for this
‘partlycloudy’ same as cloudy, with maybe a trend/derivative sensor to determine intermittent light levels
‘pouring’ above a certain rain rate
‘rainy’ sensor for this
‘snowy’ a little more complicated but maybe temperature combined with blocked UV sensor and rain sensor
‘snowy-rainy’ same as snowy but easier to assume around 0C / 32F
‘sunny’ easiest just base on UV/lux level
‘windy’ wind above a certain speed or gust above a certain speed
‘windy-variant’ not sure what this is checking
‘exceptional’ dewpoint between 50-60F, temperature between 68-77F, UV below 2.5 index, not any of the above (except sunny).

So...the actual point of this... should I make my condition template and then PR to the readme example or should we incorporate this to give another calculated sensor output? I'm leaning toward a sensor, but this would add a lot more code and wanted to run it past you before a PR without a big change with any other info. I would keep the current web api returned status as you have it and create some other named sensor. Thoughts?

@chasut
Copy link

chasut commented Jun 11, 2021

This is high on my list as well, and thinking through how I might do the same locally. I have already expanded my use of current conditions/icon selection in HA with a template sensor and pulling information from multiple sources, just not to the extent as you contemplate above... yet. I suppose the HA weather component is necessarily limited to a LCD of conditions. To me, partly cloudy, mostly cloudy, and cloudy are three very distinct and recognizable conditions. And I was hoping to be able to distinguish between them with logic as you mention. I am also not a huge fan of how WeatherFlow does its current condition. They will use something like "rain possible" even though the sky is clear or partly cloudy. To me, current condition is what it is NOW, the possibility of rain is a forecast. Dark sky does a better job of this, so for now I am using their current conditions to drive much of my NOW, sensor, at least until apple shuts it down next year.

I am certainly happy to contribute to this however I can. I think it could be a significant value-add. But I dont think the calculations will be trivial to make it work for everyone. Local latitude, current season, and sun angle/time of day will all likely need to be factored in. So, it may be more feasible as a suggested template sensor with perhaps some guidance on tweaking for local use. In any case, count me in.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Jun 11, 2021

What spurred it for me was exactly what you said above, I had a condition of raining as reported by api but there was no rain. I would like it to be actual not predicted just at you stated.
I will am open to any ideas on accomplishing any of the determinations. I think I have a handle on most of them but, as stated, I don't know how to handle some conditions at night mainly clear/cloudy/foggy. I might be able to use dewpoint and temperature for a fog determination.
I do have some other sensors at my disposal but I want to try to do this just thru my Tempest so it will benefit others also. I have a PurpleAir air quality sensor that determines particulate matter in the air; I could use it to determine between smoke and clouds but that is trivial and I'm ok with cloudy vs smoky; especially since there is no smoke in weather card.

On a slightly separate note you could look up purpleair on the HA forums to see some of the gauges I've done. If you look up pollen, I have an updated gauge with a color coordinated background to describe conditions such as cool for temperature or dry for dew point.

I will see what I can knock out this weekend. I am open to ideas either now or after I post my first attempt to see if anything can be smoothed over.

@chasut
Copy link

chasut commented Jun 11, 2021

You know, before your post, I was blissfully unaware of the Purlple Air sensor. Now I NEED one! My wife is not going to be happy with you ;)
Good to hear I am not the only one that feels that way. I will post something on Monday on the Weatherflow forum re: the current conditions. Their AI should be able to do a great job with satellite and radar data if they were so inclined. Dark sky has the best model for my area, and they are right most of the time. But they may not be around for ever. And like you said, it would be best if it could be done all local.

@briis
Copy link
Owner

briis commented Jun 12, 2021

Hi both,
I believe this is an interesting topic, but as you both point out, it can be tricky to get all the conditions right, based on the data we have local access to.

I guess the best way to start is by what Glenn has already done, trying to make some algorithms for each condition, and then tune these a as we go a long. I will be happy to make a new sensor, we could call it local_condition and then start putting in the math.

If everything fails, what would be the default condition: Cloudy?

I think it is good that you posted on the forum Glenn, there might be others already doing some of this.

As soon as I have time, I will start building it, based on what is already here.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Jun 13, 2021

@chasut enjoy:
1

I am still tweeking the backgounds on Dewpoint and Temperature that is why there are no words there yet. The UV background started it all, I saw someones background for their UV index and I modified it for other things. I was using an integration that pulled the UV data before, but I removed that integration since I have the Tempest now. So from this the UV is the only thing coming from the Tempest, I still am using the PurpleAir for Temperature/Dewpoint/Humidity and of course Air Quality. The Tempest and PurpleAir track fairly well except for humidity; the Tempest seems to run high sometimes like 99 and 100% too often.

Here is my other dash (AQI is duplicated here)
gages2
Unfortunately for me the allergy/asthma/cold&flu is pulled from an cloud API. There are local options for all but the Cold&Flu, but they are a little pricey; the allergy monitor I looked at was $8000. I have most of the Asthma already with my PurpleAir since 2.5PM is a big part of it; I have not priced ozone detectors yet.

The PurpleAir has a local web server on the device that allows you to look at the device directly, or you can look at the web map, or you can pull the data from the Rest API (local and/or web) (which is what I do).
I got the PurpleAir after wildfires last year and I do not regret that purchase at all. My wife has asthma and that little bit of information helps her out a lot.

@briis briis added the enhancement New feature or request label Jun 15, 2021
@briis
Copy link
Owner

briis commented Jun 15, 2021

I might do this, but it will not be prioritized right now. But please keep posting how you would calculate the different Conditions, then I will start adding when I have time.

@GlennGoddard
Copy link
Contributor Author

I'm not sure what happened, but I had updated a lot of the conditions in the above post...I probably closed the tab without saving the edit.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Jun 15, 2021

Establishing order of checks (priority):
‘lightning-rainy’ 10 < (sensor.wf_lightning_count_3_hours < 450 (attempt to eliminate false reports from noise on detector) & sensor.wf_precipitation_type > 0

‘lightning’ 3 hour lightning count 10 < (sensor.wf_lightning_count_3_hours < 450 (attempt to eliminate false reports from noise on detector) & sensor.wf_precipitation_type > 0

‘hail’ sensor.wf_precipitation_type = 2

‘snowy-rainy’ sensor.wf_precipitation_type > 0, sensor.wf_temperature = 32F / 0C

‘snowy’ --- snowy is a guess since there is no 'sensor' that detects snow
(temperature in C) sensor.wf_temperature < 1.44 - 0.08 * sensor.wf_humidity)
(temperature in F) sensor.wf_temperature < 34.29 - 0.14 * sensor.wf_humidity)
and
sensor.wf_temperature < 40F / 4.44C
and
sensor.wf_illuminance < 1000 if sun angle > 0

‘pouring’ sensor.wf_rain_rate > 0.31 in/hr (7.8 mm/hr) or sensor.wf_precipitation_type = 3

‘rainy’ sensor.wf_precipitation_type = 1

‘windy-variant’ Wind Speed > 25mph (11.17 m/s) & cloudy

‘windy’ Wind Speed > 25mph (11.17 m/s)

‘fog’ Fog is likely when:
Air Temperature - Dew Point < 4.5F (2.5C)
Relative Humidity > 95%
Wind Speed (above) > 5 knots / 5.7 mph / 2.57 m/s
Wind Speed (below) < 10 knots / 11.5 mph / 5.14 m/s
Clear skies overnight (???) maybe from Pressure
'> -40F (typically >14F)
*** near seawater >70% Relative Humidity and wind speed <10 knots (5.14 m/s)could cause fog (might skip this caveat for now)
Need more conditions for fog, it looks like this might throw a lot of false fog.
There is some trends for heating/cooling but most would not have a derivative sensor for air temperature.
Cooler air temp than ground temp, determine if daily temp max (24hr) is greater than current temp (more to come)
--Use visibility calculation as condition

‘cloudy’ reduced luminescence with a steady tread

‘partlycloudy’ trend of alternating lower and higher lux

‘clear-night’ sun angle < 0 and no other events

‘sunny’ no other events and sun angle > 0 (minimum of 10,000 lux)

‘exceptional --?? (looked at some weather cards and does not seem to be used)

Need to look at more data and adjust for latitude and time of day.
0.0001 lux – Moonless, overcast night sky.
0.002 lux – Moonless clear night sky with airglow.
0.27–1.0 lux – Full moon on a clear night.
3.4 lux – Dark limit of civil twilight under a clear sky.
100 lux – Very dark overcast day.
400 lux – Sunrise or sunset on a clear day.
1000 lux – Overcast day
10000–25000 lux – Full daylight (not direct sun).
32000–100000 lux – Direct sunlight.

@GlennGoddard
Copy link
Contributor Author

Still gathering information. Discussed with an individual on the WeatherFlow forums that calculates cloudy based on UV level comparison with historical data for each month for a given latitude; I could go this route but this complicates it much more than I anticipated. I am still looking for a 'simple' formula for UV or even solar radiation for latitude. I might end up breaking it up to latitude segments of 5-10 degrees for an approximation and then see how to figure the bell curves for daylight for a given time of year for a given latitude.

I will test what I have as a sensor in Home Assistant and fine tune as needed. Once I have it fairly reasonably working for me, then I'll put it on the forum for HA and hopefully get others at different latitudes to test.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Jun 28, 2021

Working on 'normal' UV levels.
I did not use the standard symbols for below, just a description.
This is just for documentation at this time, nothing to be implemented yet until I get further along.

Need the following:

Solar Declination Angle = 23.45 * sin(360/365(d-81))
d is Julian day (ie January 1st is "1")

Max Elevation at Solar Noon = 90 + h * (Latitude - Declination Angle)
h is +1 for Northern Hemisphere, -1 for Southern Hemisphere (imputed variables like elevation)
If solar elevation is >90 then subtract from 180

Elevation (already available)
for each 300 meters in elevation above sea level, UV levels increase by 4%

Assumption that weather station UV sensor sees direct sunlight most of the day.

More calculations to come.

@briis
Copy link
Owner

briis commented Jun 29, 2021

I will start looking at this, in a few weeks when my vacation is coming up, and see what I can do.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Jul 1, 2021

I think I have to reevaluate how I am determining cloudy. Today was completely overcast but UV levels were up significantly. I know that clouds can reflect some UV back down but I saw UV index of 6 without clouds (two days ago) and 11.76 with clouds (today). Solar Radiation and Light Levels were both down though (yes, I know that UV is a different sensor).

I might go with comparing current light levels to calculated light levels (based on time of day, latitude, and time of year); and also a comparison of current UV to current LUX.

I think I can get away with only calculating for the Northern Hemisphere (NH) and then make a 180 day adjustment for the Southern Hemisphere (SH). The one thing I can not get locally is Ozone levels which affect UV levels, but they are fairly constant during the SH summer/ NH winter Jan/Feb/Mar.

This has become a much deeper rabbit hole than I expected and I'm still digging down.
I am almost to the point of making my predicted UV/Lux sensor in HA so I can compare to actual and also clouds.

It is very easy to look at either the UV or Lux or Sol Rad graphs and see when there are clouds. This could be used if the day started out sunny and cloud(s) moved in causing a more rapid than drop than sunset; but if full clouds for the whole day then there would not be any drops.

Just documenting my thoughts.

@GlennGoddard
Copy link
Contributor Author

@briis is there a easy way to determine if the forecast connection is down? I know there is a similar issue here but my need is to provide the local conditions (mainly the current conditions icon) only when there is no internet. Yesterday I was without grid power and internet at home for 7 hours and I had all my WF sensors but I found that I really missed my big animated icon of the current weather conditions. So I was thinking of supplying my estimations of current conditions when either there is no connection to the weather flow server or someone has opted to not get forecast data.
I will not be able to cover all conditions but I can get it pretty close, at least for my location. I might just start with the easy status like rain (day), rain (night), sunny (day), clear (night). I won't be able determine clouds at night and clouds in day still require some work on my end but the previous is easy then I can start adding from there.

@briis
Copy link
Owner

briis commented Oct 27, 2021

@GlennGoddard There are couple of ways to test if the connection fails and/or the data we have for forecast is empty, and then based on that switch to the local format. I guess the most important part is that the dataset is formatted in the same way as the data we retrieve from WeatherFlow. I have attached a sample json file - this is what gets returned when calling the forecast. I can help with generating that, if you make the dataset and then implement the switch-over function.

forecast.json.zip

@GlennGoddard
Copy link
Contributor Author

I have not abandoned this, I have been very very very busy lately; hopefully I will have some time over the Christmas break to work on it.

@briis
Copy link
Owner

briis commented Nov 23, 2021

No worries.

@briis
Copy link
Owner

briis commented Dec 2, 2021

The Zambretti Forecast is the same that my Davis Vantage station delivers. Honestly I have never checked how accurate that is, but I assume it is better than nothing, if without Internet connection.

@badewanne1234
Copy link

badewanne1234 commented Apr 8, 2022

It would be great to the weather condition locally! I also reaslied a couple of days ago that Tempest Weather Sensor showed rainy despite there was no rain.
The forecast, provided by the government with local weather stations everywhere, stated partlycloudy instead, which was correct. Luckily, there is an integration for that provided forecast for HA (any other forecast such as SKY or whatsoever will work too).
You can take first your local weather forecast and then tempest, or, even better, first your local weather forecast, then your "country-forecast" which is probably better then temptest, and then, tempest.

I can't really contribute on this feature of full local forecast, but for now, I created a sensor which merges the weather condition in the following priority:

  1. take the local Tempest forecast which is rain, pouring, hail, lightning, lightning-rainy and windy - this works offline

Since we can't calculate anything which involves a "mixture" where we only have one of the two conditions (rain and wind), or if it is exceptional (in particular: snowy-rainy, windy-variant and exceptional), we need to check the internet forecast first. For example, we need first to check if its snowy-rainy (Internet) and just after that, if its rainy (Local), otherwise we will be having a wrong state (rainy instead of snowy-rainy.

You may adapt the parameters to your needs (lightning/lightning-rainy: strike <=1km away within the last 10mins, windy: 32-48km/h).
I created a km/h sensor to convert the m/s that is provided by the integration. If you have the m/s sensor only, then divide 32-48km/h by 3.6 and you're good to go

  1. if the above is not true, then take the condition from the "country-weather-forecast" (relies on an internet connection).

  2. if the above both is not true, then take the Tempest online weather forecast (relies on an internet connection).

You can then use this sensor to create a weather entity which has, at the time of writing, the most possible local weather condition :)

Here is the code for your configuration.yaml, you may need to adapt to your local translation (currently, 4 languages are supported by the integration. XYZ is the entity of your "other" online forecast:

# Sensor Weather state merged (Local, XYZ, Weatherflow)
  - platform: template
    sensors:
      weather_state_merged:
        value_template: >
          {% if states('weather.xyz') == 'exceptional' %}
            {{ states('weather.xyz') }}
          {% elif states('sensor.weather_home_lightning_distance') <= '1' and (as_timestamp(now()) - as_timestamp(states('sensor.weather_home_last_lightning_strike'))) < 600 and (states('sensor.weather_home_precipitation_type') == 'Rain' or states('sensor.weather_home_precipitation_type') == 'Heavy Rain')%}
            {{ 'lightning-rainy' }}
          {% elif states('sensor.weather_home_lightning_distance') <= '1' and (as_timestamp(now()) - as_timestamp(states('sensor.weather_home_last_lightning_strike'))) < 600 %}
            {{ 'lightning' }}
          {% elif states('weather.xyz') == 'snowy-rainy' %}
            {{ states('weather.xyz') }}
          {% elif states('sensor.weather_home_precipitation_type') == 'Hail' %}
            {{ 'hail' }}
          {% elif states('sensor.weather_home_rain_rate') | float >= float(10) %}
            {{ 'pouring' }}
          {% elif states('sensor.weather_home_precipitation_type') == 'Rain' %}
            {{ 'rainy' }}
          {% elif states('weather.xyz') == 'windy-variant' %}
            {{ states('weather.xyz') }}
          {% elif float(32) <= states('sensor.weather_home_wind_speed_avg_kmh') | float <= float(48) %}
            {{ 'windy' }}
          {% elif not states('weather.xyz') == 'unavailable' or not states('weather.xyz') == 'unknown' %}
            {{ states('weather.xyz') }}
          {% else %}
            {{ states('sensor.tempest_hub_weather') }}
          {% endif %}
        friendly_name: Weather home State Merged
        icon_template: mdi:weather-partly-cloudy

@GlennGoddard
Copy link
Contributor Author

@badewanne1234 this may not be a bad approach to compare local station data to WeatherFlow's current condition when there is internet and the forecast option is setup. I was working off the idea of providing the current conditions completely locally, to allow for no internet; but I like your idea of comparing local and web data for the current condition.

@badewanne1234
Copy link

@GlennGoddard
Well its not really "comparing", its preferring local weather condition if available, else take the online one. Of course I'd rather prefer to have all states to be taken local, but as stated, that goes byond my capabilities :)

So for now, I'm sticking with the above approach, and for me it works very well, the current weather condition is much more accurate now. I'm also updating the above post, I'm still finetuning the weather state entity. It will be very interesting to see how the lightning calculation works, I probably need some tweaks there, but for that I'll need data from a lightning event...
I have additional timers setup (not includeded in the above post) to not have "cloudy" as soon as it stops raining, I wait a little while before switching from "rainy" to "cloudy" for example.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Apr 29, 2022

Narrowing in on Cloudy (will need a more data points at solar noon on a sunny day).
Overview: compare measured solar radiation to calculated
Still work in progress

The higher the solar elevation the higher solar radiation (the assumption is made that the weather station's sensor is parallel with ground). 90 degress is max solar elevation.

Solar Elevation: α = sin^(-1)((sin(ϕ) * sin(δ)) + (cos(ϕ) * cos(δ) * cos(h)))
where
α is Solar Elevation
δ is declination
h is angle hour
ϕ is latitude

δ = (-23.44) * cos(360/365 * (d + 10))
where
d is number of days since January 1st (julian date) + fractional for hours

h = 15 * (LST -12)
where
LST is local solar time in 24 hr format

LST = LT + TC/60
where
TC is Time Correction Factor
LC is Local Time (24hour with fractional minute i.e. 30 mins = 0.5)

TC = 4(λ - LSTM) + EOT
where
λ is Longitude
LSTM is Local Standard Time Meridian
EOT is Equation of Time

LSTM = 15 * ΔTutc
where
ΔTutc is the difference of local time from UTC (also equal to the time zone) (example Los Angeles is -7)

EOT = 9.87 * sin(2*β) - 7.53 * cos(β) - 1.5 * sin(β)
where
β = 360/365 * (d - 81)

I think the way I will go with this is to convert this to a percentage of max solar radiation for a given time and day and compare that with the measured solar radiation. If in a certain (yet to be determined) range then it is sunny. Specifically within a range since higher than 'max' solar radiation is an indication of fluffy white clouds reflecting UV light back down to the ground (as long as not directly between sun and monitor i.e. blocking direct sun light). If outside that range and no other conditions such as rain then cloudy. This also makes the assumption that station is never in the shade (not always practical but the easiest approach). This is also only valid during 'normal' daylight hours.

Once I have a good working approximation for the Northern hemisphere, I will try to simulate for the Southern. I 'think' I have that already figured into the above calculations but I'm not entirely sure.

As much as I would love to continue to use the Greek alphabet in the code, I will use more appropriate Latin characters for variable names.

@badewanne1234 I can always use more data points to see how accurate this is; admittedly this is not ready for anyone else to try since I don't have the approximation done yet but any input is welcome.

I have considered the consideration of solar eclipses but I don't know if I want to go there. It could be done with some packages but I don't think it is worth the effort (maybe).

@badewanne1234
Copy link

wow, looks like you're an expert! Let me know if I can assist with testing etc

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Apr 29, 2022

@badewanne1234
If it is not inconvenient, could you log solar radiation at solar noon if no clouds present along with the date. As you can get the data is fine; I can (unfortunately) go months without any direct sun in the winter. Eventually I would like data with clouds but I don't want too much data (yet).

Things that would help, but not required if you don't feel comfortable:
Elevation above sea level of the station
Lat & Long (if more comfortable approximate location is fine)

Don't post here yet. Probably IM in either the Home Assistant or WeatherFlow community boards.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Apr 29, 2022

Partly Cloudy and Cloudy, I still have to figure out.
I think I need a time comparison for partly cloudy conditions to see if there are any up/down changes in solar radiation during a time frame (excluding solar noon, sunrise, and sunset).

I had no idea that figuring out local conditions would be this complicated when I started this.

@badewanne1234
Copy link

@badewanne1234 If it is not inconvenient, could you log solar radiation at solar noon if no clouds present along with the date. As you can get the data is fine; I can (unfortunately) go months without any direct sun in the winter. Eventually I would like data with clouds but I don't want too much data (yet).

Things that would help, but not required if you don't feel comfortable: Elevation above sea level of the station Lat & Long (if more comfortable approximate location is fine)

Don't post here yet. Probably IM in either the Home Assistant or WeatherFlow community boards.

Sure not a problem.
With solar radiation, you mean illuminance, right?
So ideally, I would log illuminance, sun elevation, date&time and the weather condition... Lets see if there is something available to log this automatically into a CSV file.

@GlennGoddard
Copy link
Contributor Author

@badewanne1234
Actual Solar Radiation measurement; same line in the APP as UV thou. The units are W/m^2
The tempest has UV, Brightness and Solar Radiation.
Brightness and Solar Radiation do use the same sensor and are representative, so maybe that will work.
On second thought, luminescence should work; if you could log both

@GlennGoddard
Copy link
Contributor Author

Update:

I decided to test these in Node-Red since it is easy to modify and it was very easy to pull in lat/long. I still had to establish station elevation since that is not sent to MQTT (nor does it need to be). I am currently testing 'cloudy' based on estimated Solar Radiation vs measured Solar Radiation, my unfortunate issue is that I have lots of data for cloudy days but none with any sun. I am exporting my results from Node-Red to MQTT and then using MQTT explorer to look at the output in graphical form. I am testing a few algorithms for Solar Radiation estimation and I can see that sometimes they appear too high at solar noon but I'm not sure since I have not had a sunny and non-cloudy day in weeks.

If anyone else utilizes Node-Red I would be willing to share my node for weather. I pull in the data from the Tempest via Home Home Assistant and not MQTT nor UDP directly. My intention is to get this working as much as possible before incorporating into this repo. So far we would need Latitude and Longitude added as a variable (just like elevation is already provided). I am making the assumption that the time for this container is in accurate local time (stating this in the case anyone is using an isolated machine that does not have a good time source); I think currently as long as time is within 16 mins then it will work ok. Just to note that time is important since the earth rotates 1 degree every 4 minutes, which in turn effects the received (and estimated) sunlight on the top of the weather station and therefore the Solar Radiation.

@badewanne1234
Copy link

badewanne1234 commented May 6, 2022

my temptest is exposed to the sun directly. I have some data from the DB, but I need to compare the timestamps from the different sensors (working on this). I might do a live logging instead (write to file every 2mins or so), it is supposed to be sunny mid next week here.

@rhysby
Copy link

rhysby commented Jul 28, 2022

Thanks Glenn, I've sent you a PM on the HA forum.

@GlennGoddard
Copy link
Contributor Author

Could you compare solar elevation that this outputs to what it is in your area? I think I have this right, but just want to verify.

It looks like I need to do some modifications for the southern hemisphere in the calculations for the actual estimation of Solar Radiation.

I will see if I can resurrect my node-red node for this testing. I had it where every little sub calculation had it's own block, that way I could easily find and make changes. I can let you fine tune it for your location and then with other input from people in other locations I can try to make a generalization for anywhere in the world.

@badewanne1234
Copy link

@GlennGoddard for my understanding, zambretti_text is different from local_weather_conditions, right? I saw on the documentation, that local_weather_conditions are still in development, on the other hand I read above that you have completed the local_weather_conditions (have you maybe not pushed that yet?) thank you

@GlennGoddard
Copy link
Contributor Author

Yes Zambretti is it's own sensor seperate from local current conditions. I have the current conditions effectively done, but life has gotten in the way of me finalizing and putting in the PR. So all your statements are correct. As soon as life will let me get it done, I'll get it in. There is not much left for me to do, other than make sure I correctly convert everything I have in JavaScript to Python.

@rhysby
Copy link

rhysby commented Jul 28, 2022

Ok, send it through when you are ready.

@GlennGoddard
Copy link
Contributor Author

Submitted PR for Fog Probability, if that works then Snow Probability will be next and then that should cover all remaining perquisites for local current conditions.

I still need to fine tune Solar Insolation for other Latitudes but I need more data from other latitudes

@GlennGoddard
Copy link
Contributor Author

Fog and Snow Probability are in the Dev build (I'm still working some issues, but sensors are getting to HA).

Fog has issues still, won't be able to verify Snow for a few months (myself anyways).

Note to self: try to convert ALL syntax from java-script to python.

@badewanne1234
Copy link

Hey @GlennGoddard great work as always, now with the Snow and Fog Probability which just arrived in 3.1.4.
Is this still correct, that were "your" last needed sensors before a local forecast can be implemented? Assuming it would be based on the Zambretti Algorithm?

@GlennGoddard
Copy link
Contributor Author

Zambretti is a forecast based off the Zambretti Forecaster wheel. Current Conditions is an attempt to mimic the results of the 'current conditions' received from WeatherFlow but locally.

I have actually had everything needed for some time now, and honestly I forgot about finishing moving it over to this integration since I have been running my own test state. I will see if I can port everything over this weekend.

@badewanne1234
Copy link

That would be amazing!! I would love to have full local conditions, right now this is only partially the case for me..

@GlennGoddard
Copy link
Contributor Author

@badewanne1234
There are caveats (given just the sensor in the Tempest):
Can not determine if cloudy at night
Solar Eclipse will come across as cloudy; I actually considered planing for this event, but that would add dependencies that would be rarely useful in the scheme of things.
Snow is an educated guess based on probability of snow
Fog is also an educated guess based on probability, but has been fairly accurate for me; if I don't have fog at the house, usually within a few miles there is fog.
Shade will be considered cloudy or partly cloudy.

I'm sure there will be tweaking of the calculations once more people in different locations start using this and providing feedback.

@badewanne1234
Copy link

I can totally live with these caveats. Do you have any experience yet with the snow probability sensor?
Happy to provide feedback on the sensor status, to improve the quality :)

@GlennGoddard
Copy link
Contributor Author

I will probably end up with two different sensors for Current Conditions:
(1) Main function will create a sensor with the text Home Assistant expects for weather (not exactly human readable)
(2) Sensor that converts above sensor to human readable format, this will require translations which might all be available from the Forecast section (need to verify)
Basically this will be identical to the way Zambretti Number (equivalent to 1) is converted to Zambretti Text (equivalent to 2); not exactly, but gets the point across. Translations took a little time afterwords to get sorted (thanks to other users). I will steal the translations that I can, but I might be missing some (I have not verified yet).

I 'should' be able to get (1) done this weekend. (2) won't be too hard but not 'essential' for Current Conditions, just really helpful.

@GlennGoddard
Copy link
Contributor Author

@badewanne1234 I do have some experience with the Snow Probability, and as a result the calculation was modified. It was not 'wrong' per se; I just neglected to to cap the maximum probability of snow. The way it was that if all conditions were met for snow and the temperature went into the teens (Fahrenheit) this would send a probability over 100%. The snow probability calculations came from someone else, I just put them to use with a little modification. It is not perfect, but it has proven to be more accurate then the local news weather forecasters. The more input we get from people in various locations, hopefully, the better the calculations can be.

@GlennGoddard
Copy link
Contributor Author

PR #204 submitted for local current conditions but just the Home Assistant accepted inputs. My brain is fried from too many other things, so I'm sure it is not perfect. The translations for Human Readable Current Conditions will come later once this is ironed out.

@briis
Copy link
Owner

briis commented Jan 3, 2023

I am busy all week, so I will not have time to look at the PR before the weekend.

@badewanne1234
Copy link

@briis any joy with looking into the PR yet?

@briis
Copy link
Owner

briis commented Feb 4, 2023

It is now merged and published to the dev tag. However, at my end it does not work, as there are some errors in the formula itself. I have asked @GlennGoddard to look at that.

@GlennGoddard
Copy link
Contributor Author

wind_speed was not called
snow_probability was called in wrong order

@briis
Copy link
Owner

briis commented Feb 5, 2023

Just pushing a new dev build, which now returns current_condition. Please note it will take a little while before it is pushed as there are many values that need to have data before it works.
Question: Should we have a default current_condition, so that it does not become None - could be cloudy or similar

@GlennGoddard
Copy link
Contributor Author

That is not a bad idea. For me, Cloudy is the norm in the winter. We could pass 'exceptional', as a tell-tale; not sure how the animated weather card I use would handle it since it doesn't have any icons for exceptional. I think sunny or clear-night would be better for most. Clear Night would be the most likely at night and would indicate a reboot/issue during the day.

@briis
Copy link
Owner

briis commented Feb 5, 2023

OK. I will change from None to clear-night if data is not available. Pushing it out when done, and then let me know if this is ready to be released. (README should be updated I guess)

@GlennGoddard
Copy link
Contributor Author

I need to look at fog and cloudy; they seem to be swapped with each other. When my node-red says fog, this says cloudy; when node-red says cloudy, this says fog. Everything else seems to track.

Untitled

I have my Node-Red output in 'readable' text, the capital is intentional in this case.
I will work on the readme and the plain text output with translations.
I'll probably incorporate the other issue on translations also at that time.

I'm not sure why I am inverting the fog/cloudy....???

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Feb 5, 2023

@briis
ReadMe was already updated previously in the PR for ReadMe updates from a few days ago.
Only change still needed in ReadMe is when it goes from Dev to Stable build, then the (Dev Build) comments can be removed.
Not sure if you update the version number on each dev build or only on a stable update?

@badewanne1234 current conditions is in the dev build, I have to find one issue before it reaches stable. Fog and Cloudy are swapped....it is bugging me that don't see why yet.

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Feb 6, 2023

Fog and Cloudy is not always reversed; it might be a metric/imperial thing...I set it up for all metric...so something might be translated to imperial. Wind Speed is probably the likely culprit but not sure yet. It might also be that the calculation are done with slightly different data since I was only performing every 3 mins vice every 3 seconds here...should not make a difference...but.

I think if I make a tweak to when Fog is called to a higher number from the current of 50% then that would change it.

I finished all the different translations, might need some review there. I don't have the function called yet but the work in the translation directory is done.

@GlennGoddard
Copy link
Contributor Author

Well...that is what I get for 2nd guessing myself....no fog visible at my house today...but I'm watching the news and massive fog everywhere around me.... I'm torn.... I guess I will keep this for now

@badewanne1234
Copy link

Hey @GlennGoddard have you made further progress on this? Last update was that you found an issue, need to fix before you push it to stable. I would love to have fully local conditions 💪🏻💪🏻

@jazzyisj
Copy link
Contributor

Been monitoring the local current conditions sensor and it hasn't been very accurate for me yet.
Just thought I'd throw my observations in the hat here. v3.2.2

  • Seems to default to lightning.
  • The sensor does show rain when the precipitation type is rain but it always indicates lightning-rainy when no lightning present (in reality and the lightning sensors).
  • It infrequently indicates sunny or cloudy, but for some reason last night it showed clear-night (green bar) for a while when it was still quite cloudy.

The top entity here is the current condition from Environment Canada which is usually quite accurate as I'm only a couple of kilometers from the weather station.

image

@GlennGoddard
Copy link
Contributor Author

GlennGoddard commented Oct 15, 2023

@jazzyisj
The clear-night is a known issues that will never be able to distinguish clouds at night.
I would love more input from other regions to improve this.
If you use Discord on WeatherFlow or HomeAssistant that would be better to converse and share info.
WeatherFlow: glenngoddard
HomeAssistant: GlennHA
Solar
If you could make a graph of expected and actual solar radiation, my example has an additional testing formula; and see if you are getting a partially cloudy or cloudy result.
Do you use Node-Red at all? If you do I can have you test some formula modifications.
As for the lighting, it checks lightning in the last-hour, if any then it states lighting, but if you get zero in that block then I need to do some more digging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants