Help with invalid units and ignoring records

[poblabs]

Hi there, I have 2 issues, I think maybe 3?

1. My anemometer is publishing in meters/sec, and I have the below config. However, it looks like weewx isn't saving it into mph units in the database. Any thoughts?

2. Also, when I try to set units, I get an error
   CRITICAL __main__: **** ValueError: For flagpoleWindSpeed invalid units, km_per_hour.

3. I see a lot of these errors too
   INFO user.MQTTSubscribe: (Service) TopicManager ignoring record outside of interval 1723218224.000000 1723218226.000000 1723218223.765086 'dateTime': '1723218223.7650864', 'flagpoleBatteryPercent': '90.0', 'flagpoleWindDir': '97', 'flagpoleWindGust': '2.6', 'flagpoleWindSpeed': '1.8', 'usUnits': '17'

---

In my weewx.conf:

[Accumulator]
    [[flagpoleWindGust]]
        extractor = max

[MQTTSubscribeService]
    host = localhost
    payload_type = json
    enable = true
    binding = loop
    max_loop_interval = 20
    log = true
    
    # Controls if validation errors raise an exception (stopping WeeWX from starting) or only logged.
    # Default is false
    stop_on_validation_errors = false
    [[topics]]
        unit_system = METRICWX
        [[[message]]]
            type = json
        [[[weather/rtl_433/flagpole]]]
            [[[[wind_avg_m_s]]]]
                name = flagpoleWindSpeed
                ignore = False
                contains_total = False
                conversion_type = float
                units = km_per_hour
            [[[[wind_max_m_s]]]]
                name = flagpoleWindGust
                ignore = False
                contains_total = False
                conversion_type = float
                units = km_per_hour
            [[[[wind_dir_deg]]]]
                name = flagpoleWindDir
                ignore = False
                contains_total = False
                conversion_type = int
            [[[[battery_raw]]]]
                name = flagpoleBatteryPercent
                ignore = False
                contains_total = False
                conversion_type = float

[bellrichm]

Let’s get the units working first…. The error means either the function to convert from km_per_hour to other units cannot be found or the WeeWX field flagpoleWindSpeed’s unit_group cannot be found.
What unit_group did you add flagpoleWindSpeed to? I’ll do the same and see if I can recreate/get more info.

[poblabs]

That could be my problem? I don't have any extra custom config or extension. Just performed the database action weectl database add-column flagpoleWindSpeed, and configured the mqtt subscribe service. I poked around the wiki and didn't see anything obvious in my face for it.

[bellrichm]

Check out, http://www.weewx.com/docs/5.1/custom/units/#assigning-a-unit-group
You’d want to add something like

import weewx.units
weewx.units.obs_group_dict['flagpoleWindSpeed'] = 'group_speed'

To extensions.py

[poblabs]

Awesome, thanks for the reminder. Been out of touch with weewx last few years.

I've added that and no errors. I'll keep an eye on it and the obs saved to db to make sure its in the right units.

Any insight on the ignoring record outside of interval item?

[bellrichm]

The ignoring records is due to some unfortunate (aka dumb) early design decisions. Read this, https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Date-Time-processing for some information/ideas.
I think I would just use the ignore_start_time = True option.

[poblabs]

Great thanks!

Lastly, it looks like the unit conversion isn't happening. My database is setup for us units, so I'd assume the mqtt data from m/s would be converted to mph before being added to the database, right?

[bellrichm]

Set debug = 1, restart WeeWX, run through one archive interval.
Attach, the log and I’ll take a look.

[poblabs]

Here you go!
debug.log

[poblabs]

I do have the liberty to convert these to mph before sending to mqtt. Maybe I have mixed units going on?

[bellrichm]

I only looked at one loop packet, and for that one it looked correct.

Here is the incoming data.

Aug  9 14:06:43 nebula weewxd[19176]: DEBUG user.MQTTSubscribe: (Service) MessageCallbackProvider data-> incoming topic: weather/rtl_433/flagpole, QOS: 0, retain: False, payload: b'{"battery_raw": 91, "wind_avg_m_s": 2.8, "wind_max_m_s": 3.6, "wind_dir_deg": 86}'

You have the topic unit_system = METRICWX and the units of wind_avg_m_s set to km_per_hour. So that is converted from 2.8 to 0.7777777783999998. (2.8/3.6)

Aug  9 14:06:43 nebula weewxd[19176]: DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming weather/rtl_433/flagpole: 'flagpoleBatteryPercent': '91.0', 'flagpoleWindDir': '86', 'flagpoleWindGust': '1.0000000007999998', 'flagpoleWindSpeed': '0.7777777783999998'
Aug  9 14:06:44 nebula weewxd[19176]: DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing weather/rtl_433/flagpole: 'dateTime': '1723226803.5857368', 'flagpoleBatteryPercent': '91.0', 'flagpoleWindDir': '86', 'flagpoleWindGust': '1.0000000007999998', 'flagpoleWindSpeed': '0.7777777783999998', 'usUnits': '17'

Since your WeeW unit_system is US, it then converted from 0.7777777783999998 to 1.7398436640113337 (0.7777777783999998*2.237)

Aug  9 14:06:44 nebula weewxd[19176]: DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing accumulated weather/rtl_433/flagpole: 'dateTime': '1723226804.0', 'flagpoleBatteryPercent': '91.0', 'flagpoleWindDir': '86.0', 'flagpoleWindGust': '2.2369418537288577', 'flagpoleWindSpeed': '1.7398436640113337', 'usUnits': '1'
Aug  9 14:06:44 nebula weewxd[19176]: DEBUG user.MQTTSubscribe: (Service) data-> final packet is 2024-08-09 14:06:44 EDT (1723226804): 'barometer': '29.924', 'consBatteryVoltage': '4.74', 'dateTime': '1723226804.0', 'dayET': '0.069', 'dayRain': '0.03', 'extraAlarm1': '0', 'extraAlarm2': '0', 'extraAlarm3': '0', 'extraAlarm4': '0', 'extraAlarm5': '0', 'extraAlarm6': '0', 'extraAlarm7': '0', 'extraAlarm8': '0', 'flagpoleBatteryPercent': '91.0', 'flagpoleWindDir': '86.0', 'flagpoleWindGust': '2.2369418537288577', 'flagpoleWindSpeed': '1.7398436640113337', 'forecastIcon': '7', 'forecastRule': '94', 'inHumidity': '45.0', 'insideAlarm': '0', 'inTemp': '78.9', 'monthET': '0.93', 'monthRain': '1.59', 'outHumidity': '37.0', 'outsideAlarm1': '0', 'outsideAlarm2': '0', 'outTemp': '81.6', 'radiation': '186.0', 'rain': '0.0', 'rainAlarm': '0', 'rainRate': '0.0', 'soilLeafAlarm1': '0', 'soilLeafAlarm2': '0', 'soilLeafAlarm3': '0', 'soilLeafAlarm4': '0', 'stormRain': '0.29', 'stormStart': '1723089600', 'sunrise': '1723197060', 'sunset': '1723247880', 'txBatteryStatus': '0', 'usUnits': '1', 'UV': '2.0', 'windDir': '193.0', 'windGust': '0.0', 'windGustDir': 'None', 'windSpeed': '0.0', 'windSpeed10': '2.0', 'yearET': '22.63', 'yearRain': '28.7'

What ended up in the database? For the accumulator what extractor are you using?
I can loop at more loop packets, but if one is created correctly I’d be surprised if others were not being converted…

[poblabs]

To me it looks like it's saving in the database as the m_s reading. I'm only using an extractor for wind gust and setting that to max. I'm not using one for the wind speed. Do I need to specify avg for that? I had assumed it would do that automatically in the loop?

[poblabs]

We had a power outage, so I'll do another loop to see console data vs. loop/archive data

[poblabs]

On the console I saw a 19mph gust, but the database shows it as 5.4, which if I convert 5.4 ms to mph is only 12mph, so I'm missing something

[poblabs]

Think I got it. I removed the units = km_per_hour and now the archive is matching my console. I think I was confusing myself with all the conversions. Yikes 😄

Having the extensions.py define the group for each observation was probably my missing link.

Thanks again for the help!

[bellrichm]

Oh, good to hear. Thanks for letting me know.