Author Archives: chad.e

Update

The email system in Kootnet Sensors is pretty much done, I just need to get Plotly graph’s working but everything else is good to go. Quick Graphs and Reports can be sent on a daily, weekly, monthly or yearly basis to one or more emails.

I also spent some time tweaking the performance of the Graph generations. In order to do this, I have put as much of the processing work on the SQL retrieval side, as it uses much faster C or C++ code to do it. For example, when you skip data points, it now skips it in SQLite instead of getting all, then removing data points after. There were some other more minor changes that should help performance as well.

Other updates include changes to the web interface layout (mostly around configurations), there is a single setting for the DateTime offset that is applied throughout the program, better email verifications, continual updates of things like hostname / IP address (every 5 hours I think) and a bunch of other misc fixes and refactoring.

I’m thinking I should now go through and start adding docstrings, clean ups and updating the help document.

As far as future features, I’m thinking of creating a Backup section, to backup not only data, but also configurations. I’m also thinking of re-working the remote configuration push.

That’s it for now. I’m hoping to put more time into programing, especially since its SUPER smokey out now (for the past 3 or so days).

Kootnet Sensors Update

It has been a bit since my last update. Summers has been pretty slow as far as the programming goes, however, I have made progress. Today I have an email notification system working (mostly). Kootnet Sensors can now email a Sensor Report or a Quick HTML graph as a one time go or at a set interval of X hours. I have uploaded the changes to the Developmental channel, so you can update to that and try it out if you wish to get a sneak preview.

Until next time!

High/Low Triggers + More

I have now finished the High/Low trigger recording. It’s pretty basic, if the reading goes above your “High” reading, it records to the DB as a “High” state and stays there until it goes bellow High but above Low or it goes bellow Low. If it’s between High and Low, it records a “Normal” state, and of course if it’s bellow the Low trigger, it records as a “Low” state. It also records the current state when the system starts up. Because it only records changes of state, recordings will be minimal saving space on the disk and system resources.

I have now also re-organized the Installed Sensors and SQL Recording configurations. It should make it easier to find what you’re looking for and make more room for additional sensors and recording types.

Some other misc. changes include adding the ability to change how often the Sensors Checkin and moving the Enable SQL recording into their own setting sections.

I have slowly been adding screenshots and content to the Documentation as well as adding links throughout the program that take you to the exact sections in the documentation. I suspect I’ll be at the documentation for a while before it’s all said and done, but I should probably concentrate on it now.

I suppose that’s it for the update. These changes are now in the Developmental channel and can be tried out anytime.

Changes to the installer

Due to someone having issues with the installer, I have decided to include a base version of all the required Python modules in pip wheel form. From what I could gather from the logs, one or more Python modules were missing or broken. The cause of this was most likely a bad network connection or temporarily broken modules or even just the pip servers being overloaded. Whatever the case, having all the required modules included in the installer should ensure a smoother installation experience. After adding in all the module wheels, and a few duplicates for the different pi platforms (armv6 / armv7), the size of the installer went from 1.2 MB to 45.7 MB. Probably for the best, considering those modules had to be downloaded anyway.

There are a few other nice benifits to including the modules as well, such as.

  • Faster install times, especially on the Pi Zero
  • So long as the apt-get requirements are meet, offline Installs should be possible (I still need to test it)
  • The install won’t break if there are bugs in the most current Python Modules or if the internet connection cuts out after getting the system (apt-get) requirements

There are a few downsides, such as the size I mentioned but also the fact you’ll be starting off with older modules that could include bugs or security holes. I already have the ability to update to the latest modules in the program itself (with the click of a button under Advanced), but I’m wondering if I should do something more…

I have finally found an HTML template for documentation I like, so I have started to fill in the blanks and customize the layout. I’m building the document directly into the program itself, so you can easily access it when trying out the sensor software. Once the document is closer to completion, I will also add it to my website, so people can view it for the installation instructions and troubleshooting.

On another note, I have a request to support another sensor, this makes 2 new sensor requests so far. Thanks to the Python Module I found to add support, I’m also automatically getting support for a bunch of similar sensors which include the DS18S20, DS1822, DS18B20, DS28EA00 & DS1825/MAX31850K. Support has been added but I don’t have the sensor in question to test it. I’m hoping the person will see the post/response and try it out and let me know.

Let’s see what else has been done… O yeah, I added configuration options for both the Sensor Checkin server & the Sensor Checkin itself. For the Checkin, you can enable, disable and change the URL it uses to Checkin. For the Checkin server, you can enable, disable and customize the view options. I have also added more features such as the database clean-up section to clear old data, remove sensors and shrink the database.

Right now I’m filling in the Documentation and adding a High/Low trigger recording section. That will probably take a week or two, then maybe I’ll re-organize the Installed Sensors section to have a tabbed layout per sensor company.

Beta.30.190 Released

It’s out! Fancy MQTT support and all. Time to work on the next set of enhancements.

Soon…

I keep finding minor bugs or adjustments I want to make when I test things… but only like one at a time and for small things, like logging. So that means I should be close to putting it into the stable channel.

I added a integrity check to the databases on load and upgrade (quick on load, full on upgrades). Although I’m pretty sure it doesn’t actually “fix” anything, it will notify of any issues that need closer inspection.

I think I’ll do one more test, and assuming I don’t find anything, I’ll call Beta.30.x done. I’m looking forward to seeing how many sensors are being used out there and getting some logs to help find bugs.

Hopefully a new release by tonight!

Beta.30.x is almost ready

I have been spending the last few days finding bugs and security issues then fixing them. For instance, I should now have properly “sanitized” SQL queries, by using the SQLite3 replacement of variables function. Due to this I no longer require my notes to be sanatized before going into the SQL database. If notes where used prior to version Beta.30.x, they may require some re-formatting.

I have now added particulate matter 4 to the supported sensor types. Only the Sensirion-SPS30 supports it so far but hopefully other sensors will add a wider range of detection that I can support.

This will be the first release that sends back information to help identify issues I might not find during my tests. It sends a randomly generated 32 character string (to identify unique sensors), the version, uptime and the last 40 or so lines of 2 of the 3 program logs (Primary and Sensors logs). I left out the Network log due to possible privacy concerns over IP’s being logged, however, I’ll set up a section that will allow changes to the Check-in URL as well as the ability to select which logs to include, with the default setting excluding the Network logs. Once that’s done, I’ll do a bit more testing and then release it into the stable channel.

Currently working on

So I decided it would be good to track a bit of the Sensor usage to help with troubleshooting as well as give the ability to monitor sensors uptime and such. To that end, I have created a “Call Home” type function. I know privacy is often a concern, so I came up with the following ideas (so far) to help track sensors and their issues, without compromising privacy.

To start, it sends the following information “Home” on boot and every 24 hours after that by default.

  • A randomly generated (at first run) “Sensor ID”. This is a 32 alphanumeric string. This is used to ensure each checking in sensor is unique. It can also be used to help individuals if they run into issues, as they only need to give me that ID and I can find log entries to determine and resolve issues.
  • Software Version. The version will help me identify where issues are and when issues are fixed.
  • The last 25 lines of the Primary and Hardware Sensor logs. I chose not to include the Network log, because of all the IP’s it logs, which might be a privacy concern. It also only sends logs on A) start-up and B) when the logs have changed since the last sending. This will help keep bandwidth usage to a minimum.

What I want to do is expand this to be able to change where it “checks into” so it can be used to track sensors for the person using it if desired (AKA get check-in’s yourself, instead of sending them to me). I’m making both the check-in function and check-in tracking integrated into the Sensor Software, so one can turn any sensor unit into the tracker monitor. I also want to add the ability to disable it completely for those who really don’t want any data being sent back.

In other news, I have finally set up a Configuration transition to smoothly keep settings between upgrades. It should work well for upgrades between Stable releases, but might not be perfect if you are constantly upgrading to the latest development builds.

Besides working on my sensor tracking section, I also want to add 2x new sensor “types”. First being Particulate Matter 4 (one sensor supports that level) and Noise (on the Enviro+). Once those are in and more tests have been run, I will release a new version into the stable channel.

Updates

Here is a quick recap of some things done with my programming that I haven’t posted about. I should also be releasing a new version into the stable channel in the next few days or so.

Here’s a list of some of the major changes to Kootnet Sensors in no particular order.

  • Added MQTT Support as a Broker, Publisher and Subscriber
  • New Sensor Support
  • Tweaked for faster performance
  • Added Temperature Compensation based on Raspberry Pi CPU temp + multiplication factor
  • Dummy sensor added for Testing and Demo purposes
  • Added Display Configuration to customize mini LED display usage
  • Added the ability to restart specific parts of the program without having to restart the whole application. Only changing the Main configuration, Installed sensors or Trigger variances will cause a restart of the whole app. Soon only saving Installed Sensors will cause a restart
  • A ton of behind the scene changes
  • Misc bug fixes and visual tweaks

Right now I’m just smoothing over the rough spots and fixing any bugs I find. I also want to add a FAQ’s section as well as make a decent manual.

Still Here

So I haven’t updated for more then a month, mostly due to the kid being home due to school’s closing for COVID-19. So I just havn’t set aside the time to post here, however, I have still been coding, just to a lesser degree.

I’ll see about posting some updates soon, but until then, here are a few highlights.

  • Released a new version of Kootnet Network Testers
    • Some decent additions I’ll go into later
  • Bunch o additions to Kootnet Sensors
    • Such as MQTT Support and enhanced Display support