Currently working on

So I decided it would be good to track a bit of the Sensor usage to help with troubleshooting as well as give the ability to monitor sensors uptime and such. To that end, I have created a “Call Home” type function. I know privacy is often a concern, so I came up with the following ideas (so far) to help track sensors and their issues, without compromising privacy.

To start, it sends the following information “Home” on boot and every 24 hours after that by default.

  • A randomly generated (at first run) “Sensor ID”. This is a 32 alphanumeric string. This is used to ensure each checking in sensor is unique. It can also be used to help individuals if they run into issues, as they only need to give me that ID and I can find log entries to determine and resolve issues.
  • Software Version. The version will help me identify where issues are and when issues are fixed.
  • The last 25 lines of the Primary and Hardware Sensor logs. I chose not to include the Network log, because of all the IP’s it logs, which might be a privacy concern. It also only sends logs on A) start-up and B) when the logs have changed since the last sending. This will help keep bandwidth usage to a minimum.

What I want to do is expand this to be able to change where it “checks into” so it can be used to track sensors for the person using it if desired (AKA get check-in’s yourself, instead of sending them to me). I’m making both the check-in function and check-in tracking integrated into the Sensor Software, so one can turn any sensor unit into the tracker monitor. I also want to add the ability to disable it completely for those who really don’t want any data being sent back.

In other news, I have finally set up a Configuration transition to smoothly keep settings between upgrades. It should work well for upgrades between Stable releases, but might not be perfect if you are constantly upgrading to the latest development builds.

Besides working on my sensor tracking section, I also want to add 2x new sensor “types”. First being Particulate Matter 4 (one sensor supports that level) and Noise (on the Enviro+). Once those are in and more tests have been run, I will release a new version into the stable channel.

Updates

Here is a quick recap of some things done with my programming that I haven’t posted about. I should also be releasing a new version into the stable channel in the next few days or so.

Here’s a list of some of the major changes to Kootnet Sensors in no particular order.

  • Added MQTT Support as a Broker, Publisher and Subscriber
  • New Sensor Support
  • Tweaked for faster performance
  • Added Temperature Compensation based on Raspberry Pi CPU temp + multiplication factor
  • Dummy sensor added for Testing and Demo purposes
  • Added Display Configuration to customize mini LED display usage
  • Added the ability to restart specific parts of the program without having to restart the whole application. Only changing the Main configuration, Installed sensors or Trigger variances will cause a restart of the whole app. Soon only saving Installed Sensors will cause a restart
  • A ton of behind the scene changes
  • Misc bug fixes and visual tweaks

Right now I’m just smoothing over the rough spots and fixing any bugs I find. I also want to add a FAQ’s section as well as make a decent manual.

Still Here

So I haven’t updated for more then a month, mostly due to the kid being home due to school’s closing for COVID-19. So I just havn’t set aside the time to post here, however, I have still been coding, just to a lesser degree.

I’ll see about posting some updates soon, but until then, here are a few highlights.

  • Released a new version of Kootnet Network Testers
    • Some decent additions I’ll go into later
  • Bunch o additions to Kootnet Sensors
    • Such as MQTT Support and enhanced Display support

Kootnet Network Testers

With all the updates on Kootnet Sensors, I thought it was about time to do some more work on the Network Testers. So far I have added the following.

  • View and download previous test results
  • Setup repeating tests every X amount of time
  • Set custom run options for iPerf3

I’m also working on adding a one time run of tests at a specific date and time but it’s not finished yet.

Besides features, I also fixed a bug with the new deb update system. Unfortunately the old (current release) runs the upgrade inside the program itself, so when it goes to restart the service, it kills the install and puts the install in a non-operational state 🙁 So it needs to be updated from the command line or downloaded and installed manually. Using the built-in button on the web interface will break the install and prevent it from starting up again. I’ll have to think of a way around that… or just post a warning.

Other features in the works include sending test results to email or FTP and adding aditional network tests.

I’m hoping to release these changes in the next week or so.

Kootnet Sensors Beta.29.110 released

A new version of Kootnet Sensors is out! I have added a fair amount of new functionality and tested it decently. A short-list of features added to the Web Portal includes.

  • Add notes
  • Remote management through Sensor Control is about 98% Complete
  • Database download (Raw or Zip)
  • Database upload
  • Set the port used for the HTTPS server
  • New Sensors added
  • Generate new SSL certificate or upload your own
  • Change Web Portal username & password
  • Upgrade python modules

I have also taken the time to change shell scripts into python functions where applicable. This includes the shell script created to edit configurations from the terminal. This script is now a Python script that works and looks a fair bit nicer. Among other things, it lets you edit configurations, change the Web Portal login, upgrade the program and change its autorun state.

There were some bug fixes and a lot of behind the scenes updates (Most mentioned in previous posts). A new GUI program was also created to test a local or remote sensor.

This kinda feels like my first decently stable release. I want to improve my interfaces (Web Portal) but we’ll see how my motivation goes as I have to do research on things like bootstrap, CSS and JavaScript.

Going through the ToDo list!

Now that I have had a bit of time off from programming, I managed to get on a roll for pumping through a few program changes I have been wanting to work on.

  • Cleaned up and enhanced
    • Plotly graphing code
    • Test Code
  • Added username + password in GUI Tests
    • Only runs tests if both Sensor Online & has the correct login
  • Available Configuration settings are loaded up to what is provided (By file or text content), missing items (for new settings added here on) are given default values and saved
  • Added ability to change Web Login credentials from the Web Portal
  • Added ability to change SSL certificate files from the Web Portal
    • Create new Self-Signed SSL
    • Upload custom SSL
  • Added ability to Upgrade Python3 Modules from the Web Portal
  • Removed Re-Install requirements button (Does it on every upgrade)
  • Disabled Clean SMB Upgrades (Needs to be set up, breaks install if not)
  • Multiple bugfixes

I still have a bit of work to do on the configuration report and the online service configurations logging. Then it’s just general review before releasing a new version. It should be ready within the next few days.

Bit of a break + updates

So I took a few weeks off to do other things like setting up a bunch of remote access for clients at work, due to the coronavirus and lock down some Exchange servers to limit spam.

I spent a bit of today cleaning up KootnetSensors code in preparation for a new release. I also added the ability to upload a database to a sensor, replacing the previous one. It checks to make sure it’s a valid database before replacing it, and it creates a zipped backup of the old database in the main data directory. This should come in handy for a few reasons. First, it allows you to upload a database to create graphs on a more powerful computer. Second, it allows easy sensor replacements when needed as you can upload previous databases from the web interface.

Later on, I want to be able to swap between different uploaded databases in graphing, so you can work with multiple databases on a single unit. I may also make it so you can select remote sensors to graph locally but that won’t be done anytime too soon… or will it?

I hope everyone is safe during this crazy Coronavirus time. I’m happy I live in a more remote area, where I can go outside for hikes.

Until next time!

Multiple Updates

I have managed to go through a bit of my Todo list. The following list of things has been completed!

  • Sensor Latency integration with Sensor Control
  • Generic Configuration Class use among all Configurations
  • Unit tests created for a good portion of the Flask ‘Sensor Get’ routes and all Configurations.

With the new generic configuration class in place, working with any of the 7 configurations is a lot easier, thanks to shared access functions. Saving, loading and updating configurations are all done the same way now.

With the new unit test’s I can test the program in my IDE or use a real-live sensor. I also made a bit of a front end for running the tests, so it can be accessed and used through the OS system menu easily.

I’ll probably release another version soon with the fair amount of changes since the previous release (Notes, Latency reports, tests, unified configurations, bugfixes, etc.).

Web Portal Notes!

I have now added the ability to view, create, modify and delete notes from the web portal. I’m using an HTML editor called “TinyMCE”, which works well and integrated fairly easily. It can’t embed pictures or media but It’s nice to have basic formatting, lists printing, etc. Because it saves as HTML code (which is good for compatibility), Kootnet Control Center’s note section does not support it, so viewing in Control Center will show HTML code instead of an HTML page. I’m not completely done the Notes section yet, as I still have to add the custom Date & Time stamp the user can edit to show wherein the database the note is referring to and I wanted to get the spell checker working.

On a personal note, the wife has left the province for work and has stints of 3 to 5 weeks at a time away from home before coming back for 1 or 2 weeks. So I have a little less time for programming projects.

The weather has been mostly cloudy grey and the temperature is kinda random from like +7c to -11c with wind, rain, snow, hail, etc. I’m looking forward to warmer and sunnier weather.

Back to my day job for now. I have a few work projects on the go too, like finishing an Exchange migration to Exchange 2019, replace some failed drives in a server and get an IP camera system working from outside the local network but through the VPN for security.

New Sensors Added & Standardized Configurations Created

I added 3 new Pimironi sensors. The MCP9600 (Temperature), MSA301 (Accelerometer) and the SGP30 (Gas Total VOCs). I don’t actually have any, so I can’t test them, but its all set for when I do, or if someone else does.

I have also decided to standardize my configuration files through a base inherited class. With this, the majority of functions used among the configurations are the same. So you load and save all the configurations the same way for instance. It has also cut down on the number of lines by a decent amount. I still have a few configurations to update, but I have already converted the primary configuration and the installed sensors. I’m working on Trigger variance configuration, then it’s just the Online Service Configurations and Sensor Control’s.

That’s it for tonight’s updates.