Category Archives: Random

Kootnet Ethernet Testers Update

I have spent some decent time on the Ethernet Testers and have laid most of the foundation to get things working, including the base for supporting different (or no) displays.

The next step is setting up the configuration & testing pages in the flask app. Once that is in place, I should be able to start testing it with the original display and without any displays. If all goes well, I can start adding other display support and different configuration options.

On a side note, it was cool to get my first fork on GitHub, which was of my Ethernet testers. I should get a bit more attention to the project once I finish the updates, especially with the Raspberry Pi 4 having a 1GB ethernet port.

I expect to have a release out of the new version sometime this month.

Kootnet Ethernet Tester Enhancements

So I have thought a bit about how I want the Ethernet Tester program to work and here are a few insights into what’s to come.

For starters, I’m breaking up the program into different server “Threads”. This will allow for more modular operations. Said threads would include Display, Interactions, HTTP config + access and iperf3.
Secondly, I’m creating an HTTP server for configuration and remote results access. This combined with the ability to configure the wireless through HTTP will allow the use of devices without an actual hardware display!

The HTTP configuration pages will eventually have options to do things like customize iperf3 settings, configure unit between display & secondary server, swap between supported displays, etc.

Right now I’m working on Refactoring and creating the new foundation to support these new features. This program is MUCH smaller than my Sensor program, so I don’t expect it to take too long. That being said, I will be playing around with things to see both my options and what works best. For instance, I’m going to be including all the required python files for the supported displays so you don’t need to use pip to get them (and should remain compatible regardless of how they change things upstream).

Once I get things working, I’ll post on the Raspberry Forums.

Change of Pace

I have done a lot on Kootnet Sensors, so I thought I would shift gears a bit and look over and enhance Kootnet Ethernet Testers. I have learned a lot since I made that one, so I think it’s time to fix it up and tweak it out. I have a few ideas already but I’ll keep them to myself until I think them through.

At some point, I’ll also finish up Sensor Control on Kootnet Sensors, then release another update.

Look for a new Kootnet Ethernet Tester release soon!

Re-working Triggers

So far version Alpha.28 is coming along nicely. I have finished up on showing the running status of each Variance Trigger Thread and am now refining the Variance Triggers, as well as creating a new type of trigger-based recording.

The new type of trigger-based recording is done with a high value and a low value. Once either is passed or comes back from said value, the sensor data is recorded for that period.
For example. let’s say we monitor the temperature with a high of 30c and a low of -10c. If the temperature goes above 30 or below -10, the readings are recorded. If the value returns to normal later on, those readings are recorded. It records CHANGES of state only, reducing the required writes.

I have had issues with the HTTPS server freezing when going to the “Sensor Readings Page” when Trigger Variances are enabled. I’m thinking the constant reads are crashing a sensor or two. So with that in mind, I was thinking of creating an autotest for figuring out how long it takes to get sensor data from a sensor. Once the program has that number, it can use it to help set a minimum value for sensor retrieval intervals. I’m hoping that will help prevent overtaxing sensors and keep some crashes and freezes at bay. I’ll also create a report of direct sensor access times to help set proper times and see if a sensor is taking longer then it should.

In other news, I have been playing around with solar panels and batteries. I have 2x 100-watt panels + 1x 50-watt panel. I have tested the 100 watts panels in different environments with a single Pi Zero W to see how they might operate under poor conditions. First, I tried putting one in my window with very little direct sunlight, and it managed to keep the battery’s charge level during sunlight, which means it probably had enough power to keep my Pi going, but that was about it.
Next, I put one 100 watt panel in my back yard behind my shed, and it did fairly well there when the sun was out but as soon as clouds came and or shad from the trees fell, it didn’t do so well.
So it seems solar really does need direct sunlight with no obstacles to function well. That being said, I still have to test it with my lead-acid battery (used Lithium-Ion for tests), as it may be able to charge better with less sun. I guess the next 2 tests will be putting one in direct sunlight and my lead-acid behind the shed to see how well it does.

Updated Sensor Control to allow DNS + Port

Sensor Control in the web interface now allows the use of DNS addresses and port #. AKA instead of 192.168.10.11, you may have a DNS URL setup from the outside like sensor.mycooldomain.com and a port # of 10022 through your router. Now you can enter sensor.mycooldomain.com:10022 instead of just the IP and it will work correctly, including links in reports and downloads.

The “Creating Graph” works the same as creating downloads in Sensor Control since I like the Sensor Control method of flashing a message and disabling the create button better.

I have also added the framework to put in “Tool Tips” throughout the web interface to help explain things on the fly. I only put a few in the Graphing section to start but will add more fairly soon.

Let’s see what else… I created a class to start a Thread and monitoring it so I can show the running status of different parts of the program. I just need to finish up on the Trigger monitoring, since it actually starts a thread for each sensor type. It’s already working, it’s just not showing on the information page right.

I’m making decent progress, so more to come soon.

Released Kootnet Sensors Ver. Alpha.27.336

New Release!

I have now managed to go over most of the program for testing and things seem stable. So 27.336 was the magic number. There are a lot of developments since version 26.x, but I won’t go into them tonight since most of the changes are in previous posts and definitely in the changelogs.

This release is the first to have a section dedicated to replacing the management features of the Control Center. It already does most things better than the Control Center program, especially around downloading sensor data from multiple sensors. I’ll be making big changes to the Control Center and re-branding it. It will focus on Data manipulation instead of Sensor Management. The HTTPS web portal will take up the management of single and multiple sensors. This release also adds support for both Luftdaten and Open Sense Map weather monitoring services.

I have started to customize some of the HTML components to make them a bit more modern. The checkboxes and radio boxes are slightly fancier now. A small gesture of things to come.

I feel like I should keep an SD card of each version pre-installed to see how things change over time. Thanks to GitHub keeping all versions in one form or another, it shouldn’t be too hard.

That’s it for now.

Testing, Fixes & Cleanups

I just spent the last 3 days or so testing, fixing and refactoring. The changes are definitely in the right direction but there’s a lot more to do. I’m not sure how much more refactoring I want to do before releasing it into the Stable channel since each refactor brings the possibility of new bugs, and new bugs take time to find and fix… which again reminds me I need to work on more Unit Tests. I was working on tests before, but the ‘requests’ module wouldn’t work on the local sensor for testing. I think I found a few workarounds that I’ll test later, including a httpx module that says it does work locally with flask apps and I was also thinking I could run the test through threading or a multiprocess so both tasks can run in parallel.

In other news, I created a kind of “lock” to ensure sensor data is not being retrieved from the same sensor at the same time. This should prevent any sensor retrieval issues when multiple threads access the sensors. Basically it sets a variable as True or False. If said variable indicates the sensor is in use, the program simply sleeps for 100ms and tries again. If it’s still not available, it waits some more. This will also lead to a refactoring session because I previously loaded the sensor_access module at the start and passed it as a variable to other functions. This was a failed attempt to ensure single sensor access because it was passed to other functions in threads so multiple sensor access could still occur. So now I’ll most likely remove all the sensor_access passes into module imports.

Besides all the fixes and cleanups, I also added a few minor improvements like more colour coding in the reports, adding sensor links to IPs in Sensor Control Online Checks, logging improvements and a bunch more I can’t remember at this moment.

That’s it for now.

Yay Refactoring!

After finishing up on the Sensor Control section, I committed the code and started refactoring for clarity and simplicity.

To start, I re-did Routes, Render templates and post checks from classes into flask blueprints. This allows me to break up all the flask routes into multiple py files. It took all day, but it “feels” (or should I say “smells”) better. This clean up should make it easier to do future modifications and additions. I still have to review and refactor the Sensor Control code but that will be nothing compared to moving around all the flask routes and co-functions!

There are only a few minor tweaks I want to do, then it’s testing time… which reminds me, I need to work on unit testing… Although I might start looking around for other test suites that are a bit more friendly or advanced then Python unit tests.

Things are coming along nicely. I expect to release a new version to the stable channel within the next week.

That’s it for today.

Sensor Control ‘n’ More

It’s getting time to release another version into the “Stable” Alpha channel.

I have been working on Sensor Control for the most part and have added the following features.

  • Online Status – Now Creates a new page with Sensor IP, Response Time and Sensor Name.
  • The Reports for System, Configuration and Test are pretty much the same with a few minor tweaks like colour coding response time and adding total RAM with % RAM used.
  • Added “Relay and Direct” download options.
    • With Relay, all files are collected and zipped on the local sensor
    • Download buttons are enabled upon zip creation completion
  • Download Zip of all 3 Reports – Creates a zip with all 3 reports.
  • Sensor Databases – Creates a zip of all online sensor databases.
  • Logs – Creates a zip of all online sensor logs
  • The Big Zip – Creates a zip of all online sensors Databases + logs and add all 3 reports

This section of Sensor Control is now pretty much done. I did some sloppy coding to get it working, so I’ll have to review and refactor it. I have tested using it most ways, and it seems stable, but I’m sure I’ll find an extra bug or two. I’m also not sure about the RAM limitation of the Pi’s, because even when using zipped files, the Raspberry Pi doesn’t have much RAM. I created a function to save to disk instead of memory should the database downloads be larger than available memory, but I’m pretty sure I’ll have to do a bit more to get it to writes from the HTTPS download right to disk, instead of putting all the databases into memory and then writing to disk.

Let us see… what else changed… O I’m zipping Database delivery over HTTPS by default, as it can reduce the download size of the Database A LOT! There’s more, but I’ll leave that to the changelog on GitHub.

New Release soon!

Sensor Control – Reports

I have Sensor Control Reports complete. I even upgraded the configuration report to colour code on/off items and include more settings. I still don’t really like the layout of the IP’s and functions… but I’ll worry more about that later, and just concentrate on the functionality. I have the HTML layout complete for configuring online services in Sensor Control as well.

Sensor Control is coming together fairly well. It shall be a worthy replacement for Sensor Control Center. I might end up stripping down what Sensor Control Center does, so it concentrates on data manipulation, instead of sensor manipulation. Since data manipulation is where you want real computing power, that’s the direction Sensor Control Center should go… which means it probably needs a name change… hmm something like… Kootnet Data Manipulation? or … Kootnet Sensor Companion? Probably more along the lines of Kootnet Sensors Analytics. Yeah, I think I like the last one.

This means I have a fair amount of work ahead of me. I need to finish off the new Sensor Control, then re-model Sensor Control Center into Kootnet Sensors Analytics. This means I also have to add advanced analytical abilities to Kootnet Sensors Analytics, AKA, put in some AI learning to analyze the data and find neat patterns we would miss ourselves. O I also need to work on tests.

O I added the ability to download the Sensor Database or Logs from whatever sensors you have listed in Sensor Control. one click to Download all sensor logs and Databases to review at a later time. I have also been thinking about adding a “Download the ‘Full Package'” option that would combine reports, databases, and logs of all listed sensors into a single zip file. This would make it especially easy to “grab ‘n’ go”.