Bit of a break

I took a bit of a break from my main programming projects and helped make an installer for another open-source project Darkstar. I started with a deb installer but realized a script would do a better job, especially for customizations. So that took about 3 days to get working well. Doing it even allowed me to fork the project on GitHub & create my first Pull request for someone else. So that was nice to be able to contribute. The reason I choose this project was that I was requested to set up a server for my brother, so he could more easily work on his FFXI graphics improvements. Basically, he has been making his own custom textures for the game that looks really beautiful! Unfortunately, the install instructions at that time were out of date and consist of a bunch of terminal commands and a few scripts. Without the new installer script, it took a while to get it to work at all because it was missing dependencies and was written for a much older Ubuntu install 14.04. At the time I could only get it working on an Intel system with Ubuntu. Now with the interactive installer script I made, it can be installed on the latest versions of Debian, Ubuntu and Raspbian. Being able to take advantage of the cheap Raspberry Pis is definitely a bonus. With a single terminal command, one can get set up with a Darkstar server and only need to enter a user, password and IP address. So I’m pretty happy about that.

In other news, it has been snowing A LOT here. I even had to climb my roof and remove the snow there so it wouldn’t possibly collapse. The snow on my shed roof (which I don’t bother to clear), is about 3 or 4 feet tall, and the roads are basically bumpily packed ice sheets. There’s also a huge fancy snow hill at our local no-frills parking lot that’s probably 12 or more feet tall and covers a very large amount of the parking lot… maybe … 2500 Sq ft. Anywho, it’s been a very snowy winter this month so far and although I’m not super impressed, my daughter is very much enjoying it.

I have only poked at my Sensor Project a little bit, but I was able to fix a bug that kept freezing my program after a few weeks. One of the sensors required an open and close when accessing it (it’s own custom version of it). But I didn’t close it when done with reading, so it kept pilling up open i2c connections until I couldn’t open any files at all in the program. I’m still testing to make sure it did resolve it, as it takes a few weeks for it to freeze up, but I’m fairly sure that was the issue, as it didn’t occur with any of my other sensors that were missing this one particular sensor.

Now that the installer is made for Darkstar, I’m thinking of adding additional support for sensors in my Sensor Project, but since I don’t physically have the new sensors, I won’t be able to test them, but I can at least get the backend set up and ready for testing. I’ll probably put these sensors under a “Un-Tested” section as a warning they may not function correctly.

That’s it for now. Slow progress for now due to the winter blues. Here’s hoping the sun decides to grace me with its un-clouded presence soon!

New standard Debian Installer!

I have been wanting to create a more standard installer for a while now, and I have finally gotten to it!
I decided to go with a Debian installer, as it was the simplest to figure out (after hunting down the right documentation), not to mention it’s one of the oldest ones. Originally I wanted to use a container installer like snap, AppImage, flatpack or Docker but .deb just seemed to make more sense for now…

So now Installing and Uninstalling has never been easier! Download and double click the file to install! This should make it easier for people used to Windows or Mac computers (but not compatible with Windows or Mac…). Once installed it should list kootnet-sensors in your Debian software manager. You can also remove with apt-get remove kootnet-sensors.

I have managed to get the old installer re-written in a way that will allow upgrades from the pre-deb installer to the new deb installer. It requires you to upgrade twice though, once to get the transition installer and another time to initiate the transition installer. Basically just upgrade from the web interface, wait 30 seconds, then upgrade again.

I have already done a fair bit of testing and optimizing, such as figuring out which apt-get dependencies are required and which are not. I put required to run at all stuff as required in the deb installer and sensor hardware requirements under recommends. This way you can use the management pages on Ubuntu for example, without requiring the hardware dependencies that may not be available on other Operating Systems.

After a bit more bug checking, optimizations, testing and documentation, I will release the new version and update the online documentation to match.

Sensor Control Enhanced!

I have finally finished up the Sensor Control section of the Web Portal. All configurations can now be edited and pushed to multiple sensors using Sensor Control. Once I do some testing, I’ll release this into the stable channel. Besides finishing up on the Configuration section of Sensor Control, I also added drop-down buttons for upgrading, power commands and misc other.

I would like to add the ability to review and add notes next, but I might space it out with a bit of reading. I got a new book “Modern Python Standard Library Cookbook”. I want to be able to do as much coding using the Standard Python Library as possible, so I can limit the outside requirements. This should also make my code cleaner and more efficient.

A new release is coming soon. As always, feel free to check out what is coming by upgrading to the development channel, but be warned, the development channel can be unstable.

Sensor Latency Tests Added

I finally got around to adding Sensor Latency testing. A button has been added under the “System Management” page to run it. The latency tests tell you just how long it really takes for a sensor to return a reading. This is required knowledge to set trigger variances in the seconds or less, so it doesn’t end up building a queue of reading requests waiting to get data. I have already set each sensor’s “multiple access wait times” to match more closely with how long it takes to get a sensor reading. This should improve performance when sensors are accessed at the same time.

Most sensors return data within 100ms (0.1 seconds). Some like the EnviroPhat can get a reading from each sensor within 10ms. But there’s one sensor, the light spectrum AS7262 that can take anywhere from 700ms to 8 seconds! I’m not sure exactly why, because my other spectrum sensors, like the BH1745 and the EnviroPhat, can grab readings in less than 20ms every time. I also checked the source code for the driver and how I set-up the API, and it’s set for under 30ms to get a reading for each of the six colours, so added up, it shouldn’t be much more than 200ms, and that’s assuming it can’t do them in parallel.

I have been putting off doing the configuration section under Sensor Control because I know I can actually get it to re-use my existing individual configuration pages without copying them … but it would take some creative re-working, and I have been lacking in creative motivations. Soooo… I think I’ll just copy-paste to get it working and refactor it later.

Kootnet Sensors Runs on Ubuntu

W00t, I made it so Kootnet Sensors can be run on Ubuntu (and probably other GNU/Linux systems). It just disables the hardware access functions but allows the web interface with access to “Sensor Control” to manage other sensors. Most changes were made to where the configurations are saved.

So this is awesome for testing! I can now just get it running from my Development Environment and test most HTML functions. It also does not require root access if there are no sensors.

Besides that, I also moved the sensor in use checks to the driver interface so nothing could bypass the check. The “Online Services” section of Sensor Control now works as well, except for setting the interval.

Things are coming along nicely.

Kootnet Sensors – Hardware Drivers Included

I have put the hardware drivers directly into Kootnet Sensors, instead of installing them with pip. This should ensure all the sensors work, regardless of the upstream project changes or updates I make to utilize new driver features. It took a bit of poking for the EnviroPlus sensor, but it and all other sensors I could test appear to be working well across the Pi Zero W, Pi 3b+ and the Pi4. I’m still wondering about including some of the other dependencies …

A few “bugs” have been fixed, such as making sure only one call to access a sensor is done at a time. Mostly around sensors that require a “keep-alive” thread, which bypassed the other in use checks. This should clear up errors around gas and the EnviroPlus’s particulate matter sensor.
I also made it so sensors that update multiple readings, even with single reading requests use a cache, so if the program is getting readings for all sensors (but through individual sensor reading requests), it can take from the cache for all requests made after the initial update within 1/2 a second.

The next thing I should really finish up is the Sensor Control section of the web portal. Once Configuration and Online Service sections have been done, it can pretty much do everything the PC Control Center app can do.

That’s it for now. Slow going for the short winter days and the long winter nights.

Kootnet Sensors Update

Now that Kootnet Network Testers are working well, I thought it was time to do a bit more on my Sensor Project.

I have started to integrate a version of each hardware sensor driver (the pip modules) into my program directly. The major reason for this, is I have already had 2 sensors change their functions in a way I had to fix my program to use it. This creates real problems because previous sensors don’t auto-update their drivers and conflicts start to arrive where it’s working on one but not another. I’m hoping I won’t have issues between the Pi Zero and the Pi3/4 due to the arm hardware differences. I’ll be sure to test it on all models before release, but so far the only one I am having any real issues with is the Enviro+. All the others have been entirely drop-in replacements. I think I have 4 sensors left to add drivers for, plus the Enviro+ I have to fix.
I’m happy to say that all the drivers so far have added up to about 500 Kbytes. So my installer is currently sitting at 1.5 Mbytes.

This change will also make the install much faster, as pip is not exactly a speed demon for downloading and install modules. I’m also considering adding other dependencies to the installer to make sure it will work, regardless of what happens with the dependencies own projects and even pip itself. But I’m also concerned about security with modules like Flask and the gevent since they are the internet-facing modules that you want to be patched and up to date. For now, I’ll most likely keep Flask and gevent as a pip install at the time of install, however that may change in the future… in fact, I suppose I could include the modules but then add a script to download the new version and copy it over the old one and restart the program… yeah that would probably work. I could even add an auto-update option for those modules at set intervals but disable it by default in case of issues introduced… actually… I can even have it update the module at the time of install.

Anywho, that’s what I’m working on now and once done, should improve consistent stability.

Network Testers Update

I was thinking about documentation when I realized I could integrate the instructions for using it, into the display itself. So to that end, I re-worked the hardware button operations.
With the initial release, each of the 4 hardware buttons was tied to a single command, with the exception of the 3rd button, which if pressed a second time, would initiate an update.
Now I have set the 4th button as a “Change Functions” button, that upon pressing will change the operations of the other 3 buttons, and at the same time, display a message on the screen for what each button does at the current function “level”. The change button currently has 3 levels it cycles through. The new button functions are as follows.
Primary Functions

  1. Run MTR tests
  2. Run iPerf 3 tests
  3. Nothing
  4. Change Button Functions

Secondary Functions

  1. System Information
  2. Standard Upgrade
  3. Development Upgrade
  4. Change Button Functions

Tertiary Functions

  1. Shutdown Remote Test Server
  2. Shutdown Local Unit
  3. Nothing
  4. Change Button Functions

With this change, I can tie more functions to the compatible Raspberry Pi hardware buttons. This should make it a much more useful mobile tool, especially when I start to add other types of tests it can run.

There are a few extra features, such as each instructional page viewed on the WaveShare display, now shows if the remote tester is Online or Offline. This same Online/Offline status is shown on the web portal as well.

The System Information page now shows the following for the ethernet and wifi adapter; Active/Inactive Status, DHCP or Static address setup and the current IP Address.

The about page now also displays the Primary Internet IP address along with the amount of free disk space in GB’s.

I also found and fixed a bug in the WaveShare display. If a button was pressed multiple times before the display finished its refresh, it would crash the display server. I fixed it by adding an “In Use” variable, and if “In Use”, any new commands will be dropped until no longer “In Use”.

I have already put the new update on the standard release channel.

Kootnet Network Testers 1.x.x Series Released

So the new version is out, tested and ready to be used! I should probably work on documentation next but we’ll see what happens.

The new version is now in the Standard release channel. Check it out and start your testing!

Kootnet Testers

I have fixed bugs, added validation checks and tested the testers. I think it’s ready to be released! However, it is 3 AM and I’m fairly tired, so I should probably look it over with rested eyes before actually releasing it.

That being said, it looks to be in good shape. There are proper error messages when the tests fail, the results should always look right on the E-Ink display due to splitting the output into a list instead of picking the exact character locations.

So … I think that’s that, the new release should be out tomorrow sometime.