Author Archives: chad.e - Page 2

Going through the ToDo list!

Now that I have had a bit of time off from programming, I managed to get on a roll for pumping through a few program changes I have been wanting to work on.

  • Cleaned up and enhanced
    • Plotly graphing code
    • Test Code
  • Added username + password in GUI Tests
    • Only runs tests if both Sensor Online & has the correct login
  • Available Configuration settings are loaded up to what is provided (By file or text content), missing items (for new settings added here on) are given default values and saved
  • Added ability to change Web Login credentials from the Web Portal
  • Added ability to change SSL certificate files from the Web Portal
    • Create new Self-Signed SSL
    • Upload custom SSL
  • Added ability to Upgrade Python3 Modules from the Web Portal
  • Removed Re-Install requirements button (Does it on every upgrade)
  • Disabled Clean SMB Upgrades (Needs to be set up, breaks install if not)
  • Multiple bugfixes

I still have a bit of work to do on the configuration report and the online service configurations logging. Then it’s just general review before releasing a new version. It should be ready within the next few days.

Bit of a break + updates

So I took a few weeks off to do other things like setting up a bunch of remote access for clients at work, due to the coronavirus and lock down some Exchange servers to limit spam.

I spent a bit of today cleaning up KootnetSensors code in preparation for a new release. I also added the ability to upload a database to a sensor, replacing the previous one. It checks to make sure it’s a valid database before replacing it, and it creates a zipped backup of the old database in the main data directory. This should come in handy for a few reasons. First, it allows you to upload a database to create graphs on a more powerful computer. Second, it allows easy sensor replacements when needed as you can upload previous databases from the web interface.

Later on, I want to be able to swap between different uploaded databases in graphing, so you can work with multiple databases on a single unit. I may also make it so you can select remote sensors to graph locally but that won’t be done anytime too soon… or will it?

I hope everyone is safe during this crazy Coronavirus time. I’m happy I live in a more remote area, where I can go outside for hikes.

Until next time!

Multiple Updates

I have managed to go through a bit of my Todo list. The following list of things has been completed!

  • Sensor Latency integration with Sensor Control
  • Generic Configuration Class use among all Configurations
  • Unit tests created for a good portion of the Flask ‘Sensor Get’ routes and all Configurations.

With the new generic configuration class in place, working with any of the 7 configurations is a lot easier, thanks to shared access functions. Saving, loading and updating configurations are all done the same way now.

With the new unit test’s I can test the program in my IDE or use a real-live sensor. I also made a bit of a front end for running the tests, so it can be accessed and used through the OS system menu easily.

I’ll probably release another version soon with the fair amount of changes since the previous release (Notes, Latency reports, tests, unified configurations, bugfixes, etc.).

Web Portal Notes!

I have now added the ability to view, create, modify and delete notes from the web portal. I’m using an HTML editor called “TinyMCE”, which works well and integrated fairly easily. It can’t embed pictures or media but It’s nice to have basic formatting, lists printing, etc. Because it saves as HTML code (which is good for compatibility), Kootnet Control Center’s note section does not support it, so viewing in Control Center will show HTML code instead of an HTML page. I’m not completely done the Notes section yet, as I still have to add the custom Date & Time stamp the user can edit to show wherein the database the note is referring to and I wanted to get the spell checker working.

On a personal note, the wife has left the province for work and has stints of 3 to 5 weeks at a time away from home before coming back for 1 or 2 weeks. So I have a little less time for programming projects.

The weather has been mostly cloudy grey and the temperature is kinda random from like +7c to -11c with wind, rain, snow, hail, etc. I’m looking forward to warmer and sunnier weather.

Back to my day job for now. I have a few work projects on the go too, like finishing an Exchange migration to Exchange 2019, replace some failed drives in a server and get an IP camera system working from outside the local network but through the VPN for security.

New Sensors Added & Standardized Configurations Created

I added 3 new Pimironi sensors. The MCP9600 (Temperature), MSA301 (Accelerometer) and the SGP30 (Gas Total VOCs). I don’t actually have any, so I can’t test them, but its all set for when I do, or if someone else does.

I have also decided to standardize my configuration files through a base inherited class. With this, the majority of functions used among the configurations are the same. So you load and save all the configurations the same way for instance. It has also cut down on the number of lines by a decent amount. I still have a few configurations to update, but I have already converted the primary configuration and the installed sensors. I’m working on Trigger variance configuration, then it’s just the Online Service Configurations and Sensor Control’s.

That’s it for tonight’s updates.

Bit of a break

I took a bit of a break from my main programming projects and helped make an installer for another open-source project Darkstar. I started with a deb installer but realized a script would do a better job, especially for customizations. So that took about 3 days to get working well. Doing it even allowed me to fork the project on GitHub & create my first Pull request for someone else. So that was nice to be able to contribute. The reason I choose this project was that I was requested to set up a server for my brother, so he could more easily work on his FFXI graphics improvements. Basically, he has been making his own custom textures for the game that looks really beautiful! Unfortunately, the install instructions at that time were out of date and consist of a bunch of terminal commands and a few scripts. Without the new installer script, it took a while to get it to work at all because it was missing dependencies and was written for a much older Ubuntu install 14.04. At the time I could only get it working on an Intel system with Ubuntu. Now with the interactive installer script I made, it can be installed on the latest versions of Debian, Ubuntu and Raspbian. Being able to take advantage of the cheap Raspberry Pis is definitely a bonus. With a single terminal command, one can get set up with a Darkstar server and only need to enter a user, password and IP address. So I’m pretty happy about that.

In other news, it has been snowing A LOT here. I even had to climb my roof and remove the snow there so it wouldn’t possibly collapse. The snow on my shed roof (which I don’t bother to clear), is about 3 or 4 feet tall, and the roads are basically bumpily packed ice sheets. There’s also a huge fancy snow hill at our local no-frills parking lot that’s probably 12 or more feet tall and covers a very large amount of the parking lot… maybe … 2500 Sq ft. Anywho, it’s been a very snowy winter this month so far and although I’m not super impressed, my daughter is very much enjoying it.

I have only poked at my Sensor Project a little bit, but I was able to fix a bug that kept freezing my program after a few weeks. One of the sensors required an open and close when accessing it (it’s own custom version of it). But I didn’t close it when done with reading, so it kept pilling up open i2c connections until I couldn’t open any files at all in the program. I’m still testing to make sure it did resolve it, as it takes a few weeks for it to freeze up, but I’m fairly sure that was the issue, as it didn’t occur with any of my other sensors that were missing this one particular sensor.

Now that the installer is made for Darkstar, I’m thinking of adding additional support for sensors in my Sensor Project, but since I don’t physically have the new sensors, I won’t be able to test them, but I can at least get the backend set up and ready for testing. I’ll probably put these sensors under a “Un-Tested” section as a warning they may not function correctly.

That’s it for now. Slow progress for now due to the winter blues. Here’s hoping the sun decides to grace me with its un-clouded presence soon!

New standard Debian Installer!

I have been wanting to create a more standard installer for a while now, and I have finally gotten to it!
I decided to go with a Debian installer, as it was the simplest to figure out (after hunting down the right documentation), not to mention it’s one of the oldest ones. Originally I wanted to use a container installer like snap, AppImage, flatpack or Docker but .deb just seemed to make more sense for now…

So now Installing and Uninstalling has never been easier! Download and double click the file to install! This should make it easier for people used to Windows or Mac computers (but not compatible with Windows or Mac…). Once installed it should list kootnet-sensors in your Debian software manager. You can also remove with apt-get remove kootnet-sensors.

I have managed to get the old installer re-written in a way that will allow upgrades from the pre-deb installer to the new deb installer. It requires you to upgrade twice though, once to get the transition installer and another time to initiate the transition installer. Basically just upgrade from the web interface, wait 30 seconds, then upgrade again.

I have already done a fair bit of testing and optimizing, such as figuring out which apt-get dependencies are required and which are not. I put required to run at all stuff as required in the deb installer and sensor hardware requirements under recommends. This way you can use the management pages on Ubuntu for example, without requiring the hardware dependencies that may not be available on other Operating Systems.

After a bit more bug checking, optimizations, testing and documentation, I will release the new version and update the online documentation to match.

Sensor Control Enhanced!

I have finally finished up the Sensor Control section of the Web Portal. All configurations can now be edited and pushed to multiple sensors using Sensor Control. Once I do some testing, I’ll release this into the stable channel. Besides finishing up on the Configuration section of Sensor Control, I also added drop-down buttons for upgrading, power commands and misc other.

I would like to add the ability to review and add notes next, but I might space it out with a bit of reading. I got a new book “Modern Python Standard Library Cookbook”. I want to be able to do as much coding using the Standard Python Library as possible, so I can limit the outside requirements. This should also make my code cleaner and more efficient.

A new release is coming soon. As always, feel free to check out what is coming by upgrading to the development channel, but be warned, the development channel can be unstable.

Sensor Latency Tests Added

I finally got around to adding Sensor Latency testing. A button has been added under the “System Management” page to run it. The latency tests tell you just how long it really takes for a sensor to return a reading. This is required knowledge to set trigger variances in the seconds or less, so it doesn’t end up building a queue of reading requests waiting to get data. I have already set each sensor’s “multiple access wait times” to match more closely with how long it takes to get a sensor reading. This should improve performance when sensors are accessed at the same time.

Most sensors return data within 100ms (0.1 seconds). Some like the EnviroPhat can get a reading from each sensor within 10ms. But there’s one sensor, the light spectrum AS7262 that can take anywhere from 700ms to 8 seconds! I’m not sure exactly why, because my other spectrum sensors, like the BH1745 and the EnviroPhat, can grab readings in less than 20ms every time. I also checked the source code for the driver and how I set-up the API, and it’s set for under 30ms to get a reading for each of the six colours, so added up, it shouldn’t be much more than 200ms, and that’s assuming it can’t do them in parallel.

I have been putting off doing the configuration section under Sensor Control because I know I can actually get it to re-use my existing individual configuration pages without copying them … but it would take some creative re-working, and I have been lacking in creative motivations. Soooo… I think I’ll just copy-paste to get it working and refactor it later.

Kootnet Sensors Runs on Ubuntu

W00t, I made it so Kootnet Sensors can be run on Ubuntu (and probably other GNU/Linux systems). It just disables the hardware access functions but allows the web interface with access to “Sensor Control” to manage other sensors. Most changes were made to where the configurations are saved.

So this is awesome for testing! I can now just get it running from my Development Environment and test most HTML functions. It also does not require root access if there are no sensors.

Besides that, I also moved the sensor in use checks to the driver interface so nothing could bypass the check. The “Online Services” section of Sensor Control now works as well, except for setting the interval.

Things are coming along nicely.