I finally got around to putting in a bit of security. Now people can’t just go around rebooting, shutting down or changing the settings on my Sensors. I have worked Basic HTTP Authorization into the programs. However, due to myself wanting to keep more information open, I have only added required authentication on things that could change what the sensors doing at the time. AKA you can get all sensor readings without logging in and even most configurations, but you can’t change any configurations or see the WiFi settings without logging in.
The Control Center program also now supports the Authentication and uses it by default in any commands sent. The user and password can be changed in both programs.
To make the program communication actually secure, I would have to enable HTTPS as well, because the password is sent in clear text over HTTP … but that’s where a problem occurs because if the DateTime is wrong on the sensor (by default they don’t keep track of time when powered off), the HTTPS won’t work… I’ll probably add it later anyway, but I’ll have to see if I can at least get the Control Center to ignore invalid certificate’s so one can at least sync the time correctly… I’ll have to think a bit more about that… Maybe I can just do 3rd party encryption on the password before sending it … yeah, I’ll have to think about that one more.

In other news, I had to make a few changes with the new Plotly 4.0 out as well as a new matplotlib module. The Plotly one was fairly easy but the matplotlib seems to freeze up my program now when doing plots (plot works fine, but can’t do anything else while it plots). I’ll have to do a bit more research and play around with it to get that proper again since it was nice being able to check other aspects and sensors during a live graph.

Since I do a lot of sensor reports, I re-worked the main screen to have a drop-down selector for the reports and database download. I don’t like the layout but I do like the convenience. I also changed the sensor commands to drop-down menu’s as well to help with the cluttered button arrangement. I’m still not sure I super like the overall look and feel … so I guess I’ll have to add GUI design to my research “ToDo” list.

I suppose my next bit of poking will be to bring some of the control center features into the web app itself, so more can be done without the control center. A few cool things to add would be a web-based SSH terminal to the local unit, the ability to edit the local configuration files and creating an offline Plotly graph right from the sensor.

Until next time!

So, I have actually done a fair bit since my last post. I won’t list it all today, but I’ll mention a few recent highlights.

For starters, apparently, my Windows Control Center python virtual environment wasn’t actually working on other computers! Among other things, the Python3 virtual environments get tied into the username of where it was created. So when I went to test it on my other computer, it worked (Same username)! but when I tried to put it on someone else’s computer, it did not… So after a fair amount of searching, I found something better then what I was doing (on a few levels), called pynsist. Before I basically created a virtual environment and used Inno Setup to put it all together into an installer. All needing a windows machine to do it on. Pynsist, on the other hand, is all done in Linux creating an installer with an embedded Python3 and required files! Very nice for consistency, especially when you provide the pip packages.

It took a bit to get it working, especially around pip (PyPi) packages, since it doesn’t install the pip requirements of the package you install. AKA Plotly and matplotlib were missing a lot of required 3rd party modules! Long story short, it was easier to install Python on windows, use pip to get the packages, then copy them into an include packages folder for pynsist. Worked very well after that.

Because the previous installer is not working, I uploaded a not fully finished version for the Windows control center program, but it SHOULD only be minor visual glitches, such as window sizes and a not so great looking “Sensor Display” section to have the sensors show things like sensor readings, assuming it has a supported display installed.

Anywho, I’m going on a trip tomorrow for a few days, so I don’t want to stay up too late, but a few of the enhancements include the following.

• Display functions added for mini LCD / LED displays attached to the sensor.
• New sensor support (I Think I added 7 or 8 new sensors).
• Re-worked services so everything runs from a single program. The first program launches threads of other required programs like trigger variance checks and the HTTP server for remote access.
• Added button in CC for Upgrading to the latest Development version.
• Added new sensor types: Altitude, Distance, GAS, PM, Ultra Violet.
• Added more Sensor Commands in CC: Check/install missing dependencies, Clear Logs on Sensor.
• Refactored file names for clarity.

All in all, I’m pretty happy I have made more progress. Programing is also starting to make more sense and I’m finding it easier to figure out how to do things. Whether it’s the best way to do things or not is another story. There are definitely some structural improvements that could be made now.

Until next time!

After going for a bit of a hike and camp trip with my brother, I got a slight bit more motivation to poke at the sensor programs. This time I improved the installers and uninstallers. I’m pretty sure the Control Center HTTP installer now works with other Linux distributions decently. Works on my Pi’s and Ubuntu 18.04 just fine, as well as the uninstallers.

The Sensors program now offers to install the Control Center at the start, and makes installing a lot faster if you choose not to include it. If you do have the Control Center installed as well, uninstalling the sensors program will ask if you also want to remove the Control Center program.

I’m pretty happy with the HTTP installers now and will most likely convert the SMB installers to use a zip file like HTTP, instead of rsync’ing the folders and files. Although it will take an extra step when pushing a test, it will be a lot cleaner and quicker. I’ll probably create a script to auto zip the files into the right areas, taking only as long as it takes to type in the bash alias in a terminal.

I also want to look into making a .deb installer so it can integrate with the primary package manager and creates a normal spot for removal.

That’s it for now.

A new version is out!

I actually meant to post about this when I released it on May 1st, but O well.

This version has a lot of enhancements but I’m fairly … something right now, so I’ll post about all that later.

I’m also starting to build the application as a web app, so a web browser can be used to go to a webpage on the sensor itself to see all sensors and most other variables, like system uptime, RAM usage, last update, etc, etc. It should be fairly easy to port Plotly graphing over since it outputs into an HTML page already, so sending that page with flask will be easy. The big thing I have to learn is actually web design, so I can have consistent looks and feel throughout the pages. Having it look nicer will be awesome too.

With the web design in mind, I purchased a copy of “100 Days of web in Python”. I started the first part about flask, and I’m already getting some good pointers.

That’s it for now. I’ll post some updates later on the new features.

The title is a bit of a joke since ++ is a way to add one to an integer in C++, but not so much in Python.

While I was refactoring my code, Corey Schafer released a video for cleaning up Python code. So I watched a bit of it and got some pointers to start using in my programs. I learned about the zip() and enumerate() function from the video and have begun to implement them. It does make it look better for sure.

I have updated MOST of my imports in both the sensor program and the control center. The import modifications have made things clearer for where functions and variables are stored. Variables and functions have also been relocated to places that make more sense.

That’s it for today. Until next time!

Ahh, I managed to get a few good hours in for refactoring. The imports are now standardized. AKA I import like this ‘from folder import module’, which I then access everything from it with module.whatever or module.whatever_function(). This makes it a LOT simpler to find out where everything is located. So tracking down functions and variables is easy now.

I also spent a good few hours on the local management and information HTML webpage. It uses a dark theme and displays all the useful data and links minus actual sensor readings, for which I am going to make another custom page which will have things like automatic reading updates, color changes based on sensor readings, etc. I have not even started on the page yet, but the empty functions are all set to be filled in. I’ll probably work on that one too before a new release.

The 2 things left to do before a new release are.
1. Refactor the control-center program like the RP sensor program (imports & naming).
2. Create a decent sensor readings webpage.

I suspect I’ll be ready by the end of the month. Yay motivation and progress ^_^

I’m fairly happy with my new program updates. The notes work well, the database information window shows useful database information, and I have put in a mini web interface for sensor interaction on the local web server (AKA control the sensor from a webpage served by the sensor). I might tweak the sensor report on the local web server but beyond that, it’s all about refactoring before the new release.

Starting on the local web server interface will make creating an app for the phone rather simple (web page wrapper). Although just bookmarking the sensor website is not difficult (So long as you know the port #).

I haven’t read too much more on the Machine learning yet, but it’s still progressing. I expect I’ll do some more serious reading next week.

That’s it for now.

I’m still slowly doing the notes section. I’m just finishing up on the dual DateTime entries in order to have a reliable index of notes and the ability to place a DateTime stamp for where the data is, that the note pertains to.

After this, I should do some refactoring. For one, I would like to maintain file imports so usage is imported.whatever, in order to easily see where said class, function or variable reside. So that’s going to take a bit and I’ll want to re-word some naming schemes so it doesn’t get too long like some_really_long_import.SuperBadAssClassWithCherriesOnTop()

I’ll have to remember to set aside time for refactoring, in order to bring older code up to my newer standards.

Off to do a bit of work.

I’m happy to report most of the snow is gone and many of the roads have already been cleared of debris, dirt and dust. So I have been biking and walking around a lot more.

I have made a single Python file for offline note editing and online editing. It looks decent and works fairly well. I had to do some note text sanitizing for putting into the database and retrieving, so I didn’t get things like, quotes or slashes in the note breaking the SQL query or making new lines actually show as ‘\n’ in the note when viewing.

I still need to work on the date time stamp validation since it is user editable. I also want to add 2 date time stamps per note. One entered in as a index which would always use the current time converted to UTC 0, and another user editable one for where the user wants to associate the note to specific data in the database. This will not only solve a issue of not being able to delete a note with an incorrect date but also allows tracking of when things where done.

After doing the finishing touches on the notes section, it will be time to refactor. Admittedly it’s a tad sloppy at about 700 lines of code. I just wanted to get it working first to help with motivation, once working and happy with function, I’ll have a reservoir of motivation to work on efficiency and readability.

I started my machine learning book, which looks decent and I’m kinda excited to try it out on random data. It should be able to do what I’m thinking, which is finding correlating patterns between sensors and sensor readings. It should also help in finding issues in database entries, like incorrect date stamps on notes.

I still want to look into a better GUI instead of Python Tk, but it seems there are licenses on the qt one I’m not clear on, so I’ll need to look into that more if I go that route. I’m also keeping the possibility open of using a web based interface, as it can very much be customized and allows use over the network. It would also most likely negate the need for my platform tweaks file (changing aspects of the GUI to look right on different operating systems). Of course that opens up other security issues, especially if open to the internet.

That’s it for today.

W00t, back to some program progress. I’m happy to say I have finished up the Database Information window and have created (but not finished) an Offline Database Notes Editor. It’s pretty much done already, I just have to finish up a button and add error checking.

After opening a database, the Offline Notes Editor automatically displays the first note out of the total on the screen. It also displays the current notes datetime stamp, converted to the local time. From this screen, you can add new notes, delete the current note or just read through the notes one by one. I will probably replace the Online SQL Note inserter with a replica of the Offline editor but have it connect to a single IP. Then maybe add a “Broadcast New Note” checkbox to enable sending a note to all selected sensors.

That’s it for now, I have to get some more cleaning done and such.