Refactoring!

Ahh, I managed to get a few good hours in for refactoring. The imports are now standardized. AKA I import like this ‘from folder import module’, which I then access everything from it with module.whatever or module.whatever_function(). This makes it a LOT simpler to find out where everything is located. So tracking down functions and variables is easy now.

I also spent a good few hours on the local management and information HTML webpage. It uses a dark theme and displays all the useful data and links minus actual sensor readings, for which I am going to make another custom page which will have things like automatic reading updates, color changes based on sensor readings, etc. I have not even started on the page yet, but the empty functions are all set to be filled in. I’ll probably work on that one too before a new release.

The 2 things left to do before a new release are.
1. Refactor the control-center program like the RP sensor program (imports & naming).
2. Create a decent sensor readings webpage.

I suspect I’ll be ready by the end of the month. Yay motivation and progress ^_^

New Release soon

I’m fairly happy with my new program updates. The notes work well, the database information window shows useful database information, and I have put in a mini web interface for sensor interaction on the local web server (AKA control the sensor from a webpage served by the sensor). I might tweak the sensor report on the local web server but beyond that, it’s all about refactoring before the new release.

Starting on the local web server interface will make creating an app for the phone rather simple (web page wrapper). Although just bookmarking the sensor website is not difficult (So long as you know the port #).

I haven’t read too much more on the Machine learning yet, but it’s still progressing. I expect I’ll do some more serious reading next week.

That’s it for now.

Notes updates

I’m still slowly doing the notes section. I’m just finishing up on the dual DateTime entries in order to have a reliable index of notes and the ability to place a DateTime stamp for where the data is, that the note pertains to.

After this, I should do some refactoring. For one, I would like to maintain file imports so usage is imported.whatever, in order to easily see where said class, function or variable reside. So that’s going to take a bit and I’ll want to re-word some naming schemes so it doesn’t get too long like some_really_long_import.SuperBadAssClassWithCherriesOnTop()

I’ll have to remember to set aside time for refactoring, in order to bring older code up to my newer standards.

Off to do a bit of work.

Spring is here and progress continues

I’m happy to report most of the snow is gone and many of the roads have already been cleared of debris, dirt and dust. So I have been biking and walking around a lot more.

I have made a single Python file for offline note editing and online editing. It looks decent and works fairly well. I had to do some note text sanitizing for putting into the database and retrieving, so I didn’t get things like, quotes or slashes in the note breaking the SQL query or making new lines actually show as ‘\n’ in the note when viewing.

I still need to work on the date time stamp validation since it is user editable. I also want to add 2 date time stamps per note. One entered in as a index which would always use the current time converted to UTC 0, and another user editable one for where the user wants to associate the note to specific data in the database. This will not only solve a issue of not being able to delete a note with an incorrect date but also allows tracking of when things where done.

After doing the finishing touches on the notes section, it will be time to refactor. Admittedly it’s a tad sloppy at about 700 lines of code. I just wanted to get it working first to help with motivation, once working and happy with function, I’ll have a reservoir of motivation to work on efficiency and readability.

I started my machine learning book, which looks decent and I’m kinda excited to try it out on random data. It should be able to do what I’m thinking, which is finding correlating patterns between sensors and sensor readings. It should also help in finding issues in database entries, like incorrect date stamps on notes.

I still want to look into a better GUI instead of Python Tk, but it seems there are licenses on the qt one I’m not clear on, so I’ll need to look into that more if I go that route. I’m also keeping the possibility open of using a web based interface, as it can very much be customized and allows use over the network. It would also most likely negate the need for my platform tweaks file (changing aspects of the GUI to look right on different operating systems). Of course that opens up other security issues, especially if open to the internet.

That’s it for today.

Program Progress!

W00t, back to some program progress. I’m happy to say I have finished up the Database Information window and have created (but not finished) an Offline Database Notes Editor. It’s pretty much done already, I just have to finish up a button and add error checking.

After opening a database, the Offline Notes Editor automatically displays the first note out of the total on the screen. It also displays the current notes datetime stamp, converted to the local time. From this screen, you can add new notes, delete the current note or just read through the notes one by one. I will probably replace the Online SQL Note inserter with a replica of the Offline editor but have it connect to a single IP. Then maybe add a “Broadcast New Note” checkbox to enable sending a note to all selected sensors.

That’s it for now, I have to get some more cleaning done and such.

Warmer weather, biking and program progress

So the weather went from cold to warm pretty darn quick. Most snow has melted in town and it shouldn’t be much longer until the trails are snow-free (although probably very muddy). I recently got my bike back from the shop and it was given a clean bill of health. I haven’t done much riding yet, due to the snow patches and gravel, but I think by tomorrow I should be OK for most side roads. I’m looking forward to getting back on the E-Bike!

As far as program progress goes, things went downhill for a bit there but things look promising now. I have started a “Database Information” section to get quick information out of a sensor database. When a database is selected, it shows which database is selected, its recorded date range, database size and the number of notes in the database. Next, I’ll add sensor name and IP changes with the DateTime showing when it changed, as well as how many times the sensor has rebooted.

Once that section is complete, I will add a database analytical section for quick information for things like the top 10 highest and lowest values of a column, invalid entries, etc. I still have to think of what else should be in there, but it may have a companion section using machine learning, so the program can find and list possible interesting information more dynamically.

The machine learning section will have to wait a bit, because I have to learn about machine learning 🙂 I have purchased a book already, so its on the todo list, but while I read through it, program updates will be less frequent.

I think that’s it for now.

Getting warmer & sunnier

It has been a bit since I have done anything significant on my programs. However, today was especially nice and sunny, which gives me hope for warmer times ahead! Instead of working on the programs, I decided to work on some hardware. I have setup a few mini wireless routers to use with sensors in other locations. Each mini router comes with a OpenVPN server, so its perfect for secure communications and testing.

I’m thinking of putting sensors at Meadow Creek, Nelson & Kaslo, mostly because I have clients or know people there who can host them easily enough. This will be nice in 2 ways. First, I’ll be able to test remote deployment and upgrades. Second, I would like to start recording environmental data in areas around me to see how things correlate.

Hopefully, things will warm up and pick up soon.

Alpha.23.39 Released

Another release! It has been awhile, so there are a few changes including better trigger sensor recording, OpenGL Plotly rendering for graphs and multiple configuration additions such as Wifi and the trigger variances.

In other news, I resolved most of my Ubuntu issues I was experiencing. Apparently, it was google chrome’s “Google Play Music” app that was killing my media keys. Closing chrome fixed it every time and opening it often (but not always) killed the media keys. Disabling the one music app resolved the issue. Samba (Windows network share) was also having issues when my network didn’t initialize right away, so I changed the service file for samba to require the network loaded first. It was… kinda set that way with a ‘network-online’ version, but when I replaced it with ‘network’, it loaded peachy keen. It also appears as though my dock network connection works properly now, as it usually wouldn’t work until I removed the network cable and put it back in. I’ll have to try it for a few weeks to see if it’s really fixed. All in all, I’m even more happy with my new Laptop now.

There has been a cold front that’s lowered our local temperature from a nice 0c or -3c to a mighty cold and windy -15c. This has put a large damper on motivation to get out the door, which in turn kills a good portion of my motivation in general. I don’t expect much progress until it warms up a bit and I start to do more exercise. I also have to learn a few things for work, which will take what little motivation I have.

Extra Configuration Added

Progress is still slow and I’m still looking forward to warmer weather. However, I have now added some new Sensor configurations to the control center. The wireless supplicant file can now be edited and pushed from the control center. I have also added the variance configuration file, where you can enable, disable and set variance triggers per sensor. I still have to modify the configuration load code and test all the sensors but I have already tested lumens and pressure, and it seems to be working. I’ll test the rest soon.

I have been using my new Dell XPS 13 for a few weeks now and for the most part, I really like it. I love a lot of the smaller features in Ubuntu. One thing that really annoyed me about windows, is when you go to enter your password when the screen is off, it doesn’t register the first keystroke. So you have to wait for the screen to turn on and make sure it’s ready for your password. In Ubuntu, you just type it, hit enter and when the screen turns on, you’re already in 🙂 Minor but nice when you are having to log in a bunch. Of course, windows hello and stuff solves those, but since the computer is off to the side and closed when using the docking station, it wouldn’t work anyway.

Some of the annoyances I have run into so far have been odd glitches. Here are a few of them.

1. Once my side “dock” bar on the desktop froze, and when the screen locked, it was still showing on the side. So it seems as the lock screen could be bypassed with some time and effort, also, the dock freezing is annoying.
2. Some of my multimedia keys stop working randomly and sometimes start working randomly. I have yet to figure it out and fix it. I might actually contact dell and work with them, as once its fixed in Ubuntu, its fixed for everyone.
3. Once or twice I have had a Random Freeze up, where nothing responded. Usually, it’s temporary, but once I had to force shutdown the computer. This may also tie into #4.
4. The kworker process sometimes maxes out a single thread for long periods of time. It may be related to the internet, as it stopped right after connecting online. A bit of research tells me the kworker process works with the kernel and might be a driver issue.

All in all I’m pretty happy with the system and I’m certain the few glitches will be solved (Yay open source!).

It’s a beautiful day, so I’m going out for a family walk to get some sun!

Mild progress, new laptop

I have not posted an update recently, as I have been busy with work and configuring my new laptop. I decided to go with a Dell XPS 13 Developer edition (AKA Ubuntu pre-installed). So far I’m pretty happy with the computer purchase. The price was rather high, but the design is darn good. I didn’t think I would like to type on a 13″ laptop… notebook… netbook… Ultrabook… whatever you want to call it, but I do! It’s a well-designed keyboard on a few levels. I won’t go into any of the details, as there are enough reviews of the thing for sure, but I like it! Fast, good battery life, made to run Gnu/Linux ^_^

It has been a few years since I last tried to use Ubuntu or some other Gnu/Linux variation full time. I have a good feeling about this time since a lot of the deal breakers have been solved! The following issues I had before to varying degrees have been fully fixed.

  • Netflix
  • Games (Steam)
  • TeamViewer
  • Managed Service application I use
  • Audio and Video seem to work flawlessly now

I still have a few more things I need to setup and configure, but I already have PyCharm working and my programs loaded and working. So I’ll be doing more programing soon.

I found out that Ubuntu 18.04 displays the interface of my sensor control center app different than both Windows & Raspbian, so I added GUI customizations to support Ubuntu. OpenGL works well for graphing on my Ubuntu computer, so I have enabled it by default for Ubuntu, but I’ll have to add the option to use CPU rendering in case others are not as lucky to have a working graphics card.

Just doing a quick update today. Until next time.