Category Archives: Random - Page 2

Program Progress!

W00t, back to some program progress. I’m happy to say I have finished up the Database Information window and have created (but not finished) an Offline Database Notes Editor. It’s pretty much done already, I just have to finish up a button and add error checking.

After opening a database, the Offline Notes Editor automatically displays the first note out of the total on the screen. It also displays the current notes datetime stamp, converted to the local time. From this screen, you can add new notes, delete the current note or just read through the notes one by one. I will probably replace the Online SQL Note inserter with a replica of the Offline editor but have it connect to a single IP. Then maybe add a “Broadcast New Note” checkbox to enable sending a note to all selected sensors.

That’s it for now, I have to get some more cleaning done and such.

Warmer weather, biking and program progress

So the weather went from cold to warm pretty darn quick. Most snow has melted in town and it shouldn’t be much longer until the trails are snow-free (although probably very muddy). I recently got my bike back from the shop and it was given a clean bill of health. I haven’t done much riding yet, due to the snow patches and gravel, but I think by tomorrow I should be OK for most side roads. I’m looking forward to getting back on the E-Bike!

As far as program progress goes, things went downhill for a bit there but things look promising now. I have started a “Database Information” section to get quick information out of a sensor database. When a database is selected, it shows which database is selected, its recorded date range, database size and the number of notes in the database. Next, I’ll add sensor name and IP changes with the DateTime showing when it changed, as well as how many times the sensor has rebooted.

Once that section is complete, I will add a database analytical section for quick information for things like the top 10 highest and lowest values of a column, invalid entries, etc. I still have to think of what else should be in there, but it may have a companion section using machine learning, so the program can find and list possible interesting information more dynamically.

The machine learning section will have to wait a bit, because I have to learn about machine learning 🙂 I have purchased a book already, so its on the todo list, but while I read through it, program updates will be less frequent.

I think that’s it for now.

Getting warmer & sunnier

It has been a bit since I have done anything significant on my programs. However, today was especially nice and sunny, which gives me hope for warmer times ahead! Instead of working on the programs, I decided to work on some hardware. I have setup a few mini wireless routers to use with sensors in other locations. Each mini router comes with a OpenVPN server, so its perfect for secure communications and testing.

I’m thinking of putting sensors at Meadow Creek, Nelson & Kaslo, mostly because I have clients or know people there who can host them easily enough. This will be nice in 2 ways. First, I’ll be able to test remote deployment and upgrades. Second, I would like to start recording environmental data in areas around me to see how things correlate.

Hopefully, things will warm up and pick up soon.

Alpha.23.39 Released

Another release! It has been awhile, so there are a few changes including better trigger sensor recording, OpenGL Plotly rendering for graphs and multiple configuration additions such as Wifi and the trigger variances.

In other news, I resolved most of my Ubuntu issues I was experiencing. Apparently, it was google chrome’s “Google Play Music” app that was killing my media keys. Closing chrome fixed it every time and opening it often (but not always) killed the media keys. Disabling the one music app resolved the issue. Samba (Windows network share) was also having issues when my network didn’t initialize right away, so I changed the service file for samba to require the network loaded first. It was… kinda set that way with a ‘network-online’ version, but when I replaced it with ‘network’, it loaded peachy keen. It also appears as though my dock network connection works properly now, as it usually wouldn’t work until I removed the network cable and put it back in. I’ll have to try it for a few weeks to see if it’s really fixed. All in all, I’m even more happy with my new Laptop now.

There has been a cold front that’s lowered our local temperature from a nice 0c or -3c to a mighty cold and windy -15c. This has put a large damper on motivation to get out the door, which in turn kills a good portion of my motivation in general. I don’t expect much progress until it warms up a bit and I start to do more exercise. I also have to learn a few things for work, which will take what little motivation I have.

Extra Configuration Added

Progress is still slow and I’m still looking forward to warmer weather. However, I have now added some new Sensor configurations to the control center. The wireless supplicant file can now be edited and pushed from the control center. I have also added the variance configuration file, where you can enable, disable and set variance triggers per sensor. I still have to modify the configuration load code and test all the sensors but I have already tested lumens and pressure, and it seems to be working. I’ll test the rest soon.

I have been using my new Dell XPS 13 for a few weeks now and for the most part, I really like it. I love a lot of the smaller features in Ubuntu. One thing that really annoyed me about windows, is when you go to enter your password when the screen is off, it doesn’t register the first keystroke. So you have to wait for the screen to turn on and make sure it’s ready for your password. In Ubuntu, you just type it, hit enter and when the screen turns on, you’re already in 🙂 Minor but nice when you are having to log in a bunch. Of course, windows hello and stuff solves those, but since the computer is off to the side and closed when using the docking station, it wouldn’t work anyway.

Some of the annoyances I have run into so far have been odd glitches. Here are a few of them.

1. Once my side “dock” bar on the desktop froze, and when the screen locked, it was still showing on the side. So it seems as the lock screen could be bypassed with some time and effort, also, the dock freezing is annoying.
2. Some of my multimedia keys stop working randomly and sometimes start working randomly. I have yet to figure it out and fix it. I might actually contact dell and work with them, as once its fixed in Ubuntu, its fixed for everyone.
3. Once or twice I have had a Random Freeze up, where nothing responded. Usually, it’s temporary, but once I had to force shutdown the computer. This may also tie into #4.
4. The kworker process sometimes maxes out a single thread for long periods of time. It may be related to the internet, as it stopped right after connecting online. A bit of research tells me the kworker process works with the kernel and might be a driver issue.

All in all I’m pretty happy with the system and I’m certain the few glitches will be solved (Yay open source!).

It’s a beautiful day, so I’m going out for a family walk to get some sun!

Mild progress, new laptop

I have not posted an update recently, as I have been busy with work and configuring my new laptop. I decided to go with a Dell XPS 13 Developer edition (AKA Ubuntu pre-installed). So far I’m pretty happy with the computer purchase. The price was rather high, but the design is darn good. I didn’t think I would like to type on a 13″ laptop… notebook… netbook… Ultrabook… whatever you want to call it, but I do! It’s a well-designed keyboard on a few levels. I won’t go into any of the details, as there are enough reviews of the thing for sure, but I like it! Fast, good battery life, made to run Gnu/Linux ^_^

It has been a few years since I last tried to use Ubuntu or some other Gnu/Linux variation full time. I have a good feeling about this time since a lot of the deal breakers have been solved! The following issues I had before to varying degrees have been fully fixed.

  • Netflix
  • Games (Steam)
  • TeamViewer
  • Managed Service application I use
  • Audio and Video seem to work flawlessly now

I still have a few more things I need to setup and configure, but I already have PyCharm working and my programs loaded and working. So I’ll be doing more programing soon.

I found out that Ubuntu 18.04 displays the interface of my sensor control center app different than both Windows & Raspbian, so I added GUI customizations to support Ubuntu. OpenGL works well for graphing on my Ubuntu computer, so I have enabled it by default for Ubuntu, but I’ll have to add the option to use CPU rendering in case others are not as lucky to have a working graphics card.

Just doing a quick update today. Until next time.

Python Virtual Environment

W00t, I have python virtual environments setup and working with a requirements.txt file in the Linux sensor installer script. Shortcuts & services now use the python virtual environment. Now it doesn’t matter if the main python environment is altered! I’m glad I got this working because pipenv just wasn’t working quickly enough for the poor Pis.

I’m having trouble with my configuration file upgrade section … it doesn’t always load things properly between changes and will probably be time intensive to maintain all the upgrade paths. With this in mind, I’m thinking about just resetting configuration files to default when said files change between versions. It would force one to change the config file after larger updates but at the same time, it would mean the sensor would always load up properly after an upgrade… I was thinking I could make a backup of the old configurations during the upgrade and add the ability to see previous configs in the control center for a quick reference. I’ll have to give it some more thought.

I have been giving more thought to my Sensor Triggers and it comes to mind I should probably make different types of triggers. I’ll probably split the triggers into variances (Readings differ by a variance within a given time) and min/max values (Readings exceed or go below a value, checked every X time). AKA I’ll have 3 ways to record data.

  • Record every X seconds (Interval Recording)
  • Record if the difference between 2 readings, taken every X.X seconds is more then the variance (Variance Trigger Recording)
  • Record if the sensor value is above X or below Y. Check every X Seconds (Min/Max Trigger Recording)

Those 3 types of checks should cover most project scenarios where sensor recording is required.

I have had a chance to poke the program a bit more over the past few days, so I’m happy about that. Still looking forward to warmer and sunnier weather!

Re-Configured hardware sensor config

I finally had a clear enough mind to really go through and update the raspberry pi sensor software configurations. Before I had multiple functions load the config file from the disk, but this means its reading from the disk multiple times, and the disk is one of the slower parts of a computer, especially for the Pi’s. So, I made it so the configuration py file itself loads and creates the configuration object, and all others just use that object. This way it only loads once (well technically twice because it’s used in 2 programs). This also standardizes things for simplicity, so whenever anything uses the config, it’s done the same everywhere and points to the same object instance. Easier to follow code and saves a bunch of system resources.

I’ll have to remember to do my refactoring when I’m feeling really good, as refactoring takes the most thought and planning… well that and writing the initial code 🙂 But I usually don’t know the best way to do it the first time, especially if I have to “fiddle around” to get it working, so the better work generally comes during refactoring.

During this configuration overhaul, I made a separate python file for trigger variances, since I will be making interval and trigger recording work on all sensors. With the new variance python file in place, I’ll be creating a new configuration file to enable variances per sensor & making it so the interval and trigger recording can be enabled and disabled separately in the main config file.

Motivation is especially low, so little has actually been done in the past 4 or 5 days. Something about the winter…

Potly Graph updates

So I thought it was about time to update the offline Plotly graphing module to allow the new sensors. While I was in there, I thought I might as well update a few other things. So I added some consistent markers to the XYZ readings, replaced CPU rendered graph with OpenGL (WebGL) rendering for Windows (doesn’t work properly on the Raspberry Pi).

I have to say, the OpenGL rendering is SOO much faster than using CPU rendering. I can now easily zoom in and out of really large data sets. Large for me being over 115,000 data points per sensor entry. When I do the same graph with CPU graphing, it takes more than 2 min to render at all, and I never got to the point I could zoom in. Running my mouse over readings to see the values and datetime stamp lagged a lot too. So yay for GL rendering! I just hope its fully supported on the Raspberry Pi later on, as it makes a big difference!

In other news, it was nice and sunny today! I spent a decent amount of time outside with my daughter, absorbing as much sun as I could while she played on the big snow piles.

It may take a bit longer to finish the new trigger recording code should I decide to re-work how its done … which I might … I don’t plan on changing what it does, just how it does it… I’m pretty sure I could use a few lists or a class with a ‘for’ statement or two in order to shorten the length of the code.

That’s it for now. I’ll probably release another version once I fully finish up the new trigger sensor recording. The winter continues to slow progress, but I’m still poking the code every now and then!

Finally done the trigger overhaul

OK, it did take a bit longer to get the trigger recording overhaul done, but mostly due to the downturn of motivation, however, it is all done now! well not ‘done done’, as I still have to add additional sensors to the trigger checks and make configuration options for them, but Interval recording is now recording all sensor readings and Trigger recording is working like before, but ready to have additional sensors added. I also took the time to do more code refactoring, such as creating folders for holding the program modules and consolidating more options into the configuration py file.

Off to do more holiday stuff with the family.