Progress is still slow and I’m still looking forward to warmer weather. However, I have now added some new Sensor configurations to the control center. The wireless supplicant file can now be edited and pushed from the control center. I have also added the variance configuration file, where you can enable, disable and set variance triggers per sensor. I still have to modify the configuration load code and test all the sensors but I have already tested lumens and pressure, and it seems to be working. I’ll test the rest soon.

I have been using my new Dell XPS 13 for a few weeks now and for the most part, I really like it. I love a lot of the smaller features in Ubuntu. One thing that really annoyed me about windows, is when you go to enter your password when the screen is off, it doesn’t register the first keystroke. So you have to wait for the screen to turn on and make sure it’s ready for your password. In Ubuntu, you just type it, hit enter and when the screen turns on, you’re already in 🙂 Minor but nice when you are having to log in a bunch. Of course, windows hello and stuff solves those, but since the computer is off to the side and closed when using the docking station, it wouldn’t work anyway.

Some of the annoyances I have run into so far have been odd glitches. Here are a few of them.

1. Once my side “dock” bar on the desktop froze, and when the screen locked, it was still showing on the side. So it seems as the lock screen could be bypassed with some time and effort, also, the dock freezing is annoying.
2. Some of my multimedia keys stop working randomly and sometimes start working randomly. I have yet to figure it out and fix it. I might actually contact dell and work with them, as once its fixed in Ubuntu, its fixed for everyone.
3. Once or twice I have had a Random Freeze up, where nothing responded. Usually, it’s temporary, but once I had to force shutdown the computer. This may also tie into #4.
4. The kworker process sometimes maxes out a single thread for long periods of time. It may be related to the internet, as it stopped right after connecting online. A bit of research tells me the kworker process works with the kernel and might be a driver issue.

All in all I’m pretty happy with the system and I’m certain the few glitches will be solved (Yay open source!).

It’s a beautiful day, so I’m going out for a family walk to get some sun!

I have not posted an update recently, as I have been busy with work and configuring my new laptop. I decided to go with a Dell XPS 13 Developer edition (AKA Ubuntu pre-installed). So far I’m pretty happy with the computer purchase. The price was rather high, but the design is darn good. I didn’t think I would like to type on a 13″ laptop… notebook… netbook… Ultrabook… whatever you want to call it, but I do! It’s a well-designed keyboard on a few levels. I won’t go into any of the details, as there are enough reviews of the thing for sure, but I like it! Fast, good battery life, made to run Gnu/Linux ^_^

It has been a few years since I last tried to use Ubuntu or some other Gnu/Linux variation full time. I have a good feeling about this time since a lot of the deal breakers have been solved! The following issues I had before to varying degrees have been fully fixed.

• Netflix
• Games (Steam)
• TeamViewer
• Managed Service application I use
• Audio and Video seem to work flawlessly now

I still have a few more things I need to setup and configure, but I already have PyCharm working and my programs loaded and working. So I’ll be doing more programing soon.

I found out that Ubuntu 18.04 displays the interface of my sensor control center app different than both Windows & Raspbian, so I added GUI customizations to support Ubuntu. OpenGL works well for graphing on my Ubuntu computer, so I have enabled it by default for Ubuntu, but I’ll have to add the option to use CPU rendering in case others are not as lucky to have a working graphics card.

Just doing a quick update today. Until next time.

W00t, I have python virtual environments setup and working with a requirements.txt file in the Linux sensor installer script. Shortcuts & services now use the python virtual environment. Now it doesn’t matter if the main python environment is altered! I’m glad I got this working because pipenv just wasn’t working quickly enough for the poor Pis.

I’m having trouble with my configuration file upgrade section … it doesn’t always load things properly between changes and will probably be time intensive to maintain all the upgrade paths. With this in mind, I’m thinking about just resetting configuration files to default when said files change between versions. It would force one to change the config file after larger updates but at the same time, it would mean the sensor would always load up properly after an upgrade… I was thinking I could make a backup of the old configurations during the upgrade and add the ability to see previous configs in the control center for a quick reference. I’ll have to give it some more thought.

I have been giving more thought to my Sensor Triggers and it comes to mind I should probably make different types of triggers. I’ll probably split the triggers into variances (Readings differ by a variance within a given time) and min/max values (Readings exceed or go below a value, checked every X time). AKA I’ll have 3 ways to record data.

• Record every X seconds (Interval Recording)
• Record if the difference between 2 readings, taken every X.X seconds is more then the variance (Variance Trigger Recording)
• Record if the sensor value is above X or below Y. Check every X Seconds (Min/Max Trigger Recording)

Those 3 types of checks should cover most project scenarios where sensor recording is required.

I have had a chance to poke the program a bit more over the past few days, so I’m happy about that. Still looking forward to warmer and sunnier weather!

I finally had a clear enough mind to really go through and update the raspberry pi sensor software configurations. Before I had multiple functions load the config file from the disk, but this means its reading from the disk multiple times, and the disk is one of the slower parts of a computer, especially for the Pi’s. So, I made it so the configuration py file itself loads and creates the configuration object, and all others just use that object. This way it only loads once (well technically twice because it’s used in 2 programs). This also standardizes things for simplicity, so whenever anything uses the config, it’s done the same everywhere and points to the same object instance. Easier to follow code and saves a bunch of system resources.

I’ll have to remember to do my refactoring when I’m feeling really good, as refactoring takes the most thought and planning… well that and writing the initial code 🙂 But I usually don’t know the best way to do it the first time, especially if I have to “fiddle around” to get it working, so the better work generally comes during refactoring.

During this configuration overhaul, I made a separate python file for trigger variances, since I will be making interval and trigger recording work on all sensors. With the new variance python file in place, I’ll be creating a new configuration file to enable variances per sensor & making it so the interval and trigger recording can be enabled and disabled separately in the main config file.

Motivation is especially low, so little has actually been done in the past 4 or 5 days. Something about the winter…

So I thought it was about time to update the offline Plotly graphing module to allow the new sensors. While I was in there, I thought I might as well update a few other things. So I added some consistent markers to the XYZ readings, replaced CPU rendered graph with OpenGL (WebGL) rendering for Windows (doesn’t work properly on the Raspberry Pi).

I have to say, the OpenGL rendering is SOO much faster than using CPU rendering. I can now easily zoom in and out of really large data sets. Large for me being over 115,000 data points per sensor entry. When I do the same graph with CPU graphing, it takes more than 2 min to render at all, and I never got to the point I could zoom in. Running my mouse over readings to see the values and datetime stamp lagged a lot too. So yay for GL rendering! I just hope its fully supported on the Raspberry Pi later on, as it makes a big difference!

In other news, it was nice and sunny today! I spent a decent amount of time outside with my daughter, absorbing as much sun as I could while she played on the big snow piles.

It may take a bit longer to finish the new trigger recording code should I decide to re-work how its done … which I might … I don’t plan on changing what it does, just how it does it… I’m pretty sure I could use a few lists or a class with a ‘for’ statement or two in order to shorten the length of the code.

That’s it for now. I’ll probably release another version once I fully finish up the new trigger sensor recording. The winter continues to slow progress, but I’m still poking the code every now and then!

OK, it did take a bit longer to get the trigger recording overhaul done, but mostly due to the downturn of motivation, however, it is all done now! well not ‘done done’, as I still have to add additional sensors to the trigger checks and make configuration options for them, but Interval recording is now recording all sensor readings and Trigger recording is working like before, but ready to have additional sensors added. I also took the time to do more code refactoring, such as creating folders for holding the program modules and consolidating more options into the configuration py file.

Off to do more holiday stuff with the family.

After a bit of thought, I think it makes sense to have ALL sensors available as both Interval and triggers. So that’s my next overhaul. I’m also going to add the option in config to enable to disable each one individually as well. For simplicity and making sure the sensor doesn’t start filling the storage too quickly, I’ll have Intervals enabled by default @ 5 min and Triggers disabled by default. Since I already have the 3 data point checks in place, it will be extra easy to add single checks … in fact, I’m pretty sure I can just make a single ‘single check’ 🙂 Pass the type and sensitivities as variables and bobs your uncle!

On another note, I have done preliminary work on the SQL modifier. I drew up a design idea about how to display and interact with the data. So I’ll start up the GUI soon… I’m just debating integrating it into the main control center program, instead of having it separate. I’ll be sure to make it modular as I go, so it can be integrated easily later if I so choose.

1 day later …

So I’m halfway through my Interval and Trigger modifications. There’s actually a bunch of other refactorings that have to be done to make it all work. Nothing too complicated, but I do have to step back a bit to see the big picture and figure out the best way to have it all mesh and work together.

I also took the time to re-organize my file structures for the rp-sensor program. I moved all the operation py files into a operations folder. It’s a lot cleaner now since many operational files have been added over time.

The development version of my program is broken until I get the modifications done, so I’ll most likely have it done by tomorrow (I don’t like leaving things broken). Who knows, maybe I’ll even finish it tonight … but I doubt it, since my brain is not working very well today.

In other news, I have set up my Goal Zero Yeti 150 in the shed plugged into power to keep charged. So it should keep my sensors in the back up during power outages. Later, when there is more sun, I’ll see about setting up the solar panels to keep it charged instead. It has been really dark recently, not only due to it being the shortest day of the year soon but also it has been darn cloudy. I miss summer and fall already… well maybe not the superheat of summer, but definitely the bright sun during it.

Slow winter progress but progressing all the same. At some point down the road, I also want to start my IT program, which I will use as my quick goto utility to get system information and run common commands. I might even make it with Flask or Django, so it has the ability to be used remotely later on.

I hope everyone has a great winter holiday!

New version out!  According to my website, it’s been exactly 1 month since the last release.  There have been a lot of changes since the last one, which includes a change from python sockets to Flask.  So this version is very much not compatible with the previous, as far as the Control Center communication goes.  The bigger changes include the following.

• Change from Sockets to Flask
  • Starting WebApp to control sensors with a web browser
• Enabled support for 2 new Pimoroni sensors
• Sensor Configuration now allows Config & Installed sensors to be adjusted
• Added 6 channel support to the colour spectrum recording and Live graphing
• Added sensor provided temperature offset, determined by installed hardware.  This is also recorded to the database
• Refactoring, bug fixes & tweaks

One thing that is missing is the ability to do a Plotly graph with all 6 colour channels, but it still graphs the RGB of a 6 channel sensor just fine. I have also changed the python virtual env, included in the windows installers, so you can’t do an upgrade from previous versions.  The RP sensors should upgrade just fine though. 

That’s it for now.  

So, good news first, I just got some new sensors as of … 30 min ago?  I’m really excited to test out the 6 channel spectrometer, but I also got a light and proximity sensor, Garden HAT for the Pi Zero and a few LCD displays.
Specifically, I got the following from Pimoroni.

• AS7262 – 6 channel spectral sensor
• LTR-559 – Close range (4 cm?) proximity detector & lumen sensor
• 1.12″ Mono OLED 128×128 display

In other news, I revisited my Raspberry Pi Network Testers, which are 2x Pi’s made to test Ethernet cables and routes.  It’s now been updated to my present Python & Linux programming knowledge.  AKA it now uses systemd services, bash installer scripts and works directly with the original E-Ink libraries, instead of a 3rd party one I found.  All in all, its cleaner, more efficient and functions as one would expect for use and errors.  

I’m getting closer to releasing another Sensor Control Center update, but I might tweak a few things before that.  I want to enhance my SQL note section, so you can browse, edit and delete entries.  
We’ll see what happens. 

I have also been thinking about making an I.T. technician toolkit of sorts.  Creating a simple GUI app to quickly do common IT related tasks, like quick virus scan with a self-contained scanner, ccleaner type clean-ups, show system information like IP, uptime, free mem and HDD, etc.  I think I’ll also create a built-in script runner, so you can pick from a dropdown to run whatever script you like, as well as the ability to add new ones.  If I create most things as scripts, I should be able to do almost anything … So I’m kinda looking forward to releasing a useful diagnostic tool.  

Since I have been a bit busy with work, I have not been working as much on my programs.  

Until next time!

I have been working to update the smaller stuff after those overhauls before.  Editing comments, updating help files and fixing bugs.  

Now that I have Flask in use, it allows me to show reports off the sensor itself, by going to a URL with the sensors IP (or hostname if DNS is setup right).  I’m working on http://sensor_ip:port/TestSensor.  I created a combo of all 3 Control Center reports in one.  Going to that URL with an actual IP and the right port would give a nice combo HTML report of the sensor, complete with name, version, current sensor readings, configuration setups, etc!  
The big thing I like about this is you can take your cell phone, make it a wifi hotspot the sensors connect to and goto the URL in a web browser on said phone to see its status.  If I make an actual phone app, I may be able to automate the process with a push of a button.  Cool possibilities for the future! 

I have been pondering some other back-end changes.  Since I know a lot more about python now, I might move a bunch of my install code from the bash script into the program’s first launch.  I should probably keep the bash script to a bare min, just enough to get the program needs installed, then let the program start and take over from there.  

I should probably set a time to do a purely refactoring session too.  There are a lot of places where the code could use improvement where I was trying to get it working at all.  I also really want to get the Live Graph working with threading for the data gathering.  Since right now, if the connection is not so great, the program will freeze until it times out.  So what I’m thinking of doing, is simply threading the data gathering and putting results in a queue or pool of sorts.  So long as this pool allows the program to continue while waiting for data, it should work very well.  Otherwise, I’ll set aside some time to really think about how to make it shine … or maybe replace it… Since Matplotlib is not thread safe to begin with, maybe I should look for something that is more like a multi-camera dashboard.  I’m pretty sure most people are going to want to put a dashboard of live sensors up in front of them, to monitor things like crops and such.  

I will most likely release a new version soon, as there are a lot of nice touches on this one, such as most areas are threaded to prevent program freezes now (minus the live graph), the program is no longer dependant on a 3rd party HTTP server, the Control Center sensor re-configuration section has been re-done to allow config and installed sensor changes. It also allows grabbing a sensor configuration to compare or re-distribute to other sensors.  Graphing now allows a default temperature offset from the sensor itself, which has an automatic offset based on the hardware chosen, as well as allows a custom entry.  Notes can now be entered into the Sensor Database with a date and time stamp so it will (eventually) show up in graphs beside relevant sensor data. I have created an update and program checks that will make needed adjustments to settings and files after an upgrade.  Hmm, what else… O yeah, I was able to remove the rebuild of numpy with the new Raspbian version that’s out, but I had to add ‘atlas’ i think… but just for install, not rebuild it, so that’s good.  

Lots of good improvements and it seems to be running fairly smoothly with the transition from sockets to Flask… although I must admit, I do notice a slight more delay using Flask, which makes sense due to the overhead.  The delay is also measured in ms not seconds, so its not bad. It is especially hard to notice now that I have added threading to pretty much every sensor reading grab, so it shows up as a delay in showing, not a frozen program.   

That’s it for now.