Category Archives: Project

New Sensors!

So, I'm rather excited to say that pimoroni.com has released a new attachment board called the "Breakout Garden".  It lets you hook up to 6 sensors at a time, without the need to solder! And you can swap them around all you like.  I especially like the fact it makes the sensors stick up, so the heat from the Pi itself should affect it much less, if at all!  

I might of gone overboard a little bit… not going to say how much I spent, but I will say I have a good set of 4 fully loaded sensor boards on the way.  I'm pretty sure Pimoroni was the one that had really good Python libraries, so I'm looking forward to working with them, and probably making them my primary sensor boards.  Because of the modular nature of the board itself, and I'm working with soo many different sensors, I need to make the programs modular, so it can install and be used with different kinds of sensors…. Its a bit of a undertaking, so I'm going to think about how to implement that for awhile.  

In other news, I have some decently working installers for my PC Sensor Control Center application.  Everything appears to be working well, right after install. 

I added a Configuration window to load, set and change multiple settings, such as timezone offset, Temperature Offset, SQL Querries to skip in graping, default date selections, etc. I have also made all program and file location settings compatible with any OS (I'm pretty sure…).  If the config file is missing, it creates one with default entries, so if anything messes up in it, just delete config.txt in the main app directory, and load the program again for a fresh new copy.  You can also reset it from the Configuration window.   

As you can see in the picture, I have added the GPL ver. 3 to the program.  I also had some people recommend some well documented python code to try and emulate for commenting / documenting my own code.  I'll be working on that to make the code easier to modify.  The air quality in my town is litterally off the Canadian air quality scale, and I'm finding it harder to concentrate in general, so I might put this off for a bit.

 

Picture of my Sensors

I just realized, I have not actually posted a picture of my physical sensor units yet!  I started with the Raspberry Pi 3B+ and a SenseHAT, but changed to the Raspberry Pi Zero W with a Enviro pHAT later on, due to cost, power usage and expandability.  The bigger one is the Pi + SenseHAT, the mini's are the Pi zero + Enviro pHAT. 

The Raspberry Pi 3B+ with the SenseHAT was a lot easier to deal with, since I didn't have to solder anything, just push it into the pin.  

The Raspberry Pi Zero w on the other hand, I had to not only solder the 40 Pins on the Pi (for about 6 of them, after which I found pre-soldered ones), I also had to solder the connections on the Enviro pHAT.  So 80 solder points later per sensor and they are good to go … well physically any way.  This was my first go at soldering, and thanks to a video on the Raspberry Pi's website, it went fairly well (and by that, I mean they all worked).  

So there you go, those are my physically made sensors that my program connects to over the WiFi network.  

Program License – GPL

I mentioned before that I would post my code, which I will indeed post, but the more I thought on it, the more I realized I should probably choose a license model for the program now, before actually posting any code. 

To this end, I have decided on the GNU GPL Version 3.  I am 100% for sharing and making sure it stays free for anyone who thinks its useful and wants to use it.  If it ends up being used by a company with money, I would sure appreciate funds to continue development and get a few research stations going.  I'm also very much open to adding pretty much any sensor to the units, if you send me a few for testing. 

I'm going to work on the Installer and install scripts for the next few days, so that people can easily get this up and running to try out.  Of course the hardware is still needed to be effective, but I'll be sure to include a section in the Help Menu of the PC program, that will help you make your own sensor unit. Ultimately I would like to give people the ability to digitize the world around them easily, to find out all kinds of Cool stuff.  You never know what you'll find after visualizing the data in different ways.  

I'm thinking I might have something usable by the beginning of September, 2018.  That doesn't mean bug free or pretty, but it will be usable! 

I have already spent a bit of time on the PC installer.  I have been updating my hard coded locations in the program to system variables, so it can be installed and run from any windows computer (Will have to double check if the system locations work in Linux too).  It has just passed being usable on a different computer after running the installer.  The installer puts a folder of Python 3.7 in the app directory and the shortcuts link directly to it, running the script.  The sensor installers are already working fairly well, but I would like to customize it to ask for certain variables upon running it, so its a one stop shop to configure and install.  

I'll post the update log in a later post.  

Enhancements / Plans / ChangeLog

Since I do lots of testing on the fly with multiple sensors, I thought it was about time to add some functionality to the program around sensor control.  

Up until now, I have used SSH for 90% of my sensor interaction, but there is really only 2 things I do a lot of… Update the programs and reboot the sensor.  So to that end, I have added a "Sensor Commands" section to allow just that and more.  Essentially, you check the sensor('s) you want to do something with in the main window, than click a command button like upgrade, reboot or shutdown.  It will send the command to all selected sensors.  

I added a SMB update (Windows shares) and a Online HTTP update (allows upgrading remote sensors with internet access).  I added a checkbox for the shutdown, so I don't accidently click it during quick testing.

I have also created a Windows Installer for the program, which includes a install of python 3.7 with the required libraries.  I originally just created a single .exe using pyinstaller, but Plotly didn't seem to like working with it, so I thought the next best thing was to just make the install easy.  It also runs much faster this way!  

Some of the things I hope to add in the near future include.

  • Put phpSysInfo on Sensors, using lighttpd server for system diagnostic Information
  • Update output files to local user desktop
  • Improve Graph outputs and Options at generation
  • Have update script terminate programs at the end, so the updated versions can start with improved crontab autostart
  • Improve Install scripts for ease of use
    • Ask and auto set Hostname, IP, Wifi, etc
  • Have program write config to file, and load it on boot to Include
    • IP List for Sensors
    • Default Date ranges in Graphs

I spent some time going over the back end code, and thought it best to merge a few things.  So here's yet another changelog.

ChangeLog Aug 15th, 2018

  • Merged 2 .py programs into one for socket communication
    • Merged the live sensor data send with general socket commands
  • Improved logging
  • Added Time corrections to Motion Graph (timezone)
  • added Sensor command "Terminate Sensor Programs"
  • Changed Auto Start method from rc.local to crontab scripts.  
    • Auto runs script every 1 min
    • The script checks to see if the program is running
      • If not running, start it
      • If running, do nothing
      • Allows crash recovery and applying remote updates without restarting the unit
  • Updated E-Ink Controller with Static IP 
    • Updated E-Ink & Sensor update scripts to match new IP
  • Updated Motion Database to save IP for better graph Sensor Identification 
  • (35% done) Started creating a Database "Splitter / Merger / Updater" program to allow
    • Modifications of old databases to newer models or merging with additional fields 
    • Split databases for Later analysis or archiving (X timeframe is in Y location, take just that timeframe and store in new database for later access). 
  • Updated 'Update scripts' 
    • Improved Speed (less network copies)

Using Python3 to collect and manipulate SenseHAT Sensor data

So, as I said in my last post, I want to digitize the world around me, starting with the sensors on my Raspberry Pi 3B+ and SenseHAT attachment.  

After much thinking, writing down ideas and generally trying things out, I finally got things working the way I wanted for the most part.  Here's what I have so far.

1.  Sensor Units: These are the Pi's with the SenseHAT and have the following programs running to enable the features I want (Only the HTTP server is not done by me, I just installed one). 

  • Sensor Readings to File: A simple program that reads "all" the sensors and writes the results as a string, each value separated by a ",". It creates one line each time its run.  It auto runs every 5 min through cron (Linux scheduler).  You could also do something similar with Windows "Task Scheduler".
  • HTTP Server:  The HTTP server just lists and allows downloads of the sensor data files. 
  • SenseHAT Joystick Controls: A small program auto runs with the pi and allows you to use the 5 "button" joystick to run commands and display info on the 8×8 LED grid.  It currently does the following, which helps a great deal when there is nothing attached but the SenseHAT and power.  I also have VNC / SSH enabled on the Pi's to connect remotely.  
    • 1. Display IP
    • 2. Display CPU Temp
    • 3. Display SenseHAT Temp
    • 4. Shutdown
  • Sensor Data over WiFi: Another small program runs automatically and listens on the network for my client app to request data.  

2.  Portable Sensor Data Display and File Downloader:  This is a PI with a 2.7" E-Ink display with 4 buttons.  It simply runs one program on boot to do the following.

  • Show it's own system information, including IP
  • Display Current Sensor Readings
  • Download Sensor Data Files
  • Shutdown

3.  PC: I made a more in depth GUI app for the PC, that can connect remotely to the units to gather information.  It has a lot more options, and is a bit easier to show a picture. 

Its still a work in progress, but as you can see, I can already compare Sensor readings over time.  I can also specify how many entries to skip when graphing the data files, as there should be about 288 entries for a day, after which, the program creates a new file for that day.  Plotting 288 points can not only take a bit of resources, but also isn't needed for most things being looked at. So I set it at 12 by default, so it shows a reading every hour.  

All these separate systems, apps and scripts connect through a mini router I got for $35 that's powered by USB.  This enables me to take the E-Ink pi and the mini router, power them both up off a USB battery Bank and wait about 2 min for the Units to boot and the Sensor Units to auto connect to WiFi, and I can then pull data remotely from the units, up to however far the router's and pi's range are (50-100 meters?).  Great for putting Sensors on tree's, in water, down a hole, etc, etc.  

I also enabled a OpenVPN server on the router itself, allowing me to connect through the internet and grab the sensor data with my PC program.  This can be done, even from remote sites through 3G or 4G connectivity (AKA cell phone service). Simply add a USB dongle for 3G / 4G to the router, and BAM, I can suddenly connect a lot of units, within a decent range to the internet for remote monitoring and maintenance!  

That's pretty much were I am at now.  I'm also working on other aspects like Pi Cases for weather proofing or even water and underwater deployment.  High altitude balloons are also in the future.  

I think that about does it for this post.  I'll eventually post my Python code too, but I wanted to clean it up and add some comments to explain things first (being my first Python project and all). 

 

 

A Raspberry Pi 3B+ Python Project!

I now have a use for this domain 🙂  My Python + Raspberry Pi development. 

So, about a month ago, I got a few raspberry pi 3B+ with 2x senseHAT's.  Long story short, the senseHAT is an attachment that is loaded with sensors, a 5 'button' joystick and 8×8 LED grid.  By making your own programs, you can access the hardware and get readings or have the LED's light up in any way you like, including animations.  

The raspberry pi peaked my interest for a few reasons.  

1.  They are pretty cheap at about $50 Canadian.

2.  They are small and power efficient.  AKA I can power them for a long time (few days) off a USB battery bank.  

3.  Decent support to help learn programming.  Check out their website www.raspberrypi.org

However, I was more interested in what kind of environmental data I could digitize through the sensors.  AKA more accurate local weather type information.  Before I could start however, I needed to think of how I could get and store information from the sensors and ways I could manipulate that data to find cool patterns (like when its the coldest time of day, and how and when that changes over days, months, years, etc).  

I have tried a few times to learn programing (C++, Java), but ultimately failed the self taught way, due to a lack of direction to assist with learning (AKA motivation).  But now, for about $150, I can get the pi + senseHAT to do a lot! 

I have actually come pretty far already considering I only got them a month ago, but I'll post about that later.  

This site will be active! 

w00t!