Kootnet Sensors Updates

I have made a few updates to Kootnet Sensors now. Here’s the short list.

  • Created new index.html (main web page) – It’s now a very simple page
    • Does not require any special scripts
    • Contains quick basic info
      • Sensor ID
      • Version
      • Hostname
      • Local IP
      • Disk Space check
      • Sensor’s date/time in UTC0 and with its local TimeZone
      • Link to Main Web Interface
  • Added SQLite3 Database Management Page
    • View information on Main/MQTT/Sensors Checkin Databases
    • Download Main/MQTT/Sensors Checkin databases
    • Download or Delete backups of the Main Database
    • Upload Databases
      • Replace Main/MQTT/Sensors Checkin databases
        • Replacing the Main database creates a zipped backup of the old one
      • Upload Custom databases for Graphing
    • Manage Custom Databases
      • Rename, Delete or Shrink (Vacuum) Databases
  • Updated Graphing to work with other Databases
    • Can now graph Main or MQTT Databases
    • Can graph Custom uploaded databases (Main or MQTT)
  • Create and Manage multiple IP lists in Remote Management
  • Added /robots.txt route to discourage web crawlers when set to port 80
  • Created an “Online” installer that’s only 1.4MB compared to the Offline installer at 44.5MB. The Online installer is now used when upgrading.
  • Bunch of refactoring

Things appear to be working well, so I have uploaded it into the Dev channel and it’s probably safe to try out. I’ll be running more tests in a few days and then I might release it into the Stable channel.

Kootnet Sensors MQTT Update

I have updated MQTT publishing and subscribing. There was nothing ‘wrong’ with the publishing, but I have made it more efficient by grouping all the readings to be sent into a single list made of dictionaries and sending it all at once, instead of publishing each one individually.

The MQTT Subscriber on the other hand was dropping data a lot, some sensors would only give one or two readings and never anything else, while others only randomly showed readings. I managed to fix it by using callback access inside the Python MQTT library instead of a client connection. Now all the data comes through every time.

The next step will most likely be updating graphing to allow the use of the MQTT subscriber database. I also wanted to add the ability to upload different databases for use in graphing as well as adding the ability to save multiple IP lists in remote management.

The new version has been uploaded to the Dev channel and is ready to try, which is currently Beta.32.12.

Kootnet Sensors progress

I have finally started poking at programming again. To start, I have fixed a bug in Kootnet Sensors when updating configurations of the 3rd party services. After that, I added the ability to record MQTT subscriber data to a separate MQTT SQLite database. This means you can centralize your sensor recordings, AKA send multiple sensor readings from multiple Kootnet Sensors to a single MQTT broker, such as a Kootnet Sensor and have any Kootnet sensor subscribe to the broker and record all the readings. Since MQTT is being used to transmit data, the remote recording is very flexible, for instance, you can get all readings being sent to the MQTT broker or specify specific sensors and or readings. This also means you can have multiple sensors recording data from the MQTT broker, to have redundant copies of the data for backup purposes for example.

I decided to send data over MQTT as dictionaries in string format, this way the ‘name’ in the dictionary is used as the SQL database column and the data is the sensor reading, at least that’s what I have done with sensors that have more then one data point (Magnetometer for example). I’m still debating if I should do the same for the single readings, as I’m currently using the end MQTT topic as the column name for single data point sensors (the base topic being the sensors name/ID). I suppose leaving it as is will decrease the amount of data it needs to send … I’ll have to give it some more thought.

The program changes have been published to the development channel.

That’s it for now.

Remarkable 2 paper tablet came in!

I just got my new paper tablet yesterday! So far so good. It feels really nice to write on and shows I need to work on my writing skills… Setup and sync work well and is very easy to get working. Since I just got it, I’m going to use it for a while before I write anything more about it. I plan on doing a few mock-up designs for my Network Testers interface using it later today. I’m hoping to schedule some more programming work in the near future now that I have my fancy tablet to visualize what I want before I start programming it in.

Kootnet Network Testers 1.4.0

I have finally released the new Network Testers version. It’s a minor update but does add the ability to view mtr / iperf previous results separately, set tests to run at a specific date & time (one time) and makes the web interface more compatible with Firefox.

I’m kinda waiting for my Remarkable 2 to arrive, so I can do some mock-up designs and enjoy a new toy/work tool.

Stuff

So I have been taking a bit of a break from programming, but before I did, I actually modified my Network Testers to be able to view iPerf & MTR results separately, so it’s easier to find results, but I didn’t push it as an installer yet. I’m hoping to start programming again soon (a few days?), then make a few other changes before releasing a new Network Testers version. So be on the lookout for an updated release of Koonet Network Testers!

Kootnet Sensors Beta.31.107

The new Kootnet Sensors version is in the “Stable” channel!
Here is a list of some of the changes since Beta.30.x

  • Send Reports and or Graphs by email on a daily/Weekly/Monthly/Yearly basis
  • Configure or disable Sensor Checkins or have sensors checkin to a sensor with the Checkin Server enabled.
  • New Quick Graph type and other graph back end improvements
  • Refined and enhanced view of per-sensor recording status
  • New High/Low SQL recording
  • Updated Tiny editor (note-taking section)
  • Updated Remote Sensor Management section (Sensor Control)
  • Web Portal updates for compatibility with mobile devices & Firefox
  • New Web Portal menu layout
  • New hardware sensor support
  • Added additional configuration options
  • Included base Python modules for spotty internet connections & server connection issues (Should improve install reliability and speed)
  • New Documentation / Help file (WIP)
  • Updated and enhanced GUI tests
  • Misc. bug fixes and performance enhancements

With this version out, I think I’ll work on the Network Testers.

New Sensors ordered!

So there appears to be a few new sensors on Pimoroni, including a GPS one! So I’m looking forward to adding that to the supported sensor list, that way you can record the exact location of where you are recording!

I kinda wish I had a bit of support money coming in for buying hardware since it was over $400 Canadian (245 Euros) to get the new sensors and I’m pretty sure I’ll still have to pay import fees. Guess that’s how it goes sometimes. If I could just find a good place where people would actually find this useful and give feedback on what to add to make it more useful, that would raise the chances of someone donating. If anyone is reading this and knows of such places (online forums, etc), please post a comment and let me know!

I also ordered a few new displays in hopes to add further display support to my Network Testers, of which I will be working on after Kootnet Sensors ver Beta.31.x is released.

I have finished tweaking the Web Portal on Kootnet Sensors so you can see and or scroll to all the sections on a mobile phone. It doesn’t look good in a lot of places, but at least it’s functional. I have also finished the tests section, so it’s fully operational but it could still use improvements in how and where it tests.

Another day or two and Beta.31.x should be out.

Clean up & enhanced tests

I have been going through some code and doing some clean ups. While I was looking around, I noticed the tests program needed some updates. I have added missing configurations to test and made it so when sensors are tested, it displays their results for verification. It should be finished up by tomorrow.

Another place I have to double check is the Web Portal on a mobile device. I remember there were a few things that didn’t size nicely on my phone, so I’ll see what I can do there.

Things are coming along nicely, although once I release Beta.31.x, I’ll probably take a break from the sensors for a bit and maybe look over the Network testers and tweak things there and add a new test or two.

I hope everyone had a good Thanks giving!

Kootnet Sensors Ver. Beta.31.x soon…

I have now finished updating the documentation and creating links inside the program that take you directly to the relevant section in the help document. Because the help document is actually located on the sensor itself, you’ll always have the manual available.

I have also fixed a bunch of oddities so it’s more compatible with the Firefox web browser. The logout should work properly and the formatting layout should not be overlapping anymore.

I have also enhanced the security a bit, by limiting access to certain functions, yet maintaining the ability to get non-sensitive information. When viewing the raw configurations, even when logged in, the usernames and passwords have been removed for viewing.

Testing is the main thing left to do, such as new installs, upgrade installs and reviewing functions. Next week will most likely be when I release this into the stable channel. If you want a sneak peak, upgrade to the Dev build from the Sensors Web Portal, I don’t expect there to be too many bugs right now.