I now have all the pages working on the new interface except for Graphing and Remote Management.

The next step, after finishing Graphing, Remote Management and the notification system, will be to look over the interface as a whole and tweak things to be more consistent and adjust a few minor things, like the size of text and number fields.

I’m also thinking I’ll remove the old interface altogether, as it’s adding a decent amount of bulk to the overall program.

I have not had as much time as I would have liked to work on it, but progress is continuing! Very little back-end work has been done since the last stable release, so it’s safe to upgrade to dev and check out the interface progress.

Since Kootnet Sensors has a decent amount of functionality, I thought it was time to re-visit the web interface itself. When I first made it, I really didn’t know much about site design and it shows, both visually and in the code. So now I’m remaking it for a more modern look.

After looking around for a template to use, I decided on AT Pro Admin, which is an MIT licensed template that uses pure HTML, CSS and JS. It’s small in size, quick to load and has a sweet dark/light theme toggle. It also comes with and uses Font Awesome, which I have not heard of before but I am super glad I have now! It’s loaded with very professional icons for pretty much everything. I’m going to buy the pro version to support them, even if I’m not actually going to use any pro icons.

I have already customized the template layout and added a few pages. It’s looking pretty sweet so far. I have also found a way to only load the main page once, then when clicking links, it only loads those specific pages inside the main interface. I can already notice the load time differences compared with my old interface. I’m trying to limit the required webpage plugins to keep download size low and speed high.

I’m actually not sure how long it will take to do the web interface overhaul (month?) but I have a link to both the old and new interface on the index page, so if you upgrade to the dev version, you can check it out without worrying about losing access to settings and pages. So far the layout is pretty much done the main dashboard page work along with live sensor readings and notes. I still have to customize the notes section a bit too.

So feel free to check out the latest dev version to get a sneak preview. Once I have the interface mostly done, I’ll probably replace the “Quick Graph” section with “Live Graphs” to constantly update graphs based on local or remote sensor readings. I have been having trouble getting the graph to actually graph updates, but I have already figured out how to get javascript to add new readings.

I’ll try and update the blog when I’m getting closer to completing the new interface.

I have been doing a bit of testing today, and so far, things are working great, especially around graphing through the MQTT Subscriber database (AKA using recorded data from remote sensors). New and upgrade installs have been tested and seem good as well.

So out with Beta.32.64 as the new Stable version!

Be sure to check out the improved MQTT Publisher & Subscriber as both have been enhanced decently. There’s a good chance you’ll be able to configure MQTT Publisher to send your readings to almost any sensor collecting website that supports MQTT.

If anyone is trying it, let me know if you come across any bugs!

Find installers here or if you are already using Kootnet Sensors, goto the advanced section and upgrade to the latest stable version.

P.S. Please note, due to changes to the MQTT Publisher system, your MQTT Publisher configuration will be reset to defaults.

Development version Beta.32.x is getting closer to being ready for the Stable channel. I have added some final touches around the MQTT Publisher, such as fixing some things around changing settings and adding a page to view the replacement variables for a custom string.

I have also gone through a bunch of my SQLite database code and optimized it for performance and clarity. Two particular places of improvement are the Notes section and the Sensor Checkins page, both have a significant boost in performance.

Last but maybe not least, I updated the documentation, but not the pictures in it yet … I wonder if I should bother with pictures… it’s a pretty big time hole to get new screenshots, so maybe I’ll remove pictures from a few sections.

Anywho, I’ll probably be doing testing for the most part then out with the new release!

I have revisited the MQTT Publisher and Subscriber again. I have created 3 different “types” or “ways” to send data through MQTT Publisher. The 3 types are as follows

1. Kootnet Sensor Format – Send a dictionary of all selected and available readings. Each index of the dictionary is specific to Kootnet Sensors database column names and used when writing data to the MQTT Subscriber SQLite database.
2. Topic Per Sensor – This sends each sensor reading to a specific topic, including readings with multiple variables, such as the accelerometer. So there is a separate topic for accelerometer X, Y and Z for example.
3. Custom Single String – A text string is provided by the user, of which, specially formated text inside it is replaced with actual sensor values, then sent to the base MQTT topic.

Type 1 and 2 are fairly straight forward, but type 3 deserves a bit more explination. Here is an example of a custom string.

{“DeviceID”: {{ SystemHostName }}, “Temperature”: {{ EnvTemp }}, “Humidity”: {{ Humidity }}}

In this example, a JSON (or Python dictionary) string is sent. However, all text with {{ Something }} is replaced with actual values. So if this was to be sent, it might look like this to the MQTT broker.

{“DeviceID”: MySensorName, “Temperature”: 45.23, “Humidity”: 22.5}

There will be a link above the MQTT Custom string text area to give a list of all the available text that gets replaced by sensor data. Using the MQTT Publisher custom string should allow Kootnet Sensors to work with almost all MQTT brokers being used for sensor readings collection. The MQTT Subscriber has also been updated to work with the new Publisher send types.

I also did some other misc. updates and fixes to things like Plotly graphing and the GUI test program. I need to work on updating the documentation, doing some more tests then I’ll release it to the Stable channel. After that, I kinda want to do some courses on web design, specifically HTML, CSS and javascript. I’m hoping to not only make the web interface look better, but I also want to cut down on the required 3rd party scripts to improve load time and data transmission. It’s also fairly messy… so cleaning it up would be a bounus.

This new version is in the Dev channel and seems stable, so it should be safe to try out, just be warned, the MQTT Publisher configuration gets reset on this upgrade.

I have made a few updates to Kootnet Sensors now. Here’s the short list.

• Created new index.html (main web page) – It’s now a very simple page
  • Does not require any special scripts
  • Contains quick basic info
    • Sensor ID
    • Version
    • Hostname
    • Local IP
    • Disk Space check
    • Sensor’s date/time in UTC0 and with its local TimeZone
    • Link to Main Web Interface
• Added SQLite3 Database Management Page
  • View information on Main/MQTT/Sensors Checkin Databases
  • Download Main/MQTT/Sensors Checkin databases
  • Download or Delete backups of the Main Database
  • Upload Databases
    • Replace Main/MQTT/Sensors Checkin databases
      • Replacing the Main database creates a zipped backup of the old one
    • Upload Custom databases for Graphing
  • Manage Custom Databases
    • Rename, Delete or Shrink (Vacuum) Databases
• Updated Graphing to work with other Databases
  • Can now graph Main or MQTT Databases
  • Can graph Custom uploaded databases (Main or MQTT)
• Create and Manage multiple IP lists in Remote Management
• Added /robots.txt route to discourage web crawlers when set to port 80
• Created an “Online” installer that’s only 1.4MB compared to the Offline installer at 44.5MB. The Online installer is now used when upgrading.
• Bunch of refactoring

Things appear to be working well, so I have uploaded it into the Dev channel and it’s probably safe to try out. I’ll be running more tests in a few days and then I might release it into the Stable channel.

I have updated MQTT publishing and subscribing. There was nothing ‘wrong’ with the publishing, but I have made it more efficient by grouping all the readings to be sent into a single list made of dictionaries and sending it all at once, instead of publishing each one individually.

The MQTT Subscriber on the other hand was dropping data a lot, some sensors would only give one or two readings and never anything else, while others only randomly showed readings. I managed to fix it by using callback access inside the Python MQTT library instead of a client connection. Now all the data comes through every time.

The next step will most likely be updating graphing to allow the use of the MQTT subscriber database. I also wanted to add the ability to upload different databases for use in graphing as well as adding the ability to save multiple IP lists in remote management.

The new version has been uploaded to the Dev channel and is ready to try, which is currently Beta.32.12.

I have finally started poking at programming again. To start, I have fixed a bug in Kootnet Sensors when updating configurations of the 3rd party services. After that, I added the ability to record MQTT subscriber data to a separate MQTT SQLite database. This means you can centralize your sensor recordings, AKA send multiple sensor readings from multiple Kootnet Sensors to a single MQTT broker, such as a Kootnet Sensor and have any Kootnet sensor subscribe to the broker and record all the readings. Since MQTT is being used to transmit data, the remote recording is very flexible, for instance, you can get all readings being sent to the MQTT broker or specify specific sensors and or readings. This also means you can have multiple sensors recording data from the MQTT broker, to have redundant copies of the data for backup purposes for example.

I decided to send data over MQTT as dictionaries in string format, this way the ‘name’ in the dictionary is used as the SQL database column and the data is the sensor reading, at least that’s what I have done with sensors that have more then one data point (Magnetometer for example). I’m still debating if I should do the same for the single readings, as I’m currently using the end MQTT topic as the column name for single data point sensors (the base topic being the sensors name/ID). I suppose leaving it as is will decrease the amount of data it needs to send … I’ll have to give it some more thought.

The program changes have been published to the development channel.

That’s it for now.

I just got my new paper tablet yesterday! So far so good. It feels really nice to write on and shows I need to work on my writing skills… Setup and sync work well and is very easy to get working. Since I just got it, I’m going to use it for a while before I write anything more about it. I plan on doing a few mock-up designs for my Network Testers interface using it later today. I’m hoping to schedule some more programming work in the near future now that I have my fancy tablet to visualize what I want before I start programming it in.

I have finally released the new Network Testers version. It’s a minor update but does add the ability to view mtr / iperf previous results separately, set tests to run at a specific date & time (one time) and makes the web interface more compatible with Firefox.

I’m kinda waiting for my Remarkable 2 to arrive, so I can do some mock-up designs and enjoy a new toy/work tool.