INSTAR Deutschland GmbH

FHEM with the INSTAR MQTT Server



Just to get a quick overview over the FHEM user interface, I am going to install inside Raspbian on a Raspberry Pi 2 - if you are using a different operating system, please take a look at the installation guide.

FHEM Installation

  • Import repository gpg key: sudo wget http://debian.fhem.de/archive.key | apt-key add -

  • Add repository to sudo nano /etc/apt/sources.list: deb http://debian.fhem.de/nightly/ /

  • Update your package administration: sudo apt-get update

  • Install fhem: sudo apt-get install fhem

  • Access the web user interface on the IP of your FHEM server - e.g. 192.168.2.64 : http://192.168.2.64:8083/fhem

Make FHEM look Pretty(ier)

The FHEM UI does not look so good. Fortunately it is fully customizable - you can even download ready-to-use themes, like the haus-automatisierung.com skin, that I modified slightly for this tutorial. Skins don't change the button layout of the user interface and everything is purely cosmetic - so there is no need to install it. But if you want to follow along, download the either the original skin or the INSTAR modified version:

  1. Copy the content of the www directory from the is repository to the www directory inside your FHEM installation.
  2. Access your FHEM user interface and click on Unsorted and select the FHEMWEB / Web device:

INSTAR FHEM Style

  1. Select the stylesheetPrefix Attribute and type in instar_theme (or hausautomatisierung_com) and confirm. Reload the website to see the result:

INSTAR FHEM Style








The IPCAM Module

Adding your INSTAR HD or Full HD Camera

IPCAM enables the event- or time-controlled video recording in INSTAR network cameras. It is also possible to control the camera in PTZ mode or via CGI commands. We can start by defining a network camera device to trigger snapshots on events:

define <name> IPCAM <ip[:port]> 

FHEM Raspberry Pi

As an example we can add a local INSTAR Full HD camera that takes 5 snapshots with 10 seconds delay per call. First we need to define our camera - we choose a camera running on IP 192.168.2.116 and port 80 (if it is not 80 you have to add it to your camera's IP like: 192.168.178.21:8081) and gave it the name OfficeCam

define OfficeCam IPCAM  192.168.2.116:80

We can refer to the Command Reference for available attributes that we can assign to our camera.

This configuration uses a credentials file ipcams.conf located in /opt/fhem/credentials (make sure to sudo chown fhem:dialout and sudo chmod 664 both the folders (credentials & snapshots) and the config file !)

$credentials{OfficeCam}{username} = "admin";
$credentials{OfficeCam}{password} = "instar";

FHEM Raspberry Pi

Timed snapshots

attr OfficeCam basicauth {USERNAME}:{PASSWORD}
attr OfficeCam credentials /opt/fhem/credentials/ipcams.conf
attr OfficeCam path tmpfs/snap.jpg?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam snapshots 1
attr OfficeCam storage /opt/fhem/www/snapshots
attr OfficeCam event-on-update-reading snapshots
attr OfficeCam alias IN-8015 Full HD
attr OfficeCam group Surveillance
attr OfficeCam icon it_camera
attr OfficeCam room Office
attr OfficeCam verbose 5

FHEM Raspberry Pi

The attribute snapshot (we choose 1) defines the number of snapshots that should be taken when our IPCAM device is triggered by an event. If you choose more than 1 snapshot, you can also define a attr OfficeCam delay 10 in seconds between each shot. We also set the storage path on our Raspberry Pi to /opt/fhem/www/snapshots (make sure that directory exist and can be written to by the FHEM user) and told the IPCAM module that those snapshots can be found under tmpfs/snap.jpg?user={USERNAME}&pwd={PASSWORD} - the 1080p snapshot path of every INSTAR HD and Full HD camera.

Note: In most cases it won't be necessary to load the 1920x1080 pixel image - that is why your INSTAR Full HD camera offers 3 resolutions that you can pick from:

  1. /tmpfs/snap.jpg?usr=admin&pwd=instar Snapshot (1080p)
  2. /tmpfs/auto.jpg?usr=admin&pwd=instar Snapshot (320p)
  3. /tmpfs/auto2.jpg?usr=admin&pwd=instar Snapshot (160p)

To be able to access those snapshots we can use the integrated webserver inside FHEM called HTTPSRV and tell it to serve the content of the storage directory define <name> HTTPSRV <infix> <directory> <friendlyname>:

  • name is a custom name for the device used by the FHEM UI.
  • infix a prefix for the FHEMWEB-Base-URL under which the content of the directory will be served.
  • directory the absolute path to the public directory on your RaspPi.
  • friendlyname the name that is used in the FHEM side navigation.
define SnapServer HTTPSRV snaps /opt/fhem/www/snapshots OfficeCam-Snapshots

You can test the server by uploading an image, e.g. A19041608083210.jpg, and accessing the URL http://192.168.2.64:8083/fhem/snaps/A19041608083210.jpg with your web browser.

We can now define a job that retrieves those snapshots for us once every 10min:

define OfficeCam_Timelapse at +*00:01:00 get OfficeCam image

FHEM Raspberry Pi

The timelapse job should now trigger the IPCAM module which will then upload images to our storage path on the RaspberryPi. Via the HTTPSRV module we are now able to load the latest snapshot with the following URL: http://192.168.2.64:8083/fhem/snaps/OfficeCam_snapshot.jpg.

FHEM Raspberry Pi

This image can be embedded inside a website. You can download this HTML code and unzip it to /opt/fhem/www/snapshots as an example. This page will be hosted under http://192.168.2.64:8083/fhem/snaps and show you the latest snapshot inside an jQuery lightbox:

FHEM Raspberry Pi

Camera Control

attr OfficeCam pathCmd param.cgi?
attr OfficeCam cmd01 cmd=pushhostalarm?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd02 cmd=setinfrared&-infraredstat=auto?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd03 cmd=setinfrared&-infraredstat=close?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd04 cmd=setmdattr&-name=1&-enable=0&cmd=setmdattr&-name=2&-enable=0&cmd=setmdattr&-name=3&-enable=0&cmd=setmdattr&-name=4&-enable=0?cmd=setpirattr&-pir_enable=0?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd05 cmd=setmdattr&-name=1&-enable=1&cmd=setmdattr&-name=2&-enable=1&cmd=setmdattr&-name=3&-enable=1&cmd=setmdattr&-name=4&-enable=1?cmd=setpirattr&-pir_enable=1?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd06 cmd=preset&-act=goto&-number=0?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd07 cmd=preset&-act=goto&-number=1?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd08 cmd=preset&-act=goto&-number=2?user={USERNAME}&pwd={PASSWORD}
attr OfficeCam cmd09 cmd=ptzctrl&-step=0&-act=hscan?user={USERNAME}&pwd={PASSWORD}







Adding your Cameras Live Video

You can use the IFRAME module to add the cameras live video:

define OfficeCamView weblink iframe http://192.168.2.116:80/mjpegstream.cgi?-chn=12&-usr=admin&-pwd=instar
attr OfficeCamView htmlattr width="640" height="352" frameborder="0" marginheight="0" marginwidth="0"
attr OfficeCamView group Surveillance
attr OfficeCamView room Office
attr OfficeCamView verbose 5

Just as it was the case with the snapshots, you can pick for 3 different resolutions for the live video stream - just substitute the channel number in the MJPEG path above:

  1. /mjpegstream.cgi?-chn=11 (1080p)
  2. /mjpegstream.cgi?-chn=12 (320p)
  3. /mjpegstream.cgi?-chn=11 (160p)

To edit the this view once added it use the following URL http://192.168.2.64:8083/fhem?detail=OfficeCamView where you have to replace the OfficeCamView with the name you choose in the commands above.

FHEM Raspberry Pi








Motion Detection and Notification

In this tutorial step we want to use our cameras Motion Detection to trigger an FTP Upload of a Snapshot to the Raspberry Pi running FHEM. We will need to install a FTP Server and implement a way to notify FHEM every time that a new file was uploaded.

Adding a FTP Server

We are going to use ProFTPD that can be installed on our RasPi with the following command:

sudo apt update
sudo apt install proftpd-basic

FHEM Raspberry Pi

You might have been asked to choose between a standalone server mode or from inetd during the installation - choose the standalone mode. To configure ProFTPD manually open the configuration file in Nano:

sudo nano /etc/proftpd/proftpd.conf

We are deactivating IPv6 - that we don't need, renaming our server to FHEM and making sure that the standalone server type is active.

FHEM Raspberry Pi

Now we need to make some changes to our user configuration for the default ProFTPd user ftpuser by creating the following file:

sudo nano /etc/proftpd/conf.d/proftp-custom.conf

And adding the following text:

# Ftp user doesn't need a valid shell
<Global>
    RequireValidShell off
</Global>
 
# Default directory is ftpusers home
DefaultRoot ~ ftpuser
 
# Limit login to the ftpuser group
<Limit LOGIN>
    DenyGroup !ftpuser
</Limit>

FHEM Raspberry Pi

With the user configuration done we now have to add the ftpuser to our Linux system:

sudo adduser ftpuser --shell /bin/false --home /var/www/upload

We make sure that the ftpuser cannot be used to access our server by SSH --shell /bin/false and uses the home directory that was created by ProFTPd --home /var/www/upload.

FHEM Raspberry Pi

In this example I have chosen the UNIX password to be instar - if you choose a different password you will have to substitute instar with it in the following steps.

Now with everything set we just have to restart the FTP service:

sudo service proftpd restart

We can use our browser to test that the FTP server is running by accessing the IP address of our Raspberry Pi prefixed by an ftp://: ftp://192.168.2.64:21/:

FHEM Raspberry Pi

You should now be able to login with username = ftpuser and password = instar - of course we only see an empty directory here for now.

FHEM Raspberry Pi

  • r: Read
  • w: Write
  • x: eXecute

Right now only the ftpuser can read and write files to the ProFTPd home directory - which can be a little bit frustrating some times when we need to add a file via the fhem or default pi user (that only have the right to read). We can change the rights recursively for the directory with the following command:

sudo chmod -R 666 /var/www/upload/

FHEM Raspberry Pi

User Group Global
rw- rw- rw-

With 666 the owner (ftpuser) it's group as well as every global user gets the rights to read, write but not execute rw-

Binary Octal Permission
000 0
001 1 –x
010 2 -w-
011 3 -wx
100 4 r–
101 5 r-x
110 6 rw-
111 7 rwx

We can now create a folder that we want to use as upload folder for our office camera:

sudo mkdir officecam
sudo chown ftpuser:ftpuser officecam/
sudo chmod -R 666 /var/www/upload/officecam/







Camera FTP Upload

FHEM Raspberry Pi

We can now configure our INSTAR camera to upload files to the FTP server inside the FTP Menu. Use the IP address of your Raspberry Pi, the port 21 and type in the folder that you created inside the ftpuser home directory as storage path (prepend ./). The login is the ftpuser login - username = ftpuser and password = instar. You will see that the test image was successfully uploaded to the officecam directory:

FHEM Raspberry Pi

Now all we have to do is to activate the alarm FTP upload in case of an alarm inside the Alarm Actions:

FHEM Raspberry Pi








Configuring FHEM

We will start by creating a dummy device for the Alarm Events in FHEM and assigning it to our Office room:

define OfficeCam.Alarm dummy

FHEM Raspberry Pi

We now can use the event-on-update-reading attribute to trigger an event every time a new snapshot is uploaded by our camera:

FHEM Raspberry Pi

The variable our device is now watching is new_snapshot which we can set by the following command: setreading OfficeCam.Alarm new_snapshot test:

FHEM Raspberry Pi

But FHEM also allows us to send commands by HTTP through our web browser (or by a curl command on your Linux terminal). The corresponding URL looks like this http://192.168.2.64:8083/fhem?cmd=setreading%20OfficeCam.Alarm%20new_snapshot%20test2&XHR=1:

FHEM Raspberry Pi

Note if calling the URL does not work and results in an HTTP ERROR 400, run the following command in the FHEM command line attr WEB csrfToken none and try again. See the FHEM Forum for more details.

FHEM Raspberry Pi








Creating a Watch Daemon

To notify FHEM when a snapshot is uploaded we need to install the inotify tools and create a shell script inside the FHEM directory alarmwatch.sh. This script should run as a background service, always checking if a new file was created inside the watch folder and call the URL above with the name of the file as value (instead of test or test2):

sudo apt install inotify-tools
sudo nano /opt/fhem/alarmwatch.sh
sudo chmod 744 /opt/fhem/alarmwatch.sh

The script contains the following (s. below):

#!/bin/bash

cd /var/www/upload/officecam

# use inotify to watch multiple create events inside the camera upload folder
inotifywait -m -e create /var/www/upload/officecam |
while true; do
    while read $T path action file; do
        # cleanUp delete oldest file when 100 files are reached
        rm -f `ls -td -1 /var/www/upload/officecam/** | awk 'NR>100'`
        echo "'$file' was created in directory '$path' via '$action'"
        # Get FHEM Command URL to set new_snapshot to current file name
        curl "http://admin:instar@localhost:8083/fhem?cmd=setreading%20OfficeCam.Alarm%20new_snapshot%20$file&XHR=1"
    done
done

If you have chosen something different for any of those - please adjust the script accordingly:

  • Upload directory: /var/www/upload/officecam
  • FHEM Username: admin
  • FHEM Password: instar
  • FHEM Port: 8083
  • Alarm dummy device: OfficeCam.Alarm
  • Snapshot variable: new_snapshot

To be able to execute the script we need to change the owner from root to fhem:

sudo chown fhem:dialout alarmwatch.sh

FHEM Raspberry Pi

We can now start sh /opt/fhem/alarmwatch.sh the script manually and test if it is working by triggering an alarm on our camera:

FHEM Raspberry Pi

  1. The alarm is triggered
  2. Our camera uploads a single snapshot into the officecam directory
  3. inotify detects that a new file was created and triggers our script
  4. FHEM is contacted with the HTTP command and new_snapshot is set to the filename of the most current upload

We now have to create a background service that automatically starts our script as a system daemon. This can be done with SystemD - sudo nano /etc/systemd/system/OfficeCamWatch.service:

[Unit]
Description=OfficeCam Alarm Watch

[Service]
Type=oneshot
ExecStart=/opt/fhem/alarmwatch.sh

[Install]
WantedBy=multi-user.target

FHEM Raspberry Pi

Now we can reload SystemD and start our service:

sudo systemctl daemon-reload
sudo systemctl enable OfficeCamWatch.service
sudo systemctl start OfficeCamWatch.service

MQTT Integration

You can find an extended introduction on how to integrate your INSTAR Full HD (with integrated MATT Broker) into FHEM in the INSTAR MQTT Broker section. But the FHEM Installation there was based on the Windows version. There isn't much of a difference once you have FHEM up and running. But installing FHEM (as described above) as well as it's dependencies (see below) obviously differs.

MQTT Prerequisite

sudo apt-get install libmodule-pluggable-perl
sudo cpan install Net::MQTT:Simple
sudo cpan install Net::MQTT:Constants

Installation of Node-Red

sudo apt-get install build-essential

wget https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered



bash update-nodejs-and-nodered

FHEM Raspberry Pi

node-red-start
node-red-stop

sudo systemctl enable nodered.service
sudo systemctl disable nodered.service

sudo systemctl start nodered.service
sudo systemctl stop nodered.service

FHEM Raspberry Pi

You can now access the Node-RED interface through the following URL http://192.168.2.64:1880/ - where 192.168.2.64 has to be substituted by your server address (Raspberry Pi):

FHEM Raspberry Pi