WWW.DUMAIS.IO
ARTICLES
OVERLAY NETWORKS WITH MY SDN CONTROLLERSIMPLE LEARNING SWITCH WITH OPENFLOWINSTALLING KUBERNETES MANUALLYWRITING A HYPERVISOR WITH INTEL VT-X CREATING YOUR OWN LINUX CONTAINERSVIRTIO DRIVER IMPLEMENTATIONNETWORKING IN MY OSESP8266 BASED IRRIGATION CONTROLLERLED STRIP CONTROLLER USING ESP8266.OPENVSWITCH ON SLACKWARESHA256 ASSEMBLY IMPLEMENTATIONPROCESS CONTEXT ID AND THE TLBTHREAD MANAGEMENT IN MY HOBBY OSENABLING MULTI-PROCESSORS IN MY HOBBY OSNEW HOME AUTOMATION SYSTEMINSTALLING AND USING DOCKER ON SLACKWARESYSTEM ON A CHIP EMULATORUSING JSSIP AND ASTERISK TO MAKE A WEBPHONEC++ WEBSOCKET SERVERSIP ATTACK BANNINGBLOCK CACHING AND WRITEBACKBEAGLEBONE BLACK BARE METAL DEVELOPEMENTARM BARE METAL DEVELOPMENTUSING EPOLLMEMORY PAGINGIMPLEMENTING HTTP DIGEST AUTHENTICATIONSTACK FRAME AND THE RED ZONE (X86_64)AVX/SSE AND CONTEXT SWITCHINGHOW TO ANSWER A QUESTION THE SMART WAY.REALTEK 8139 NETWORK CARD DRIVERREST INTERFACE ENGINECISCO 1760 AS AN FXS GATEWAYHOME AUTOMATION SYSTEMEZFLORA IRRIGATION SYSTEMSUMP PUMP MONITORINGBUILDING A HOSTED MAILSERVER SERVICEI AM NOW HOSTING MY OWN DNS AND MAIL SERVERS ON AMAZON EC2DEPLOYING A LAYER3 SWITCH ON MY NETWORKACD SERVER WITH RESIPROCATEC++ JSON LIBRARYIMPLEMENTING YOUR OWN MUTEX WITH CMPXCHGWAKEUPCALL SERVER USING RESIPROCATEFFT ON AMD64CLONING A HARD DRIVECONFIGURING AND USING KVM-QEMUUSING COUCHDBINSTALLING COUCHDB ON SLACKWARENGW100 MY OS AND EDXS/LSENGW100 - MY OSASTERISK FILTER APPLICATIONCISCO ROUTER CONFIGURATIONAASTRA 411 XML APPLICATIONSPA941 PHONEBOOKSPEEDTOUCH 780 DOCUMENTATIONAASTRA CONTACT LIST XML APPLICATIONAVR32 OS FOR NGW100ASTERISK SOUND INJECTION APPLICATIONNGW100 - DIFFERENT PROBLEMS AND SOLUTIONSAASTRA PRIME RATE XML APPLICATIONSPEEDTOUCH 780 CONFIGURATIONUSING COUCHDB WITH PHPAVR32 ASSEMBLY TIPAP7000 AND NGW100 ARCHITECTUREAASTRA WEATHER XML APPLICATIONNGW100 - GETTING STARTEDAASTRA ALI XML APPLICATION

INSTALLING AND USING DOCKER ON SLACKWARE

2015-06-06

Docker

Imagine you want an asterisk system complete with mariadb and apache but you don't want to install all that on your day-to-day system. You could create a VM. but vms are heavy and have quite some overhead. What about backups? you can't copy live running image (not with qemu at least).

Enter Docker. Docker makes this really simple. Docker encloses an environment like a chroot would. But it acts more like a vm. With docker, the equivalent of a vm is a container. You create your filesystem and install the needed software in your container and run it. But Docker lets you run 1 command only. You start the container, the command executes, in its environment, and the container stops. All this using the host's kernel but separated with namespaces and cgroups. Nothing prevents you from running a script as the command. So you make a script that starts httpd and asterisk then load bash. the container will run as long as bash doesn't exit. so if you attach your session on the container and "exit" bash, the container will stop amd asterisk and httpd will shutdown. The container does not start "init" (unless it's the command you chose to invoke) so it's not like an entire OS is brought up. Only the command you run is ran in the container's FS but with the host's kernel

Docker allows you export running containers to a tarball. the tarball contains the entire FS of the container (not the memory) You can then import it anywhere where Docker runs.

Installing docker on slackware

These are the instructions for installing docker 1.7.0 on slackware 14.1 with kernel 3.11.1. I also tried with slackware 14.0 but the rc.S script does not mount the cgroup hierarchy properly So I modified it to do it like in 14.1 I could not get it to work with kernel 3.10.17 and did not bother to troubleshoot since I had a 3.11.1 on hand. 1.7.0-rc1 did not work for me. I needed commit 6cdf8623d52e7e4c5b5265deb5f5b1d33f2e6e95 in. So I cloned the bleeding edge from git but then 2h later rc2 came out.

Pre-requisites

Before anything, you should make sure that your kernel is compiled with

  • CONFIG_DM_THIN_PROVISIONING
  • CONFIG_BRIDGE
  • CONFIG_BRIDGE_NETFILTER
  • CONFIG_MEMCG
  • CGROUP_DEVICE
  • CONFIG_NF_NAT_IPV4
  • NF_NAT_IPV4
  • IP_NF_TARGET_MASQUERADE
  • IP_NF_TARGET_NETMAP
  • CONFIG_IP_NF_TARGET_REDIRECT
  • CONFIG_DEVPTS_MULTIPLE_INSTANCES
  • CONFIG_MACVLAN
  • CONFIG_VETH
  • CONFIG_NETFILTER_XT_MATCH_ADDRTYPE
  • CONFIG_BLK_CGROUP
  • CONFIG_CGROUP_PERF
  • CONFIG_CFS_BANDWIDTH
  • CONFIG_EXT4_FS_POSIX_ACL
  • CONFIG_EXT4_FS_SECURITY
  • BLK_DEV_THROTTLING

I actually made a script to enable all those settings

#!/bin/sh CFG=("CONFIG_DM_THIN_PROVISIONING" \ "CONFIG_BRIDGE" \ "CONFIG_BRIDGE_NETFILTER" \ "CONFIG_MEMCG" \ "CGROUP_DEVICE" \ "CONFIG_NF_NAT_IPV4" \ "NF_NAT_IPV4" \ "IP_NF_TARGET_MASQUERADE" \ "IP_NF_TARGET_NETMAP" \ "CONFIG_IP_NF_TARGET_REDIRECT" \ "CONFIG_DEVPTS_MULTIPLE_INSTANCES" \ "CONFIG_MACVLAN" \ "CONFIG_VETH" \ "CONFIG_NETFILTER_XT_MATCH_ADDRTYPE" \ "CONFIG_BLK_CGROUP" \ "CONFIG_CGROUP_PERF" \ "CONFIG_CFS_BANDWIDTH" \ "CONFIG_EXT4_FS_POSIX_ACL" \ "CONFIG_EXT4_FS_SECURITY" \ "BLK_DEV_THROTTLING"); for i in ${CFG[*]}; do sed -i "/^# *$i/c\\$i=y" $1 done

Maybe some other flags need to be set on your kernel, but these were all the ones I was missing. There is a uitlity that you can download to check those settings: https://github.com/docker/docker/blob/master/contrib/check-config.sh

Download, compile, and prepare the environment

At first, I tried downloading the binaries but docker was complaining about "Udev sync is not supported". I found out that was because the binary is statically link and it causes some problems that I didn't care to look into. So I opted for building from source. The first step is to get "go". I didn't want to leave this on my system so I just put it in a temporary place and then deleted it.

wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz tar -zxf go1.4.2.linux-amd64.tar.gz mv go /opt #You should make that permanent if you intend to keep Go after building Docker PATH=$PATH:/opt/go/bin GOPATH=/opt/go:/opt/go/src/github.com/docker/docker/vendor # download docker source. wget https://github.com/docker/docker/archive/v1.7.0-rc2.tar.gz tar -zxf v1.7.0-rc2.tar.gz cd docker-1.7.0-rc2 #docker won't build because there is a header file that won't be found. There is a patch #for that but let's do it manually here: sed -i "/ioctl\.h/c\#include \r\n#include \r\n#include \r\n#include \r\n#include \r\n" daemon/graphdriver/btrfs/btrfs.go #note that DOCKER_GIT_COMMIT needs to match the version you have downloaded GOROOT=/opt/go AUTO_GOPATH=1 DOCKER_GITCOMMIT="395cced " ./hack/make.sh dynbinary cp bundles/1.7.0-rc2/dynbinary/docker-1.7.0-rc2 /usr/sbin/docker cp bundles/1.7.0-rc2/dynbinary/dockerinit-1.7.0-rc2 /usr/sbin/dockerinit #remove go... or not. It's up to you. If you leave it there, you might want to permanently add it to your PATH rm -Rf /opt/go

Prepare network bridge

In my case, since I was already using KVM/qemu, I had a bridge alrady setup. But this is what would be needed

#create bridge brctl addbr br0 ifconfig eth0 down ifconfig br0 192.168.1.253 netmask 255.255.255.0 broadcast 192.168.1.255 up # add eth0 as member of the bridge and bring it up. brctl stp br0 off brctl setfd br0 1 brctl sethello br0 1 brctl addif br0 eth0 ifconfig eth0 0.0.0.0 promisc up # setup default gateway. route add default gw 192.168.1.1

You might want that last example to run at boot time. There is a way to setup a bridge with the init scripts but I just added those lines in rc.local before launching the docker daemon.

There is currently no easy way to assign a static IP to your container. Docker will choose an IP in the range of you bridge. But this isn't perfect. It seems to pick some addresses that are already used on my network. Issue 6743 on githubis opened for that. But for the time being, I've hacked the code to make this possible. I won't create a pull request since they are already working on a more elegant solution. But meanwhile, you can download my fork if you need it. My fork on github.. That repo contains the patch to build on slackware and also adds a "--ipv4-adress=A.B.C.D" option to docker run.

Auto start

Finally, you should add this in your rc.local script.

/usr/sbin/docker -d -b br0&

Building a container, running it and doing backups

My use case

What I'm looking to do is to isolate my home automation services in one container that I can easily transport from one computer to another (or even to a VM). In case I get a hardware failure, I want to reduce the downtime of my house services. Those include Asterisk, httpd, MariadDB, CouchDB, DHAS, cron jobs, and some more. I want to be able to make a daily backup of the container and always be able to launch it from somehwere else where Docker is installed.

Creating a container

docker run vbatts/slackware:14.1 -ti /bin/bash

That command will create a container from a base image "slackware 14.1" from docker hub. The image will be downloaded automatically. Then bash will be invoked. the -t -i flags will give you an interactive TTY attached so you will be able to interact with bash. From there, install whatever you need in the container. Download gcc, install it, etc. Once you are done, exit bash. By exiting bash, the container stops. You can now commit your changes to a new base image

docker commit ContainerID awesomeNewImage

Now you have a base image of your own that you can share with other people. Now create another container for your real use case from the base image.

docker run -tid --restart=always awesomeNewImage /root/start.sh

The -d flag runs the contain in the background. You can access it using "docker attach". --restart=always will make the container restart automatically when you exit it and when the docker daemon starts (after a host reboot for example). When you were setting up you base image, you could have created a start.sh script that invokes asterisk, httpd, mysqld and couchdb then bash. To detach from the container without stopping it (leaving your command running) you can Ctrl-P, Ctrl-Q.

Backup/Restore

An easy way to backup is to regularly export the container (Cron job?).

docker export -o backup.tar

This will create a tarball container the entire filesystem of your container. Then to restore, either locally or on some other machine runner docker:

cat backup.tar | docker import - restoredbackup

SYSTEM ON A CHIP EMULATOR

2015-04-09

I was looking at some online NES emulators the other day and it gave me an idea. What If I created my own emulator but for my own architecture instead of emulating an existing one? Of course that would be easier because I can decide what the architecture looks like instead of having to go through the specs of the platform I want to emulate. So I decided to create my own instructions with an assembler and a disassembler. The assembler converts the assembly code to a binary format (obviously) and the result can be downloaded. Again, this is my own virtual assembly and my own architecture so the instructions are encoded in my own format. Each instructions are (inneficiently)encoded on 64bit including a references to 2 operands, 1 immediate 32bit value and a condition code. Each instruction is conditional.

The emulator can be found at http://www.dumaisnet.ca/vm
The documentation about the architecture and the instruction set: Documentation


Since it is written in 100% client-side javascript, it was very easy to reuse the code and make command-line tools using node.js. I've created 3 tools:

  • An assembler: convert your source code to binary
  • A disassembler: convert binary file to assembly.
  • A Simulator: execute a binary

You can find the project on github at hrefhttps://github.com/pdumais/patasm

USING JSSIP AND ASTERISK TO MAKE A WEBPHONE

2015-04-07

Building a web-based phone is easy enough with asterisk and jssip. At the time of writing this, I was using asterisk 11.6 (LTS) and jssip 0.6.21.

My web application is hosted on a local webserver that resides on the same server as asterisk. Because of that, it is not possible to tell asterisk to bond on port 80 or 443 for its internal websocket server. So I've configured asterisk to bond on port 8088. But having a webapp that wants to communicate on such a port will not work if the clien is behind a firewall that blocks outside access to non standard ports. So I've configured Apache to proxy websocket connections to a chosen URL to the asterisk server on the local host.

Configuring asterisk

step-by-step instructions:

  • Make sure res_srtp is loaded in asterisk (you may need to install libsrtp and rebuild asterisk). This is absolutely needed for webrtc to work.
  • Enable websockets in asterisk http.conf [general] enabled=yes bindaddr=127.0.0.1 bindport=8088
  • Make a proxy entry in apache for wss://www.dumaisnet.ca/pbxws -> ws://127.0.0.1:8088/ws <VirtualHost *:443> SSLEngine On ... ServerName www.dumaisnet.ca <Location /pbxws> ProxyPass ws://127.0.0.1:8088/ws nocanon ProxyPassReverse ws://127.0.0.1:8088/ws </Location> </VirtualHost>
  • Generate DTLS certificates for asterisk mkdir /folder.to.keys cd <asterisk src directory>/contrib/scripts ./ast_tls_cert -C wwwdumaisnet.ca -O "Dumaisnet" -d /folder.to.keys
  • Configure a SIP peer in sip.conf transport=ws avpf=yes videosupport=no encryption=yes icesupport=yes directmedia=no force_avp=yes dtlsenable=yes dtlsverify=no dtlscertfile=/folder.to.keys/asterisk.pem dtlsprivatekey=/folder.to.key/asterisk.pem dtlssetup=actpass

Building the SIP client

jssip is very easy to use. The following example should register to your server and automatically answer an incoming call

C++ WEBSOCKET SERVER

2015-04-01

I recently wanted to learn a bit more about websockets. And by that, I don't mean how to use websockets from javascript but rather how the server part works and what the protocol looks like. So I decided to write my own server library. I followed RFC6455 but there is still some things I need to change in order to be fully compliant. There's not much to say about the library other than it is very easy to use. I did try libwebsocket before. It is pretty complete but I felt like it was a little more complicated than it should. So although my library is not as complete as libwebsocket, it is easier to use and will be good enough for most of my projects.

My code is hosted on github: https://github.com/pdumais/websocket