WWW.DUMAISNET.CA

System On A Chip emulatorLast edited on Apr 9, 2015

I was looking at some online NES emulators the other day and it gave me an idea. What If I created my own emulator but for my own architecture instead of emulating an existing one? Of course that would be easier because I can decide what the architecture looks like instead of having to go through the specs of the platform I want to emulate. So I decided to create my own instructions with an assembler and a disassembler. The assembler converts the assembly code to a binary format (obviously) and the result can be downloaded. Again, this is my own virtual assembly and my own architecture so the instructions are encoded in my own format. Each instructions are (inneficiently)encoded on 64bit including a references to 2 operands, 1 immediate 32bit value and a condition code. Each instruction is conditional.

The emulator can be found at http://www.dumaisnet.ca/vm
The documentation about the architecture and the instruction set: Documentation


Since it is written in 100% client-side javascript, it was very easy to reuse the code and make command-line tools using node.js. I've created 3 tools:

  • An assembler: convert your source code to binary
  • A disassembler: convert binary file to assembly.
  • A Simulator: execute a binary

The command-line tools can be found here: nodejsvm.tar



Using Jssip and asterisk to make a webphoneLast edited on Apr 7, 2015

Building a web-based phone is easy enough with asterisk and jssip. At the time of writing this, I was using asterisk 11.6 (LTS) and jssip 0.6.21.

My web application is hosted on a local webserver that resides on the same server as asterisk. Because of that, it is not possible to tell asterisk to bond on port 80 or 443 for its internal websocket server. So I've configured asterisk to bond on port 8088. But having a webapp that wants to communicate on such a port will not work if the clien is behind a firewall that blocks outside access to non standard ports. So I've configured Apache to proxy websocket connections to a chosen URL to the asterisk server on the local host.

Configuring asterisk

step-by-step instructions:

  • Make sure res_srtp is loaded in asterisk (you may need to install libsrtp and rebuild asterisk). This is absolutely needed for webrtc to work.
  • Enable websockets in asterisk http.conf
    [general]
    enabled=yes
    bindaddr=127.0.0.1
    bindport=8088
    
  • Make a proxy entry in apache for wss://www.dumaisnet.ca/pbxws -> ws://127.0.0.1:8088/ws
    <VirtualHost *:443>
    SSLEngine On
    ...
    ServerName www.dumaisnet.ca
    <Location /pbxws>
        ProxyPass ws://127.0.0.1:8088/ws nocanon
        ProxyPassReverse ws://127.0.0.1:8088/ws
    </Location>
    </VirtualHost>
    
  • Generate DTLS certificates for asterisk
    mkdir /folder.to.keys
    cd <asterisk src directory>/contrib/scripts
    ./ast_tls_cert -C wwwdumaisnet.ca -O "Dumaisnet" -d /folder.to.keys
    
  • Configure a SIP peer in sip.conf
    transport=ws
    avpf=yes
    videosupport=no
    encryption=yes
    icesupport=yes
    directmedia=no
    force_avp=yes
    dtlsenable=yes
    dtlsverify=no
    dtlscertfile=/folder.to.keys/asterisk.pem
    dtlsprivatekey=/folder.to.key/asterisk.pem
    dtlssetup=actpass
    

Building the SIP client

jssip is very easy to use. The following example should register to your server and automatically answer an incoming call

<script type="text/javascript" src="jssip-0.6.21.js" />
<script>
var userId = "webclient";
var domain = "dumaisnet.ca";
var config = {
    'ws_servers': 'wss://www.dumaisnet.ca/pbxws',
    'uri': "sip:"+userId+"@"+domain,
    'display_name': "web phone",
    'authorization_user': userId,
    'password': 'somepassword'
};
var userAgent = new JsSIP.UA(config);

userAgent.on('registered', function(e){ 
        // successfully registered
});
userAgent.on('unregistered', function(e){
});
userAgent.on('disconnected', function(e){
});
userAgent.on('registrationFailed', function(e){
});

// WARNING: When this handler was not defined, this UA
// would reply 405 to any INVITE.
userAgent.on('newRTCSession', function(e){
    e.session.answer();
    e.session.on('ended', function(e){
    });
    e.session.on('accepted', function(e){
    });
    e.session.on('failed', function(e){
    });
});

userAgent.start();

</script>


C++ websocket serverLast edited on Apr 1, 2015

I recently wanted to learn a bit more about websockets. And by that, I don't mean how to use websockets from javascript but rather how the server part works and what the protocol looks like. So I decided to write my own server library. I followed RFC6455 but there is still some things I need to change in order to be fully compliant. There's not much to say about the library other than it is very easy to use. I did try libwebsocket before. It is pretty complete but I felt like it was a little more complicated than it should. So although my library is not as complete as libwebsocket, it is easier to use and will be good enough for most of my projects.

My code is hosted on github: https://github.com/pdumais/websocket



Sip attack banningLast edited on Mar 30, 2015

New and improved version

I wrote this article a few years ago and posted a c++ application I wrote for automatically invoking iptables to hosts that are abusing my Asterisk server.

I rewrote the application, but this time using Perl. I use Net::Pcap to sniff on the network. The script runs as a daemon and looks for traffic going out of the LAN. It filters SIP responses and will automatically invoke iptables to block hosts to which it sees asterisk sending more than 10 (configurable) responses higher or equal than 400 to a remote host. Only responses sent for REGISTER and INVITE are filtered.

Script

#!/usr/bin/perl

use Net::Pcap;
use Socket;

# WARNING: configure this
my $to = "pat\@localhost";
my $pbxPort = 5070;
my $device = "eth0";
my $exemptNet = "192.168"; # This will exclude local interface as well.
my $maxfailcount = 10 ; # ban after that number of errors in a row.





my $err;
my %peers = ();
my $filterString = "udp port $pbxPort";

open(my $logFile, '>>', "/var/log/astban.log") or die "Can't open log file";


print "Listening on interface: $device\r\n";

$pid = fork();
if ($pid !=0)
{
    exit 0;
}

print $logFile localtime() . " astban started\r\n";
$logFile->flush();

my $pcap = Net::Pcap::open_live($device, 2048, 0, 0, \$err);
Net::Pcap::compile($pcap, \$filter, $filterString, 1, $net);
Net::Pcap::setfilter($pcap, $filter);

Net::Pcap::loop($pcap, -1, \&packetHandler, 0);




sub packetHandler 
{
    my($userData, $header, $packet) = @_;

    # Note that this is not bulletproof UDP parsing, we don't check fragmentation
    # nor for duplicate sequence numbers. This is just a quick&dirty implementation.
    my $IHL = ord(substr($packet,14,1)) &0x0F;
    my $udpStart = 14+($IHL*4);
    #my $payloadStart = $tcpStart+((((ord(substr($packet,$tcpStart+12,1))&0xF0)>>4)-1)*4);
    my $payloadStart = $udpStart+8;
    my $payload = substr($packet,$payloadStart);

    #@p = map(sprintf("%x",ord),split //,substr($packet,$tcpStart));
    #print "dump: @p\r\n\r\n";


    my $destination  = sprintf("%d.%d.%d.%d",
                               ord(substr($packet, 30, 1)),
                               ord(substr($packet, 31, 1)),
                               ord(substr($packet, 32, 1)),
                               ord(substr($packet, 33, 1)));
    $payload =~ /^SIP\/2.0 ([0-9]*) /;
    $code = $1;
    if ($payload =~ /^call-id:[ *|](.*)[\r|]$/im)
    {
        $callid = $1;
    }
    if ($payload =~ /^cseq:[ *|].* (.*)[\r|]$/im)
    {
        $method = $1;
    }


    if ($code)
    {
        if ($destination !~ /^$exemptNet/)
        {
            if (!exists($peers{$destination}))
            {
                $peers{$destination} = { 'failcount' => 0 };
            }

            #only filter for REGISTER and INVITE since we can get some errors for Notify and OPTIONS. it doesn't count.
            if ($method =~ /REGISTER/i | $method =~ /INVITE/i)
            {
                # if error response code is an error code and sent to a node outside of the network, flag it.
                if ($code >=400)
                {
                    {
                       $peers{$destination}{'failcount'}++;
                    }
                }
                else
                {
                    # reset the counter if we got something good, it means we are a friend
                   $peers{$destination}{'failcount'} = 0;
                }
            }
            print $logFile localtime() . " Match: $code, method: $method, dest: $destination callid: [$callid], fail:". $peers{$destination}{'failcount'} ."\r\n";
    
            if ($peers{$destination}{'failcount'} >= $maxfailcount)
            {
                print $logFile localtime() . " ERROR: Too many failures for hos. Banning\r\n";
                my $iptables = "iptables -A INPUT -s $destination -p udp --destination-port $pbxPort -j DROP";
                system($iptables);    

                open($m, "|/usr/sbin/sendmail -t");
                print $m "To: $to\n";
                print $m "From: root\@localhost\n";
                print $m "Subject: Astban is banning $destination\n\n";
                print $m "That guy was causing nothing but trouble!";
                close($m);
            }
            $logFile->flush();
        }
    }
}



Block caching and writebackLast edited on Dec 19, 2014

I recently wrote a disk driver for my x86-64 OS. I also wrote a block caching mechanism with delayed writeback to disk

Block Caching

reading/writing blocks is at a layer under the filesystem. so there is no notion of available/used blocks. This layer only reads/writes and caches blocks.

Reading a block

when a read request is executed, the requested block is first searched for in the cache. If a block is already cached, that data is returned. If the block does not exist in the cache, a new cache entry is created and is marked as "pending read". The new cache entry is associated with the device and block number that is requested. The request will then block the current thread until the block is gets its "pending read" status cleared. This will be done by the IRQ. When a new block needs to be created, it is done atomically so that only one block at a time can be created. That mechanism will prevent two similar read access that occur at the same time to issue 2 read requests.

When a block is read from disk, it is kept in the cache. Everytime it is accessed, a timestamp is update to keep track of the latest access.

Scheduling

A function called schedule_io() is called at the following times:

  • At the end of a disk IRQ.
  • After a cache entry is marked "pending read" and the disk driver is not busy (so no pending operation would trigger an IRQ)

The schedule_io() function iterates through the list of cache entries and finds an entry that is "pending read" and then requests the disk driver to read the sector from disk. Several different algorithms can be used in this function to make schedule_io() choose which "pending read" entry to use. A common algorithm is the "elevator" algorithm where the scheduler will choose to execute a read operation for a sector that is the closest to the last one read. This is limit seeking on disk. An elevator that needs to go to floors 5,2,8,4 will not jump around all those floors. If the elevator is currently at floor 3, it will do: 2,4,5,8.

That is not the algorithm I chose to implement though. To keep it simple (and tgus very inneficient), my scheduler just picks the first "pending read" entry it sees in the block cache list. When there is no more read requests, the scheduler proceeds with write requests. So read requests will always have higher priority. This is good for speed, but bad for reliability of data persistance.

Updating a block

when data needs to be written to an existing block, the block could be loaded in memory previously. This means that it was either read earlier for some other reasons or it was read and a small portion of it was updated. Either way, it is already in the cache and it needs to be written back to disk. In that case, the "pending write" flag will be set on it and when the scheduler picks it up, it will send a write request to the disk driver.

The following scenarios could occur:

  • Trying to read while write pending
    It doesn't matter. The block will be be read directly (from memory). This could happen after writing into a block and reading it right away. You would want the updated version.
  • Trying to write a block that does not exist yet in the cache
    This means that the block was never read and we just wanna overwrite whatever is in it. A cache entry will be created for the block and data will be copied in it. The Write pending" flag will be set
  • Trying to update while write pending
    This call would need to block until the block is finished writing back on disk. because we want to avoid updating in the middle of write

Block cache list

To keep things simple (and again very inneficient), I chose to implement the block cache list as a fixed-size array. A better approach would be to store the entries in a tree and let it grow as long as there is available memory.

Each cache entry is as follows:

#define CACHE_WRITE_PENDING 1
#define CACHE_READ_PENDING 2
#define CACHE_BLOCK_VALID 4
#define CACHE_IN_USE 8
#define CACHE_FILL_PENDING 16

struct block_cache_entry
{
    unsigned long block;
    char *data;
    unsigned char device;
    volatile unsigned char flags;
    unsigned long lastAccess;
} __attribute((packed))__;

Each entry has a field to determine the sector number on disk and the device number on which the sector belongs. lastAccess is used for the cache purging alorithm. The flags field is a combination of the following bits:

  • CACHE_WRITE_PENDING: The block does contain valid data but is not flushed to disk yet, but it should be.
  • CACHE_READ_PENDING: The block does not contain data yet and is waiting for a disk read operation to fill it
  • CACHE_BLOCK_VALID: The entry is valid. If 0, the entry is invalid and is free to use for caching. if 1, it contains valid data that belongs to a sector on disk.
  • CACHE_IN_USE: The entry is in use by the cache subsystem and should be be purged.
  • CACHE_FILL_PENDING: The entry was created to a write operation but does not contain data yet. So it cannot be read nor flushed to disk, but it should not be purged either.

Clearing cached block

when there is no space left in the cache block list (in my case, because the fixed-size array is full, but when the tree cannot grow anymore for the tree version), cached blocks must cleared. The block cache will find the blocks with oldest access time and that are not pending write or read and will free them. obviously, this is a very simple algorithm that does not take into account the frequency of access, or probability of access given the location on disk. But it works.

ATA driver

Just for reference, here is a sample of the disk driver. The full source code can be download at the end of this article, but I will show a portion of it here anyway

void atahandler(unsigned short base, unsigned char channel)
{
    unsigned char val;
    INPORTB(val,base+7);
    if (val&1)
    {
        INPORTB(val,base+1);
        pf("ERROR! %x\r\n",val);
    }
    else if (pendingRequest[channel].pending)
    {
        pendingRequest[channel].pending = 0;
        callback(pendingRequest[channel].dev,pendingRequest[channel].block,pendingRequest[channel].count);
    }
    INPORTB(val,busMasterRegister+2+(channel*8));
    OUTPORTB(4,busMasterRegister+2+(channel*8));
}

void atahandler1()
{
    atahandler(CH1BASE,0);
}

void atahandler2()
{
    atahandler(CH2BASE,1);
}

void ata_select_device(unsigned short dev, unsigned char slave)
{
    unsigned char val;
    unsigned int reg = CH1BASE - (dev<<7);

    if ((channelSlaveSelection&(1<<dev)) == ((slave&1)<<dev)) return;

    //OUTPORTB(0xA0 | (slave<<4),dev+ATA_REG_HDDEVSEL); // send "Select" command
    OUTPORTB(0xE0 | (slave<<4),reg+ATA_REG_HDDEVSEL); // send "Select" command
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);

    channelSlaveSelection &= ~(1<<dev);    
    channelSlaveSelection |= ((slave&1)<<dev);    

}

void ata_init_dev(unsigned short dev, unsigned char slave)
{
    unsigned int i;
    unsigned char val;
    unsigned int val2;
    unsigned int signature;
    unsigned int reg = CH1BASE - (dev<<7);

    pf("ATA DEVICE %x,%x: ",reg,slave);
    
    ata_select_device(dev,slave);

    OUTPORTB(ATA_CMD_IDENTIFY,reg+ATA_REG_COMMAND); // identify
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);
    INPORTB(val,reg+0x206);

    while (1)
    {
        INPORTB(val,reg+ATA_REG_STATUS);
        if (val == 0)
        {
            pf("none\r\n");
            return;
        } 
        else if (val&ATA_SR_ERR)
        {
            // Identify command does not work for ATAPI devices. Need to send IDENTIFY_PACKET.
            // but we will just ignore it here, we dont care about cdroms yet.
            pf("ATAPI\r\n");
            return;
        } 
        else if (!(val&ATA_SR_BSY)&&(val&ATA_SR_DRQ))
        {
            break;
        }
    }



    for (i=0;i<128;i++)
    {
        INPORTL(val2,reg+ATA_REG_DATA);
        ((unsigned int*)tmpBuffer)[i] = val2;
    }

    val2 = *((unsigned int *)(tmpBuffer+164));
    if (val2 & (1<<26))
    {
        val2 = *((unsigned int *)(tmpBuffer+200));
        pf("supported 48bit LBA device of size %x\r\n",val2*512);
        OUTPORTB(3,reg+ATA_REG_FEATURES); // DMA
    }
    else
    {
        pf("unsupported drive\r\n");
    }

}

void init_ata(atairqcallback irqcallback)
{
    int dev;

    callback = irqcallback;

    dev = pci_getDevice(0x8086,0x7010);
    pci_enableBusMastering(dev);

    busMasterRegister = pci_getBar(dev,4) & ~1;
    OUTPORTL(PRDT1,busMasterRegister+0x04); // set PRDT1 
    OUTPORTL(PRDT2,busMasterRegister+0x0C); // set PRDT2
    pf("IDE bus mastering Device: %x, bar4=%x\r\n", dev,busMasterRegister);

    ata_init_dev(0,0);
    ata_init_dev(0,1);
    ata_init_dev(1,0);
    ata_init_dev(1,1);

    // Warning: we hardcode irq 14 and 15 here, but we should read it from PCI device
    registerIRQ(&atahandler1,14);
    registerIRQ(&atahandler2,15);
}

void convertDevId(unsigned int dev, unsigned int *device, unsigned char *slave)
{
    switch (dev)
    {
        case 0:
            *device = 0;
            *slave = 0;
        break;
        case 1:
            *device = 0;
            *slave = 1;
        break;
        case 2:
            *device = 1;
            *slave = 0;
        break;
        case 3:
            *device = 1;
            *slave = 1;
        break;
    }
}

int ata_read(unsigned int dev, unsigned long sector, char* buffer, unsigned long count)
{
    unsigned int device;
    unsigned char slave,val;
    unsigned short reg,bmr;
    struct PRD *prdt = (struct PRD*)PRDT1;


    convertDevId(dev,&device,&slave);
    ata_select_device(device,slave);
    pendingRequest[device].pending = 1;
    pendingRequest[device].block = sector;
    pendingRequest[device].count = count;
    pendingRequest[device].dev = dev;

    if (device==0)
    {
        bmr = busMasterRegister;
        reg = CH1BASE;
    }
    else
    {
        bmr = busMasterRegister+8;
        reg = CH2BASE;
    }

    // setup PRD
    prdt[device].addr = buffer;
    prdt[device].size = count*512; // sector size = 512
    prdt[device].reserved = 0x8000;

    // Stop DMA
    OUTPORTB(0b00001000,bmr);

    // write sector count and LBA48 address
    OUTPORTB(((count>>8)&0xFF),reg+2);
    OUTPORTB(((sector>>24)&0xFF),reg+3);
    OUTPORTB(((sector>>32)&0xFF),reg+4);
    OUTPORTB(((sector>>40)&0xFF),reg+5);
    OUTPORTB((count&0xFF),reg+2);
    OUTPORTB((sector&0xFF),reg+3);
    OUTPORTB(((sector>>8)&0xFF),reg+4);
    OUTPORTB(((sector>>16)&0xFF),reg+5);
    OUTPORTB(0x25,reg+7);

    // Start DMA (read)
    OUTPORTB(0b00001001,bmr);
}

int ata_write(unsigned int dev, unsigned long sector, char* buffer, unsigned long count)
{
    unsigned int device;
    unsigned char slave,val;
    unsigned short reg,bmr;
    struct PRD *prdt = (struct PRD*)PRDT1;

    convertDevId(dev,&device,&slave);
    ata_select_device(device,slave);
    pendingRequest[device].pending = 1;
    pendingRequest[device].block = sector;
    pendingRequest[device].count = count;
    pendingRequest[device].dev = dev;

    if (device==0)
    {
        bmr = busMasterRegister;
        reg = CH1BASE;
    }
    else
    {
        bmr = busMasterRegister+8;
        reg = CH2BASE;
    }

    // setup PRD
    prdt[device].addr = buffer;
    prdt[device].size = count*512; // sector size = 512
    prdt[device].reserved = 0x8000;

    // Stop DMA
    OUTPORTB(0b00000000,bmr);

    // write sector count and LBA48 address
    OUTPORTB(((count>>8)&0xFF),reg+2);
    OUTPORTB(((sector>>24)&0xFF),reg+3);
    OUTPORTB(((sector>>32)&0xFF),reg+4);
    OUTPORTB(((sector>>40)&0xFF),reg+5);
    OUTPORTB((count&0xFF),reg+2);
    OUTPORTB((sector&0xFF),reg+3);
    OUTPORTB(((sector>>8)&0xFF),reg+4);
    OUTPORTB(((sector>>16)&0xFF),reg+5);
    OUTPORTB(0x35,reg+7);

    // Start DMA (read)
    OUTPORTB(0b00000001,bmr);
}

Download

block_cache.c
block_cache.h
ata.c (the disk driver)



Pages:12345678910