Painless IPSec setup for a home network.

I have written a script to painlessly set up fully secured encrypted network traffic between systems on a small LAN or home network. Although more specialized than other IPSec solutions such as SWAN or Racoon, it does provide for the common use case of securing otherwise sensitive protocols such as nfs, dns, web based device configuration pages, and samba that are often sent in the clear or with minimal authentication.

Network security has always been something of a pain, different protocols require completely different systems that must be independently learned and implemented. A signed certificate for https, a kerberos server for secure NFS, the woefully uncommon DNSSEC for DNS as well as many protocols that have no easy way to add security other than piggybacking on something such as SSH or stunnel.

IPSec makes all of this a non-issue. All traffic between hosts is encrypted and verified, no matter what the underlying protocol is. The encryption and verification is done in the kernel and is fully transparent to apps. Unfortunately, it also can be a bit complicated to set up in general due to the need for a daemon to handle the public key exchange, However for the common case of secure communications within a LAN between known hosts a lot of the usual complications can go away since you can pre-distribute the keys beforehand resulting in a simpler setup. I have written a script to automate this process.

Continue Reading »

Cuckoo Byte Stuffing Algorithm

Cuckoo Byte Stuffing

I introduce a byte stuffing algorithm for general use that has many useful properties that are not found together in other common algorithms. It obtains very low statistical overhead on arbitrary data without signifigantly increasing the entropy of the data stream. It does this by permuting the escape characters, taking them from a PRNG stream, rather than modifying the raw data. This means that the compressibility and radix locality of the encoded stream is preserved.

Similar to Cuckoo Hashing the algorithm has a pair of escape codes at any time from which values are ejected and a new one is chosen when they are shown to increase the size of the coded stream. By only ejecting a choice for an escape character on conflict, the algorithm is self optimizing, when the symbols of the data stream are not uniformly distributed, cuckoo stuffing achieves greater than the maximum theoretical efficiency  for random data. In particular, ASCII and UTF8 data have zero overhead for arbitrarily long streams even when they contain embedded null characters.

  • True online operation requiring zero bytes of lookahead, data can be output as soon as it is generated. By adding a single byte of lookahead the average size overhead can be further reduced. Said lookahead can be done opprotunistically based on data availability without changing the decoder.
  • The encoded byte stream preserves locality and compressibility, long runs of similar values remain long runs of similar values in encoded form. When possible, the stream remains unchanged aiding debugging by hardware protocol analyzers. This means that the encoded stream may directly be used as a radix tree key with the same efficiency as the unencoded stream.
  • Worst case 2x overhead but average case 0.2%, On any form of structured data with a non uniform distribution, the overhead asymptotically approaches zero. unlike other probabalistic algorithms the average case is over arbitrary data, not just random data. There are no generic pathological patterns to exploit.
  • The encoding/decoding algorithm requires only 7 bytes of RAM and no buffer. An implementation that takes only about 100 words of program memory on an atmega 8 bit mcu is provided. The PRNG is chosen to be particularly fast on both 8 bit and 32/64 bit architectures. Coding is extremely fast as PRNG only needs to be run on the rare collision.
  • Sample implementation released under completely free open source BSD style license.


Encoding takes a byte stream with arbitrary binary data and produces a byte stream that never has a byte equal to zero within it. Decoding inverts this process.

The algorithm relys on a PRNG that can generate a random sequence of bytes. At any given point, there are two escape codes active, e1 and e2. e1 is used to replace the zero byte, e2 is used to indicate that a literal e1 or e2 needs to be inserted. When a literal e1 or e2 occurs in the stream, it is replaced by the sequence e2 e1 or e2 e2 respectively and a new escape value is chosen by iterating the PRNG until the next number that is not zero and not equal to the other escape code. A new code is chosen only when a two byte escape code is generated.

Bytestream format

The decoding rules are as follows:

e1 -> 0
e2 e1 -> e1  and replace e1 with next code in sequence.
e2 e2 -> e2  and replace e2 with next code in sequence.
e2 x | x not in {e1,e2} -> e2 x
x -> x

The encoding rules are the opposite:

0 -> e1
e1 -> e2 e1  and replace e1 with next code in sequence.
e2 x | x not in {e1,e2,0} -> e2 x
e2 -> e2 e2  and replace e2 with next code in sequence.
x -> x


The PRNG chosen is based on an 8 bit xorshift algorithm, It performs very well on the Diehard tests and although specified as an 8 bit algorithm, it admits a particularly fast implementation on 32 or 64 bit architectures where the 4 middle ops can be replaced by a single shift in a full register.

#define INIT_CODE_STATE {  .x = 21, .y = 229, .z = 181, .w = 51 }
struct ecode_generator_state { uint8_t x, y, z, w; };

next_ecode(struct ecode_generator_state *s, uint8_t verboten)
        do {
                uint8_t t = s->x ^ (s->x << 3);
                s->x = s->y; s->y = s->z; s->z = s->w;
                s->w = s->w ^ (s->w >> 5) ^ (t ^ (t >> 2));
        } while (!s->w || s->w == verboten);
        return s->w;

The initial values for E1 and E2 are

#define INIT_E1 0xC1
#define INIT_E2 0xF8

These were chosen to give a good chance of zero overhead on common data streams.

  • Neither occurs in ASCII
  • Neither will ever appear in valid UTF8 data
  • Do not correspond to any common pad values in binary data protocols


Optimizing Haskell compiler jhc 0.8.1 is out

A new version of the jhc optimizing haskell compiler has been released with a new licence, new libraries, and lots of improvements.

After a long hiatus, I have released a new version of my haskell compiler jhc.

  • New license, jhc is now released under a permissive BSD style licence rather than the GPL. The license is compatible with that of ghc allowing code mixing between them.
  • New library layout based around the standards, there are now haskell98 and haskell2010 packages that are guarenteed to be future proof strictly compatible with the respective standards. A package haskell-extras contains the additonal libraries from ghc’s base.
  • Native support for complex and vector SIMD primitives, exposed via type
    functions. for instance ‘foo :: Complex_ Float32_’ for hardware accelerated
    complex 32 bit floats for instance. These are unboxed only for now, full
    library Num support in the works.
  • support for android as a target, you must install the android NDK to use this.
  • Support for embedded ARM architectures imported from Kiwamu Okabe’s branch
    allowing targeting bare hardware with no OS.
  • user defined kinds, introduced with the ‘kind’ keyword otherwise looking like
    ‘type’ declarations.
  • export/import lists now allow namespace qualifiers kind, class, type, or data
    to explicitly only import or export the specific named entity. As an
    extension allowed by this, classes and types no longer are in the same
    namespace and can share names.
  • ForeignPtr’s now have working finalizers when collected by the RTS.
  • CTYPE pragma to allow promoting arbitrary C types to FFIable entities.

Whaw 0.2 released.

Window tiling is nice; automatically placing your windows to not overlap and maximally use screen space. Switching fully to tiling window managers is not always feasible. Whaw is a program I wrote to tile windows on your screen completely independently of your window manager. It plays nicely with anything from modern window managers to using no window manager at all, calculating the proper arrangement then sending commands to your manager to carry out the placement.


Get it at

Cigar Box Breadboarding Tool – Part 1

As a kid, I always like those 300-in-One Electronic Project Lab toys. Although not really useful for real work, it was nice to have everything in one place. I decided to attempt to build one the way I want it to be. So far, I just have the power supply and external connectors set up. I can set it to unipolar +9V  or +18Vor bipolar +9/GND/-9 volts for analog work. Plus dedicated +5V and +3.3V rails. I used the dead simple method of generating a bipolar supply, I simple used two battery packs in series. Eventually I may upgrade to some sort of switcher circuit, but you can’t beat the pair of batteries for a low noise power source.

I included easy attachment points for connecting an oscilloscope or multimeter and phono jacks for routing audio signals and testing homebrew audio effects. By putting them near the edge I can move the board around without the cables interfering with teh breadboard or falling out.

I am using a random cigar box, they make great project boxes and can be gotten cheaply or for free from cigar shops. It turned out quite attractive if I do say so with its red/black motif. I wonder if I can find solid black breadboards to keep it on theme.

After this, I will add some panel meters and LEDs, some of the more common things needed when breadboarding. I am leaning towards a rotary encoder, a couple buttons, and a Voltmeter at the moment. Unlike the 300-in-1 style kit, I will have proper debouncing and quadrature circuitry built in to give nice clean logic level outputs and high impedance inputs.

P1000960 P1000962 P1000963 P1000966 P1000967P1000968gofpower

Panel Volt/Ammeter

Two inexpensive panel meters fit quite nicely in a Hammond stomp box case. Perfect for a voltage and current meter.

From Panel Volt/Ammeter

Continue Reading »

Simple, reusable ZIF “socket” for SMD parts.

I show how to use a pair of magnets to temporarily mount a surface mount chip to a breakout board.

Surface mount chips are becoming more common nowadays but they are harder to use for prototyping not being breadboard friendly. Using a simple SMD breakout board (like one of these) and some powerful magnets (I use these ones) you can conveniently and easily temporarily attach SMD chips to the breakout board.



You simply use a magnet to hold the chip onto the board.

Continue Reading »

Update on labeling your chips

A while ago I started  labeling my chips when breadboarding. Having a bin of properly labeled common jelly bean chips is just an extremely useful thing to have in arms reach when prototyping or experimenting with circuits. One thing that is annoying is having the decoupling capacitors right over the top of the chips blocking the layout and not doing a very good job decoupling due to it’s large leads adding dreaded inductance.

Then after spilling some parts I noticed this.

From Misc Projects

The axial decoupling caps fit directly in the grooves in the breadboard! Let’s take advantage of that….

Continue Reading »

Experimenting with a homebrew time domain reflectometer

Have a big coil of wire you don’t know the length of? don’t bother measuring it, just measure how long it takes for a signal to go through it and multiply by the speed of light and there you go.


Here is a simple one chip time-domain reflectometer. It consists of a pulse generator that can drive pulses with a rise time on the order of a nanosecond. I hook it to just one end of the unknown wire and set it off, the first step you see on the screen is my generated pulse. the next step is my signal being reflected off the other end of the cable and coming back to hit me. I have the meter set to 25 nanoseconds a division, so it took about 75 nanoseconds for the signal to come back, rule of thumb for the speed of light is a foot a nanosecond in vacuum and about 2/3 of that for an electric signal in copper. so 75 feet * 2/3 = 50 feet, and since the signal had to go down the wire and come back, you end up with 25 feet of wire in my unknown length coil.

The circuit is based on the one on this page which also has more discussion on how a TDR works.

Making a conductance substitution box

Resistance substitution boxes are a well known feature of any electronics lab and a common beginning project, allowing you to dial in a specific resistance measured in ohms directly. The inverse of a resistance box is the conductance box, allowing you to dial in a specific conductance measured in siemens. The most straightforward method of creating a resistance box is to use 10 position dials to connect resistors in series, a conductance box can be made by using BCD to connect resistors in parallel.

I used some dials that were BCD or binary coded decimal that had connections for one,two,four, and eight producing a binary digit by connecting the corresponding bits in parallel. While this form of dial cannot be directly used for a resistor box, it can be used to dial in conductance directly in siemens. This is due to the fact that connecting resistors in parallel will increase their conductance, as opposed to series which increases resistance and decreases conductance.

Continue Reading »