The Socialhacker
http://socialhacker.com/index.xml
Content on The Socialhacker
Hugo  gohugo.io
enus
anton@socialhacker.com (Anton Staaf)
anton@socialhacker.com (Anton Staaf)
Copyright © Anton Staaf. <a href="http://socialhacker.com/copyright/">Some rights reserved</a>.
Mon, 06 Mar 2017 00:00:00 +0000

2D Rotation
http://socialhacker.com/posts/2d_rotation/
Thu, 05 Sep 2013 21:55:00 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/2d_rotation/
<p>I'm hoping with this post to show one way that complex numbers come about and
how they are a natural way of representing rotations in 2D. To do this I will
introduce a small amount of Clifford algebra and some infinite series
expansions. A benefit of understanding rotations in this way will be that it
generalizes to arbitrary dimensions, though you probably only care about the
two and three dimensional cases.</p>
<p>A friend of mine recently asked me about 2D rotations. He had started with a
<a href="http://en.wikipedia.org/wiki/Rotation_matrix">2x2 rotation matrix</a>:</p>
<p><span class="math">\[
\left( \begin{array}{cr}
cos(\theta) & sin(\theta)\\
sin(\theta) & cos(\theta)\\
\end{array} \right)
\left( \begin{array}{c} x\\ y\\ \end{array} \right) =
\left( \begin{array}{c}
cos(\theta) * x  sin(\theta) * y\\
sin(\theta) * x + cos(\theta) * y\\
\end{array} \right)
\]</span></p>
<p>After playing with it for a while he concluded that the second row of the
matrix was redundant and you could just store the first row, or really any
row or column. This reminded him of something I had said a while back about
using complex numbers to represent rotations. This prompted me to write a
first version of this post as an email in response. Then he wrote again
saying that it seemed like the two component thing was the result of
<span class="math">\(e^{i\theta}\)</span>. So I have added a section on the exponential map and Taylor
series.</p>
<h2 id="complexnumbers">Complex Numbers</h2>
<p>First let's start with a bit of a review of complex numbers.
You've probably been introduced to them at some point as a
number with two components, a real portion and an imaginary
portion, where the imaginary portion is signified by having
a lower case <span class="math">\(i\)</span> associated with it.</p>
<p><span class="math">\[(a + b i)\]</span></p>
<p>And they were probably being used to fill in a gap when
someone tried to take the square root of a negative number.
Probably it was right around when you learned the quadratic
formula. Some people are introduced to complex numbers when
they start playing with fractals; the Mandelbrot set, Julia
sets, and others are defined in the complex plane. The
complex plane is what you get if you treat the real and
imaginary parts of a complex number as the <span class="math">\(x\)</span> and <span class="math">\(y\)</span>
components of a 2D vector. A lot of neat results come from
that mapping. Later I'll show a different way to associate
the components of a complex number with the <span class="math">\(x\)</span> and <span class="math">\(y\)</span>
components of a 2D vector, but first, let's remember what
happens when you multiply two complex numbers together.</p>
<p><span class="math">\[(a_1 + b_1 i) * (a_2 + b_2 i)\]</span></p>
<p>You probably remember FOIL (First Outer Inner Last) which
tells you how to go about doing this multiplication. First
multiple the a's together, then the outer elements <span class="math">\((a_1
\text{and} b_2 i)\)</span>, then the inner elements <span class="math">\((b_1 i
\text{and} a_2)\)</span>, and finally the last elements, the b's.</p>
<p><span class="math">\[(a_1 * a_2) + (a_1 * b_2 i) + (b_1 i * a_2) + (b_1 i * b_2 i)\]</span></p>
<p>With complex numbers we're free to move the i's around inside
each product, so we'll move them to the end of each one, and
outside of the parentheses.</p>
<p><span class="math">\[(a_1 * a_2) + (a_1 * b_2) i + (b_1 * a_2) i + (b_1 * b_2) i * i\]</span></p>
<p>The i * i that we get from the multiplication of the two
imaginary components <span class="math">\((b_1 \text{and} b_2)\)</span> of the two
complex numbers could also be written <span class="math">\(i^2\)</span>. The whole
reason for inventing <span class="math">\(i\)</span> was to provide an answer to the
question of what <span class="math">\(x\)</span> is in the following.</p>
<p><span class="math">\[x = \sqrt{1}\]</span></p>
<p>So, <span class="math">\(i^2 = 1\)</span>, and thus the result of multiplying two
imaginary components of a complex number together is a real
number, just one that has the opposite sign than it would
otherwise. So, taking this into account and merging the two
terms in the middle that each have a single <span class="math">\(i\)</span> in them we
can simplify the result of multiplying two complex numbers
down to the following.</p>
<p><span class="math">\[(a_1 * a_2) + (a_1 * b_2 + b_1 * a_2) i  (b_1 * b_2)\]</span></p>
<p>Finally, moving the imaginary portion to the end we get:</p>
<p><span class="math">\[(a_1 * a_2  b_1 * b_2) + (a_1 * b_2 + b_1 * a_2) i\]</span></p>
<p>Which is another complex number <span class="math">\((a_3 + b_3 i)\)</span> where
<span class="math">\(a_3\)</span> and <span class="math">\(b_3\)</span> are defined as:</p>
<p><span class="math">\[\begin{align*}
a_3 &= a_1 * a_2  b_1 * b_2 \\
b_3 &= a_1 * b_2 + b_1 * b_2
\end{align*}\]</span></p>
<p>With that basic review of complex numbers, let's move on to
talk about the Clifford algebra.</p>
<h2 id="cliffordalgebra">Clifford Algebra</h2>
<p>Above we had said that you could make a 2D plane by
associating the real and imaginary components of a complex
number with the <span class="math">\(x\)</span> and <span class="math">\(y\)</span> elements of a 2D vector.
But we didn't really talk about what that meant or how you
could distinguish an <span class="math">\(x\)</span> component from a <span class="math">\(y\)</span> component.
There has been an enormous amount written about this
subject, and I highly recommend Geometrical Vectors by
Gabriel Weinreich. But for now, I'm going to introduce two
new things, sort of like <span class="math">\(i\)</span> from the complex numbers,
called <span class="math">\(\mathbf{e_1}\)</span> and <span class="math">\(\mathbf{e_2}\)</span>. And we will
use them to distinguish between the two components of a 2D
vector. So, our vectors will look like this.
<span class="math">\((x \mathbf{e_1} + y \mathbf{e_2})\)</span>
These things, we'll call them bases, have a few properties:
a base times itself equals one, and swapping the position of
two bases in a term negates the term.</p>
<p><span class="math">\[
\mathbf{e_i e_i} = 1\\
\mathbf{e_i e_j} =  \mathbf{e_j e_i}
\]</span></p>
<p>These vectors behave just like you would expect a vector to
behave. You can add and subtract them, and you can multiply
them by scalars:</p>
<p><span class="math">\[\begin{align*}
(x_1 \mathbf{e_1} + y_1 \mathbf{e_2}) +
(x_2 \mathbf{e_1} + y_2 \mathbf{e_2}) &=
((x_1 + x_2) \mathbf{e_1} + (y_1 + y_2) \mathbf{e_2}) \\
c * (x_1 \mathbf{e_1} + y_1 \mathbf{e_2}) &=
((c * x_1) \mathbf{e_1} + (c * y_1) \mathbf{e_2})
\end{align*}\]</span></p>
<p>But they also have another interesting trick, if we multiply
two of these vectors together we get an interesting new
object.</p>
<p><span class="math">\[(x_1 \mathbf{e_1} +
y_1 \mathbf{e_2}) *
(x_2 \mathbf{e_1} +
y_2 \mathbf{e_2})\]</span></p>
<p>using FOIL you get</p>
<p><span class="math">\[(x_1 * x_2) \mathbf{e_1 e_1} +
(x_1 * y_2) \mathbf{e_1 e_2} +
(y_1 * x_2) \mathbf{e_2 e_1} +
(y_1 * y_2) \mathbf{e_2 e_2}\]</span></p>
<p>Since <span class="math">\(\mathbf{e_1 e_1} = 1\)</span> and <span class="math">\(\mathbf{e_2 e_2} = 1\)</span>
the first and last terms turn into scalars and you get</p>
<p><span class="math">\[(x_1 * x_2) +
(x_1 * y_2) \mathbf{e_1 e_2} +
(y_1 * x_2) \mathbf{e_2 e_1} +
(y_1 * y_2)\]</span></p>
<p>Now, the middle two terms can be combined by swapping the
<span class="math">\(\mathbf{e_2 e_1}\)</span> in the second term to be <span class="math">\(\mathbf{e_1
e_2}\)</span> and negating the whole term. So the result is</p>
<p><span class="math">\[(x_1 * x_2) +
(x_1 * y_2  y_1 * x_2) \mathbf{e_1 e_2} +
(y_1 * y_2)\]</span></p>
<p>Finally, grouping the first and last term together you get</p>
<p><span class="math">\[(x_1 * x_2 + y_1 * y_2) +
(x_1 * y_2  y_1 * x_2) \mathbf{e_1 e_2}\]</span></p>
<p>We see that the first term is just the dot product of the
two vectors, and the second term is just the 2D cross
product <sup class="footnoteref" id="fnref:1"><a class="footnote" href="#fn:1">1</a></sup> times something
weird, this <span class="math">\(\mathbf{e_1 e_2}\)</span> term. So we've got some
new odd object here: a scalar plus a scalar times a strange
pair of bases. If we replace the dot and cross product
values with the letters <span class="math">\(a\)</span> and <span class="math">\(b\)</span> we have <span class="math">\(a + b
\mathbf{e_1 e_2}\)</span>.</p>
<p>As you may know, the dot product of two vectors is equal to
the cosine of the angle between them times the length of
each vector. Similarly, the length of the cross product of
two vectors is equal to the sine of the angle between them
times the length of each vector. These sines and cosines
allow us to get back to rotation.</p>
<h2 id="2drotationrevisited">2D Rotation Revisited</h2>
<p>Now what happens if we multiply one of our original vectors
times one of these?</p>
<p><span class="math">\[(x \mathbf{e_1} + y \mathbf{e_2}) *
(a + b \mathbf{e_1 e_2})\]</span></p>
<p>We get:</p>
<p><span class="math">\[(a * x) \mathbf{e_1} +
(b * x) \mathbf{e_1 e_1 e_2} +
(a * y) \mathbf{e_2} +
(b * y) \mathbf{e_2 e_1 e_2}\]</span></p>
<p>Which after simplification is:</p>
<p><span class="math">\[(a * x  b * y) \mathbf{e_1} + (b * x + a * y) \mathbf{e_2}\]</span></p>
<p>Which looks a lot like a rotation matrix application. <sup class="footnoteref" id="fnref:2"><a class="footnote" href="#fn:2">2</a></sup> Especially when you
remember that if the two vectors we started with are unit length then
a and b are equal to the cos and sin of the angles between
the two vectors. In which case we have:</p>
<p><span class="math">\[a = cos(\theta)\\
b = sin(\theta)\]</span></p>
<p>and after substitution:</p>
<p><span class="math">\[
\begin{align*}
&(cos(\theta) * x  sin(\theta) * y) \mathbf{e_1} +\\
&(sin(\theta) * x + cos(\theta) * y) \mathbf{e_2}
\end{align*}
\]</span></p>
<h2 id="complexnumbersrevisited">Complex Numbers Revisited</h2>
<p>You can probably see where this is going. Let's try
multiplying two of these 'scalar plus scalar times base pairs'
together.</p>
<p><span class="math">\[
(a_1 + b_1 \mathbf{e_1 e_2}) *
(a_2 + b_2 \mathbf{e_1 e_2})
\]</span></p>
<p>Again, using FOIL we get:</p>
<p><span class="math">\[
(a_1 * a_2) +
(a_1 * b_2) \mathbf{e_1 e_2} +
b_1 \mathbf{e_1 e_2} * a_2 +
b_1 \mathbf{e_1 e_2} * b_2 \mathbf{e_1 e_2}
\]</span></p>
<p>It is fine to move the scalars around, so let's clean this up:</p>
<p><span class="math">\[
(a_1 * a_2) +
(a_1 * b_2) \mathbf{e_1 e_2} +
(a_2 * b_1) \mathbf{e_1 e_2} +
(b_1 * b_2) \mathbf{e_1 e_2 e_1 e_2}
\]</span></p>
<p>We can combine the two <span class="math">\(\mathbf{e_1 e_2}\)</span> terms:</p>
<p><span class="math">\[
(a_1 * a_2) +
(a_1 * b_2 + a_2 * b_1) \mathbf{e_1 e_2} +
(b_1 * b_2) \mathbf{e_1 e_2 e_1 e_2}
\]</span></p>
<p>Next we deal with the <span class="math">\(\mathbf{e_1 e_2 e_1 e_2}\)</span> term. If
we swap the middle <span class="math">\(\mathbf{e_2 e_1}\)</span> into <span class="math">\(\mathbf{e_1
e_2}\)</span> and negate the whole thing we get <span class="math">\(\mathbf{e_1 e_1
e_2 e_2}\)</span>. And since a base times itself is <span class="math">\(1\)</span> we have
<span class="math">\(1 * 1\)</span> or <span class="math">\(1\)</span>. So, the last term can be replaced
with <span class="math">\((b_1 * b_2)\)</span>, causing the entire thing to turn
into:</p>
<p><span class="math">\[
\begin{align*}
&(a_1 * a_2  b_1 * b_2) +\\
&(a_1 * b_2 + a_2 * b_1) \mathbf{e_1 e_2}
\end{align*}
\]</span></p>
<p>And poof, we have another one of these scalar plus scalar
times <span class="math">\(\mathbf{e_1 e_2}\)</span> things. The fact that
<span class="math">\(\mathbf{e_1 e_2 e_1 e_2} = 1\)</span> should have given it away:
<span class="math">\(\mathbf{e_1 e_2}\)</span> could also be called <span class="math">\(i\)</span> and these
are complex numbers (or at least they behave like them,
which in some sense means that they are them).</p>
<p>If complex numbers really do represent rotations in 2D then
we would expect that there would be a way to combine two
rotations that are represented by complex numbers, resulting
in a single complex number that represents the combined
rotation. And you are probably not surprised to learn that
there is, and it is just complex multiplication (which we
worked out above). To show this let's assume that <span class="math">\((a_1 +
b_1 \mathbf{e_1 e_2})\)</span> represents a rotation by <span class="math">\(\theta\)</span>
radians and <span class="math">\((a_2 + b_2 \mathbf{e_1 e_2})\)</span> represents a
rotation by <span class="math">\(\phi\)</span> radians. So:</p>
<p><span class="math">\[
a_1 = cos(\theta)\\
b_1 = sin(\theta)\\
a_2 = cos(\phi)\\
b_2 = sin(\phi)
\]</span></p>
<p>If we rewrite the result of multiplying two complex numbers
together and substitute these cosines and sines in we get:</p>
<p><span class="math">\[
\begin{align*}
&(cos(\theta) * cos(\phi)  sin(\theta) * sin(\phi)) +\\
&(cos(\theta) * sin(\phi) + cos(\phi) * sin(\theta))
\mathbf{e_1 e_2}
\end{align*}
\]</span></p>
<p>Now we have to remember (or rederive, but this post is
getting too long as it is) the sin and cos of the sum of
angles formulas. They are, probably not surprisingly:</p>
<p><span class="math">\[
\begin{align*}
cos(\theta + \phi) &=
cos(\theta) * cos(\phi)  sin(\theta) * sin(\phi)\\
sin(\theta + \phi) &=
cos(\theta) * sin(\phi) + cos(\phi) * sin(\theta)\\
\end{align*}
\]</span></p>
<p>And thus, we can rewrite the product of two complex numbers
that represent rotations as:</p>
<p><span class="math">\[cos(\theta + \phi) + sin(\theta + \phi) \mathbf{e_1 e_2}\]</span></p>
<p>And indeed, the product of two complex numbers that
represent rotations, is itself a rotation, one that is the
sum of the two independent rotations.</p>
<h2 id="exponentialmap">Exponential Map</h2>
<p>The exponential map is a special function that is its own
derivative. It is fascinating, and I recommend reading more
about it. Here I'm just going to quickly show one fantastic
result. I'll need to use the words Taylor Series and
infinity, but I'm not going to justify them much. It can be
thought of as an infinite sum of terms, or as the
exponentiation <span class="math">\(e^x\)</span>.</p>
<p><span class="math">\[exp(x) = e^x = \sum_{k=0}^{\infty} \frac{x^k}{k!}\]</span></p>
<p>This function is valid for all complex numbers, but we will
limit the input to just the imaginary numbers (the imaginary
portion of a complex number). To make the notation a little simpler I'll use <span class="math">\(i\)</span>
instead of <span class="math">\(\mathbf{e_1 e_2}\)</span>. I'll show that by doing
this we effectively map an infinite line to the circle of radius
one centered at the origin of the complex plane. Points on the line
can be thought of as representing angles in radians. And every time
we move <span class="math">\(2 \pi\)</span> along the line we make one full loop around the
circle. So, there are many points on the line that map to the same
point on the circle. These all represent the same rotation.</p>
<p><span class="math">\[exp(x i) = e^{x i} = \sum_{k=0}^{\infty} \frac{(x i)^k}{k!}\]</span></p>
<p>First, let's look at the <span class="math">\((x i)^k\)</span> term. The first few
terms in the infinite sum and the generalization for an
arbitrary value of <span class="math">\(k\)</span> are:</p>
<p><span class="math">\[
\begin{align*}
(x i)^0 &= x^0 = 1\\
(x i)^1 &= x^1 i = x i\\
(x i)^2 &= x^2 i i = 1 x^2\\
(x i)^3 &= x^3 i i i = 1 x^3 i\\
&\vdots\\
(x i)^k &=
x^k i^k =
\begin{cases}
1^{\frac{k}{2}} x^k & k\;\text{is even}
\\[1ex]
1^{\frac{k1}{2}} x^k i & k\;\text{is odd}
\end{cases}\\
\end{align*}
\]</span></p>
<p>If we group all of the even and all of the odd terms
together we get:</p>
<p><span class="math">\[
e^{x i} =
\sum_{k=0,2,4...}^{\infty} \frac{1^{\frac{k}{2}} x^k}{k!} +
\sum_{k=1,3,5...}^{\infty} \frac{1^{\frac{k1}{2}} x^k}{k!}i
\]</span></p>
<p>We can turn the sums back into simple sums from zero to
infinity by transforming <span class="math">\(k\)</span> in each of them. In the
first one we can replace <span class="math">\(k\)</span> with <span class="math">\(2k\)</span>, and in the
second we can replace <span class="math">\(k\)</span> with <span class="math">\(2k+1\)</span>. The result is:</p>
<p><span class="math">\[
e^{x i} =
\sum_{k=0}^{\infty} \frac{1^k x^{2k}}{(2k)!} +
\sum_{k=0}^{\infty} \frac{1^k x^{2k+1}}{(2k+1)!}i
\]</span></p>
<p>It turns out that the first sum is actually the Taylor
series expansion for <span class="math">\(cos(x)\)</span> and the second sum is the
Taylor series expansion for <span class="math">\(sin(x)\)</span>. So this can be
rewritten as:</p>
<p><span class="math">\[ e^{x i} = cos(x) + sin(x) i \]</span></p>
<p>This beautiful result is known as <a href="http://en.wikipedia.org/wiki/Euler's_formula">Euler's formula</a>.</p>
<div class="footnotes">
<hr>
<ol>
<li id="fn:1"><p>In 2D the cross product as we normally think of it doesn't
exist, but the value above can be thought of as the Z
component of a 3D cross product of two vectors that lie in
the XY plane. Since the two vectors have zero Z
components, the X and Y components of their cross product
is zero, so the overall cross product length is just the
length of the Z portion.</p>
<a class="footnotereturn" href="#fnref:1"><sup>[return]</sup></a></li>
<li id="fn:2"><p>There is a bit more subtlety to this, the actual group
action we care about is ATA* <sup class="footnoteref" id="fnref:3"><a class="footnote" href="#fn:3">3</a></sup> where A is a complex
number, T is a vector and A* is the conjugate of the
complex number. That action works even if the complex
number isn't unit, and it generalizes up to higher
dimensions (TA, what we did here, doesn't).</p>
<a class="footnotereturn" href="#fnref:2"><sup>[return]</sup></a></li>
<li id="fn:3"><p>The construction of a complex number as a multiplication
of two vectors using the odd base combination rules gives
a cool way to ask for a rotation between two vectors. If
you use the full ATA* group operation the rotation is
twice the angle between the two vectors though. This
turns out to be important in higher dimensions.</p>
<a class="footnotereturn" href="#fnref:3"><sup>[return]</sup></a></li>
</ol>
</div>

Mandelbrot
http://socialhacker.com/posts/fractals/mandelbrot/
Wed, 16 May 2012 00:00:00 +0000
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/fractals/mandelbrot/
<p><a href="http://socialhacker.com/applications/mandelbrot/mandelbrot.html">Mandelbrot</a> is a javascript
implementation of a simple mandelbrot set calculator.</p>
<p></p>
<p>
<pre><code class="languagejavascript">//
// Set the pixel at location <x,y> in the image data to <r,g,b,a>. The r, g,
// and b elements are the red, green, and blue color components. The a element
// is the alpha, or transparency value. The r, g, b, and a values can range
// from 0 to 255.
//
function set_pixel(image_data, x, y, r, g, b, a)
{
// Compute the index into the pixel data. Each pixel is represented by four
// sequential values in the image_data.data array.
index = (x + y * image_data.width) * 4;
image_data.data[index + 0] = r;
image_data.data[index + 1] = g;
image_data.data[index + 2] = b;
image_data.data[index + 3] = a;
}
//
// Compute the number of iterations required for a single orbit of the
// mandelbrot equation to escape from a circle of radius 2 centered at the
// origin of the complex plane.
//
function compute_escape_time(c_r, c_i, maximum_iterations)
{
z_r = 0.0;
z_i = 0.0;
for (i = 0; i < maximum_iterations; i++)
{
//
// Compute a single iteration of Z^2 + C. Complex numbers are of the
// form "real + imaginary * i", so to square them you end up computing:
//
// (real + imaginary * i) * (real + imaginary * i)
// =
// real * real + imaginary * imaginary * i^2 + 2 * real * imaginary * i
// =
// real * real  imaginary * imaginary + 2 * real * imaginary * i
//
// So the new real value is (real * real  imaginary * imaginary)
// And the new imaginary value is (2 * real * imaginary)
//
// The reason the first + turns into a  is because i^2 is defined to be
// equal to 1, that's what makes the numbers imaginary.
//
temp = (z_r * z_r)  (z_i * z_i) + c_r;
z_i = (2.0 * z_r * z_i) + c_i;
z_r = temp;
// Check to see if the magnitude of the complex number is larger than
// 2. We check whether the squared magnitude is greater than 4 here to
// avoid taking a square root, because square roots are slow.
if (((z_r * z_r) + (z_i * z_i)) > 4.0)
return i;
}
// Return the maximum iterations value if we never escaped.
return maximum_iterations;
}
//
// Draw a Mandelbrot set filling in a canvas specified by element_id.
//
function draw_mandelbrot(element_id)
{
// Lookup the canvas and get a 2D rendering context from it.
element = document.getElementById(element_id);
context = element.getContext("2d");
//
// Construct an image data object to hold the pixels before they are
// drawn to the screen.
//
image_data = context.createImageData(element.width,
element.height);
//
// Iterate over every pixel in the canvas and compute the escape time for
// the corresponding point in the complex plane.
//
for (y = 0; y < image_data.height; y++)
{
for (x = 0; x < image_data.width; x++)
{
//
// This scales the <x, y> values into a smaller range of the
// complex plane. But it doesn't take into account the size of the
// canvas element. So if the canvas element changes size, this will
// need to change as well. This can be made automatic, but it would
// obcure the meaning for this example. It would be a good
// experiment to change these values and try and make the result
// general.
//
iterations = compute_escape_time((x  350.0) / 200.0,
(y  250.0) / 200.0,
100);
//
// This is a simple use of the iteration count. We are only looking
// at the bottom bit of the resulting escape time iterations. This
// results in the zebra striping look.
//
if (iterations & 1)
r = g = b = 255;
else
r = g = b = 0;
//
// Write the computed color value to the image_data at the current
// location.
//
set_pixel(image_data, x, y, r, g, b, 255);
}
}
// Finally, copy the resulting image to the canvas for display.
context.putImageData(image_data, 0, 0);
}
</code></pre>
</p>
<p>Syntax highlighted with <a href="http://prismjs.com">Prism</a>.</p>

Git Filter Branch
http://socialhacker.com/posts/git_filter_branch/
Sun, 25 Mar 2012 03:14:00 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/git_filter_branch/
<p>This HOWTO describes the steps required to extract a portion of a
large git repository into it's own separate repository.</p>
<p>I have found this useful numerous times because when I first started
to use git I created a single repository for all of my personal
projects. Since then I've come to appreciate having many smaller
repositories. So whenever I want to open up some new chunk of code
I use these steps to extract the history for just that chunk into a
new repository and then publish that.</p>
<h2 id="setup">Setup</h2>
<p>I always start by doing a fresh clone of the repository that I want
to extract from, so that I know I'm not messing with my primary
development environment. This has the added advantage that as long
as you don't clone with recursive, you won't have any submodules
checked out. If you have submodules then the filter branch can get
into trouble trouble.</p>
<pre><code>git clone git://source.socialhacker.com/... temp_clone
</code></pre>
<p>Then just to make sure I'm not going to do something stupid, I remove
the remote from the newly cloned repository.</p>
<pre><code>cd temp_clone
git remote rm origin
</code></pre>
<p>Now we're ready to run git filterbranch. In particular, I run the
subdirectory filter which replays all of your commits, only keeping
changes to a given subdirectory. You need to add the pruneempty
option to cause filterbranch to not include empty commits.</p>
<pre><code>git filterbranch \
pruneempty \
subdirectoryfilter &lt;directory&gt; \
 \
all
</code></pre>
<p>Now, at this point I like to fix up the history a little. In
particular I fix up the commit messages with the name and email
address I've decided on. For a while I hadn't set these correctly
and my early commits have something pretty useless. This will only
work if you're the only committer to your repository. If you have
multiple people committing, then this will wipe out their author names
and email addresses from their commits.</p>
<pre><code>git filterbranch \
f \
envfilter "export GIT_AUTHOR_NAME='Anton Staaf'; export GIT_AUTHOR_EMAIL='anton@socialhacker.com';" \
HEAD
git filterbranch \
f \
envfilter "export GIT_COMMITTER_NAME='Anton Staaf'; export GIT_COMMITTER_EMAIL='anton@socialhacker.com';" \
HEAD
</code></pre>
<p>The final step before you can push your new repository is to remove
all of the old information about the commits that you no longer want
to be visible. The first line below clears out the reflog, so that
it doesn't maintain references to the old state of the repository.
The second line does a garbage collection run on the repository. This
will remove any objects that are no longer referenced.</p>
<pre><code>git reflog expire expire=now all
git gc aggressive prune=now
</code></pre>
<p>At this point you can add a new remote and push your repository.</p>

Diet Planner
http://socialhacker.com/posts/diet_planner/
Sat, 17 Mar 2012 00:00:00 +0000
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/diet_planner/
<p>The <a href="http://socialhacker.com/applications/diet_planner/diet_planner.html">Diet Planner</a> is a simple
javascript application that can calculate grams of fat, protien, and
carbohydrates needed to gain or lose weight given various body metrics.</p>

Brute Force SSH Attack Protection
http://socialhacker.com/posts/ssh_protection/
Tue, 12 Oct 2010 05:19:37 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/ssh_protection/
<p>This HOWTO describes a couple things that you can do to secure your
SSH server on a Linux machine (Ubuntu, RedHat, Suse...).</p>
<p>This is useful because there are script kiddies all around trying to
break into computers. And I imagine that botnets writers will take
more interest in Linux as it's market share increases.</p>
<p>The pattern that I have seen is of many many requests from the same
IP address trying to guess users and passwords. Most of the requests
are trying to guess the root password.</p>
<p>There are a couple things we can do to slow these attackers. The
most obvious is to configure ssh to only allow logins from a couple
select users. And to disallow remote login by the root user. We can
also use IPTables to only allow a limited number of connections per
minute. And finaly, we can move the SSH server to a different port
on the machine. I don't know if this actually causes the attackers
any pause however. They may just be trying all of the open ports.</p>
<p>There are more complex solutions to the problem. Port knocking or
log parsing come to mind. But I've opted for the simplest solution
that doesn't impact usability in my case.</p>
<p>The use of IPTables to limit repeated connections is based on work
by Kevin van Zonneveld. You can see his approach on <a href="http://kevin.vanzonneveld.net/techblog/article/block_brute_force_attacks_with_iptables/">his blog</a></p>
<h2 id="whatisiptables">What is IPTables</h2>
<p>IPTables is part of the kernels network stack (I think). It is a
user configurable state machine that can be used to filter packets as
they are received, before they are forwarded or before they are
transmitted.</p>
<p>Our configuration will drop incoming packets that meet a specific set
of rules.</p>
<h2 id="sshdconfiguration">SSHD configuration</h2>
<p>The file /etc/ssh/sshd_config is used to configure the ssh server on
your linux machine. The changes I made to mine were to change "Port
22" to "Port xxxx" and to add "AllowUsers yyyy zzzz wwww" where xxxx
is the new port you want SSH to listen to. yyyy, zzzz and wwww are
the users that you want to have remove access. I also made sure
that the line "PermitRootLogin no" existed and was not commented
out.</p>
<h2 id="sshconfiguration">SSH configuration</h2>
<p>If you have changed the port that sshd listens to then you will
probably want to configure your ssh clients on any machine that you
would like to access your server from. In your home directory on
each of these machines you should find "~/.ssh/". In that directory
you can create a config file. It's just called config. Put the
following in that file. Again, xxxx is the new port that your ssh
server is listening to.</p>
<pre><code>Host your.server.name
Port xxxx
</code></pre>
<h2 id="networkscripts">Network scripts</h2>
<p>In Ubuntu there are directories that contain scripts to run when an
interface comes up or goes down. These are convenient places to put
the IPTables commands needed to drop attackers packets. The
directory for scripts to run when a network interface comes up is
/etc/network/ifup.d. And the directory for scripts to run when a
network interface goes down is /etc/network/ifdown.d. We will
create one file in the ifup.d directory and a symlink in the
ifdown.d directory. We do this to consolidate the logic in a single
location. We can use the MODE variable to determine if the interface
is coming up or going down.</p>
<p>In /etc/network/ifup.d/sshprotection put the following.</p>
<pre><code><pre>
#!/bin/bash
SSH_IFACE="eth1"
SSH_PORT=xxxx # This should be the port you've moved your ssh server to, or 22 if you haven't moved it.
SSH_PERIOD=60
SSH_COUNT=8
#
# Only add the rules to the interface that SSH is actually listening on.
#
if [ "$IFACE" != "$SSH_IFACE" ]; then
exit 0
fi
case "$MODE" in
start)
IPTABLES_ACTION="A"
;;
stop)
IPTABLES_ACTION="D"
;;
esac
/sbin/iptables $IPTABLES_ACTION INPUT \
i $IFACE \
p tcp \
dport $SSH_PORT \
m state \
state NEW \
m recent \
set \
name SSH
/sbin/iptables $IPTABLES_ACTION INPUT \
i $IFACE \
p tcp \
dport $SSH_PORT \
m state \
state NEW \
m recent \
update \
seconds $SSH_PERIOD \
hitcount $SSH_COUNT \
rttl \
name SSH \
j DROP</pre>
</code></pre>
<p>This file needs to be executable by root. You can use the following
command line to make it so.</p>
<pre><code>chmod u+x /etc/network/ifup.d/sshprotection
</code></pre>
<p>And in /etc/network/ifdown.d create a symlink to the sshprotection
file in ifup.d using the following command line. This command line
assumes you're in the ifdown.d directory.</p>
<pre><code>ln s ../ifup.d/sshprotection
</code></pre>

Port Forwarding
http://socialhacker.com/posts/port_forwarding/
Wed, 20 May 2009 23:10:10 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/port_forwarding/
<p>This HOWTO describes the steps required to setup your RedHat (well any Linux
distro) firewall to forward the port used by gtkgnutella to a machine on your
internal network. This is useful because it allows your gtkgnutella client
to behave in a non firewalled mode and thus more of the gnutella network is
available to you. In particular, other machines that are behind firewalls that
can handle push requests will become available to you.</p>
<h2 id="whatisportforwarding">What is Port Forwarding?</h2>
<p>Port forwarding is a feature of the IPTables system. It allows one
computer to forward connections made to it so that another computer
can actually process the request. If you want a very simple metaphor
you can think of it as mail forwarding. Each computer has a number
of addresses called ports, and IPTables allows (among other things)
connections to these ports to be sent to another computer. The most
common use of port forwarding that I am aware of is allowing servers
to run on machines that would normally be hidden behind a firewall.</p>
<h2 id="firewallscript">Firewall script</h2>
<p>I am going to assume that you are using the default firewall script
that comes with RedHat or whatever distribution you are running. My
system is currently running RedHat 8.0 (heh, not anymore). And I am
using a firewall script called rc.firewall2.4. You should be able to
find it in your /etc/rc.d directory. If you don't find it it may be
that I had to install it and just don't remember. :) You can search
for rc.firewall2.4 on Google and find many copies.</p>
<h2 id="setup">Setup</h2>
<p>My goal was to make gtkgnutella work in a nonfirewalled mode from
within my firewalled LAN. To do this people suggested a line of the
form:</p>
<pre><code>$IPTABLES t nat I PREROUTING p tcp i $EXTIF dport 6346 j DNAT to 192.168.0.2:6346
</code></pre>
<p>Where $IPTABLES is the iptables executable, $EXTIF is the external
ethernet interface (I use two ethernet cards in my firewall), port
6346 is the gtkgnutella port and 192.168.0.2 is the machine on my
internal network on which I wished to run gtkgnutella.</p>
<p>With the rc.firewall2.4 script this doesn't quite work. The reason
is that by default any connection that would open a new session from
the outside world is dropped. This is done with the line:</p>
<pre><code>$IPTABLES P FORWARD DROP
</code></pre>
<p>The solution is to add a rule into the FORWARD chain that causes
connections from the outside world to port 6346 to not be dropped,
but instead to be accepted. Then the PREROUTING rule above will be
encountered and the connection will be forwarded to the internal
machine as desired. The line to accomplish this is:</p>
<pre><code>$IPTABLES A FORWARD i $EXTIF o $INTIF p tcp dport 6346 j ACCEPT
</code></pre>
<p>I placed this right before the line:</p>
<pre><code>$IPTABLES A FORWARD j LOG
</code></pre>
<p>I did this because this line will will add a rule that causes all
rules added after it to not be checked.</p>

Virtual Hosting and SSL
http://socialhacker.com/posts/socialhacker/virtual_hosting_anton_ssl/
Tue, 05 May 2009 00:28:33 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/socialhacker/virtual_hosting_anton_ssl/
<p>All right, now that we have some blogs up. We will want to enable
SSL/TLS security, also known as https. The reason you want this may be
obvious to some. But to spell it out for you, if you don't use https
to connect to your administrative pages in Movable Type while you're
sipping on your latte, then everyone else that's on that wireless
network can see you type your password in, plain as day. And by "can",
I mean there is nothing preventing them from watching your traffic.
Most likely, no one is, but you never know.</p>
<p>There are a couple of complications that we will need to work through
though. Firstly, SSL and name based virtual hosting are mostly
incompatible. And secondly, unless you get a Certificate Authority,
such as Verisign, to sign your SSL key you and your security conscious
visitors will be presented with an ugly message from the browser
saying that your site is trying to identify itself with an invalid
security certificate.</p>
<p>So, why are SSL and name based virtual hosting mostly incompatible?
Well, it turns out that the way name based virtual hosting works is
for the browser to send to your web server the name of the server it's
trying to connect to. And then your web server uses that information
to look through the list of virtually hosted domains until it finds a
match. Then the page the browser wants is sent from the server to the
browser. But if you have connected to the web server using an https
connection then the communication channel needs to be encrypted. So
the server needs to send the browser the public key of the domain the
browser is trying to connect to. But the server doesn't yet know what
domain the browser is trying to connect to, because that information
is part of what will eventually be encrypted and sent by the browser.
Apache will just serve the first certificate it finds in this case.
So all but the first domain in your list of virtually hosted domains
will cause the browser to issue an additional warning, that the
certificate is for the wrong domain. This isn't a big problem for
large hosting companies or business, they can just assign a separate
IP address to each domain. And then use that extra information to
configure the web server, allowing it to pass the correct domain
specific key back to the browser. For us little guys, that's not
really a suitable approach. You can do something similar by having
your web server listen on a bunch of different ports, one for each
domains SSL connection. But then your URLs will have to have the port
number in them as well. Not a really classy solution. There is a
solution to this problem in the works. It's called "Server Name
Indication" or SNI, and it's part of the TLS protocol. Unfortunately
it's still not readily available (see
<a href="http://www.alexanderkiel.net/2008/04/22/statusoftlssni/">this</a> page).
So, what's the solution? It's pretty simple actually, just use
one certificate for all of your domains. There is an extension that
allows for multiple domain names to be associated with a single
certificate. When a browser receives such a certificate, it looks at
all of the domain names and if any of them match it is satisfied.
There seem to be some security issues with this feature (see
<a href="http://nils.toedtmann.net/pub/subjectAltName.txt">this</a>
page). But since we are mainly interested in using it to authenticate
ourselves with our own server, it's not a big deal, I think. And that
brings us to our second issue. We can pay Verisign to generate a
certificate for us, or we can sign it ourself. If you pay Verisign (or
other Certificate Authority) then anyone that browses your web site
with a secure https connection will feel right at home, secure even.
If you sign the certificate yourself, then viewers will be presented
with a nastygram from their browser. Since I am mainly interested in
being able to securely access my servers from insecure networks, I'm
happy to sign the certificates myself.</p>
<p>First you need to enable SSL in your Apache configuration. In the
Apache2 configuration for Ubuntu these files are located in
/etc/apache2. You'll need to make sure that your ports.conf file
contains something like the following:</p>
<pre><code>Listen 80
<IfModule mod_ssl.c>
Listen 443
</IfModule>
</code></pre>
<p>This causes Apache to listen on port 443 for connections as well as
port 80. Port 443 is the https port. And your conf.d/namevirtualhosts
file should look something like:</p>
<pre><code>NameVirtualHost *:80
NameVirtualHost *:443
</code></pre>
<p>This tells Apache to look up virtual hosts by name for traffic coming
in on either port. Next you need to make sure that your
conf.d/ssl_certificate file looks something like:</p>
<pre><code>SSLCertificateFile /etc/apache2/ssl/serverwide.crt
SSLCertificateKeyFile /etc/apache2/ssl/serverwide.key
</code></pre>
<p>You can see that I've called my certificate and key files serverwide
to make it obvious that they are used by all domains served by this
server. It is very important that these files have their permissions
set so that only root can read them. You'll also need to make symlinks
from the ssl.conf and ssl.load files in your modsavailable directory
to your modsenabled directory.</p>
<p>Now we're ready to generate and sign our key and certificate. Once you
have generated the certificate you can inspect it's contents with this
command.</p>
<pre><code>openssl x509 in serverwide.crt noout text
</code></pre>
<p>To generate a key use the following command. You will be asked for a
pass phrase. It is important that you remember this pass phrase or
your key will be lost to you forever. Or at least until computers are
powerful enough to brute force crack your key.</p>
<pre><code>openssl genrsa des3 rand /dev/urandom out serverwide.key 1024
</code></pre>
<p>Once you have a key you can create and sign a certificate with the
following command. You will be asked for the pass phrase you entered
above. This is because the key is protected by that pass phrase and
can't be used without it. This certificate will be valid for one year.</p>
<pre><code>openssl req config server.config new key serverwide.key out serverwide.crt x509 days 365
</code></pre>
<p>Most of the options we need to pass to OpenSSL to create and sign the
certificate can be passed in a configuration file. The command line
above assumes the configuration file is called server.config. Below
is an example server.config file, the main lines of interest are in
the alt_names section. The alt_names section is where you can put all
of the virtually hosted domains on your server. The browser will look
for a match with any of those domains when the server passes it the
certificate we have just generated. I also found that subjectAltName
had to be in both the v3_req and v3_ca sections.</p>
<pre><code>[ req ]
default_bits = 1024
default_md = sha1
distinguished_name = req_distinguished_name
prompt = no
string_mask = nombstr
req_extensions = v3_req
x509_extensions = v3_ca
[ req_distinguished_name ]
countryName = &lt;country code&gt;
stateOrProvinceName = &lt;state&gt;
localityName = &lt;city&gt;
organizationName = &lt;whatever, could be your name&gt;
organizationalUnitName = &lt;again, whatever&gt;
commonName = www.domain1.com
emailAddress = webmaster@domain1.com
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[ v3_ca ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = www.domain1.com
DNS.2 = www.domain2.com
DNS.3 = www.domain3.com
</code></pre>
<p>The key that we have generated will have a pass phrase associated
with it (the one you entered when you generated the key). This pass
phrase will need to be entered every time you restart your apache
server. There are ways of removing this extra security, but if you
want to do that I'll let you look that one up elsewhere. For me with
a server running on a UPS, I reboot or restart apache a couple of
times a year at most. The added security is well woth it. If your
server get's compromised, and it will eventually, then your security
key will be coppied. And that is as they say, a bad thing.</p>
<p>And finally, in your site configuration file, probably in
/etc/apache2/sitesavailable, you will need to add the following:</p>
<pre><code><VirtualHost *:443>
ServerName www.domain1.com
ServerAlias domain1.com *.domain1.com
DocumentRoot /var/www/domain1
SSLEngine On
</VirtualHost>
</code></pre>
<p>This is the Virtual host configuration file for your secured site.
You'll want to add any additional configurations to it from your
normal *:80 configuration section.</p>

Virtual Hosting and Movable Type
http://socialhacker.com/posts/socialhacker/virtual_hosting_anton_mt/
Sat, 02 May 2009 11:33:15 0800
anton@socialhacker.com (Anton Staaf)
http://socialhacker.com/posts/socialhacker/virtual_hosting_anton_mt/
<p>It was a lot harder than I anticipated to get Movable Type to run
from a single global install on all of my virtually hosted domains.
So in the spirit of sharing, here's how I did it.</p>
<p>First, install movable type into a directory at the root of your web
servers directory structure. I put mine in /var/www/shared/. The
resulting directory structure contained:</p>
<dl>
<dt>/var/www/shared/cgibin/mt</dt>
<dd>
This directory contains pretty much everything in the Movable Type
tarball.
</dd>
<dt>/var/www/shared/mtstatic</dt>
<dd>
This directory is the mtstatic directory from the Movable Type
tarball. I moved it out to the top level of the shared directory
because I have Apache configured to not serve documents from the
cgibin directory.
</dd>
<dt>/var/www/shared/conf</dt>
<dd>
This directory will contain the configuration files for all of the
sites you will be virtually hosting.
</dd>
</dl>
<p>Next configure apache for each of your virtually hosted domains. I
assume you already have virtual hosting up and running. The
configuration file for socialhacker.com looks like this:</p>
<pre><code><VirtualHost *:80>
ServerName www.socialhacker.com
ServerAlias socialhacker.com *.socialhacker.com
DocumentRoot /var/www/socialhacker
AddHandler cgiscript .cgi
SetEnv MT_HOME /var/www/shared/cgibin/mt
SetEnv MT_CONFIG /var/www/shared/conf/socialhacker.cgi
ScriptAlias /cgibin/ /var/www/shared/cgibin/
Alias /mtstatic/ /var/www/shared/mtstatic/
</VirtualHost>
</code></pre>
<p>The important bits here are how you set up the alias' and the
environment variables. And all of this is covered in <a href="http://www.movabletype.org/documentation/enterprise/managingmultipleinstancesof.html">this article</a>.</p>
<p>Finally, you need to create a configuration file in the shared/conf
directry for your site. I did this by hand, which you'll need to do as
well since Movable Type won't set most of these options in the
configuration file it generates.</p>
<pre><code>CGIPath /cgibin/mt/
StaticWebPath /mtstatic/
StaticFilePath /var/www/shared/mtstatic/
PluginPath /var/www/shared/cgibin/mt/plugins/
TemplatePath /var/www/shared/cgibin/mt/tmpl/
WeblogTemplatesPath /var/www/shared/cgibin/mt/default_templates/
ObjectDriver DBI::mysql
Database <your database name>
DBUser <your database user>
DBPassword <your database password>
DBHost localhost
MailTransfer smtp
SMTPServer smtp.<your provider>.com _(probably)_
</code></pre>
<p>And there's the real magic. You have to specify paths to the Static
content as well as both Template directories, and the Plugin
directory. If you forget the first template directory you won't get
very far as the administrative pages won't load. But if you forget
the second template path you'll get all the way to making a post and
find that all of the files in your blog directory are zero length. If
you forget the Plugin path you'll be missing all of your plugins.</p>
<p>Hopefully you've found this useful in setting up your own Movable Type blogs.</p>