Ryan Schulze

Virtualbox to 4.2.12 problems with Ubuntu

April 16, 2013April 16, 2013Ryan12.04, 12.10, 4.2.10, 4.2.12, ubuntu, virtualbox

After updating Virtualbox to 4.2.12 and updating the guest additions on my Ubuntu 12.10 the guest suddenly started having problems with the graphics driver (X.org wouldn’t start correctly, only starts in “rescue” mode). I did a bit of googling, and it seems other users are also affected (mainly 12.04 and 12.10 Ubuntu), and it seems to affect people who upgraded from virtualbox guest addditions 4.2.10 -> 4.2.12

Here the quick and dirty workaround to the problem: download the 4.2.10 guest additions and use them.

wget http://download.virtualbox.org/virtualbox/4.2.10/VBoxGuestAdditions_4.2.10.iso
mount -o loop VBoxGuestAdditions_4.2.10.iso /mnt
cd /mnt/
bash VBoxLinuxAdditions.run

wget http://download.virtualbox.org/virtualbox/4.2.10/VBoxGuestAdditions_4.2.10.iso

mount -o loop VBoxGuestAdditions_4.2.10.iso /mnt

cd /mnt/

bash VBoxLinuxAdditions.run

Will need to reboot or make sure the new modules are loaded and restart X afterwards.

Picture galleries from last fall

March 24, 2013March 24, 2013Ryancave, death valley, grand canyon, national park, pictures, sequoia

This weekend I finally got around to uploading some pictures from our trip last fall to the Grand Canyon, Las Vegas (no pics), Death Valley, and Sequoia & Kings Canyon National Parks.
The cave pictures at sequoia national park are from crystal cave.

[album id=34 template=dopefish]

[album id=35 template=dopefish]

[album id=36 template=dopefish]

Script to start minion in tmux

February 15, 2013Ryanbash, minion, scripting, security, tmux

Minion is a security project from Mozilla (link). It provides a user-friendly web interface to various security scanner tools. There is a webcast demonstrating the software (link).

The software requires a few services to run, and since I like having one script take care of starting everything with the right parameters, I threw together a simple shell script that sets up a tmux session with the services started in windows with the names of the services.

#!/bin/bash

session="minion"
http="0.0.0.0:8000"

cd /opt/minion

#-------------------------------------------------------------------------------

if [[ "$(tmux list-sessions|cut -d: -f1|grep "^${session}$")" == "${session}" ]]
then
  tmux -2 attach-session -t "${session}"
  exit
fi
tmux new-session -s ${session} -d

window="${session}:1"
tmux send-keys -t "${window}" 'top' Enter

window="${session}:2"
tmux new-window -t "${window}" -n "task-engine"
tmux send-keys -t "${window}" 'source env/bin/activate' Enter
tmux send-keys -t "${window}" 'minion-task-engine --debug' Enter

window="${session}:3"
tmux new-window -t "${window}" -n "plugin-service"
tmux send-keys -t "${window}" 'source env/bin/activate' Enter
tmux send-keys -t "${window}" 'minion-plugin-service --debug' Enter

window="${session}:4"
tmux new-window -t "${window}" -n "frontend"
tmux send-keys -t "${window}" 'source env/bin/activate' Enter
tmux send-keys -t "${window}" 'cd frontend' Enter
tmux send-keys -t "${window}" 'python manage.py syncdb' Enter
tmux send-keys -t "${window}" "python manage.py runserver ${http}" Enter

tmux -2 attach-session -t "${session}"

#!/bin/bash

session="minion"

http="0.0.0.0:8000"

cd /opt/minion

#-------------------------------------------------------------------------------

if [[ "$(tmux list-sessions|cut -d: -f1|grep "^${session}$")" == "${session}" ]]

then

tmux -2 attach-session -t "${session}"

exit

tmux new-session -s ${session} -d

window="${session}:1"

tmux send-keys -t "${window}" 'top' Enter

window="${session}:2"

tmux new-window -t "${window}" -n "task-engine"

tmux send-keys -t "${window}" 'source env/bin/activate' Enter

tmux send-keys -t "${window}" 'minion-task-engine --debug' Enter

window="${session}:3"

tmux new-window -t "${window}" -n "plugin-service"

tmux send-keys -t "${window}" 'source env/bin/activate' Enter

tmux send-keys -t "${window}" 'minion-plugin-service --debug' Enter

window="${session}:4"

tmux new-window -t "${window}" -n "frontend"

tmux send-keys -t "${window}" 'source env/bin/activate' Enter

tmux send-keys -t "${window}" 'cd frontend' Enter

tmux send-keys -t "${window}" 'python manage.py syncdb' Enter

tmux send-keys -t "${window}" "python manage.py runserver ${http}" Enter

tmux -2 attach-session -t "${session}"

How to break down CIDR subnets in Bash

December 11, 2012September 27, 2014Ryanbash, cidr, ip, math, subnet

I was playing around with subnets in bash recently and needed an elegant/easy way to split up a subnet into smaller subnets. First I used 2 functions I found on stackoverflow.com to convert an IP addresse to and from an integer. After that it was just a bit of math in bash to split up any networks too big.
Any network larger than $maxSubnet gets split up.
Here the useful code:

#!/bin/bash
. functions_ip2long.sh

input='10.1.4.0/22 10.2.2.0/24 10.3.3.128/25'
output=
maxSubnet=24

for pair in ${input}
do
  # split up the network and netmask
  network=${pair%/*}
  netmask=${pair#*/}
  if [[ ${netmask} -ge ${maxSubnet} ]]
  then # network is smaller than max. size. add it to the array
    output="${output} ${network}/${netmask}"
  else # network is too big. split it up into smaller chunks and add them to the array
    for i in $(seq 0 $(( $(( 2 ** (${maxSubnet} - ${netmask}) )) - 1 )) )
    do # convert network to long integer, add n * ${maxSubnet} and then convert back to string
      output="${output} $( INET_NTOA $(( $( INET_ATON ${network} ) + $(( 2 ** ( 32 - ${maxSubnet} ) * ${i} )) )) )/${maxSubnet}"
    done
  fi
done

echo ${output}|tr ' ' '\n'

#!/bin/bash

. functions_ip2long.sh

input='10.1.4.0/22 10.2.2.0/24 10.3.3.128/25'

output=

maxSubnet=24

for pair in ${input}

# split up the network and netmask

network=${pair%/*}

netmask=${pair#*/}

if [[ ${netmask} -ge ${maxSubnet} ]]

then # network is smaller than max. size. add it to the array

output="${output} ${network}/${netmask}"

else # network is too big. split it up into smaller chunks and add them to the array

for i in $(seq 0 $(( $(( 2 ** (${maxSubnet} - ${netmask}) )) - 1 )) )

do # convert network to long integer, add n * ${maxSubnet} and then convert back to string

output="${output} $( INET_NTOA $(( $( INET_ATON ${network} ) + $(( 2 ** ( 32 - ${maxSubnet} ) * ${i} )) )) )/${maxSubnet}"

done

echo ${output}|tr ' ' '\n'

Output of script:

10.1.4.0/24
10.1.5.0/24
10.1.6.0/24
10.1.7.0/24
10.2.2.0/24
10.3.3.128/25

10.1.4.0/24

10.1.5.0/24

10.1.6.0/24

10.1.7.0/24

10.2.2.0/24

10.3.3.128/25

How to get the intersecting area of two polygons in MySQL

December 3, 2012September 27, 2014RyanGeometric Objects, Intersection, MySQL, polygon, spatial, stored procedure

I was playing around with spatial features of MySQL this weekend and stumbled into a problem where I was looking for the area of two rectangles that overlap. MySQL provides a function to check if they overlap, but no function to extract the region that overlaps.

I’ve never written a stored routine in MySQL before, so I decided it would be a good exercise to try making one. As you can see the function is pretty straightforward and it assumes you are working with rectangles, but other than that it does what it is supposed to.
You pass the function 2 polygons (e.g. Intersection(a.poly,b.poly)), and it returns the intersecting area as a new polygon.

DROP FUNCTION IF EXISTS Intersection;
DELIMITER //
CREATE FUNCTION Intersection(g1 POLYGON, g2 POLYGON) RETURNS POLYGON DETERMINISTIC
BEGIN
  DECLARE minx1, minx2, maxx1, maxx2 DOUBLE;
  DECLARE miny1, miny2, maxy1, maxy2 DOUBLE;
  DECLARE miny,  maxy,  minx,  maxx DOUBLE;
  SET minx1 =    LEAST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));
  SET maxx1 = GREATEST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));
  SET miny1 =    LEAST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));
  SET maxy1 = GREATEST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));
  SET minx2 =    LEAST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));
  SET maxx2 = GREATEST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));
  SET miny2 =    LEAST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));
  SET maxy2 = GREATEST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));
  SET minx = IF ( minx2 > minx1 , minx2 , minx1 );
  SET maxx = IF ( maxx2 > maxx1 , maxx1 , maxx2 );
  SET miny = IF ( miny2 > miny1 , miny2 , miny1 );
  SET maxy = IF ( maxy2 > maxy1 , maxy1 , maxy2 );
  RETURN POLYGON(LINESTRING(POINT(minx,miny),POINT(maxx,miny), POINT(maxx,maxy), POINT(minx,maxy),POINT(minx,miny)));
END //
DELIMITER ;

DROP FUNCTION IF EXISTS Intersection;

DELIMITER //

CREATE FUNCTION Intersection(g1 POLYGON, g2 POLYGON) RETURNS POLYGON DETERMINISTIC

BEGIN

DECLARE minx1, minx2, maxx1, maxx2 DOUBLE;

DECLARE miny1, miny2, maxy1, maxy2 DOUBLE;

DECLARE miny, maxy, minx, maxx DOUBLE;

SET minx1 = LEAST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));

SET maxx1 = GREATEST(X(PointN(ExteriorRing(g1),1)),X(PointN(ExteriorRing(g1),2)),X(PointN(ExteriorRing(g1),3)),X(PointN(ExteriorRing(g1),4)));

SET miny1 = LEAST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));

SET maxy1 = GREATEST(Y(PointN(ExteriorRing(g1),1)),Y(PointN(ExteriorRing(g1),2)),Y(PointN(ExteriorRing(g1),3)),Y(PointN(ExteriorRing(g1),4)));

SET minx2 = LEAST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));

SET maxx2 = GREATEST(X(PointN(ExteriorRing(g2),1)),X(PointN(ExteriorRing(g2),2)),X(PointN(ExteriorRing(g2),3)),X(PointN(ExteriorRing(g2),4)));

SET miny2 = LEAST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));

SET maxy2 = GREATEST(Y(PointN(ExteriorRing(g2),1)),Y(PointN(ExteriorRing(g2),2)),Y(PointN(ExteriorRing(g2),3)),Y(PointN(ExteriorRing(g2),4)));

SET minx = IF ( minx2 > minx1 , minx2 , minx1 );

SET maxx = IF ( maxx2 > maxx1 , maxx1 , maxx2 );

SET miny = IF ( miny2 > miny1 , miny2 , miny1 );

SET maxy = IF ( maxy2 > maxy1 , maxy1 , maxy2 );

RETURN POLYGON(LINESTRING(POINT(minx,miny),POINT(maxx,miny), POINT(maxx,maxy), POINT(minx,maxy),POINT(minx,miny)));

END //

DELIMITER ;

Example comparing some rectangles in 2 tables using the function:

CREATE TABLE `poly1` (
  `id` int(10) unsigned NOT NULL,
  `poly` polygon NOT NULL,
  PRIMARY KEY (`id`),
  SPATIAL KEY `poly` (`poly`)
) ENGINE=MyISAM;

CREATE TABLE `poly2` (
  `id` int(10) unsigned NOT NULL,
  `poly` polygon NOT NULL,
  PRIMARY KEY (`id`),
  SPATIAL KEY `poly` (`poly`)
) ENGINE=MyISAM;

-- Insert some test data into the tables
insert into poly1 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));
insert into poly1 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(6,0),POINT(10,0),POINT(10,2),POINT(6,2),POINT(6,0)))));
insert into poly1 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(10,10),POINT(20,10),POINT(20,5),POINT(10,5),POINT(10,10)))));

insert into poly2 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));
insert into poly2 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(7,0), POINT(7,2), POINT(3,2),POINT(3,0)))));
insert into poly2 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(7,0), POINT(7,2), POINT(5,2),POINT(5,0)))));
insert into poly2 (id,poly) VALUES (4, GEOMFROMWKB(POLYGON(LINESTRING(POINT(8,0),POINT(10,0),POINT(10,2),POINT(8,2),POINT(8,0)))));
insert into poly2 (id,poly) VALUES (5, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(6,0), POINT(6,2), POINT(3,2),POINT(3,0)))));
insert into poly2 (id,poly) VALUES (6, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(11,0),POINT(11,2),POINT(5,2),POINT(5,0)))));
insert into poly2 (id,poly) VALUES (7, GEOMFROMWKB(POLYGON(LINESTRING(POINT(12,8),POINT(18,8),POINT(18,4),POINT(12,4),POINT(12,8)))));

SELECT
 a.id as 'poly1',
 b.id as 'poly2',
 cast(100/Area(a.poly)*Area(Intersection(a.poly,b.poly)) as decimal(4,2)) as '% of poly2 in poly1'
FROM
 poly1 a,
 poly2 b
WHERE
 MBROverlaps(b.poly,a.poly);

CREATE TABLE `poly1` (

`id` int(10) unsigned NOT NULL,

`poly` polygon NOT NULL,

PRIMARY KEY (`id`),

SPATIAL KEY `poly` (`poly`)

) ENGINE=MyISAM;

CREATE TABLE `poly2` (

`id` int(10) unsigned NOT NULL,

`poly` polygon NOT NULL,

PRIMARY KEY (`id`),

SPATIAL KEY `poly` (`poly`)

) ENGINE=MyISAM;

-- Insert some test data into the tables

insert into poly1 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));

insert into poly1 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(6,0),POINT(10,0),POINT(10,2),POINT(6,2),POINT(6,0)))));

insert into poly1 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(10,10),POINT(20,10),POINT(20,5),POINT(10,5),POINT(10,10)))));

insert into poly2 (id,poly) VALUES (1, GEOMFROMWKB(POLYGON(LINESTRING(POINT(1,0),POINT(4,0), POINT(4,2), POINT(1,2),POINT(1,0)))));

insert into poly2 (id,poly) VALUES (2, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(7,0), POINT(7,2), POINT(3,2),POINT(3,0)))));

insert into poly2 (id,poly) VALUES (3, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(7,0), POINT(7,2), POINT(5,2),POINT(5,0)))));

insert into poly2 (id,poly) VALUES (4, GEOMFROMWKB(POLYGON(LINESTRING(POINT(8,0),POINT(10,0),POINT(10,2),POINT(8,2),POINT(8,0)))));

insert into poly2 (id,poly) VALUES (5, GEOMFROMWKB(POLYGON(LINESTRING(POINT(3,0),POINT(6,0), POINT(6,2), POINT(3,2),POINT(3,0)))));

insert into poly2 (id,poly) VALUES (6, GEOMFROMWKB(POLYGON(LINESTRING(POINT(5,0),POINT(11,0),POINT(11,2),POINT(5,2),POINT(5,0)))));

insert into poly2 (id,poly) VALUES (7, GEOMFROMWKB(POLYGON(LINESTRING(POINT(12,8),POINT(18,8),POINT(18,4),POINT(12,4),POINT(12,8)))));

SELECT

a.id as 'poly1',

b.id as 'poly2',

cast(100/Area(a.poly)*Area(Intersection(a.poly,b.poly)) as decimal(4,2)) as '% of poly2 in poly1'

FROM

poly1 a,

poly2 b

WHERE

MBROverlaps(b.poly,a.poly);

Result:

poly1	poly2	% of poly2 in poly1
1	2	33.33
1	5	33.33
2	2	25.00
2	3	25.00
3	7	36.00

poly1 poly2 % of poly2 in poly1

1 2 33.33

1 5 33.33

2 2 25.00

2 3 25.00

3 7 36.00