Posted in Programming

How to install the latest Nmap for Debian/Ubuntu

A quick & dirty script to download the latest version of nmap (sourcecode) and generate a deb and install it (so that it’s correctly in the package management). Yes, I know this is not much more than a glorified configure && make && checkinstall

Posted in Programming

Bash snippet, verify ctrl+c

Lately I’ve been working on a pair of more elaborate scripts using ncat and openssl to transfer data between hosts. I’ll get around to posting it eventually, but until then a few small snippets that people may find useful.

Today we will catch ctrl+c and ask the user if he really want’s to terminate the script.

The initialize() and cleanup() are my usual function names I have in every script, making sure general settings and variables are defined and that on exit any tempfiles get deleted.
What has been added was a trap for the INT signal (ctrl+c) which calls the verify_quit() function, giving the user 10 seconds to press ctrl+c again to exit (via cleanup()) or return back to wherever we were in the code. There is one unavoidable caveat, the first ctrl+c will kill whatever the script was doing before it jumps into the verify_quit() function.

Posted in Programming

Simple “try” function for bash

Made a nice little try() function today for simplifying checking/dealing with  return codes from commands. It uses the function text() I posted earlier to colorfy output: How to easily add colored text output in bash scripts. The function accepts 2 parameters, how it should behave if a problem occurs and the command to be executed: try <silent|warn|fatal> command

silent: save the return status in the global variable command_status
warn: if the command has a return code > 0, print a warning and save the return status in the global variable command_status
fatal: if the command has a return code > 0, print an error and exit

Obviously not as versatile as a python try/except, bu streamlines verifying the command return codes.
Example Usage:

Output
Warning: ‘false‘ failed with return code –1
ls: cannot access doesnotexist: No such file or directory
Error: ‘ls -al doesnotexist‘ failed with return code –2

File: error_handling.sh

Posted in Internet Stuff

Virtualbox to 4.2.12 problems with Ubuntu

After updating Virtualbox to 4.2.12 and updating the guest additions on my Ubuntu 12.10 the guest suddenly started having problems with the graphics driver (X.org wouldn’t start correctly, only starts in “rescue” mode). I did a bit of googling, and it seems other users are also affected (mainly 12.04 and 12.10 Ubuntu), and it seems to affect people who upgraded from virtualbox guest addditions 4.2.10 -> 4.2.12

Here the quick and dirty workaround to the problem: download the 4.2.10 guest additions and use them.

Will need to reboot or make sure the new modules are loaded and restart X afterwards.

Posted in Photography

Picture galleries from last fall

This weekend I finally got around to uploading some pictures from our trip last fall to the Grand Canyon, Las Vegas (no pics), Death Valley, and Sequoia & Kings Canyon National Parks.
The cave pictures at sequoia national park are from crystal cave.

[Not a valid template] [Not a valid template] [Not a valid template]

 

Posted in Internet Stuff, Security, Tech

Script to start minion in tmux

Minion is a security project from Mozilla (link). It provides a user-friendly web interface to various security scanner tools. There is a webcast demonstrating the software (link).

The software requires a few services to run, and since I like having one script take care of starting everything with the right parameters, I threw together a simple shell script that sets up a tmux session with the services started in windows with the names of the services.

Posted in Internet Stuff, Programming

How to break down CIDR subnets in Bash

I was playing around with subnets in bash recently and needed an elegant/easy way to split up a subnet into smaller subnets. First I used 2 functions I found on stackoverflow.com to convert an IP addresse to and from an integer. After that it was just a bit of math in bash to split up any networks too big.
Any network larger than $maxSubnet gets split up.
Here the useful code:

Output of script:

 

Posted in Internet Stuff, Programming

How to get the intersecting area of two polygons in MySQL

I was playing around with spatial features of MySQL this weekend and stumbled into a problem where I was looking for the area of two rectangles that overlap.  MySQL provides a function to check if they overlap, but no function to extract the region that overlaps.

I’ve never written a stored routine in MySQL before, so I decided it would be a good exercise to try making one. As you can see the function is pretty straightforward and it assumes you are working with rectangles, but other than that it does what it is supposed to.
You pass the function 2 polygons (e.g. Intersection(a.poly,b.poly)), and it returns the intersecting area as a new polygon.

Example comparing some rectangles in 2 tables using the function:

Result:

 

Posted in Photography

Picture galleries from trips this summer

Since we are going on a small vacation tomorrow I wanted to get a couple picture galleries up before I start filling my drive up with new pictures. These are all from small trips/excursions we took over the summer. Unfortunately no pictures from the canoe on the st. croix river.

[Not a valid template] [Not a valid template] [Not a valid template]
Posted in Server, Tech

How to find the fingerprints of public keys in authorized_keys

If you use keys for SSH authentication (and you should) then you have probably run into the situation that the auth.log shows that someone logged in, even which local user was used (e.g. root), but you have no idea which of the keys in ~/.ssh/autorized_keys was used. The first step you can do to see what is going on, is increasing the log level of the SSH daemon:

/etc/ssh/sshd_config

That will spit out the fingerprint of the SSH key used to log in. Example log entry for a successful login:

Now that we have the fingerprint of the ssh key used to login, we will need ssh-keygen to spit out the fingerprints of the public keys in ~/.ssh/authorized_keys to be able to compare them. So I wrote a little wrapper called ssh-fingerprint.sh around ssh-keygen to feed it all the public keys from authorized_keys (if you want you can even fit the whole while loop as a oneliner):

Posted in Programming

How to check if a IP (ipv4) address is valid in pure Bash

Here is a small bash function to check if a IP is valid (4 octets, each octet < 256). I find it somewhat elegant since instead of using a lot of case/if/then constructs or a crazy long regex it splits the IP into each octet (and stores them in an array, and then uses a combination of regex and bit shifting to check each octet.

The function will return 0 if the IP is valid, and 1 or higher if it encountered an error (you can check with the $? variable directly after calling the function)
Example:

Posted in Me

silica.dopefish.de – catching data leaks

I was testing a PHP script for security flaws today and needed a way to check if it was possible to “accidently” leak sensitive data to an external server. So I threw together a small script to save the contents of the $_SERVER $_COOKIE $POST and $_GET arrays and dump them in a file. Since it worked so good I decided to leave the site online.

Why did I call it silica? Because silica gel absorbs and stores fluids like this website absorbs information thrown at it. How to use it? Just get your application to connect to silica.dopefish.de instead of where it intended to connect to (by overwriting variables, DNS foo, be creative) and see what happens.

Base Domain: silica.dopefish.de
The webserver will catch any URL (regardless what path or filename) and log the environment. The only exception is the logfile.

Logfile: http://silica.dopefish.de/access.log
The log is automatically emptied every 10 minutes, so save the output if you need it.

Posted in Internet Stuff, Security

Updated OSSEC Web UI 0.3 files for OSSEC 2.6

OSSEC is an open source HIDS (Host-based Intrusion Detection System), and a pretty darn good one too. It also has a simple web front-end to view what’s going on, search through alerts and stuff like that  (called OSSEC Web UI, I’ll just call it “WUI” here). Unfortunately the code is a bit outdated (the last official update was from 2008 as far as I can tell) and it doesn’t support newer features of OSSEC like polling data from a database. Something I’d like to tackle if I find the time 😉

The latest version of OSSEC is 2.6, and due to some small changes to the format of the logs WUI no longer works out-of-the-box. I had a look at the code this weekend and am providing patches and downloads of the files needed to change to get everything running again with OSSEC 2.6.

List of changes:

  • Works with the OSSEC 2.6 alert log file format
  • Changed Rule ID Link to better work with the new documentation wiki
  • Added “user” field to alert output
  • Widened the layout by a few pixels (to 1000px) and changed the CSS / alert layout to make the individual alerts better readable
  • Moved some of the hardcoded formatting to CSS

Download Download changed files
Download Patch download

 

Posted in Tech

Wireshark Version 1.8.0 released

A new version of Wireshark was released yesterday (1.8.0), head on over to http://www.wireshark.org/download.html to download it for your OS.

New and updated features since version 1.6:

  • Wireshark supports capturing from multiple interfaces at once.
  • You can now add, edit, and save packet and capture file annotations.
  • Wireshark, TShark, and their associated utilities now save files using the pcap-ng file format by default. (Your copy of Wireshark might still use the pcap file format if pcap-ng is disabled in your preferences.)
  • Decryption key management for IEEE 802.11, IPsec, and ISAKMP is easier.
  • OID resolution is now supported on 64-bit Windows.
  • The “Save As” menu item has been split into “Save As”, which lets you save a file using a different filename and “Export Specified Packets”, which lets you have more control over which packets are saved.
  • TCP fast retransmissions are now indicated as an expert info note, rather than a warning, just as TCP retransmissions are.
  • TCP window updates are no longer colorized as “Bad TCP”.
  • TShark’s command-line options have changed. The previously undocumented -P option is now -2 option for performing a two-pass analysis; the former -S option is now the -P option for printing packets even if writing to a file, and the -S option is now used to specify a different line separator between packets.
  • GeoIP IPv6 databases are now supported.
Posted in Me

How to use Cluster SSH (cssh) and Mosh together

A colleague told me about mosh today, just saying “it’s an alternative to SSH that addresses many of the problems” doesn’t do it credit. Go look at the page and have a look, don’t worry it’s a nice page and I’ll still be here waiting when you come back. I do a lot of work on servers across the globe, and believe me, although subtle, the difference between a 10ms connection and a 100+ ms connection is definitely noticeable 😉 That lag all but disappears with mosh. Mosh doesn’t support X11/SSH-Agent forwarding yet, but it’s on their roadmap.

Back to topic, I also use cssh for working on multiple servers simultaneously. I’d recommend making a copy of your .cshrc file (cp -a ~/.csshrc ~/.csshrc_mosh) and working on the copy. Edit the  ~/.csshrc_mosh and change the lines ssh= and ssh_args= (diff of my files):

You will have to remove any parameters in ssh_args= and move them to your ~/.ssh/config due to the difference in command line parameters between mosh and ssh. Now all you have to do is call cssh with the new config file, e.g. with an alias to make things easier alias mcssh=”cssh -C ${HOME}/.csshrc_mosh” And your done.

Posted in Internet Stuff

Playing around with Exim, Spamassassin, DKIM and SPF

I was playing around with my mail server today and decided to have a look at DKIM and SPF. Jump to the bottom for some useful links. You will have to be able to change TXT entries for your domain/subdomains to do any of this. Sending a mail to check-auth@verifier.port25.com will generate an auto-reply with the result of your mail servers settings. I also noticed that Exim wasn’t listening to the IPv6 interface and promptly fixed that, and have received my first mail via IPv6.

Some spamassassin scores for your local/user settings:

SPF:
Syntax: http://www.openspf.org/SPF_Record_Syntax
Tools & Information: http://www.openspf.org/
Wizard: http://www.spfwizard.net/

DKIM:
Exim walkthrough: http://mikepultz.com/2010/02/using-dkim-in-exim/
Domain specific settings: http://www.debian-administration.org/users/lee/weblog/41
Tools: http://dkimcore.org/tools/
Wizard: https://www.port25.com/dkim-wizard/

Posted in Server, Tech

New Virtualbox version, script for easy update of extension pack

VirtualBox 4.1.10 was released yesterday with a few nice things in the changelog. Updating virtualbox itself is easy, just download the package and update it. Since I seem to stumble over the update of the extension pack every time (on my headless system) I thought I’d write a small script this time so I don’t have to rethink it next time (automatically downloads and installs the current extpack):

 

Posted in Gaming

Floor plans for PAYDAY Map “Diamond Heist”

I’ve played PAYDAY the last few weeks, it is a 4 player CO-OP FPS (think of Left 4 Dead in a different setting). One of the maps has caused me a bit of difficulty (Diamond Heist). There is an achievement called “Are Those The Blue Ones?” where you have to collect all the sapphires before the alarm goes off, and I kept forgetting where they were. Also keeping track of the cameras was a bit of a pain, so today I threw them all together on a map for easier reference (right-click “Save link as …” to download):

Graphics sources:
http://www.overkillsoftware.com/payday/
http://openclipart.org/detail/125695/diamant–diamond-by-lmproulx
http://openclipart.org/detail/31891/surveillance-camera-by-maxim2

Posted in Internet Stuff, Server

New Server (with IPv6)

I got a new server and I finished moving everything to the new hardware this week. If you notice anything behaving strange just drop me a note. I also set up IPv6 on all my VMs so will be playing around with that a bit. It’s a shame that hardly any providers give customers IPv6 addresses, migrating from IPv4 to IPv6 will take forever if none of the users can use them (and in the meantime IPv4 addresses are getting more and more expensive for me).

The new server is from hetzner since I have been very satisfied with them in the past. Impressive hardware features for low prices and (what really makes me keep choosing them) fantastic technical support. I’ve been with a few companies in the past, but none have had as good support as hetzner. Sooner or later hardware will die, and having a hosting provider that responds promptly and knowledgeably to request is priceless.

Posted in Programming, Tech

How to add locking to a shell script (the easy way)

I haven’t posted anything with bash here for a while, so today I’ll throw in a little snippet to use flock to make sure a script is only running once.  This is very handy in cron jobs that you want to run often, but there shouldn’t be multiple instances of the script running at the same time.
Since it is small and easy I’d recommend adding it to any code you don’t want running multiple times since “that script” you just wrote, that runs 10 minutes now, might turn into a monster in 6 months and run 45 minutes when things change (data grows, more stuff to do).  Better safe than sorry.

Basically we rely on flock to do the heavy lifting and we just add some logic around it:

man flock will show you more details to the parameters used and even some examples. Basically it will use trap to make sure the lock is released if the script ends in any way. 200 is a random file descriptor I chose for this example, it can be anything numeric. flock -xn means it will attempt to acquire an exclusive lock, and if that fails it will exit with an error.

Putting this somewhere at the top of your script will simply end the script if it finds an existing lock. flock has a few other options like -wait or nor using -n that allow you to not exit but wait for the lock to end (with wait a variable amount of seconds). And thus with a bit of creativity enabling you to only lock specific parts of the code (e.g. database calls, file changes, …) and handling failed lock attempts more gracefully than an exit.