How to compare package version strings in bash

This is a little function I use to compare package version strings. Sometimes they can get complex with multiple different delimiters or strings in them. I cheated a bit by using sort –version-sort for the actual comparison. If you are looking for a pure bash version to compare simpler strings (e.g. compare 1.2.4 with 1.10.2), I’d suggest this stackoverflow posting. The function takes three parameters (the version strings and the comparison you want to apply) and uses the return code to signal if the result was valid or not. This gives the function a somewhat natural feel, for example compare_version 3.2.0-113.155 “<” 3.2.0-130.145 would return true. Aside from < and […]

Continue reading

Convert configuration files to ansible templates

I’ve been playing around with ansible a lot lately, and I noticed that while changing stuff from “installed and configured manually” to “installed and configured by ansible” I was running into quite a few configuration files that needed to be manually turned into templates. It can be quite tedious to replace values in a configuration file with placeholders and put all those placeholders in a .yml file with default values. Automating this is something I would have typically done in perl, but since I wanted to learn more about using regex in bash I decided to have a go at it in bash using regex and ${BASH_REMATCH} The script takes a configuration […]

Continue reading

How to install the latest Nmap for Debian/Ubuntu

A quick & dirty script to download the latest version of nmap (sourcecode) and generate a deb and install it (so that it’s correctly in the package management). Yes, I know this is not much more than a glorified configure && make && checkinstall

Continue reading

Bash snippet, verify ctrl+c

Lately I’ve been working on a pair of more elaborate scripts using ncat and openssl to transfer data between hosts. I’ll get around to posting it eventually, but until then a few small snippets that people may find useful. Today we will catch ctrl+c and ask the user if he really want’s to terminate the script.

The initialize() and cleanup() are my usual function names I have in every script, making sure general settings and variables are defined and that on exit any tempfiles get deleted. What has been added was a trap for the INT signal (ctrl+c) which calls the verify_quit() function, giving the user 10 seconds to […]

Continue reading

Simple “try” function for bash

Made a nice little try() function today for simplifying checking/dealing with  return codes from commands. It uses the function text() I posted earlier to colorfy output: How to easily add colored text output in bash scripts. The function accepts 2 parameters, how it should behave if a problem occurs and the command to be executed: try <silent|warn|fatal> command silent: save the return status in the global variable command_status warn: if the command has a return code > 0, print a warning and save the return status in the global variable command_status fatal: if the command has a return code > 0, print an error and exit Obviously not as versatile as a python try/except, bu streamlines verifying […]

Continue reading

How to break down CIDR subnets in Bash

I was playing around with subnets in bash recently and needed an elegant/easy way to split up a subnet into smaller subnets. First I used 2 functions I found on stackoverflow.com to convert an IP addresse to and from an integer. After that it was just a bit of math in bash to split up any networks too big. Any network larger than $maxSubnet gets split up. Here the useful code:

Output of script:

 

Continue reading

How to get the intersecting area of two polygons in MySQL

I was playing around with spatial features of MySQL this weekend and stumbled into a problem where I was looking for the area of two rectangles that overlap.  MySQL provides a function to check if they overlap, but no function to extract the region that overlaps. I’ve never written a stored routine in MySQL before, so I decided it would be a good exercise to try making one. As you can see the function is pretty straightforward and it assumes you are working with rectangles, but other than that it does what it is supposed to. You pass the function 2 polygons (e.g. Intersection(a.poly,b.poly)), and it returns the intersecting area […]

Continue reading

How to check if a IP (ipv4) address is valid in pure Bash

Here is a small bash function to check if a IP is valid (4 octets, each octet < 256). I find it somewhat elegant since instead of using a lot of case/if/then constructs or a crazy long regex it splits the IP into each octet (and stores them in an array, and then uses a combination of regex and bit shifting to check each octet.

The function will return 0 if the IP is valid, and 1 or higher if it encountered an error (you can check with the $? variable directly after calling the function) Example:

Continue reading

How to add locking to a shell script (the easy way)

I haven’t posted anything with bash here for a while, so today I’ll throw in a little snippet to use flock to make sure a script is only running once.  This is very handy in cron jobs that you want to run often, but there shouldn’t be multiple instances of the script running at the same time. Since it is small and easy I’d recommend adding it to any code you don’t want running multiple times since “that script” you just wrote, that runs 10 minutes now, might turn into a monster in 6 months and run 45 minutes when things change (data grows, more stuff to do).  Better safe than sorry. […]

Continue reading

How to increase Fraps performance with a ramdisk

I recently started playing Battlefield 3 and remembered that I have a Fraps license so I installed it and started recording some stuff. Unsurprisingly the performance made a big dip when I recorded. A glance at my PC told me the harddrive was at fault, probably bringing the whole system down due to IO. Since my PC has more than enough RAM I decided to set up a 5Gb Ramdisk to see if that helped. It did, when writing the video files to the ramdisk I hardly had any performance hit. Unfortunately 5GB isn’t going to last long while recording 1920×1080 @ 40FPS (a few minutes footage at most). Here is my little […]

Continue reading

How to build an efficient GeoIP SQL table

This here is a very handy little script I threw together to generate a geoip.sql table for quickly determining which country a IP is from. I already hear you saying “Just convert the IP to an INT and use BETWEEN, how hard can it be”. And you are right, that works. And it may even be your easiest solution, but it just isn’t fast. And if you are planning on hammering the table with thousands of queries you are going to end up looking for something fast. A while back I found a very interesting posting at www.jcole.us that described how to use Spacial Indexes together with MySQL’s GIS to speed up […]

Continue reading

How to easily add colored text output in bash scripts

Here is small snippet that can give your shell scripts some nice output: colortext.sh As with the debug.sh script, just download it to the same directory as your own script and add it with

It contains one simple function called text with the syntax text “text to be output”. Color can be red, green, yellow, blue or grey. The function does not automatically add a linebreak to the putput, so pop a \n in there if you need it. I prefer using it together with printf for clean and easy color output. Here are some examples of how the function can be used, and below the corresponding output:

Output: […]

Continue reading

How to add debugging to shellscripts

Debugging bash scripts is pretty straightforward, throwing around a couple echo and set -x quickly gives you what you need. But what if you want to add a nice breakpoint,  debugging to lots of paces in the code or turn all debugging on or off at once? Then this little script I wrote is the right thing for you: debug.sh just download it to the same directory as your script and include it with the following line:

It contains 4 simple functions that will make your bash coding easier. debug and breakpoint both print the argument with a timestamp to STDERR You can turn off all the functions by adding a […]

Continue reading

Disemvoweling

Talk about weird words … ok, according to Wikipedia disemvoweling is the term for replacing or removing vowels from words. Commonly used as a tool for moderating.  I’m pretty sure everyone has run across  certain disemvoweled  words on the internet like f*ck or sh*t. Anyway I went and made a pure html/javascript page that does just that, removes any vowels from an inputted text. The usefullness can certainly be argued, it was more for me to brush up on my javascript and css skills. http://www.dopefish.de/projects/disemvoweling/

Continue reading

captcha cracking

This is a pretty old posting from 2009 I just recently discovered in my “drafts” directory. Nowadays there are probably easier and more elegant ways of defeating a captcha, but for old times sake, here is my simple approach. ———————– Eclectic and Marko were so kind as to “provide” me a captcha to play around with. Took me a few days of poking around and googling but in the end it was easier than I had thought. As long as there aren’t and logic errors in the code (e.g. bad or no session handling) you probably won’t get around some kind of OCR. As OCR software I decided to use […]

Continue reading

Wireshark remote capturing

yeah, this is real simple stuff, not really worth writing a script for it. but on the other hand it saves me from remembering how to do it every time I need it (which isn’t often). So here is a little script to setup remote capturing with wireshark. All it basically does is ssh to the remote host and tcpdump sucking the output via stdout through the ssh connection to a local pipe, that is then used by wireshark to display the stream. Because of this you may want to make sure you aren’t capturing your own ssh data when doing this 😉

Continue reading

MySQL selecting IPs via CIDR

Quick little snippet here for selecting IPs from a database based off a CIDR subnet. First off a table structure with some test data:

Now let’s say we want all IPs from the subnet 173.192.175.16/28, using a simple 173.192.175.% would provide false results since you don’t want the whole /24. If your IP is stored as an unsigned int (good for you) than you can use this snippet to search for matching IPs:

If your IP is stored as a varchar (for whatever reason), the only difference is a inet_aton() around the IP field.

No matter which one you use, the result will be:

Continue reading

Controlling SSH identities

SSH has a few strange undocumented “features”. One of which is the way it handles identities via agent and command line. It is possible to specify an identity file to use for ssh via the -i parameter (ssh -i identity_file $host). What the manpage doesn’t mention, is that the specified identity isn’t forced for the connection, it is just added to the list of possible identities. To make matters worse, ssh tries the identities from the agent first. So if agent forwarding is enabled and valid for the destination the ssh command will never use the identity specified with -i. Why is this “bad”? Because the identity specified may be […]

Continue reading

Script of the day – clean up stale .ssh/known_hosts

This little script takes an IP or hostname as a parameter, and if there is an offending key in the .ssh/known_hosts it removes it and replaces it with the current valid one useful if you are moving/reinstalling a large amount of servers …

Continue reading

bash: using the content of a variable as variable name

Since the implementation of Arrays in Bash is somewhat lacking compared to higher level programming languages (only one-dimensional), and hash lists require a bit of work to set up, you may run into a situation where you have a small list of key/value pairs that are both variable and you need to store. There are various solutions for the problem, e.g. creating two arrays (one for the keys, one for the values, and combining them by using the same index values for the entries), or using the functions from the link above to build a hash list. For me the easiest way to solve the problem, if I only have […]

Continue reading