How to compare package version strings in bash

This is a little function I use to compare package version strings. Sometimes they can get complex with multiple different delimiters or strings in them. I cheated a bit by using sort –version-sort for the actual comparison. If you are looking for a pure bash version to compare simpler strings (e.g. compare 1.2.4 with 1.10.2), I’d suggest this stackoverflow posting. The function takes three parameters (the version strings and the comparison you want to apply) and uses the return code to signal if the result was valid or not. This gives the function a somewhat natural feel, for example compare_version 3.2.0-113.155 “<” 3.2.0-130.145 would return true. Aside from < and […]

Continue reading

How to easily switch between ansible versions

Lately I’ve run into issues with different versions of ansible (1.9 handling async better, 2.x having more modules and handling IPv6 better) and having to test playbooks and roles against different versions to make sure they work. TO make life easier I put this little function in my .bashrc to switch back and forth between ansible versions. It checks out the specified version from github if it needs to, and switches over to it (just for that terminal, not the system). Usage is straight forward ansible_switch <branch> , i.e. ansible_switch 2.1  (or whatever branch you want, here is a list of all branches). It is currently limited to stable branches, but you can change line […]

Continue reading

A script to diff files/directories on two different servers

Ok,  short one today. This is a straightforward script that simplifies comparing directories on different servers. There is no magic in it, it just rsyncs the directories to a local temp directory and runs diff against them (then deletes the directory afterwards). Mainly intended for config files, I wouldn’t recommend trying to diff gigabytes of binaries with it.

 

Continue reading

Renewing “Let’s Encrypt” SSL certificates

Let’s Encrypt provides free DV SSL certificates for everyone and is now in the open beta phase. I’m not going to go into the details of which of the clients are best, since that depends entirely on your use case (I use acme-tiny and a rule in varnish to intercept all calls to /.well-known/acme-challenge/). Since the certificates are only valid for 90 days, I often see people suggesting to just renew them via cronjob every 2 months. I find this to be really awful advice, if that renewal fails for any reasons (network problems, local problems, problems with let’s encrypt) the next renewal is a month after the certificate expired. It is also […]

Continue reading

Convert configuration files to ansible templates

I’ve been playing around with ansible a lot lately, and I noticed that while changing stuff from “installed and configured manually” to “installed and configured by ansible” I was running into quite a few configuration files that needed to be manually turned into templates. It can be quite tedious to replace values in a configuration file with placeholders and put all those placeholders in a .yml file with default values. Automating this is something I would have typically done in perl, but since I wanted to learn more about using regex in bash I decided to have a go at it in bash using regex and ${BASH_REMATCH} The script takes a configuration […]

Continue reading

How to prevent changes to a tag via svn hook

A colleague of mine recently asked if it was possible to keep people from committing changes to tags in subversion. I thought “Hey, that should be easy to do via the pre-commit hook. I bet someone already made one that I can just test and use“. Either my google-fu failed me or the request wasn’t as common as I had anticipated, because surprisingly I couldn’t find any hooks that truly accomplish blocking changes to a tag (probably right after I post this someone will say “hey, why didn’t you look $here, it is exactly what you wanted“). I found people looking for such a feature, and I found a hook […]

Continue reading

Native tcp/udp sockets in bash cheatsheet

Bash has a nifty feature to open tcp and udp connections and read/write data to that connection. The Advanced Bash Scripting Guide and the bash man page offers some information and examples, and google has some odd examples, but all in all there isn’t much variety of information on the internet on the topic. This feature is enabled in Bash at compile time with the flag –enable-net-redirections It works by assigning a connection to a file descriptor with exec. Protocol must be udp or tcp, hostname must be either an IP or a FQDN. Use any free file-descriptor (3 or higher usually).

Use &- to close the connection instead of leaving it in the CLOSE_WAIT status. […]

Continue reading

How to install the latest Nmap for Debian/Ubuntu

A quick & dirty script to download the latest version of nmap (sourcecode) and generate a deb and install it (so that it’s correctly in the package management). Yes, I know this is not much more than a glorified configure && make && checkinstall

Continue reading

Bash snippet, verify ctrl+c

Lately I’ve been working on a pair of more elaborate scripts using ncat and openssl to transfer data between hosts. I’ll get around to posting it eventually, but until then a few small snippets that people may find useful. Today we will catch ctrl+c and ask the user if he really want’s to terminate the script.

The initialize() and cleanup() are my usual function names I have in every script, making sure general settings and variables are defined and that on exit any tempfiles get deleted. What has been added was a trap for the INT signal (ctrl+c) which calls the verify_quit() function, giving the user 10 seconds to […]

Continue reading

Simple “try” function for bash

Made a nice little try() function today for simplifying checking/dealing with  return codes from commands. It uses the function text() I posted earlier to colorfy output: How to easily add colored text output in bash scripts. The function accepts 2 parameters, how it should behave if a problem occurs and the command to be executed: try <silent|warn|fatal> command silent: save the return status in the global variable command_status warn: if the command has a return code > 0, print a warning and save the return status in the global variable command_status fatal: if the command has a return code > 0, print an error and exit Obviously not as versatile as a python try/except, bu streamlines verifying […]

Continue reading

Script to start minion in tmux

Minion is a security project from Mozilla (link). It provides a user-friendly web interface to various security scanner tools. There is a webcast demonstrating the software (link). The software requires a few services to run, and since I like having one script take care of starting everything with the right parameters, I threw together a simple shell script that sets up a tmux session with the services started in windows with the names of the services.

Continue reading

How to break down CIDR subnets in Bash

I was playing around with subnets in bash recently and needed an elegant/easy way to split up a subnet into smaller subnets. First I used 2 functions I found on stackoverflow.com to convert an IP addresse to and from an integer. After that it was just a bit of math in bash to split up any networks too big. Any network larger than $maxSubnet gets split up. Here the useful code:

Output of script:

 

Continue reading

How to find the fingerprints of public keys in authorized_keys

If you use keys for SSH authentication (and you should) then you have probably run into the situation that the auth.log shows that someone logged in, even which local user was used (e.g. root), but you have no idea which of the keys in ~/.ssh/autorized_keys was used. The first step you can do to see what is going on, is increasing the log level of the SSH daemon: /etc/ssh/sshd_config

That will spit out the fingerprint of the SSH key used to log in. Example log entry for a successful login:

Now that we have the fingerprint of the ssh key used to login, we will need ssh-keygen to […]

Continue reading

How to check if a IP (ipv4) address is valid in pure Bash

Here is a small bash function to check if a IP is valid (4 octets, each octet < 256). I find it somewhat elegant since instead of using a lot of case/if/then constructs or a crazy long regex it splits the IP into each octet (and stores them in an array, and then uses a combination of regex and bit shifting to check each octet.

The function will return 0 if the IP is valid, and 1 or higher if it encountered an error (you can check with the $? variable directly after calling the function) Example:

Continue reading

How to use Cluster SSH (cssh) and Mosh together

A colleague told me about mosh today, just saying “it’s an alternative to SSH that addresses many of the problems” doesn’t do it credit. Go look at the page and have a look, don’t worry it’s a nice page and I’ll still be here waiting when you come back. I do a lot of work on servers across the globe, and believe me, although subtle, the difference between a 10ms connection and a 100+ ms connection is definitely noticeable 😉 That lag all but disappears with mosh. Mosh doesn’t support X11/SSH-Agent forwarding yet, but it’s on their roadmap. Back to topic, I also use cssh for working on multiple servers simultaneously. I’d recommend making a copy of your […]

Continue reading

New Virtualbox version, script for easy update of extension pack

VirtualBox 4.1.10 was released yesterday with a few nice things in the changelog. Updating virtualbox itself is easy, just download the package and update it. Since I seem to stumble over the update of the extension pack every time (on my headless system) I thought I’d write a small script this time so I don’t have to rethink it next time (automatically downloads and installs the current extpack):

 

Continue reading

How to add locking to a shell script (the easy way)

I haven’t posted anything with bash here for a while, so today I’ll throw in a little snippet to use flock to make sure a script is only running once.  This is very handy in cron jobs that you want to run often, but there shouldn’t be multiple instances of the script running at the same time. Since it is small and easy I’d recommend adding it to any code you don’t want running multiple times since “that script” you just wrote, that runs 10 minutes now, might turn into a monster in 6 months and run 45 minutes when things change (data grows, more stuff to do).  Better safe than sorry. […]

Continue reading

How to build an efficient GeoIP SQL table

This here is a very handy little script I threw together to generate a geoip.sql table for quickly determining which country a IP is from. I already hear you saying “Just convert the IP to an INT and use BETWEEN, how hard can it be”. And you are right, that works. And it may even be your easiest solution, but it just isn’t fast. And if you are planning on hammering the table with thousands of queries you are going to end up looking for something fast. A while back I found a very interesting posting at www.jcole.us that described how to use Spacial Indexes together with MySQL’s GIS to speed up […]

Continue reading

How to install ruby 1.9.2 on Ubuntu 10.04

The current LTS version of Ubuntu is 10.04 and the most current version of ruby it ships with is 1.9.1. Unfurtunately 1.9.1 wasn’t that great of a release and anyone using the 1.9 branch really should use the stable 1.9.2. After doing a bit of researching I found some information on how the best approach to get ruby installed is. Downloading the source, compiling it and registering the installed version with the package manager. The following little bash script takes care of installing ruby 1.9.2 on a ubuntu or debian based system (or any other version if you change the $Version variable in the script). The script just consolidates information […]

Continue reading

How to easily add colored text output in bash scripts

Here is small snippet that can give your shell scripts some nice output: colortext.sh As with the debug.sh script, just download it to the same directory as your own script and add it with

It contains one simple function called text with the syntax text “text to be output”. Color can be red, green, yellow, blue or grey. The function does not automatically add a linebreak to the putput, so pop a \n in there if you need it. I prefer using it together with printf for clean and easy color output. Here are some examples of how the function can be used, and below the corresponding output:

Output: […]

Continue reading