Ryan Schulze

Ansible 2.7 oddities

October 18, 2018Ryan

One of the changes with ansible 2.7 is how you are supposed to pass a list of packages to a package module. instead of passing a list via with_items (which then get’s squashed into one call anyway), you just pass the whole list.
Apt as an example:

ansible <= 2.6

- name: Install packages
  apt: pkg: "{{ item }}"
  state: present
  with_items: "{{ packages }"

- name: Install packages

apt: pkg: "{{ item }}"

state: present

with_items: "{{ packages }"

ansible <= 2.7

- name: Install packages
  apt:
    pkg: "{{ packages }}"
    state: present

- name: Install packages

apt:

pkg: "{{ packages }}"

state: present

What the documentation doesn’t touch is how to pass multiple lists.
The error output suggests using ['{{ list1 }}', '{{ list2 }}'], but ansible throws a fit if you do that. Turns out you can also concatenate lists with the + sign. So the solution looks like this:

- name: Install packages
  apt:
    pkg: "{{ packages + more_packages }}"
    state: present

- name: Install packages

apt:

pkg: "{{ packages + more_packages }}"

state: present

Selfhosting email, and sending email to Microsoft.

September 25, 2018October 19, 2018Ryanemail, microsoft, MTA, SMTP

About once a year one of my outgoing email servers will magically pop up on Microsofts blacklist of email servers. The exact status in SNDS is “Blocked due to user complaints or other evidence of spamming“, which is a bit ironic since the weekly volume of emails that go to Microsoft controlled domains seldom is higher than 5 and they are all personal emails.
The error message the MTA recieves is “550 5.7.1 Unfortunately, messages from [xxx.xxx.xxx.xxx] weren’t sent. Please contact your Internet service provider since part of their network is on our block list (S3140). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors.” in case anyone ends up here via google.

I’m not going to bitch and moan or attempt to guess why the server ends up on the list since it isn’t something I can influence
(just submit a support request and wait), instead I’ll post some tips and pointers to useful tools you should be using:

SPF – Sender Policy Framework
Specify which IPs are allowed too send mail for your domains, also determine what should happen with mail that is recieved from unauthorized IPs.

DKIM – DomainKeys Identified Mail
Outgoing mail servers for a domain can sign the emails and thereby allow recipients to verify that the email really came from a valid mail server and is not from a spoofed sender.

DMARC – Domain-based Message Authentication, Reporting & Conformance
DMARC allows you to specify policies for domains or subdomains in regards to “what should happen if an email fails DKIM or SPF” and where/if to send reports for Emails recieved by other mail servers.

SNDS – Smart Network Data Service
Allows you to monitor the IPs of your mailservers as viewed by the Outlook.com system.

JMRP – Junk Email Reporting Program
Forwards the full message with headers of any email marked as “junk” or “phishing” by Outlook.com users

MX Toolbox
MXtoolbox is a great website for testing your mail server settings, they also have a free_monitoring service for one IP, so you can get alerted if your IP shows up on a Blacklist.

Last but not least I use the following small script to monitor the status of my servers according to Microsoft. I call the script via a daily cronjob, and if a server is blacklisted, it sends an alert. Go to Automated Data Access first to generate an API key.

#!/usr/bin/env bash

sdns_key="&lt;&lt;insert API key here&gt;&gt;"
recipient="&lt;&lt;email to send alerts to&gt;&gt;"

if [[ "${sdns_key}" == '' ]]; then
        exit 0
fi

_URL_ip="https://sendersupport.olc.protection.outlook.com/snds/ipStatus.aspx?key=${sdns_key}"
_URL_data="https://sendersupport.olc.protection.outlook.com/snds/data.aspx?key=${sdns_key}"

content="$(curl -s "${_URL_ip}")"

if [[ ${#content} -gt 0 ]]; then
        {
        echo "${content}"
        echo
        curl -s "${_URL_data}"
    } | mailx -s "Microsoft SNDS status" ${recipient}
    exit 1
fi

#!/usr/bin/env bash

sdns_key="<<insert API key here>>"

recipient="<<email to send alerts to>>"

if [[ "${sdns_key}" == '' ]]; then

exit 0

_URL_ip="https://sendersupport.olc.protection.outlook.com/snds/ipStatus.aspx?key=${sdns_key}"

_URL_data="https://sendersupport.olc.protection.outlook.com/snds/data.aspx?key=${sdns_key}"

content="$(curl -s "${_URL_ip}")"

if [[ ${#content} -gt 0 ]]; then

{

echo "${content}"

echo

curl -s "${_URL_data}"

} | mailx -s "Microsoft SNDS status" ${recipient}

exit 1

And if all else fails and your IP does end up blacklisted in SNDS, you can go here to submit a ticket to get back off (although I’d suggest checking your mail server logs and the status of the IP on other RBLs first, just to make sure).

http://go.microsoft.com/fwlink/?LinkID=614866

Odroid XU4 case

September 11, 2018April 1, 2022Ryanodroid

It’s been quiet here lately. I moved back to Germany and have taken on new responsibilities at work. We are slowly getting settled in, although i still have to get the network at the new apartment up and running the way I want it. I expect to post here more often once I have more time for side projects.

One thing I recently acquired and really like, is a new case for my Odroid XU4.
The small 5v Noctua fan fits very nicely in it and is virtually silent, together with the large heatsink it stays cool enough and seldom has to throttle the CPU.
I prefer the odroid over a raspberry pi for “small server” tasks since it has a Gbit interface (that isn’t shared on the USB bus) and supports eMMC modules (no one enjoys waiting on I/O).

Updating foscam firmware via API

June 12, 2018Ryan

Foscam just released a new round of firmware updates. The suggested process to deploy firmware updates involves using the web interface. I’m hoping newer models are less of a hassle, but older models require the user to install an .exe and use internet explorer 10 or hope that your browser is old enough to allow you to enable NPAPI.

That pretty much rules out Linux, Mac, and up-to-date browsers on Windows.

But fear not, the devices have an API (for some reason they deleted any references/links to it from their forum, but I found a version here).

The API comes with its own share of odd quirks, but to update the firmware without a browser you can use the following CURL command

curl --output - --connect-timeout 15 --max-time 180 -H 'Expect:' --form 'image=@FOSIPC_A_patch_ver_whatever.bin' --request POST 'http://192.168.0.123/cgi-bin/CGIProxy.fcgi?cmd=fwUpgrade&usr=admin&pwd=password'

Replace the username and password with an account with admin privileges, the IP with the IP of your device, and the .bin file with the latest patch for your device.

Note: If your admin username or password contain any non alphanumeric characters, you need to Urlencode them. There are multiple ways to do this (perl, python, jq), but I’ll assume you have curl installed and can do
echo -n '$uper$secure' | curl -Gso /dev/null -w %{url_effective} --data-urlencode @- "" | cut -c 3-

Privacy Policy

May 22, 2018May 22, 2018Ryan

Soon new laws will go into effect in the EU regarding data privacy. I assume anyone reading this has already noticed everyone updating their privacy policies. The German version of the laws exempt private websites, but the exact definition of what “private” means is a bit unclear.

To err on the side of caution we also updated our privacy policy, our impressum, and took the necessary steps behind the scenes to be complaint with the laws. The following links/forms are also in the policy itself, but for easier access:

[gdpr_preferences text=”Privacy Preferences”]

[gdpr_request_form type=”export-data”]

[gdpr_request_form type=”delete”]

Submit a request to rectify data:

[gdpr_request_form type=”rectify”]