How to fix Mono crashing on Odroid XU4

Recently I’ve been noticing my Sonarr and Radarr applications behaving erratic (sometimes not responsive, sometimes not performing tasks, not searching or adding content, but at other times behaving totally fine). A quick look at the logs told me the applications were crashing and being restarted by systemctl, after a few crashes they seemed to stabilize.
Today I had some time to dig deeper into the issue. I had already searched the Internet for general issues, but it didn’t seem to be a widespread problem. I assumed it might have to do with the ARM architecture, but Raspberry Pi users didn’t seem to be having these issues.

In the past I had difficulty reproducing the issue, but today I was in luck, every time I tried to kick off the “Process Monitored Downloads” task in Radarr, it would start working on the task and then crash and restart. The core issue turned out to be oddly specific to the Odroid XU4 hardware.

The XU4 has eight CPU cores, four A7 cores running at 1.4GHz, and four A15 cores running at 2GHz.

 

Whenever the mono process moved from an A7 to an A15 core (or vice versa) the process crashed.
Since both Sonarr and Radarr are Mono applications, they were both affected. Pinning the applications to either the A15 or the A7 CPUs resolved the problem.
taskset --cpu-list  can be used to change the CPU affinity of a process.

First look up the systemctl  service files for Radarr and Sonarr (e.g. via systemctl status radarr ). Edit the service files by prefixing the ExecStart  command with /usr/bin/taskset -c 0-3  (for the A7 cores) or /usr/bin/taskset -c 4-7  (for the A15 cores).
Then reload the systemctl files ( systemctl daemon-reload ) and restart the service.

Example:

If the service files are managed via the package manager, you may want to create an systemctl override instead of editing the service file, so that the package manager doesn’t overwrite your changes:

 

Now you probably want to see if it worked and want to know how to check which core a process is running on? There a re a few options:

htop: launch htop . Press <F2> , go to Columns , and add PROCESSOR  from Available Columns. The CPUs are numbered 1-8 in htop (as opposed to 0-7 by the system).

ps: the PSR column can display the core a process is on. The CPUs are numbered 0-7 in ps.

taskset: can display the current affinity of a process.

 

Odroid XU4 case

It’s been quiet here lately. I moved back to Germany and have taken on new responsibilities at work. We are slowly getting settled in, although i still have to get the network at the new apartment up and running the way I want it. I expect to post here more often once I have more time for side projects.

Odroid Xu4

One thing I recently acquired and really like, is a new case for my Odroid XU4.
The small 5v Noctua fan fits very nicely in it and is virtually silent, together with the large heatsink it stays cool enough and seldom has to throttle the CPU.
I prefer the odroid over a raspberry pi for “small server” tasks since it has a Gbit interface (that isn’t shared on the USB bus) and supports eMMC modules (no one enjoys waiting on I/O).