Monthly Archives: May 2015

Getting your locale right

So, you get this annoying error (in Debian, some Ubuntu, or perhaps any other Linux or even Unix system):

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

Perhaps you get it when running apt-get?
You Google for answers and you find nothing useful and/or just contradictory information?
You are connecting over SSH?
You are connecting from another system (like Mac OS X)?
Read on…
In it simplest incarnation, the problem looks like this.

$ man some-none-existing-program
man: can't set the locale; make sure $LC_* and $LANG are correct
No manual entry for some-none-existing-program

$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LANGUAGE=en_GB:en
LC_CTYPE=UTF-8
LC_NUMERIC=sv_SE.UTF-8
LC_TIME=sv_SE.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=sv_SE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=sv_SE.UTF-8
LC_NAME=sv_SE.UTF-8
LC_ADDRESS=sv_SE.UTF-8
LC_TELEPHONE=sv_SE.UTF-8
LC_MEASUREMENT=sv_SE.UTF-8
LC_IDENTIFICATION=sv_SE.UTF-8
LC_ALL=

What happens here, in the second case is that “UTF-8” is not a valid LC_CTYPE (in Debian) and since LC_ALL is not set it can not fall back properly. That is all. Why is LC_CTYPE invalid? Perhaps because you have used ssh from Mac OS X (or something else):

mac $ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

mac $ ssh user@debianhost

The point is, here LC_CTYPE=UTF-8, and it is valid in OS X. But it is not valid in Debian. That is all there is to it. Try:

# This will fix the problem
mac $ LC_CTYPE=en_US.UTF-8 ssh user@debianhost

# This will produce the problem even from a Debian machine
debianclient $ LC_CTYPE=UTF-8 ssh user@debianhost

# This will (kind of) eleminate the problem when already logged in
$ LC_CTYPE=en_US.UTF-8 locale
-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8)
LANG=en_US.UTF-8
LANGUAGE=en_GB:en
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=sv_SE.UTF-8
LC_TIME=sv_SE.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=sv_SE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=sv_SE.UTF-8
LC_NAME=sv_SE.UTF-8
LC_ADDRESS=sv_SE.UTF-8
LC_TELEPHONE=sv_SE.UTF-8
LC_MEASUREMENT=sv_SE.UTF-8
LC_IDENTIFICATION=sv_SE.UTF-8
LC_ALL=

So, the problem is that all the guides on the internet presume you have a problem with your locales on your server, which you dont! The locale is probably fine. It is just that the ssh client computer has an LC_CTYPE which does not happen to be valid on the server.

The Debian Locale guide itself is of course correct. And it states that you should not set LC_ALL=en_US.UTF-8 (!). That is however the only “working” answer I found online.

Solution
I guess the easiest thing is to add a line to your .profile file (on the server):

$ cat .profile 
.
.
.
LC_CTYPE="en_US.UTF-8"

If you are not using a bourne-shell you probably know how to set a variable for your shell. Any other method that will set LC_CTYPE to a locale valid on your Debian machine will also work. Clearly, you dont need (and you probably should not) set LC_ALL.

Not happy with en_US.UTF-8?
Not everyone use the American locale. To find out what locales are available/valid on your system:

$ locale -a
C
C.UTF-8
en_GB.utf8
en_US.utf8
POSIX
sv_SE.utf8

Want others?

$ sudo dpkg-reconfigure locales

or read the Debian Locale Guide.

Upgrading Qnap TS109 from Wheezy to Jessie

The Qnap TS-109 runs Debian Wheezy just fine, but it has to be upgraded from Squeeze (no direct install). How about upgrading Wheezy to Jessie? This device is old and slow, so I decided to find out, and if it does not work, so be it.

Debian links:

Below follows a shorter version of the upgrade guide, focusing on what I actually did with my Qnap.

Getting ready
First you should of course backup your stuff if the device contains anything you can not lose.

It can also be a good idea to clean out packages that you dont need (to both avoid problems when upgrading them, and to save time during the upgrade):

$ sudo dpkg -l
$ sudo apt-get purge SOMEPACKAGES

It is also adviced to make sure you dont have packages on hold. You can do this with:

$ sudo dpkg --get-selections | grep 'hold$'

I confirmed that my system was then fully wheezy-updated

$ sudo apt-get update

$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

With that, I decided I was ready for an upgrade to Jessie.

Upgrade itself
I first updated /etc/apt/sources.list.

Old version:

deb http://ftp.se.debian.org/debian/ wheezy main
deb-src http://ftp.se.debian.org/debian/ wheezy main non-free

deb http://security.debian.org/ wheezy/updates main
deb-src http://security.debian.org/ wheezy/updates main non-free

deb http://ftp.df.lth.se/debian wheezy-backports main

New version:

deb http://ftp.se.debian.org/debian/ jessie main
deb-src http://ftp.se.debian.org/debian/ jessie main non-free

deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main non-free

I just deleted the backports source.

And begin upgrade:

$ sudo apt-get update
$ sudo apt-get upgrade

That went fine, so I did a reboot. It took longer than usual, but it came up. So I proceeded with:

$ sudo apt-get dist-upgrade

…which went fine, and I rebooted.

System came up and it seems good! If I encounter problems later I will write about it here.

Node.js Benchmark on Raspberry Pi (v1)

I have experimented a bit with Node.js and Raspberry Pi lately, and I have found the performance… surprisingly bad. So I decided to run some standard tests: benchmark-octane (v9).

Octane is essentially run like:

$ npm install benchmark-octane
$ cd node_modules/benchmark-octane
$ node run.js

The distilled result of Octane is a total run time and a score. Here are a few results:

                         OS             Node.js                   Time    Score
QNAP TS-109 500MHz       Debian        v0.10.29 (Debian)         3350s      N/A
Raspberry Pi v1 700MHz   OpenWrt BB    v0.10.35 (self built)     2267s      140
Raspberry Pi v1 700MHz   Raspbian       v0.6.19 (Raspbian)       2083s      N/A
Raspberry Pi v1 700MHz   Raspbian       v0.12.2 (self built)     2176s      104
Eee701 Celeron 900Mhz    Xubuntu       v0.10.25 (Ubuntu)          171s     1655
Athlon II X2@3Hz         Xubuntu       v0.10.25 (Ubuntu)           49s     9475
MacBook Air i5@1.4Ghz    Mac OS X      v0.10.35 (pkgsrc)           47s    10896
HP 2560p i7@2.7Ghz       Xubuntu       v0.10.25 (Ubuntu)           41s    15450

Score N/A means that one test failed and there was no final score.

When I first saw the RPi performance I thought I had done something wrong building (using a cross compiler) Node.js myself for RPi and OpenWRT. However Node.js with Raspbian is basically not faster, and also RPi ARMv6 with FPU is not much faster than the QNAP ARMv5 without FPU.

I think the Eee701 serves as a good baseline here. At first glance, possible reasons for the RPi underperformance relative to the Celeron are:

  • Smaller cache (16kb of L1 cache and L2 only available to GPU, i Read) compared to Celeron (512k)
  • Bad or not well utilised FPU (but there at least is one on the RPi)
  • Node.js (V8) less optimized for ARM

I found that I have benchmarked those to CPUs against each other before. That time the Celeron was twice as fast as the RPi, and the FPU of the RPi performed decently. Blaming the small cache makes more sense to me, than blaming the people who implemented ARM support in V8.

The conclusion is that Raspberry Pi (v1 at least) is extremely slow running Node.js. Other benchmarks indicate that RPi v2 is significantly faster.

Eee701 in 2015

My eee701 is not doing very much anymore, but sometimes it is handy to have it around. I have not upgraded it since Lubuntu 13.10, and that version is not supported anymore. I found that:

  1. Since 13.10 is abandoned upgrading with apt-get generated errors.
  2. 15.04 Lubuntu desktop ISO complains that the hard drive is less than 4.1Gb.
  3. 14.04.2 Lubuntu desktop ISO complains that the hard drive is less than 4.5Gb.

Thus, the standard upgrade or installation paths were blocked. And I was not very interested in putting lots of effort into getting my eee701 running a current system.

Instead, i tried the Ubuntu mini-iso. That was very nice! The iso itself is written with dd (rather than the Startup Disk Creator) to USB drive. The installation is text (curses) based, but very guided (just like Debian). I choose a single 4GB ext4 partition for root, no swap (since I have 2GB RAM) and “Xubuntu minimal” desktop. Keyboard, Wifi and timezones were all correctly set up. When installation was complete and system restarted I had 1.8Gb used and 1.7Gb available. Not even a web browser was installed, but Xubuntu itself was fine.

Not bad at all!

Seeding Xubuntu and Lubuntu

It is not much I do to contribute to the Ubuntu community so when Ubuntu 15.04 was released a few weeks ago I downloaded 4 isos with bittorrent and kept seeding them for the benefit of Ubuntu users.

Now (2015-05-10):

Image                                Size     Ratio
lubuntu-15.04-desktop-i386.iso      696Mb       120
lubuntu-15.04-desktop-amd64.iso     690Mb      60.0
xubuntu-15.04-desktop-i386.iso      970Mb      68.5
xubuntu-15.04-desktop-amd64.iso     963Mb      68.2

Does this mean anything?

If the ratio has anything to do with the popularity of different ubuntu versions:

  1. It surprised my that Lubuntu is so popular
  2. It surprised me that i386 is still very popular

There are, however, a number of factors that could disturb the correlation between my ratio and the true popularity of different Ubuntu flavours:

  • The way Lubuntu vs Xubuntu push people to the torrent (rather than the direct http/ftp) download, I found it harder to find the Lubuntu torrent, though.
  • Lubuntu and Xubuntu may have different audiences, with different attitude to torrent downloads.
  • i386 and amd86 may have different audiences as well
  • If more downloaders mean more seeders, and people on average seed close to 1.0 (or even above) then my Ratio may mean very little.
  • The tracker may have identified me as a stable seeder and sent downloaders of less popular images (both to download and seed) my way, in an attempt to provide equally good service for everyone (no idea if trackers do this).

The i386-heaviness of Lubuntu indicates that there should be correlation between general popularity and my Ratio.

Raspberry Pi (v1), OpenWrt (14.07) and Node.js (v0.10.35 & v0.12.2)

Since I gave up running NetBSD on my Raspberry pi I decided it was time to try OpenWrt. And, to my surprise I also managed to cross compile Node.js!

Install OpenWrt on Raspberry Pi (v1@700MHz)
I installed OpenWrt Barrier Breaker (the currently stable release) using the standard instructions.

After you have put the image on an SD-card with dd, it is quite easy to resize the root partition:

  1. copy the second partition to an image file using dd
  2. use fdisk to delete the second partition, and create a new, bigger
  3. format the new partition with mkfs.ext4
  4. mount the image file using mount -o loop
  5. mount the new second partition
  6. copy all data from image file to second partition using cp -a

If you want to, you can edit /etc/config/network while you are anyway working with the OpenWrt root partition:

#config interface 'lan'
#	option ifname 'eth0'
#	option type 'bridge'
#	option proto 'static'
#	option ipaddr '192.168.1.1'
#	option netmask '255.255.255.0'
#	option ip6assign '60'
#	option gateway '?.?.?.?'
#	option dns '?.?.?.?'
config interface 'lan'
	option ifname 'eth0'
	option proto 'dhcp'
	option macaddr 'XX:XX:XX:XX:XX:XX'
	option hostname 'rpiopenwrt'

Probably you want to disable dnsmasq, odhcpd and firewall too:

.../etc/init.d/$ chmod -x dnsmasq firewall odhcpd

OR (depending on your idea of what is the right way)

.../etc/rc.d$ sudo rm S60dnsmasq S35odhcpd K85odhcpd S19firewall

Also, it is a good idea to edit config.txt (on the DOS partition):

gpu_mem=1

I don’t know if 1 is really a legal value, but it worked for me, and I had much more memory available than when gpu_mem was not set.

Node.js4 added 2015-10-03
For Node.js, check Node.js 4 builds.

Building Node.js v0.12.2
I downloaded and built Node.js v0.12.2 on a Xubuntu machine with an x64 cpu. On such a machine you can download the standard OpenWrt toolchain for Raspberry Pi.

I replaced configure and cpu.cc in the standard sources with the files from This Page (they are meant for v0.12.1 but they work equally good for v0.12.2).

I then found an a gist that gave me a good start. I modified it, and ended up with:

#!/bin/sh -e

export STAGING_DIR=...path to your toolchain...

#Tools
export CSTOOLS="$STAGING_DIR"
export CSTOOLS_INC=${CSTOOLS}/include
export CSTOOLS_LIB=${CSTOOLS}/lib
export ARM_TARGET_LIB=$CSTOOLS_LIB

export TARGET_ARCH="-march=armv6j"

#Define the cross compilators on your system
export AR="arm-openwrt-linux-uclibcgnueabi-ar"
export CC="arm-openwrt-linux-uclibcgnueabi-gcc"
export CXX="arm-openwrt-linux-uclibcgnueabi-g++"
export LINK="arm-openwrt-linux-uclibcgnueabi-g++"
export CPP="arm-openwrt-linux-uclibcgnueabi-gcc -E"
export LD="arm-openwrt-linux-uclibcgnueabi-ld"
export AS="arm-openwrt-linux-uclibcgnueabi-as"
export CCLD="arm-openwrt-linux-uclibcgnueabi-gcc ${TARGET_ARCH} ${TARGET_TUNE}"
export NM="arm-openwrt-linux-uclibcgnueabi-nm"
export STRIP="arm-openwrt-linux-uclibcgnueabi-strip"
export OBJCOPY="arm-openwrt-linux-uclibcgnueabi-objcopy"
export RANLIB="arm-openwrt-linux-uclibcgnueabi-ranlib"
export F77="arm-openwrt-linux-uclibcgnueabi-g77 ${TARGET_ARCH} ${TARGET_TUNE}"
unset LIBC

#Define flags
export CXXFLAGS="-march=armv6j"
export LDFLAGS="-L${CSTOOLS_LIB} -Wl,-rpath-link,${CSTOOLS_LIB} -Wl,-O1 -Wl,--hash-style=gnu"
export CFLAGS="-isystem${CSTOOLS_INC} -fexpensive-optimizations -frename-registers -fomit-frame-pointer -O2"
export CPPFLAGS="-isystem${CSTOOLS_INC}"
export CCFLAGS="-march=armv6j"

export PATH="${CSTOOLS}/bin:$PATH"

./configure --without-snapshot --dest-cpu=arm --dest-os=linux --without-npm

bash --norc

Run this script in the Node.js source directory. If everything goes fine it configures the Node.js build, and leaves you with a shell where you can simply run:

$ make

If compilation is fine, you find the node binary in the out/Release folder. Copy it to your OpenWrt Raspberry Pi.

Building Node.js v0.10.35
I first successfully built Node.js v0.10.35.

The (less refined) script for configuring that I used was:

#!/bin/sh -e

export STAGING_DIR=...path to your toolchain...

#Tools
export CSTOOLS="$STAGING_DIR"
export CSTOOLS_INC=${CSTOOLS}/include
export CSTOOLS_LIB=${CSTOOLS}/lib
export ARM_TARGET_LIB=$CSTOOLS_LIB
export GYP_DEFINES="armv7=0"

#Define our target device
export TARGET_ARCH="-march=armv6"
export TARGET_TUNE="-mfloat-abi=hard"

#Define the cross compilators on your system
export AR="arm-openwrt-linux-uclibcgnueabi-ar"
export CC="arm-openwrt-linux-uclibcgnueabi-gcc"
export CXX="arm-openwrt-linux-uclibcgnueabi-g++"
export LINK="arm-openwrt-linux-uclibcgnueabi-g++"
export CPP="arm-openwrt-linux-uclibcgnueabi-gcc -E"
export LD="arm-openwrt-linux-uclibcgnueabi-ld"
export AS="arm-openwrt-linux-uclibcgnueabi-as"
export CCLD="arm-openwrt-linux-uclibcgnueabi-gcc ${TARGET_ARCH} ${TARGET_TUNE}"
export NM="arm-openwrt-linux-uclibcgnueabi-nm"
export STRIP="arm-openwrt-linux-uclibcgnueabi-strip"
export OBJCOPY="arm-openwrt-linux-uclibcgnueabi-objcopy"
export RANLIB="arm-openwrt-linux-uclibcgnueabi-ranlib"
export F77="arm-openwrt-linux-uclibcgnueabi-g77 ${TARGET_ARCH} ${TARGET_TUNE}"
unset LIBC

#Define flags
export CXXFLAGS="-march=armv6"
export LDFLAGS="-L${CSTOOLS_LIB} -Wl,-rpath-link,${CSTOOLS_LIB} -Wl,-O1 -Wl,--hash-style=gnu"
export CFLAGS="-isystem${CSTOOLS_INC} -fexpensive-optimizations -frename-registers -fomit-frame-pointer -O2 -ggdb3"
export CPPFLAGS="-isystem${CSTOOLS_INC}"
export CCFLAGS="-march=armv6"

export PATH="${CSTOOLS}/bin:$PATH"

./configure --without-snapshot --dest-cpu=arm --dest-os=linux
bash --norc

Running node on the Raspberry Pi
Back on the Raspberry Pi you need to install a few packages:

# ldd ./node 
	libdl.so.0 => /lib/libdl.so.0 (0xb6f60000)
	librt.so.0 => not found
	libstdc++.so.6 => not found
	libm.so.0 => /lib/libm.so.0 (0xb6f48000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6f34000)
	libpthread.so.0 => not found
	libc.so.0 => /lib/libc.so.0 (0xb6edf000)
	ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0xb6f6c000)
# opkg update
# opkg install librt
# opkg install libstdcpp

That is all! Now you should be ready to run node. The node binary is about 13Mb (the v0.10.35 was 19Mb perhaps becuase of -ggdb3), so it is not optimal to deploy it to other typical OpenWrt hardware.

Final comments
I ran a few small programs to test, and they were fine. I guess some more testing would be appropriate. The performance is very comparable to Node.js built and executed on Raspbian.

I think RaspberryPi+OpenWrt+Node.js is a very interesting and competitive combination for microservices!