Rust to the Pi

Introduction

Cross compilation is the action of compiling for a different target platform than the one you are building on (Example: Compiling from a Linux machine and run your program on Windows, a Raspberry Pi, an Arduino board…). Sometimes your target platform does not have enough resources to compile the code, sometimes it’s just too slow. In any case, cross compilation is the solution and Rust supports a great number of platforms.

In this article, we are going to try to build rust-motd. It is a simple enough program so that it should not involve too much weird magic. Yet it is complex enough so that it should be representative of at least some issues you would face.

For the impatient reader, there is a TLDR section at the end.

Method 1: Cross compiling in the rust docker image (Recommended)

Prerequisite

In this method, we’ll compile everything inside the official rust image. The only thing you will need is a working docker install.

Just run the following command to get a shell inside the rust docker image.

docker run -it --rm -v $(pwd):/usr/src/myapp -w /usr/src/myapp rust:1.58.0 bash

Cargo

Let’s start by taking a look at the rust documentation on cross compilation.

Well seems easy enough. All the tooling is already provided by our docker environment. We just need to install rust toolchain for Raspberry Pi. A quick internet search gives us that the target for Raspberry Pi 2+ is armv7-unknown-linux-gnueabihf.

rustup target add armv7-unknown-linux-gnueabihf

git clone https://github.com/rust-motd/rust-motd.git
cd rust-motd
cargo build --target armv7-unknown-linux-gnueabihf

-- A few logs later --
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/usr/src/myapp/rust-motd/target/release/build/ring-6a04e67138f6a894/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("armv7-unknown-linux-gnueabihf")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_armv7-unknown-linux-gnueabihf = None
  CC_armv7_unknown_linux_gnueabihf = None
  TARGET_CC = None
  CC = None
  CROSS_COMPILE = None
  CFLAGS_armv7-unknown-linux-gnueabihf = None
  CFLAGS_armv7_unknown_linux_gnueabihf = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = None

  --- stderr
  running "arm-linux-gnueabihf-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-mfpu=vfpv3-d16" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/release/build/ring-2de21f60affcb1e7/out/aesv8-armx-linux32.o" "/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux32.S"
  thread 'main' panicked at 'failed to execute ["arm-linux-gnueabihf-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-mfpu=vfpv3-d16" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/release/build/ring-2de21f60affcb1e7/out/aesv8-armx-linux32.o" "/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux32.S"]: No such file or directory (os error 2)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/build.rs:653:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

Well, we are out of luck, it seems that we are missing arm-linux-gnueabihf-gcc here.

Getting the Cross-Compiler

In the previous step, we were missing arm-linux-gnueabihf-gcc. It is a part of the GNU toolchain for ARM, and it is the actual cross-compiler we’ll be using. A quick search in the Debian package registry gives us the proper package.

# Install gcc-arm-linux-gnueabihf
apt-get update
apt-get install -y gcc-arm-linux-gnueabihf

# Now that we've got a cross-compiler, let's try to compile again

cargo build --target armv7-unknown-linux-gnueabihf

-- A few logs later --
          /usr/bin/ld: /usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/debug/deps/hyper-32b5b56be2b66aa1.hyper.6096c872-cgu.0.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/debug/deps/hyper-32b5b56be2b66aa1.hyper.6096c872-cgu.0.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/debug/deps/hyper-32b5b56be2b66aa1.hyper.6096c872-cgu.0.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/debug/deps/hyper-32b5b56be2b66aa1.hyper.6096c872-cgu.0.rcgu.o: error adding symbols: file in wrong format
          collect2: error: ld returned 1 exit status
          
error: could not compile `hyper` due to previous error
  warning: build failed, waiting for other jobs to finish...
error: build failed

A linking problem, we can fix that by giving the correct linker to cargo. One of the way of doing that is to add a file .cargo/config.toml

mkdir .cargo
echo -e '[target.armv7-unknown-linux-gnueabihf]\nlinker = "arm-linux-gnueabihf-gcc"' > .cargo/config.toml

cargo build --target armv7-unknown-linux-gnueabihf

It builds \o/. Now let’s check the binary.

file target/armv7-unknown-linux-gnueabihf/debug/rust-motd

-- Result --
target/armv7-unknown-linux-gnueabihf/debug/rust-motd: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=a62a2fe866db10ebe2359352540697161554e11b, for GNU/Linux 3.2.0, with debug_info, not stripped

As you can see, the binary is compiled for arm. We can now try it out. (You’ll need to get out of the docker image first, and to add a config file, but I’m sure you can figure that out on yourself :D)

scp rust-motd/target/armv7-unknown-linux-gnueabihf/debug/rust-motd [[ raspberry pi ]]:
ssh heimdall ./rust-motd

-- Result --
Weather report: New York, New York

     \  /       Partly cloudy
   _ /"".-.     -2(-6) °C
     \_(   ).   ↓ 15 km/h
     /(___(__)  16 km
                0.0 mm

Up 1day 13h 55m 57s

System Services:
  Accounts: inactive
  Cron:     inactive

Filesystems  Device     Mount  Type  Used    Total
  root       /dev/root  /      ext4  4.1 GB  7.4 GB
  [===============================================]
rust-motd

Awesome, it works !

Method 2: Cross Compiling directly on host system

Prerequisite

With this method, we’ll compile everything without the docker container. You need to have a running rust setup.

Cargo

Let’s start (again) by taking a look at the rust documentation on cross compilation.

We just need to install rust toolchain for raspberry pi. The target for raspberry pi 2+ is armv7-unknown-linux-gnueabihf.

rustup target add armv7-unknown-linux-gnueabihf
git clone https://github.com/rust-motd/rust-motd.git
cd rust-motd
cargo build --target armv7-unknown-linux-gnueabihf

-- A few logs later --
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/usr/src/myapp/rust-motd/target/release/build/ring-6a04e67138f6a894/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("armv7-unknown-linux-gnueabihf")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_armv7-unknown-linux-gnueabihf = None
  CC_armv7_unknown_linux_gnueabihf = None
  TARGET_CC = None
  CC = None
  CROSS_COMPILE = None
  CFLAGS_armv7-unknown-linux-gnueabihf = None
  CFLAGS_armv7_unknown_linux_gnueabihf = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = None

  --- stderr
  running "arm-linux-gnueabihf-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-mfpu=vfpv3-d16" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/release/build/ring-2de21f60affcb1e7/out/aesv8-armx-linux32.o" "/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux32.S"
  thread 'main' panicked at 'failed to execute ["arm-linux-gnueabihf-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-mfpu=vfpv3-d16" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/usr/src/myapp/rust-motd/target/armv7-unknown-linux-gnueabihf/release/build/ring-2de21f60affcb1e7/out/aesv8-armx-linux32.o" "/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux32.S"]: No such file or directory (os error 2)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/build.rs:653:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

Well, unsurprisingly, we face the same issue as in method 1. We are lacking arm-linux-gnueabihf-gcc.

Getting the Cross Compiler

In the previous step, we were missing arm-linux-gnueabihf-gcc. It is a part of the GNU toolchain for ARM (available here). We’ll need to download it and add it to our path.

Important: You need to select the one adapted to

your host system (x86_64 linux for me)
your target system (arm-none-linux-gnueabihf for raspberry pi 2+).
the GCC version installed on your target Raspberry Pi

wget -O gcc-arm-none-linux-gnueabihf.tar.xz "https://developer.arm.com/-/media/Files/downloads/gnu-a/10.2-2020.11/binrel/gcc-arm-10.2-2020.11-x86_64-arm-none-linux-gnueabihf.tar.xz?revision=d0b90559-3960-4e4b-9297-7ddbc3e52783&hash=764D949F016397C3B36F225171E9AF38"
mkdir gcc-arm-none-linux-gnueabihf
tar -xf gcc-arm-none-linux-gnueabihf.tar.xz -C ./gcc-arm-none-linux-gnueabihf --strip-components=1
export PATH="$(pwd)/gcc-arm-none-linux-gnueabihf/bin:$PATH"

By default, cargo uses arm-linux-gnueabihf-gcc for target armv7, we need to configure the compiler and the linker.

export CC_armv7_unknown_linux_gnueabihf=arm-none-linux-gnueabihf-gcc
export CC=arm-none-linux-gnueabihf-gcc

mkdir .cargo
echo -e '[target.armv7-unknown-linux-gnueabihf]\nlinker = "arm-none-linux-gnueabihf-gcc"' > .cargo/config.toml

cargo build --target armv7-unknown-linux-gnueabihf

It builds \o/. Now let’s check the binary.

file target/armv7-unknown-linux-gnueabihf/debug/rust-motd

-- Result --
target/armv7-unknown-linux-gnueabihf/debug/rust-motd: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=a62a2fe866db10ebe2359352540697161554e11b, for GNU/Linux 3.2.0, with debug_info, not stripped

As you can see, the binary is compiled for arm. We can now try it out. (You’ll need to get out of the docker image first, and to add a config file but I’m sure you can figure that out on yourself :D)

scp rust-motd/target/armv7-unknown-linux-gnueabihf/debug/rust-motd [[ raspberry pi ]]:
ssh heimdall ./rust-motd

-- Result --
Weather report: New York, New York

     \  /       Partly cloudy
   _ /"".-.     -2(-6) °C
     \_(   ).   ↓ 15 km/h
     /(___(__)  16 km
                0.0 mm

Up 1day 13h 55m 57s

System Services:
  Accounts: inactive
  Cron:     inactive

Filesystems  Device     Mount  Type  Used    Total
  root       /dev/root  /      ext4  4.1 GB  7.4 GB
  [===============================================]
rust-motd

Awesome, it works (again)!

Troubleshooting

File not found arm-none-linux-gnueabihf-gcc

If you see that cargo still can’t find arm-none-linux-gnueabhf-gcc, verify that it is in your path and that it you can use it.

ls gcc-arm-10.2-2020.11-mingw-w64-i686-arm-none-linux-gnueabihf/bin
-- Result --
arm-none-linux-gnueabihf-addr2line.exe  arm-none-linux-gnueabihf-dwp.exe         arm-none-linux-gnueabihf-gcc-ranlib.exe  arm-none-linux-gnueabihf-gdb.exe       arm-none-linux-gnueabihf-lto-dump.exe  arm-none-linux-gnueabihf-size.exe
arm-none-linux-gnueabihf-ar.exe         arm-none-linux-gnueabihf-elfedit.exe     arm-none-linux-gnueabihf-gcc.exe         arm-none-linux-gnueabihf-gfortran.exe  arm-none-linux-gnueabihf-nm.exe        arm-none-linux-gnueabihf-strings.exe
arm-none-linux-gnueabihf-as.exe         arm-none-linux-gnueabihf-g++.exe         arm-none-linux-gnueabihf-gcov-dump.exe   arm-none-linux-gnueabihf-gprof.exe     arm-none-linux-gnueabihf-objcopy.exe   arm-none-linux-gnueabihf-strip.exe
arm-none-linux-gnueabihf-c++.exe        arm-none-linux-gnueabihf-gcc-10.2.1.exe  arm-none-linux-gnueabihf-gcov-tool.exe   arm-none-linux-gnueabihf-ld.bfd.exe    arm-none-linux-gnueabihf-objdump.exe
arm-none-linux-gnueabihf-c++filt.exe    arm-none-linux-gnueabihf-gcc-ar.exe      arm-none-linux-gnueabihf-gcov.exe        arm-none-linux-gnueabihf-ld.exe        arm-none-linux-gnueabihf-ranlib.exe
arm-none-linux-gnueabihf-cpp.exe        arm-none-linux-gnueabihf-gcc-nm.exe      arm-none-linux-gnueabihf-gdb-add-index   arm-none-linux-gnueabihf-ld.gold.exe   arm-none-linux-gnueabihf-readelf.exe

gcc-arm-10.2-2020.11-mingw-w64-i686-arm-none-linux-gnueabihf/bin/arm-none-linux-gnueabihf-gcc
-- Result --
bash: no such file or directory: gcc-arm-10.2-2020.11-mingw-w64-i686-arm-none-linux-gnueabihf/bin/arm-none-linux-gnueabihf-gcc

This usually means that you did not download a binary matching your host system architecture.

GCC version problem

If you compile everything successfully but still can’t run it with this error:

./rust-motd: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.32' not found (required by ./rust-motd)
./rust-motd: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.33' not found (required by ./rust-motd)

This usually means that you downloaded a version of arm-none-linux-gnueabihf-gcc which is not compatible with the gcc version installed on the pi. You probably need to try an older version.

OpenSSL

When cross-compiling your own program, you might encounter issues with openssl. Something along those lines:

error: failed to run custom build command for `openssl-sys v0.9.72`
Caused by:
  process didn't exit successfully: `.../target/release/build/openssl-sys-bd16977361a30d49/build-script-main` (exit status: 101)
  ... 
  run pkg_config fail: "pkg-config has not been configured to support cross-compilation.\n\nInstall a sysroot for the target platform and configure it via\nPKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a\ncross-compiling wrapper for pkg-config and set it via\nPKG_CONFIG environment variable."
  --- stderr
  thread 'main' panicked at '
  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.
  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.
  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.
  $HOST = x86_64-unknown-linux-gnu
  $TARGET = armv7-unknown-linux-gnueabihf
  openssl-sys = 0.9.72
warning: build failed, waiting for other jobs to finish...
error: build failed

This means that you are missing openssl bindings to compile against. You can either add the openssl crate to your Cargo.toml with the “vendor” feature that will compile openssl for you:

[dependencies]
openssl = { version = "0.10", features = ["vendored"] }

Or you can compile openssl yourself before.

# Cross Compile openssl
export MACHINE=armv7
export ARCH=arm
export CC=arm-linux-gnueabihf-gcc # Or arm-linux-none-gnueabihf-gcc dependending on the method you are following

wget -q https://www.openssl.org/source/openssl-1.1.1m.tar.gz
tar xzf openssl-1.1.1m.tar.gz
cd openssl-1.1.1m
./config shared
make
cd ..

export OPENSSL_LIB_DIR="$(pwd)/openssl-1.1.1m/"
export OPENSSL_INCLUDE_DIR="$(pwd)/openssl-1.1.1m/include"

TLDR

For the really impatient, here how to cross-compile rust-motd for Raspberry Pi 2+. If you face any issues, well you’re out of luck, and you’re going to have to read it the article through.

# Run it in a debian docker container to avoid GCC version mismatches
docker run -it --rm -v $(pwd):/usr/src/myapp -w /usr/src/myapp rust:1.58.0 bash

# Cloning rust-motd   
git clone https://github.com/rust-motd/rust-motd.git
cd rust-motd

# Install GCC Cross compiler
apt-get update
apt-get install -y gcc-arm-linux-gnueabihf

# Setup rust toolchain
rustup target add armv7-unknown-linux-gnueabihf

# Setup cargo linker
mkdir .cargo
echo -e '[build]\n\n[target.armv7-unknown-linux-gnueabihf]\nlinker = "arm-linux-gnueabihf-gcc"' > .cargo/config.toml

# Cross Compile rust-motd
cargo build --target armv7-unknown-linux-gnueabihf

# Checking the binary output
file target/armv7-unknown-linux-gnueabihf/debug/rust-motd

Conclusion

Dear reader, we are now at the end of our journey. I hope this article will give you everything you need to know to cross compile stuff without too much hair pulling.

Introduction#

Method 1: Cross compiling in the rust docker image (Recommended)#

Prerequisite#

Cargo#

Getting the Cross-Compiler#

Method 2: Cross Compiling directly on host system#

Prerequisite#

Cargo#

Getting the Cross Compiler#

Troubleshooting#

File not found arm-none-linux-gnueabihf-gcc#

GCC version problem#

OpenSSL#

TLDR#

Conclusion#

Sources#

Introduction

Method 1: Cross compiling in the rust docker image (Recommended)

Prerequisite

Cargo

Getting the Cross-Compiler

Method 2: Cross Compiling directly on host system

Prerequisite

Cargo

Getting the Cross Compiler

Troubleshooting

File not found arm-none-linux-gnueabihf-gcc

GCC version problem

OpenSSL

TLDR

Conclusion

Sources