Docker Image / Samba

olilarkin · 17 January 2020 23:37

Hello,

We have got iPlug2 building VST3 plugins on Linux so I have started playing with the Elk board again

It’s great to see the instructions for the docker image elkpi-sdk/running_docker_container_on_macos.md at master · elk-audio/elkpi-sdk · GitHub

Any reason it would not possible to publish a docker image on docker hub with the cross compiling toolchain all set up?
Also do you know if it is essential to do the samba stuff? Docker images i have used often map the the folder where you execute to the image

thanks!

oli

Stefano · 18 January 2020 16:35

Hi @olilarkin,

That’s wonderful to hear! Many people are looking for higher-level and cross-platform frameworks that are able to spit out VST3 for Linux.

None at all, other than we are not very expert with Docker, at least in our core team

If anyone would like to contribute that, we would be super happy!

Pretty much the same reason for the question above. As I wrote in the doc, those instructions are adapted almost verbatim from the CROPS eSDK docs, so if you want, please try out if direct mapping works and let us know

Cheers,
Stefano

frederic · 20 January 2020 08:46

I did try to mount a volume instead of using the samba option in macOS. I remember having problems although I’m not sure what happened. It should work theoretically. To somehow automate the building process, what I ended up doing is some simple scripts for samba mounting in macOS, and modified the CROPS eSDK init python script to automatically compile source files.

in the ELK workshop at ADC however they had a build server to which you could drop source files via HTTP and would return a compiled binary. I think something like that could be implemented using Docker and that would be really awesome.

Dauq · 27 January 2020 21:56

@frederic To use a docker mount instead of samba under OSX, I think you need to create and mount a Case Sensitive OSX volume into docker.

The build chain has name conflicts when running in the default case insensitive (osx) volumes. I don’t think Elk owns these dependencies to make this go away.

See: Dependency for ProJucer

Dauq · 27 January 2020 22:02

@Stefano Can we add this to a README.md or FAQ?

In this millenium, it’s common to want to use docker and avoid samba.

FYI, the link above has helpful info and sample commands for running with docker. Suspect it’s a 30 minute to half day task for someone at Elk to test this out. And use of docker containers is a great way to setup automated builds and tests.

frederic · 28 January 2020 12:37

Thanks for the tip @Dauq, I’ll try the case sensitive thing at some point.

Stefano · 29 January 2020 01:49

Sure,
we’ll prepare some update in the README of the SDK repo if this helps.

Let us give a shot first, we don’t have many macOS machines around so it might not be immediate.

toani · 20 2020 18:13

Hi!

(Posting here due to the title Docker Image / Samba)

This may be basic. I couldn’t pass files to the docker image. Neither source files nor the cross-compiling installation script.

I went opened a Docker Quickstart Terminal window. I followed all instructions for MacOs. When I connect in finder via Samba, the guest credentials to not work.

It either shakes as a negative or there is a dialog that states the user/pass is wrong. Does anyone know why?

A workaround to this is the docker command as shown here:

docker cp elk_juce_example.zip samba:/workdir/code/elk_juce_example.zip

I hope this is helpful. Thanks.

Dauq · 21 2020 09:19

Use a docker mount of a case sensitive osx file system. See post above.

toani · 27 2020 20:14

I created the CaseSensitive volume. Copied the sh file on a directory /Volumes/CaseSensitive/elkbuild/workdir. Here is my sequence:

docker run -it --rm -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir busybox chown -R 1000:1000 /workdir
mkdir /Volumes/CaseSensitive/elkbuild
mkdir /Volumes/CaseSensitive/elkbuild/workdir

Then I open a bash session and manually copy the file:

docker run -it --rm -v /Users/user/location/Elk/SW/:/SW -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir bash

Then I try to install the toolchain. But I get an error.

CaseSensitive $ docker run -it --rm -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir     busybox chown -R 1000:1000 /workdir
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
bdbbaa22dec6: Pull complete 
Digest: sha256:6915be4043561d64e0ab0f8f098dc2ac48e077fe23f488ac24b665166898115a
Status: Downloaded newer image for busybox:latest
CaseSensitive $ cp /SW/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh /workdir/
cp: directory /workdir does not exist
CaseSensitive $ docker run -it --rm -v /Users/location/Elk/SW/:/SW -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir bash
bash-5.0# cp /SW/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh /workdir/
bash-5.0# ls /workdir/
elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh
bash-5.0# exit
exit
CaseSensitive $ docker run --rm -it -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir crops/extsdk-container --url /workdir/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh
Unable to find image 'crops/extsdk-container:latest' locally
latest: Pulling from crops/extsdk-container
0a01a72a686c: Pull complete 
cc899a5544da: Pull complete 
19197c550755: Pull complete 
716d454e56b6: Pull complete 
5a3ee5b98d0c: Pull complete 
583476fc4e3c: Pull complete 
8aca12f7c43a: Pull complete 
15774e0ad4e8: Pull complete 
fb69732d8be9: Pull complete 
6ad118a8a032: Pull complete 
91d36b431fe1: Pull complete 
Digest: sha256:7c61541a79800b427945402b73d26aacf994fedba4673b5ffb669bedc74233cb
Status: Downloaded newer image for crops/extsdk-container:latest
Traceback (most recent call last):
  File "/usr/bin/esdk-launch.py", line 101, in <module>
    os.chmod(esdk_installer, stat.S_IXUSR | oldmode)
OSError: [Errno 1] Operation not permitted: '/workdir/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh'

Running with sudo gives me a different kind of error:

sudo docker run --rm -it -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir crops/extsdk-container --url /workdir/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh
Password:
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

See 'docker run --help'.

So I went back and run again:

docker run -it --rm -v /Volumes/CaseSensitive/elkbuild/workdir:/workdir busybox chown -R 1000:1000 /workdir
cp /Users/myuser/Elk/SW/elk-glibc-x86_64-elk-sika-image-dev-cortexa7t2hf-neon-vfpv4-raspberrypi3-toolchain-1.0.sh /Volumes/CaseSensitive/elkbuild/workdir

The run this what appeared to be successful because it started like this:

ELK Poky based distribution for Yocto Project SDK installer version 1.0   =======================================================================
You are about to install the SDK to "/workdir". Proceed [Y/n]? Y
Extracting SDK........................tar: ./sysroots/x86_64-pokysdk-linux/usr/libexec/arm-elk-linux-gnueabi/gcc/arm-elk-linux-gnueabi/8.3.0/cc1plus: Wrote only 3584 of 10240 bytes
.tar: ./sysroots/x86_64-pokysdk-linux/usr/libexec/arm-elk-linux-gnueabi/gcc/arm-elk-linux-gnueabi/8.3.0/plugin/gengtype: Cannot write: No space left on device

Followed by an infinity of similar failures: tar: ./sysroots/x86_64-pokysdk-linux/lib/libnss_files-2.29.so: Cannot write: No space left on device.

Since my volume was only 6Gb then I resized it and repeated with 22Gb this time. I got exactly the same errors.

I also tried

CaseSensitive $ docker run --rm -it -v /volumes/casesensitive/elkbuild/workdir:/workdir crops/extsdk-container
sudo: unknown user: sdkuser
sudo: unable to initialize policy plugin

I guess I am about to set up a Ubuntu machine. Can’t get docker to work on my OSX El Capitan.

Any ideas? Help. Thanks.

Dauq · 27 2020 20:50

Have you tried increasing the disk image size for docker?

toani · 28 2020 04:32

Very helpful hint. Thank you.

I don’t have the app because of El Capitan. I found here how to do it from the docker toolbox.
So I resized it to 19Gb. When I bash into the system, and do a df -h I see the 19Gb went elsewhere but to /workdir where they are most needed. Figuring that out.

Filesystem                Size      Used Available Use% Mounted on
overlay                  17.4G     67.5M     16.4G   0% /
tmpfs                    64.0M         0     64.0M   0% /dev
tmpfs                   494.7M         0    494.7M   0% /sys/fs/cgroup
shm                      64.0M         0     64.0M   0% /dev/shm
/Users                  210.5G    173.3G     37.1G  82% /SW
tmpfs                   890.5M    890.5M         0 100% /workdir
/dev/sda1                17.4G     67.5M     16.4G   0% /etc/resolv.conf
/dev/sda1                17.4G     67.5M     16.4G   0% /etc/hostname
/dev/sda1                17.4G     67.5M     16.4G   0% /etc/hosts
tmpfs                   494.7M         0    494.7M   0% /proc/asound
tmpfs                   494.7M         0    494.7M   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                   494.7M         0    494.7M   0% /proc/scsi
tmpfs                   494.7M         0    494.7M   0% /sys/firmware

Stefano · 28 2020 09:35

Hi @toani,
I’m not a macOS user but for the permission rights maybe take a look at this reply in another thread:
https://forum2.elk.audio/t/dependency-for-projucer/92/19?u=stefano

toani · 1 2020 05:29

Don’t want to go off-topic but this may save time. I recommend (non developers like me) going straight with a linux machine or VM. Get a Ubuntu system. Everything goes through out the package and the installation of packages and dependencies is much easier. This is after how it was all conceived after all so it works. See this post.

frederic · 3 2020 09:10

Well, the Docker pipeline can be improved a bit, but in my case I was really quite close to a “one-click” service for cross compiling a project from macOS with some scripts I wrote to automate things. I stopped development because I need to focus on something else now, but once I get back to my elk project I’ll try to prepare the scripts and make them public together with some instructions.