ARM64 Support #682

New Issue

davidsbond · 2020-10-16T19:56:23Z

davidsbond commented

2020-10-16 19:56:23 +00:00

Hello, I'm trying to run vikunja on arm64 architecture, when I attempt to pull the image I get the following error:

Failed to pull image "vikunja/api:latest": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/vikunja/api:latest": failed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:f962b6a3b5aa1af0693a4e8f17266deac96ad1894fcda40c89fbe583451b9a48: not found

I suspect this is because you push images for linux/arm but not for linux/arm64. Could this be added to the build process?

Hello, I'm trying to run vikunja on arm64 architecture, when I attempt to pull the image I get the following error: ``` Failed to pull image "vikunja/api:latest": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/vikunja/api:latest": failed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:f962b6a3b5aa1af0693a4e8f17266deac96ad1894fcda40c89fbe583451b9a48: not found ``` I suspect this is because you push images for linux/arm but not for linux/arm64. Could this be added to the build process?

konrad commented

2020-10-16 20:20:40 +00:00

AFAIK you need an arm64 host to build docker images for arm64. I don't have one, so I'm afraid I can't do that.

Do you have another idea?

AFAIK you need an arm64 host to build docker images for arm64. I don't have one, so I'm afraid I can't do that. Do you have another idea?

davidsbond commented

2020-10-16 20:23:53 +00:00

There is a docker CLI plugin called buildx which allows you to target multiple platforms. I've used this to build arm64 images on amd64 hosts.

As far as building with go, you can set the GOOS and GOARCH environment variables to change the go build binary output

There is a docker CLI plugin called [buildx](https://docs.docker.com/buildx/working-with-buildx/) which allows you to target multiple platforms. I've used this to build arm64 images on amd64 hosts. As far as building with go, you can set the GOOS and GOARCH environment variables to change the `go build` binary output

davidsbond commented

2020-10-16 20:25:04 +00:00

If you build your images using buildx, you wouldn't need to do anything with GOOS and GOARCH since you actually build your binary within the Dockerfile by the looks

If you build your images using `buildx`, you wouldn't need to do anything with GOOS and GOARCH since you actually build your binary within the Dockerfile by the looks

konrad commented

2020-10-16 20:33:42 +00:00

Oh we already have binaries for pretty much all plattforms (including arm64). Since we need cgo support because of sqlite, we can't build that by just setting GOOS and GOARCH though but that's a whole other story.

I've never heard of buildx, it looks really promising. I'm not sure if we could put that in the ci though since this would require a) changing settings on the docker daemon of the auto-provisioned hosts and b) would require a modified drone docker builder plugin since the command is different (I'm not even sure if the plugin is using the command or just communicating with the docker daemon directly? Either way).

Would you be up for a PR?

Oh we already have binaries for [pretty much all plattforms](https://dl.vikunja.io/api/master) (including arm64). Since we need cgo support because of sqlite, we can't build that by just setting `GOOS` and `GOARCH` though but that's a whole other story. I've never heard of buildx, it looks really promising. I'm not sure if we could put that in the ci though since this would require a) changing settings on the docker daemon of the auto-provisioned hosts and b) would require a modified drone docker builder plugin since the command is different (I'm not even sure if the plugin is using the command or just communicating with the docker daemon directly? Either way). Would you be up for a PR?

davidsbond commented

2020-10-16 20:37:08 +00:00

Sure, I can take a look. I've not used drone before but will do some reading!

👍 1

konrad commented

2020-10-16 20:43:43 +00:00

Awesome!

The drone plugin we're using is the official one (https://github.com/drone/drone-docker), nothing fancy (yet 🙂 )

Drone plugins are really just docker images, so if you know your way around docker I think you will feel pretty much at home.

Awesome! The drone plugin we're using is [the official one](http://plugins.drone.io/drone-plugins/drone-docker/) (https://github.com/drone/drone-docker), nothing fancy (yet 🙂 ) Drone plugins are really just docker images, so if you know your way around docker I think you will feel pretty much at home.

pragtich commented

2020-11-14 07:21:03 +00:00

Hi there,

I happened to be playing with the same challenge.

After some playing with Drone, it became apparent that the easiest way to get this to work, is to have a runner native on ARM64. I tried it on one of these cheap SBC's (NanoPi Neo2) and that seems to work (512 MB RAM seems to be a bit on the small side, somewhat easy to run out of memory).

But then I found this post:
https://www.docker.com/blog/multi-platform-docker-builds/
and I have to say that this is utter genius. I installed QEMU (through Homebrew) and Docker Desktop on my Macbook (OSX), and docker build seems to work as advertised, and the ARM64 machine can successfully pull from Docker hub and run it.

I've not yet tested it on the full Vikunja stack, only on the api server, I want to try that later on but perhaps this might help you to get ARM64 support quite easily?

Hi there, I happened to be playing with the same challenge. After some playing with Drone, it became apparent that the easiest way to get this to work, is to have a runner native on ARM64. I tried it on one of these cheap SBC's (NanoPi Neo2) and that seems to work (512 MB RAM seems to be a bit on the small side, somewhat easy to run out of memory). But then I found this post: https://www.docker.com/blog/multi-platform-docker-builds/ and I have to say that this is utter genius. I installed QEMU (through Homebrew) and Docker Desktop on my Macbook (OSX), and `docker build` seems to work as advertised, and the ARM64 machine can successfully pull from Docker hub and run it. I've not yet tested it on the full Vikunja stack, only on the api server, I want to try that later on but perhaps this might help you to get ARM64 support quite easily?

konrad commented

2020-11-14 18:39:10 +00:00

That looks indeed very promising.

Please correct me if I'm wrong, but does buildx have docker-in-docker support (yet)? I think that's a requirement if we'd use that to build and publish arm64 images. IIRC buildx is still an experimental thing so we might need to wait until we can fully use that.

I'd like to automate as much as possible, especially regarding publishing etc.

That looks indeed very promising. Please correct me if I'm wrong, but does buildx have docker-in-docker support (yet)? I think that's a requirement if we'd use that to build and publish arm64 images. IIRC buildx is still an experimental thing so we might need to wait until we can fully use that. I'd like to automate as much as possible, especially regarding publishing etc.

davidsbond commented

2020-11-14 19:53:53 +00:00

Sorry, I haven't had much of a chance to take a look into this yet, life got busy!

I currently have projects running via github actions that use buildx to push arm images, although I'm not sure if the underlying ubuntu is a docker image or not. @pragtich if you think you can make it work then please give it a go, not sure when I'll have time to properly take a look.

Sorry, I haven't had much of a chance to take a look into this yet, life got busy! I currently have projects running via github actions that use buildx to push arm images, although I'm not sure if the underlying ubuntu is a docker image or not. @pragtich if you think you can make it work then please give it a go, not sure when I'll have time to properly take a look.

pragtich commented

2020-11-16 10:32:10 +00:00

That looks indeed very promising.

Please correct me if I'm wrong, but does buildx have docker-in-docker support (yet)? I think that's a requirement if we'd use that to build and publish arm64 images.

I am just an enthousiast playing around in 'hobby time', and just beginning with docker, so I am afraid I cannot comment on this question. For my learning: why do you need docker-in-docker support?
I did run builds in buildx (running on an Intel amd64 system) that generated arm64 containers that pull and run on my arm64 system (pragtich/vikunja-frontend:v0.15.0 and pragtich/vikunja-api:v0.15.1), but have yet to test them out together in a working system: I might get a chance to try that in the coming few days.

IIRC buildx is still an experimental thing so we might need to wait until we can fully use that.

That's up to you: Experimental features must not be used in production environments.
As far as I understand, just building on an ARM64 host is still the more certain option.

Following that note: building api containers on my NanoPi Neo2 went without a hitch, but the frontend part was very slow and finally errored out. This might be because of the 'just' 512Mb memory that my system has, I might test it on another board that has a tad more memory.

I'd like to automate as much as possible, especially regarding publishing etc.

Makes sense.

> That looks indeed very promising. > > Please correct me if I'm wrong, but does buildx have docker-in-docker support (yet)? I think that's a requirement if we'd use that to build and publish arm64 images. I am just an enthousiast playing around in 'hobby time', and just beginning with docker, so I am afraid I cannot comment on this question. For my learning: why do you need docker-in-docker support? I did run builds in buildx (running on an Intel amd64 system) that generated arm64 containers that pull and run on my arm64 system (pragtich/vikunja-frontend:v0.15.0 and pragtich/vikunja-api:v0.15.1), but have yet to test them out together in a working system: I might get a chance to try that in the coming few days. >IIRC buildx is still an experimental thing so we might need to wait until we can fully use that. That's up to you: [Experimental features must not be used in production environments](https://docs.docker.com/buildx/working-with-buildx/). As far as I understand, just building on an ARM64 host is still the more certain option. Following that note: building `api` containers on my NanoPi Neo2 went without a hitch, but the `frontend` part was very slow and finally errored out. This might be because of the 'just' 512Mb memory that my system has, I might test it on another board that has a tad more memory. > > I'd like to automate as much as possible, especially regarding publishing etc. Makes sense.

konrad commented

2020-11-16 18:54:48 +00:00

why do you need docker-in-docker support?

Drone itself uses docker containers to run the build and test steps. Therefore, to build docker images from whithin a docker container we need support for buildx.

> why do you need docker-in-docker support? Drone itself uses docker containers to run the build and test steps. Therefore, to build docker images from whithin a docker container we need support for buildx.

pragtich commented

2020-11-23 16:10:08 +00:00

Okay, I'll try and set up a test case on a AMD64 machine and see how far I can get with cross-building a Docker image for AMD64 through Drone. Sounds like it could be a challenge. I wonder if the Drone build images will be able to access the QEMU stuff: good chance that things might stall there.

Since I'm renovating the house, it might take a while :-)

Okay, I'll try and set up a test case on a AMD64 machine and see how far I can get with cross-building a Docker image for AMD64 through Drone. Sounds like it could be a challenge. I wonder if the Drone build images will be able to access the QEMU stuff: good chance that things might stall there. Since I'm renovating the house, it might take a while :-)

pragtich commented

2020-12-01 21:36:29 +00:00

So I did some playing around, and indeed did not find an option where the docker-in-docker process allows cross-building. I'm no expert on Docker, though.

What does look promising, is the buildx (experimental) method with QEMU. I got it to successfully cross compile from Drone by using the Drone Exec runner (also experimental, just like buildx), so that the build gets run not inside docker, but on a machine which has the QEMU stuff installed.

A bit frustrating to me is, that you end up recompiling all the binaries that you already had built.

Anyway, with my limited knowledge, these seem to be the simplest options:

Adding an ARM64 build server. My small ARM boards have only 512MB memory which is clearly insufficient. Perhaps I might try on an Odroid C2 that I can repurpose, this has 2GB, if you are interested in the test. Update: yes, building works well on that board.
Drone Exec runner on a system with Docker and QEMU support installed.

@konrad, I might need to consider this mostly done w.r.t. the depth of my available knowledge...

So I did some playing around, and indeed did not find an option where the docker-in-docker process allows cross-building. I'm no expert on Docker, though. What does look promising, is the `buildx` (experimental) method with QEMU. I got it to successfully cross compile from Drone by using the Drone Exec runner (also experimental, just like buildx), so that the build gets run not inside docker, but on a machine which has the QEMU stuff installed. A bit frustrating to me is, that you end up recompiling all the binaries that you already had built. Anyway, with my limited knowledge, these seem to be the simplest options: 1. Adding an ARM64 build server. My small ARM boards have only 512MB memory which is clearly insufficient. Perhaps I might try on an Odroid C2 that I can repurpose, this has 2GB, if you are interested in the test. **Update**: yes, building works well on that board. 2. Drone Exec runner on a system with Docker and QEMU support installed. @konrad, I might need to consider this mostly done w.r.t. the depth of my available knowledge...

konrad commented

2021-02-04 22:11:48 +00:00

There are now (b8ccfbd190) arm64 docker images in addition to the already existing arm builds! 🎉

You should now be able to just pull the vikunja/api and vikunja/frontend images on an arm or arm64 machine.

I couldn't get buildx to work in drone so I figured I'd just get an odroid which is now sitting at my desk, happily building arm images. It is also twice as fast as the previous arm build server I had while building two images at the same time.

Closing this as resolved, please let me know if you have any issues.

If you found this to be useful, please consider buying me a coffe or supporting me through github sponsors.

There are now (https://kolaente.dev/vikunja/api/commit/b8ccfbd1906f05bbeb0038657e8822372300e02d) arm64 docker images in addition to the already existing arm builds! 🎉 You should now be able to just pull the `vikunja/api` and `vikunja/frontend` images on an arm or arm64 machine. I couldn't get buildx to work in drone so I figured I'd just get an odroid which is now sitting at my desk, happily building arm images. It is also twice as fast as the previous arm build server I had while building two images at the same time. Closing this as resolved, please let me know if you have any issues. If you found this to be useful, please consider [buying me a coffe](https://www.buymeacoffee.com/kolaente) or [supporting me through github sponsors](https://github.com/sponsors/kolaente?o=esb).

konrad closed this issue

2021-02-04 22:11:48 +00:00

pragtich commented

2021-02-17 17:56:35 +00:00

Konrad,

That is great news. I installed it right away, and it is running great. Thank you for your effort!

pragtich

Konrad, That is great news. I installed it right away, and it is running great. Thank you for your effort! pragtich

👍 1

konrad commented

2021-02-17 18:47:33 +00:00

Glad it works!