Automate all the things #2

New Issue

konrad · 2023-01-31T20:58:41Z

konrad commented

2023-01-31 20:58:41 +00:00

We need a pipeline for Drone to automatically build and push new releases of the chart (Is there a build step? I don't know enough about Helm).

Should we only publish when tagging a release in this repo? Should the version number follow the release numbers for the frontend and api?

![image](/attachments/e216bd37-dfaf-4c8a-b649-a9a263e50320) We need a pipeline for Drone to automatically build and push new releases of the chart (Is there a build step? I don't know enough about Helm). Should we only publish when tagging a release in this repo? Should the version number follow the release numbers for the frontend and api?

image.png

193 KiB

vlasov-y commented

2023-02-02 07:49:50 +00:00

Is there a build step? I don't know enough about Helm

In order to publish the chart as a package in Gitea.

Run helm package .

It will produce a tarball with the chart

Push that chart to the repository

You can use either curl to do that or helm cm-push plugin. This is described in the contributing section.
I would suggest to use curl, because it is present in most CIs and you can avoid installation of the cm-push plugin for Helm.

curl --user '<username>:<password>' -X POST --upload-file './<archive>.tgz' https://kolaente.dev/api/packages/vikunja/helm/api/charts

You can just specify ./*.tgz as an archive name.

Should we only publish when tagging a release in this repo?

Chart version is specified in the Chart.yaml file, hence, I would suggest triggering pipeline on change in Chart.yaml only. Other files not important as a trigger source.

Should the version number follow the release numbers for the frontend and api?

I am not sure, that got your point :) Anyway, there is a line in the Chart.yaml that stands for application version. I have build the chart in the way, when this particular value is used as a tag for both frontend and api images. So that, you can just set appVersion to newer version in order to update both applications.

Nevertheless, end user is able to override the application version during the deployment (without chart rebuild), just setting up tags values. If tag is not an empty string, appVersion from Chart.yaml is ignored. There is a place where this behavior is implemented: API, Frontend.

P.S. @konrad feel free to ping me with mentioning my login in the message :) I have spotted this issue occasionally. Or just assign it on me.

> Is there a build step? I don't know enough about Helm In order to publish the chart as a package in Gitea. 1. Run `helm package .` It will produce a tarball with the chart 2. Push that chart to the repository You can use either curl to do that or helm cm-push plugin. This is described in the [contributing section.](https://kolaente.dev/vikunja/helm-chart#contributing) I would suggest to use curl, because it is present in most CIs and you can avoid installation of the cm-push plugin for Helm. ```shell curl --user '<username>:<password>' -X POST --upload-file './<archive>.tgz' https://kolaente.dev/api/packages/vikunja/helm/api/charts ``` You can just specify `./*.tgz` as an archive name. > Should we only publish when tagging a release in this repo? Chart version is specified in the [Chart.yaml](https://kolaente.dev/vikunja/helm-chart/src/commit/496bf81f8e68169f829e8d13a081f9ddc07d068f/Chart.yaml#L13) file, hence, I would suggest triggering pipeline on change in Chart.yaml only. Other files not important as a trigger source. > Should the version number follow the release numbers for the frontend and api? I am not sure, that got your point :) Anyway, there is a [line](https://kolaente.dev/vikunja/helm-chart/src/commit/496bf81f8e68169f829e8d13a081f9ddc07d068f/Chart.yaml#L14) in the Chart.yaml that stands for application version. I have build the chart in the way, when this particular value is used as a tag for both frontend and api images. So that, you can just set appVersion to newer version in order to update both applications. Nevertheless, end user is able to override the application version during the deployment (without chart rebuild), just setting up [tags values](https://kolaente.dev/vikunja/helm-chart/src/commit/496bf81f8e68169f829e8d13a081f9ddc07d068f/values.yaml#L36). If tag is not an empty string, appVersion from Chart.yaml is ignored. There is a place where this behavior is implemented: [API](https://kolaente.dev/vikunja/helm-chart/src/commit/496bf81f8e68169f829e8d13a081f9ddc07d068f/templates/api/deployment.yaml#L49), [Frontend](https://kolaente.dev/vikunja/helm-chart/src/commit/496bf81f8e68169f829e8d13a081f9ddc07d068f/templates/frontend/deployment.yaml#L49). P.S. @konrad feel free to ping me with mentioning my login in the message :) I have spotted this issue occasionally. Or just assign it on me.

konrad commented

2023-02-02 10:14:55 +00:00

I would suggest to use curl, because it is present in most CIs and you can avoid installation of the cm-push plugin for Helm.

Drone runs with Docker images only - it has nothing by default and everything we need as long as there's a docker image for it. Looks like there's a docker image with cm-push we could use so it does not really matter which route we go.

Chart version is specified in the Chart.yaml file, hence, I would suggest triggering pipeline on change in Chart.yaml only. Other files not important as a trigger source.

What happens if a chart with the same version is published again? Does it make sense to template the version number in Chart.yml and build a pipeline around it so that it would only publish when tagging the repo? (replacing the version number in the chart file with the tag name)

I am not sure, that got your point :) Anyway, there is a line in the Chart.yaml that stands for application version.

There might be occasions where the last api and frontend release have different version numbers (as is the case right now). Probably better to specify them individually, even if the numbers will be the same most of the time.

I do however aim to have at least the same minor versions for both the frontend and the api. For example, 0.20.0 were released for both at the same time. The question is if we should follow that convention when releasing new chart versions? (For the version numbers of the chart)

> I would suggest to use curl, because it is present in most CIs and you can avoid installation of the cm-push plugin for Helm. Drone runs with Docker images only - it has nothing by default and everything we need as long as there's a docker image for it. [Looks like there's a docker image with cm-push](https://hub.docker.com/r/truegunner/ci-base-helm) we could use so it does not really matter which route we go. > Chart version is specified in the Chart.yaml file, hence, I would suggest triggering pipeline on change in Chart.yaml only. Other files not important as a trigger source. What happens if a chart with the same version is published again? Does it make sense to template the version number in Chart.yml and build a pipeline around it so that it would only publish when tagging the repo? (replacing the version number in the chart file with the tag name) > I am not sure, that got your point :) Anyway, there is a line in the Chart.yaml that stands for application version. There might be occasions where the last api and frontend release have different version numbers (as is the case right now). Probably better to specify them individually, even if the numbers will be the same most of the time. I do however aim to have at least the same minor versions for both the frontend and the api. For example, 0.20.0 were released for both at the same time. The question is if we should follow that convention when releasing new chart versions? (For the version numbers of the chart)

vlasov-y commented

2023-02-04 06:42:48 +00:00

Drone runs with Docker images only - it has nothing by default and everything we need as long as there's a docker image for it. Looks like there's a docker image with cm-push we could use so it does not really matter which route we go.

Okay :) this is a minor technical detail

What happens if a chart with the same version is published again? Does it make sense to template the version number in Chart.yml and build a pipeline around it so that it would only publish when tagging the repo? (replacing the version number in the chart file with the tag name)

It will be overriden in the repository. I see that you want to use tags to version the chart, but I am not sure that it is a best way to go, while it is possible anyway. I always prefer to set version in Chart by hands and update it in commits.

There might be occasions where the last api and frontend release have different version numbers (as is the case right now). Probably better to specify them individually, even if the numbers will be the same most of the time.

Let's image the following case.

Chart version	Chart appVersion	Frontend tag	Backend tag	What to do
0.1.0	0.22.0	0.22.0	0.22.0	Leave tags in values empty, so that appVersion will be used
0.1.1	0.22.1	0.22.1	0.22.0	Set backed tag in Values to 0.22.0, but leave frontend as empty
0.2.0	0.23.0	0.23.0	0.23.0	Leave tags in values empty, so that appVersion will be used
0.2.1	0.23.5	0.23.5	0.23.3	Set backed tag in Values to 0.22.0, but leave frontend as empty

I do however aim to have at least the same minor versions for both the frontend and the api. For example, 0.20.0 were released for both at the same time. The question is if we should follow that convention when releasing new chart versions? (For the version numbers of the chart)

Chart version is to be increased in case of any change to its content, even a single line. Saying change, I mean value or template change. If we fix typos or whitespaces - we can use the same version.

Version name of chart should not be the same as appVersion, but rules that if try to follow are:

Update Chart feature version in case of small changes to chart template or appVersion feature version update
Update Chart minor version in case of appVersion minor version update
Update Chart major version in case of not backward-compatible changes to the chart version or appVersion major update

P.S. @konrad Sorry for the long reploy timeout, had a hard week

> Drone runs with Docker images only - it has nothing by default and everything we need as long as there's a docker image for it. Looks like there's a docker image with cm-push we could use so it does not really matter which route we go. Okay :) this is a minor technical detail > What happens if a chart with the same version is published again? Does it make sense to template the version number in Chart.yml and build a pipeline around it so that it would only publish when tagging the repo? (replacing the version number in the chart file with the tag name) It will be overriden in the repository. I see that you want to use tags to version the chart, but I am not sure that it is a best way to go, while it is possible anyway. I always prefer to set version in Chart by hands and update it in commits. > There might be occasions where the last api and frontend release have different version numbers (as is the case right now). Probably better to specify them individually, even if the numbers will be the same most of the time. Let's image the following case. | Chart version | Chart appVersion | Frontend tag | Backend tag | What to do | | ------------- | ---------------- | ------------ | ----------- | --------------------------------------------------------------- | | 0.1.0 | 0.22.0 | 0.22.0 | 0.22.0 | Leave tags in values empty, so that appVersion will be used | | 0.1.1 | 0.22.1 | 0.22.1 | 0.22.0 | Set backed tag in Values to 0.22.0, but leave frontend as empty | | 0.2.0 | 0.23.0 | 0.23.0 | 0.23.0 | Leave tags in values empty, so that appVersion will be used | | 0.2.1 | 0.23.5 | 0.23.5 | 0.23.3 | Set backed tag in Values to 0.22.0, but leave frontend as empty | > I do however aim to have at least the same minor versions for both the frontend and the api. For example, 0.20.0 were released for both at the same time. The question is if we should follow that convention when releasing new chart versions? (For the version numbers of the chart) Chart version is to be increased in case of **any** change to its content, even a single line. Saying change, I mean value or template change. If we fix typos or whitespaces - we can use the same version. Version name of chart should not be the same as appVersion, but rules that if try to follow are: 1. Update Chart feature version in case of small changes to chart template or appVersion feature version update 2. Update Chart minor version in case of appVersion minor version update 3. Update Chart major version in case of not backward-compatible changes to the chart version or appVersion major update P.S. @konrad Sorry for the long reploy timeout, had a hard week

konrad commented

2023-02-14 10:10:19 +00:00

Thanks for the explanations! Sorry for the late reply, there's been too much going on last week outside Vikunja.

Chart version is to be increased in case of any change to its content, even a single line. Saying change, I mean value or template change. If we fix typos or whitespaces - we can use the same version.

Okay, that makes sense. I think my reasoning to only set the version number and publish based on a tag name was to accidentally prevent overriding an old version when changing something without increasing the version number in the same commit. As long as increasing the version number is a manual task, that will happen at some point. Better make it explicit via tags.

If we fix typos or whitespaces - we can use the same version.

But as long as the chart will be published on each push to the main branch, this would override the last release even if nothing changed?

Thanks for the explanations! Sorry for the late reply, there's been too much going on last week outside Vikunja. > Chart version is to be increased in case of any change to its content, even a single line. Saying change, I mean value or template change. If we fix typos or whitespaces - we can use the same version. Okay, that makes sense. I think my reasoning to only set the version number and publish based on a tag name was to accidentally prevent overriding an old version when changing something without increasing the version number in the same commit. As long as increasing the version number is a manual task, that will happen at some point. Better make it explicit via tags. > If we fix typos or whitespaces - we can use the same version. But as long as the chart will be published on each push to the main branch, this would override the last release even if nothing changed?

vlasov-y commented

2023-02-21 13:48:17 +00:00

Hi @konrad !

Better make it explicit via tags.

Okay, that will work.

But as long as the chart will be published on each push to the main branch, this would override the last release even if nothing changed?

Not the last one, but one that matches version field in the Chart.yaml, but yes, it will override.

Hi @konrad ! > Better make it explicit via tags. Okay, that will work. > But as long as the chart will be published on each push to the main branch, this would override the last release even if nothing changed? Not the last one, but one that matches version field in the Chart.yaml, but yes, it will override.

👍 1

vlasov-y commented

2023-02-21 13:49:59 +00:00

@konrad, I have debugged the chart and pushed it to packages (0.1.2)! I have Vikunja launched on my k3s cluster at https://vikunja.vlasov.pro. Let me know if You need credentials and want to test it.

It uses unstable tag for both api and frontend for now, and we will migrate to stable version after new Dockerfiles will propagate to stable version.

If it is possible, we can make a call and I will show all technical points You want to know about.

@konrad, I have debugged the chart and pushed it to packages (0.1.2)! I have Vikunja launched on my k3s cluster at https://vikunja.vlasov.pro. Let me know if You need credentials and want to test it. It uses unstable tag for both api and frontend for now, and we will migrate to stable version after new Dockerfiles will propagate to stable version. If it is possible, we can make a call and I will show all technical points You want to know about.

konrad commented

2023-02-23 15:11:48 +00:00

I have debugged the chart and pushed it to packages (0.1.2)! I have Vikunja launched on my k3s cluster at https://vikunja.vlasov.pro.

Looks good! Two things I wonder:

Can we put the api and frontend on the same (sub) domain?
Can we configure openid auth through the chart?

If it is possible, we can make a call and I will show all technical points You want to know about.

I'd love to :) I'll ping you via mail to figure out the details.

> I have debugged the chart and pushed it to packages (0.1.2)! I have Vikunja launched on my k3s cluster at https://vikunja.vlasov.pro. Looks good! Two things I wonder: * Can we put the api and frontend on the same (sub) domain? * Can we configure openid auth through the chart? > If it is possible, we can make a call and I will show all technical points You want to know about. I'd love to :) I'll ping you via mail to figure out the details.

vlasov-y commented

2023-02-27 08:58:20 +00:00

Can we put the api and frontend on the same (sub) domain?

Right now - no, but I can implement that as well.

Can we configure openid auth through the chart?

Chart passes complete config.yaml to the API, so any key that are present in you configuration can be passed to the API.

> Can we put the api and frontend on the same (sub) domain? Right now - no, but I can implement that as well. > Can we configure openid auth through the chart? Chart passes complete config.yaml to the API, so any key that are present in you configuration can be passed to the API.

konrad commented

2023-02-27 09:30:35 +00:00

Thanks for the meeting!

Some notes.

To publish the chart properly on Artifacthub, we need to claim ownership of the Vikunja org first so that it shows up as official. This needs a metadata file and a way to serve that from our Helm repo (Gitea)
I'll try to add a drone pipeline to publish automatically on git tag.

Thanks for the meeting! Some notes. 1. To publish the chart properly on Artifacthub, we need to claim ownership of the Vikunja org first so that it shows up as official. This needs a [metadata file](https://github.com/artifacthub/hub/blob/master/docs/metadata/artifacthub-repo.yml) and a way to serve that from our Helm repo (Gitea) 2. I'll try to add a drone pipeline to publish automatically on git tag.

vlasov-y commented

2023-02-27 09:36:04 +00:00

@konrad

Thanks for the call resume you have posted above!

I have a question regarding hosting on one domain. It is obvious that traffic will be splited basic on URI path. If I want to host API at /qwer1234 - what config option I have to set so that API will be aware of such path?

@konrad Thanks for the call resume you have posted above! I have a question regarding hosting on one domain. It is obvious that traffic will be splited basic on URI path. If I want to host API at /qwer1234 - what config option I have to set so that API will be aware of such path?

konrad commented

2023-02-27 09:45:07 +00:00

There's no option to make the api aware of that path. I think you'd need to work with the X-Forwarded-Path header, but I never tested that.

The bigger problem is hosting the frontend on a sub path as the bundle assumes it is hosted on a domain. There's a docs article on how to build is with a subpath in mind: https://vikunja.io/docs/running-vikunja-in-a-subdirectory/ but that's static.

TBH I think it would be fine if we just wouldn't support hosting on a sub path and start figuring something out when people ask for it.

There's no option to make the api aware of that path. I think you'd need to work with the `X-Forwarded-Path` header, but I never tested that. The bigger problem is hosting the frontend on a sub path as the bundle assumes it is hosted on a domain. There's a docs article on how to build is with a subpath in mind: https://vikunja.io/docs/running-vikunja-in-a-subdirectory/ but that's static. TBH I think it would be fine if we just wouldn't support hosting on a sub path and start figuring something out when people ask for it.

vlasov-y commented

2023-02-27 10:47:01 +00:00

TBH I think it would be fine if we just wouldn't support hosting on a sub path and start figuring something out when people ask for it.

That sound great! It was always easier to enable CORS and create a subdomain then play with different paths on the same domain.

> TBH I think it would be fine if we just wouldn't support hosting on a sub path and start figuring something out when people ask for it. That sound great! It was always easier to enable CORS and create a subdomain then play with different paths on the same domain.

👍 1

konrad referenced a pull request that will close this issue

2023-03-10 17:39:42 +00:00

feat: add ci config #3

konrad commented

2023-03-10 17:40:14 +00:00

PR for automation is up: #3

PR for automation is up: https://kolaente.dev/vikunja/helm-chart/pulls/3

konrad closed this issue

2023-03-10 21:30:03 +00:00

konrad referenced this issue from a commit

2023-03-10 21:30:03 +00:00