Commit Graph

76 Commits (e11d7fe91e61300601cf89dfeca304b69577d4d4)

Author SHA1 Message Date
Paul Spooren e11d7fe91e rootfs: add malta/be as target
This allows testing of mips_24kc in CIs.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-14 22:16:14 -10:00
Paul Spooren b24eb19785 add downstream dependencie and quote version
The VERSION variable is still missing in downstream jobs even though the
documentation states it could be added like that. Add the VERSION to the
trigger job and also quote it, to avoid possible YAML parsing issues.

Adding a *depend* to the triggered job makes the CI report the
downstream success state instead of instantly marking the upstream job
as success. Meaning, if a downstream SDK or IB job fails, the upstream
job is marked as failed.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-13 10:20:58 -10:00
Paul Spooren 8fade93d39 fixup docker-sdk.sh TARGET usage
copy paste error in docker-sdk used TARGET rather than ARCH as Docker
container tag.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-11 09:04:41 -10:00
Paul Spooren 750df8e0ae add VERSION to individual jobs
Neither pipeline nor global variables are passed to downstream jobs, add
them individually per job even if it is always the same version.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-11 09:02:38 -10:00
Paul Spooren b16454f65b pass VERSION env to downstream jobs
GitLab calls dynamically created jobs "downstream". They don't seem to
inherit any env variables from the main CI job, therefore add the
VERSION variable specifically.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-10 23:03:59 -10:00
Paul Spooren 961306c0a0 remove minor release from version
The OpenWrt branches only contains the major number

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-10 22:01:09 -10:00
Paul Spooren c3b582b50e version fixup
BRANCH was missing in the generate_targets.sh script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-10 21:04:49 -10:00
Paul Spooren 962d0fb6c0 replace BRANCH with VERSION
The generate_targets.sh script requires to run on a specific branch to
only generate jobs for existing targets. The BRANCH variable was however
used for VERSION rather than the branch running on.

This changes the behaviour by automatically selecting the correct branch
and only taking the new VERSION env variable into account.

By doing so new Docker container tags include `snapshot` rather than
`master`, however for compatibility with existing setups the `master`
tag is still added.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-11 08:06:37 +02:00
Paul Spooren 435653e3d5 remove obsolete TARGET env from rootfs
The TARGET variable is determined via the job name. DRY

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-07 08:20:31 +02:00
Paul Spooren 55df8c7cef support ARCH tagging
For the rootfs containers it can be of interest to run a root container
based on used architecture. Say a CI builds a specific architecture and
a container should run the specific packages. Instead of maintaining the
mapping upstream this should happen via container tags.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-07 08:20:31 +02:00
Paul Spooren 2c562e71c0 deploy: Retry twice
Currently some deploy job fails because of hub.docker.com issues, just
retry the deploy job in such cases.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-05 13:49:45 -10:00
Paul Spooren c48da5786d fixup: docker-sdk TARGET_TAG delimiter
The script tried to tag Docker images with targets like
`target/subtarget` while `/` can't be used in tag names.

Use `tr` to replace `/` with `-`.

Error message in CI:

```
Error parsing reference: "openwrtorg/sdk:cns3xxx/generic-master" is not
a valid repository/tag: invalid reference format
```

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 14:27:29 -10:00
Paul Spooren ca9e4c93d6 generate_targets: adapt to dump script name
The `dumpinfo.pl` script is now part of upstream openwrt.git but was
renamed to `dump-target-info.pl` to have a more verbose name.

Reflect that in the `gen_targets.sh` scripts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 07:35:26 -10:00
Paul Spooren ab85512508 dynamic downstream job creation based on targets
Instead of having a hard coded list of available targets, this approach
makes use os the dumpinfo.pl script that automatically finds available
targets existing in the OpenWrt buildroot. The advantage is to
automatically adapt to available targets in different branches.

Additionally the tagging of SDK containers is improved. While it is
still possible to use SDK based on target tag, it is now also possible
to chose the target based on architecutre. This way less SDK container
are build as one architecutre covers multiple targets.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-03 09:10:16 +02:00
Paul Spooren 570ecf2da5 Fix targets containing underscores
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 11:37:41 -10:00
Paul Spooren f56033a6e0 targets: fixup SDK target missing underscore
The underscore is now used to determine the target, it was only updated
for the ImageBuilders but not SDKs. This commit fixed the deploy step
for SDKs.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 09:56:25 -10:00
Paul Spooren 368c530d00 testing
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 09:36:25 -10:00
Petr Štetiar 50129f2522 testing explicit build stage
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar cd139b3266 ci: remove unused TARGETS variable
Seems like TARGETS variable is not used anywhere.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 5ea4c28324 ci: targets: remove ixp4xx target
It's not available in snapshot and 19.07 release anymore.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar d69bf76835 ci: targets: add bcm targets from master
They were renamed from brcm to bcm, add as well new bcm2711 subtarget.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar be87273cd4 ci: targets: build brcm targets only for releases
In master they were renamed from brcm to bcm.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar fe1ed2330a ci: targets: build ar71xx only for releases
ar71xx is source-only in master for some time.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 2e7fa1b36d ci: targets: remove ar7/ac49x and ar7/generic
Seems like those targets don't exist anymore.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar e76c77d806 ci: refactor IB/SDK targets for DRY
We've already target specified in the CI job name, so just use this
value for TARGET variable instead of duplicating the same value.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 6a0584a6d5 ci: fix imagebuilder test for 19.07 as well
That check is currently failing due to different x86 image generation
code refactoring which changed naming of resulting images so they're now
different in master squashfs-combined Vs combined-squashfs in 19.07.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-05-27 09:47:31 -10:00
Paul Spooren a83630cde4 allow defining of key folder variable
This allows to overwrite GNUPGHOME and USIGNHOME for local testing.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-05-21 08:24:50 -10:00
Paul Spooren 0b66aeac08 sdk: run syncs and sleeps between IO intense steps
This tries to fix failing CIs when files are not found which should have
been there.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-05-21 08:23:14 -10:00
Baptiste Jonglez 2176ce8311 docker-build: Fix compatibility with older Docker versions 2020-05-20 11:24:08 -10:00
Petr Štetiar a7c9b69238 ci: fix imagebuilder test for x86/64
That check is currently failing due to x86 image generation code
refactoring which changed naming of resulting images.

 ls: cannot access './bin/targets/x86/64/*combined-squashfs.img.gz': No such file or directory

Fixes #48
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-05-05 20:40:06 -10:00
Paul Spooren 87db4a5c78 docker-common: add 18.06 usign key
Without the key verification may fail as both usign and pgp is used when
signatures are available.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-10 16:51:48 +01:00
Paul Spooren 7805358c4f add "GP key for 18.06 release builds" key
This key is used for the more recnet 18.06 releses and currently
missing.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren 355d3c8d56 docker-common: verify usign sha256 sums
This was added to packages.git CI before[0].

Use -c instead of --check as alpine does not support to longer version.

[0]: eb5a9990ed

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren f3e9e3230c sdk: run make prereq in tests
Add the tests from packages.git ci[0] to docker test.

[0]: a6d5b5d1c1

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren 200f563a8a set pre-build as first stage again
This was changed without fully thinking on it and should obviously be
corrected.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-08 13:27:21 +01:00
Paul Spooren 931d156460 readd docker login on deploy
This was removed during debugging and accidently forgotten to readd. It
fails to pass if no Docker credentials are given, which would be always
the case except on openwrtorg/master. In the future a test docker hub
account should be used instead of disabling it.

Fixes: 475f2f1a97

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-07 09:37:13 -10:00
Paul Spooren 475f2f1a97 fixup prebuild deploy container
define USIGNHOME and GNUPGHOME to store the keys

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 18:16:03 -10:00
Paul Spooren a940fa094c make sdk test more verbose
using compile V=s should give some details in case this test fails.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 12:51:54 -10:00
Paul Spooren 54d80c0363 use deploy container
This sets up gpg/usign once and reuses it for the ~160 sdk/ib containes

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 12:29:29 -10:00
Paul Spooren 806910a4b8 fixup find -printf
-printf is apperently not always available

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:19:14 -10:00
Paul Spooren d5e893c8f0 use find to replace previous glob behaviour
to make shellcheck happy we can no longer use glob, so now the script
use find to get the actual filename.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:00:41 -10:00
Paul Spooren e026c4154b replace true with exit when download fails
previously multiple targets were combined in a single job so failing
targets should not stop others from being deployed. However as there are
now single builds per target this is no longer required.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:00:06 -10:00
Paul Spooren e3770961ad expand gitignore with usign/ & gpg/
these files usually appear after local testing

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 13:56:14 -10:00
Paul Spooren 30ed87f52e Update CI badge after move to gitlab.com
The former test instance of fe80.eu is no longer used, instead the
gitlab.com server run the scripts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 11:13:19 -10:00
Petr Štetiar 34adbafaeb ci: refactor into separate YAML files under .gitlab/ci/ dir
It makes the YAML files more comprehensible.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Petr Štetiar 964f42b5c4 fix warnings reported by shellcheck and remove pythonisms
In ./docker-download.sh:
 SC2086: Double quote to prevent globbing and word splitting.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2086: Double quote to prevent globbing and word splitting.

In ./gen-targets.sh:
 SC2188: This redirection doesn't have a command. Move to its command (or use 'true' as no-op).
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2188: This redirection doesn't have a command. Move to its command (or use 'true' as no-op).
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.

In ./docker-imagebuilder.sh:
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

In ./docker-rootfs.sh:
 SC2039: In POSIX sh, == in place of = is undefined.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

In ./docker-sdk.sh line:
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Petr Štetiar 43c17ad72b ci: add shellcheck for shell script checking
In order to make the shell scripts more robust.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Lucas Ramage 567eaa2716 Fix typo in README 2019-10-03 18:53:26 -10:00
Paul Spooren 2db99b240b introduce gen-targets.sh script
This improves readability as it auto generates the target list.
2019-10-02 20:46:24 -10:00
Paul Spooren 7d1ad316b6 Dockerfile: install rsync
Some packages need rsync to be installed on the host.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-19 11:12:03 -10:00