Commit Graph

216 Commits (c0ddfae186b8bb658f8f11e30393a2b5bc9b82ca)

Author SHA1 Message Date
Paul Spooren 2c562e71c0 deploy: Retry twice
Currently some deploy job fails because of hub.docker.com issues, just
retry the deploy job in such cases.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-05 13:49:45 -10:00
Paul Spooren c48da5786d fixup: docker-sdk TARGET_TAG delimiter
The script tried to tag Docker images with targets like
`target/subtarget` while `/` can't be used in tag names.

Use `tr` to replace `/` with `-`.

Error message in CI:

```
Error parsing reference: "openwrtorg/sdk:cns3xxx/generic-master" is not
a valid repository/tag: invalid reference format
```

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 14:27:29 -10:00
Paul Spooren ca9e4c93d6 generate_targets: adapt to dump script name
The `dumpinfo.pl` script is now part of upstream openwrt.git but was
renamed to `dump-target-info.pl` to have a more verbose name.

Reflect that in the `gen_targets.sh` scripts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 07:35:26 -10:00
Paul Spooren ab85512508 dynamic downstream job creation based on targets
Instead of having a hard coded list of available targets, this approach
makes use os the dumpinfo.pl script that automatically finds available
targets existing in the OpenWrt buildroot. The advantage is to
automatically adapt to available targets in different branches.

Additionally the tagging of SDK containers is improved. While it is
still possible to use SDK based on target tag, it is now also possible
to chose the target based on architecutre. This way less SDK container
are build as one architecutre covers multiple targets.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-03 09:10:16 +02:00
Paul Spooren 570ecf2da5 Fix targets containing underscores
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 11:37:41 -10:00
Paul Spooren f56033a6e0 targets: fixup SDK target missing underscore
The underscore is now used to determine the target, it was only updated
for the ImageBuilders but not SDKs. This commit fixed the deploy step
for SDKs.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 09:56:25 -10:00
Paul Spooren 368c530d00 testing
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-01 09:36:25 -10:00
Petr Štetiar 50129f2522 testing explicit build stage
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar cd139b3266 ci: remove unused TARGETS variable
Seems like TARGETS variable is not used anywhere.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 5ea4c28324 ci: targets: remove ixp4xx target
It's not available in snapshot and 19.07 release anymore.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar d69bf76835 ci: targets: add bcm targets from master
They were renamed from brcm to bcm, add as well new bcm2711 subtarget.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar be87273cd4 ci: targets: build brcm targets only for releases
In master they were renamed from brcm to bcm.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar fe1ed2330a ci: targets: build ar71xx only for releases
ar71xx is source-only in master for some time.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 2e7fa1b36d ci: targets: remove ar7/ac49x and ar7/generic
Seems like those targets don't exist anymore.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar e76c77d806 ci: refactor IB/SDK targets for DRY
We've already target specified in the CI job name, so just use this
value for TARGET variable instead of duplicating the same value.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-06-30 14:12:03 -10:00
Petr Štetiar 6a0584a6d5 ci: fix imagebuilder test for 19.07 as well
That check is currently failing due to different x86 image generation
code refactoring which changed naming of resulting images so they're now
different in master squashfs-combined Vs combined-squashfs in 19.07.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-05-27 09:47:31 -10:00
Paul Spooren a83630cde4 allow defining of key folder variable
This allows to overwrite GNUPGHOME and USIGNHOME for local testing.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-05-21 08:24:50 -10:00
Paul Spooren 0b66aeac08 sdk: run syncs and sleeps between IO intense steps
This tries to fix failing CIs when files are not found which should have
been there.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-05-21 08:23:14 -10:00
Baptiste Jonglez 2176ce8311 docker-build: Fix compatibility with older Docker versions 2020-05-20 11:24:08 -10:00
Petr Štetiar a7c9b69238 ci: fix imagebuilder test for x86/64
That check is currently failing due to x86 image generation code
refactoring which changed naming of resulting images.

 ls: cannot access './bin/targets/x86/64/*combined-squashfs.img.gz': No such file or directory

Fixes #48
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-05-05 20:40:06 -10:00
Paul Spooren 87db4a5c78 docker-common: add 18.06 usign key
Without the key verification may fail as both usign and pgp is used when
signatures are available.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-10 16:51:48 +01:00
Paul Spooren 7805358c4f add "GP key for 18.06 release builds" key
This key is used for the more recnet 18.06 releses and currently
missing.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren 355d3c8d56 docker-common: verify usign sha256 sums
This was added to packages.git CI before[0].

Use -c instead of --check as alpine does not support to longer version.

[0]: eb5a9990ed

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren f3e9e3230c sdk: run make prereq in tests
Add the tests from packages.git ci[0] to docker test.

[0]: a6d5b5d1c1

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-12-03 22:17:10 +01:00
Paul Spooren 200f563a8a set pre-build as first stage again
This was changed without fully thinking on it and should obviously be
corrected.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-08 13:27:21 +01:00
Paul Spooren 931d156460 readd docker login on deploy
This was removed during debugging and accidently forgotten to readd. It
fails to pass if no Docker credentials are given, which would be always
the case except on openwrtorg/master. In the future a test docker hub
account should be used instead of disabling it.

Fixes: 475f2f1a97

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-07 09:37:13 -10:00
Paul Spooren 475f2f1a97 fixup prebuild deploy container
define USIGNHOME and GNUPGHOME to store the keys

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 18:16:03 -10:00
Paul Spooren a940fa094c make sdk test more verbose
using compile V=s should give some details in case this test fails.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 12:51:54 -10:00
Paul Spooren 54d80c0363 use deploy container
This sets up gpg/usign once and reuses it for the ~160 sdk/ib containes

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-11-06 12:29:29 -10:00
Paul Spooren 806910a4b8 fixup find -printf
-printf is apperently not always available

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:19:14 -10:00
Paul Spooren d5e893c8f0 use find to replace previous glob behaviour
to make shellcheck happy we can no longer use glob, so now the script
use find to get the actual filename.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:00:41 -10:00
Paul Spooren e026c4154b replace true with exit when download fails
previously multiple targets were combined in a single job so failing
targets should not stop others from being deployed. However as there are
now single builds per target this is no longer required.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 14:00:06 -10:00
Paul Spooren e3770961ad expand gitignore with usign/ & gpg/
these files usually appear after local testing

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 13:56:14 -10:00
Paul Spooren 30ed87f52e Update CI badge after move to gitlab.com
The former test instance of fe80.eu is no longer used, instead the
gitlab.com server run the scripts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-11 11:13:19 -10:00
Petr Štetiar 34adbafaeb ci: refactor into separate YAML files under .gitlab/ci/ dir
It makes the YAML files more comprehensible.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Petr Štetiar 964f42b5c4 fix warnings reported by shellcheck and remove pythonisms
In ./docker-download.sh:
 SC2086: Double quote to prevent globbing and word splitting.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2086: Double quote to prevent globbing and word splitting.

In ./gen-targets.sh:
 SC2188: This redirection doesn't have a command. Move to its command (or use 'true' as no-op).
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2188: This redirection doesn't have a command. Move to its command (or use 'true' as no-op).
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2027: The surrounding quotes actually unquote this. Remove or escape them.
 SC2086: Double quote to prevent globbing and word splitting.

In ./docker-imagebuilder.sh:
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

In ./docker-rootfs.sh:
 SC2039: In POSIX sh, == in place of = is undefined.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

In ./docker-sdk.sh line:
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2155: Declare and assign separately to avoid masking return values.
 SC2086: Double quote to prevent globbing and word splitting.
 SC2105: continue is only valid in loops.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Petr Štetiar 43c17ad72b ci: add shellcheck for shell script checking
In order to make the shell scripts more robust.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-11 11:00:08 -10:00
Lucas Ramage 567eaa2716 Fix typo in README 2019-10-03 18:53:26 -10:00
Paul Spooren 2db99b240b introduce gen-targets.sh script
This improves readability as it auto generates the target list.
2019-10-02 20:46:24 -10:00
Paul Spooren 7d1ad316b6 Dockerfile: install rsync
Some packages need rsync to be installed on the host.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-19 11:12:03 -10:00
Paul Spooren 133f2ad712 split TARGETS in stacks of 10
ImageBuilder/SDKs are build for each target + master & 19.07-snapshot
branch. This is to much for most CI runners, this separates the job and
also allows running it in parrallel if more workers are available.

Also add PGP key for 19.07 release builds.

Introduce $CI_COMMIT_REF_SLUG for testing.

Introduce $SKIP_ROOTFS for older builds not supporting docker images.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-19 11:11:36 -10:00
Paul Spooren 35daea4a9a Update README with new available branches
Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-18 20:17:39 -10:00
Paul Spooren c8beb1bdf8 add sudo/curl to ImageBuilder/SDK
Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-17 13:09:48 -10:00
Paul Spooren 19285e351d use GNUHOME to use ./gpg
This prevents mixing of keys when running the scripts locally for
testing, the same is done for the using keys.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-16 16:21:01 -10:00
Paul Spooren ee1c7dad1b docker-common: remove login as it's done by CI
This makes more sense as local users wouldn't use this function anyway.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-14 14:58:05 -10:00
Paul Spooren 86a90d8755 fixup: remove further outdated targets
* gemini/raidsonic
* gemini/wiligear
* malta/be64
* malta/le
* malta/le64
* mediatek/32

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-07 18:42:29 -10:00
Paul Spooren c636775737 fixup: support other branches for rootfs
Initially the rootfs would only support snapshot builds as the stable
18.06.x wouldn't support Docker due to procd limitations. This was fixed
and therefore 19.07.x should also be supported.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-08 03:00:38 +02:00
Paul Spooren 816805813f fixup: remove outdated targets
Some targets still exists online, but were lastly updated late may.

* adm5120
* adm8668
* au1000
* mcs814x
* xburst

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-08 03:00:28 +02:00
Paul Spooren 35d0ebb47a fixup: rootfs uses a different Dockerfile
ImagBuilder and SDK use a different Dockerfile than the rootfs, this was
missed while refactoring for GitLab.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-08 02:41:16 +02:00
Paul Spooren ca0a3d6d23 fixup: remove target ppc44x/generic
This target doen't seem to be build anymore:

lede-imagebuilder-ppc44x.Linux-x86_64.tar.xz
lede-ppc44x-canyonlands-squashfs.img
lede-ppc44x-default.manifest
lede-ppc44x-taishan-squashfs.img
lede-ppc44x-uImage
lede-sdk-ppc44x_gcc-5.4.0_musl-1.1.15.Linux-x86_64.tar.xz

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-08 02:26:09 +02:00