886 views
# basic tasks ## building `./compile.sh` ### debugging add `DEBUG=yes` optionally `SHOW_COMMANDS=yes` #### inventory ``` ./compile.sh inventory cat output/info/all_boards_all_branches.json | jq . ``` ### slightly more advanced building see commands in `lib/functions/cli/commands.sh` specifically shortcuts most likely commands needed for granularity * kernel * kernel-config * uboot * rootfs ### recipes #### make sure your user has docker privileges stop being a curmudgeon the flexibility is real, and its better system hygegine. the tool is optimized for docker. ``` lane@rock-5b:~/GIT/build$ docker ps Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json": dial unix /var/run/docker.sock: connect: permission denied lane@rock-5b:~/GIT/build$ sudo usermod -a -G docker ${USER} lane@rock-5b:~/GIT/build$ docker ps Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json": dial unix /var/run/docker.sock: connect: permission denied lane@rock-5b:~/GIT/build$ #surprise you have to log out lane@rock-5b:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ``` #### I'm old and mean and dont want to use docker ``` export PREFER_DOCKER=no #BECAUSE i dont want docker ## you should suck it up and want docker unless you have a compelling usecase otherwise #### just build a kernel you'll need to pick an SBC and branch so that it can target. example if i wanted rockchip64-edge ```bash #export PREFER_DOCKER=no #BECAUSE i dont want docker ## you should suck it up and want docker ./compile.sh BOARD=tinkerboard-2 BRANCH=edge kernel ``` #### just build a rootfs ```bash ./compile.sh PREFER_DOCKER=no BOARD=rockpi-4bplus BRANCH=edge BUILD_MINIMAL=no BUILD_DESKTOP=no COMPRESS_OUTPUTIMAGE=sha,img EXPERT=yes RELEASE=bookworm rootfs ``` #### build a desktop rootfs run from a good arm box and make your life easier ```bash ./compile.sh EXPERT=yes BOARD=tinkerboard-2 BRANCH=edge RELEASE=sid BUILD_DESKTOP=yes DESKTOP_ENVIRONMENT=cinnamon rootfs ``` #### Ignore the cache and make me a f-ing kernel I don't care that nothing changed, power is free in American, build me something anyway `ARTIFACT_IGNORE_CACHE=yes` `./compile.sh kernel-config` will also work and is cleaner. ### adding kernel patches use the b4 tool `b4 am -o- "https://lore.kernel.org/all/xxxxx/" > patch/kernel/xxx-edge/xxx.patch` ## WARNINGS ### cache the cache is great don't share the `cache` directory between docker and non_docker, or alternate `DOCKER_ARMBIAN_BASE_IMAGE=` settings or there will be pain AKA use seperate git checkouts and working directories for non-docker, docker, and custom docker ## pro moves find available build options first generate info file (if new run) `./compile.sh json-info` `cat output/info.json |jq '.[].BOARD_DESC_ID' -r |sort -u|fzf` lookup stuff about a board `cat output/info.json |jq '.[]|select(.BOARD_DESC_ID=="bananapim64-edge")'` silly command to look cool ```bash BOARD_DESC_ID=$(cat output/info.json |jq '.[].BOARD_DESC_ID' -r |sort -u|fzf) cat output/info.json |jq ".[]|select(.BOARD_DESC_ID==\"${BOARD_DESC_ID}\") | .BOARD_FILE_HARDWARE_DESC,.BOARD_POSSIBLE_BRANCHES,.in" ``` ### build with 2 boxes Okay going to cache advantage of the OCI ORAS feature here.. start unauthed registry on my build server ``` docker run -it -p 5000:5000 -d --name oci_armbian_cache registry:2 ``` okay lets be lazy and portforward the registry on the big host over ssh. lets listen on our docker interface ``` export DOCKER_GW_IP=$(docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}') ssh -A -L ${DOCKER_GW_IP}:5000:172.17.0.1:5000 armbian-builder-lanefu.example.com ``` confirm it works ``` ane@rock-5b:~$ curl -i localhost:5000/ HTTP/1.1 200 OK Cache-Control: no-cache Date: Sat, 11 Mar 2023 14:54:51 GMT Content-Length: 0 ``` make sure your user has docker privileges if not see howto at top ``` lane@rock-5b:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ``` check out armbian then lazily copy my userpatches directory ``` git clone git@github.com:armbian/build.git cd build #scp -pr armbian-builder-lanefu.example.com:GIT/build/userpatches . rsync -auv armbian-builder-lanefu.example.com:GIT/build/userpatches . ``` time to do the real deal.. build some kernels n stuff on the kernel host lets just give it aproject path of `/poc/` ``` export DOCKER_GW_IP=$(docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}') ./compile.sh OCI_TARGET_BASE=${DOCKER_GW_IP}:5000/poc/ EXPERT=yes TEXT_IS_TOO_DARK=yes rock5b-midstream kernel ./compile.sh OCI_TARGET_BASE=${DOCKER_GW_IP}:5000/poc/ EXPERT=yes TEXT_IS_TOO_DARK=yes rock5b-midstream uboot ``` now build an image from your rootfs host using the artifacts ``` #export MY_IP=$(ip route|fgrep default|egrep -o 'src.*'|cut -d' ' -f2) export DOCKER_GW_IP=$(docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}') ./compile.sh OCI_TARGET_BASE=${DOCKER_GW_IP}:5000/poc/ EXPERT=yes TEXT_IS_TOO_DARK=yes rock5b-midstream build ``` # signal flow all sequencing is extermely function driven.. When i'm listing files that's just a breadcrumb ## basic there's lots of pre-processing etc happening, but this is the main execution stream. `./compile.sh -> lib/single.sh -> lib/functions/cli/entrypoint.sh -> lib/functions/cli/utils-cli.sh` # Quirks ## docker * will default to docker if present * but your account needs to be in the docker group.... so that it can be executed without sudo * or `PREFER_DOCKER=no` # what's different ## a lot * modular AF (via extensions) * lots of tools to dynamically generate functionality ## deprecated things * don't use sudo it wants to do it for you * GCC selection * default is native GCC now * `uboot_use_gcc` * `kernel_use_gcc` * LIB_TAG IS DEAD * PACKAGE_LIST_ADDITIONAL * need to use some of the native hooks and functions * will default to docker if present * but your account needs to be in the docker group.... so that its sudoless * or `PREFER_DOCKER=no` * KERNEL_ONLY * use the `kernel` command for the build ## CLEAN_LEVEL * slightly revised arguments use * `make-kernel` * `make-uboot` * `debs` * add more info here ## cache * very advanced caching accross the board * experimental OCI storage * you can deploy local insecure docker registry and use an argument like * `docker run -it -p 5000:5000 registry:2` you should add persistent storage * `OCI_TARGET_BASE=172.17.0.1:5000/mybuild` # Some knowledge straight from the artifact-kernel.sh # Mastery ## Understand Hookpoints Hookpoints allow you to make add functions prefixed with any of these [hook points](https://docs.armbian.com/Developer-Guide_Extensions-Hooks/) in your config file. you can use them to use [existing functions](https://github.com/armbian/build/tree/main/lib/functions) or [extensions](https://docs.armbian.com/Developer-Guide_Extensions/) ## Understand Extensions Extensions are a pluggable way to add a rich library of functions to armbian build framework. Once enabled via a `enable_extension` call, They are generally available for use anywhere. https://docs.armbian.com/Developer-Guide_Extensions/#what-is-an-extension "Specifically, extension files should not contain any code outside of functions – they should do nothing when sourced." **NOTE** slight edge-case that hooks related to build dependencies may get executed durring the Docker bootstrapping for cache management purposes regardless of being enabled... those extensions aren't otherwise available unless enabled with `enable_extension` Explaination: ``` <+Armbian-Discord> <r​pardini> enable_extension is needed to actually use the extension, indeed. <+Armbian-Discord> <r​pardini> for the specific case of Docker (only in the launcher, which creates the Dockerfile, but then re-launches itself), all extensions found both in core and in userpatches are enabled automatically, and their add_host_dependencies hook is run. That is specifically to allow consistent caching of the dependencies so Dockerfile does not keep changing and re-downloading stuff all the time. <+Armbian-Discord> <r​pardini> (after the Dockerfile is built, and compile.sh is re-launched under Docker, you'd need enable_extension to use other hooks) ``` ### extension wisdom #### call an extension from another one from rpardini > Just call enable_extension "bar" from your foo extension. Do that outside of a function/hook. Internally, the framework will postpone the enablement of bar until foo is finished enabling. for an example see extensions/image-output-vhdx.sh #### ordering from rpardini: > For the ordering... all non-numeric hooks run as if they were prepended with 500_ > so say you have extension_prepare_config__something but want that to run earlier than other similar extension_prepare_config hooks in other extensions, just rename it to extension_prepare_config__250_something #### add more hookpoints from rpardini: > Finally the final pro-move is that you can define new extension points/hooks using, well, hooks > see for example extensions/grub.sh -- which itself calls call_extension_method , allowing for further extensions; and extensions/grub-with-dtb.sh which extends the grub extension. ### extension debugging `SHOW_EXTENSIONS=yes` flag will show more information, but is known to cause blindness ## Understand aggregation `lib/tools/aggregation.py` tells you a LOT # customizing ## custom config First read up on hook points and extensions (see above). a primary custom config file is still referenced by its name without the prefix. will show example `userpatches/config-lanefu-sid.conf` this would be access in a example command like ```bash ./compile.sh BOARD=pinebook-pro lanefu-sid build ``` ```bash # you could put common things in a file and source source common-lanefu.conf ### ### Generic Configuration. BOARD should be defined via command line ### export BRANCH=edge export RELEASE=sid export DEST_LANG="en_US.UTF-8" # sl_SI.UTF-8, en_US.UTF-8 export BUILD_MINIMAL=no # No minimal. Fully working server cli please export BUILD_DESKTOP=no # No desktop. You install it later after jumpstarting. export INSTALL_HEADERS=no # dont install kernel headers package declare -g MAINTAINER="maintainer person" declare -g MAINTAINERMAIL="me@example.com" ## tell users this config file is being used echo "lanefu-generic config for ${BOARD}" # early config hook, after family set variables. used for overrides ## adds default package to the common rootfs cache.. if you don't maintain your own rootfs cache and prefer upstream don't use this function function user_config__010_extra_packages_rootfs() { display_alert "opinionated install" "lanefu_custom" "info" add_packages_to_rootfs vim mtr } ## adds extra packages on top of rootfs durring image build.. wont be cached. function user_config__011_extra_packages_image() { display_alert "customized install" "lanefu_custom" "info" add_packages_to_image nginx docker.io } ## should probably leverage other hooks and functions to modify EDITOR env vars for system since i'm a systemhttps://github.com/armbian/build/blob/main/lib/functions/configuration/package-lists.sh#L43 function user_config__200_vim_snob() { display_alert "vim forever" "lanefu_custom" "info" ## this function will impact all cache per documentation remove_packages nano emacs } ```