Evaluate swupdate

Type of update systems

  • image based
    • mender.io (only a/b system; client/server)
    • swupdate (a/b + rescue system; client/hawkbit server)
    • rauc (a/b + rescue system; only client)
  • revisioned binary patches
    • swupd
    • ostree

mostly based on whitepaper swupdate/mender/resion comparison from matt porter
https://lists.linuxfoundation.org/pipermail/automotive-discussions/2016-May/002061.html

above study sponsored by automotive group: (suggests to use OSTree)
http://events.linuxfoundation.org/sites/events/files/slides/Open%20Source%20secure%20software%20updates%20for%20Linux-based%20IVI%20systems.pdf p.22

Other work

  • ostree aka binary diff via git-like repository
    • rollback/upgrade through checkout of new rootfs
    • small incremental updates (->security updates?)
    • actively used: automotive(agl)/gnome continuous
  • not looked into container based, due to lacking base system upgrade

u-boot: boot count / boot limit support

in case rootfs fails to boot, we will boot back into swupdate after bootlimit attempts

  • CONFIG_BOOTCOUNT_LIMIT http://www.denx.de/wiki/view/DULG/UBootBootCountLimit
    • feature has been consolidated in 2012.07 (gitk drivers/bootcount)
    • probably not enabled in any of our platforms

      - definitely disabled dss20/dss11e
      - to be checked: 1gb-t1?/1gb-sdc?/dssip?)

    • could be implemented in boot.scr / fw-env update

SWupdate evaluation

  • mode of operation
    • we would use it as rescue-system, (not full a/b images):
      • boot into swupdate initramfs, perform update, then reboot into rootfs
      • replace our shell based flash script
    • puts higher strain on our u-boot environment
      • assumes fw-env flag is used to select either rescue/rootfs to boot
      • single buffer environment in 1gb/dssip
      • sensitive data in fw_env: mac address in 1gb
    • alternative: only used it for flashing the rootfs
      • keep existing kexec boot system of rescue-system and only use swupdate to download and flash images as standalone tool
      • no dependency on u-boot, rescue-system abstracts details about u-boot
      • drawback no kexec on dssip yet
    • alternative2: use a/b images on dss20/dss11e/dssip
      • run swupdate either as a daemon or cronjob, upgrade in the background then reboot into the other image
  • better hardware configuration management
    • /etc/hwrevsion: with single line: board + revision, e.g. "dss20 1.0"
    • sw-description in update image, can match individual board names, hence bundle several platforms
    • our 'machine.desc' describing devices/volumes and ways to flash are now moved into the image format
  • size evaluation
    • adding swupdate binary to the rescue-system, (needed to drop gnupg(600kB) due to build failures)
      reflash_roots.sh(1) swupdate
      dss20 3 MB 6.0 MB
      dss11-sdc 4.5 MB 7.5 MB
    • swupdate config
      • compile with dynamic libraries / static built failed due link error against libconfig/libcurl
      • no lua/webserver support
      • no openssl/sha256 enabled
      • curl support
      • mtd/ubi support
    • felt boot time is longer with new rescue system on sdc / no difference noticed on dss20

    (1) includes gnupg, numbers are taken from repository with binary images
    https://git.digitalstrom.org/dss-oe/rescue-images/tree/master

  • open points / ideas
  • difference to reflash_rootfs
    reflash_rootfs swupdate
    yes no verify streamed image before flashing
    yes no bzip2/xz compression
    yes yes stream into emmc/block device
    yes no stream into raw nand device
    yes xx stream into ubi volume
  • streaming test
    • on legacy at91 platforms we need streaming support since neither flash/ram can hold the update
    dss20 dss11-sdc dss11-1gb
    streaming ok Raw NAND not streamable (1)

    used attached sw-description to create image according to
    https://sbabic.github.io/swupdate/swupdate.html#building-a-single-image

      [NOTIFY] : SWUPDATE running :  [extract_files] : Installing STREAM digitalstrom-devel-rootfs-dss11-sdc.jffs2, 68681728 bytes
      [NOTIFY] : SWUPDATE running :  [install_single_image] : Found installer for stream digitalstrom-devel-rootfs-dss11-sdc.jffs2 flash
      [NOTIFY] : SWUPDATE running :  [install_flash_image] : Copying digitalstrom-devel-rootfs-dss11-sdc.jffs2 into /dev/mtd6
      [NOTIFY] : SWUPDATE failed [0] ERROR handlers/flash_handler.c : flash_write_nand : 460 : Raw NAND not streamable
      [NOTIFY] : SWUPDATE failed [0] ERROR handlers/flash_handler.c : install_flash_image : 707 : I cannot copy digitalstrom-devel-rootfs-dss11-sdc.jffs2 into mtd6
      [NOTIFY] : SWUPDATE running :  [install_single_image] : Installer for flash not successful !
      

reflash_rootfs downloads the image twice, the first is a verification run. Only after the image is verified we erase the old rootfs. This only applies to the streaming case, I assume when downloading the image first, swupdate will verify the checksum before starting the update.

Summary

Swupdate can be used as a drop in replacement for our reflash_rootfs script, with the least effort. Open points are security (gnupg vs. openssl) and streaming support on the legacy at91 platforms. This only replaces part of our rescue-system. The boot process suggested by swupdate relies on u-boot features, fw-env / evtl. bootlimit. We might consider to keep our kexec based boot system to stay u-boot agnostic and only use swupdate to download and flash images. The image format used by swupdate is superior to our image format. Ours consisting only of a bare rootfs link.

From a quick glance at mender.io it sems to have a very clean source https://github.com/mendersoftware/mender/blob/master/main.go. But since it doesn't offer a rescue-system configuration but only a/b images, I didn't pursue to evaluate it further. The use of 'go' language is another barrier. mender provides a yocto layer

rauc seems the closest alternative to swupdate, it uses glib, offering containers like vector/list and a mainloop which eases implementation. Documentation says it uses squashfs as the image format, which is not suitable for streaming. Source code also supports http of individual files, but streaming seems not supported. Documentation is less mature, first commit 2015-04.

OStree is used by Automative Grade Linux, it needs more work to prepare images on the server (binary diffs). Gnome uses it not as working environment, but as continuous testing environment to have all the latest feature of the HEAD branch. Overall complexity seems higher server+client. We should keep an eye on it, since it's actively used by other projects, it might develop fast.

At the time being only swupdate offers all must-have features we need to support all of our platforms

Collected Stuff found on WWW/ML

TODO, needs to be sorted

streaming

https://groups.google.com/forum/#!topic/swupdate/sMoAKb07dKA

image + tarball

  • each of them small enough to be downloaded, verified completely before flashing
  • this could be taken further into cutting an image into pieces, and flash them piece wise at an offset

binary diff (bsdiff)

https://wiki.ubuntu.com/ImageBasedUpgrades
http://dev.chromium.org/developers/design-documents/software-updates-courgette

https://groups.google.com/forum/#!topic/swupdate/7Z3-JpGoAx0

image creation and signing

https://groups.google.com/forum/#!topic/swupdate/scsvoFf0ohc
https://groups.google.com/forum/#!topic/swupdate/R0papgK1RqY

only need to sign sw-description file, which includes hashes of all other files. cpio archive generated afterwards

swupdate swu creation
https://groups.google.com/forum/#!searchin/swupdate/creating$20swu$20file%7Csort:relevance/swupdate/a5fPrDRZT5A/07zA1FlGEwAJ
https://groups.google.com/forum/#!searchin/swupdate/creating$20swu$20file%7Csort:relevance/swupdate/R0papgK1RqY/078zmJfVDwAJ

ubi

https://groups.google.com/forum/#!topic/swupdate/_SDJzzSYlyI
declare ubi volume to use all available space, (not specifying number of bytes)

swupdate + wks

https://groups.google.com/forum/#!topic/swupdate/xf86Ip82Gzk

sw-description a/b example

https://groups.google.com/forum/#!topic/swupdate/7gMdrtAiWJM

TODO

Streaming support

http://redmine.digitalstrom.org/issues/15954

Summary:

  • streaming compressed ubifs/jffs2 filesystems is not supported
  • ubifs already uses compression, but only per block. Hence there must be a lot of redundancy between blocks on our rootfs

Evtl. we might compress the rootfs better if we switch to squashfs, that evtl applies compression to the whole rootfs

sw-description (648 Bytes) Andreas Fenkart, 01/18/2017 09:29 AM