RoadMap

2.0

Overall state: ‘Test && polish period’.

Milestones

  • New deb package layout: mini-buildd (the service), mini-buildd-utils (for users), python3-mini-buildd, mini-buildd-doc

  • Code migration to python 3.

  • Update to django 2.2 (minimal version).

  • Switch to twisted as http backend.

  • Support secure data channels (HTTPS && SFTP).

  • API upgrade to json.

  • Event message queue.

  • Periodic task scheduler (cron, internal use).

  • Automated repository snasphots (weekly, monthly, yearly).

  • WEB UI rewrite.

2.0 will

  • be ‘upgrade-compatible’ with 1.0 (especially, there is no SQL scheme change).

  • not be run-time compatible with 1.0.

Open Issues

Attention

2.0: Test and document new setup feature

Attention

2.0: Test and document new stop/start behaviour on ongoing builds/packaging

Attention

2.0: Last steps && QA

  • Check all warnings/errors in daemon.log after bootstrap.

  • Test mini-buildd-ssh-*-command setup.

  • General GAU tests.

  • Test upgrading from 1.0.x.

  • roadmap.rst: Add hr news for 2.0

Attention

2.0: Bugs (irrelevant for release)

  • SSL: Sporadic errors reading event queue
    • It seems occurrence is reduced when running with debug logs

    • Locking render_GET() does not seem to help

    Reproduce:

    ./devel profile _ssl updatetestall
    (...)
    "ValueError: invalid literal for int() with base 16: b''"
    
  • Web: While setup is running, web ‘mbd-off’ display does not work (daemon.is_alive() remains true somehow?)

Upgrading from 1.x

Attention

WIP, quite raw for now

buster

Please rather use bullseye.

Extra APT sources: Debian Backports (buster-backports) and Hellfield Archive (buster-ab-stable).

aptitude is imho the easiest way. Select the suggestion that de-installs py2 variants, and upgrades all the rest:

aptitude install mini-buildd/buster-ab-stable

Error

500 Internal Error: [Errno 2] No such file or directory: 'S.gpg-agent.extra'

We see this error seemingly random on buster installations (w/ 2.2.27-1~bpo10+1) only so far.

  • Removing a very old left-over ~/.gnupg/gpg.conf seem to have helped so far on an actual host

  • In test suite / chroot, it usually pops up with profile ‘_all’

  • Never yet seen with default (no) profile or ‘_debian’

generic post

  • Update Daemon.Gnupg keyserver: keyserver.ubuntu.com

  • Re-check all repositories
    • will regenrate repositories with new Content support (else you need to wait for internal cron)

  • Remove ~/.mini-buildd.pid (in case there is stale leftover – no longer used by mini-buildd (except when running with initv)

  • In case you have a custom upload setup, you can now add a custom dput snippet for convenience. See API call ‘dput_conf’:.

  • In case you hate the autogenerated complicated “Origin” text, you may now customize it. See Daemon Configuration => Extra Options.

  • Recommended to re-create all chroots – run API call ‘remake_chroots’ (web: Builders)

    mini-buildd >= 1.1.0 no longer removes ‘/etc/sudoers’ in chroot snapshots prior to builds.

    This file was part of a workaround for a sbuild bug; it is present in chroots created with mini-buildd versions < 1.0.5.

    In case you really have not yet re-created those chroots, you really MUST DO IT NOW, else your build runs may gain root privileges.

    To re-create chroots, enter the Web configuration frontend as ‘admin’ user, and first run ‘Remove’ and then ‘PCA’ on each chroot instance.

  • Re-check/upgrade your bindings settings. --httpd-bind is now deprecated in favor of --http-endpoint

    The same holds true for the ftpd_bind option in Daemon instance. I.e., all 1.0.x syntax is still supported, but upgrading avoids warnings and possible breakage should compatibility ever be removed.

  • Defaults: Now prefixes ‘~’ to rolling:

    The mandatory version appendix for rolling distributions now defaults to ~CODENAME (i.e., like “~SID” or “~STRETCH”).

    Using ‘CODENAME’ alone may call for versioning trouble once you convert the distribution to non-rolling (i.e., like “STRETCH” is bigger for version comparison than “90” or “9”).

    This change does not affect existing instances; to follow this recommendation in this case, you need to reconfigure the current resp. rolling Distributions.

  • Defaults: Adapt to ‘~bpo8’ (one number) scheme.

    The default ‘codename version’ is now only a single number for Debian distributions wheezy upwards (for example, ‘8’ instead of ‘80’ for jessie), following the updated convention in Debian.

    This change does not affect existing Source instances; to follow this recommendation in this case, you need to reconfigure the current resp. Source, save and empty string for ‘Codeversion override’ (Extras), and “PCA” the Source.

2.2 Brainstorm

Attention

2.2: Debian * Update debconf to support encryption (https) oob (see local branch [sic])

Attention

2.2: new features

  • Dependency check on package migration

  • watcher support: Autoport and/or alarm on Debian Package Updates

  • Automatic SSH upload support, either by scripting the setup or integrating SSH w/ twisted

  • Make piuparts actually work (non-EXPERIMENTAL)

  • cron: ccache expire

Attention

2.2: django

  • External user management (ldap, sso?) - python3-django-casclient?
    • Extended User Role Concept, with advanced right management workflows?

  • SQL schema migration (with django >= 1.7)

Attention

2.2: refactoring

  • gnupg: Switch completely to ‘python3-gpg’ (NOT ‘python3-gpgme’)

  • Archive: Rethink Archive::mbd_ping (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=740576)

  • Can use of dataclasses simplify sorting support?

Attention

2.2: repository

  • Release: Alternatives for NotAutomatic, ButAutomo…)?? Can we use more finegrained PRIOS on archives?

  • sid-test-wildwest: Add extra suite for arbitrary package (deb+src) installs?

Attention

2.2: setup

  • admin/model objects: Show “used by count” (so we can see what objects may be obsolete)?

  • sync config between instances?

Bummers

Important

http://tools.ietf.org/html/rfc5147: This, in theory, could be some way to create a TOC for text/plain (buildlogs and such). However

Important

twisted: