mini_buildd Package

mini_buildd Package

api Module

class mini_buildd.api.Command(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: object

ADMIN = 3
ARGUMENTS = []
AUTH = 0
COMMAND = None
COMMON_ARG_VERSION = ([u'--version', u'-V'], {u'action': u'store', u'default': u'', u'metavar': u'VERSION', u'help': u'\nlimit command to that version. Use it for the rare case of\nmultiple version of the same package in one distribution (in\ndifferent components), or just as safeguard\n'})
CONFIRM = False
LOGIN = 1
NEEDS_RUNNING_DAEMON = False
NONE = 0
STAFF = 2
arg_false2none(key)
classmethod docstring()
classmethod get_default_args()
has_flag(flag)
class mini_buildd.api.GetDputConf(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Get recommended dput config snippet.

Usually, this is for integration in your personal ~/.dput.cf.

COMMAND = u'getdputconf'
run(daemon)
class mini_buildd.api.GetKey(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Get GnuPG public key.

COMMAND = u'getkey'
run(daemon)
class mini_buildd.api.GetSourcesList(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Get sources.list (apt lines).

Usually, this output is put to a file like ‘/etc/sources.list.d/mini-buildd-xyz.list’.

ARGUMENTS = [([u'codename'], {u'help': u'codename (base distribution) to get apt lines for'}), ([u'--repository', u'-R'], {u'action': u'store', u'default': u'.*', u'metavar': u'REPO', u'help': u'repository name regex.'}), ([u'--suite', u'-S'], {u'action': u'store', u'default': u'.*', u'metavar': u'SUITE', u'help': u'suite name regex.'}), ([u'--with-deb-src', u'-s'], {u'action': u'store_true', u'default': False, u'help': u'also list deb-src apt lines.'}), ([u'--with-extra-sources', u'-x'], {u'action': u'store_true', u'default': False, u'help': u'also list extra sources needed.'})]
COMMAND = u'getsourceslist'
run(daemon)
class mini_buildd.api.List(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

List packages matching a shell-like glob pattern; matches both source and binary package names.

ARGUMENTS = [([u'pattern'], {u'help': u'list source packages matching pattern'}), ([u'--with-rollbacks', u'-r'], {u'action': u'store_true', u'default': False, u'help': u'also list packages on rollback distributions'}), ([u'--distribution', u'-D'], {u'action': u'store', u'default': u'', u'metavar': u'DIST', u'help': u'limit distributions to those matching this regex'}), ([u'--type', u'-T'], {u'action': u'store', u'default': u'', u'metavar': u'TYPE', u'help': u'package type: dsc, deb or udeb (like reprepo --type)'})]
AUTH = 1
COMMAND = u'list'
run(daemon)
class mini_buildd.api.LogCat(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Cat last n lines of the mini-buildd’s log.

ARGUMENTS = [([u'--lines', u'-n'], {u'action': u'store', u'default': 500, u'type': <type 'int'>, u'metavar': u'N', u'help': u'cat (approx.) the last N lines'})]
AUTH = 2
COMMAND = u'logcat'
run(daemon)
class mini_buildd.api.Meta(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Call arbitrary meta functions for models; usually for internal use only.

ARGUMENTS = [([u'model'], {u'help': u"Model path, for example 'source.Archive'"}), ([u'function'], {u'help': u"Meta function to call, for example 'add_from_sources_list'"})]
AUTH = 3
COMMAND = u'meta'
run(daemon)
class mini_buildd.api.Migrate(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Migrate a source package (along with all binary packages).

ARGUMENTS = [([u'package'], {u'help': u'source package name'}), ([u'distribution'], {u'help': u"distribution to migrate from (if this is a '-rollbackN' distribution, this will perform a rollback restore)"}), ([u'--version', u'-V'], {u'action': u'store', u'default': u'', u'metavar': u'VERSION', u'help': u'\nlimit command to that version. Use it for the rare case of\nmultiple version of the same package in one distribution (in\ndifferent components), or just as safeguard\n'})]
AUTH = 2
COMMAND = u'migrate'
CONFIRM = True
run(daemon)
class mini_buildd.api.Port(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Port an internal package.

An internal ‘port’ is a no-changes (i.e., only the changelog will be adapted) rebuild of the given locally-installed package.

ARGUMENTS = [([u'package'], {u'help': u'source package name'}), ([u'from_distribution'], {u'help': u'distribution to port from'}), ([u'to_distributions'], {u'help': u'comma-separated list of distributions to port to (when this equals the from-distribution, a rebuild will be done)'}), ([u'--version', u'-V'], {u'action': u'store', u'default': u'', u'metavar': u'VERSION', u'help': u'\nlimit command to that version. Use it for the rare case of\nmultiple version of the same package in one distribution (in\ndifferent components), or just as safeguard\n'})]
AUTH = 2
COMMAND = u'port'
CONFIRM = True
NEEDS_RUNNING_DAEMON = True
run(daemon)
class mini_buildd.api.PortExt(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Port an external package.

An external ‘port’ is a no-changes (i.e., only the changelog will be adapted) rebuild of any given source package.

ARGUMENTS = [([u'dsc'], {u'help': u'URL of any Debian source package (dsc) to port'}), ([u'distributions'], {u'help': u'comma-separated list of distributions to port to'})]
AUTH = 2
COMMAND = u'portext'
CONFIRM = True
NEEDS_RUNNING_DAEMON = True
run(daemon)
class mini_buildd.api.PrintUploaders(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Print all GPG ids allowed to upload to repositories.

ARGUMENTS = [([u'--repository', u'-R'], {u'action': u'store', u'default': u'.*', u'metavar': u'REPO', u'help': u'repository name regex.'})]
AUTH = 3
COMMAND = u'printuploaders'
NEEDS_RUNNING_DAEMON = True
run(daemon)
class mini_buildd.api.Remove(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Remove a source package (along with all binary packages).

ARGUMENTS = [([u'package'], {u'help': u'source package name'}), ([u'distribution'], {u'help': u'distribution to remove from'}), ([u'--version', u'-V'], {u'action': u'store', u'default': u'', u'metavar': u'VERSION', u'help': u'\nlimit command to that version. Use it for the rare case of\nmultiple version of the same package in one distribution (in\ndifferent components), or just as safeguard\n'})]
AUTH = 3
COMMAND = u'remove'
CONFIRM = True
run(daemon)
class mini_buildd.api.Retry(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Retry a previously failed package.

ARGUMENTS = [([u'package'], {u'help': u'source package name'}), ([u'version'], {u'help': u"source package's version"}), ([u'--repository', u'-R'], {u'action': u'store', u'default': u'*', u'metavar': u'REPO', u'help': u'Repository name -- use only in case of multiple matches.'})]
AUTH = 2
COMMAND = u'retry'
CONFIRM = True
NEEDS_RUNNING_DAEMON = True
run(daemon)
class mini_buildd.api.SetUserKey(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Set a user’s GnuPG public key.

ARGUMENTS = [([u'key'], {u'help': u'GnuPG public key; multiline inputs will be handled as ascii armored full key, one-liners as key ids'})]
AUTH = 1
COMMAND = u'setuserkey'
CONFIRM = True
run(_daemon)
class mini_buildd.api.Show(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Show a source package.

ARGUMENTS = [([u'package'], {u'help': u'source package name'}), ([u'--verbose', u'-v'], {u'action': u'store_true', u'default': False, u'help': u'verbose output'})]
COMMAND = u'show'
run(daemon)
class mini_buildd.api.Start(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Start the Daemon (engine).

ARGUMENTS = [([u'--force-check', u'-C'], {u'action': u'store_true', u'default': False, u'help': u'run checks on instances even if already checked.'})]
AUTH = 3
COMMAND = u'start'
run(daemon)
class mini_buildd.api.Status(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Show the status of the mini-buildd instance.

COMMAND = u'status'
chroots_str()
has_chroot(codename, arch)
repositories_str()
run(daemon)
class mini_buildd.api.Stop(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Stop the Daemon (engine).

ARGUMENTS = []
AUTH = 3
COMMAND = u'stop'
run(daemon)
class mini_buildd.api.Subscription(args, request=None, msglog=<logging.Logger object at 0x7fc4ae4f7110>)

Bases: mini_buildd.api.Command

Manage subscriptions to package notifications.

A package subscription is a tuple ‘PACKAGE:DISTRIBUTION’, where both PACKAGE or DISTRIBUTION may be empty to denote all resp. items.

ARGUMENTS = [([u'action'], {u'help': u'action to run', u'choices': [u'list', u'add', u'remove']}), ([u'subscription'], {u'help': u'subscription pattern'})]
AUTH = 1
COMMAND = u'subscription'
run(daemon)
mini_buildd.api.cls

alias of Subscription

builder Module

class mini_buildd.builder.Build(breq, gnupg, sbuild_jobs)

Bases: mini_buildd.misc.Status

BUILDING = 1
CHECKING = 0
FAILED = -1
UPLOADED = 10
UPLOADING = 2
architecture
build()

Note

SUDO WORKAROUND for http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608840

Includes all ‘sudo’ prefix for chroot-setup-commands below.

See also

Chroot

clean()
distribution
key
package
sbuildrc_path
took
upload()
upload_result_to
version
class mini_buildd.builder.LastBuild(build)

Bases: mini_buildd.misc.API

Subset of ‘Build’ for pickled statistics.

mini_buildd.builder.build(daemon_, breq)
mini_buildd.builder.build_close(daemon, build)

Close build. Just continue on errors, but log them; guarantee to remove it from the builds dict.

mini_buildd.builder.run(daemon_)

changes Module

class mini_buildd.changes.Changes(file_path)

Bases: debian.deb822.Changes

BUILDREQUEST_RE = <_sre.SRE_Pattern object at 0x7fc4b0685030>
BUILDRESULT_RE = <_sre.SRE_Pattern object at 0x7fc4b04d41e8>
TYPES = {0: u'', 1: u'_mini-buildd-buildrequest', 2: u'_mini-buildd-buildresult'}
TYPE_BREQ = 1
TYPE_BRES = 2
TYPE_DEFAULT = 0
add_file(file_name)
bres_stat
buildlog_name
dsc_file_name
dsc_name
file_name
file_path
gen_buildrequests(daemon, repository, dist, suite_option)

Build buildrequest files for all architectures.

gen_buildresult(path=None)
classmethod gen_changes_file_name(package, version, arch, mbd_type=0)

Gen any changes file name.

Always strip epoch from version, and handle special mini-buildd types.

>>> Changes.gen_changes_file_name("mypkg", "1.2.3-1", "mips")
u'mypkg_1.2.3-1_mips.changes'
>>> Changes.gen_changes_file_name("mypkg", "7:1.2.3-1", "mips")
u'mypkg_1.2.3-1_mips.changes'
>>> Changes.gen_changes_file_name("mypkg", "7:1.2.3-1", "mips", mbd_type=Changes.TYPE_BREQ)
u'mypkg_1.2.3-1_mini-buildd-buildrequest_mips.changes'
>>> Changes.gen_changes_file_name("mypkg", "7:1.2.3-1", "mips", mbd_type=Changes.TYPE_BRES)
u'mypkg_1.2.3-1_mini-buildd-buildresult_mips.changes'
classmethod gen_dsc_file_name(package, version)
gen_file_name(arch, mbd_type)
get_files(key=None)
get_pkg_id(with_arch=False, arch_separator=u':')
get_pkglog_dir(installed, relative=True)

Package log path for this changes file: REPOID/[_failed]/PACKAGE/VERSION/ARCH

In case the changes is bogus (i.e, cannot produce a valid path for us, like a wrong distribution), None is returned.

get_spool_dir()
is_new()
magic_auto_backports
magic_backport_mode
move_to_pkglog(installed)
remove()
save(gnupg=None)

Write to file (optionally signed).

>>> import tempfile
>>> t = tempfile.NamedTemporaryFile()
>>> c = Changes(t.name)
>>> c["key"] = "ASCII value"
>>> c.save(None)
>>> c["key"] = "Ünicöde «value»"
>>> c.save(None)
tar(tar_path, add_files=None)
type
untar(path)
upload(hopo)
upload_buildrequest(local_hopo)
upload_failed_buildresult(gnupg, hopo, retval, status, exception)

daemon Module

class mini_buildd.daemon.Changelog(file=None, max_blocks=None, allow_empty_author=False, strict=True, encoding='utf-8')

Bases: debian.changelog.Changelog

Changelog class with some extra functions.

>>> cl = Changelog(mini_buildd.misc.open_utf8("./examples/doctests/changelog"), max_blocks=100)
>>> cl.find_first_not("mini-buildd@buildd.intra")
(u'Stephan S\xfcrken <absurd@debian.org>', u'1.0.0-2')
>>> cl = Changelog(mini_buildd.misc.open_utf8("./examples/doctests/changelog.ported"), max_blocks=100)
>>> cl.find_first_not("mini-buildd@buildd.intra")
(u'Stephan S\xfcrken <absurd@debian.org>', u'1.0.0-2')
>>> cl = Changelog(mini_buildd.misc.open_utf8("./examples/doctests/changelog.oneblock"), max_blocks=100)
>>> cl.find_first_not("mini-buildd@buildd.intra")
(u'Stephan S\xfcrken <absurd@debian.org>', u'1.0.1-1~')
>>> cl = Changelog(mini_buildd.misc.open_utf8("./examples/doctests/changelog.oneblock.ported"), max_blocks=100)
>>> cl.find_first_not("mini-buildd@buildd.intra")
(u'Mini Buildd <mini-buildd@buildd.intra>', u'1.0.1-1~')
find_first_not(author)

Find (author,version+1) of the first changelog block not by given author.

class mini_buildd.daemon.DSTPackage(tpl_dir, version=None)

Bases: mini_buildd.misc.TmpDir

class mini_buildd.daemon.Daemon

Bases: object

classmethod get_active_chroots()
classmethod get_active_or_auto_reactivate_remotes()
classmethod get_active_remotes()
classmethod get_active_repositories()
get_keyring_package()
get_status()
classmethod get_subscription_objects()
classmethod get_test_package(id_)
is_busy()
is_running()
classmethod logcat(lines)
mbd_get_sources_list(codename, repo_regex, suite_regex, prefixes, with_extra_sources)
classmethod meta(model, func, msglog)
classmethod parse_distribution(dist)

Get repository, distribution and suite model objects (plus rollback no) from distribtion string.

port(package, from_dist, to_dist, version)
portext(dsc_url, to_dist)
start(force_check=False, msglog=<logging.Logger object at 0x7fc4adb2e1d0>)
stop(msglog=<logging.Logger object at 0x7fc4adb2e1d0>)
update_to_model(obj)
class mini_buildd.daemon.DebianVersion(version)

Bases: debian.debian_support.Version

gen_external_port(default_version)

Generate an ‘external port’ version.

This currently just appends the given default version appendix. For example:

1.2.3 -> 1.2.3~test60+1

gen_internal_port(from_mandatory_version_regex, to_default_version)

Generate an ‘internal port’ version.

Tests for the (recommended) Default layout:

>>> sid_regex = r"~testSID\+[1-9]"
>>> sid_default = "~testSID+1"
>>> sid_exp_regex = r"~testSID\+0"
>>> sid_exp_default = "~testSID+0"
>>> wheezy_regex = r"~test70\+[1-9]"
>>> wheezy_default = "~test70+1"
>>> wheezy_exp_regex = r"~test70\+0"
>>> wheezy_exp_default = "~test70+0"
>>> squeeze_regex = r"~test60\+[1-9]"
>>> squeeze_default = "~test60+1"
>>> squeeze_exp_regex = r"~test60\+0"
>>> squeeze_exp_default = "~test60+0"

sid->wheezy ports:

>>> DebianVersion("1.2.3-1~testSID+1").gen_internal_port(sid_regex, wheezy_default)
u'1.2.3-1~test70+1'
>>> DebianVersion("1.2.3-1~testSID+4").gen_internal_port(sid_regex, wheezy_default)
u'1.2.3-1~test70+4'
>>> DebianVersion("1.2.3-1~testSID+4fud15").gen_internal_port(sid_regex, wheezy_default)
u'1.2.3-1~test70+4fud15'
>>> DebianVersion("1.2.3-1~testSID+0").gen_internal_port(sid_exp_regex, wheezy_exp_default)
u'1.2.3-1~test70+0'
>>> DebianVersion("1.2.3-1~testSID+0exp2").gen_internal_port(sid_exp_regex, wheezy_exp_default)
u'1.2.3-1~test70+0exp2'

wheezy->squeeze ports:

>>> DebianVersion("1.2.3-1~test70+1").gen_internal_port(wheezy_regex, squeeze_default)
u'1.2.3-1~test60+1'
>>> DebianVersion("1.2.3-1~test70+4").gen_internal_port(wheezy_regex, squeeze_default)
u'1.2.3-1~test60+4'
>>> DebianVersion("1.2.3-1~test70+4fud15").gen_internal_port(wheezy_regex, squeeze_default)
u'1.2.3-1~test60+4fud15'
>>> DebianVersion("1.2.3-1~test70+0").gen_internal_port(wheezy_exp_regex, squeeze_exp_default)
u'1.2.3-1~test60+0'
>>> DebianVersion("1.2.3-1~test70+0exp2").gen_internal_port(wheezy_exp_regex, squeeze_exp_default)
u'1.2.3-1~test60+0exp2'

No version restrictions: just add default version

>>> DebianVersion("1.2.3-1").gen_internal_port(".*", "~port+1")
u'1.2.3-1~port+1'
gen_internal_rebuild()

Generate an ‘internal rebuild’ version.

If the version is not already a rebuild version, just append the rebuild appendix, otherwise replace the old one. For example:

1.2.3 -> 1.2.3+rebuilt20130215100453
1.2.3+rebuilt20130215100453 -> 1.2.3+rebuilt20130217120517

Code samples:

>>> regex = r"^1\.2\.3\+rebuilt{s}$".format(s=DebianVersion.stamp_regex())
>>> bool(re.match(regex, DebianVersion("1.2.3").gen_internal_rebuild()))
True
>>> bool(re.match(regex, DebianVersion("1.2.3+rebuilt20130215100453").gen_internal_rebuild()))
True
classmethod stamp()
classmethod stamp_regex(stamp=None)
class mini_buildd.daemon.KeyringPackage(identity, gpg, debfullname, debemail, tpl_dir=u'/usr/share/doc/mini-buildd/examples/packages/archive-keyring-template')

Bases: mini_buildd.misc.TmpDir

class mini_buildd.daemon.Keyrings

Bases: object

Hold/manage all gnupg keyrings (for remotes and all repository uploaders).

close()
get_remotes()
get_uploaders()
set_needs_update()
mini_buildd.daemon.get()
mini_buildd.daemon.run()

mini-buildd ‘daemon engine’ run.

django_settings Module

class mini_buildd.django_settings.SMTPCreds(creds)

Bases: object

SMTP creds string parser. Format “USER:PASSWORD@smtp|ssmtp://HOST:PORT”.

>>> d = SMTPCreds(":@smtp://localhost:25")
>>> (d.user, d.password, d.protocol, d.host, d.port)
(u'', u'', u'smtp', u'localhost', 25)
>>> d = SMTPCreds("kuh:sa:ck@smtp://colahost:44")
>>> (d.user, d.password, d.protocol, d.host, d.port)
(u'kuh', u'sa:ck', u'smtp', u'colahost', 44)
mini_buildd.django_settings.configure(smtp_string, loglevel)

Configure django.

mini_buildd.django_settings.get_django_secret_key(home)

This method creates once django’s SECRET_KEY and/or returns it.

Parameters:home (string) – mini-buildd’s home directory.
Returns:string – the (created) key.

ftpd Module

class mini_buildd.ftpd.FtpDHandler(*args, **kwargs)

Bases: pyftpdlib.handlers.FTPHandler

on_disconnect()
on_file_received(file_name)

Make any incoming file read-only as soon as it arrives; avoids overriding uploads of the same file.

on_incomplete_file_received(file_name)
class mini_buildd.ftpd.Incoming

Bases: object

Tool collection for some extra incoming directory handling.

classmethod get_changes()
classmethod is_changes(file_name)
classmethod remove_cruft()

Remove cruft files from incoming.

classmethod remove_cruft_files(files)

Remove all files from list of files not mentioned in a changes file.

classmethod requeue_changes(queue)

Re-queue all existing changes in incoming.

We must feed the the user uploads first, so the daemon does not get any yet-unknown build results (hence the sorting).

mini_buildd.ftpd.run(bind, queue)
mini_buildd.ftpd.shutdown()

gnupg Module

class mini_buildd.gnupg.BaseGnuPG(home)

Bases: object

add_keyring(keyring)
add_pub_key(key)
export(dest_file, identity=u'')
gen_secret_key(template)
get_first_sec_key()
get_first_sec_key_fingerprint()
get_pub_colons(type_regex=u'pub')
get_pub_key(identity)
get_sec_colons(type_regex=u'sec')
recv_key(keyserver, identity)
sign(file_name, identity=None)
verify(signature, data=None)
class mini_buildd.gnupg.Colons(colons_line)

Bases: object

Provide a colon->name mapping for the gpg script-parsable ‘–with-colons’ output.

See /usr/share/doc/gnupg/DETAILS.gz.

creation_date
expiration_date
key_id
type
user_id

fingerprint for ‘fpr’ type

class mini_buildd.gnupg.GnuPG(template, fullname, email)

Bases: mini_buildd.gnupg.BaseGnuPG

get_pub_key(identity=None)
prepare()
remove()
class mini_buildd.gnupg.TmpGnuPG

Bases: mini_buildd.gnupg.BaseGnuPG, mini_buildd.misc.TmpDir

>>> gnupg = TmpGnuPG()
>>> gnupg.gen_secret_key("Key-Type: DSA\nKey-Length: 1024\nName-Real: Üdo Ümlaut\nName-Email: test@key.org")
>>> gnupg.get_first_sec_key().type
u'sec'
>>> gnupg.get_first_sec_key().user_id
u'\xdcdo \xdcmlaut <test@key.org>'
>>> gnupg.get_first_sec_key().key_id  
u'...'
>>> gnupg.get_first_sec_key_fingerprint().user_id  
u'...'
>>> t = tempfile.NamedTemporaryFile()
>>> t.write("A test file\n")
>>> t.flush()
>>> gnupg.sign(file_name=t.name, identity="test@key.org")
>>> gnupg.verify(t.name)
>>> pub_key = gnupg.get_pub_key(identity="test@key.org")
>>> gnupg.close()
>>> tgnupg = TmpGnuPG()
>>> tgnupg.add_pub_key(pub_key)
>>> tgnupg.verify(t.name)
>>> tgnupg.close()

httpd Module

class mini_buildd.httpd.StaticWithIndex

Bases: cherrypy._cptools.HandlerTool

mini_buildd.httpd.run(bind, wsgi_app)

Run the CherryPy WSGI Web Server.

Parameters:
  • bind (string) – the bind address to use.
  • wsgi_app (WSGI-application) – the web application to process.

misc Module

class mini_buildd.misc.API

Bases: object

Helper class to implement an API check.

Inheriting classes must define an __API__ class attribute that should be increased on incompatible changes, and may then check via api_check() method.

api_check()
class mini_buildd.misc.BlockQueue(maxsize)

Bases: Queue.Queue

Wrapper around Queue to get put() block until <= maxsize tasks are actually done. In Queue.Queue, task_done() is only used together with join().

This way can use the Queue directly to limit the number of actually worked-on items for incoming and builds.

load
put(item, **kwargs)
task_done()
class mini_buildd.misc.ConfFile(file_path, snippet=u'', comment=u'#')

Bases: object

ConfFile generation helper.

>>> ConfFile("/tmp/mini_buildd_test_conf_file", "my_option=7").add("my_2nd_option=42").save()
add(snippet)
save()
class mini_buildd.misc.Distribution(dist, meta_map=None)

Bases: object

A mini-buildd distribution string.

>>> d = Distribution("squeeze-test-stable")
>>> d.codename, d.repository, d.suite
(u'squeeze', u'test', u'stable')
>>> d.get()
u'squeeze-test-stable'
>>> d = Distribution("squeeze-test-stable-rollback5")
>>> d.is_rollback
True
>>> d.codename, d.repository, d.suite, d.rollback
(u'squeeze', u'test', u'stable', u'rollback5')
>>> d.get()
u'squeeze-test-stable-rollback5'
>>> d.rollback_no
5
codename
get(rollback=True)
has_lintian_suppress()
is_rollback
repository
rollback
rollback_no

Rollback (int) number: ‘rollback0’ -> 0

suite
class mini_buildd.misc.HoPo(bind)

Bases: object

Convenience class to parse bind string “hostname:port”

test_bind()

Check that we can bind to that port (not already bound, permissions)

class mini_buildd.misc.Keyring(service)

Bases: object

get(host, user=u'')
reset_save_policy()
set(key, password)
class mini_buildd.misc.PkgLog(repository, installed, package, version)

Bases: object

classmethod get_path(repository, installed, package, version=None, architecture=None, relative=False)
classmethod make_relative(path)
class mini_buildd.misc.Status(stati)

Bases: object

Helper class to implement an internal status.

Inheriting classes must give a stati dict to init.

get_status()

Get raw (integer) status.

set_status(status, desc=u'')

Set status with optional description.

status
status_desc
class mini_buildd.misc.TmpDir(tmpdir=None)

Bases: object

Use with contextlib.closing() to guarantee tmpdir is purged afterwards.

close()
classmethod file_dir(file_name)
tmpdir
class mini_buildd.misc.UserURL(url, username=None)

Bases: object

URL with a username attached.

>>> U = UserURL("http://admin@localhost:8066")
>>> (U.username, U.plain, U.full)
(u'admin', u'http://localhost:8066', u'http://admin@localhost:8066')
>>> U = UserURL("http://example.org:8066", "admin")
>>> (U.username, U.plain, U.full)
(u'admin', u'http://example.org:8066', u'http://admin@example.org:8066')
>>> UserURL("http://localhost:8066")
Traceback (most recent call last):
  ...
Exception: UserURL: No username given
>>> UserURL("http://admin@localhost:8066", "root")
Traceback (most recent call last):
  ...
Exception: UserURL: Username given in twice, in URL and parameter
full

URL string with username.

plain

URL string without username.

username
mini_buildd.misc.b642u(base64_bytestream)

Convert base46 string to unicode.

>>> u = b642u('w5xuaWNvZGUgc3Ryw7xuZw==')
>>> u.__class__.__name__
'unicode'
>>> print(u)
Ünicode strüng
mini_buildd.misc.call(args, run_as_root=False, value_on_error=None, log_output=True, error_log_on_fail=True, **kwargs)

Wrapper around subprocess.call().

>>> call(["echo", "-n", "hallo"])
u'hallo'
>>> call(["id", "-syntax-error"], value_on_error="Kapott")
u'Kapott'
mini_buildd.misc.call_sequence(calls, run_as_root=False, value_on_error=None, log_output=True, rollback_only=False, **kwargs)

Run sequences of calls with rollback support.

>>> call_sequence([(["echo", "-n", "cmd0"], ["echo", "-n", "rollback cmd0"])])
>>> call_sequence([(["echo", "cmd0"], ["echo", "rollback cmd0"])], rollback_only=True)
mini_buildd.misc.check_multiprocessing()

Multiprocessing needs shared memory. This may be use to check for misconfigured shm early for better error handling.

mini_buildd.misc.clone_log(dst, src=u'mini_buildd')

Setup logger named ‘dst’ with the same handlers and loglevel as the logger named ‘src’.

mini_buildd.misc.codename_has_lintian_suppress(codename)

Test if the distribution (identified by the codename) has a recent lintian with the ‘–suppress-tags’ option.

mini_buildd.misc.dont_care_run(func, *args, **kwargs)
mini_buildd.misc.fromdos(string)
mini_buildd.misc.get_cpus()
mini_buildd.misc.guess_codeversion(release)

Guess the ‘codeversion’ aka the first two digits of a Debian release version; for releases without version, this falls back to the uppercase codename.

In Debian,
  • point release <= sarge had the ‘M.PrN’ syntax (with 3.1 being a major release).
  • point release in squeeze used ‘M.0.N’ syntax.
  • point releases for >= wheezy have the ‘M.N’ syntax (with 7.1 being a point release).
  • testing and unstable do not gave a version in Release and fall back to uppercase codename

Ubuntu just uses YY.MM which we can use as-is.

>>> guess_codeversion({"Origin": "Debian", "Version": "3.1r8", "Codename": "sarge"})
u'31'
>>> guess_codeversion({"Origin": "Debian", "Version": "4.0r9", "Codename": "etch"})
u'40'
>>> guess_codeversion({"Origin": "Debian", "Version": "6.0.6", "Codename": "squeeze"})
u'60'
>>> guess_codeversion({"Origin": "Debian", "Version": "7.0", "Codename": "wheezy"})
u'70'
>>> guess_codeversion({"Origin": "Debian", "Version": "7.1", "Codename": "wheezy"})
u'70'
>>> guess_codeversion({"Origin": "Debian", "Codename": "jessie"})
u'JESSIE'
>>> guess_codeversion({"Origin": "Debian", "Codename": "sid"})
u'SID'
>>> guess_codeversion({"Origin": "Ubuntu", "Version": "12.10", "Codename": "quantal"})
u'1210'
mini_buildd.misc.hash_of_file(file_name, hash_type=u'md5')

Helper to get any hash from file contents.

mini_buildd.misc.list_get(list_, index, default=None)
mini_buildd.misc.log_call_output(log, prefix, output)
mini_buildd.misc.md5_of_file(file_name)
mini_buildd.misc.mkdirs(path)
mini_buildd.misc.nop(*_args, **_kwargs)
mini_buildd.misc.open_utf8(path, mode=u'r', **kwargs)
mini_buildd.misc.pkg_fmt(status, distribution, package, version, extra=None, message=None)

Generate a package status line.

mini_buildd.misc.qualname(obj)
mini_buildd.misc.run_as_thread(thread_func=None, daemon=False, **kwargs)
mini_buildd.misc.sbuild_keys_workaround()

Create sbuild’s internal key if needed (sbuild needs this one-time call, but does not handle it itself).

mini_buildd.misc.setup_console_logging(level=10)
mini_buildd.misc.sha1_of_file(file_name)
mini_buildd.misc.skip_if_keep_in_debug(func, *args, **kwargs)
mini_buildd.misc.sose_call(args)
>>> sose_call(["echo", "-n", "hallo"])
u'hallo'
>>> sose_call(["ls", "__no_such_file__"])
Traceback (most recent call last):
...
Exception: SoSe call failed (ret=2): ls __no_such_file__
mini_buildd.misc.strip_epoch(version)

Strip the epoch from a version string.

mini_buildd.misc.subst_placeholders(template, placeholders)

Substitue placeholders in string from a dict.

>>> subst_placeholders("Repoversionstring: %IDENTITY%%CODEVERSION%", { "IDENTITY": "test", "CODEVERSION": "60" })
u'Repoversionstring: test60'
mini_buildd.misc.tail(file_object, lines, line_chars=160)
mini_buildd.misc.taint_env(taint)
mini_buildd.misc.timedelta_total_seconds(delta)

python 2.6 compat for timedelta.total_seconds() from python >= 2.7.

mini_buildd.misc.u2b64(unicode_string)

Convert unicode string to base46.

>>> b64 = u2b64("Ünicode strüng")
>>> b64.__class__.__name__
'str'
>>> b64
'w5xuaWNvZGUgc3Ryw7xuZw=='
mini_buildd.misc.web_login(host, user, credentials, proto=u'http', login_loc=u'/accounts/login/', next_loc=u'/mini_buildd/')

packager Module

class mini_buildd.packager.LastPackage(package)

Bases: mini_buildd.misc.API

Subset of ‘Package’ for pickled statistics.

class mini_buildd.packager.Package(daemon, changes)

Bases: mini_buildd.misc.Status

BUILDING = 1
CHECKING = 0
FAILED = -2
INSTALLED = 10
INSTALLING = 2
REJECTED = -1
add_buildresult(bres)
install()

Install package to repository.

This may throw on error, and if so, no changes should be done to the repo.

move_to_pkglog()
notify()
precheck()
took
mini_buildd.packager.package_close(daemon, package)

Close package. Just continue on errors, but log them; guarantee to remove it from the packages dict.

mini_buildd.packager.run(daemon, changes)

reprepro Module

Run reprepro commands.

class mini_buildd.reprepro.Reprepro(basedir)

Bases: object

Abstraction to reprepro repository commands.

Locking

This implicitly provides a locking mechanism to avoid parallel calls to the same repository from mini-buildd itself. This rules out any failed call due to reprepro locking errors in the first place.

For the case that someone else is using reprepro manually, we also always run it with ‘–waitforlock’.

check()
install(changes, distribution)
install_dsc(dsc, distribution)
list(pattern, distribution, typ=None, list_max=50)
migrate(package, src_distribution, dst_distribution, version=None)
reindex()
remove(package, distribution, version=None)
show(package)

root_urls Module

setup Module

mini_buildd.setup.log_exception(log, message, exception, level=40)

urls Module

views Module

class mini_buildd.views.AccountProfileView(**kwargs)

Bases: django.views.generic.base.TemplateView

This just adds repositories to context.

get_context_data(**kwargs)
mini_buildd.views.api(request)
mini_buildd.views.error(request, code, meaning, description, api_cmd=None)
mini_buildd.views.error400_bad_request(request, description=u'Bad request', api_cmd=None)
mini_buildd.views.error401_unauthorized(request, description=u'Missing authorization', api_cmd=None)
mini_buildd.views.error404_not_found(request, description=u'The requested resource could not be found', api_cmd=None)
mini_buildd.views.error405_method_not_allowed(request, description=u'The resource does not allow this request', api_cmd=None)
mini_buildd.views.error500_internal(request, description=u'Sorry, something went wrong', api_cmd=None)
mini_buildd.views.home(request)
mini_buildd.views.log(request, repository, package, version)

webapp Module

class mini_buildd.webapp.WebApp

Bases: django.core.handlers.wsgi.WSGIHandler

This class represents mini-buildd’s web application.

classmethod dumpdata(app_path)
classmethod loaddata(file_name)
classmethod remove_system_artifacts()

Bulk-remove all model instances that might have produced cruft on the system (i.e., outside mini-buildd’s home).

classmethod set_admin_password(password)

This method sets the password for the administrator.

Parameters:password (string) – The password to use.