kojismokydingo.builds

Koji Smoky Dingo - Build Utilities

Functions for working with Koji builds

author:

Christopher O'Brien <obriencj@gmail.com>

license:

GPL v3

class BuildFilter(session, limit_tag_ids=None, lookaside_tag_ids=None, imported=None, cg_list=None, btypes=None, state=None)[source]

Bases: object

Parameters:

  • session (ClientSession) -- an active koji client session

  • limit_tag_ids (Iterable[int] | None) -- if specified, builds must be tagged with one of these tags

  • lookaside_tag_ids (Iterable[int] | None) -- if specified, builds must not be tagged with any of these tags

  • imported (bool | None) -- if True, only imported builds are returned. If False, only unimported builds are returned. Default, does not test whether a build is imported or not.

  • cg_list (Iterable[str] | None) -- If specified, only builds which are produced by the named content generators will be returned.

  • btypes (Iterable[str] | None) -- Filter for the given build types, by name. Default, any build type is allowed.

  • state (BuildState | None) -- Filter by the given build state. Default, no filtering by state.

filter_by_btype(build_infos)[source]
Parameters:

build_infos (Iterable[BuildInfo])

Return type:

Iterable[BuildInfo]

filter_by_state(build_infos)[source]
Parameters:

build_infos (Iterable[BuildInfo])

Return type:

Iterable[BuildInfo]

filter_by_tags(build_infos)[source]
Parameters:

build_infos (Iterable[BuildInfo])

Return type:

Iterable[BuildInfo]

filter_imported(build_infos)[source]
Parameters:

build_infos (Iterable[BuildInfo])

Return type:

Iterable[BuildInfo]

class BuildNEVRCompare(binfo)[source]

Bases: NEVRCompare

An adapter for RPM-style Name, Epoch, Version, Release comparisons of a build info dictionary. Used by the build_nvr_sort function.

Parameters:

binfo (BuildInfo)

build: BuildInfo
class NEVRCompare(name, epoch, version, release)[source]

Bases: object

A comparison for RPM-style Name, Epoch, Version, Release sorting. Used by the nvr_sort function.

Parameters:

  • name (str)

  • epoch (str | None)

  • version (str | None)

  • release (str | None)

evr: Tuple[str, str, str]
n: str
build_dedup(build_infos)[source]

Given a sequence of build info dictionaries, return a de-duplicated list of same, with order preserved.

All None infos will be dropped.

Parameters:

build_infos (Iterable[BuildInfo]) -- build infos to be de-duplicated.

Return type:

List[BuildInfo]

build_id_sort(build_infos, dedup=True, reverse=False)[source]

Given a sequence of build info dictionaries, return a de-duplicated list of same, sorted by the build ID

All None infos will be dropped.

Parameters:

  • build_infos (Iterable[BuildInfo]) -- build infos to be sorted and de-duplicated

  • dedup (bool) -- remove duplicate entries. Default, True

  • reverse (bool) -- reverse the sorting. Default, False

Return type:

List[BuildInfo]

build_nvr_sort(build_infos, dedup=True, reverse=False)[source]

Given a sequence of build info dictionaries, sort them by Name, Epoch, Version, and Release using RPM's variation of comparison

If dedup is True (the default), then duplicate NVRs will be omitted.

All None infos will be dropped.

Parameters:

  • build_infos (Iterable[BuildInfo]) -- build infos to be sorted and de-duplicated

  • dedup (bool) -- remove duplicate entries. Default, True

  • reverse (bool) -- reverse the sorting. Default, False

Return type:

List[BuildInfo]

bulk_move_builds(session, srctag, dsttag, build_infos, force=False, notify=False, size=100, strict=False)[source]

Move a large number of builds using multicalls of tagBuildBypass and untagBuildBypass.

Parameters:

  • session (ClientSession) -- an active koji session

  • srctag (int | str | TagInfo) -- Source tag's name or ID. Builds will be removed from this tag.

  • dsttag (int | str | TagInfo) -- Destination tag's name or ID. Builds will be added to this tag.

  • build_infos (Iterable[BuildInfo]) -- Build infos to be tagged

  • force (bool) -- Force tagging/untagging. Re-tags if necessary, bypasses policy. Default, False

  • notify (bool) -- Send tagging/untagging notifications. Default, False

  • size (int) -- Count of tagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Return type:

List[Tuple[BuildInfo, Any]]

bulk_move_nvrs(session, srctag, dsttag, nvrs, force=False, notify=False, size=100, strict=False)[source]

Move a large number of NVRs using multicalls of tagBuildBypass and untagBuildBypass.

NVRs will be resolved to builds. If strict is True then any missing builds will result in a NoSuchBuild exception being raised. Otherwise missing builds will be ignored.

Parameters:

  • session (ClientSession) -- an active koji session

  • srctag (int | str | TagInfo) -- Source tag's name or ID. Builds will be removed from this tag.

  • dsttag (int | str | TagInfo) -- Destination tag's name or ID. Builds will be added to this tag.

  • nvrs (Iterable[str | int]) -- Builds to be tagged, specified by NVR or build ID

  • force (bool) -- Force tagging/untagging. Re-tags if necessary, bypasses policy. Default, False

  • notify (bool) -- Send tagging/untagging notifications. Default, False

  • size (int) -- Count of tagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Return type:

List[Tuple[BuildInfo, Any]]

bulk_tag_builds(session, tag, build_infos, force=False, notify=False, size=100, strict=False)[source]
Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Destination tag's name or ID

  • build_infos (Iterable[BuildInfo]) -- Build infos to be tagged

  • force (bool) -- Force tagging. Re-tags if necessary, bypasses policy. Default, False

  • notify (bool) -- Send tagging notifications. Default, False

  • size (int) -- Count of tagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Raises:

NoSuchTag -- If tag does not exist

Return type:

List[Tuple[BuildInfo, Any]]

bulk_tag_nvrs(session, tag, nvrs, force=False, notify=False, size=100, strict=False)[source]

Tags a large number of builds using multicall invocations of tagBuildBypass.

The entire list of NVRs is validated first, checking that such a build exists. If strict is True then a missing build will cause a NoSuchBuild exception to be raised. If strict is False, then missing builds will be omitted from the tagging operation.

The list of builds is de-duplicated, preserving order of the first instance found in the list of NVRs.

Returns a list of tuples, pairing build info dicts with the result of a tagBuildBypass call for that build.

Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Destination tag's name or ID

  • nvrs (Iterable[str | int]) -- list of NVRs

  • force (bool) -- Bypass policy, retag if necessary. Default, follow policy and do not re-tag.

  • notify (bool) -- Start tagNotification tasks to send a notification email for every tagging event. Default, do not send notifications. Warning, sending hundreds or thousands of tagNotification tasks can be overwhelming for the hub and may impact the system.

  • size (int) -- Count of tagging calls to make per multicall chunk. Default is 100

  • strict (bool) -- Stop at the first failure. Default, continue after any failures. Errors will be available in the return results.

Raises:

  • NoSuchBuild -- If strict and an NVR does not exist

  • NoSuchTag -- If tag does not exist

  • koji.GenericError -- If strict and a tag policy prevents tagging

Return type:

List[Tuple[BuildInfo, Any]]

bulk_untag_builds(session, tag, build_infos, force=False, notify=False, size=100, strict=False)[source]
Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Destination tag's name or ID

  • build_infos (Iterable[BuildInfo]) -- Build infos to be untagged

  • force (bool) -- Force untagging. Bypasses policy. Default, False

  • notify (bool) -- Send untagging notifications. Default, False

  • size (int) -- Count of untagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Raises:

NoSuchTag -- If tag does not exist

Return type:

List[Tuple[BuildInfo, Any]]

bulk_untag_nvrs(session, tag, nvrs, force=False, notify=False, size=100, strict=False)[source]

Untags a large number of builds using multicall invocations of untagBuildBypass.

The entire list of NVRs is validated first, checking that such a build exists. If strict is True then a missing build will cause a NoSuchBuild exception to be raised. If strict is False, then missing builds will be omitted from the untagging operation.

The list of builds is de-duplicated, preserving order of the first instance found in the list of NVRs.

Returns a list of tuples, pairing build info dicts with the result of an untagBuildBypass call for that build.

Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Tag's name or ID

  • nvrs (Iterable[str | int]) -- list of NVRs

  • force (bool) -- Bypass policy checks

  • notify (bool) -- Start tagNotification tasks to send a notification email for every untagging event. Default, do not send notifications. Warning, sending hundreds or thousands of tagNotification tasks can be overwhelming for the hub and may impact the system.

  • size (int) -- Count of untagging calls to make per multicall chunk. Default is 100

  • strict (bool) -- Stop at the first failure. Default, continue after any failures. Errors will be available in the return results.

Raises:

  • NoSuchBuild -- If strict and an NVR does not exist

  • NoSuchTag -- If tag does not exist

  • koji.GenericError -- If strict and a tag policy prevents untagging

Return type:

List[Tuple[BuildInfo, Any]]

decorate_builds_btypes(session, build_infos, with_fields=True)[source]

Augments a list of build_info dicts with two new keys:

  • archive_btype_ids is a list of BType IDs for the build

  • archive_btype_names is a list of BType names for the build

Returns a tuple of the original input of build_infos. Any build_infos which had not been previously decorated will be mutated with their new keys.

If with_fields is True (the default) then any special btype fields will be merged into the build info dict as well.

Parameters:

  • session (ClientSession) -- an active koji session

  • build_infos (Iterable[BuildInfo]) -- list of build infos to decorate and return

  • with_fields (bool) -- also decorate btype-specific fields. Default, True

Return type:

List[DecoratedBuildInfo]

decorate_builds_cg_list(session, build_infos)[source]

Augments a list of build_info dicts with two or four new keys:

  • archive_cg_ids is a list of content generator IDs for each archive of the build

  • archive_cg_names is a list of content generator names for each archive of the build

Returns a tuple of the original input of build_infos. Any build_infos which had not been previously decorated will be mutated with their new keys.

Parameters:

  • session (ClientSession) -- an active koji client session

  • build_infos (Iterable[BuildInfo]) -- list of build infos to decorate and return

Return type:

List[DecoratedBuildInfo]

decorate_builds_maven(session, build_infos)[source]

For any build info which does not have a maven_group_id and hasn't already been checked for additional btype data, augment the btype data.

Parameters:

  • session (ClientSession) -- an active koji client session

  • build_infos (Iterable[BuildInfo]) -- build info dicts to decorate with maven GAV info

Return type:

List[DecoratedBuildInfo]

filter_builds_by_state(build_infos, state=BuildState.COMPLETE)[source]

Given a sequence of build info dicts, return a generator of those matching the given state.

Typically only COMPLETE and DELETED will be encountered here, the rest will rarely result in a build info dict existing.

If state is None then no filtering takes place

Parameters:

  • build_infos (Iterable[BuildInfo]) -- build infos to filter through

  • state (BuildState) -- state value to filter for. Default: only builds in the COMPLETE state are returned

Return type:

Iterable[BuildInfo]

filter_builds_by_tags(session, build_infos, limit_tag_ids=(), lookaside_tag_ids=())[source]
Parameters:

  • session (ClientSession) -- an active koji client session

  • build_infos (Iterable[BuildInfo]) -- build infos to filter through

  • limit_tag_ids (Iterable[int]) -- tag IDs that builds must be tagged with to pass. Default, do not limit to any tag membership.

  • lookaside_tag_ids (Iterable[int]) -- tag IDs that builds must not be tagged with to pass. Default, do not filter against any tag membership.

Return type:

Iterable[BuildInfo]

filter_imported_builds(build_infos, by_cg=(), negate=False)[source]

Given a sequence of build info dicts, yield those which are imports.

build_infos may have been decorated by the decorate_builds_cg_list function. This provides an accurate listing of the content generators which have been used to import the build (if any). In the event that they have not been thus decorated, the cg filtering will rely on the cg_name setting on the build itself, which will only have been provided if the content generator reserved the build ahead of time.

If by_cg is empty and negate is False, then only builds which are non-CG imports will be emitted.

If by_cg is empty and negate is True, then only builds which are non-imports will be emitted (ie. those with a task).

If by_cg is not empty and negate is False, then only builds which are CG imports from the listed CGs will be emitted.

If by_cg is not empty and negate is True, then only builds which are CG imports but not from the listed CGs will be emitted.

by_cg may contain the string "any" to indicate that it matches all content generators. "any" should not be used with negate=True, as this will always result in no matches.

Parameters:

  • build_infos (Iterable[BuildInfo]) -- build infos to filter through

  • by_cg (Iterable[str]) -- Content generator names to filter for

  • negate (bool) -- whether to negate the test, Default False

Return type:

Iterable[BuildInfo]

gather_buildroots(session, build_ids)[source]

For each build ID given, produce the list of buildroots used to create it.

Parameters:

  • session (ClientSession) -- an active koji session

  • build_ids (Iterable[int]) -- build IDs to fetch buildroots for

Return type:

Dict[int, List[dict]]

gather_component_build_ids(session, build_ids, btypes=None)[source]

Given a sequence of build IDs, identify the IDs of the component builds used to produce them (installed in the buildroots of the archives of the original build IDs)

Returns a dict mapping the original build IDs to a list of the discovered component build IDs.

If btypes is None, then all component archive types will be considered. Otherwise, btypes may be a sequence of btype names and only component archives of those types will be considered.

Parameters:

  • session (ClientSession) -- an active koji session

  • build_ids (Iterable[int]) -- Build IDs to collect components for

  • btypes (Iterable[str] | None) -- Component archive btype filter. Default, all types

Return type:

Dict[int, List[int]]

gather_rpm_sigkeys(session, build_ids)[source]

Given a sequence of build IDs, collect the available sigkeys for each rpm in each build.

Returns a dict mapping the original build IDs to a set of the discovered sigkeys.

Parameters:

  • session (ClientSession) -- an active koji session

  • build_ids (Iterable[int]) -- IDs of builds to gather keys from

Return type:

Dict[int, Set[str]]

gather_wrapped_builds(session, task_ids, results=None)[source]

Given a list of task IDs, identify any which are wrapperRPM tasks. For each which is a wrapperRPM task, associate the task ID with the underlying wrapped build info from the request.

Parameters:

  • session (ClientSession) -- an active koji session

  • task_ids (Iterable[int]) -- task IDs to check

  • results (dict | None) -- optional results dict to store mapping of task_ids to build dict in. If not specified a new dict will be created and returned.

Return type:

Dict[int, BuildInfo]

iter_bulk_move_builds(session, srctag, dsttag, build_infos, force=False, notify=False, size=100, strict=False)[source]

Moves a large number of builds from one tag to another using multicall invocations of tagBuildBypass and untagBuildBypass. Builds are specified by build info dicts.

yields lists of tuples containing a build info dict and either None if tagging/untagging was successful, or a fault dict if the tagging or untagging failed. This gives the caller a chance to record the results of each multicall and to present feedback to a user to indicate that the operations are continuing.

Parameters:

  • session (ClientSession) -- an active koji session

  • srctag (int | str | TagInfo) -- Source tag's name or ID. Builds will be removed from this tag.

  • dsttag (int | str | TagInfo) -- Destination tag's name or ID. Builds will be added to this tag.

  • build_infos (Iterable[BuildInfo]) -- Build infos to be tagged

  • force (bool) -- Force tagging/untagging. Re-tags if necessary, bypasses policy. Default, False

  • notify (bool) -- Send tagging/untagging notifications. Default, False

  • size (int) -- Count of tagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Raises:

NoSuchTag -- If either the source or destination tag do not exist

Return type:

Iterator[List[Tuple[BuildInfo, Any]]]

iter_bulk_tag_builds(session, tag, build_infos, force=False, notify=False, size=100, strict=False)[source]

Tags a large number of builds using multicall invocations of tagBuildBypass. Builds are specified by build info dicts.

yields lists of tuples containing a build info dict and the result of the tagBuildBypass call for that build. This gives the caller a chance to record the results of each multicall, and to present feedback to a user to indicate that the operations are continuing.

Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Destination tag's name or ID

  • build_infos (Iterable[BuildInfo]) -- Build infos to be tagged

  • force (bool) -- Force tagging. Re-tags if necessary, bypasses policy. Default, False

  • notify (bool) -- Send tagging notifications. Default, False

  • size (int) -- Count of tagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Raises:

NoSuchTag -- If tag does not exist

Return type:

Iterator[List[Tuple[BuildInfo, Any]]]

iter_bulk_untag_builds(session, tag, build_infos, force=False, notify=False, size=100, strict=False)[source]

Untags a large number of builds using multicall invocations of untagBuildBypass. Builds are specified by build info dicts.

yields lists of tuples containing a build info dict and the result of the untagBuildBypass call for that build. This gives the caller a chance to record the results of each multicall, and to present feedback to a user to indicate that the operations are continuing.

Parameters:

  • session (ClientSession) -- an active koji session

  • tag (int | str | TagInfo) -- Tag's name or ID

  • build_infos (Iterable[BuildInfo]) -- Build infos to be untagged

  • force (bool) -- Force untagging, bypasses policy. Default, False

  • notify (bool) -- Send untagging notifications. Default, False

  • size (int) -- Count of untagging operations to perform in a single multicall. Default, 100

  • strict (bool) -- Raise an exception and discontinue execution at the first error. Default, False

Raises:

NoSuchTag -- If tag does not exist

Return type:

Iterator[List[Tuple[BuildInfo, Any]]]

iter_latest_maven_builds(session, tag, pkg_names=None, inherit=True)[source]

Yields ((G, A, V), build_info) representing the latest build for each GAV in the tag.

If pkg_names is given, then only those builds matching the given package names are yielded.

Parameters:

  • session (ClientSession) -- an active koji client session

  • tag (int | str | TagInfo) -- the tag to load latest maven builds from

  • pkg_names (Iterable[str] | None) -- optional filter for build with the given package name

  • inherit (bool) -- if True also yield information for builds in the tag's parents

Raises:

NoSuchTag -- if tag could not be resolved

Return type:

Iterator[Tuple[Tuple[str, str, str], BuildInfo]]

latest_maven_builds(session, tag, pkg_names=None, inherit=True)[source]

Returns a dict mapping each (G, A, V) to a build_info dict, representing the latest build for each GAV in the tag.

If pkg_names is given, then only those builds matching the given package names are returned.

Parameters:

  • session (ClientSession) -- an active koji client session

  • tag (int | str | TagInfo) -- the tag to load latest maven builds from

  • pkg_names (Iterable[str] | None) -- optional filter for build with the given package name

  • inherit (bool) -- if True also yield information for builds in the tag's parents

Raises:

NoSuchTag -- if tag could not be resolved

Return type:

Dict[Tuple[str, str, str], BuildInfo]