Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial dependency loader (Hawkey based) #13

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

lhh
Copy link

@lhh lhh commented Nov 29, 2018

I'm working on other bits as well for pungi integration, but, wanted to make sure I got the layouts right.

This uses Hawkey to interface with libdnf and pull/parse package and repository metadata:

https://github.com/rpm-software-management/libdnf/tree/master/python/hawkey

@lhh lhh force-pushed the master branch 6 times, most recently from 730e5ba to e29f59f Compare November 29, 2018 19:28
@jguiditta
Copy link
Member

@lhh can you add that reference we talked about when you get a chance? I will go over this in depth on Monday, and I think that would be helpful to be able to look at

@lhh
Copy link
Author

lhh commented Dec 1, 2018

Added. Within the code we specifically call things in package-py.cpp in the above repository which is extracted from repository information provided via libdnf.

The bits that actually gather and parse repository metadata is in libdnf directly.

@fuzzball81
Copy link
Member

@lhh We should update setup.py to include rpmreq as a requirement. No need to worry about any of the other requirements files, they are driven by setup.py. I will review the rest of the changes next week.

@lhh
Copy link
Author

lhh commented Dec 1, 2018 via email

@fuzzball81
Copy link
Member

@lhh maybe a second pull request with that change isolated. That way if we find the change causes issues with testing on Travis we can decide what to do from there, but this pull request is not blocked.

Copy link
Member

@jguiditta jguiditta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts on approach.

"""

def __init__(self, **kwargs):
if 'config' not in kwargs:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we just collapse this down to a config param, and then add a helper method to parse this into whatever ConfigManager needs below to initialize it? Then we can have an explicit signature for init, which I think makes usage clearer (if would have 4 params: config, logger, filename, version, which could then be documented, so one know what to pass in).

self.log = kwargs['logger']
self.provided_log = True
else:
self.log = logging.getLogger(__name__)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use our logging helper of the same name, so the actual backend we log to can be specified. Hmm, actually, it should probably go one stepfuther. Shouldn't this just be in the config file,and then the Configmanager loads the proper logger defined there? In that casewedon't even need to take the logger here,just set it up in the manager

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, we can have a logging configuration snippet that then just wraps our existing logging infra - good idea.

So, we then don't need it as part of the init function at all, which is good.

atkinson/pungi/depends.py Show resolved Hide resolved
self.wrong_version = transmogrify_unmet(wrong_version)
self.unmet = transmogrify_unmet(unmet_deps)

if 'logger' in kwargs:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather not see us interacting with args passed to the object on instantiation here, and simply always use self.logger.

return str_format.format(fname, len(self.met), len(self.wrong_version), len(self.unmet))


def transmogrify_met(met_deps):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure where to put this, so I am going to drop it here. I am proposing a slight change in the design of this, suggesting that DependencySet do more what it sounds like (to me, at least), and return a set of Dependency objects. I notice that met, wrong_version and un_met all return hashes with the same fields, so why not simply add a type field or similar to denote met, un_met, or wrong_version. Then your transmogrify methods change to simply populate the current object based on type, and the Set object calls in the init something like:

self.met = [[met_dep for Dependency(type='met', met_dep) in met_deps]]
self.wrong_version = [[wrong for Dependency(type='wrong_version', wrong) in wrong_version]]
self.unmet = [[unmet for Dependency(type='unmet', unmet) in unmet_deps]]

return (epoch, version, release)


def dep_to_nevr(dep):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These could probably remain outside the object, making them more reusable (and possibly moving them to toolchest?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to making a toolchest rpm module.

- http://whatever-location-2/x86_64
build-version-2:
- http://whatever-location-3/arch/
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the nice doc string here!

return (epoch, version, release)


def dep_to_nevr(dep):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to making a toolchest rpm module.

return ret


def _main(argv):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point we should remove this. But that can be at a later time when we have more of the runner infrastructure setup. We could however pull this out to a contrib or manual test area that just calls this module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants