0.7.0

Perceval 0.7 - (2017-03-21)

New features and improvements:

• New set of backends added:
  • Hyperkitty
  • NNTP
  • Slack
• RateLimitError exception added for handling rate limit errors.
• Code was cleaned to follow most of the PEP8 style guidelines.

Backend improvements:

• git
  • retry calls on SSH commands were added to avoid temporal server errors
• github
  • HHTP 404 errors are managed when user's organizations are fetched
  • generic RateLimitError exception is used

Bugs fixed:

• In Mediawiki backend, the log messages written when a revision is not
  found were set to ERROR when the real level should have been WARNING.
• The URL used to fetch jobs in Jenkins was not common to all servers.
• When UUIDs are generated with some input data, some errors may be raised
  due to problems encoding invalid characters on the input. To avoid these
  problems, a surrogate escape control error has been set when data is
  encoded to UTF8. (#123)
• Handle Meetup requests rate limit. (#126)

0.6.0

Perceval 0.6 - (2017-02-02)

Backend improvements:

• bugzillarest
  • messages in client errors were improved
• git
  • new method is_detached()
• mbox
  • ignore messages with invalid dates on Date header
• phabricator
  • retry requests on HTTP 502 and 503 errors

Bugs fixed:

• The mbox class from Python's mailbox module fails when it tries to
  decode non-ascii unix-from headers. This header is used as a separator
  between messages. When this error is found, the class stops reading messages
  from the mbox. Wrapping mbox class to override the way messages are read
  was needed to catch the exception and decode the header using UTF-8.
• When a user does not exists on Phabricator, the API does not return an
  error. It returns an empty list. The case where an empty list is returned
  was not managed by the parser, which raised exceptions.
• In gerrit, the identifier of the change, Change-Id (or id), is not unique.
  What it is unique in a gerrit sever is the number of each change and review.
  This number is used now instead of id as the identifier of a review
• When Git repositories are reset to the current status on upstream, some of
  them cannot deal with origin reference because it is ambiguous. Replacing
  it by FETCH_HEAD works on those repositories with defined branches on
  the origin.
• Git repositories in detached state do not need to be reset after git fetch
  is called. This call is now skipped when a repository in this state
  is in use. (#105)

0.5.0

Perceval 0.5.0 - (2017-01-17)

New features and improvements:

• New set of backends added:
  • Askbot
  • Meetup
  • RSS
• Definition of perceval.backends namespace and dynamic loading of backends.
  These two features allow to have third party backends or packages of
  backends that can be imported and used at runtime.
• Mozilla's backends were moved to their own package: perceval-mozilla.
• Commands were refactorized generalizing their usage into BackendCommand
  class which can run any type of backend. This was possible thanks to the
  creation of BackendCommandArgumentParser class, that defines, manages and
  parses those arguments needed to run a command; the definition of pre_init()
  and post_init() methods during the initialization of the instance; and to
  the implementation of setup_cache() as a public function of the cache
  module.

Backend improvements:

• bugzilla
  • set maximum number of bugs requested on CSV queries
• git
  • parse commit trailers
  • new methods is_empty() and count_objects()
  • set missing encodings for the command output
  • cleaning up of the module
• jenkins
  • ignore invalid job builds
• supybot
  • parse action and bot empty lines
  • parse user actions with the format *nick msg
  • generate item ids using the body of the message

Bugs fixed:

• The field 'timestamp' on metadata was not generated in UTC. The call
  to datetime.now() does not generate a timestamp in UTC. It does using
  the timezone of the system. The right way is to call to datetime.utcnow()
  method. (#92)
• The docker image for Perceval purged the git package after installing
  perceval. This made impossible to run the backend for Git because
  Perceval needs of git command under the hood. (#95)
• Git empty repositories threw errors while fetching commits. Those were
  raised because on empty repositories, those which do not have any history
  or are only initialized, there are some commands that cannot be run, like
  a pull or log. If any of this commands is called an error is be returned.
  It was fixed checking whether the repository is empty and returning
  an empty list of commits for those cases. (#102 and #107)

0.4.0

Perceval 0.4.0 - (2016-11-03)

New features and improvements:

• category field was added to items metadata to classify the type of
  the item generated with each backend.
• The tag attribute added to the backends allows to mark the items
  with a custom label.
• Two class methods, has_caching and has_resuming, are part now
  of Backend class interface to notify whether a backend supports
  caching and/or resuming of items.

Backend improvements:

• jenkins
  • support blacklist of jobs
• mediawiki
  • use API pages methods by default
• phabricator
  • fetch and include projects data assigned to each task
• redmine
  • fetch and include users data
• remo
  • support new version of the API
• supybot
  • parse messages written by special bots

Bugs fixed:

• Filepaths on merge commits were not captured on Git backend. This was
  neccesary in those cases where merge commits only include data about
  lines added and removed because the filepaths were not parsed and
  included on the item data. (#63)
• The url argument on the Gerrit backend was set to optional. It is
  mandatory. Thus, it was set to positional on the argument parser. (#60)
• Newer versions of Phabricator fixed a bug on API Conduit regarding
  'constraints' parameter. The Phabricator client was modified to fix
  this bug, too. (#80)
• Python's requests library decompresses gzip-encoded responses, but
  in some cases is only able to decompress some parts due to encoding
  issues or to mixed contents. This problem was fixed downloading and
  storing the orinal/raw data (compressed or decompressed) for furthed
  processing.
• Jira backend did not return items in order, from oldest to newest. (#89)
• Dates with invalid timezones were not parsed. In those cases, the
  the dates will be converted usin UTC by default. (#73)

0.3.0

Perceval 0.3.0 - (2016-09-19)

New features and improvements:

• New set of backends added:
  • Phabricator
  • Redmine
• Add support for creating PyPi packages

Backend improvements:

• jira
  • fetch additional information about custom fields
• mediawiki
  • add a flag which ignores the MAX_RECENT_DAYS constraint when the
    backend is tested

Bugs fixed:

• Cache tests for Redmine backend checked the values retrieved from the
  repository but not from the cache.
• Timestamps generated to fetch data from a given date included invalid
  timezone information for Mediawiki API (>=1.27). It only works with Zulu
  dates. (#54)
• Date strings that included information after the timezone were not parsed:
  Thu, 14 Aug 2008 02:07:59 +0200 CEST. (#57)

0.2.0

Perceval 0.2.0 - (2016-07-20)

New features and improvements:

• New set of backends added:
  • Bugzilla (REST API)
  • Confluence
  • Discourse
  • Gmane
  • Jenkins
  • Kitsune (Mozilla)
  • Mediawiki
  • Pipermail
  • ReMo (Mozilla)
  • Supybot
  • Telegram
• The origin of the fetched data is configurable.
• Unit tests for GitHub, Jira and Stack Exchange were added. Other tests
  were added and improved. Now, the unit tests framework covers a 83% of
  the source code.

Backend improvements:

• gerrit
  • support server version 2.8
• git
  • filtering by branches
  • so far, the full log was read before parsing it; now, it is parsed and processed
    while is being read
• github
  • full control of GitHub API rate limit
  • the program can be sent to sleep until the rate limit is reset again
• mbox
  • fetches messages since a given date
• pipermail
  • fetches messages from a mod_mbox interface (i.e: Apache)

Bugs fixed:

• Dates that included parentheses sections were not parsed:
  2005 15:20:32 -0100 (GMT+1).
• An encoding error was raised when version.py module was imported. (#32)
• The call in chain of functions utcnow() and timestamp() from the
  module time, produced wrong timezones on the GitHub backend.
• Action IRC messages (leading with a single *) were ignored. (#48)
• The backoff field received in a Stack Exchange API response was
  ignored. When this field is set, any client must wait the number of
  seconds specified on it before sending any new request.
• The query used in Gerrit to retrieve the reviews was badly formed when
  the blacklist filter contained two or more reviews. (#50)

0.1.0

Perceval 0.1.0 - (2016-03-30)

New features and improvements:

• Supports Python 3.4 and newer versions.
• Fetches and caches information from several software repositories:
  • Bugzilla
  • Gerrit
  • Git
  • GitHub
  • Jira
  • MBox
  • Stack Exchange
• Metadata fields are added to fetched items.
• Dates and times used to request data are always converted to UTC.
• Unit testing framework is available. Currently, these
  unit tests cover the 62% of the source code.

Bugs fixed:

• Some Git commit log entries may not contain information about files.
  Before this was fixed, Perceval raised an exception with a "Unexpected
  end of log stream" message. (#8)
• Empty Git commit logs raised erros when were parsed. (#17)
• Character ^M) produced some parsing errors in the Git backend. (#21)