Indy 2.3.0

This release supports the Maven group-artifact (GA) cache. It is performance critical when we switch to path-mapped storage and retrieve files from a group which contains hundreds or thousands of repositories. The GA cache will tell which repositories contains the requested artifact.

Indy 2.2.0

This release includes below major feature/improvements.

1. Static-proxy support
2. Some NPM fixes.
3. Performance improvements for Httproxy.

Indy 2.1.0

This release addresses a critical latency increase when accessing content in groups that have large numbers (1000's) of constituent repositories, and the new path-mapped storage layer is enabled. In such a case, the extra latency involved with querying the Cassandra database for each constituent repository is leading to a large slow-down for content downloads.

This release addresses the problem by refactoring the content searching mechanism for groups, allowing a query for all matches of a given path in a set of constituent repositories.

Indy 2.0.0

Release notes:

• NEW FEATURE: Cassandra-driven path-to-file mapping for artifact storage

This will enable us to avoid file locking altogether, by using UUIDs for physical filenames and directory structures, then mapping those physical filenames to the paths they represent inside the database. As we overwrite files like maven-metadata.xml, the database entry will update to the new physical file, and the old one will be marked for garbage collection. This enables readers for that file which are already in progress to continue / finish before the file is removed. Similarly, file deletion is a logical operation with a gc mark-and-sweep strategy for the physical storage.

• Improvements to the Honeycomb.io metric reporting subsystem

This enables us to trace sub-spans for threaded / concurrent operations that serve a user request, such as gathering and parsing Maven metadata files. We've also introduced sampling, which will allow us to control the flow of data to Honeycomb.

• Calculate-and-store affected-by groups relevant to a promotion request, instead of recalculating thousands of times
• Implementation of Cassandra-based Not-Found Cache (NFC) [currently disabled, pending performance testing]
• Implementation of remote repository SSL / reachability validation [currently disabled, pending migration process]
• Calculation and sizing of promotion batches based on promotion thread-pool size
• SpecialPathInfo retrieval takes into consideration packageType for the repository
• Improved error handling for failed remote repo downloads
• Improved SSL certificate generation performance for termination of SSL connections that facilitate generic-http proxy caching
• Fix maven-metadata.xml downloads with huge version lists
• Remove some sun.* package imports
• Diagnostic REST endpoint for retrieving groups affected by a given repo / group

Indy 1.9.9

This release corrects a data-loading problem in the new groups-affected-by reverse mapping of the InfinispanStoreDataManager, which was preventing content storage actions from expiring associated, cached metadata.

It also includes preliminary support for sending metrics to Honeycomb (http://honeycomb.io)

Indy 1.9.8

This release is a more comprehensive fix for observed latency problems, over and above those fixed in Indy 1.9.7.2. Additional fixes here include processing certain de-indexing and NFC maintenance events asynchronously instead of synchronously, and optimizing the ability to determine what groups' available content will be affected by a change in some concrete repository. We use a reverse-mapping cache to handle the latter logic.

Changes:

• Push de-indexing process for the content index (during content uploads) to an asynchronous thread-driven execution (PR#1476)
• Extract getArtifactStore() call from deep / large loop in getGroupsAffectedBy calculation (PR#1472)
• Remove duplicate / unused dependencies (PR#1474)
• Thread off NFC cleanup operations that reference getGroupsAffectedBy method (PR#1477)
• Register cache listener for NFC changes in content manager (PR#1479)
• Implement reverse-cache to optimize getGroupsAffectedBy method (PR#1480)
• Fix NPE when store is null in deletion use case (PR#1481)
• Thread off getGroupsAffectedBy calls in NFC content listener (PR#1482)

Indy 1.9.7.2

This release is a safe fallback in case Indy 1.9.8 has a problem. The purpose is to fix a major performance regression in the promotion subsystem.

Changes:

• Push de-indexing process for the content index (during content uploads) to an asynchronous thread-driven execution (PR#1476)
• Extract getArtifactStore() call from deep / large loop in getGroupsAffectedBy calculation (PR#1472)

Indy 1.9.7.1 (regression fix)

Fix a NPE created when calculating the groups affected by a given repository in cases where a group is deleted as the calculation takes place.

• #1463

Indy 1.9.7

This release attempts to improve artifact upload latencies on large deployments that have many, many repository definitions. We also improve some metrics in the repository definition system and add a new cumulative-call metering device that will report the total number of calls for any metered method call. These cumulative counts will be reported as each REST request completes.

Changelog:

• Refactor InfinispanStoreDataManager (repo definitions) to use Externalizable for database persistence
• Refactor DefaultArtifactStoreQuery / InfinispanStoreDataManager to avoid iterating all ArtifactStore instances where possible, instead preferring first-level filtering by StoreKey + StoreDataManager.get() call
• Add cumulative-call metrics, which are summarized in the final log for each REST call
• Add ability to specify a metering ratio, where calls will be sampled at a configurable ratio and metered
• Add metrics for ArtifactStoreQuery methods using a StoreDataManager CDI decorator to wrap the default query class

Indy 1.9.6

This release is intended to address some subtle yet irritating bugs related to cache expiration from Koji content and associated metadata aggregation. It also aims to improve visibility into the content upload process so we can continue diagnosing the elevated latencies we've seen over the past couple months. Two other releases have carried these fixes, but have failed in production due to unforseen regressions or miscommunication in the team. Our new release process is designed to improve the latter problem, and our new load testing configuration has helped the former.

Changelog (1.9.5 to 1.9.6):

• Fix cascade of cache and metadata expirations flowing from expiration of the metadata cache generated from Koji and the aggregated metadata cache
• Implement better SLI measurement of content uploads, complete with subtraction of content transfer time from latency calculation
• Implement low-level content-read and content-write APM measurements
• Implement cumulative timing recording to the logging MDC, which is reported in the END message to the org.commonjava.rest.inbound logger
• Add MD5 / SHA1 headers to HEAD response for content
• Disable deflate content-encoding support since it causes problems with certain http clients
• Add support for validating remote repositories when they are added or updated (disabled by default)
• Deprecate listContents call in IndyContentClientModule in favor of browse module api
• Add timestamps for content accesses to tracked content entries (part of folo content tracking)
• Reuse existing remote repositories when modifying group membership from implied repositories POM event listener
• Add preliminary support for reporting metrics to Prometheus (currently all-or-nothing)
• Allow disabling content index (for simple, reverse-proxy Indy instances)
• Fix HTTP 400 response when uploading snapshot-versioned maven-metadata.xml files
• Skip writing http-metadata.json files for upstream 404 responses and for file types like .md5 and .sha1 (these JSON files preserve upstream response headers to make downstream response to the user more consistent)

Dependency Upgrades (1.9.5 to 1.9.6):

• Infinispan (9.4.15.Final)
• Keycloak (7.0.1)
• Swagger-ui (3.23.11)
• Jackson (2.10.1)