Merge branch 'master' of github.com:tulios/kafkajs into lint-disallow…

…-focused-tests

.github/workflows/combine.yml

@@ -0,0 +1,18 @@
+name: "Combine Dependabot PRs"
+on:
+  workflow_dispatch:
+
+jobs:
+  combine-prs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: maadhattah/combine-dependabot-prs@main
+        with:
+          branchPrefix: "dependabot"
+          mustBeGreen: true
+          combineBranchName: "combined-prs"
+          ignoreLabel: "nocombine"
+          baseBranch: "master"
+          openPR: true
+          allowSkipped: false

CHANGELOG.md

@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [1.16.0] - 2022-02-09
+
+### Added
+  - Allow manual heartbeating from inside `eachMessage` handler #1255
+  - Add `rebalancing` consumer event #1067 #1079
+  - Add overload typings for all event types #1202
+  - Return `configSource` in `admin.decribeConfigs` #1023
+  - Add `topics` property to `admin.fetchOffsets` to fetch offsets for multiple topics #992 #998
+  - Improve error output from `admin.createTopic` #1104
+  - Export Error classes #1254
+  - Validate `brokers` list contains strings #1284
+  - Throw error when failing to stop or disconnect consumer #960
+
+### Changed
+
+  - Don't commit offsets from `consumer.seek` when `autoCommit` is `false` #1012
+  - Do not restart the consumer on non-retriable errors #1274
+  - Downgrade consumer rebalance error log to `warn` #1279
+  - Make default round-robin partitioner topic-aware #1112
+
+### Fixed
+  - Fix `offset` type of `consumer.seek` #981
+  - Fix crash when used in Electron app built with electron-builder #984
+  - Improve performance of Fetch requests #985
+  - Fix crash when using topics with name of built-in Javascript functions #995
+  - Fix type of consumer constructor to require config object #1002
+  - Fix message type to allow `null` key #1037
+  - Respect `heartbeatInterval` when invoking `heartbeat` concurrently #1026
+  - Fix type of `timestamp` of `LoggerEntryContent` to be string #1082
+  - Fix return type of `admin.describeAcls` #1118
+  - Fix consumer getting stuck in `DISCONNECTING` state if in-flight requests time out during disconnect #1208
+  - Fix failed serialization of BigInts when logging #1234
+  - Fix crash when committing offsets for a topic before consumer initialization #1235
+  - Reauthenticate to all brokers on demand #1241
+  - Remove unnecessary warn log when calling `admin.deleteTopicRecords` with offset `-1` #1265
+  - Handle empty control batches #1256
+  - Send empty topic array as null when fetching metadata #1184
+
 ## [1.15.0] - 2020-11-24
 ### Added
   - Initial work for static membership #888

CONTRIBUTING.md

@@ -0,0 +1,40 @@
+# Contributing to KafkaJS
+
+Thank you for considering contributing to KafkaJS!
+
+Reading and following these guidelines will help us make the contribution process easy and effective for everyone involved. It also communicates that you agree to respect the time of the developers managing and developing these open source projects.
+
+## Getting Started
+
+Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:
+
+- Search for existing issues and PRs before creating your own.
+- The maintainers of KafkaJS are people that volunteer their time. We try to address issues and PRs in a timely manner, but cannot make any guarantees. Please don't @mention individual maintainers to try to get their attention.
+
+### Issues
+
+Issues should be used to report problems with the library, request a new feature, or to discuss potential changes before a PR is created. Please follow the issue template that's provided when you first create an issue in order to collect all the necessary information.
+
+Issues are not a support channel. Please use [StackOverflow](https://stackoverflow.com/questions/tagged/kafkajs), [Slack](https://kafkajs-slackin.herokuapp.com/) or other online resources instead. [Limited support from a maintainer](https://github.com/sponsors/Nevon?frequency=one-time&sponsor=Nevon) is available to sponsors.
+
+If you find an issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one. Adding a [reaction](https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) can also help be indicating to our maintainers that a particular problem is affecting more than just the reporter.
+
+### Pull Requests
+
+PRs are welcome and can be a quick way to get your fix or improvement out. If you've never contributed before, see [the contribution guidelines on our website](https://kafka.js.org/docs/contribution-guide) for practical information on how to get started.
+
+In general, PRs should:
+
+- Only fix/add the functionality in question **OR** address wide-spread whitespace/style issues, not both.
+- Add tests for fixed or changed functionality.
+- Address a single concern.
+- Update the [Typescript type definitions](./types) if your change introduces any new or affects existing interfaces.
+- Include documentation if it changes the functionality of the library. Our [documentation](https://kafka.js.org/docs/getting-started) is in the [`/docs`](./docs/) folder of the repo.
+
+If your PR introduces a change in functionality or adds new functionality, always open an issue first to discuss your proposal before implementing it. This is especially crucial for breaking changes, which will almost always be rejected unless discussed first. For bug fixes this is not required, but still recommended.
+
+Once a PR is merged and the master build is successful, a pre-release version of KafkaJS will be published to NPM in the [beta channel](https://www.npmjs.com/package/kafkajs/v/beta), which you can use until a there has been a stable release made containing your change.
+
+## Getting Help
+
+Join our [Slack community](https://kafkajs-slackin.herokuapp.com/) if you have questions about the contribution process or otherwise want to get in touch.

azure-pipelines.yml

@@ -30,7 +30,7 @@ jobs:
     steps:
     - task: NodeTool@0
       inputs:
-        versionSpec: '10.x'
+        versionSpec: '14.x'
     - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
       displayName: should skip tests
     - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -45,7 +45,7 @@ jobs:
     steps:
     - task: NodeTool@0
       inputs:
-        versionSpec: '10.x'
+        versionSpec: '14.x'
     - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
       displayName: should skip tests
     - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -64,7 +64,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -89,7 +89,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -114,7 +114,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -139,7 +139,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -166,7 +166,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -194,7 +194,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: ./scripts/pipeline/shouldRunTests.sh && SKIP_TESTS=true && echo "Only non-code has changed!" || test true
         displayName: should skip tests
       - bash: test $SKIP_TESTS && echo "Skipped!" || yarn install
@@ -225,7 +225,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - task: Npm@1
         inputs:
           command: publish
@@ -245,7 +245,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: |
           git config user.name "Website Build Bot"
           git config user.email "<>"
@@ -263,7 +263,7 @@ jobs:
           git add .
           DOC_VERSION=$(echo "$VERSION" | sed -e "s/^refs\/tags\/v//")
           git commit -m "Versioned docs for $DOC_VERSION"
-          git push origin master
+          git push origin HEAD:master
         displayName: commit
         env:
           VERSION: $(Build.SourceBranch)
@@ -284,7 +284,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: yarn install
         displayName: yarn install
       - bash: ./scripts/pipeline/checkBetaEligibility.js
@@ -312,7 +312,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: yarn install
         displayName: yarn install
       - bash: ./scripts/pipeline/generateBetaReleaseVersion.js
@@ -359,7 +359,7 @@ jobs:
     steps:
       - task: NodeTool@0
         inputs:
-          versionSpec: '10.x'
+          versionSpec: '14.x'
       - bash: git config core.autocrlf true || test true
         displayName: git config core.autocrlf
       - bash: git config --global user.name "${GH_NAME}" || test true

docker-compose.0_10.yml

@@ -35,6 +35,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092,SSL://kafka1:29093,SSL_HOST://localhost:9093,SASL_SSL://kafka1:29094,SASL_SSL_HOST://localhost:9094
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -82,6 +84,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29095,PLAINTEXT_HOST://localhost:9095,SSL://kafka2:29096,SSL_HOST://localhost:9096,SASL_SSL://kafka2:29097,SASL_SSL_HOST://localhost:9097
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -129,6 +133,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:29098,PLAINTEXT_HOST://localhost:9098,SSL://kafka3:29099,SSL_HOST://localhost:9099,SASL_SSL://kafka3:29100,SASL_SSL_HOST://localhost:9100
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'

docker-compose.0_11.yml

@@ -35,6 +35,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092,SSL://kafka1:29093,SSL_HOST://localhost:9093,SASL_SSL://kafka1:29094,SASL_SSL_HOST://localhost:9094
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -82,6 +84,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29095,PLAINTEXT_HOST://localhost:9095,SSL://kafka2:29096,SSL_HOST://localhost:9096,SASL_SSL://kafka2:29097,SASL_SSL_HOST://localhost:9097
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -129,6 +133,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:29098,PLAINTEXT_HOST://localhost:9098,SSL://kafka3:29099,SSL_HOST://localhost:9099,SASL_SSL://kafka3:29100,SASL_SSL_HOST://localhost:9100
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'

docker-compose.1_1.yml

@@ -35,6 +35,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092,SSL://kafka1:29093,SSL_HOST://localhost:9093,SASL_SSL://kafka1:29094,SASL_SSL_HOST://localhost:9094
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -82,6 +84,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29095,PLAINTEXT_HOST://localhost:9095,SSL://kafka2:29096,SSL_HOST://localhost:9096,SASL_SSL://kafka2:29097,SASL_SSL_HOST://localhost:9097
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -129,6 +133,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:29098,PLAINTEXT_HOST://localhost:9098,SSL://kafka3:29099,SSL_HOST://localhost:9099,SASL_SSL://kafka3:29100,SASL_SSL_HOST://localhost:9100
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'

docker-compose.2_2.yml

@@ -35,6 +35,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092,SSL://kafka1:29093,SSL_HOST://localhost:9093,SASL_SSL://kafka1:29094,SASL_SSL_HOST://localhost:9094
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -81,6 +83,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29095,PLAINTEXT_HOST://localhost:9095,SSL://kafka2:29096,SSL_HOST://localhost:9096,SASL_SSL://kafka2:29097,SASL_SSL_HOST://localhost:9097
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'
@@ -127,6 +131,8 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SSL:SSL,SSL_HOST:SSL,SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL
       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:29098,PLAINTEXT_HOST://localhost:9098,SSL://kafka3:29099,SSL_HOST://localhost:9099,SASL_SSL://kafka3:29100,SASL_SSL_HOST://localhost:9100
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
+      KAFKA_DEFAULT_REPLICATION_FACTOR: '2'
+      KAFKA_NUM_PARTITIONS: '2'
       KAFKA_DELETE_TOPIC_ENABLE: 'true'
       KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
       KAFKA_SSL_KEYSTORE_FILENAME: 'kafka.server.keystore.jks'