Home

Pulsar Wiki

Pulsar Improvement Proposals

We encourage you to document any big change or feature or any addition to public user APIs through a design document to be discussed with the community.

Proposed

• PIP 198: Standardize PR Naming Convention using GitHub Actions
• PIP-195: New bucket based delayed message tracker
• PIP-194: Pulsar client: seek command add epoch
• PIP-193: Sink preprocessing Function
• PIP-192: New Pulsar Broker Load Balancer
• PIP-191: Support batched message using entry filter
• PIP-190: Simplify documentation release and maintenance strategy
• PIP-189: No batching if only one message in batch
• PIP-188: Cluster migration or Blue-Green cluster deployment support in Pulsar
• PIP-187: Add API to analyse a subscription backlog and provide a accurate value
• PIP-186: Introduce two phase deletion protocol based on system topic
• PIP-184: Topic specific consumer priorityLevel
• PIP-183: Reduce unnecessary REST call in broker
• PIP-182: Provide new load balance placement strategy implementation for ModularLoadManagerStrategy
• PIP-181: Pulsar Shell
• PIP-180: Shadow Topic, an alternative way to support readonly topic ownership
• PIP-179: Support the admin API to check unknown request parameters
• PIP-178: Multiple snapshots for transaction buffer
• PIP-177: Add the classLoader field for SchemaDefinition
• PIP-176: Refactor Doc Bot
• PIP-175: Extend time based release process
• PIP-174: Provide new implementation for broker dispatch cache
• PIP-173: Create a built-in Function implementing the most common basic transformations
• PIP-165: Auto release client useless connections
• PIP-162: LTS Releases
• PIP-161: Exclusive Producer: new mode ExclusiveWithFencing
• PIP-160: Make transactions work more efficiently by aggregation operation for transaction log and pending ack store
• PIP-157: Bucketing topic metadata to allow more topics per namespace
• PIP-156: Build and Run Pulsar Server on Java 17
• PIP-155: Drop support for Python2
• PIP-154: Max active transaction limitation for transaction coordinator
• PIP-148: Create Pulsar client release notes
• PIP-144: Making SchemaRegistry implementation configurable
• PIP-143: Support split bundle by specified boundaries
• PIP-137: Pulsar Client Shared State API
• PIP-136: Sync Pulsar policies across multiple clouds
• PIP-135: Include MetadataStore backend for Etcd
• PIP-130: Apply redelivery backoff policy for ack timeout
• PIP-124: Create init subscription before sending message to DLQ
• PIP-123: Introduce Pulsar metadata CLI tool
• PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder
• PIP-121: Pulsar cluster level auto failover
• PIP-120: Enable client memory limit by default
• PIP-119: Enable consistent hashing by default on KeyShared dispatcher
• PIP-118: Do not restart brokers when ZooKeeper session expires
• PIP 117: Change Pulsar Standalone defaults
• PIP 116: Create Pulsar Writing Style Guide
• PIP 112: Generate Release Notes Automatically
• PIP-111: Add messages produced by Protocol Handler When checking maxMessagePublishBufferSizeInMB
• PIP 110: Topic metadata
• PIP 108: Add method to help user judge if consumer queue has message PIP
• PIP 107: Introduce the chunk message ID PIP
• PIP 106: Broker extensions to provide operators of enterprise-wide clusters better control and flexibility
• PIP 105: Support pluggable entry filter in Dispatcher
• PIP 104: Add new consumer type: TableView
• PIP 101: Add seek by index feature for consumer
• PIP 100: Pulsar pluggable topic factory
• PIP 97: Asynchronous Authentication Provider
• PIP 94: Support level increment delay for ReconsumerLater interface
• PIP 80: Unified namespace-level admin API
• PIP 79: Reduce redundant producers from partitioned producer
• PIP 59: gPRC Protocol Handler
• PIP 49: Permission levels and inheritance
• PIP 48: hierarchical admin api
• PIP 46: Next-gen Proxy
• PIP 35: Improve topic lookup for topics that have high number of partitions
• PIP 21: Pulsar Edge Component

Accepted

• PIP-152: Support subscription level dispatch rate limiter setting
• PIP 149: Making the REST Admin API fully async
• PIP 132: Include message header size when check maxMessageSize for non-batch message on the client side
• PIP 131: Resolve produce chunk messages failed when topic level maxMessageSize is set
• PIP 129: Introduce intermediate state for ledger deletion
• PIP 109: Introduce Bot to Improve Efficiency of Developing Docs
• PIP 108: Pulsar Feature Matrix (Client and Function)
• PIP 106: Negative acknowledgment backoff
• PIP 99: Pulsar Proxy Extensions
• PIP 98: Redesign Pulsar Information Architecture
• PIP 96: Message payload processor for Pulsar client
• PIP 95: Smart Listener Selection with Multiple Bind Addresses
• PIP 93: Transaction performance tools
• PIP 92: Topic policy across multiple clusters
• PIP 91: Separate lookup timeout from operation timeout
• PIP 90: Expose broker entry metadata to the client
• PIP 89: Structured document logging
• PIP 87: Upgrade Pulsar Website Framework (Docusaurus)
• PIP 88: Replicate schemas across multiple
• PIP 86: Pulsar Functions: Preload and release external resources
• PIP 85: Add Schema Information to Message in Java Client API
• PIP 85: Expose Pulsar-Client via Function/Connector BaseContext
• PIP 84: Pulsar client: Redeliver command add epoch
• PIP 83: Pulsar client: Message consumption with pooled buffer
• PIP 82: Tenant and namespace level rate limiting
• PIP 81: Split the individual acknowledgments into multiple entries
• PIP 78: Generate Docs from Code Automatically
• PIP 77: Contribute Supernova to Apache Pulsar
• PIP 76: Streaming Offload
• PIP 75: Replace protobuf code generator
• PIP 74: Pulsar client memory limits
• PIP 73: Configurable data source priority for message reading
• PIP 72: Introduce Pulsar Interface Taxonomy: Audience and Stability Classification
• PIP 71: Pulsar SQL migrate SchemaHandle to presto decoder
• PIP 70: Introduce lightweight broker entry metadata
• PIP 69: Schema design for Go client
• PIP 68: Exclusive Producer
• PIP 67: Pulsarctl - An alternative tools of pulsar-admin
• PIP 66: Pulsar Function Mesh
• PIP 65: Adapting Pulsar IO Sources to support Batch Sources
• PIP 64: Introduce REST endpoints for producing, consuming and reading messages
• PIP 63: Readonly Topic Ownership Support
• PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories
• PIP 61: Advertised multiple addresses
• PIP 60: Support Proxy server with SNI routing
• PIP 58 : Support Consumers Set Custom Retry Delay
• PIP 57: Improve Broker's Zookeeper Session Timeout Handling
• PIP 56: Python3 Migration
• PIP 55: Refresh Authentication Credentials
• PIP 54: Support acknowledgement at batch index level
• PIP 53: Contribute DotPulsar to Apache Pulsar
• PIP 52: Message dispatch throttling relative to publish rate
• PIP 51: Tenant policy support
• PIP 50: Package Management
• PIP 47: Time Based Release Plan
• PIP 45: Pluggable metadata interface
• PIP 44: Separate schema compatibility checker for producer and consumer
• PIP 43: producer send message with different schema
• PIP 42: KoP - Kafka on Pulsar
• PIP 41: Pluggable Protocol Handler
• PIP 40: Pulsar Manager
• PIP 39: Namespace Change Events
• PIP 38: Batch Receiving Messages
• PIP 37: Large message size handling in Pulsar
• PIP 33: Replicated subscriptions
• PIP 36: Max Message Size
• PIP 34: Add new subscribe type Key_shared
• PIP 32: Go Function API, Instance and LocalRun
• PIP 31: Transaction Support
• PIP 30: change authentication provider API to support mutual authentication
• PIP 29: One package for both pulsar-client and pulsar-admin
• PIP 28: Pulsar Proxy Gateway Improvement
• PIP 27: Add checklist in github pull request template
• PIP 26: Delayed Message Delivery
• PIP 25: Token based authentication
• PIP 24: Simplify memory settings
• PIP 23: Message Tracing By Interceptors
• PIP 22: Pulsar Dead Letter Topic
• PIP 20: Mechanism to revoke TLS authentication
• PIP 19: Pulsar SQL
• PIP 17: Tiered storage for Pulsar topics
• PIP 16: Pulsar "instance" terminology change
• PIP 15: Pulsar Functions
• PIP 14: Topic compaction
• PIP 13: Subscribe to topics represented by regular expressions
• PIP 12: Introduce builder for creating Producer Consumer Reader
• PIP 11: Short topic names
• PIP 10: Remove cluster for namespace and topic names
• PIP 9: Adding more Security checks to Pulsar Proxy
• PIP 8: Pulsar beyond 1M topics
• PIP 7: Pulsar Failure domain and Anti affinity namespaces
• PIP 6: Guaranteed Message Deduplication
• PIP 5: Event time
• PIP 4: Pulsar End to End Encryption
• PIP 3: Message dispatch throttling
• PIP 2: Non Persistent topic
• PIP 1: Pulsar Proxy

Discarded

• PIP 18: Pulsar Replicator
• PIP 94: Message converter at broker level

Contributor documentation

• Community Meetings
• Release Candidate Validation
• Pulsar Improvement Proposal (PIP)
• Pulsar Improvement Proposal Template

Committers documentation

• Committer Guide
• Create GPG keys to sign release artifacts
• Release process
• Tools and Tips