Open Source Promotion Plan Summer 2020 Apache Pulsar Tasks

This page contains the tasks for program https://isrc.iscas.ac.cn/summer2020

Introduce

This program is hosted by ISCAS(Institute of Software Chinese Academy of Sciences) and openEuler community. It is very similar with Google Summer of Code. This year(2020) is their first time to host this event. Apache Pulsar has been selected as one program project.

Program home page, https://isrc.iscas.ac.cn/summer2020/#/

Apache Pulsar introduce

1. Project Name: Apache Pulsar

2. Website: http://pulsar.apache.org

3. Project Introduction:

   Apache Pulsar is a cloud-native, distributed messaging and streaming platform that manages hundreds of billions of events per day. Pulsar was originally developed and deployed inside Yahoo as the consolidated messaging platform connecting critical Yahoo applications such as Yahoo Finance, Yahoo Mail, and Flickr, to data. Pulsar was contributed to open source by Yahoo in 2016 and became a top-level Apache Software Foundation project in 2018. Apache Pulsar has been used in a lot of companies: https://pulsar.apache.org/en/powered-by/

4. Tags: Distributed system, messaging system, streaming platform, Big Data, Infrastructure.

Tasks

1. Pulsar .NET/C# client feature catch up:

1. Description: Recently community contributed .NET/C# client library for Apache Pulsar. But lack of some of the features in Pulsar main repo. This task is to catch up the missing features in .NET/C# client.
2. Level: Middle
3. Mentor: Jia Zhai (翟佳), email: [email protected]
4. Requirements: familiar with C# language.
5. OutPut:
   • complete related features catch up.
   • Add unit test for each feature.
   • PRs get reviewed and merged into repo.
6. Related Repo: https://github.com/apache/pulsar-dotpulsar

2. Schema independent from the topic

1. Description: Schema is widely used in data storage systems and Apache Pulsar uses Apache Avro to manage schemas such as schema definition, evolution, and data encoding. Currently, schemas are stored for each topic. In the scenario where the same schema is used for different topics, we store redundant schemas. This task is aimed to support the same schema that can be reused between different topics, make schema independent from the topic. And while being compatible with the existing architecture.
2. Level: Middle
3. Mentor: Penghui Li(李鹏辉), email: [email protected]
4. Requirements: familiar with Java language.
5. Output:
   • complete the feature
   • Add unit test for the feature
   • Add compatibility test
   • PRs get reviewed and merged into repo.

3. Read compacted topic in Pulsar SQL

1. Description: Compacted topic compact the history version message of the same message key. This is more like a row update in the database. Pulsar SQL can query data and analyze data from Pulsar topics. Currently, Pulsar SQL can't read data from the compacted topic, this task is to support reading data from the compacted topics in Pulsar SQL.
2. Level: Middle
3. Mentor: Ran Gao(高冉), email: [email protected]
4. Requirements: familiar with Java language.
5. Output:
   • complete the feature
   • Add unit test for the feature
   • PRs get reviewed and merged into repo.

4. Pulsar Go client feature catch up:

1. Description: Recently community contributed go client library for Apache Pulsar. But lack of some of the features in Pulsar main repo. This task is to catch up the missing features in go client.
2. Level: Middle
3. Mentor: Xiaolong Ran (冉小龙), email: [email protected]
4. Requirements: familiar with go language.
5. OutPut:
   • complete related features catch up.
     • Support key-based batcher for key_shared (https://github.com/apache/pulsar-client-go/issues/172)
     • Support multi pulsar cluster addresses (https://github.com/apache/pulsar-client-go/issues/177)
     • Check max message size in go client (https://github.com/apache/pulsar-client-go/issues/240)
     • Support schema for go client
   • Add unit test for each feature.
   • PRs get reviewed and merged into repo.
6. Related Repo: https://github.com/apache/pulsar-client-go

============= As the requirements from the Sponsor, we need also to provide a Chinese version.

#“开源软件供应链点亮计划——暑期2020”任务清单“ -- Apache Pulsar

1. 丰富 Pulsar .NET 客户端相关功能

1. 项目标题：丰富 Pulsar .NET 客户端相关功能
2. 项目描述：Pulsar .NET/C# 客户端是近期刚刚捐献给 Apache Pulsar 的一个新项目。现在 Pulsar .NET 客户端 已经具备了基础的 Pub/Sub 功能，但是缺少其他一些Java Client 具备的高级的功能。这个项目参与者可以和导师协商感兴趣的功能，理解现有的 Java 客户端的高级功能，并在 .NET 的client 端中进行相关的实现。这个项目为参与者提供消息系统的高级功能的深入理解。
3. 项目难度： 中
4. 项目导师： jiazhai (翟佳), email: [email protected]
5. 项目产出要求：

• 完成相关功能的实现。
• 完成单元测试和集成测试。
• 完成 PR 的发布，解决社区的 Review Comments，并合并到 Apache 代码库中。

6. 项目技能要求：

• .NET/C#

7. 相关开源软件仓库列表： https://github.com/apache/pulsar-dotpulsar