雅虎的开源消息发布/订阅平台Pulsar

雅虎开源了发布/订阅消息平台Pulsar, 借助Pulsar可以向用户提供消息发布和订阅的功能，可运行在一个集群或多个集群上，并且能够通过API对其进行管理：添加/移除用户、添加计算和存储容量、核算和监控等等。Yahoo内部已经把Pulsar用在了多项服务上如邮件、财经、体育等频道。雅虎希望Pulsar能够与其他开源项目进行集成，促使Pulsar得到更广泛的应用。

源码下载：https://github.com/yahoo/pulsar

新闻来源：Yahoo! Open Sources Pulsar, a Pub/Sub Messaging Platform（https://www.infoq.com/news/2016/09/pulsar/）
-----------------------------------------------------------------------

Apache Pulsar - distributed pub-sub messaging system

pulsar.apache.org/ 

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.

Learn more about Pulsar at https://pulsar.apache.org

Main features

• Horizontally scalable (Millions of independent topics and millions of messages published per second)
• Strong ordering and consistency guarantees
• Low latency durable storage
• Topic and queue semantics
• Load balancer
• Designed for being deployed as a hosted service:
  • Multi-tenant
  • Authentication
  • Authorization
  • Quotas
  • Support mixing very different workloads
  • Optional hardware isolation
• Keeps track of consumer cursor position
• REST API for provisioning, admin and stats
• Geo replication
• Transparent handling of partitioned topics
• Transparent batching of messages

Repositories

This repository is the main repository of Apache Pulsar. Pulsar PMC also maintains other repositories for components in the Pulsar ecosystem, including connectors, adapters, and other language clients.

• Pulsar Core

Helm Chart

• Pulsar Helm Chart

Ecosystem

• Pulsar Adapters

Clients

• .NET/C# Client
• C++ Client
• Go Client
• NodeJS Client
• Python Client
• Reactive Java Client

Dashboard & Management Tools

• Pulsar Manager

Website

• Pulsar Site

CI/CD

• Pulsar CI

Archived/Halted

• Pulsar Connectors (bundled as pulsar-io)
• Pulsar Translation
• Pulsar SQL (Pulsar Presto Connector) (bundled as pulsar-sql)
• Ruby Client

Pulsar Runtime Java Version Recommendation

• pulsar ver > 2.10 and master branch

Components	Java Version
Broker	17
Functions / IO	17
CLI	17
Java Client	8 or 11 or 17

• 2.8 <= pulsar ver <= 2.10

Components	Java Version
Broker	11
Functions / IO	11
CLI	8 or 11
Java Client	8 or 11

• pulsar ver < 2.8

Components	Java Version
All	8 or 11

Build Pulsar

Requirements

• JDK

  Pulsar Version	JDK Version
  master and 2.11 +	JDK 17
  2.8 / 2.9 / 2.10	JDK 11
  2.7 -	JDK 8

• Maven 3.6.1+

• zip

Note:

This project includes a Maven Wrapper that can be used instead of a system-installed Maven. Use it by replacing mvn by ./mvnw on Linux and mvnw.cmd on Windows in the commands below.

It's better to use CMD rather than Powershell on Windows. Because maven will activate the windows profile which runs rename-netty-native-libs.cmd.

Build

Compile and install:

$ mvn install -DskipTests

Compile and install individual module

$ mvn -pl module-name (e.g: pulsar-broker) install -DskipTests

Minimal build (This skips most of external connectors and tiered storage handlers)

mvn install -Pcore-modules,-main -DskipTests

Run Unit Tests:

$ mvn test

Run Individual Unit Test:

$ mvn -pl module-name (e.g: pulsar-client) test -Dtest=unit-test-name (e.g: ConsumerBuilderImplTest)

Run Selected Test packages:

$ mvn test -pl module-name (for example, pulsar-broker) -Dinclude=org/apache/pulsar/**/*.java

Start standalone Pulsar service:

$ bin/pulsar standalone

Check https://pulsar.apache.org for documentation and examples.

Setting up your IDE

Read https://pulsar.apache.org/contribute/setup-ide for setting up IntelliJ IDEA or Eclipse for developing Pulsar.

Documentation

Note:

For how to make contributions to Pulsar documentation, see Pulsar Documentation Contribution Guide.

Contact

Mailing lists

Name	Scope	Subscribe	Unsubscribe	Archives
users@pulsar.apache.org	User-related discussions	Subscribe	Unsubscribe	Archives
dev@pulsar.apache.org	Development-related discussions	Subscribe	Unsubscribe	Archives

Slack

Pulsar slack channel at https://apache-pulsar.slack.com/

You can self-register at https://communityinviter.com/apps/apache-pulsar/apache-pulsar

Report a security vulnerability

To report a vulnerability for Pulsar, contact the Apache Security Team. When reporting a vulnerability to security@apache.org, you can copy your email to private@pulsar.apache.org to send your report to the Apache Pulsar Project Management Committee. This is a private mailing list.

https://github.com/apache/pulsar/security/policy contains more details.

Crypto Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See The Wassenaar Arrangement for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software: Pulsar uses the SSL library from Bouncy Castle written by http://www.bouncycastle.org.

from https://github.com/apache/pulsar