Skip to main content
Version: 2.8.0

Managing Namespaces

Pulsar namespaces are logical groupings of topics.

Namespaces can be managed via:

Namespaces resources#

Create namespaces#

You can create new namespaces under a given tenant.

Use the create subcommand and specify the namespace by name:


$ pulsar-admin namespaces create test-tenant/test-namespace

Get policies#

You can fetch the current policies associated with a namespace at any time.

Use the policies subcommand and specify the namespace:


$ pulsar-admin namespaces policies test-tenant/test-namespace{  "auth_policies": {    "namespace_auth": {},    "destination_auth": {}  },  "replication_clusters": [],  "bundles_activated": true,  "bundles": {    "boundaries": [      "0x00000000",      "0xffffffff"    ],    "numBundles": 1  },  "backlog_quota_map": {},  "persistence": null,  "latency_stats_sample_rate": {},  "message_ttl_in_seconds": 0,  "retention_policies": null,  "deleted": false}

List namespaces#

You can list all namespaces within a given Pulsar tenant.

Use the list subcommand and specify the tenant:


$ pulsar-admin namespaces list test-tenanttest-tenant/ns1test-tenant/ns2

Delete namespaces#

You can delete existing namespaces from a tenant.

Use the delete subcommand and specify the namespace:


$ pulsar-admin namespaces delete test-tenant/ns1

Configure replication clusters#

Set replication cluster#

It sets replication clusters for a namespace, so Pulsar can internally replicate publish message from one colo to another colo.

$ pulsar-admin namespaces set-clusters test-tenant/ns1 \  --clusters cl1

Get replication cluster#

It gives a list of replication clusters for a given namespace.

$ pulsar-admin namespaces get-clusters test-tenant/cl1/ns1

cl2

Configure backlog quota policies#

Set backlog quota policies#

Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take corresponding action after the limit is reached.

  1. producer_request_hold: broker will hold and not persist produce request payload

  2. producer_exception: broker disconnects with the client by giving an exception.

  3. consumer_backlog_eviction: broker will start discarding backlog messages

    Backlog quota restriction can be taken care by defining restriction of backlog-quota-type: destination_storage

$ pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --policy producer_request_hold test-tenant/ns1

N/A

Get backlog quota policies#

It shows a configured backlog quota for a given namespace.

$ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1


{  "destination_storage": {    "limit": 10,    "policy": "producer_request_hold"  }}

Remove backlog quota policies#

It removes backlog quota policies for a given namespace

$ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1

N/A

Configure persistence policies#

Set persistence policies#

Persistence policies allow to configure persistency-level for all topic messages under a given namespace.

  • Bookkeeper-ack-quorum: Number of acks (guaranteed copies) to wait for each entry, default: 0

  • Bookkeeper-ensemble: Number of bookies to use for a topic, default: 0

  • Bookkeeper-write-quorum: How many writes to make of each entry, default: 0

  • Ml-mark-delete-max-rate: Throttling rate of mark-delete operation (0 means no throttle), default: 0.0

$ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1

N/A

Get persistence policies#

It shows the configured persistence policies of a given namespace.

$ pulsar-admin namespaces get-persistence test-tenant/ns1


{  "bookkeeperEnsemble": 3,  "bookkeeperWriteQuorum": 2,  "bookkeeperAckQuorum": 2,  "managedLedgerMaxMarkDeleteRate": 0}

Configure namespace bundles#

Unload namespace bundles#

The namespace bundle is a virtual group of topics which belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.

$ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1

N/A

Split namespace bundles#

Each namespace bundle can contain multiple topics and each bundle can be served by only one broker. If a single bundle is creating an excessive load on a broker, an admin splits the bundle using this command permitting one or more of the new bundles to be unloaded thus spreading the load across the brokers.

$ pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1

N/A

Configure message TTL#

Set message-ttl#

It configures message’s time to live (in seconds) duration.

$ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1

N/A

Get message-ttl#

It gives a message ttl of configured namespace.

$ pulsar-admin namespaces get-message-ttl test-tenant/ns1

100

Remove message-ttl#

Remove a message TTL of the configured namespace.

$ pulsar-admin namespaces remove-message-ttl test-tenant/ns1

100

Clear backlog#

Clear namespace backlog#

It clears all message backlog for all the topics that belong to a specific namespace. You can also clear backlog for a specific subscription as well.

$ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1

N/A

Clear bundle backlog#

It clears all message backlog for all the topics that belong to a specific NamespaceBundle. You can also clear backlog for a specific subscription as well.

$ pulsar-admin namespaces clear-backlog  --bundle 0x00000000_0xffffffff  --sub my-subscription test-tenant/ns1

N/A

Configure retention#

Set retention#

Each namespace contains multiple topics and the retention size (storage size) of each topic should not exceed a specific threshold or it should be stored for a certain period. This command helps configure the retention size and time of topics in a given namespace.

$ pulsar-admin set-retention --size 100 --time 10 test-tenant/ns1

N/A

Get retention#

It shows retention information of a given namespace.

$ pulsar-admin namespaces get-retention test-tenant/ns1


{  "retentionTimeInMinutes": 10,  "retentionSizeInMB": 100}

Configure dispatch throttling for topics#

Set dispatch throttling for topics#

It sets message dispatch rate for all the topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.

note
  • If neither clusterDispatchRate nor topicDispatchRate is configured, dispatch throttling is disabled.
  • If topicDispatchRate is not configured, clusterDispatchRate takes effect.
  • If topicDispatchRate is configured, topicDispatchRate takes effect.
$ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \  --msg-dispatch-rate 1000 \  --byte-dispatch-rate 1048576 \  --dispatch-rate-period 1

Get configured message-rate for topics#

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)

$ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1


{  "dispatchThrottlingRatePerTopicInMsg" : 1000,  "dispatchThrottlingRatePerTopicInByte" : 1048576,  "ratePeriodInSecond" : 1}

Configure dispatch throttling for subscription#

Set dispatch throttling for subscription#

It sets message dispatch rate for all the subscription of topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.

$ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \  --msg-dispatch-rate 1000 \  --byte-dispatch-rate 1048576 \  --dispatch-rate-period 1

Get configured message-rate for subscription#

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)

$ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1


{  "dispatchThrottlingRatePerTopicInMsg" : 1000,  "dispatchThrottlingRatePerTopicInByte" : 1048576,  "ratePeriodInSecond" : 1}

Configure dispatch throttling for replicator#

Set dispatch throttling for replicator#

It sets message dispatch rate for all the replicator between replication clusters under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.

$ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \  --msg-dispatch-rate 1000 \  --byte-dispatch-rate 1048576 \  --dispatch-rate-period 1

Get configured message-rate for replicator#

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)

$ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1


{  "dispatchThrottlingRatePerTopicInMsg" : 1000,  "dispatchThrottlingRatePerTopicInByte" : 1048576,  "ratePeriodInSecond" : 1}

Configure deduplication snapshot interval#

Get deduplication snapshot interval#

It shows configured deduplicationSnapshotInterval for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)

$ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1

Set deduplication snapshot interval#

Set configured deduplicationSnapshotInterval for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval. brokerDeduplicationEnabled must be set to true for this property to take effect.

$ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000

Remove deduplication snapshot interval#

Remove configured deduplicationSnapshotInterval of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)

$ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1

Namespace isolation#

You can use the Pulsar isolation policy to allocate resources (broker and bookie) for a namespace.

Unload namespaces from a broker#

You can unload a namespace, or a namespace bundle, from the Pulsar broker that is currently responsible for it.

pulsar-admin#

Use the unload subcommand of the namespaces command.


$ pulsar-admin namespaces unload my-tenant/my-ns