base/frameworks/cluster/pools.bro

Cluster

Defines an interface for managing pools of cluster nodes. Pools are a useful way to distribute work or data among nodes within a cluster.

Namespace:Cluster
Imports:base/frameworks/cluster/main.bro, base/utils/hash_hrw.bro
Source File:/scripts/base/frameworks/cluster/pools.bro

Summary

State Variables

Cluster::logger_pool: Cluster::Pool A pool containing all the logger nodes of a cluster.
Cluster::logger_pool_spec: Cluster::PoolSpec &redef The specification for Cluster::logger_pool.
Cluster::proxy_pool: Cluster::Pool A pool containing all the proxy nodes of a cluster.
Cluster::proxy_pool_spec: Cluster::PoolSpec &redef The specification for Cluster::proxy_pool.
Cluster::worker_pool: Cluster::Pool A pool containing all the worker nodes of a cluster.
Cluster::worker_pool_spec: Cluster::PoolSpec &redef The specification for Cluster::worker_pool.

Types

Cluster::PoolNode: record Store state of a cluster within the context of a work pool.
Cluster::PoolNodeTable: table  
Cluster::PoolSpec: record A pool specification.
Cluster::RoundRobinTable: table  

Functions

Cluster::hrw_topic: function Retrieve the topic associated with the node mapped via Rendezvous hash of an arbitrary key.
Cluster::register_pool: function Registers and initializes a pool.
Cluster::rr_log_topic: function Distributes log message topics among logger nodes via round-robin.
Cluster::rr_topic: function Retrieve the topic associated with the node in a round-robin fashion.

Detailed Interface

State Variables

Cluster::logger_pool
Type:Cluster::Pool
Default:
{
   spec=[topic="", node_type=Cluster::PROXY, max_nodes=<uninitialized>, exclusive=F]
   nodes={

   }
   node_list=[]
   hrw_pool=[sites={

   }]
   rr_key_seq={

   }
   alive_count=0
}

A pool containing all the logger nodes of a cluster. The pool’s node membership/availability is automatically maintained by the cluster framework.

Cluster::logger_pool_spec
Type:Cluster::PoolSpec
Attributes:&redef
Default:
{
   topic="bro/cluster/pool/logger"
   node_type=Cluster::LOGGER
   max_nodes=<uninitialized>
   exclusive=F
}

The specification for Cluster::logger_pool.

Cluster::proxy_pool
Type:Cluster::Pool
Default:
{
   spec=[topic="", node_type=Cluster::PROXY, max_nodes=<uninitialized>, exclusive=F]
   nodes={

   }
   node_list=[]
   hrw_pool=[sites={

   }]
   rr_key_seq={

   }
   alive_count=0
}

A pool containing all the proxy nodes of a cluster. The pool’s node membership/availability is automatically maintained by the cluster framework.

Cluster::proxy_pool_spec
Type:Cluster::PoolSpec
Attributes:&redef
Default:
{
   topic="bro/cluster/pool/proxy"
   node_type=Cluster::PROXY
   max_nodes=<uninitialized>
   exclusive=F
}

The specification for Cluster::proxy_pool.

Cluster::worker_pool
Type:Cluster::Pool
Default:
{
   spec=[topic="", node_type=Cluster::PROXY, max_nodes=<uninitialized>, exclusive=F]
   nodes={

   }
   node_list=[]
   hrw_pool=[sites={

   }]
   rr_key_seq={

   }
   alive_count=0
}

A pool containing all the worker nodes of a cluster. The pool’s node membership/availability is automatically maintained by the cluster framework.

Cluster::worker_pool_spec
Type:Cluster::PoolSpec
Attributes:&redef
Default:
{
   topic="bro/cluster/pool/worker"
   node_type=Cluster::WORKER
   max_nodes=<uninitialized>
   exclusive=F
}

The specification for Cluster::worker_pool.

Types

Cluster::PoolNode
Type:

record

name: string

The node name (e.g. “manager”).

alias: string

An alias of name used to prevent hashing collisions when creating site_id.

site_id: count

A 32-bit unique identifier for the pool node, derived from name/alias.

alive: bool &default = F &optional

Whether the node is currently alive and can receive work.

Store state of a cluster within the context of a work pool.

Cluster::PoolNodeTable
Type:table [string] of Cluster::PoolNode
Cluster::PoolSpec
Type:

record

topic: string &default = "" &optional

A topic string that can be used to reach all nodes within a pool.

node_type: Cluster::NodeType &default = Cluster::PROXY &optional

The type of nodes that are contained within the pool.

max_nodes: count &optional

The maximum number of nodes that may belong to the pool. If not set, then all available nodes will be added to the pool, else the cluster framework will automatically limit the pool membership according to the threshhold.

exclusive: bool &default = F &optional

Whether the pool requires exclusive access to nodes. If true, then max_nodes nodes will not be assigned to any other pool. When using this flag, max_nodes must also be set.

A pool specification.

Cluster::RoundRobinTable
Type:table [string] of int

Functions

Cluster::hrw_topic
Type:function (pool: Cluster::Pool, key: any) : string

Retrieve the topic associated with the node mapped via Rendezvous hash of an arbitrary key.

Pool:the pool of nodes to consider.
Key:data used for input to the hashing function that will uniformly distribute keys among available nodes.
Returns:a topic string associated with a cluster node that is alive or an empty string if nothing is alive.
Cluster::register_pool
Type:function (spec: Cluster::PoolSpec) : Cluster::Pool

Registers and initializes a pool.

Cluster::rr_log_topic
Type:function (id: Log::ID, path: string) : string

Distributes log message topics among logger nodes via round-robin. This will be automatically assigned to Broker::log_topic if Cluster::enable_round_robin_logging is enabled. If no logger nodes are active, then this will return the value of Broker::default_log_topic.

Cluster::rr_topic
Type:function (pool: Cluster::Pool, key: string &default = "" &optional) : string

Retrieve the topic associated with the node in a round-robin fashion.

Pool:the pool of nodes to consider.
Key:an arbitrary string to identify the purpose for which you’re requesting the topic. e.g. consider using a name-spaced key like “Intel::cluster_rr_key” if you need to guarantee that a group of messages get distributed in a well-defined pattern without other messages being interleaved within the round-robin. Usually sharing the default key is fine for load-balancing purposes.
Returns:a topic string associated with a cluster node that is alive, or an empty string if nothing is alive.
Copyright 2016, The Bro Project. Last updated on October 12, 2018. Created using Sphinx 1.7.5.