0.9.8.10
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
Classes | Public Member Functions | Private Types | Private Attributes | List of all members
Hypertable::ApplicationQueue Class Reference

Application queue. More...

#include <ApplicationQueue.h>

Inheritance diagram for Hypertable::ApplicationQueue:
Inheritance graph
[legend]
Collaboration diagram for Hypertable::ApplicationQueue:
Collaboration graph
[legend]

Classes

class  ApplicationQueueState
 Application queue state shared among worker threads. More...
 
class  GroupState
 Tracks group execution state. More...
 
class  RequestRec
 Request record. More...
 
class  Worker
 Application queue worker thread function (functor) More...
 

Public Member Functions

 ApplicationQueue ()
 Default constructor used by derived classes only. More...
 
 ApplicationQueue (int worker_count, bool dynamic_threads=true)
 Constructor initialized with worker thread count. More...
 
virtual ~ApplicationQueue ()
 Destructor. More...
 
std::vector< Thread::id > get_thread_ids () const
 Returns all the thread IDs for this threadgroup. More...
 
void shutdown ()
 Shuts down the application queue. More...
 
bool wait_for_idle (std::chrono::time_point< std::chrono::steady_clock > deadline, int reserve_threads=0)
 Wait for queue to become idle (with timeout). More...
 
void join ()
 Waits for a shutdown to complete. More...
 
void start ()
 Starts application queue. More...
 
void stop ()
 Stops (pauses) application queue, preventing non-urgent requests from being executed. More...
 
virtual void add (ApplicationHandler *app_handler)
 Adds a request (application request handler) to the application queue. More...
 
virtual void add_unlocked (ApplicationHandler *app_handler)
 Adds a request (application request handler) to the application queue. More...
 
size_t backlog ()
 Returns the request backlog Returns the request backlog, which is the number of requests waiting on the request queues for a thread to become available. More...
 

Private Types

typedef std::unordered_map
< uint64_t, GroupState * > 
GroupStateMap
 Hash map of thread group ID to GroupState. More...
 
typedef std::list< RequestRec * > RequestQueue
 Individual request queue. More...
 

Private Attributes

ApplicationQueueState m_state
 Application queue state object. More...
 
ThreadGroup m_threads
 Boost thread group for managing threads. More...
 
std::vector< Thread::id > m_thread_ids
 Vector of thread IDs. More...
 
bool joined
 Flag indicating if threads have joined after a shutdown. More...
 
bool m_dynamic_threads
 Set to true if queue is configured to allow dynamic thread creation. More...
 

Detailed Description

Application queue.

Helper class for use by server applications that are driven by messages received over the network. This class can be used in conjunction with the ApplicationHandler class to implement an incoming request queue. Worker threads pull handlers (requests) off the queue and carry them out. The following features are supported:

Groups

Because a set of worker threads pull requests from the queue and carry them out independently, it is possible for requests to get executed out of order relative to the order in which they arrived in the queue. This can cause problems for certain request sequences such as appending data to a file in the FsBroker, or fetching scanner results from a scanner using multiple readahead requests. Groups are a way to give applications the ability to serialize a set of requests. Each request has a group ID that is returned by the ApplicationHandler::get_group_id method. Requests that have the same group ID will get executed in series, in the order in which they arrived in the application queue. Requests with group ID 0 don't belong to any group and will get executed independently with no serialization order.

Prioritization

The ApplicationQueue supports two-level request prioritization. Requests can be designated as urgent which will cause them to be executed before other non-urgent requests. Urgent requests will also be executed even when the ApplicationQueue has been paused. In Hypertable, METADATA scans and updates are marked urgent which allows them procede and prevent deadlocks when the application queue gets paused due to low memory condition in the RangeServer. The ApplicationHandler::is_urgent method is used to signal if a request is urgent.

Definition at line 89 of file ApplicationQueue.h.

Member Typedef Documentation

typedef std::unordered_map<uint64_t, GroupState *> Hypertable::ApplicationQueue::GroupStateMap
private

Hash map of thread group ID to GroupState.

Definition at line 107 of file ApplicationQueue.h.

Individual request queue.

Definition at line 121 of file ApplicationQueue.h.

Constructor & Destructor Documentation

Hypertable::ApplicationQueue::ApplicationQueue ( )
inline

Default constructor used by derived classes only.

Definition at line 331 of file ApplicationQueue.h.

Hypertable::ApplicationQueue::ApplicationQueue ( int  worker_count,
bool  dynamic_threads = true 
)
inline

Constructor initialized with worker thread count.

This constructor sets up the application queue with a number of worker threads specified by worker_count.

Parameters
worker_countNumber of worker threads to create
dynamic_threadsDynamically create temporary thread to carry out requests if none available.

Definition at line 341 of file ApplicationQueue.h.

virtual Hypertable::ApplicationQueue::~ApplicationQueue ( )
inlinevirtual

Destructor.

Definition at line 354 of file ApplicationQueue.h.

Member Function Documentation

virtual void Hypertable::ApplicationQueue::add ( ApplicationHandler app_handler)
inlinevirtual

Adds a request (application request handler) to the application queue.

The request queue is designed to support the serialization of related requests. Requests are related by the thread group ID value in the ApplicationHandler. This thread group ID is constructed in the Event object.

Parameters
app_handlerPointer to request to add

Implements Hypertable::ApplicationQueueInterface.

Definition at line 428 of file ApplicationQueue.h.

virtual void Hypertable::ApplicationQueue::add_unlocked ( ApplicationHandler app_handler)
inlinevirtual

Adds a request (application request handler) to the application queue.

The request queue is designed to support the serialization of related requests. Requests are related by the thread group ID value in the ApplicationHandler. This thread group ID is constructed in the Event object.

Note
This method is defined for symmetry and just calls add
Parameters
app_handlerPointer to request to add

Implements Hypertable::ApplicationQueueInterface.

Definition at line 473 of file ApplicationQueue.h.

size_t Hypertable::ApplicationQueue::backlog ( )
inline

Returns the request backlog Returns the request backlog, which is the number of requests waiting on the request queues for a thread to become available.

Returns
Request backlog

Definition at line 481 of file ApplicationQueue.h.

std::vector<Thread::id> Hypertable::ApplicationQueue::get_thread_ids ( ) const
inline

Returns all the thread IDs for this threadgroup.

Returns
vector of Thread::id

Definition at line 365 of file ApplicationQueue.h.

void Hypertable::ApplicationQueue::join ( )
inline

Waits for a shutdown to complete.

This method returns when all application queue threads exit.

Definition at line 397 of file ApplicationQueue.h.

void Hypertable::ApplicationQueue::shutdown ( )
inline

Shuts down the application queue.

All outstanding requests are carried out and then all threads exit. join can be called to wait for completion of the shutdown.

Definition at line 374 of file ApplicationQueue.h.

void Hypertable::ApplicationQueue::start ( )
inline

Starts application queue.

Definition at line 406 of file ApplicationQueue.h.

void Hypertable::ApplicationQueue::stop ( )
inline

Stops (pauses) application queue, preventing non-urgent requests from being executed.

Any requests that are being executed at the time of the call are allowed to complete.

Definition at line 416 of file ApplicationQueue.h.

bool Hypertable::ApplicationQueue::wait_for_idle ( std::chrono::time_point< std::chrono::steady_clock >  deadline,
int  reserve_threads = 0 
)
inline

Wait for queue to become idle (with timeout).

Parameters
deadlineReturn by this time if queue does not become idle
reserve_threadsNumber of threads that can be active when queue is idle
Returns
false if deadline was reached before queue became idle, true otherwise

Definition at line 386 of file ApplicationQueue.h.

Member Data Documentation

bool Hypertable::ApplicationQueue::joined
private

Flag indicating if threads have joined after a shutdown.

Definition at line 320 of file ApplicationQueue.h.

bool Hypertable::ApplicationQueue::m_dynamic_threads
private

Set to true if queue is configured to allow dynamic thread creation.

Definition at line 325 of file ApplicationQueue.h.

ApplicationQueueState Hypertable::ApplicationQueue::m_state
private

Application queue state object.

Definition at line 311 of file ApplicationQueue.h.

std::vector<Thread::id> Hypertable::ApplicationQueue::m_thread_ids
private

Vector of thread IDs.

Definition at line 317 of file ApplicationQueue.h.

ThreadGroup Hypertable::ApplicationQueue::m_threads
private

Boost thread group for managing threads.

Definition at line 314 of file ApplicationQueue.h.


The documentation for this class was generated from the following file: