0.9.8.10
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
Public Member Functions | Private Types | Private Member Functions | Private Attributes | List of all members
Hypertable::TableInfoMap Class Reference

Manages live range map and set of log names that can be safely removed. More...

#include <TableInfoMap.h>

Collaboration diagram for Hypertable::TableInfoMap:
Collaboration graph
[legend]

Public Member Functions

 TableInfoMap ()
 Constructor. More...
 
 TableInfoMap (HyperspaceTableCachePtr schema_cache)
 Constructor with HyperspaceTableCache. More...
 
virtual ~TableInfoMap ()
 Destructor. More...
 
bool lookup (const String &table_id, TableInfoPtr &info)
 Returns the TableInfo object for a given table. More...
 
void get (const String &table_id, TableInfoPtr &info)
 Gets the TableInfo object for a table, creating one if not found. More...
 
void promote_staged_range (const TableIdentifier &table, RangePtr &range, const char *transfer_log)
 Adds a staged range. More...
 
bool remove (const String &table_id, TableInfoPtr &info)
 Removes a table from the map. More...
 
void get_all (std::vector< TableInfoPtr > &tv)
 Gets all TableInfo objects in map. More...
 
void get_ranges (Ranges &ranges, StringSet *remove_ok_logs=0)
 Gets set of live RangeData objects and corresponding transfer logs that can be safely removed. More...
 
void clear ()
 Clears the map. More...
 
bool empty ()
 Determines if map is empty. More...
 
void merge (TableInfoMap *other)
 Merges in another map. More...
 
void merge (TableInfoMap *other, vector< MetaLog::EntityPtr > &entities, StringSet &transfer_logs)
 Merges in another map and updates RMSL with merged entities. More...
 

Private Types

typedef std::map< String,
TableInfoPtr
InfoMap
 table_id-to-TableInfoPtr map type More...
 

Private Member Functions

void merge_unlocked (TableInfoMap *other)
 Merges in another map (without locking mutex). More...
 

Private Attributes

std::mutex m_mutex
 Mutex for serializing access to members More...
 
HyperspaceTableCachePtr m_schema_cache
 Hyperspace table cache More...
 
InfoMap m_map
 table_id-to-TableInfoPtr map More...
 

Detailed Description

Manages live range map and set of log names that can be safely removed.

This class is used to maintain an active set of ranges, organized by table ID. It maps table IDs to TableInfo objects holding a set of active Range objects. It also manages a set of transfer log names that can be safely removed from the system. The maintenance scheduler is responsible for removing commit log fragments (including linked transfer logs). It does this by obtaining the set of live ranges, computing some statistics for each range and then removing the log fragments that contain no data that hasn't been compacted. However, loading a range is a two step process that introduces a race condition: 1) the range's transfer log is linked into the commit log and 2) the range is persisted to the RSML. If the maintenance scheduler does its commit log purging work between steps #1 and #2, it' possible for a newly added transfer log to get removed before the Range has been fully loaded. To avoid this race condition we've introduced MetaLogEntityRemoveOkLogs which contains the set of transfer logs that have been linked in and which may be safely removed. The MetaLogEntityRemoveOkLogs entity is modified and persisted atomically to the RSML with new Range entities by the promote_staged_range() method. The get_ranges() method returns a consistent snapshot of the current set of live ranges and the set of transfer logs in MetaLogEntityRemoveOkLogs.

Definition at line 67 of file TableInfoMap.h.

Member Typedef Documentation

table_id-to-TableInfoPtr map type

Definition at line 210 of file TableInfoMap.h.

Constructor & Destructor Documentation

Hypertable::TableInfoMap::TableInfoMap ( )
inline

Constructor.

Definition at line 71 of file TableInfoMap.h.

Hypertable::TableInfoMap::TableInfoMap ( HyperspaceTableCachePtr  schema_cache)
inline

Constructor with HyperspaceTableCache.

Objects constructed with this constructor will use the provided schema cache to lookup schemas instead of reading them from Hyperspace.

Parameters
schema_cacheTable schema cache

Definition at line 79 of file TableInfoMap.h.

TableInfoMap::~TableInfoMap ( )
virtual

Destructor.

Definition at line 42 of file TableInfoMap.cc.

Member Function Documentation

void TableInfoMap::clear ( )

Clears the map.

Definition at line 191 of file TableInfoMap.cc.

bool TableInfoMap::empty ( )

Determines if map is empty.

Returns
true if map is empty, false otherwise

Definition at line 196 of file TableInfoMap.cc.

void TableInfoMap::get ( const String table_id,
TableInfoPtr info 
)

Gets the TableInfo object for a table, creating one if not found.

Parameters
table_idTable identifier string
infoOutput parameter to hold TableInfo object
Exceptions
Exceptionwith code set to Error::RANGESERVER_SCHEMA_PARSE_ERROR

Definition at line 56 of file TableInfoMap.cc.

void TableInfoMap::get_all ( std::vector< TableInfoPtr > &  tv)

Gets all TableInfo objects in map.

Parameters
tvOutput vector to hold TableInfo objects

Definition at line 163 of file TableInfoMap.cc.

void TableInfoMap::get_ranges ( Ranges ranges,
StringSet remove_ok_logs = 0 
)

Gets set of live RangeData objects and corresponding transfer logs that can be safely removed.

This method is used to fetch all of the live range objects in the map. The maintenance scheduler is responsible for removing commit log fragments (including linked transfer logs). It does this by obtaining the set of live ranges, computing some statistics for each range and then removing the log fragments that contain no data that hasn't been compacted. However, loading a range is a two step process that introduces a race condition: 1) the range's transfer log is linked into the commit log and 2) the range is persisted to the RSML. If the maintenance scheduler does its commit log purging work between steps #1 and #2, it' possible for a newly added transfer log to get removed before the Range has been fully loaded. To avoid this race condition we've introduced MetaLogEntityRemoveOkLogs which contains the set of transfer logs that have been linked in and which may be safely removed. The MetaLogEntityRemoveOkLogs entity is modified and persisted atomically to the RSML with new Range entities by the promote_staged_range() method. This method returns a consistent snapshot of the current set of live ranges and the set of transfer logs in MetaLogEntityRemoveOkLogs.

Parameters
rangesOutput parameter to hold RangeData objects
remove_ok_logsPointer to string set to hold logs that can be removed
See also
promote_staged_range

Definition at line 170 of file TableInfoMap.cc.

bool TableInfoMap::lookup ( const String table_id,
TableInfoPtr info 
)

Returns the TableInfo object for a given table.

Parameters
table_idTable identifier string
infoOutput parameter to hold TableInfo object
Returns
true if mapping found for table_id, false otherwise.

Definition at line 47 of file TableInfoMap.cc.

void TableInfoMap::merge ( TableInfoMap other)

Merges in another map.

This method locks m_mutex and calls merge_unlocked(). It should only be called for ranges that have already been persisted into the RSML from a previous call to promote_staged_range() such as during the loading of ranges from the RSML at system startup.

Parameters
otherMap to merge in

Definition at line 202 of file TableInfoMap.cc.

void TableInfoMap::merge ( TableInfoMap other,
vector< MetaLog::EntityPtr > &  entities,
StringSet transfer_logs 
)

Merges in another map and updates RMSL with merged entities.

This method atomically adds transfer_logs to Global::remove_ok_logs, persists entities and Global::remove_ok_logs to the RSML, and merges in another map. entities and transfer_logs should correspond to the ranges in other that are to be merged in. This method performs these steps atomically (with m_mutex locked) to avoid the race condition described in get_ranges()

Parameters
otherMap to merge in
entitiesVector of range entities
transfer_logsSet of transfer logs
See also
get_ranges

Definition at line 207 of file TableInfoMap.cc.

void TableInfoMap::merge_unlocked ( TableInfoMap other)
private

Merges in another map (without locking mutex).

For each TableInfo object in other, if there is no corresponding TableInfo object in this object's map, the TableInfo object is directly inserted into this object's map (m_map). If there is already a TableInfo object in this object's map, then all of the RangeData objects are fetched from the other TableInfo object and inserted into this object's TableInfo object. At the end of the method other is cleared.

Parameters
otherMap to merge in

Definition at line 221 of file TableInfoMap.cc.

void TableInfoMap::promote_staged_range ( const TableIdentifier table,
RangePtr range,
const char *  transfer_log 
)

Adds a staged range.

When staging is complete for a range, the Range object is added with a call to this method. Since a range's transfer log cannot be removed until the range has been added, this method adds the range and adds the range's tranfer log to the the global MetaLogEntityRemoveOkLogs object (Global::remove_ok_logs) in one atomic transaction. This method performs the following actions with m_mutex locked:

Parameters
tableTable identifier
rangeRange object
transfer_logTransfer log for range
See also
stage_range

Definition at line 102 of file TableInfoMap.cc.

bool TableInfoMap::remove ( const String table_id,
TableInfoPtr info 
)

Removes a table from the map.

Parameters
table_idTable identifier string
infoOutput parameter to hold removed TableInfo object
Returns
true if table exists in map and is removed, false otherwise.

Definition at line 152 of file TableInfoMap.cc.

Member Data Documentation

InfoMap Hypertable::TableInfoMap::m_map
private

table_id-to-TableInfoPtr map

Definition at line 219 of file TableInfoMap.h.

std::mutex Hypertable::TableInfoMap::m_mutex
private

Mutex for serializing access to members

Definition at line 213 of file TableInfoMap.h.

HyperspaceTableCachePtr Hypertable::TableInfoMap::m_schema_cache
private

Hyperspace table cache

Definition at line 216 of file TableInfoMap.h.


The documentation for this class was generated from the following files: