Tkrzw
Classes | Public Member Functions | Static Public Member Functions | List of all members
tkrzw::ShardDBM Class Referencefinal

Sharding database manager adapter. More...

#include <tkrzw_dbm_shard.h>

Classes

class  Iterator
 Iterator for each record. More...
 

Public Member Functions

 ShardDBM ()
 Default constructor. More...
 
virtual ~ShardDBM ()
 Destructor. More...
 
Status Open (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT) override
 Opens a database file. More...
 
Status OpenAdvanced (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT, const std::map< std::string, std::string > &params={}) override
 Opens a database file, in an advanced way. More...
 
Status Close () override
 Closes the database file. More...
 
Status Process (std::string_view key, RecordProcessor *proc, bool writable) override
 Processes a record with a processor. More...
 
Status Get (std::string_view key, std::string *value=nullptr) override
 Gets the value of a record of a key. More...
 
Status Set (std::string_view key, std::string_view value, bool overwrite=true, std::string *old_value=nullptr) override
 Sets a record of a key and a value. More...
 
Status Remove (std::string_view key, std::string *old_value=nullptr) override
 Removes a record of a key. More...
 
Status Append (std::string_view key, std::string_view value, std::string_view delim="") override
 Appends data at the end of a record of a key. More...
 
Status ProcessEach (RecordProcessor *proc, bool writable) override
 Processes each and every record in the database with a processor. More...
 
Status Count (int64_t *count) override
 Gets the number of records. More...
 
Status GetFileSize (int64_t *size) override
 Gets the current file size of the database. More...
 
Status GetFilePath (std::string *path) override
 Gets the path of the database file. More...
 
Status Clear () override
 Removes all records. More...
 
Status Rebuild () override
 Rebuilds the entire database. More...
 
Status RebuildAdvanced (const std::map< std::string, std::string > &params={}) override
 Rebuilds the entire database, in an advanced way. More...
 
Status ShouldBeRebuilt (bool *tobe) override
 Checks whether the database should be rebuilt. More...
 
Status Synchronize (bool hard, FileProcessor *proc=nullptr) override
 Synchronizes the content of the database to the file system. More...
 
Status SynchronizeAdvanced (bool hard, FileProcessor *proc=nullptr, const std::map< std::string, std::string > &params={}) override
 Synchronizes the content of the database to the file system, in an advanced way. More...
 
Status CopyFile (const std::string &dest_path) override
 Copies the content of the database files to other files. More...
 
std::vector< std::pair< std::string, std::string > > Inspect () override
 Inspects the database. More...
 
bool IsOpen () const override
 Checks whether the database is open. More...
 
bool IsWritable () const override
 Checks whether the database is writable. More...
 
bool IsHealthy () const override
 Checks whether the database condition is healthy. More...
 
bool IsOrdered () const override
 Checks whether ordered operations are supported. More...
 
std::unique_ptr< DBM::IteratorMakeIterator () override
 Makes an iterator for each record. More...
 
std::unique_ptr< DBMMakeDBM () const override
 Make a new DBM object of the same concrete class. More...
 
DBMGetInternalDBM () const
 Gets the pointer to the first internal database object. More...
 
- Public Member Functions inherited from tkrzw::ParamDBM
virtual ~ParamDBM ()=default
 Destructor. More...
 
- Public Member Functions inherited from tkrzw::DBM
virtual ~DBM ()=default
 Destructor. More...
 
virtual Status Process (std::string_view key, RecordLambdaType rec_lambda, bool writable)
 Processes a record with a lambda function. More...
 
virtual std::string GetSimple (std::string_view key, std::string_view default_value="")
 Gets the value of a record of a key, in a simple way. More...
 
virtual std::map< std::string, std::string > GetMulti (const std::initializer_list< std::string > &keys)
 Gets the values of multiple records of keys. More...
 
virtual std::map< std::string, std::string > GetMulti (const std::vector< std::string > &keys)
 Gets the values of multiple records of keys, with a vector. More...
 
virtual Status SetMulti (const std::initializer_list< std::pair< std::string, std::string >> &records, bool overwrite=true)
 Sets multiple records. More...
 
virtual Status SetMulti (const std::map< std::string, std::string > &records, bool overwrite=true)
 Sets multiple records, with a map of strings. More...
 
virtual Status CompareExchange (std::string_view key, std::string_view expected, std::string_view desired, std::string *actual=nullptr)
 Compares the value of a record and exchanges if the condition meets. More...
 
virtual Status Increment (std::string_view key, int64_t increment=1, int64_t *current=nullptr, int64_t initial=0)
 Increments the numeric value of a record. More...
 
int64_t IncrementSimple (std::string_view key, int64_t increment=1, int64_t initial=0)
 Increments the numeric value of a record, in a simple way. More...
 
virtual Status ProcessEach (RecordLambdaType rec_lambda, bool writable)
 Processes each and every record in the database with a lambda function. More...
 
virtual int64_t CountSimple ()
 Gets the number of records, in a simple way. More...
 
virtual int64_t GetFileSizeSimple ()
 Gets the current file size of the database, in a simple way. More...
 
virtual std::string GetFilePathSimple ()
 Gets the path of the database file, in a simple way. More...
 
virtual bool ShouldBeRebuiltSimple ()
 Checks whether the database should be rebuilt, in a simple way. More...
 
virtual Status Export (DBM *dbm)
 Exports all records to another database. More...
 

Static Public Member Functions

static Status RestoreDatabase (const std::string &old_file_path, const std::string &new_file_path, const std::string &class_name="", int64_t end_offset=-1)
 Restores a broken database as a new healthy database. More...
 

Additional Inherited Members

- Public Types inherited from tkrzw::DBM
typedef std::function< std::string_view(std::string_view, std::string_view)> RecordLambdaType
 Lambda function type to process a record. More...
 

Detailed Description

Sharding database manager adapter.

All operations except for Open and Close are thread-safe; Multiple threads can access the same database concurrently. Every opened database must be closed explicitly to avoid data corruption.

This class is a wrapper of PolyDBM for sharding the database into multiple instances.

Constructor & Destructor Documentation

◆ ShardDBM()

tkrzw::ShardDBM::ShardDBM ( )
explicit

Default constructor.

◆ ~ShardDBM()

virtual tkrzw::ShardDBM::~ShardDBM ( )
virtual

Destructor.

Member Function Documentation

◆ Open()

Status tkrzw::ShardDBM::Open ( const std::string &  path,
bool  writable,
int32_t  options = File::OPEN_DEFAULT 
)
overridevirtual

Opens a database file.

Parameters
pathA path of the file.
writableIf true, the file is writable. If false, it is read-only.
optionsBit-sum options for opening the file.
Returns
The result status.

Implements tkrzw::DBM.

◆ OpenAdvanced()

Status tkrzw::ShardDBM::OpenAdvanced ( const std::string &  path,
bool  writable,
int32_t  options = File::OPEN_DEFAULT,
const std::map< std::string, std::string > &  params = {} 
)
overridevirtual

Opens a database file, in an advanced way.

Parameters
pathA path of the file.
writableIf true, the file is writable. If false, it is read-only.
optionsBit-sum options for opening the file.
paramsOptional parameters. All parameters for PolyDBM::OpenAdvanced are supported. Moreover, the parameter "num_shards" specifies the number of shards. Each shard file has a suffix like "-00003-of-00015". If the number of shards is not specified and existing files match the path, it is implicitly specified. If there are no matching files, 1 is implicitly set.
Returns
The result status.

Implements tkrzw::ParamDBM.

◆ Close()

Status tkrzw::ShardDBM::Close ( )
overridevirtual

Closes the database file.

Returns
The result status.

Implements tkrzw::DBM.

◆ Process()

Status tkrzw::ShardDBM::Process ( std::string_view  key,
RecordProcessor proc,
bool  writable 
)
overridevirtual

Processes a record with a processor.

Parameters
keyThe key of the record.
procThe pointer to the processor object.
writableTrue if the processor can edit the record.
Returns
The result status.

If the specified record exists, the ProcessFull of the processor is called. Otherwise, the ProcessEmpty of the processor is called.

Implements tkrzw::DBM.

◆ Get()

Status tkrzw::ShardDBM::Get ( std::string_view  key,
std::string *  value = nullptr 
)
overridevirtual

Gets the value of a record of a key.

Parameters
keyThe key of the record.
valueThe pointer to a string object to contain the result value. If it is nullptr, the value data is ignored.
Returns
The result status.

Reimplemented from tkrzw::DBM.

◆ Set()

Status tkrzw::ShardDBM::Set ( std::string_view  key,
std::string_view  value,
bool  overwrite = true,
std::string *  old_value = nullptr 
)
overridevirtual

Sets a record of a key and a value.

Parameters
keyThe key of the record.
valueThe value of the record.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
old_valueThe pointer to a string object to contain the old value. Assignment is done even on the duplication error. If it is nullptr, it is ignored.
Returns
The result status.

Reimplemented from tkrzw::DBM.

◆ Remove()

Status tkrzw::ShardDBM::Remove ( std::string_view  key,
std::string *  old_value = nullptr 
)
overridevirtual

Removes a record of a key.

Parameters
keyThe key of the record.
old_valueThe pointer to a string object to contain the old value. If it is nullptr, it is ignored.
Returns
The result status.

Reimplemented from tkrzw::DBM.

◆ Append()

Status tkrzw::ShardDBM::Append ( std::string_view  key,
std::string_view  value,
std::string_view  delim = "" 
)
overridevirtual

Appends data at the end of a record of a key.

Parameters
keyThe key of the record.
valueThe value to append.
delimThe delimiter to put after the existing record.
Returns
The result status.

If there's no existing record, the value is set without the delimiter.

Reimplemented from tkrzw::DBM.

◆ ProcessEach()

Status tkrzw::ShardDBM::ProcessEach ( RecordProcessor proc,
bool  writable 
)
overridevirtual

Processes each and every record in the database with a processor.

Parameters
procThe pointer to the processor object.
writableTrue if the processor can edit the record.
Returns
The result status.

The ProcessFull of the processor is called repeatedly for each record. The ProcessEmpty of the processor is called once before the iteration and once after the iteration.

Implements tkrzw::DBM.

◆ Count()

Status tkrzw::ShardDBM::Count ( int64_t *  count)
overridevirtual

Gets the number of records.

Parameters
countThe pointer to an integer object to contain the result count.
Returns
The result status.

Implements tkrzw::DBM.

◆ GetFileSize()

Status tkrzw::ShardDBM::GetFileSize ( int64_t *  size)
overridevirtual

Gets the current file size of the database.

Parameters
sizeThe pointer to an integer object to contain the result size.
Returns
The result status.

Implements tkrzw::DBM.

◆ GetFilePath()

Status tkrzw::ShardDBM::GetFilePath ( std::string *  path)
overridevirtual

Gets the path of the database file.

Parameters
pathThe pointer to a string object to contain the result path.
Returns
The result status.

Implements tkrzw::DBM.

◆ Clear()

Status tkrzw::ShardDBM::Clear ( )
overridevirtual

Removes all records.

Returns
The result status.

Implements tkrzw::DBM.

◆ Rebuild()

Status tkrzw::ShardDBM::Rebuild ( )
overridevirtual

Rebuilds the entire database.

Returns
The result status.

Implements tkrzw::DBM.

◆ RebuildAdvanced()

Status tkrzw::ShardDBM::RebuildAdvanced ( const std::map< std::string, std::string > &  params = {})
overridevirtual

Rebuilds the entire database, in an advanced way.

Parameters
paramsOptional parameters.
Returns
The result status.

The parameters work in the same way as with PolyDBM::RebuildAdvanced.

Implements tkrzw::ParamDBM.

◆ ShouldBeRebuilt()

Status tkrzw::ShardDBM::ShouldBeRebuilt ( bool *  tobe)
overridevirtual

Checks whether the database should be rebuilt.

Parameters
tobeThe pointer to a boolean object to contain the result decision.
Returns
The result status.

Implements tkrzw::DBM.

◆ Synchronize()

Status tkrzw::ShardDBM::Synchronize ( bool  hard,
FileProcessor proc = nullptr 
)
overridevirtual

Synchronizes the content of the database to the file system.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
procThe pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored.
Returns
The result status.

Implements tkrzw::DBM.

◆ SynchronizeAdvanced()

Status tkrzw::ShardDBM::SynchronizeAdvanced ( bool  hard,
FileProcessor proc = nullptr,
const std::map< std::string, std::string > &  params = {} 
)
overridevirtual

Synchronizes the content of the database to the file system, in an advanced way.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
procThe pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored.
paramsOptional parameters.
Returns
The result status.

The parameters work in the same way as with PolyDBM::OpenAdvanced.

Implements tkrzw::ParamDBM.

◆ CopyFile()

Status tkrzw::ShardDBM::CopyFile ( const std::string &  dest_path)
overridevirtual

Copies the content of the database files to other files.

Parameters
dest_pathA path prefix to the destination files.
Returns
The result status.

Copying is done while the content is synchronized and stable. So, this method is suitable for making a backup file while running a database service. Each shard file is copied and the destination file also has the same suffix.

Reimplemented from tkrzw::DBM.

◆ Inspect()

std::vector<std::pair<std::string, std::string> > tkrzw::ShardDBM::Inspect ( )
overridevirtual

Inspects the database.

Returns
A vector of pairs of a property name and its value.

Implements tkrzw::DBM.

◆ IsOpen()

bool tkrzw::ShardDBM::IsOpen ( ) const
overridevirtual

Checks whether the database is open.

Returns
True if the database is open, or false if not.

Implements tkrzw::DBM.

◆ IsWritable()

bool tkrzw::ShardDBM::IsWritable ( ) const
overridevirtual

Checks whether the database is writable.

Returns
True if the database is writable, or false if not.

Implements tkrzw::DBM.

◆ IsHealthy()

bool tkrzw::ShardDBM::IsHealthy ( ) const
overridevirtual

Checks whether the database condition is healthy.

Returns
True if the database condition is healthy, or false if not.

Implements tkrzw::DBM.

◆ IsOrdered()

bool tkrzw::ShardDBM::IsOrdered ( ) const
overridevirtual

Checks whether ordered operations are supported.

Returns
True if ordered operations are supported, or false if not.

Implements tkrzw::DBM.

◆ MakeIterator()

std::unique_ptr<DBM::Iterator> tkrzw::ShardDBM::MakeIterator ( )
overridevirtual

Makes an iterator for each record.

Returns
The iterator for each record.

Implements tkrzw::DBM.

◆ MakeDBM()

std::unique_ptr<DBM> tkrzw::ShardDBM::MakeDBM ( ) const
overridevirtual

Make a new DBM object of the same concrete class.

Returns
The new file object.

Implements tkrzw::DBM.

◆ GetInternalDBM()

DBM* tkrzw::ShardDBM::GetInternalDBM ( ) const

Gets the pointer to the first internal database object.

Returns
The pointer to the first internal database object, or nullptr on failure.

◆ RestoreDatabase()

static Status tkrzw::ShardDBM::RestoreDatabase ( const std::string &  old_file_path,
const std::string &  new_file_path,
const std::string &  class_name = "",
int64_t  end_offset = -1 
)
static

Restores a broken database as a new healthy database.

Parameters
old_file_pathThe path of the broken database.
new_file_pathThe path of the new database to be created.
class_nameThe name of the database class. If it is empty, the class is guessed from the file extension.
end_offsetThe exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly.
Returns
The result status.