Riak in Depth

Riak lineage

NoSQL database Riak already know. Basho.com is distributed by the company, and the Enterprise version has support for the Community version and free version of commercial technology that can get support.

Are divided into two major NoSQL lineage,   Dynamo, Riak, which is divided into the family of Cassandra, HBase, HyperTable based on Google’s Big Table paper, such as Amazon Dynamo paper belongs to a series.

Document store database JSON documents that are stored in the data model, Value Key Value store format to take, and takes the form of a data model similar to MongoDB or CouchDB and has

MongoDB from Migration is because of these characteristics, is known to many users.

Riak of technical features

Ring structure-based architecture

Riak, as already mentioned, because it has the structure of the series, Dynamo Ring has a structure in a distributed data structure that stores By Data Key Hash algorithm, is to get to the proper node structure.

 

Riak Clustering of units and vnode can be divided into the node, the node refers to a physical server and vnode logical server with one instance of the concept so if you think that is one of the physical servers in the multiple logical servers (vnode) can be installed.

Riak Runtime the Ring structure in the event of dynamic can be set. (Node to add or subtract.) In this process, depending on the structure of the Riak data automatically modified Ring redistributed.

Master node architecture

Mongodb and consists of the nodes that store data, Request Routing (mongos, mongod) architecture is the master-slave concept, Riak, the master node does not have a separate hand this the end of the Single Failure Point means that there is no single point of failure) (balancing the load of each node to the L4 switch on the front, or using a web server such as Apache or NginX using a software load balancer (Reverse Proxy) can be configured.

※ front of the load balancer is not possible to determine the disability status of Riak vnode Using the Client SDK provided by Riak, fast fault detection and using Connection Pooling In addition, improved performance and efficiency of resource use can be improved.

Data Replication (Replication)

NoSQL structure has to replicate each data internally do some data replication is referred to as “N-Value” policy is that the default setting has three copies typically

(Extended or deleted) one data distributed storage system having three copies typically shown in the recent fashion, the reason for having three working when impaired when compared to a two-node is to maintain the Active state.

Data consisting of real-time data replication because replication of data between nodes is accomplished, of course, precisely the consistency of the data is not guaranteed. WRITE, and then, if replicated to other nodes in the READ READ If done before READ can be means that the old data. (Millisecond units, but)

CAP theory

Published by Professor Eric Brewer of the University of California at Berkeley in 2002, the CAP theory to the theory of distributed computing, distributed computing environment Consistency, Availability, and Partitioning Features and can be satisfied only two dualtheories.

 

   Consistency access to any node of the distributed nodes, even if the data value is equal to the functional characteristics. (Query the data during replication when the other data values ​​can be Query Consistency does not provide system)

   Availability features offered by more than one node of the clustered nodes, FAIL, even if the request can not be processed normally.

   Partition Tolerance impaired even if the network communication between nodes clustering is the ability to perform all the services normally.

Riak is implemented as a double-A and P in a direction that support ..

Riak data structure

Bucket

Bucket every Property keyspace (RDBMS to multiple Bucket RDBMS database to look into some sort of container. Hitting the table), and contains the attributes can be defined. (Replica of the number that is being replicated, the actual physical storage type, etc.)

Data Structure

Riak similar to the other NoSQL Key-Value storage structure has Seems to be easy to understand if you think (Hash Table). Unique Key from the corresponding Bucket, Value must be a JSON Document or Binary data can be stored. Specific details of the data structures in the Header part contains, metadata or Riak to provide user-defined Header Index defined to specify relationships with other Object Link.

Riak Architecture

Riak general characteristics of investigated until now. Riak architecture and some of the technical features go a little deeper look at.

 

 Riak Architecture Stack

Storage BackEnd

Riak to store data when the physical store, you can choose the following four major type of repository support.

 

BitCask

Basho created in storage, Erlang Bitcask base.

Provides faster performance on your own Memory Key Index, but the contrary, requires a lot of memory compared to other storage backend.

Bitcask append only database, change or delete data on impossible, because change is the deletion of data occurs History to manage and save a new version of the form will take., To periodically delete the old version of Garbage Collection tasks.

InnoStore

Wrapping a storage Oracle’s InnoDB Innostore strengths in mass storage, he said. InnoDB Hey, as long as the MySQL Backend as verified reliability high, Commercial Support from the Oracle side to get so personal opinion is a little higher in terms of stability to ensure that would not be possible.

Recommended InnoStore so basically from Riak but BitCask if that requires a lot of memory, whereas InnoStore a lot of memory, because it does not need to be used in this scenario is possible. InnoStore design Precautions during separate Disk InnoDB DB files and Log files should be stored by dividing one can achieve the same performance.

LevelDB

Google announced some time ago, DB, BigTable theory rooted LevelDB was implemented.

Finally Memory, Memory, currently when I saw writing primarily with the purposes of the test, such as is used, such as Redis or Memcached uses cache memory to use would be made ​​possible. (Subjective thought)

Storage BackEnd these four, depending on the purpose and the purpose Riak in Riak Cluster mixed in so that you can use.

When BitCask + LevelDB documents give a clear guide on how to select a feature or sangeneun for four Storage BackEnd does not want to use the Default and recommended area BitCask Secondary Index recommend.

Riak KV

 

Riak KV Key Value Storage, Riak of the main data storage structure that is responsible for. For Key Value Pair Data stored data. Query in two ways to support the Secondary Index and MR (Map & Reduce).

Map & Reduce

 

MR imaging to divide the vnode of several actions related to the retrieval of data, then (Map), the result of a combination of the return that form (Reduce). Use MR, The Map and the Reduce Function Map Function and Reduce Function defined in the request sent to write Erlang or JavaScript based are available.

(Note: Riak MR http://wiki.basho.com/MapReduce.html )

Map example) data from the field, finding 600 more than the value of “High” Map Function (JavaScript version)

Function (value, keyData, arg) {

 var data = Riak.mapValuesJson (value) [0];

 if (data.High && data.High> 600.00)

return [value.key];

 else

return [];

}

Actually using Riak When calling on REST + JSON as follows: HTTP Request sends

{“Inputs”: “goog”,

 “Query”: [{“map”: {“language”: “javascript”,

              “Source”: “function (value, keyData, arg) {var data = Riak.mapValuesJson (value) [0]; if (data.High && parseFloat (data.High)> 600.00) return [value.key]; else return [];} “,

              “Keep”: true}}]

}

Map & Reduce Example) The following example is an example of including the Map and Reduce.

Map to calculate the monthly deviation of sales by (maximum – minimum: data.High-data.Low) and the deviation of the monthly revenue from the Reduce step is an example of finding the greatest value.

/ * Map function to compute the daily variance and key it by the month * /

function (value, keyData, arg) {

 var data = Riak.mapValuesJson (value) [0];

 var month = value.key.split (‘-‘). slice (0, 2). join (‘-‘);

 var obj = {};

 obj [month] = data.High – data.Low;

 return [obj];

}

/ * Reduce function to find the maximum variance per month * /

function (values, arg) {

 return [values.reduce (function (acc, item) {

         for (var month in item) {

             if (acc [month]) {acc [month] = (acc [month] <item [month])? item [month]: acc [month];}

             else {acc [month] = item [month];}

         }

         return acc;

        })

     ];

}

Secondary Index

Riak’s a powerful new features in this release that supports one of the Secondary Index is the ability to easily talk, RDB Roach sleep on the table, as well as the Primary Key Multiple Column particular Record Query Index foot when a fast response time guarantee. (Riak multiple of one of the KeyStore Index at the same time you can make.)

When you enter or edit data using the REST PUT / POST method, how to define in the Index Riak in the header of the “x-riak-index {index name} _ {data type}:} {index date format” technology. following is an example of the index specified by the user in the Key Value Store, called “twitter” and “email” fields.

curl-X POST \

-H ‘x-riak-index-twitter_bin: rustyio’ \

-H ‘x-riak-index-email_bin: rusty@basho.com’ \

-D ‘… user data …’ \

http://localhost:8098/buckets/users/keys/rustyk

For more information about how to query the data using the Index HTTP GET / {bucket} / index / {index} / {index value} can be described as below in the bucket called “users” in a “twitter_bin” index “rustyio” Object with a value of the query is an example haeohneun.

localhost: 8098/buckets/users/index/twitter_bin/rustyio

February 2012 Riak Secondary Index has a few limitations: First, you can search simultaneously using multiple Index no search results for Sorting and Paging is not supported. (Range Search is available – for example, between age 20 to 30 years old with Object Index Search)

Index File Object to the form that stores the information will be stored in the same vnode, Index information is internally stored in Google LevelDB storage backend. Ring Topology Key / Value Hash value based on the value of Key Features vnode visit Only specific vnode receiving the request, because the hand Query Index based Index value which has been stored in the vnode not be able to find. Hash search the Index based on the value (as described above for Object Index Object is stored in the same vnode, because it is impossible for this reason.) To perform the entire vnode index Query Query distributed doeseo performed because, based Query Index gives a burden on the entire node. (1 / N vnode) performed in the

Riak Search

Riak Search Full Text Search (FTS) that enables the search with the contents of the Value Function. Riak Search was implemented with Apache’s Lucene ( http://lucene.apache.org FTS ) to the default Riak in order to optimize Lucene Riak Erlang implementation of the original Lucene and Java to implement because it is part of the was re-implemented in Erlang.

 In advance before the architecture described in the diagram in Riak KV and Riak Search by separating representation, but physically Riak Search is a part of Riak KV. (KV install in the Search go.), Which combined with the design ideas and Riak Search as a separate module without detaching, which relieves the burden by combining many components that need to be managed in the operating

Search for West Search How to Search Index understand there is a need. Riak Search Index storage mechanism using a technique called the “Term Partitioning”. Search Dictionary-based or can be simply cut by a tab or space units Analyzer, which is responsible in the Key / Value Value Text Parsing anti Term is divided into units of the unit to cut the Term, the Term Extraction also, and user requirements, and can be used in various ways, such as direct Analyzer can produce also.

http://wiki.basho.com/Riak-Search—Indexing.html

To use, Term Index to the Term Term extraction is distributed over the entire Cluster. Thus, the N-Value and Key-Value stores in much the same way as you divide replica also stored in the form.

Do so by using the Term Partitioning Query Search Keyword come through the case of a single word, which is stored in the vnode Hash value, because it can not find the Search operation does not occur over the entire node. In some cases, if the door to use of other operations, such as AND, OR, or use wild card characters only (Secondary Index compared with that part). 1 or ~ 1 / Query distributed over N one vnode can be performed.

Map & Reduce, Secondary Index, Search Compare

MapReduce

Riak Search

Secondary Indexs

Query Types

Implement the query using the Map and the Reduce Function

Lucene SOLR query using text search, wildcard model and Boolean search (AND, OR)

Comparing the same values ​​(Equality), a range-based queries Range Query Support

Index Locality

N / A

N vnode in the Search Index is distributed using Riak KV and Saved same back-end

Index Saved Object is stored in the same vnode

Data Storage Backend LevelDB Use

Vnodes Queried

Depends on input

Query performs vnode Map & Reduce logic determined

1 per term queries; 1 / N for trailing wild card

Term Index “is stored as one of the” Term “if you want to search vNode performed.

Wildcard Search if performed over the full (1 / N) vNode in

1 / N per request (aka coverage query)

Performed distributed across the full (1 / N) vnode on

Supported Data Types

Erlang and JavaScript, JSON to Map Reduce using

Integer, Date and Text

Binary and Integer

Suggested Use Cases

Value in the Full Text Search

Tag-based search

Poor Use Cases

Object from a large number of complex operations that case

Searching for common terms in documents

Limitation

Backend Storage must use LevelDB

Simultaneously using multiple Index Query impossible.

Anti-Entropy Fault Tolerance

Extraction

*

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s