-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1 from inaka/cabol.221.riak_adapter_impl
[Close #221] Cabol.221.riak adapter impl
- Loading branch information
Showing
18 changed files
with
8,301 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,3 +8,6 @@ ebin | |
rel/example_project | ||
.concrete/DEV_MODE | ||
.rebar | ||
.erlang.mk/ | ||
logs/ | ||
sumo_db_riak.d |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
PROJECT = sumo_db_riak | ||
|
||
CONFIG ?= test/test.config | ||
|
||
DEPS = sumo_db riakc iso8601 | ||
|
||
dep_sumo_db = git https://github.com/inaka/sumo_db.git 01db604 | ||
dep_riakc = git https://github.com/inaka/riak-erlang-client.git 2.1.1-R18 | ||
dep_iso8601 = git https://github.com/zerotao/erlang_iso8601.git 0d14540 | ||
|
||
TEST_DEPS = katana_test mixer | ||
|
||
dep_katana_test = git https://github.com/inaka/katana-test.git 0.0.1 | ||
dep_mixer = git https://github.com/inaka/mixer.git 0.1.5 | ||
|
||
BUILD_DEPS = inaka_mk | ||
|
||
dep_inaka_mk = git https://github.com/inaka/inaka.mk.git 1.0.0 | ||
|
||
DEP_PLUGINS = inaka_mk | ||
|
||
LOCAL_DEPS := tools compiler syntax_tools common_test inets test_server dialyzer wx | ||
|
||
include erlang.mk | ||
|
||
ERLC_OPTS := +'{parse_transform, lager_transform}' | ||
ERLC_OPTS += +warn_unused_vars +warn_export_all +warn_shadow_vars +warn_unused_import +warn_unused_function | ||
ERLC_OPTS += +warn_bif_clash +warn_unused_record +warn_deprecated_function +warn_obsolete_guard +strict_validation | ||
ERLC_OPTS += +warn_export_vars +warn_exported_vars +warn_missing_spec +warn_untyped_record +debug_info | ||
|
||
# Common Test Config | ||
TEST_ERLC_OPTS += +'{parse_transform, lager_transform}' +debug_info | ||
CT_OPTS = -cover test/sumo_db_riak.coverspec -vvv -erl_args -boot start_sasl -config ${CONFIG} | ||
|
||
erldocs: | ||
erldocs . -o docs | ||
|
||
changelog: | ||
github_changelog_generator --token ${TOKEN} | ||
|
||
EDOC_OPTS += todo, report_missing_types |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,150 @@ | ||
# sumo_db_riak | ||
# sumo_db_riak | ||
|
||
This is the [Riak](http://docs.basho.com/riak/latest/) adapter for [sumo_db](https://github.com/inaka/sumo_db). | ||
|
||
|
||
## Implementation Notes | ||
|
||
These are some implementation notes you should know: | ||
|
||
1. This **Riak** adapter is implemented using [Riak Data Types](http://docs.basho.com/riak/latest/dev/using/data-types/). | ||
2. The modeled **sumo** docs/entities are mapped to [Riak Maps](http://docs.basho.com/riak/latest/dev/using/data-types/#Maps). | ||
3. This Riak adapter allows nested docs/entities, ang again, those entities are treat them | ||
as Riak Maps too. | ||
4. Bulk operations such as: `delete_all/1`, `delete_by/2`, `find_all/1,4` and `find_by/2,4,5`, | ||
they were optimized using streaming. Records are streamed in chunks (using [Riak 2i](http://docs.basho.com/riak/latest/dev/using/2i/) | ||
to stream keys first), and then the current operation is applied. E.g: supposing | ||
we are executing a `find_all/1`, all keys are fetched first, and then the values | ||
corresponding to each key are fetched (remember that fetch a doc by the key is the | ||
fastest way, O(1)). This allows better memory and cpu efficiency. | ||
5. Query functions were implemented using [Riak Search](http://docs.basho.com/riak/latest/dev/search/search-data-types/) | ||
on Data Types, to get better performance and flexibility. | ||
|
||
|
||
## Riak | ||
|
||
### Install Riak | ||
|
||
To install/upgrade **Riak** please follow the instructions in this link: | ||
[Installing and Upgrading Riak](http://docs.basho.com/riak/latest/ops/building/installing). | ||
|
||
### Initial Configurations | ||
|
||
Due to the fact that **Riak** comes with default configuration, we need to | ||
change some parameters required by `sumo_db`. | ||
|
||
**Riak** has a main configuration file `riak.conf`, which you can find into | ||
your installation path `$YOUR_INSTALL_PATH/etc/riak.conf`. | ||
|
||
> **Note:** For more information check this link [Configuration Files](http://docs.basho.com/riak/latest/ops/advanced/configs/configuration-files). | ||
First parameter to change is the default **Riak** backend from **Bitcask** to | ||
**LevelDB**. This change also enables the use of [Riak Secondary Indexes](http://docs.basho.com/riak/latest/ops/advanced/configs/secondary-index/). | ||
|
||
storage_backend = leveldb | ||
|
||
Then proceed to enable search capabilities: | ||
|
||
search = on | ||
|
||
> **Note:** For more information check this link [Riak Search Settings](http://docs.basho.com/riak/latest/ops/advanced/configs/search/). | ||
### Configuring Riak Data Types and Search | ||
|
||
First, let's create and activate a bucket type simply called maps that is set up | ||
to store Riak maps: | ||
|
||
Because `sumo_db_riak` adapter is implemented using **Riak Data Types**, and docs/entities | ||
are mapped to **Riak Maps**, we need to create a [Riak Bucket Type](http://docs.basho.com/riak/latest/dev/advanced/bucket-types/) | ||
for those docs that will be stored. | ||
|
||
Taking as example our [tests](./test), let's call the bucket type `maps`. | ||
|
||
$ riak-admin bucket-type create maps '{"props":{"datatype":"map"}}' | ||
$ riak-admin bucket-type activate maps | ||
|
||
Now, let's create a search index called `sumo_test_index` using the default | ||
schema: | ||
|
||
$ curl -XPUT $RIAK_HOST/search/index/sumo_test_index \ | ||
-H 'Content-Type: application/json' \ | ||
-d '{"schema":"_yz_default"}' | ||
|
||
With our index created, we can associate our new `sumo_test_index` index with | ||
our `maps` bucket type: | ||
|
||
$ riak-admin bucket-type update maps '{"props":{"search_index":"sumo_test_index"}}' | ||
|
||
Now we can start working with **Riak** from `sumo_db`. | ||
|
||
> **Note:** For more information check this link [Riak Data Types and Search](http://docs.basho.com/riak/latest/dev/search/search-data-types/#Maps-Example). | ||
|
||
## Getting Started | ||
|
||
To start use `sumo_db` with this Riak adapter `sumo_db_riak` is pretty easy, you only has to | ||
follow these steps: | ||
|
||
1. Add `sumo_db_riak` as dependencies in your project. | ||
|
||
Using **erlang.mk**: | ||
|
||
```erlang | ||
DEPS = sumo_db sumo_db_riak | ||
|
||
dep_sumo_db_riak = git https://github.com/inaka/sumo_db_riak.git 0.0.1 | ||
``` | ||
|
||
Using **Rebar**: | ||
|
||
```erlang | ||
{deps, [ | ||
{sumo_db_riak, {git, "https://github.com/inaka/sumo_db_riak.git", {tag, "0.0.1"}}} | ||
]}. | ||
``` | ||
|
||
2. You need at least one doc/entity, let's use [sumo_test_people_riak](./test/sumo_test_people_riak.erl) | ||
as example. | ||
|
||
3. Provide the configuration file, e.g.: [test.config](./tests/test.config). | ||
|
||
4. Now you can run your app and start using `sumo` from there. | ||
|
||
### Running sumo from Erlang console | ||
|
||
Start the Erlang console, adding the path to your beams and config file | ||
|
||
$ erl -pa ebin deps/*/ebin -config tests/test.config | ||
|
||
Within the console: | ||
|
||
```erlang | ||
> sumo_db_riak:start(). | ||
08:30:22.591 [info] Application lager started on node nonode@nohost | ||
08:30:22.598 [info] Application crypto started on node nonode@nohost | ||
08:30:22.601 [info] Creating wpool ETS table | ||
08:30:22.601 [info] Application worker_pool started on node nonode@nohost | ||
08:30:22.602 [info] Application quickrand started on node nonode@nohost | ||
08:30:22.603 [info] Application uuid started on node nonode@nohost | ||
08:30:22.635 [info] Application sumo_db started on node nonode@nohost | ||
08:30:22.635 [info] Application sumo_db_riak started on node nonode@nohost | ||
{ok,[sasl,syntax_tools,compiler,goldrush,lager,crypto, | ||
worker_pool,quickrand,uuid,sumo_db,sumo_db_riak]} | ||
|
||
% from here you can start using sumo | ||
|
||
> sumo:find_all(sumo_test_people_riak). | ||
[] | ||
``` | ||
|
||
|
||
## Contact Us | ||
|
||
For **questions** or **general comments** regarding the use of this library, | ||
please use our public [hipchat room](http://inaka.net/hipchat). | ||
|
||
If you find any **bugs** or have a **problem** while using this library, please | ||
[open an issue][issue] in this repo (or a pull request :)). | ||
|
||
And you can check all of our open-source projects at | ||
[inaka.github.io](http://inaka.github.io) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
[{elvis, [ | ||
{config, [ | ||
#{dirs => ["src", "test"], | ||
filter => "*.erl", | ||
ruleset => [ | ||
{elvis_style, line_length, #{limit => 80}}, | ||
{elvis_style, no_tabs}, | ||
{elvis_style, macro_names}, | ||
{elvis_style, macro_module_names}, | ||
{elvis_style, operator_spaces, #{rules => [ | ||
{right, ","}, | ||
{right, "++"}, | ||
{left, "++"} | ||
]}}, | ||
{elvis_style, nesting_level, #{level => 3}}, | ||
{elvis_style, god_modules, #{limit => 25}}, | ||
{elvis_style, no_if_expression}, | ||
{elvis_style, used_ignored_variable}, | ||
{elvis_style, no_behavior_info}, | ||
{elvis_style, module_naming_convention, | ||
#{regex => "^([a-z][a-z0-9]*_?)*(_SUITE)?$", ignore => []} | ||
}, | ||
{elvis_style, state_record_and_type}, | ||
{elvis_style, no_spec_with_records} | ||
] | ||
}, | ||
#{dirs => ["."], | ||
filter => "Makefile", | ||
rules => [{elvis_project, no_deps_master_erlang_mk, #{ignore => []}}] | ||
}, | ||
#{dirs => ["."], | ||
filter => "rebar.config", | ||
rules => [{elvis_project, no_deps_master_rebar, #{ignore => []}}] | ||
}, | ||
#{dirs => ["."], | ||
filter => "elvis.config", | ||
rules => [{elvis_project, old_configuration_format}] | ||
} | ||
]} | ||
]} | ||
]. |
Oops, something went wrong.