Export a riak-kv
node's objects to a file tree.
Given that:
- A desire to have a tangible representation of a
riak-kv
's node's content á lamysqldump
exists; - Existing tools that could fulfil the above, all fail in one way or another.
That is:
-
Every incantation that I could think of of
riak-admin backup <node> <cookie> <filename> [node|all]
... produced, apart from this depreciation warning:
riak-admin backup has been deprecated and its use is not recommended. Please see the documentation for more information on how to safely backup a cluster.
... exactly zero files.
-
https://github.com/basho-labs/riak-data-migrator
git clone https://github.com/basho-labs/riak-data-migrator.git cd riak-data-migrator mvn clean mvn package cd target java -jar riak-data-migrator-0.2.9.jar -h
Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) [...] ... 7 more
-
https://www.npmjs.com/package/riak-bucket-exporter
npm install -g riak-bucket-exporter
/usr/local/bin/riak-bucket-exporter -> /usr/local/lib/node_modules/riak-bucket-exporter/bin/riak-bucket-exporter.js > wtf8@0.2.0 install /usr/local/lib/node_modules/riak-bucket-exporter/node_modules/wtf8 > node-gyp rebuild CXX(target) Release/obj.target/wtf8/wtf8.o In file included from ../wtf8.cc:4: In file included from ../../nan/nan.h:74: ../../nan/nan_new.h:23:43: error: no member named 'Handle' in namespace 'v8' template <typename T> v8::Local<T> To(v8::Handle<v8::Integer> i); ~~~~^ [...] fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. make: *** [Release/obj.target/wtf8/wtf8.o] Error 1 gyp ERR! build error gyp ERR! stack Error: `make` failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) [...] npm ERR! Failed at the wtf8@0.2.0 install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /Users/david/.npm-cache/_logs/2020-01-24T12_11_00_858Z-debug.log
-
... I ultimately resorted to concoct the shell script included below;
There are no command line arguments to supply; parameterization is purely driven by environment variables. These, and their defaults are:
HOST=localhost
PORT=8098
BUCKET_TYPE=default
So, a compeletely parameterized invocation could look like:
HOST=example.com PORT=1234 BUCKET_TYPE=cusomt-bt rkvd.sh
This will create a file tree in the current working directory, conforming to this pattern:
${BUCKET_TYPE}/${BUCKET}/${KEY}/{head,body}
... where head
and body
will contain the corresponding sections of the response to the HTTP request below:
GET /types/${BUCKET_TYPE}/buckets/${BUCKET}/keys/${KEY}
The script will simply walk the listed buckets and their keys to fetch each object and store it as described above;
Curiously, riak-kv
's HTTP API does not seem to provide a means to list available bucket-types; so, to discover these, one could:
riak-admin bucket-type list
The script doesn't bother to do walk these as well but one could easily wrap a loop around the output of the above to achieve that:
( export BUCKET_TYPE; while IFS='' read BUCKET_TYPE ; do rkvd.sh ; done < <( riak-admin bucket-type list | sed -e 's/\s*(active)//' ))
Note this implies that rkvd.sh
be run on the machine that hosts the riak-kv
node; (and you probably should sudo riak-admin ...
too).
The script is only as complete as I currently need it to be. There's probably "some" ground left to cover.