Gian Merlino gianm

Joins

About this document

This is a design document describing the mechanics of joins in Druid. It is related to the initial join support proposal (apache/druid#8728) but covers different things: the proposal is more about motivation, design considerations, scope, and direction. This document is more about implementation.

Virtual join segments

Mechanics

See http://druid.io/docs/0.11.0-rc2/operations/tls-support.html for reference docs.

Quick start:

Add "simple-client-sslcontext" to druid.extensions.loadList in common.runtime.properties.
Add the properties below to common.runtime.properties.
For local testing, create a new self-signed certificate using the following command (but for production you should use a real certificate signed by a CA). Note that with these certificates the important part is the CN, which must match the hostname of your server. The command for testing purposes is: openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 3650 -out cert.pem
Create pkcs12 bundle with: openssl pkcs12 -inkey key.pem -in cert.pem -export -out key.p12 -name druid
Create keystore: keytool -importkeystore -destkeystore imply-keystore.jks -srckeystore key.p12 -srcstoretype PKCS12
Create truststore: keytool -import -alias druid -file cert.pem -keystore imply-truststore.jks

	1) from:(git@apache.org) to:(dev@druid.apache.org)
	Do this: Skip Inbox, Apply label "apache-gitbox"

	2) to:(dev@druid.apache.org) "This is an automated email from the ASF dual-hosted git repository." "https://gitbox.apache.org/repos"
	Do this: Skip Inbox, Apply label "apache-gitbox"

	3) ((to:(dev@druid.apache.org) OR to:(dev@druid.incubator.apache.org)) -from:git@apache.org -"This is an automated email from the ASF dual-hosted git repository") -"https://gitbox.apache.org/repos"
	Do this: Skip Inbox, Apply label "apache-druid-dev"

	#!/bin/bash -eux

	#
	# Backports patches from master to a release branch.
	#
	# Usage:
	# export GIT_TOKEN=...
	# druid-backport [branch] [pr-number], like: "druid-backport 0.12.1 5706"
	#
	# Bugs:

	84 import com.google.common.annotations.VisibleForTesting;
	1 import com.google.common.base.CharMatcher;
	53 import com.google.common.base.Charsets;
	260 import com.google.common.base.Function;
	14 import com.google.common.base.Functions;
	52 import com.google.common.base.Joiner;
	9 import com.google.common.base.Objects;
	74 import com.google.common.base.Optional;
	446 import com.google.common.base.Preconditions;
	134 import com.google.common.base.Predicate;

	<code_scheme name="Calcite style" version="173">
	<option name="OTHER_INDENT_OPTIONS">
	<value>
	<option name="INDENT_SIZE" value="2" />
	<option name="TAB_SIZE" value="2" />
	</value>
	</option>
	<option name="LINE_SEPARATOR" value=" " />
	<option name="GENERATE_FINAL_LOCALS" value="true" />
	<option name="GENERATE_FINAL_PARAMETERS" value="true" />

	#!/usr/bin/env python

	# Copyright 2017 Imply Data, Inc.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

	#!/bin/bash -eux

	# Note: assumes "base" and "milestone" are the same (for the backport PR)

	base="$1"
	pr="$2"
	myrepo="gianm"
	backport="backport-$pr-to-$base"

	if [ "$(curl -u "$myrepo:$GIT_TOKEN" -s https://api.github.com/repos/druid-io/druid/pulls/"$pr" \|jq -r '.merged')" != "true" ]

	@Test
	public void testGroupByWithLookupDimensionSpec()
	{
	Map<String, String> map = new HashMap<>();
	map.put("automotive", "health105");
	map.put("business", "health20");
	map.put("entertainment", "travel47");
	map.put("health", "health55");
	map.put("mezzanine", "health09");
	map.put("news", "health0000");

	dataSources:
	- spec:
	dataSchema:
	dataSource: druid-metrics
	parser:
	type: string
	parseSpec:
	format: json
	timestampSpec:
	column: timestamp