Skip to content

Instantly share code, notes, and snippets.

@sploiselle
Created April 1, 2020 16:02
Show Gist options
  • Save sploiselle/530eeee74050ba32959b732da7c626c4 to your computer and use it in GitHub Desktop.
Save sploiselle/530eeee74050ba32959b732da7c626c4 to your computer and use it in GitHub Desktop.

You can start the materialized binary with:

./materialized_(linux|mac) --dev --w=1

And then launch into the SQL interface (e.g. psql -h 0.0.0.0 -p 6875 -d materialize)

And here is the part I'm having trouble testing: actually connecting to a Kerberized Kafka cluster:

CREATE SOURCE quotes
  FROM KAFKA BROKER '<broker host>' TOPIC '<topic>'
  WITH (
    security_protocol = 'sasl_plaintext', 
    sasl_kerberos_keytab = '<path to keytab>',
    sasl_kerberos_principal = '<principal val>',
    sasl_kerberos_kinit_cmd = '<kinit cmd>',
    sasl_kerberos_min_time_before_relogin = '<val>'
  )
  FORMAT AVRO 
    USING CONFLUENT SCHEMA REGISTRY '<schema registry>';

I assume you're using a Schema Registry; if not, you could just change to FORMAT BYTES since all I'm trying to see is if this will connect to the Kafka cluster.

You can also specify sasl_mechanisms and sasl_kerberos_service_name but I believe those properly default through rdkafka; if not here's another statement to try:

CREATE SOURCE quotes
  FROM KAFKA BROKER '<broker host>' TOPIC '<topic>'
  WITH (
    security_protocol = 'sasl_plaintext',
    sasl_mechanisms = 'GSSAPI',
    sasl_kerberos_service_name = 'kafka',
    sasl_kerberos_keytab = '<path_to_keytab>',
    sasl_kerberos_principal = '<principal val>',
    sasl_kerberos_kinit_cmd = '<kinit cmd>',
    sasl_kerberos_min_time_before_relogin = '<val>'
  )
  FORMAT AVRO USING CONFLUENT SCHEMA REGISTRY '<schema registry>';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment