jq is useful utility, but I use it infrequently and each time I have to check what was the filters syntax. This cheat sheet helps we to recall what is the expression that I need. Hopefully it would help others too. For full guide refer to the official jq manual.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Based on ideas here, expanded to enable Hive support | |
https://www.linkedin.com/pulse/running-spark-2xx-cloudera-hadoop-distro-cdh-deenar-toraskar-cfa/ | |
wget https://archive.apache.org/dist/spark/spark-2.4.8/spark-2.4.8-bin-without-hadoop.tgz | |
tar -xvzf spark-2.4.8-bin-without-hadoop.tgz | |
cd spark-2.4.8-bin-without-hadoop | |
cp -R /etc/spark2/conf/* conf/ | |
cp /etc/hive/conf/hive-site.xml conf/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Requirements: kafka-python gssapi krbticket | |
import os | |
import time | |
from kafka import KafkaConsumer, KafkaProducer | |
from krbticket import KrbConfig, KrbCommand | |
try: | |
os.environ['KRB5CCNAME'] = '/tmp/krb5cc_<myusername>' | |
kconfig = KrbConfig(principal='araujo', keytab='/path/to/<myusername>.keytab') | |
KrbCommand.kinit(kconfig) |
You can use ssacli
(smart storage administrator command line interface) tool to manage any of supported HP Smart Array Controllers in your Proxmox host without need to reboot your server to access Smart Storage Administrator in BIOS. That means no host downtime when managing your storage.
CLI is not as convenient as GUI interface provided by BIOS or desktop utilities, but still allows you to fully manage your controller, physical disks and logical drives on the fly with no Proxmox host downtime.
ssacli
replaces older hpssacli
, but shares the same syntax and adds support for newer servers and controllers.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
""" | |
Example of Python Data Frame with SparkSession. | |
""" | |
from pyspark.conf import SparkConf | |
from pyspark.sql import SparkSession | |
from pyspark.sql.functions import * | |
from pyspark.sql.types import * |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from flask import Flask, request, jsonify, json, abort | |
from flask_cors import CORS, cross_origin | |
import pandas as pd | |
app = Flask(__name__) | |
cors = CORS(app) | |
app.config['CORS_HEADERS'] = 'Content-Type' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys | |
from datetime import date,datetime,timedelta | |
import datetime | |
import string | |
from pexpect import * | |
from os import remove,listdir | |
import os | |
import pprint | |
def chunks(l, n): |
NewerOlder