Sunil Sattiraju lordlinus

PostgreSQL is Enough

Background and Cron Jobs

Spark HWC integration - HDP 3 Secure cluster

Prerequisites :

Kerberized Cluster
Enable hive interactive server in hive
Get following details from hive for spark or try this HWC Quick Test Script

Meta Gist

PostgreSQL and PostGIS queries
CartoCSS styles
CARTO.js blocks
DI.js blocks
Leaflet.js blocks
Other blocks
Others

Applied Functional Programming with Scala - Notes

1. Mastering Functions

A function is a mapping from one set, called a domain, to another set, called the codomain. A function associates every element in the domain with exactly one element in the codomain. In Scala, both domain and codomain are types.

val square : Int => Int = x => x * x

General
- Mastering Observable - http://docs.couchbase.com/developer/java-2.0/observables.html
Android
- Grokking RxJava, Part 1-4 - http://blog.danlew.net/2014/09/15/grokking-rxjava-part-1/
- Loading data from multiple sources with RxJava - http://blog.danlew.net/2015/06/22/loading-data-from-multiple-sources-with-rxjava/
- Don't break the chain: use RxJava's compose() operator - http://blog.danlew.net/2015/03/02/dont-break-the-chain/
Pro RxJava
- Hot and Cold Observable - http://davesexton.com/blog/post/Hot-and-Cold-Observables.aspx
- To use subject or not to use subject - http://davesexton.com/blog/post/To-Use-Subject-Or-Not-To-Use-Subject.aspx
RxJava Threading Examples - http://www.grahamlea.com/2014/07/rxjava-threading-examples/

Essential JavaScript Links

I no longer mantain this list. There are lots of other very comprehensive JavaScript link lists out there. Please see those, instead (Google "awesome JavaScript" for a start).

	// define the name of the Azure Databricks notebook to run
	val notebookToRun = ???

	// define some way to generate a sequence of workloads to run
	val jobArguments = ???

	// define the number of workers per job
	val workersPerJob = ???

	import java.util.concurrent.Executors

	# This function is used to reduce memory of a pandas dataframe
	# The idea is cast the numeric type to another more memory-effective type
	# For ex: Features "age" should only need type='np.int8'
	# Source: https://www.kaggle.com/gemartin/load-data-reduce-memory-usage
	def reduce_mem_usage(df):
	""" iterate through all the columns of a dataframe and modify the data type
	to reduce memory usage.
	"""
	start_mem = df.memory_usage().sum() / 1024**2
	print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))