merkle-db.spark.table-rdd

A TableRDD represents the lazy application of a scan over the records in a table. The scan may be bounded by a minimum and maximum key to load just a subset of the records in the table, as well as specifying a subset of the fields to load for each record.

The RDD effectively has the type PairRDD[(Key, Map)], where the keys are the serialized table keys and the values are fully-decoded records. A table scan RDD does not depend on any parent RDDs, since it is a direct data source.

scan

(scan spark-ctx init-store table)(scan spark-ctx init-store table scan-opts)

with-op-scope

macro

(with-op-scope spark-ctx scope-name & body)

Apply a Spark context operation scope around the statements in the body.