org.apache.lens.cube.metadata

Class Storage

java.lang.Object

org.apache.lens.cube.metadata.AbstractCubeTable

org.apache.lens.cube.metadata.Storage

All Implemented Interfaces:

Named, PartitionMetahook

Direct Known Subclasses:

DBStorage, HDFSStorage

public abstract class Storage
extends AbstractCubeTable
implements PartitionMetahook

Storage is Named Interface which would represent the underlying storage of the data.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Storage.LatestInfo
static class	Storage.LatestPartColumnInfo

Field Summary

Fields inherited from class org.apache.lens.cube.metadata.AbstractCubeTable

LOG

Constructor Summary

Constructors

Constructor and Description
Storage(org.apache.hadoop.hive.ql.metadata.Table hiveTable)

Method Summary

Methods

Modifier and Type	Method and Description
void	addPartition(org.apache.hadoop.hive.ql.metadata.Hive client, StoragePartitionDesc addPartitionDesc, Storage.LatestInfo latestInfo) Add single partition to storage.
void	addPartitions(org.apache.hadoop.hive.ql.metadata.Hive client, String factOrDimTable, UpdatePeriod updatePeriod, List<StoragePartitionDesc> storagePartitionDescs, Map<Map<String,String>,Storage.LatestInfo> latestInfos) Add given partitions in the underlying hive table and update latest partition links
void	dropPartition(org.apache.hadoop.hive.ql.metadata.Hive client, String storageTableName, List<String> partVals, Map<String,Storage.LatestInfo> updateLatestInfo, Map<String,String> nonTimePartSpec) Drop the partition in the underlying hive table and update latest partition link
String	getPrefix() Get the name prefix of the storage
static String	getPrefix(String name) Get the name prefix of the storage
Set<String>	getStorages()
org.apache.hadoop.hive.ql.metadata.Table	getStorageTable(org.apache.hadoop.hive.ql.metadata.Hive client, org.apache.hadoop.hive.ql.metadata.Table parent, StorageTableDesc crtTbl) Get the storage table descriptor for the given parent table.
CubeTableType	getTableType()

Methods inherited from class org.apache.lens.cube.metadata.AbstractCubeTable

addProperties, alterWeight, equals, getAllFieldNames, getColumns, getName, getProperties, getWeight, hashCode, removeProperty, toString, weight

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.lens.cube.metadata.PartitionMetahook

commitAddPartitions, commitDropPartition, preAddPartitions, preDropPartition, rollbackAddPartitions, rollbackDropPartition

Constructor Detail

Storage

public Storage(org.apache.hadoop.hive.ql.metadata.Table hiveTable)

Method Detail

getPrefix

public String getPrefix()

Get the name prefix of the storage

Returns:

Name followed by storage separator

getTableType

public CubeTableType getTableType()

Specified by:

getTableType in class AbstractCubeTable

getStorages

public Set<String> getStorages()

Specified by:

getStorages in class AbstractCubeTable

getPrefix

public static String getPrefix(String name)

Get the name prefix of the storage

Parameters:

name - Name of the storage

Returns:

Name followed by storage separator

getStorageTable

public org.apache.hadoop.hive.ql.metadata.Table getStorageTable(org.apache.hadoop.hive.ql.metadata.Hive client,
                                                       org.apache.hadoop.hive.ql.metadata.Table parent,
                                                       StorageTableDesc crtTbl)
                                                         throws org.apache.hadoop.hive.ql.metadata.HiveException

Get the storage table descriptor for the given parent table.

Parameters:

client - The metastore client

parent - Is either Fact or Dimension table

crtTbl - Create table info

Returns:

Table describing the storage table

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

addPartition

public void addPartition(org.apache.hadoop.hive.ql.metadata.Hive client,
                StoragePartitionDesc addPartitionDesc,
                Storage.LatestInfo latestInfo)
                  throws org.apache.hadoop.hive.ql.metadata.HiveException

Add single partition to storage. Just calls #addPartitions.

Parameters:

client -

addPartitionDesc -

latestInfo -

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

addPartitions

public void addPartitions(org.apache.hadoop.hive.ql.metadata.Hive client,
                 String factOrDimTable,
                 UpdatePeriod updatePeriod,
                 List<StoragePartitionDesc> storagePartitionDescs,
                 Map<Map<String,String>,Storage.LatestInfo> latestInfos)
                   throws org.apache.hadoop.hive.ql.metadata.HiveException

Add given partitions in the underlying hive table and update latest partition links

Parameters:

client - hive client instance

factOrDimTable - fact or dim name

updatePeriod - update period of partitions.

storagePartitionDescs - all partitions to be added

latestInfos - new latest info. atleast one partition for the latest value exists for each part column

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

dropPartition

public void dropPartition(org.apache.hadoop.hive.ql.metadata.Hive client,
                 String storageTableName,
                 List<String> partVals,
                 Map<String,Storage.LatestInfo> updateLatestInfo,
                 Map<String,String> nonTimePartSpec)
                   throws org.apache.hadoop.hive.ql.metadata.HiveException

Drop the partition in the underlying hive table and update latest partition link

Parameters:

client - The metastore client

storageTableName - TableName

partVals - Partition specification

updateLatestInfo - The latest partition info if it needs update, null if latest should not be updated

nonTimePartSpec -

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException