org.apache.lens.cube.metadata

Class Storage

java.lang.Object

org.apache.lens.cube.metadata.AbstractCubeTable

org.apache.lens.cube.metadata.Storage

All Implemented Interfaces:

Named, PartitionMetahook

Direct Known Subclasses:

DBStorage, HDFSStorage

public abstract class Storage
extends AbstractCubeTable
implements PartitionMetahook

Storage is Named Interface which would represent the underlying storage of the data.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Storage.LatestInfo
static class	Storage.LatestPartColumnInfo

Constructor Summary

Constructors

Constructor and Description
Storage(org.apache.hadoop.hive.ql.metadata.Table hiveTable)

Method Summary

Modifier and Type	Method and Description
List<org.apache.hadoop.hive.ql.metadata.Partition>	addPartitions(org.apache.hadoop.hive.ql.metadata.Hive client, String factOrDimTable, UpdatePeriod updatePeriod, List<StoragePartitionDesc> storagePartitionDescs, Map<Map<String,String>,Storage.LatestInfo> latestInfos, String tableName) Add given partitions in the underlying hive table and update latest partition links
void	dropPartition(org.apache.hadoop.hive.ql.metadata.Hive client, String storageTableName, List<String> partVals, Map<String,Storage.LatestInfo> updateLatestInfo, Map<String,String> nonTimePartSpec) Drop the partition in the underlying hive table and update latest partition link
String	getPrefix() Get the name prefix of the storage
static String	getPrefix(String name) Get the name prefix of the storage
Set<String>	getStorages()
static org.apache.hadoop.hive.ql.metadata.Table	getStorageTable(String storageTableNamePrefix, org.apache.hadoop.hive.ql.metadata.Hive client, org.apache.hadoop.hive.ql.metadata.Table parent, StorageTableDesc crtTbl) Get the storage table descriptor for the given parent table.
CubeTableType	getTableType()
void	updatePartition(org.apache.hadoop.hive.ql.metadata.Hive client, String fact, org.apache.hadoop.hive.ql.metadata.Partition partition) Update existing partition
void	updatePartitions(String storageTable, org.apache.hadoop.hive.ql.metadata.Hive client, String fact, List<org.apache.hadoop.hive.ql.metadata.Partition> partitions) Update existing partitions

Methods inherited from class org.apache.lens.cube.metadata.AbstractCubeTable

addProperties, alterWeight, equals, getAllFieldNames, getColumns, getName, getProperties, getWeight, hashCode, now, removeProperty, toString, weight

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.lens.cube.metadata.PartitionMetahook

commitAddPartitions, commitDropPartition, commitUpdatePartition, preAddPartitions, preDropPartition, preUpdatePartition, rollbackAddPartitions, rollbackDropPartition, rollbackUpdatePartition

Constructor Detail

Storage

public Storage(org.apache.hadoop.hive.ql.metadata.Table hiveTable)

Method Detail

getPrefix

public String getPrefix()

Get the name prefix of the storage

Returns:

Name followed by storage separator

getTableType

public CubeTableType getTableType()

Specified by:

getTableType in class AbstractCubeTable

getStorages

public Set<String> getStorages()

Specified by:

getStorages in class AbstractCubeTable

getPrefix

public static String getPrefix(String name)

Get the name prefix of the storage

Parameters:

name - Name of the storage

Returns:

Name followed by storage separator

getStorageTable

public static org.apache.hadoop.hive.ql.metadata.Table getStorageTable(String storageTableNamePrefix,
                                                                       org.apache.hadoop.hive.ql.metadata.Hive client,
                                                                       org.apache.hadoop.hive.ql.metadata.Table parent,
                                                                       StorageTableDesc crtTbl)
                                                                throws org.apache.hadoop.hive.ql.metadata.HiveException

Get the storage table descriptor for the given parent table.

Parameters:

storageTableNamePrefix - Storage table prefix based on update period

client - The metastore client

parent - Is either Fact or Dimension table

crtTbl - Create table info

Returns:

Table describing the storage table

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

addPartitions

public List<org.apache.hadoop.hive.ql.metadata.Partition> addPartitions(org.apache.hadoop.hive.ql.metadata.Hive client,
                                                                        String factOrDimTable,
                                                                        UpdatePeriod updatePeriod,
                                                                        List<StoragePartitionDesc> storagePartitionDescs,
                                                                        Map<Map<String,String>,Storage.LatestInfo> latestInfos,
                                                                        String tableName)
                                                                 throws org.apache.hadoop.hive.ql.metadata.HiveException

Add given partitions in the underlying hive table and update latest partition links

Parameters:

client - hive client instance

factOrDimTable - fact or dim name

updatePeriod - update period of partitions.

storagePartitionDescs - all partitions to be added

latestInfos - new latest info. atleast one partition for the latest value exists for each part column

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

updatePartition

public void updatePartition(org.apache.hadoop.hive.ql.metadata.Hive client,
                            String fact,
                            org.apache.hadoop.hive.ql.metadata.Partition partition)
                     throws org.apache.hadoop.hive.metastore.api.InvalidOperationException,
                            org.apache.hadoop.hive.ql.metadata.HiveException

Update existing partition

Parameters:

client - hive client instance

fact - fact name

partition - partition to be updated

Throws:

org.apache.hadoop.hive.metastore.api.InvalidOperationException

org.apache.hadoop.hive.ql.metadata.HiveException

updatePartitions

public void updatePartitions(String storageTable,
                             org.apache.hadoop.hive.ql.metadata.Hive client,
                             String fact,
                             List<org.apache.hadoop.hive.ql.metadata.Partition> partitions)
                      throws org.apache.hadoop.hive.metastore.api.InvalidOperationException,
                             org.apache.hadoop.hive.ql.metadata.HiveException

Update existing partitions

Parameters:

client - hive client instance

fact - fact name

partitions - partitions to be updated

Throws:

org.apache.hadoop.hive.metastore.api.InvalidOperationException

org.apache.hadoop.hive.ql.metadata.HiveException

dropPartition

public void dropPartition(org.apache.hadoop.hive.ql.metadata.Hive client,
                          String storageTableName,
                          List<String> partVals,
                          Map<String,Storage.LatestInfo> updateLatestInfo,
                          Map<String,String> nonTimePartSpec)
                   throws org.apache.hadoop.hive.ql.metadata.HiveException

Drop the partition in the underlying hive table and update latest partition link

Parameters:

client - The metastore client

storageTableName - TableName

partVals - Partition specification

updateLatestInfo - The latest partition info if it needs update, null if latest should not be updated

nonTimePartSpec -

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException