This project has retired. For details please refer to its Attic page.
Lens –

Lens session configuration


The configuration parameters and their default values
No. Property Name Default Value Description
1 hive.metastore.batch.retrieve.max 100 Maximum number of objects (tables/partitions) can be retrieved from metastore in one batch. The higher the number, the less the number of round trips is needed to the Hive metastore server, but it may also cause higher memory requirement at the client side.
2 hive.metastore.batch.retrieve.table.partition.max 500 Maximum number of table partitions that metastore internally retrieves in one batch.
3 hive.metastore.client.connect.retry.delay 1 Number of seconds for the client to wait between consecutive connection attempts
4 hive.metastore.client.socket.timeout 20 MetaStore Client socket timeout in seconds
5 hive.metastore.connect.retries 5 Number of retries while opening a connection to metastore
6 hive.metastore.failure.retries 3 Number of call retries when Hive Metastore calls fail with Thrift errros
7 hive.metastore.uris The hive metastore server URI that the lens server is talking to
8 lens.cube.query.completeness.threshold 100 The query will fail if data completeness is less than the set threshold given that the flag "lens.cube.query.fail.if.data.partial" is set as true
9 lens.cube.query.disable.aggregate.resolver false Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.
10 lens.cube.query.disable.auto.join false Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.
11 lens.cube.query.fail.if.data.partial true Whether to fail the query of data is partial
12 lens.cube.query.promote.select.togroupby true Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.
13 lens.query.add.insert.overwrite true Prefix query with insert overwrite clause if the query is persistent. User can disable if user gave the clause himself.
14 lens.query.cancel.on.timeout true Specifies whether to attempt cancellation of a query whose execution takes longer than the timeout value specified while submitting the query for execution. The default value is true.
15 lens.query.enable.mail.notify false When a query ends, whether to notify the submitter by mail or not.
16 lens.query.enable.metrics.per.query false Generates gauge metrics for each query to measure time taken with unique id appended for each query. Should be enabled only for performance measurements. Should not be enabled in day to day production environment.
17 lens.query.enable.persistent.resultset false Whether to enable persistent resultset for queries. When enabled, server will fetch results from driver, custom format them if any and store in a configured location. The file name of query output is queryhandle-id, with configured extensions
18 lens.query.enable.persistent.resultset.indriver true Whether the result should be persisted by driver. Currently only HiveDriver persists the results in a HDFS location.
19 lens.query.hdfs.output.path hdfsout The directory under the parent result directory, in which HiveDriver will persist the results, if persisting by driver is enabled. This directory should exist and should have world writable permissions sothat all users will be able put query outputs here.
20 lens.query.http.notification.mediatype application/json This is the media type for Query Http notifications. Accepted types are "application/json" and "application/xml". The default value is "application/json"
21 lens.query.http.notification.type.FINISHED false Setting this property to true will enable query FINISHED notifications which includes SUCCESSFUL, FAILED and CANCELLED queries. The notification will have eventtype = "FINISHED", eventtime = long event time and query = org.apache.lens.api.query.LensQuery instance. The mediatype for eventtype and eventtime will be TEXT/PLAIN and the mediatype for query will be based on property lens.query.http.notification.mediatype. Default value of this property is false.
22 lens.query.http.notification.urls These are the http end points for Query http notifications. Users can specify more than one comma separated end points for a query. Url parameter values that include special characters should be encoded. Please note that if this property is not set, no http notification will be sent out by lens server for the query.
23 lens.query.output.charset.encoding UTF-8 The charset encoding for formatting query result. It supports all the encodings supported by java.io.OutputStreamWriter.
24 lens.query.output.compression.codec org.apache.hadoop.io.compress.GzipCodec The codec used to compress the query output, if compression is enabled
25 lens.query.output.enable.compression false Whether to compress the query result output
26 lens.query.output.file.extn .csv The extension name for the persisted query output file. If file is compressed, the extension from compression codec will be appended to this extension.
27 lens.query.output.footer The value of custom footer that should be written, if any. This footer will be added in formatting driver persisted results.
28 lens.query.output.formatter The query result output formatter for the query. If no value is specified, then org.apache.lens.lib.query.FileSerdeFormatter will be used to format in-memory result sets, org.apache.lens.lib.query.FilePersistentFormatter will be used to format driver persisted result sets.
29 lens.query.output.header The value of custom header that should be written, if any. If no value column names will be used as header.
30 lens.query.output.write.footer false Whether to write footer as part of query result. When enabled, total number of rows will be written as part of header.
31 lens.query.output.write.header false Whether to write header as part of query result formatting. When enabled the user given header will be added in case of driver persisted results, and column names chosen will be added as header for in-memory results.
32 lens.query.prefetch.inmemory.resultset true When set to true, specified number of rows of result set will be pre-fetched if the result set is of type InMemoryResultSet and query execution is not asynchronous i.e. query should be launched with operation as EXECUTE_WITH_TIMEOUT. Suggested usage of this property: It can be used by client to stream as well as persist results in server for queries that finish fast and produce results with fewer rows (should be less than number of rows pre-fetched). Note that the results are streamed to the client early, without waiting for persistence to finish. Default value of this property is true.
33 lens.query.prefetch.inmemory.resultset.rows 100 Specifies the number of rows to pre-fetch when lens.query.prefetch.inmemory.resultset is set to true. Default value is 100 rows.
34 lens.query.result.email.cc When query ends, the result/failure reason will be sent to the user via email. The mail would be cc'ed to the addresses provided in this field.
35 lens.query.result.fs.read.url Http read URL for FileSystem on which result is present, if available. For example webhdfs as http read url should http://host:port/webhdfs/v1. Currently we support only webhdfs url as the http url for HDFS file system
36 lens.query.result.output.dir.format The format of the output if result is persisted in hdfs. The format should be expressed in HQL.
37 lens.query.result.output.serde org.apache.lens.lib.query.CSVSerde The default serde class name that should be used by org.apache.lens.lib.query.FileSerdeFormatter for formatting the output
38 lens.query.result.parent.dir file:///tmp/lensreports The directory for storing persisted result of query. This directory should exist and should have writable permissions by lens server
39 lens.query.result.size.format.threshold 10737418240 The maximum allowed size of the query result. If exceeds, no server side formatting would be done.
40 lens.query.result.split.multiple false Whether to split the result into multiple files. If enabled, each file will be restricted to max rows configured. All the files will be available as zip.
41 lens.query.result.split.multiple.maxrows 100000 The maximum number of rows allowed in each file, when splitting the result into multiple files is enabled.
42 lens.query.timeout.millis 86400000 The runtime(millis) of the query after which query will be timedout and cancelled. Default is 1 day.
43 lens.session.aux.jars List of comma separated jar paths, which will added to the session
44 lens.session.cluster.user Session level config which will determine which cluster user will access hdfs
45 lens.session.loggedin.user The username used to log in to lens. e.g. LDAP user
46 lens.session.metastore.exclude.cubetables.from.nativetables true Exclude cube related tables when fetching native tables