org.apache.lens.server.scheduler

Class SchedulerServiceImpl

java.lang.Object

org.apache.hive.service.AbstractService

org.apache.hive.service.CompositeService

org.apache.lens.server.BaseLensService

org.apache.lens.server.scheduler.SchedulerServiceImpl

All Implemented Interfaces:

Externalizable, Serializable, org.apache.hive.service.Service, HealthCheckable, LensService, SchedulerService, SessionValidator

public class SchedulerServiceImpl
extends BaseLensService
implements SchedulerService

This class handles all the scheduler operations.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SchedulerServiceImpl.WaitingInstanceHandler

Nested classes/interfaces inherited from class org.apache.lens.server.BaseLensService

BaseLensService.SessionContext

Nested classes/interfaces inherited from class org.apache.hive.service.CompositeService

org.apache.hive.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hive.service.Service

org.apache.hive.service.Service.STATE

Field Summary

Fields inherited from interface org.apache.lens.server.api.scheduler.SchedulerService

NAME

Constructor Summary

Constructors

Constructor and Description
SchedulerServiceImpl(org.apache.hive.service.cli.CLIService cliService) Instantiates a new scheduler service.

Method Summary

Modifier and Type	Method and Description
void	deleteJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle) Delete a job.
void	expireJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle) End a job by specifying an expiry time.
AlarmService	getAlarmService()
List<SchedulerJobHandle>	getAllJobs(String user, SchedulerJobState state, Long start, Long end) Get all jobs matching the filter parameters.
Collection<SchedulerJobStats>	getAllJobStats(String state, String user, String jobName, long startTime, long endTime)
HealthStatus	getHealthStatus() Returns the health status of the service.
SchedulerJobInstanceInfo	getInstanceDetails(SchedulerJobInstanceHandle instanceHandle) Instance details for an instance.
XJob	getJobDefinition(SchedulerJobHandle jobHandle) Returns the definition of a job.
SchedulerJobInfo	getJobDetails(SchedulerJobHandle jobHandle) Returns the details of a job.
List<SchedulerJobInstanceInfo>	getJobInstances(SchedulerJobHandle jobHandle, Long numResults) Returns handles for last numResults instances for the job.
SchedulerJobStats	getJobStats(SchedulerJobHandle handle, String state, long startTime, long endTime) Returns stats for a job.
QueryExecutionService	getQueryService()
SchedulerDAO	getSchedulerDAO()
SchedulerEventListener	getSchedulerEventListener()
SchedulerQueryEventListener	getSchedulerQueryEventListener()
void	init(org.apache.hadoop.hive.conf.HiveConf hiveConf)
boolean	killInstance(LensSessionHandle sessionHandle, SchedulerJobInstanceHandle instanceHandle) Kills a running job instance.
LensSessionHandle	openSessionAsUser(String user) Create session as user for scheduling the job with no auth.
void	rerunInstance(LensSessionHandle sessionHandle, SchedulerJobInstanceHandle instanceHandle) Reruns a failed/killed/completed job instance.
void	resumeJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle) Resume a job from a given time.
void	scheduleJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle) Schedule a job.
void	setQueryService(QueryExecutionService queryService)
void	start() How the restarts are handled? Get all the instances with state Running or New.
void	stop()
SchedulerJobHandle	submitAndScheduleJob(LensSessionHandle sessionHandle, XJob job) Submit a job and also schedule it.
SchedulerJobHandle	submitJob(LensSessionHandle sessionHandle, XJob job) Submit a job.
void	suspendJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle) Suspend a job.
void	updateJob(LensSessionHandle sessionHandle, SchedulerJobHandle jobHandle, XJob newJobDefinition) Update a job with new definition.

Methods inherited from class org.apache.lens.server.BaseLensService

acquire, acquire, closeSession, getCliService, getHiveSessionHandle, getLensConf, getLensConf, getNumberOfSessions, getServerDomain, getSession, getSessionInfo, getSessionManager, getValidPath, openSession, openSession, prepareStopping, readExternal, release, release, removePrefixBeforeURI, restoreSession, validateSession, writeExternal

Methods inherited from class org.apache.hive.service.CompositeService

getServices

Methods inherited from class org.apache.hive.service.AbstractService

getHiveConf, getName, getServiceState, getStartTime, register, unregister

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lens.server.api.SessionValidator

validateSession

Constructor Detail

SchedulerServiceImpl

public SchedulerServiceImpl(org.apache.hive.service.cli.CLIService cliService)
                     throws LensException

Instantiates a new scheduler service.

Parameters:

cliService - the cli service

Throws:

LensException

Method Detail

init

public void init(org.apache.hadoop.hive.conf.HiveConf hiveConf)

Specified by:

init in interface org.apache.hive.service.Service

Overrides:

init in class org.apache.hive.service.CompositeService

start

public void start()

How the restarts are handled? Get all the instances with state Running or New. If They are running then check the query status. If Query is finished, take query parameters and update the instance. If query is still running then do nothing. If the state is New then Kill the instance and rerun it.

Specified by:

start in interface org.apache.hive.service.Service

Overrides:

start in class org.apache.hive.service.CompositeService

stop

public void stop()

Specified by:

stop in interface org.apache.hive.service.Service

Overrides:

stop in class org.apache.hive.service.CompositeService

getHealthStatus

public HealthStatus getHealthStatus()

Returns the health status of the service.

Specified by:

getHealthStatus in interface HealthCheckable

Returns:

openSessionAsUser

public LensSessionHandle openSessionAsUser(String user)
                                    throws LensException

Create session as user for scheduling the job with no auth.

Specified by:

openSessionAsUser in interface SchedulerService

Returns:

LensSessionHandle

Throws:

LensException

getAllJobs

public List<SchedulerJobHandle> getAllJobs(String user,
                                           SchedulerJobState state,
                                           Long start,
                                           Long end)
                                    throws LensException

Description copied from interface: SchedulerService

Get all jobs matching the filter parameters.

Specified by:

getAllJobs in interface SchedulerService

Returns:

List of all job handles matching the parameters.

Throws:

LensException

submitJob

public SchedulerJobHandle submitJob(LensSessionHandle sessionHandle,
                                    XJob job)
                             throws LensException

Submit a job.

Specified by:

submitJob in interface SchedulerService

Parameters:

sessionHandle - handle for this session.

job - job to be submitted.

Returns:

unique id for the submitted job.

Throws:

LensException - the lens exception

scheduleJob

public void scheduleJob(LensSessionHandle sessionHandle,
                        SchedulerJobHandle jobHandle)
                 throws LensException

Schedule a job.

Specified by:

scheduleJob in interface SchedulerService

Parameters:

sessionHandle - handle for the current session.

jobHandle - handle for the job to be scheduled.

Throws:

LensException - the lens exception

submitAndScheduleJob

public SchedulerJobHandle submitAndScheduleJob(LensSessionHandle sessionHandle,
                                               XJob job)
                                        throws LensException

Description copied from interface: SchedulerService

Submit a job and also schedule it.

Specified by:

submitAndScheduleJob in interface SchedulerService

Parameters:

sessionHandle - handle for the session.

job - job definition.

Returns:

unique id of the job which is submitted and scheduled.

Throws:

LensException - the lens exception

getJobDefinition

public XJob getJobDefinition(SchedulerJobHandle jobHandle)
                      throws LensException

Returns the definition of a job.

Specified by:

getJobDefinition in interface SchedulerService

Parameters:

jobHandle - handle for the job

Returns:

job definition

Throws:

LensException - the lens exception

getJobDetails

public SchedulerJobInfo getJobDetails(SchedulerJobHandle jobHandle)
                               throws LensException

Returns the details of a job. Details may contain extra system information like id for the job.

Specified by:

getJobDetails in interface SchedulerService

Parameters:

jobHandle - handle for the job

Returns:

job details for the job

Throws:

LensException - the lens exception

updateJob

public void updateJob(LensSessionHandle sessionHandle,
                      SchedulerJobHandle jobHandle,
                      XJob newJobDefinition)
               throws LensException

Update a job with new definition.

Updates will be applied only for newer instances. Running instances will be running with old definition

Specified by:

updateJob in interface SchedulerService

jobHandle - handle for the job which you want to update.

Throws:

LensException - the lens exception

expireJob

public void expireJob(LensSessionHandle sessionHandle,
                      SchedulerJobHandle jobHandle)
               throws LensException

End a job by specifying an expiry time.

Specified by:

expireJob in interface SchedulerService

Parameters:

sessionHandle - handle for the current session.

jobHandle - handle for the job

Throws:

LensException - the lens exception

suspendJob

public void suspendJob(LensSessionHandle sessionHandle,
                       SchedulerJobHandle jobHandle)
                throws LensException

Suspend a job.

If the job is not in scheduled state, it will return true. Once a job is suspended, no further instances of that job will run. Any running instances of that job will continue normally.

Specified by:

suspendJob in interface SchedulerService

Parameters:

sessionHandle - handle for the current session.

jobHandle - handle for the job

Throws:

LensException - the lens exception

resumeJob

public void resumeJob(LensSessionHandle sessionHandle,
                      SchedulerJobHandle jobHandle)
               throws LensException

Resume a job from a given time.

Specified by:

resumeJob in interface SchedulerService

Parameters:

sessionHandle - handle for the session.

jobHandle - handle for the job

Throws:

LensException - the lens exception

deleteJob

public void deleteJob(LensSessionHandle sessionHandle,
                      SchedulerJobHandle jobHandle)
               throws LensException

Delete a job.

Specified by:

deleteJob in interface SchedulerService

Parameters:

sessionHandle - handle for the session.

jobHandle - handle for the job

Throws:

LensException - the lens exception

getAllJobStats

public Collection<SchedulerJobStats> getAllJobStats(String state,
                                                    String user,
                                                    String jobName,
                                                    long startTime,
                                                    long endTime)
                                             throws LensException

Specified by:

getAllJobStats in interface SchedulerService

Parameters:

state - filter for status, if specified only jobs in that state will be returned, if null no entries will be removed from result

user - filter for user who submitted the job, if specified only jobs submitted by the given user will be returned, if not specified no entries will be removed from result on basis of userName

jobName - filter for jobName, if specified only the jobs with name same as given name will be considered , else no jobs will be filtered out on the basis of name.

startTime - if specified only instances with scheduleTime after this time will be considered.

endTime - if specified only instances with scheduleTime before this time will be considered.

Returns:

A collection of stats per job

Throws:

LensException

getJobStats

public SchedulerJobStats getJobStats(SchedulerJobHandle handle,
                                     String state,
                                     long startTime,
                                     long endTime)
                              throws LensException

Returns stats for a job.

Specified by:

getJobStats in interface SchedulerService

Parameters:

handle - handle for the job

state - filter for status, if specified only jobs in that state will be returned, if null no entries will be removed from result

startTime - if specified only instances with scheduleTime after this time will be considered.

endTime - if specified only instances with scheduleTime before this time will be considered.

Throws:

LensException - the lens exception

rerunInstance

public void rerunInstance(LensSessionHandle sessionHandle,
                          SchedulerJobInstanceHandle instanceHandle)
                   throws LensException

Reruns a failed/killed/completed job instance.

If the instance is not in a terminal state, then this operation will be a no-op and will return false.

Specified by:

rerunInstance in interface SchedulerService

Parameters:

sessionHandle - handle for the session.

instanceHandle - handle for the instance

Throws:

LensException - the lens exception

getJobInstances

public List<SchedulerJobInstanceInfo> getJobInstances(SchedulerJobHandle jobHandle,
                                                      Long numResults)
                                               throws LensException

Returns handles for last numResults instances for the job.

Specified by:

getJobInstances in interface SchedulerService

Parameters:

jobHandle - handle for the job

numResults - - number of results to be returned, default 100.

Returns:

list of instance ids for the job

Throws:

LensException - the lens exception

killInstance

public boolean killInstance(LensSessionHandle sessionHandle,
                            SchedulerJobInstanceHandle instanceHandle)
                     throws LensException

Description copied from interface: SchedulerService

Kills a running job instance.

If the job instance is already completed or not in running state, this will be a no-op and will return false.

Specified by:

killInstance in interface SchedulerService

Parameters:

sessionHandle - handle for the session.

instanceHandle - handle for the instance

Throws:

LensException - the lens exception

getInstanceDetails

public SchedulerJobInstanceInfo getInstanceDetails(SchedulerJobInstanceHandle instanceHandle)
                                            throws LensException

Instance details for an instance.

Specified by:

getInstanceDetails in interface SchedulerService

Parameters:

instanceHandle - handle for the instance.

Returns:

details for the instance.

Throws:

LensException - the lens exception

getSchedulerDAO

@Generated(value="lombok")
public SchedulerDAO getSchedulerDAO()

getQueryService

@Generated(value="lombok")
public QueryExecutionService getQueryService()

setQueryService

@Generated(value="lombok")
public void setQueryService(QueryExecutionService queryService)

getSchedulerEventListener

@Generated(value="lombok")
public SchedulerEventListener getSchedulerEventListener()

getSchedulerQueryEventListener

@Generated(value="lombok")
public SchedulerQueryEventListener getSchedulerQueryEventListener()

getAlarmService

@Generated(value="lombok")
public AlarmService getAlarmService()