Schedulers, and Why We Built Ours From Scratch
05/11/2017 • by Nicole Hitner • 0 comments
We didn’t have to, you know. A scheduler service just performs automated tasks in the background, and most operating systems come with one built right in. You can tell Microsoft’s Task Scheduler to run a report and email the output to CMO Bob on Monday mornings at 10 AM, and it will quietly go about making that happen while you turn the rest of your CPU’s attention to other activities. We could have easily taken this approach and hooked into whatever scheduler was available to us for a given installation. But we didn’t.
The reason we didn’t is because generic schedulers are like Swiss army knives: they can perform a wide variety of tasks, but they’re usually not the best possible tool for any given one. We wanted Exago’s scheduler to be optimized for executing reports and to mesh with the application’s other processes. And you know what they say: if you want something done right, do it yourself.
While the Exago Scheduler is that—an asynchronous report execution engine—it’s also a great deal more. Sure, it can execute a report the same time each week and store the output as a temporary file, but it can also be configured to send those reports somewhere, like to an inbox or folder. In addition, it comes packaged with a remote execution application and so can be installed on any number of servers to distribute immediate report executions across the network. It also handles exceptions intelligently and comes with a queueing program designed to improve the load balancing of scheduled tasks. It is, in short, a specialized tool built specifically for Exago and its users’ purposes.
Designed to Scale
One of Exago Scheduler’s defining characteristics is that it is separate from the main application. Because businesses using Exago BI are constantly expanding and acquiring new users, it was crucial that the Scheduler be able to handle an ever-expanding workload. To this end, the Scheduler was decoupled from the application UI so that one could be installed without the other, helping to facilitate horizontal scaling.
Above: The Exago Web Application, Web Service API, and Scheduling Service are installed separately. (On a Linux system, the Scheduler is installed as a standalone application managed by a series of scripts.)
As more users access the application and the number of scheduled tasks increases, all the system administrator needs to do is add a new Scheduler server to spread out the workload. This is especially easy to do on cloud services like AWS, as the Scheduler just becomes part of the base image loaded onto any new servers added to your cluster. From there, jobs are distributed in a round-robin fashion among the servers, unless a program called the Scheduler Queue is deployed to further optimize load balancing. The Scheduler Queue creates a repository of report schedules that get assigned to available servers at the appointed runtime, improving performance and maximizing efficiency.
Built to Perform
Indeed, the Exago Scheduler is more than just a scheduler—it’s a remote execution engine. If you have a cluster of servers waiting in the wings for their scheduled jobs to come due for execution, there’s a good chance they’re sitting idle most of the time. We decided to take advantage of that latent processing power by using the scheduler to load balance immediate report executions as well. With remote execution turned on, the web server can focus exclusively on communicating with the client, leaving execution to the remote server cluster otherwise occupied with schedules.
Enabling remote execution goes a long way toward improving Exago’s performance, especially as the number of users grows and more reports are being executed concurrently.
In addition to scheduling and remote execution, the Scheduler is designed to kick off subtasks once a report has been executed.
Email: The Scheduler can be set directly from the UI to email scheduled reports to any number of recipients.
Batch Reporting: The Scheduler can be set to refer to a table of email addresses stored in a database and use that table to send a unique filtered version of the report based on the recipients’ credentials. Batch reporting can also be used purely as an emailing tool, with no reports attached.
Saving to a Repository: The Scheduler can be set to save report output to a specified directory or folder if desired.
Exceptional Exception Handling
The Exago Scheduler, therefore, notifies users when an execution has resulted in an abnormal end (or has, in industry speak, “abended”), prompting them to seek out and resolve the root of the issue. Generic schedulers do not typically do this. Instead, they soak up processing power in a continuous effort to execute that report until explicitly instructed to stop. Not only does the Exago Scheduler cancel failed executions, it also disambiguates situations like No Data Qualified reports, which might otherwise be interpreted as errors. If, for example, a weekly sales report is scheduled for execution every Friday afternoon, but there are no sales for a given week, the Scheduler will run the blank report and provide a No Data Qualified message instead of abending the task.
So while the Exago Scheduler does require separate installation and configuration, its utility to the reporting process more than makes up for this inconvenience. To learn more about the Exago Scheduler’s capabilities, including batch reporting and scheduling to repositories, visit our Knowledge Base.