To schedule ETL processes on Linux, we always use the cron scheduler. The task to do this is very easy, while understanding the syntax of the cron schedules is a bit more complicated.

To add a new schedule, you need to edit the crontab file; to do this, type the following command from the Linux command line:

crontab –e

Depending on the distribution, you can also use some graphical tools to do the same, but as I always suggest, it is better to learn the command-line commands to give you complete freedom and portability.

To schedule a process, you need to specify the time at which the command will run. To do this, there is a particular syntax that lets you specify minutes, hours, the month date, and the month year. The format of any of these items is as follows:

You can follow some simple rules to specify ranges or multiple values for any of the preceding items, summarized as follows:

So, let's suppose that we want to schedule a job in the <book_samples>/sample1 directory called export-job.kjb to execute at every half an hour, at 20 and 50 minutes past the hour every day, except on weekends. To do this, you need to add a new schedule by typing the following command:

20,50 * * * 1-5 <pdi_home>/kitchen.sh –file <books_samples>/export-job.kjb

After you have typed the command, save and quit the crontab editor by pressing the Esc key; then type :wq, and then press the Enter key. The schedule will immediately be available. As you can see, in this case, you need to specify the complete path to the PDI scripts and to the export-job.kjb job. Of course, scheduling a Pan script is just a matter of changing the script name.

Further references about how to schedule tasks with the cron scheduler can be found on Wikipedia at http://en.wikipedia.org/wiki/Cron.

Go to Control Panel | Administrative Tools | Task Scheduler.

Select Create a Basic Task from the menu on the right.

The Create Basic Task Wizard dialog will be displayed. Enter the task name (for example, Example PDI job). Click on Next.

Select the schedule you want to apply. You can either select the schedule based on the calendar or on an event. Let us suppose our schedule is based on the calendar and we want our program to be scheduled daily. Select the Daily radio button and click on Next.

Insert the time you want the event to fire and its recurrence multiplicity (Recur every). We want, for example, the event to fire everyday at 4.00 A.M. Click on Next.

Select the action you want to accomplish; in our case, we select Start a program. Click on Next.

In the Program/script field, insert the full path and the name of the script you want to start (Kitchen.bat or Pan.bat). In the Add arguments field, insert the usual information you pass along to the Kitchen or Pan script to start your job or transformation properly. Click on Next.

At the end, a summary dialog will open showing you all the parameters you inserted to completely define your schedule.

Click on Finish.

Your job will now appear in the list of waiting jobs.

The at command works the same as cron, so it becomes very easy to schedule our job with Kitchen or our transformation with Pan.

Let's suppose that we want to schedule a job in the <book_samples>/sample1 directory called export-job.kjb to be executed everyday at 8:00 AM with the exception of the weekends. To do this, you need to add a new schedule by typing the following command:

at 8:00 /every:M,T,W,Th,F <pdi_home>\Kitchen.bat /file:<books_samples>\export-job.kjb

After you type the command, the schedule immediately becomes available. As you can see, in this case, you need to specify the complete path to the PDI scripts and to the export-job.kjb job.