Python - Scheduled Task Execution

Cause#

Recently, I was writing an automatic check-in script and wanted to execute it at 12 o'clock every day on the server. This led to a lot of similar timing issues.

This article will provide an answer.

Executing the Program Every xx Seconds#

This is basic and can be achieved using time.sleep. Here is the program:

# Print every 5 seconds
from datetime import datetime
from time import sleep


def print_message():
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    print("https://soapffz.com")


while True:
    print_message()
    sleep(5)

From this, we can try our goal:

from datetime import datetime
from time import sleep


def print_message():
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    print("https://soapffz.com")


while True:
    while True:
        now = datetime.now()
        if now.hour == 0 and now.minute == 0:
            break
        sleep(20)
    print_message()

Although this method is simple and straightforward, the while loop will continuously consume CPU resources, so it is generally not recommended.

threading.Timer Timer#

The above method is too aggressive, so let's try a more elegant method.

There is a Timer class in the threading module. It starts a new thread to execute the scheduled task, so it is a non-blocking function.

If you are using multiple threads, you need to consider thread safety. In that case, you can use the threading.Timer module.

The Timer() function accepts three parameters:

Timer(10, task, ()).start()
Delay in seconds before executing the task
The task to be executed, i.e., the function
Arguments to be passed to the function (tuple)

Delayed Execution of the Program for xx Seconds, Only Once#

from datetime import datetime
from time import sleep
from threading import Timer


def print_message():
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    print("https://soapffz.com")


Timer(10, print_message).start()

Executing the Program Every xx Seconds, Continuously#

This is achieved by moving the Timer() statement into the function being executed.

from datetime import datetime
from time import sleep
from threading import Timer


def print_message():
    Timer(10, print_message).start()
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    print("https://soapffz.com")


print_message()

Similarly, if you need to execute the program every 24 hours (regardless of the starting time), you just need to change the time in the Timer function to 86400.

schedule Module for Scheduled Task Execution#

Official documentation: https://schedule.readthedocs.io/en/stable/

This is a lightweight library for scheduling tasks. Let's look at a few examples:

from datetime import datetime
from time import sleep
import schedule


def print_message():
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    print("https://soapffz.com")

schedule.every(15).seconds.do(print_message)
schedule.every().minutes.do(print_message)
schedule.every().hour.do(print_message)
schedule.every().day.at("10:30").do(print_message)
schedule.every(5).to(10).days.do(print_message)
schedule.every().wednesday.at("13:15").do(print_message)

while True:
    schedule.run_pending()
    sleep(1)

The above code means:

Execute the task every 15 seconds
Execute the task every minute
Execute the task every hour
Execute the task at 10:30 every day
Execute the task every 5 to 10 days
Execute the task at this time every Wednesday
run_pending: run all pending tasks

Note that there is one duplicate print in the output, not sure why~ you can discuss it in the comments.

What is schedule.run_pending()?

schedule is actually just a timer. In the while True loop, schedule.run_pending() keeps schedule running and queries the above tasks. In the tasks, you can set different times to run. It is similar to setting a cron job in Linux.

Therefore, schedule has certain limitations and can only be used to execute small-scale scheduled tasks. What are the limitations of schedule?

The function job that needs to be scheduled should not be an infinite loop type, which means that the thread should have an exit point after execution. This is because if the thread freezes, it will be a very tricky problem; and the next scheduled task will start a new thread, which can become a disaster if the number of executions increases.

If the time interval set by schedule is shorter than the execution time of job, thread accumulation will still occur, which means that if the execution time of job is 1 hour, but the scheduled task is set to run every 5 minutes, threads will accumulate.

There is also a library called sched.scheduler, which has similar usage to Timer(). You can explore it if you are interested.

Summary#

Finally, I compared the first method using datetime + sleep with the third method using schedule.

Server configuration: 1 core CPU, 1838MB memory

The consumption of the first method during normal operation is as follows:

The consumption of the third method during normal operation is as follows:

Although the third method seems to have an advantage during normal operation, when the specified time is reached

The former is stable and does not consume much CPU, while the latter goes crazy, occupying all the memory and CPU, and sending me a lot of messages.

But I checked the program and there is no problem, so personally I recommend using the former, which is a bit more secure. Double loops can be fatal.

That's it, end of this article.

References: