Linux: Using flock to ensure only one instance of the script is running
đ Wiki page | đ Last updated: Oct 31, 2022It's very easy to create race conditions by accidentally running multiple instances of the script at the same time.
You can use this method to prevent that from happening simply by using the flock
command (part of the standard util-linux
package).
tl;dr
Example script:
#!/bin/bash
exec 7>/tmp/mylock
flock 7 || exit 1
echo "[$$] Starting..."
sleep 15
echo "[$$] Done..."
Example output (with three instances of the script running at the same time):
[1179045] Starting...
[1179045] Done...
[1179046] Starting...
[1179046] Done...
[1179047] Starting...
[1179047] Done...
How does it work?
First, let's create a simple script that will simulate some processing:
#/bin/bash
echo "[$$] Starting..."
sleep 15
echo "[$$] Done..."
Save that into a file named test
and make it executable:
chmod +x test
Now, let's try to run multiple instances of the script:
./test & ./test &
(we're using ampersand at the end of each command to put it into the background)
[1183086] Starting...
[1183087] Starting...
[1183086] Done...
[1183087] Done...
As we can see from the output, there is an overlap between the processing phase of those two scripts.
So, what can we do to prevent this from happening?
Using flock outside of the script
The first method is to just use flock
as an outside wrapper for our script:
flock /tmp/test_lock -c ./test &
We're using the flock
command to obtain the exclusive lock on the /tmp/test_lock
file (we could specify a file, directory, or a descriptor - more on that later), and we're specifying the command to run once the lock has been acquired with -c
.
If we try to run multiple instances of this whole command:
flock /tmp/test_lock -c ./test & flock /tmp/test_lock -c ./test &
The result now should be as expected:
[1183024] Starting...
[1183024] Done...
[1183040] Starting...
[1183040] Done...
But if you try to run the test
script in another terminal at the same, there'll be an overlap again.
This can easily happen in practice (for example, the command gets manually executed in the shell, while another instance is running from the crontab).
Using flock inside the script
To prevent that from happening, we need to put flock
inside our script (and somehow tie it to the duration of our script).
A simple way to that is by using file descriptors:
exec 7>/tmp/mylock
We're using exec
to assign the file descriptor (for reading that file) to our process. You can think of a file descriptor as a handle to access that file. 7 is an arbitrary number I decided to use here, but we could use any non-negative number (as long as it's not in use - standard file descriptors are 0 - STDIN, 1 - STDOUT, and 2 - STDERR).
Unless we explicitly close it, this descriptor will typically stay attached to our process until the process exits.
Now we just need to tell flock
to use that file descriptor:
flock 7 || exit 1
Without any additional options, flock
will typically block until the lock is acquired (exit
is just a safeguard here if something goes wrong).
Full script:
#!/bin/bash
exec 7>/tmp/mylock
flock 7 || exit 1
echo "[$$] Starting..."
sleep 15
echo "[$$] Done..."
We can specify the timeout with -w
option:
flock -w 500 7 || exit 1
This tells flock
to wait 500 seconds before giving up.
If you want to exit immediately instead of waiting for the lock, you can use -n
option.
flock -n 7 || exit 1
Alternatives
Alternatives include:
lockfile
(provided byprocmail
)- manually creating and removing a pid file (which usually involves a race condition in the
if
statement) - directory variant of the previous alternative (which is using atomic
mkdir
operation)
I'll update this page with more examples.
Ask me anything / Suggestions
If you find this site useful in any way, please consider supporting it.