Puppet File Sync Not Working – LOCK_FAILURE

I had a recent issue where Puppet was not properly syncing code from the code-staging directory to the code directory.  I verified it was pulling the new code from my Git repository to code-staging without issue.  However, file-sync was not pushing the new code to the code directory.

Here is what I was seeing in the /var/log/puppetlabs/puppetserver/puppetserver.log

2017-06-26 11:08:49,026 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:08:54,051 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:08:59,077 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:04,103 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:09,129 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:14,155 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.

I had no idea what this meant, and I wasn’t sure how to resolve it so I took a snapshot of my Puppet Master VM and tried a few things.

The first thing I tried was going to the directory indicated and taking a look:

ll /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/production.git/
total 44
drwxr-xr-x 7 pe-puppet pe-puppet 4096 Jun 26 11:03 ./
drwxr-xr-x 3 pe-puppet pe-puppet 4096 Apr 25 2016 ../
drwxr-xr-x 2 pe-puppet pe-puppet 4096 Apr 25 2016 branches/
-rw-r—– 1 pe-puppet pe-puppet 307 Jun 26 11:03 config
-rw-r—– 1 pe-puppet pe-puppet 148 Jun 26 10:28 FETCH_HEAD
-rw-r–r– 1 pe-puppet pe-puppet 23 Apr 25 2016 HEAD
drwxr-xr-x 2 pe-puppet pe-puppet 4096 Apr 25 2016 hooks/
drwxr-xr-x 3 pe-puppet pe-puppet 4096 Apr 25 2016 logs/
drwxr-xr-x 4 pe-puppet pe-puppet 4096 Jun 26 10:28 objects/
drwxr-xr-x 4 pe-puppet pe-puppet 4096 Apr 25 2016 refs/
-rw-r—– 1 pe-puppet pe-puppet 41 Jun 26 10:28 synced-commit

/opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git/production.git had the same contents but for one file:

synced-commit.lock

I wasn’t sure this file belonged there, so I  removed it.  Once I did that, the file-sync service stopped throwing errors and successfully synced my files!

Hope this helps!

2 thoughts on “Puppet File Sync Not Working – LOCK_FAILURE

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s