Category Archives: Puppet

Using the Puppet CA API From Windows

Puppet Enterprise exposes a number of RESTful APIs that can be used to help automate the solution and integrate it with other things. One need I’ve run into is the need to revoke and remove certificates from Puppet nodes in an automated fashion. My previous approach involved using SSH to connect to the Puppet Master server and run the puppet cert clean command, but I’m not a huge fan of that. With some effort, I found out how to talk to the API using Postman and PowerShell in a Windows environment. Postman was good for initial testing of the API, while I use PowerShell to fully automate solutions. I’ve outlined a step-by-step on how to set this up below:

Basics

The base URI for the puppet CA API is:

https://*puppet master server FQDN*:8140/puppet-ca/v1

The default port is 8140, which is configurable.

Authorization

Authorization and authentication were the most difficult parts for me to figure out. Unlike the other API endpoints in Puppet, you don’t use the normal token method. The CA API uses certificate authentication and authorization is granted based on the Subject Name of the certificate your client presents to the Puppet server. By default, the ONLY machine allowed to talk to the endpoint is your Puppet Master server itself, so without modification you can’t do much with the API.

You can change the authorization rules to allow other machines to connect. You can see the configuration for this in the /etc/puppetlabs/puppetserver/conf.d/auth.conf:

{
"allow-unauthenticated": true,
"match-request": {
"method": "get",
"path": "/puppet-ca/v1/certificate/",
"query-params": {},
"type": "path"
},
"name": "puppetlabs certificate",
"sort-order": 500
},
{
"allow": [
"puppetmaster.domain.com"
],
"match-request": {
"method": [
"get",
"put",
"delete"
],
"path": "/puppet-ca/v1/certificate_status",
"query-params": {},
"type": "path"
},
"name": "puppetlabs certificate status",
"sort-order": 500
},
{
"allow-unauthenticated": true,
"match-request": {
"method": "get",
"path": "/puppet-ca/v1/certificate_revocation_list/ca",
"query-params": {},
"type": "path"
},
"name": "puppetlabs crl",
"sort-order": 500
},
{
"allow-unauthenticated": true,
"match-request": {
"method": [
"get",
"put"
],
"path": "/puppet-ca/v1/certificate_request",
"query-params": {},
"type": "path"
},
"name": "puppetlabs csr",
"sort-order": 500
}

You’ll see an array of rules defined in this file, each one granting access to particular API endpoints. In this case, I’m most concerned with the certificate endpoints shown above. (For details on the layout of this file, see Puppet’s Docs here)

The endpoint rules that specify “allow-unauthenticated” are freely-accessible without authentication, so most of this article doesn’t apply to them. Just make a call from Postman or Curl like normal.

However, the certificate_status endpoint has an “allow” property, which lists all of the nodes that are allowed to access the endpoint. By default, it appears the name of your Puppet Master server appears here.

Normally, you could probably add entries to this list, restart your Puppet Master services, and go. The issue is this file is actually managed by Puppet, and your changes would be overwritten the next time the Puppet agent runs.

This setting is actually governed by the puppet_enterprise::profile::certificate_authority::client_whitelist setting. This can be set a couple of ways. The first way is to log into the Puppet Master GUI and do the following:

  1. Go to Inventory and select your Puppet Master server
  2. Select the “Groups” tab and click the PE Certificate Authority Group
  3. Click the “Classes” tab
  4. Set the client_whitelist parameter under puppet_enterprise::profile::certificate_authority

certificate_authorityNormally, this would work, but when the Puppet agent runs you might get the following error:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Puppet_enterprise::Profile::Master] is already declared; cannot redeclare on node

The workaround I found in a Q/A article suggested to just add the setting to your common.yaml and have Hiera set the setting instead. This worked well for me. My common.yaml file looks like this:

# Allows the listed machines to communicate with the puppet-ca API:
puppet_enterprise::profile::certificate_authority::client_whitelist:
– server1.mydomain.com
– server2.mydomain.com

Once this was pushed to the Puppet Master server, I did a Puppet agent run using puppet agent -t from the server and it applied the settings. Checking auth.conf again, I now see this:

{
"allow": [
"puppetmaster.domain.com",
"server1.domain.com",
"server2.domain.com"
],
"match-request": {
"method": [
"get",
"put",
"delete"
],
"path": "/puppet-ca/v1/certificate_status",
"query-params": {},
"type": "path"
},

Now that my servers are authorized to access the API, I can make calls using a client certificate to authenticate to the API.

Authentication

The next section shows you how to setup Postman and PowerShell to authenticate to the API. If you setup your authorization correctly as shown above, you should be able to hit the APIs.

Using Postman

To use Client Cert authentication to the Puppet API, you can setup Postman using the following method

Import the cert into Postman:

  1. Click Settings in Postman
  2. Go to Certificates
  3. Click the “Add Certificate link”
  4. Add the cert using the following settings
    • Host – Specify the FQDN of the host you want to present the cert to. Don’t specify any of the URI path, just the FQDN and port.
    • CRT File – Use the PEM file in the certs directory
    • KEY File – Use the PEM file in the private_keys directory
    • NO passphrase

Postman_client_cert

Once that is done, you can issue a GET command to a URI like this and get a response:

https://puppetmasterserver.domain.com:8140/puppet-ca/v1/certificate_statuses/key

The “key” portion of the URI is required, but the word “key” is arbitrary. I think you can pretty much type anything you want there.

This yields a response much like the following:

cert_statuses

If you get a “Forbidden” error, you either have the URI slightly wrong or you don’t have the authorization correct. The array of names in the “allow” section of the API rule MUST match the Subject Name of the certificate.

Using PowerShell

To get this to work with PowerShell, you have to export your Puppet certs as a PFX and reference them in a Invoke-RestMethod call.

To create a PFX from the certs, do the following:

  1. Install Openssl
      • If you have Git for Windows installed, you already have this. Just change to c:\program files\Git\usr\bin
  2. Run the following
C:\Program Files\Git\usr\bin\openssl.exe pkcs12 -export -out "c:\temp\server1.domain.com.pfx" -inkey "C:\ProgramData\PuppetLabs\puppet\etc\ssl\private_keys\server1.domain.com.pem" -in "C:\ProgramData\PuppetLabs\puppet\etc\ssl\certs\server1.domain.com.pem"

Don’t specify a export password.

Once that is done, call the following Cmdlet:

Invoke-RestMethod -Uri "https://puppetmaster.domain.com:8140/puppet-ca/v1/certificate_statuses/key" -Certificate (Get-PfxCertificate -FilePath C:\temp\server1.domain.com.pfx) -Headers @{"Content-Type" = "application/json" }

Viola! That’s it.

References

Puppet File Sync Not Working – LOCK_FAILURE

I had a recent issue where Puppet was not properly syncing code from the code-staging directory to the code directory.  I verified it was pulling the new code from my Git repository to code-staging without issue.  However, file-sync was not pushing the new code to the code directory.

Here is what I was seeing in the /var/log/puppetlabs/puppetserver/puppetserver.log

2017-06-26 11:08:49,026 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:08:54,051 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:08:59,077 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:04,103 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:09,129 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.
2017-06-26 11:09:14,155 ERROR [clojure-agent-send-off-pool-3] [p.e.file-sync-errors] Error syncing repo :puppet-code: File sync successfully fetched from the server repo, but update-ref result was LOCK_FAILURE on 8c346001ee2f834a4be05d3d9788d2d712b212c5. Name: puppet-code. Directory: /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git.

I had no idea what this meant, and I wasn’t sure how to resolve it so I took a snapshot of my Puppet Master VM and tried a few things.

The first thing I tried was going to the directory indicated and taking a look:

ll /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/production.git/
total 44
drwxr-xr-x 7 pe-puppet pe-puppet 4096 Jun 26 11:03 ./
drwxr-xr-x 3 pe-puppet pe-puppet 4096 Apr 25 2016 ../
drwxr-xr-x 2 pe-puppet pe-puppet 4096 Apr 25 2016 branches/
-rw-r—– 1 pe-puppet pe-puppet 307 Jun 26 11:03 config
-rw-r—– 1 pe-puppet pe-puppet 148 Jun 26 10:28 FETCH_HEAD
-rw-r–r– 1 pe-puppet pe-puppet 23 Apr 25 2016 HEAD
drwxr-xr-x 2 pe-puppet pe-puppet 4096 Apr 25 2016 hooks/
drwxr-xr-x 3 pe-puppet pe-puppet 4096 Apr 25 2016 logs/
drwxr-xr-x 4 pe-puppet pe-puppet 4096 Jun 26 10:28 objects/
drwxr-xr-x 4 pe-puppet pe-puppet 4096 Apr 25 2016 refs/
-rw-r—– 1 pe-puppet pe-puppet 41 Jun 26 10:28 synced-commit

/opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git/production.git had the same contents but for one file:

synced-commit.lock

I wasn’t sure this file belonged there, so I  removed it.  Once I did that, the file-sync service stopped throwing errors and successfully synced my files!

Hope this helps!

Puppet Enterprise – Adding Windows Scheduled Tasks

So, continuing on the path I’ve been on, I’ve had to create quite a few custom “resources” in my Puppet profiles to deploy or configure items I could not find right out-of-the-box.  In this case, I have a server that requires a standard set of Windows scheduled tasks.

For this purpose, I created a new pseudo-resource called “windows_scheduled_task”.  As with the other items I’ve published, I call this a pseudo-resource because it’s not really a Puppet resource.  It’s a custom class that is used just like a resource.  The approach I took here leverages PowerShell and assumes the presence of the ScheduledTasks module, which is only available in PowerShell v4 and higher.

The class requires the use of a module class (.pp file) and an accompanying template file (.epp).  The .pp file goes in the manifests folder in your module, and the template in your templates folder.  The assumed folder structure is like so:

/manifests/windows_server
  /scheduled_task.pp
/templates/windows_server
  /scheduled_task_add.epp

If you change the paths, that’s OK, but you have to make sure the class namespace in the .pp file matches your new folder structure. The default is

class windows_server::scheduled_task()

which assumes the folder /manifests/windows_server

You also have to make sure the epp() function call in the .pp file references the correct path to the template (if you change it). Right now, it’s set to look at /templates/windows_server/scheduled_task_add.epp.

Here is the .pp file class:

class windows_server::scheduled_task()
{

  define windows_scheduled_task
  (
    String $description = "No description.",
    String $path = "",
    String $executionTimeLimit = "01.00:00:00",
    String $userName = "NT AUTHORITY\\SYSTEM",
    String $password = "",
    Boolean $deployEnabled = true,
    Array[Hash] $actions,
    Array[Hash] $triggers = []
  )
  {
    #  name (string)                - Specifies the name of the task
    #  description (string)         - Specifies a description of the task
    #  path (string)                - Specifies the folder to place the task in.  Default is "\" (the root folder).  NOTE:  This must begin with a slash but not end with one!  Example:  /Restore
    #  executionTimeLimit (string)  - Specifies the length of time the task can run before being automatically stopped.  Specify as a TimeSpan.
    #  deployEnabled (bool)         - Determines whether the task should deployed in an enabled state or not.  This state is not enforced going forward.
    #  actions (Hash[]) -
    #    workingDirectory (string)      - Specifies the working directory for the action.  Default is C:\windows\system32
    #    command (string)               - Specifies the command to execute.
    #    arguments (string[])           - Specifies the arguments to pass to the command.
    #    isPowerShell (bool)            - If specified, then the command and arguments are automatically constructed.  You only need pass the powershell script you want to run for the command.

    #  triggers (Hash[]) -
    #    atDateTime (String)          - Specifies the date and time to start running the task.
    #    repetitionInterval (string)  - Specifies how often to re-run the task after the atDateTime occurs.  Specify as a Timespan.
    #    repetitionDuration (string)  - Specifies how long to repeat the task executions for.  Specify as a Timespan.  Default is [Timespan]::MaxValue (forever)

    #  If your command is a PowerShell script, you have to escape double-quotes with backslashes.
    #  Example:
    #  windows_server::scheduled_task::windows_scheduled_task { 'Test Scheduled Task':
    #   userName          =>  $taskCredentials['userName'],
    #   password          =>  $taskCredentials['password'],
    #   path              => '\MyTasks',
    #   actions           => [{
    #    isPowerShell        => true,
    #    command             => "c:\\scripts\\Run-MyPowerShellScript.ps1 -Param1 value1 -Param2 \"value 2\" -Param3 ${puppetVariableHere}  "
    #   }],
    #   triggers              => [{
    #    atDateTime          => "9/1/2016 12:30 AM",
    #    repetitionInterval  => "00:30:00"
    #   }],
    #}

    exec { "scheduled_task_${title}" :
      command       => epp("windows_server/scheduled_task_add.epp", {
                        name                => $name,
                        description         => $description,
                        path                => $path,
                        executionTimeLimit  => $executionTimeLimit,
                        userName            => $userName,
                        password            => $password,
                        deployEnabled       => $deployEnabled,
                        actions             => $actions,
                        triggers            => $triggers
                      }),
      onlyif        => "if ( ScheduledTasks\\Get-ScheduledTask | Where-Object { \$_.TaskName -ieq \"${name}\" -and \$_.TaskPath -ieq \"${path}\\\" } ) { \$host.SetShouldExit(99); exit 99 }",
      returns       => [0],
      provider      => powershell,
      logoutput     => true,
    }
  }
}

The template file is here:

<%- | String $name,
      String $description = "No description",
      String $path = "\\",
      String $executionTimeLimit = "01.00:00:00",
      String $userName = "NT AUTHORITY\\SYSTEM",
      String $password = "",
      Boolean $deployEnabled = true,
      Array[Hash] $actions,
      Array[Hash] $triggers = []
|
  #  name (string) - Specifies the name of the task
  #  description (string) - Specifies a description of the task
  #  path (string) - Specifies the folder to place the task in.  Default is "\" (the root foler)
  #  executionTimeLimit (string) - Specifies the length of time the task can run before being automatically stopped.  Specify as a TimeSpan.
  #  userName (string) - Specifies the user to execute the task as.  Default is local system,.
  #  password (string) - Specifies the password for the given user.
  #  actions (Hash[]) -
  #    workingDirectory (string) - Specifies the working directory for the action.  Default is C:\windows\system32
  #    command (string) - Specifies the command to execute.
  #    arguments (string[]) - Specifies the arguments to pass to the command.
  #    isPowerShell (bool) - If specified, then the command and arguments are automatically constructed.  You only need pass the powershell script you want to run for the command.

  #  triggers (Hash[]) -
  #    atDateTime (String) - Specifies the date and time to start running the task.
  #    repetitionInterval (string) - For daily repetition - Specifies how often to re-run the task after the atDateTime occurs.  Specify as a Timespan.
  #    repetitionDuration (string) - For daily repetition - Specifies how long to repeat the task executions for.  Specify as a Timespan.  Default is [Timespan]::MaxValue (forever)
  #    daysOfTheWeek (Array[string]) - For weekly repetition - Specifies the days of the week to run the task.  Specify an array of Monday | Tuesday | Wednesday | Thursday | Friday | Saturday | Sunday
  #    weeksInterval (Integer) - For weekly repetition - Specifies whether to run the schedule every week (value 1) or every n weeks (value n).  Default is every 1 week.
%>
$acts = @();
<% $actions.each | Hash $act | { -%>
$arg = @();
<%  if ( $act['isPowerShell'] )
{
  $cmd = "powershell.exe"
-%>
$arg += "-noprofile"
$arg += "-command `"<%= regsubst($act['command'],'\"', '\\\`"', 'GI') -%>`""
<% }
else
{
  $cmd = $act['command']
  if ( $act['arguments'] and is_array($act['arguments']) )
  {
    $act['arguments'].each | String $ar |
    { -%>
$arg += "<%= $ar -%>";
<%
    }
  }
  else
  { -%>
$arg += "<%= $act['arguments'] -%>"
<%}
}
if ( $act['workingDirectory'] )
{
  $wd = "-WorkingDirectory \"${act['workingDirectory']}\" "
}
else
{
  $wd = ""
} -%>
$params = @{}
if ( $arg )
{
  $params.Add("Argument", ($arg -join " "))
}

$acts += New-ScheduledTaskAction <%= $wd -%>-Execute "<%= $cmd -%>" @params
<% } -%>

$params = @{};
$trigs = @();
<% $triggers.each | Hash $trig |
{
  if ( $trig['weeksInterval'] or $trig['daysOfTheWeek'] )
  {
    #  Weekly Trigger:
    if ( $trig['weeksInterval'] )
    {
      $weeksInterval = $trig['weeksInterval']
    }
    else
    {
      $weeksInterval = 1
    }
-%>
$trigs += New-ScheduledTaskTrigger -Weekly -At "<%= $trig['atDateTime'] -%>" -WeeksInterval <%= $weeksInterval %> -DaysOfWeek <%= $trig['daysOfTheWeek'].join(",") %>;
<%
  }
  else
  {
    if ( $trig['repetitionDuration'] )
    {
      $repDuration = "<%= $trig['repetitionDuration'] -%>"
    }
    else
    {
      $repDuration = "([TimeSpan]::MaxValue)"
    }
#  Daily Trigger:
-%>
$trigs += New-ScheduledTaskTrigger -Once -At "<%= $trig['atDateTime'] -%>" -RepetitionInterval "<%= $trig['repetitionInterval'] -%>" -RepetitionDuration <%= $repDuration -%>;
<%
  }
}
-%>
if ( $trigs )
{
  $params.Add("Trigger", $trigs);
}

<% if ( $path == "" )
{
  $taskPath = "\\"
}
else
{
  $taskPath = $path
}
-%>
$sett = New-ScheduledTaskSettingsSet -ExecutionTimeLimit "<%= $executionTimeLimit -%>" -RunOnlyIfIdle:$false -DontStopOnIdleEnd;
$task = Register-ScheduledTask -TaskName "<%= $name -%>" -TaskPath "<%= $taskPath -%>" -Action $acts -Force -User "<%= $userName -%>" -Settings $sett<% if ( $password != "" ) { %> -Password "<%= $password -%>"<% } %> -RunLevel Highest @params;
<% if ( $deployEnabled == false ) { -%>
$task = $task | Disable-ScheduledTask;
<% } -%>

You can get both in my PuppetResources GitHub repo here.

Here is an example of a sample Scheduled Task:

mymodule::scheduled_task::windows_scheduled_task { 'Sample Scheduled Task':
    userName          =>  'MyTaskUserName',
    password          =>  'MyTaskPassword',
    deployEnabled     =>  true,
    description       => 'This task does some stuff.',
    actions           => [{
      command             => "c:\\scripts\\test-powershellscript.ps1",
      isPowerShell        => true
    }],
    triggers              => [{
      atDateTime          => "9/1/2016 11:00 PM",
      weeksInterval       => 1,
      daysOfTheWeek       => ["Monday","Tuesday","Wednesday","Thursday","Friday"]
    }],
  }

Enjoy!

Adding SQL Server Agent Jobs using Puppet

I find Puppet Enterprise to be very useful for configuring our many SQL Servers.  It does a nice job of setting up the SQL Instance and doing some base configuration.  There were a few things I wanted to add that it didn’t do out of the box that I thought I’d share.  One need I had was there was a set of specific SQL Agent jobs that I deployed out to our servers that I wanted Puppet to lay down for me.  I was able to build a pseudo-resource using the PE sqlserver forge module and some T-SQL.  I call it a pseudo-resource because it’s not a real resource in Puppet (with all the backing Ruby classes), but it behaves very much like a resource.

To do this, I needed the puppetlabs/sqlserver module and I had to create two files in my Puppet code repository.

NOTE: You must have Puppet Enterprise to use the puppetlabs/sqlserver module!

The first file I had to create was a T-SQL template that would generate the code needed to add the SQL Agent job.  This template is not 100% fully-featured, and a lot  more variables can be added to fully flesh out all of its options, but this is a very solid start.  I named this file sql_agent_job.epp and dropped it in my “templates” folder.  It looks like this:

<%- | String $name, String $description, String $notifyOperator = "", Array[Hash] $steps, Any $schedules = undef | -%>
BEGIN TRANSACTION
BEGIN TRY
  DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]';
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
END;

This isn’t the complete file (see my link below for the entire thing), but it gives you the idea. The template gets called by the .pp class file, which is below.

The second file is the actual Puppet class (.pp extension).  This is the file that implements the template and makes the whole thing “resource-like”.  This file belongs in your “manifests” folder in your repository or module:

class sqlserver::sql_agent_job()
{
  define sql_agent_job
  (
    String $sqlInstanceName,
    String $description,
    String $notifyOperator,
    Array[Hash] $steps,
    Any $schedules = undef
  )
  {
   sqlserver_tsql { "${title}_${sqlInstanceName}_sql_agent_job" :
      instance    => $sqlInstanceName,
      command     => epp("sqlserver/sql_add_job.epp", {
                        name            => $name,
                        description     => $description,
                        notifyOperator  => $notifyOperator,
                        steps           => $steps,
                        schedules       => $schedules
                      }),
      onlyif      => "IF NOT EXISTS ( SELECT * FROM msdb.dbo.sysjobs WHERE name = '${name}' ) BEGIN
                        THROW 51000, '${name} job not present.', 10;
                    END;"
    }
  }
}

Note:  You have to make sure the call to epp(…) above points to the path your template is at.  In the example above, I presume it’s in the same module in the templates/sqlserver folder.  Your folder structure should look roughly like this:

manifests/
     /sqlserver/sql_add_job.pp
templates/
     /sqlserver/sql_add_job.epp

This is the resource you will actually drop in you profile classes to add jobs to servers. The input parameters are as follows:

#  PARAMETERS:
    # name                          => (namevar) Specifies the name of the agent job.   - https://msdn.microsoft.com/en-us/library/ms182079.aspx
    # sqlInstanceName               => Specifies the SQL Server instance.
    # description                   => Specifies the description on the job.
    # notifyOperator                => Specifies the name of the job operator to notify.
    # steps                         => An array of hashes specifying the job steps:
    #   name                          => String - The name of the job step
    #   command                       => String - The T-SQL to execute
    #   database                      => String - The name of the database to execute against if the subsystem is TSQL.
    #   onSuccess                     => Integer - 3(next)|2(quitfail)|1(quitsuccess)|4(gotostep), default is 1
    #   onFail                        => Integer - 3(next)|2(quitfail)|1(quitsuccess)|4(gotostep), default is 2
    #   onSuccessStepId               => Integer - The stepid to go to on success
    #   onFailStepId                  => Integer - The stepid to to go in failure
    #   subsystem                     => String - Specify either "TSQL" or "CmdExec".  Default is TSQL.
    #   outputFileName                => String - Specify the path to the file to write the output to.
    # schedules                     => (optional) A hash specifying a job schedule.     - https://msdn.microsoft.com/en-us/library/ms366342.aspx
    #   frequencyType                 => Integer - 1(once)|4(daily)|8(weekly)|16(monthly), default 4
    #   frequencyInterval             => Integer - (once) - not used | (daily) - every frequencyInterval days | (weekly) - frequencyinterval determines day of wek | (monthly) - determines day of the month
    #   frequencySubdayType           => Integer - 1(attime)|4(minutes)|8(hours), default 1
    #   frequencySubdayInterval       => Integer - number of minutes/hours
    #   frequencyRecurrenceFactor     => Integer - Number of weeks/months between exectutions.  Nonzero value required if frequencytype is 8|16|32 (not used otherwise).  Default is 0.
    #   activeStartTime               => "HHMMSS, default 0",
    #   activeEndTime                 => "HHMMSS, default 235959"

You’ll probably notice the parameter names and values are pretty much identical to the input parameters for sp_add_job, sp_add_jobstep and sp_add_jobschedule stored procedures. A trick I use when I want to take a job and add it to Puppet is to add the job to SQL Server first, set it up the way I want, then script the job out. The parameters in the T-SQL script will pretty much translate to the sql_agent_job resource.

Here is an example of a profile with the sql_agent_job resource in use:

profile::sqlserver::component::sql_agent_job::sql_agent_job { "${name}_my_agent_job":
      name                  => "My SQL Agent Job",
      sqlInstanceName       => $name,
      description           => 'This is my SQL Agent Job being deploying wiht Puppet.',
      notifyOperator        => 'SQLTeam',
      steps                 => [{
                                name      => 'Execute Test Script',
                                database  => 'master',
                                subsystem => 'TSQL',
                                command   => "SELECT 'test data here'",
                                onSuccess => 1,
                                onFail    => 2
                              }],
      schedules             => {
                                frequencyType           => 4,
                                frequencyInterval       => 1,
                                frequencySubdayType     => 4,
                                frequencySubdayInterval => 30,
                                activeStartTime         => 000000,
                                activeEndTime           => 235959
                              },
    }
The full versions of these files can be found in my GitHub repository here:
Enjoy!

Creating a Directory Tree in Puppet

As you can probably tell from the flurry of blog posts I’ve made concerning Puppet, I’m going through the process of learning and setting up Puppet Enterprise.

One thing that irked me early on is the inability of the file resource to create a directory if the parent directory does not exist.  For example:

file { 'mydirectory' :
  ensure         => 'directory',
  path           => 'c:/parentdir/childdir'
}

If c:\parentdir does not exist, this fails.

Error: Cannot create C:/parentdir/childdir; parent directory C:/parentdir does not exist
Error: /Stage[main]/Profile::Myclass/File[mydirectory]/ensure: change from absent to directory failed: Cannot create C:/parentdir/childdir; parent directory C:/parentdir does not exist

You can alternately specify it like this to get it to work:

file { ['c:/parentdir', 'c:/parentdir/childdir'] :
  ensure         => 'directory'
}

This works, and for the most part is OK.  In my case though, I have the user provide the directory name through a class parameter:

class myclass ([String] $mydirectory)
{
  file { 'mydirectory' :
   ensure         => 'directory',
   path           => $mydirectory
  }
}

If the user specifies c:/parentdir/childdir, and c:/parentdir does not exist, it explodes.  I could adjust the code and advise my users to pass in arrays of strings representing the directories, but that’s not very clear or clean.

Fortunately, Puppet supports PowerShell and PowerShell is awesome:

class myclass (String $directory)
{
   exec { 'mydirectory' :
     command => "c:\\windows\\system32\\windowspowershell\\v1.0\\powershell.exe -noprofile -noninteractive -command \"New-Item -ItemType Directory -Path \"$directory\" \"",
     onlyif  => "c:\\windows\\system32\\windowspowershell\\v1.0\\powershell.exe -noprofile -noninteractive -command \"if (Test-Path -Path \"$directory\" -PathType Container) { exit 99 }\""
   } 
}

This code block creates the entire directory tree without issue.  The onlyif parameter ensures that the exec block is not fired off if the directory already exists.

Enjoy!

Puppet Agent on Windows – Module not found

The first step I take when developing a new Puppet configuration is to install the Puppet Agent on a standalone test Windows server and build the configuration files locally there.  I then use the puppet apply utility to test it and make sure it works.  This saves a lot of time since it avoids having to do hundreds of pushes and merge requests to our source control system as I tweak and debug the config files to get them working the way I want.

I had some challenges getting this setup initially though.  I attempted to follow advice given to me by my Puppet SE, and researched and tried to implement Roles and Profiles as a means of developing layered configurations.  It make sense to do it this way, especially as your configuration base grows, but it requires a bit of know-how to get working properly.  One of the major stumbling blocks I hit was getting Puppet to recognize classes located in a non-standard directory.  The normal, standard directory structure looks like this:

C:/ProgramData/PuppetLabs/code
    /modules # This is the default $basemodulepath
    /environments/production
        /manifests  # This is where it expects site.pp and any other code you write
        /modules     # Your downloaded and custom modules can also go here

In my case, I wanted to create a “site” directory in which I stored my role and profile configurations per the design above.  My structure looked like this:

c:/programdata/puppetlabs/code/environments/production
    /site
        /profile/manifests
        /role/manifests

Since this was not in the default $basemodulepath directory  or the environment module directory I’d receive an error stating the class could not be found:

ModuleNotFound

This is easy enough to figure out.  Puppet is highly configurable, and as such you can add additional directories to the list of those it looks in for classes it can use.  In my case, I simply edited the environment.conf file found at C:\ProgramData\PuppetLabs\code\environments\production\environment.conf  and commented-in the modulepath variable.  I then added my site folder.  I changed this line:

# modulepath = ./modules:$basemodulepath

To look like this:

modulepath = modules:site:$basemodulepath

However, I found I would still receive the same error as before.  A clue for me was when I ran the puppet config print modulepath command:

PS C:\ProgramData\PuppetLabs\code\environments\production\manifests&amp;gt; (puppet config print modulepath) -split &quot;;&quot;

You can see it lists the following paths:

C:/ProgramData/PuppetLabs/code/environments/production/modules
C:/ProgramData/PuppetLabs/code/modules
C:/opt/puppetlabs/puppet/modules

None of these were my site directory.  It’s as if the change I made to environment.conf was simply ignored.

Essentially, I found it was.  Even though the inital example show in the environment.conf files shows this (note the colon delimiter):

# modulepath = ./modules:$basemodulepath

I found the Windows Agent uses semicolons, not colons as a delimiter for multiple paths.  This is kind of documented here.

Path Separator

Make sure to use a semi-colon (;) as the path separator on Windows, e.g., modulepath=path1;path2

Plain enough, but this document does not reference the environment.conf file specifically, or even the Puppet Agent (this seems to be just a general Windows thing).  Also, the Puppet Agent installer lays down the environment.conf file with the colons in place, so it’s very misleading.

In any case, I found that if I changed the file to look like this, everything worked:

modlepath = modules;site;$basemodulepath

Running puppet config print modulepath confirmed my site path now shows up:

C:/ProgramData/PuppetLabs/code/environments/production/modules
C:/ProgramData/PuppetLabs/code/environments/production/site
C:/ProgramData/PuppetLabs/code/modules
C:/opt/puppetlabs/puppet/modules

So, in summary, if you are using any non-standard paths for your modules or classes on a Windows machine, make sure and use semicolons to delimit multiple paths for the modulepath setting, rather than the default colon.

Confusing, but easy to fix fortunately.

hiera-eyaml and Puppet Enterpise – Command not found?

I’m in the process of evaluating Puppet Enterprise as a configuration management solution for my company.  A glaring issue I hit early on is figuring out how to secure credentials that are fed to the various Puppet configurations.  By default, there is no way I’m aware of to obfuscate credentials in the configuration areas (including hiera files and class parameters in the GUI).  This is an issue as I can’t expose certain credentials to the general public.

Fortunately, hiera-eyaml was easy-to-find and does the trick.  There’s a lot of good documentation out there on how to set this up, and I won’t belabor that point, but to a Puppet noob the documentation makes a lot of assumptions.  The main assumption I want to clear up is how to get it up-and-running on your Puppet Master server using the eyaml utility from the CLI.

 

The GitHub document appears easy-to-follow:

https://github.com/TomPoulton/hiera-eyaml

The first step makes perfect sense, and worked without issue:

puppetserver gem install hiera-eyaml

The problem was after this.  I could not call the eyaml executable.  If I typed “eyaml –help”, “eyaml encrypt” or any valid variation of the command I received a “eyaml:  command not found” error.

Long story short, the issue is the Puppet master server does not have the ruby interpreter setup by default for command line use. The command above does make hiera-eyaml available for the Puppet software’s use, and you can go about configuring  it and using as stated in the GitHub readme for Puppet, but the eyaml calls will not work for you on the CLI.  The assumption they make is that you know to install the Ruby interpreter and gem separately for CLI usage.  To do this, do the following from the Puppet master (or any Linux station):

apt-get install ruby
gem install hiera-eyaml

Now the ruby interpreter is available for use to you on the CLI and you can call the eyaml executable as noted in the GitHub article.

I’m sure this is obvious to a Ruby/Linux expert, but it took me about 3/4 of a day to figure this out, so hopefully this helps save someone some time down the road.