IGGroup mcollective-puppetng

coordinate puppet runs across many hosts

Download .zip Download .tar.gz View on GitHub

mcollective-puppetng is an mcollective agent and application for running puppet on many systems.

It uses a small subset of code from the puppetlabs puppet agent, but is otherwise a substantial rewrite with an emphasis on reliability, detecting and handling failures, and reporting back the outcome of puppet runs. Many of the configurables the puppetlabs agent did are not available in this yet, but noop is.

Usually, it will report the errors that you would see on the puppet console.

With the original agent, our experience was that it was quite hard to know for certain the run was successful on all systems.

Benefits and Features

  • Extended checking of failures and timeouts during the puppet run.
  • Will retry in some scenarios
  • Reports back errors.
  • Most timeouts and maximum retry attempts are configurable.
  • Can manage the number of concurrent puppet runs.
  • Prints a summary of failed hosts at the end of the run.
  • Coloured output.
  • Extendible using observers which can log progress of the run.



The application coordinates the puppet runs across many agents.

  • util/puppetng/colorize.rb - for colorizing the output
  • application/puppetng.rb
  • It requires the agent DDL at agent/puppetng/puppetng.ddl

Example command:

		  mco puppetng run --concurrency 50

The mcollective agent has two functions, run and check_run. When the run function is called, a unique ID 'runid' is passed (the same for all nodes in a run), which can then be provided to the check_run to get the status of it.

The puppet run is monitored by a daemon process backgrounded by the agent. It monitors the puppet state directories, and when it detects change in progress it writes the state to a JSON file (under /tmp by default). check_run mostly just serves up this file.

  • /usr/libexec/mcollective/mcollective/agent/puppetng.rb - starts the daemon or reads reports (using util/ code).
  • /usr/libexec/mcollective/mcollective/agent/puppetng.ddl - DDL for the agent. See for available inputs and outputs.
  • /usr/libexec/mcollective/mcollective/util/puppetng/managed_puppet_run.rb - code for launching and monitoring a run.
  • /usr/libexec/mcollective/mcollective/util/puppetng/puppet_run_registry.rb - for reading JSON reports from disk.
  • /usr/local/sbin/puppetng_agent - a daemon which just passes arguments and backgrounds the managed_puppet_run.rb code.



About IG Group

IG Group is a world-leading provider of financial spread betting and CFDs. We are an established member of the FTSE 250 and have a market capitalisation of £1.7 billion (as at June 2011).

With headquarters in London and offices across Europe, as well as in the US, Japan, Singapore and Australia, our international network is growing rapidly. In the last five years we have opened 11 new offices.

The Group has built a reputation for its advanced trading technology, competitive pricing and reliability. We have won a number of awards and independent research has revealed that over 60% of the UK's active spread bettors hold an account with IG Index.*

*Investment Trends UK Financial Spread Betting & Contracts for Difference Report (November 2011).

About IG Careers Open Source