module

Krikri::PythonModuleRunner

Constants

BASIC_PY_SHIM = "import json\nimport os\nimport subprocess\nimport sys\nimport tempfile\n\n\nclass AnsibleModule(object):\n def __init__(self, argument_spec=None, bypass_checks=False, no_log=False,\n supports_check_mode=False, **kwargs):\n self.argument_spec = argument_spec or {}\n self.supports_check_mode = supports_check_mode\n self._warnings = []\n self.tmpdir = tempfile.gettempdir()\n self.check_mode = False\n self.params = {}\n self._name = os.path.splitext(os.path.basename(sys.argv[0]))[0]\n raw_args = self._read_args()\n self.check_mode = bool(\n raw_args.pop('_ansible_check_mode', False)\n or os.environ.get('ANSIBLE_CHECK_MODE') == '1')\n self._apply_argument_spec(raw_args)\n\n def _read_args(self):\n env_args = os.environ.get('ANSIBLE_MODULE_ARGS')\n if env_args:\n try:\n return json.loads(env_args)\n except ValueError:\n self.fail_json(msg='ANSIBLE_MODULE_ARGS env var is not valid JSON')\n raw = ''\n try:\n if not sys.stdin.isatty():\n raw = sys.stdin.read()\n except Exception:\n raw = ''\n raw = raw.strip()\n if not raw:\n return {}\n try:\n parsed = json.loads(raw)\n except ValueError:\n self.fail_json(msg='Failed to decode JSON module parameters.')\n if isinstance(parsed, dict) and 'ANSIBLE_MODULE_ARGS' in parsed:\n parsed = parsed['ANSIBLE_MODULE_ARGS']\n if not isinstance(parsed, dict):\n self.fail_json(msg='Module parameters must be a JSON object.')\n return parsed\n\n def _cast(self, name, value, spec):\n kind = spec.get('type', 'str')\n if kind == 'bool':\n if isinstance(value, bool):\n return value\n text = str(value).strip().lower()\n if text in ('yes', 'on', '1', 'true'):\n return True\n if text in ('no', 'off', '0', 'false', ''):\n return False\n self.fail_json(msg=\"argument '%s' is not a valid boolean\" % name)\n if kind == 'int':\n try:\n return int(value)\n except (TypeError, ValueError):\n self.fail_json(msg=\"argument '%s' is not a valid integer\" % name)\n if kind == 'float':\n try:\n return float(value)\n except (TypeError, ValueError):\n self.fail_json(msg=\"argument '%s' is not a valid float\" % name)\n if kind in ('dict', 'json'):\n if isinstance(value, dict):\n return value\n try:\n parsed = json.loads(value)\n except (TypeError, ValueError):\n self.fail_json(msg=\"argument '%s' is not valid JSON\" % name)\n if not isinstance(parsed, dict):\n self.fail_json(msg=\"argument '%s' is not a dict\" % name)\n return parsed\n if kind == 'list':\n if isinstance(value, list):\n return value\n try:\n parsed = json.loads(value)\n except (TypeError, ValueError):\n parsed = str(value).split(',')\n return parsed\n if kind == 'path':\n return os.path.expanduser(os.path.expandvars(str(value)))\n return value\n\n def _apply_argument_spec(self, raw_args):\n for key, spec in self.argument_spec.items():\n for alias in spec.get('aliases', []) or []:\n if alias in raw_args and key not in raw_args:\n raw_args[key] = raw_args[alias]\n missing = []\n for key, spec in self.argument_spec.items():\n if key in raw_args:\n value = self._cast(key, raw_args[key], spec)\n if spec.get('type') == 'list' and spec.get('elements'):\n element_spec = {'type': spec['elements']}\n value = [self._cast(key, element, element_spec)\n for element in value]\n self.params[key] = value\n elif 'default' in spec:\n self.params[key] = spec['default']\n elif spec.get('required'):\n missing.append(key)\n else:\n self.params[key] = None\n if missing:\n self.fail_json(msg='missing required arguments: %s'\n % ', '.join(sorted(missing)))\n for key, value in raw_args.items():\n if key.startswith('_ansible_') or key in self.params:\n continue\n self.params[key] = value\n\n def warn(self, message):\n self._warnings.append(str(message))\n\n def deprecate(self, message, **kwargs):\n self._warnings.append('DEPRECATED: %s' % message)\n\n # Real basic.py logs to the systemd journal (when the target has\n # python-systemd) or syslog with the ident\n # 'ansible-<module_name>' at LOG_INFO; where neither is reachable\n # (containers, sandboxed exec contexts) python's syslog module\n # itself silently no-ops. Real basic.py only raises when *msg*\n # isn't a string; the actual syslog write never fails the module\n # - so here a swallowed exception is the documented worst case,\n # never an AttributeError like before (sr_fingerprint via\n # linux-system-roles.firewall/.kdump).\n def log(self, msg, log_args=None):\n if isinstance(msg, bytes):\n msg = msg.decode('utf-8', 'replace')\n try:\n import syslog\n syslog.openlog('ansible-%s' % self._name, 0, syslog.LOG_USER)\n syslog.syslog(syslog.LOG_INFO, str(msg))\n except Exception:\n pass\n\n def run_command(self, args, check_rc=False, cwd=None,\n environ_update=None, **kwargs):\n if isinstance(args, str):\n argv = args.split()\n else:\n argv = [str(a) for a in args]\n env = os.environ.copy()\n if environ_update:\n env.update({k: str(v) for k, v in environ_update.items()})\n proc = subprocess.Popen(argv, cwd=cwd, stdout=subprocess.PIPE,\n stderr=subprocess.PIPE, env=env)\n out, err = proc.communicate()\n rc = proc.returncode\n if check_rc and rc != 0:\n self.fail_json(msg='Command failed with rc %d: %s'\n % (rc, err.decode('utf-8', 'replace')))\n return (rc, out.decode('utf-8', 'replace'),\n err.decode('utf-8', 'replace'))\n\n # Mirrors real basic.py's AnsibleModule.get_bin_path\n # (delegate to module_utils.common.process.get_bin_path):\n # absolute paths pass through, then opt_dirs, then PATH. Not\n # found + required fails via fail_json like real basic.py;\n # not required raises ValueError for the caller to catch\n # (systemd_units via linux-system-roles.systemd calls it\n # with neither, and real ansible-playbook still succeeds\n # there because systemctl is found).\n def get_bin_path(self, arg, required=False, opt_dirs=None):\n paths = []\n if os.path.isabs(arg):\n paths.append(arg)\n for d in (opt_dirs or []):\n paths.append(d)\n paths.extend(os.environ.get('PATH', os.defpath).split(os.pathsep))\n for d in paths:\n candidate = os.path.join(d, arg)\n if os.path.isfile(candidate) and os.access(candidate, os.X_OK):\n return candidate\n msg = ('Failed to find required executable %s in paths: %s'\n % (arg, ':'.join(paths)))\n if required:\n self.fail_json(msg=msg)\n raise ValueError(msg)\n\n def exit_json(self, **kwargs):\n result = dict(kwargs)\n result.setdefault('changed', False)\n if self._warnings:\n result['warnings'] = self._warnings\n sys.stdout.write(json.dumps(result) + '\\n')\n sys.exit(0)\n\n def fail_json(self, msg='Module failed', **kwargs):\n result = dict(kwargs)\n result['failed'] = True\n result['msg'] = msg\n if self._warnings:\n result['warnings'] = self._warnings\n sys.stdout.write(json.dumps(result) + '\\n')\n sys.exit(1)"

The ansible/module_utils bundle a new-style module's from ansible.module_utils.basic import AnsibleModule import needs. Real Ansible never relies on ansible-core being installed on the target - the AnsiballZ wrapper bundles module_utils INTO the module payload it ships - so every new-style role-private module runs on any target with a python3. This engine runs the raw module script instead, so on a target with no ansible-core installed the import died with ModuleNotFoundError and the module printed no result JSON ("MODULE FAILURE") - hard-FAILING the task where real Ansible ran it successfully (found via newrelic.newrelic-infra's own "Setup agent config *NIX" task: the role ships its own library/merge_yaml.py, which took the py_module path and failed on every fresh target while real ansible-playbook succeeded). The shim covers what corpus role-private modules actually use - params parsing/validation against argument_spec (with type coercion, defaults, aliases, required), check_mode, exit_json/fail_json, warn/run_command, log, get_bin_path - plus the ansible/module_utils/_text helpers modules import directly; not the whole real basic.py surface; anything beyond that fails exactly as before this shim existed.

CONVERTERS_PY_SHIM = "def to_bytes(value, errors='surrogate_or_strict', encoding='utf-8'):\n if isinstance(value, bytes):\n return value\n if errors in ('surrogate_or_strict', 'surrogate_or_replace',\n 'surrogate_or_xmltext', 'surrogate_then_replace'):\n errors = 'surrogateescape'\n try:\n return str(value).encode(encoding, errors)\n except (UnicodeEncodeError, LookupError):\n return str(value).encode(encoding, 'replace')\n\ndef to_text(value, errors='surrogate_or_strict', encoding='utf-8'):\n if isinstance(value, bytes):\n if errors in ('surrogate_or_strict', 'surrogate_or_replace',\n 'surrogate_or_xmltext', 'surrogate_then_replace'):\n errors = 'surrogateescape'\n try:\n return value.decode(encoding, errors)\n except (UnicodeDecodeError, LookupError):\n return value.decode(encoding, 'replace')\n if isinstance(value, str):\n return value\n return str(value)\n\ndef to_native(value, errors='surrogate_or_strict', encoding='utf-8'):\n return to_text(value, errors, encoding)\n\ndef to_basestring(value):\n return to_text(value)"

In modern ansible-core the real text-conversion implementation moved from ansible/module_utils/_text.py to ansible/module_utils/common/text/converters.py - _text remains only as a deprecated re-export shim - and newer roles import the new path directly. bodsch.users' own library/multi_users.py does exactly that (from ansible.module_utils.common.text.converters import to_native, round 813275) and died with ModuleNotFoundError: No module named 'ansible.module_utils.common' on a target without ansible-core, while real Ansible - which ships both paths - succeeded on the same task. So this file ships alongside _text.py, self-contained rather than importing from it, since role code may import either path (or both) and real Ansible keeps both importable. Same to_bytes/to_text/to_native surface and the same surrogateescape mapping of the Ansible error-handler spellings as the _text shim above.

TEXT_PY_SHIM = "def to_bytes(value, errors='surrogate_or_strict', encoding='utf-8'):\n if isinstance(value, bytes):\n return value\n if errors in ('surrogate_or_strict', 'surrogate_or_replace',\n 'surrogate_or_xmltext', 'surrogate_then_replace'):\n errors = 'surrogateescape'\n try:\n return str(value).encode(encoding, errors)\n except (UnicodeEncodeError, LookupError):\n return str(value).encode(encoding, 'replace')\n\ndef to_text(value, errors='surrogate_or_strict', encoding='utf-8'):\n if isinstance(value, bytes):\n if errors in ('surrogate_or_strict', 'surrogate_or_replace',\n 'surrogate_or_xmltext', 'surrogate_then_replace'):\n errors = 'surrogateescape'\n try:\n return value.decode(encoding, errors)\n except (UnicodeDecodeError, LookupError):\n return value.decode(encoding, 'replace')\n if isinstance(value, str):\n return value\n return str(value)\n\ndef to_native(value, errors='surrogate_or_strict', encoding='utf-8'):\n return to_text(value, errors, encoding)\n\ndef to_basestring(value):\n return to_text(value)"

The ansible/module_utils/_text helpers a new-style module can import DIRECTLY alongside basic (nbde_server_tang via linux-system-roles.nbde_server does from ansible.module_utils._text import to_native) - without this file the import dies with ModuleNotFoundError at module top level, before AnsibleModule is ever constructed. Only the to_bytes/to_text/to_native surface modules actually import; the error-handler spellings real _text.py maps (surrogate_or_strict et al) become surrogateescape on py3 like the real code.

Class methods

write_module_utils_bundle(work_dir : String) : Nil

Writes the shim bundle above into work_dir as a real ansible/module_utils package tree. The module script itself sits in work_dir too, and Python puts the script's own directory first on sys.path - so the shim shadows any installed ansible-core exactly when it's written, and the import resolves to it instead of dying with ModuleNotFoundError. Written ONLY for a target where the probe import failed (see py_module.cr): where real ansible-core IS installed the module keeps running against the real basic.py, unchanged from pre-shim behavior.

Source

Instance methods

build_kv_argv(params : Hash(String, String)) : Array(String)

The old-style key=value argv line (one entry per param).

Source
build_module_args(params : Hash(String, String), check_mode : Bool) : String

The module's argument dict: the substituted task params (already stringified by the parser) re-typed as JSON where they parse - the parser JSON-encodes list/dict-valued params verbatim, so "['a','b']" becomes a real array for the module, the way real Ansible passes typed args. Plus real Ansible's own reserved _ansible_* keys a new-style module's AnsibleModule reads.

Source
find_source(module_name : String, role_path : String | Nil, playbook_dir : String | Nil) : String | Nil

Finds a role-private module source for module_name, or nil. Search roots mirror real Ansible's two most-used locations: the current role's own library/ and the playbook-adjacent library/. First match wins (real Ansible's own nearest-first order).

Takes the role's ROOT directory directly (task.role_path, always set - see role_loader.cr's task.role_path = role_dir), not role_files_dir (only set when the role actually ships a files/ subdirectory - existing_dir returns nil otherwise). A role with no files/ dir at all (linux-system-roles.storage/.logging/ .timesync, none of them ship one) could never resolve its own library/*.py modules through the old files/-derived path, so sr_fingerprint/blivet/timesync_provider fell straight back to "unavailable modules" - the exact scope cut 0.9.819 was supposed to have already closed for role-private modules. Found re-testing linux-system-roles.storage/logging/timesync.

Source
missing_interpreter_line?(source : String, new_style : Bool) : Bool

Real Ansible refuses to ship a module payload whose first line is not a #! interpreter line: ActionBase._execute_module's if not module_shebang and module_style != 'binary' guard raises "module (name) is missing interpreter line" as a controller-side AnsibleError (failed task, nothing executed on the target). New-style modules are exempt in practice - the AnsiballZ wrapper embeds its own shebang. This engine runs the raw script with the target's python3 instead of honoring the file's own line, so without this guard a shebangless old-style module - which real ansible-playbook FAILS - silently succeeded (found by the py_module podman-diff edge cases: only the new-style fixture survived the real side without a shebang).

Source
new_style?(source : String) : Bool

Real Ansible's own new-style detection (ansiballz): a module importing ansible.module_utils gets its args as a JSON dict (via the ANSIBLE_MODULE_ARGS env var its basic.py reads when no argv is given); everything else is old-style key=value argv.

Source
parse_module_output(stdout : String) : JSON::Any | Nil

Parses the module's stdout into its result JSON: real modules print a JSON object (pretty or single-line), possibly preceded by other output (warnings, prints) that real Ansible also strips. Walks backwards from the end for the first offset where a JSON object parse succeeds.

Source
short_name(module_name : String) : String

Short module name for the FQCN spellings a task can write (sr_fingerprint, linux_system_roles.sr_fingerprint, ...).

Source