How to parse a nested YAML config file in Python and Ruby

if you have a complex config schema, you may need to store it in a YAML or JSON format

having been used to .INI style configs, I recently had to store nested values and INI style gets very complex, very fast.

For instance in YAML:

---
person:
  Joe:
    age: 32
    children:
      - Katie
      - Frank

  Bob:
    age: 43
    children:
      - Lisa

to get the names of Joe’s children, in JSON or YAML would look something like this,

data['person']['Joe']['children']
['Katie','Frank']

in INI, this would be something like,

[person]
[person/joe]
age=32
children=Katie,Frank

This is really ugly and not nested visually. To get an individual child’s name, you would need to additionally parse a comma separated string. Fugly.

Much better to use YAML. I prefer YAML over JSON because its much easier for human readability, although the language interpreter converts YAML into JSON during run-time

Heres a Python and Ruby example on how to parse this sample Config file

config.yaml

---
scanners:
  hostname:
    web01.nyc.mycorp.com:
      port: 9900
      scans:
        - "cisco scan"
        - "network sec scan"
        - "windows sec scan"
    web05.tex.mycorp.com:
      port: 9923
      scans:
        - "tex network"
        - "infra scan"

 

The Py and Rb parser scripts are structurally very similar,

Python

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import yaml

def get_config(*args):
 with open('config.yaml', 'r') as f:
     conf = yaml.load(f) 
 
     # get section
     section = args[0]

     # check if Config file has Section
     if not conf.has_key(section):
         print "key missing"
         quit()

     # get values
     argList = list(args) # convert tuple to list
     argList.pop(0) # remove Section from list
 
     # create lookup path
     parsepath = "conf['" + section + "']"
 
     for arg in argList:
         parsepath = parsepath + "['" + arg + "']"
     return eval(parsepath)
 f.close()

scans = get_config('scanners','hostname','web01.nyc.mycorp.com','scans')
print scans
['cisco scan', 'network sec scan', 'windows sec scan']

or if you want a list of all hostnames,

scans = list(get_config('scanners','hostname'))
['web01.nyc.mycorp.com', 'web05.tex.mycorp.com']

 

here’s a simple one-liner if you want to bypass error checking

for key, value in yaml.load(open('config.yaml'))['scanners']['hostname'].iteritems():
    print key, value
web01.nyc.mycorp.com {'port': 9900, 'scans': ['cisco scan', 'network sec scan', 'windows sec scan']}
web05.tex.mycorp.com {'port': 9923, 'scans': ['tex network', 'infra scan']}

Ruby

#!/usr/bin/env ruby
require 'rubygems'
require 'yaml'

# get config file value
def get_config(*args)
   conf = YAML.load_file($config_file)

   # get section
   section = args[0]

   # check if Config file has Section
   if not conf.has_key?(section)
       puts "Config file does not have section information for #{section}")
       break
    end

   # remove 1st element, "section"
   args.shift
   parsepath = "conf[\"#{section}\"]"
   args.each do |arg|
       parsepath = parsepath+"[\"#{arg}\"]"
   end

   # Handle errors
   begin
     return eval(parsepath)
   rescue
      puts "Config file does not have information for #{parsepath}")
      exit(1)
   end
end #EOF

scans = get_config('scanners','hostname','web05.tex.mycorp.com','scans')
puts scans
['tex network', 'infra scan']
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s