Saturday, May 24, 2008

python + elixir + pyyaml == yay!

And so... I'm playing with a little toy script to ease some pain at work. We've got a bunch of bills in a very large database, the bill data our application reads/modifies is split across a number of tables.

At the moment, the developers manually run SQL scripts to peek at various bits of data. Definitely a huge waste of time.

So, I'm playing with Python + Elixir + PyYAML to get a script that I can just pass in the bill ID and it'll query the 5-6 tables and serialize the bill to something more human readable. If I play my cards right, I'll never have to deal with Toad again.

Elixir works its magic and I can query my database. The autoload didn't work, and there are a couple tables with 50-100+ columns. Serializing it to YAML without repeating myself is now the trick.

I'm sure there's a better way to do this, but here's what I came up with... applied to the Elixir tutorial. Not the most exciting thing ever. But, it's the start of building a better testing framework. Doing the same for the real tables will make it much easier to compare a bill before and after running through our application.

#!/usr/bin/python

# -*- coding: latin-1 -*-



from elixir import *



from yaml import load, dump

try:

from yaml import CLoader as Loader

from yaml import CDumper as Dumper

except ImportError:

from yaml import Loader, Dumper





def _toYamlRep(ent):

"""

Given an elixir entity, query the entity's members via its __dict__

and return a dict

"""


ret = {}

for (k,v) in ent.__dict__.items():

if k.startswith('_') or k == 'row_type':

# don't print out the 'hidden' keys

continue

if v:

ret[k] = str(v)

return ret



class YamlEntity(Entity):

def toYamlRep(self):

"""

Wrap the _toYamlRep() dict in another dict, use the class' name as the header.

"""


return {self.__class__.__name__ : _toYamlRep(self) }



class Movie(YamlEntity):

title = Field(String(30))

year = Field(Integer())

description = Field(Text())

director = ManyToOne('Director')



def __repr__(self):

return '<Movie: "%s" (%d)>'%(self.title,self.year)





class Director(YamlEntity):

name = Field(String(60))

movies = OneToMany('Movie')



def __repr__(self):

return '<Director: "%s">'%(self.name)





def main():

metadata.bind = "sqlite://"

setup_all()

create_all()



rscott = Director(name="Ridley Scott")

glucas = Director(name="George Lucas")

alien = Movie(title="Alien", year=1979, director=rscott)

swars = Movie(title="Star Wars", year=1977, director=glucas)

brunner = Movie(title="Blade Runner", year=1982, director=rscott)



session.flush()



for m in Movie.query().all():

print dump(m.toYamlRep(),Dumper=Dumper,default_flow_style=False)



cleanup_all()



if __name__ == '__main__':

main()






And, here's the output:



Movie:
director: '<Director: "George Lucas">'
director_id: '2'
id: '3'
title: Star Wars
year: '1977'

Movie:
director: '<Director: "Ridley Scott">'
director_id: '1'
id: '4'
title: Alien
year: '1979'

Movie:
director: '<Director: "Ridley Scott">'
director_id: '1'
id: '5'
title: Blade Runner
year: '1982'

cruisecontrol == teh suck

Every time I try to dive into configuring CruiseControl, I hate it a little bit more.

Wednesday, May 21, 2008

can it be? finally done? ... WTF?

And now, the servers have been installed (all except for the HP-UX I may give up on).

after the following loop through the tech support desk:
  1. go to system A
  2. go to system B
  3. go to system C
  4. go to system D
  5. go to system F
  6. no, go to system C. enter text 'XYZ A'
  7. FAIL
  8. no, go to system C. enter text 'XYZ B'
  9. SUCCESS

The admin rights have been granted.

It seems obvious now, but if you're not given a definitive answer when you ask "Doesn't Umbrella Corp already have a pool of licenses we can pull from?", keep asking.

Turns out, we had the licenses all along. SWEET ZOMBIE JESUS, that was good times.

Monday, May 19, 2008

progress?

When last we saw "Steve's lame-assed adventures in IT", I was relating my struggles with our IT department. This morning's update:
  • I reply, "I don't know what group name you're asking for? How do I find that information?"
  • Tech-support person #6 replies, "Global group used by your team to access servers."
  • I reply, "I have a 'global group to access servers'? I log in using my XYZ domain account. I don't know what group you're asking for. Is the global group one of the access groups listed in service request system F?"
  • Tech-support person #6 replies, "Yes, global group is a access group."

I don't know if this is progress or not.

Further complicating matters... the process improvements Jeff and I have suggested appear to have helped us meet this month's release. The group's communication is improving, the code reviews caught some subtle problems, and it seems like meeting the deadline may not only be because of the handful of items they dropped off the release plan. And, Jeff and I are spitballing some ideas to for a unit testing framework so the developers can test their modifications without manually running SQL statements to verify the result of their changes.

It seems like we could be successful. If only we're not ground up by the monthly release chaos.

But, is it worth it? I'd hate to run away after only 2 months, but it gets very discouraging. There are at least two friends who have jobs to fill, one unknown but potentially awesome and another known and good-to-awesome.

I don't feel like I'm growing technically. But, I'm definitely growing in terms of dealing with projects, customers, and other developers onshore-and-offshore. Maybe the pain is worth it. On the plus side, Jeff and I have the complete support of the business-side to make the changes we're planning. They're suprisingly enthusiastic.

Sunday, May 18, 2008

are you there Mordac? it's me steve

Because the purpose of my life is to serve as a warning for others, please note the following:
  1. Don't spend the first 3 weeks after your initial "I need software X" request, lackadaisically monitoring the request's progress every 2-3 days. Expect to spend 1-3 hours per day following up on e-mails and on the phone.
  2. Don't bother trying to navigate the 'helpful' self-service website, or the dozen+ service request web apps. Go directly to the Help Line. While one-on-one interaction with actual people is scary, at least there will be someone to hear your plaintive cries.
  3. If tech-support person #1 helpfully forwards your e-mail to person #2 for clarification, don't assume that person #1 would take note of person #2's out-of-office reply that says they'll be out for the rest of the month.
  4. Don't assume #3 wouldn't happen twice. The same day.
  5. Assume every person 'helping' you is actually actively preventing you from reaching your
    goal through a clever misinformation campaign meant to drive you slowly insane.
As the owner of the action item, it does make me feel slightly better to be an obstruction to the people waiting on me to get this done. Their pitiful cries bring delight to my cruel, cruel heart.

Last week's status meeting:

Person Z asks, "Steve, any progress on ?"

I reply, "I've got it installed on the two wintel servers I had admin rights to. I can't find anyone to give me a straight answer on who owns the other two wintel servers, or the 'right' process for requesting admin access. I've entered a Unix service request on the fifth server, I assume someone may see it eventually."

The assorted people in the room nod sympathetically, apparently my struggles are par for the course.

Project Manager F asks, "Is there anything we can do to help?"

"Pray."


Mordac's latest trick, masquerading as other IT service members:
  • As persona #1, point me to intranet URL #1 telling me to use service request system A to apply for admin rights.
  • I submit request, push the approval through my management chain.
  • As persona #2, reject request. "System A is only to request admin access to your desktops. Not servers. You need to follow the set of instructions at this URL."
  • URL #2 describes in the vaguest way possible how to gain access to various intranet web applications through service request system B
  • I reply, "WAH? Person #1 told me this, and its clearly documented at URL #1. If that's wrong, it needs to be updated. I follow your instructions, but I don't see which access group I need to belong to gain access rights, can you clarify?"
  • Persona #2 forwards it along to persona #3.
  • Persona #3 says "You need to follow the instructions at URL #3, and use service request system C."
  • System C looks like its asking for the right information. I submit the request.
  • The request is rejected by Persona #4, "You can't do that for your specific user id. You have to enter your team's group name."
  • This is the first I've heard that my team has a group name.
  • No one on my team is aware we have a group name, or what it may be. Not even those who already have admin on those servers.
  • No reply from Persona #4 yet.