Tuesday, November 25, 2008

further adventures in wasting time

The 2 days I'd never get back eventually turned into around 4-5 days total of developing, creating unit tests, system testing, and correcting various bugs the tests didn't find.

And now 1 more day. Backing the changes out.

Released ~2 weeks ago; customers and internal support are finally installing the new version and KABOOM! Unknown references in the C Runtime library.

Superfantastic news! Turns out our development / production testing environment isn't at the same HP-UX patch level as most customers' systems. Thanks, Oracle 11g! Thanks, budget constraints that made someone decide they had to stick it on the same server we use for support!

Wednesday, November 12, 2008

twitterpated

I'm all aflutter over the news that HBO has ordered the pilot for Game of Thrones -- http://grrm.livejournal.com/58155.html

Monday, November 10, 2008

'tis the season

For zombies. Sweet, sweet zombies.


http://iamnotinfected.com/

and

http://survivetheoutbreak.com/

great ringtone, or greatest ringtone?

don't gamble

I should learn not to bet for or against the weather.

I'd let the leaves in the yard get away from me -- decided to take a few days off leaf duty until the maples dropped more of their leaves.

I wasn't able to do the raking Saturday, due to a bunch of errands that couldn't wait till Sunday. Decided the weekend was a waste raking-wise due to the predicted weekend rain.

Woke up Sunday to see that it hadn't rained. Decided I didn't want to get started on the leaves since the clouds looked ominous, and instead started on the rain gutters. Ick. No rain by the time I'd finished, and I was filthy anyway, so I started in on the leaves. Promised myself I'd go until the rain started.

No rain. For the four hours it took me to get the leaves off the front and side yard. Lame. Stupid rain, you were supposed to give me an excuse to go inside and watch bad TV.

16 new bags o' leaves to add to the 10+ from previous weekends. Still have the backyard to do, and about a quarter of the leaves from the maples in the front yard.

Wednesday, November 05, 2008

tell me what to do, lazyweb!

I want a laptop. Or a netbook. I don't particularly want to buy both.

After trying to type on a Eee PC 900(?), if I go the netbook route I definitely want to actually try typing on it in person. Right now, leaning towards a Eee 1000H or maybe the Samsung NC10.

Or, do I just spend another $800 or so and get a just-as-portable but more powerful system? Leaning towards either the Lenovo Thinkpad X200 (non-tablet) or buckling to peer pressure and getting a MacBook.

Or... do I go for an even bigger model so that Eclipse isn't all scrunched up? Or, do I just use it for 'fun' projects where Vim is all I'll need?

Yeah, I don't know what I want, and I've got conflicting requirements. Small! Big! Cheap! Oh, alright, spend 3x as much.

A decent keyboard and sturdy build are my only consistent criteria.

Monday, October 27, 2008

2 days I'll never get back

Through sheer happenstance, one of the test databases for our new system was (mis)configured in such a way that an Oracle sequence was generating very large ID's for a certain field.

Very large as in 123,456,789,236

Guess which horrible application uses a 'long' type to hold those IDs? Guess which horrible application is supported on both HP-UX and Windows? Guess what type might be 32bit on Windows and 64bit on HP-UX? Ding! Ding! Ding!

Spent all day Friday and today rejiggering the code to use 'long long'. Most of today was spent un-rejiggering and re-rejiggering the code to work around Oracle Pro*C's inability to use 'long long' as a host variable -- today I had to swap all that out of the embedded SQL in order to use character strings and stroll/sprintf to move values from the strings to 'long long' and vice-versa.

I had thought the code was 'OK' on HP-UX... after all these recent changes, I'm not so sure. The code to transfer the values from the query results to their 'long' variable instances seems wrong. In any case... it'll work the same on both platforms now.

I take some solace from finding the problem before any customer. We have at least 3 customers nearing 1-billion bills -- although the horrible app is 10+ years old, the rate they're adding bills to their systems grows every year. They can also start/change their sequences whenever they feel like it. So, maybe we had another 1-5 years before it became a major emergency.

Wednesday, October 22, 2008

disappointment, thy name is tiny keyboard

I've been resolutely avoiding purchasing an Eee PC (or other netbook) until I actually could get my hands on one and try typing.

Tried out an Eee 900? at BestBuy the other day -- not sure which exact 900 model. Too small. Maybe I'd eventually get used to it, but I've decided I definitely should go with the slightly larger Eee 1000H.

Tuesday, October 21, 2008

not spam

I never imagined I'd get an e-mail with subject line "Help with goat talk" and it would not be spam.

Tuesday, October 14, 2008

which is better?

  1. Laughing/crying at the right-wing, loony spam mails I get from my kook of an Aunt.
  2. Laughing at the "in no way related" e-mail containing the facts behind the rumors that inevitably follows a few days later
The latest example, a forward from the doublespeak-tastic 'American Family Council', with the subject line 'Fwd: If you think things are bad now, just wait....'
{snip}

In case you may think I’m a “the sky is falling” type of person, you should know: When it comes to predictions, I am a very reserved person. But not on this one. I cannot overstate the damage a liberal victory will do to our country. The upcoming election is the most important in the history of our nation. Yes, if the liberals win you will lose some of your religious freedoms and free speech rights. Churches and pastors will not be exempt. You will not be allowed to say certain things about a particular group. Every item of the homosexual agenda will be approved. All the laws protecting the unborn will be wiped away.

{snip}
And 5 days later, the next e-mail from her with the subject line, 'In Fairness to B. Obama. AND J. McCain.....Snopes.com' :
There are so many derogatory Political Emails floating thru space out there, that I am personally tired of having to wade thru them all - and acknowledge that it's hard to get the straight-forward Truth....

I think it might do us all well to consult those who have done the research on these (either bogus or real-truth) emails. (See links below)

Click here: snopes.com: Politics (Barack Obama)

Click here: snopes.com: Politics (John McCain)
My favorite part of the inevitable follow-up e-mail is the lack of self-awareness or any hint of an apology for adding to the onslaught of misinformation herself. Despite the fact that no one has asked her to, I'm glad she's warned us that she's "personally tired of having to wade thru" all those spam messages she passes along.

I've decided whenever I get baited into a political or religious discussion with my family, I'll make the saving throw by chiming in with, "While I appreciate that these Gay Agendas (*music sting*) make our meetings well organized, they're really making me re-think my conversion to Secret Islamism."

P.S. Also plus too as well : http://www.factcheck.org/

worst. impression. ever.

Dear Autoflush Toilet,

You ain't foolin' me! You're not a bidet!

Why does your sensor think my back is transparent?

You're horrible.

Love,
Steve



P.S. Now that you're thinking about me pooping, I'd like to point out that I deftly avoided use of any onomatopoeia.

P.P.S. Mostly because I couldn't connect the dots between this post and the punchline using the consignment store named "Grunts & Postures" near the first apartments of my wife and I.

P.P.P.S. First one to comment with a "full of shit" joke gets 15 demerits.

P.P.P.P.S. Yes, despite the demerits, it is still worth it.

Wednesday, October 08, 2008

hudson == teh awesome

Hudson is great. Lightyears beyond CruiseControl. Everything configurable through the (awesome) UI, a ecosystem of plugins, and incredibly simple installation.

We don't have that complex a build, but our SCM team looks at me like I'm a crazy person when I point out yet another issue with the build. After the umpteenth time I've found yet another build problem, I decided to give Hudson a try. We need to checkout some modules from the trunk, and others from a branch. It was bad enough when their chosen solution to that problem meant we couldn't automatically trigger the builds with a check-in -- builds had to be forced from the UI, then a pre-build step did a CVS checkout of the modules from the appropriate branches. The last straw was realizing that the pre-build script also contained some logic that overwrote any changes we made to our build.xml file with some out-of-synch version of build.xml that one of them had created at some point in the past. The sort of thing you'd like a heads up about.

With Hudson, builds have been a snap to set up -- monitor these CVS modules, run this script from the repository if any changes are checked-in, the build artifacts will match this regex pattern allow download of them from the UI, keep only X number of builds on the CI server.

Hudson doesn't let you checkout from a mixture of HEAD/branch either -- but it's simple enough to set up multiple build commands, the first of which is doing the branch checkout. We won't get a complete changelog or CI automation, but it's good enough.

Tuesday, September 30, 2008

lessons in conflation : compare != sort

For months, the business people have complained that the rules engine sporadically produces different results on Windows than HP-UX. They'd complained previous developers said, "sorting is different on Windows and Unix". I did my best to restrain any reply more caustic than, "That's silly."

Until this month they couldn't point me to bills that had the problem at the same time as we had log files from our regression tests (or find me any archived e-mails from the original developers). Given that we're probably the only people running the application on both HP-UX and Windows -- during our regression testing on the two platforms -- the bug hasn't been that high a priority.

Finally tracked down the cause of the issue to be differences in the qsort() algorithm of the C runtime lib -- specifically how qsort() handles two elements that compare as equal (!). For some reason, the Windows qsort() performs some extra comparisons and 1 extra swap.

The list we're sorting looks something like :

1. $23.96
2. $23.96
3. $23.96
4. $23.96
5. $40.95

On Unix, qsort() and this particular compare (descending order by amount) results in the following :

5. $40.95
1. $23.96
2. $23.96
3. $23.96
4. $23.96

On Windows, its qsort() and the same compare function resulted in:

5. $40.95
3. $23.96
1. $23.96
2. $23.96
4. $23.96

The values we're comparing are equal... but what we're actually sorting is the value and a reference to where it started in the list. Because the sorting produces different results we end up annotating the bill lines differently on Windows than Unix.

Changing how we do the comparison fixed the problem. Now we compare left/right values, if they are equal we compare the originating line number of the values.

And now we get consistent results no matter the platform:

5. $40.95
1. $23.96
2. $23.96
3. $23.96
4. $23.96


Next time, in Steve's job rants:
  • Typedefs on top of typedefs on top of typedefs. Oh yeah, and because of typedefs A, B, C, and D it turns out 95% of calls within the system look like pass-by-value but are in fact pass-by-reference. It took me ~2 months to realize that was going on. 4 months later and it still gives me the heebie jeebies.

Saturday, September 27, 2008

loving/hating Fit

We waste a huge amount of time on our legacy code base because we have no Unit tests, and no standard way the project's developers test their changes.

It's typically some error-prone manual process -- run some SQL in Toad, copy results to Excel, run the bill, re-run SQL, compare to the old results with eyeballs, pray.

I read about Fit a few months ago, and it seemed like it'd solve a lot of our problems. Blackbox testing would help build tests up around our legacy codebase. It'd also help bridge the communication gaps between onshore/offshore development, and development/business.

Finally took the time to play with it this week -- specifically, DbFit. Without any customization, I was able to write some simple SQL in a wiki page to generate a new test case for the issue I was working on this week.

I'm in love. It seems like with some minor customization we can make some huge leaps in our productivity. But I really hate trying to navigate the Fit/Fitnesse web sites for reference information. Ugh.

additional offshoring adventures

My project at work is a set of Java applications for which communicate over UDP / RMI. The port numbers they talk over is configurable. In our hosted environment our servers may be running the apps for multiple clients. So, the users (or our tech support people) have to configure unique port #'s in each set of config files if there are multiple s running against different databases. It's a annoying, but the least of our worries at the moment.

Monday this week, I realize our configuration editor doesn't give the user the ability to edit a certain set of port #s. I ask the offshore team to add that ability to the config UI.

When I come in the next day the offshore lead has decided the existing way we do things is too error prone. He's made a great change so that the configurable settings are just a range of port #'s and the application(s) will pick unused ports within the range.

After reviewing the changes, the only issue I found was in some output files created by the apps once they pick a port number. Once the port # is selected, an output file will be written. It was implemented with a DataOutputStream to write an int primitive to the file, and read back in with DataInputStream. I'd prefer it to be human readable, so I respond with some notes saying to make the file a text file rather than a data file.

I come in the following day, and find that the only thing he's changed is to give the file a ".txt" file extension.

Wah?

It's the 'little' things like this that drive me crazy. Things I assume don't need much specification. It's particularly annoying because this engineer is so awesome -- he's always quick to turn around any solution, and always asks great questions on our calls. And, he did a great job of taking the initiative to fix a major flaw just a day earlier.

Tuesday, September 16, 2008

further adventures in offshoring

Have not had time to do a code review for the project I've inherited. Decided to use FindBugs to pick off low hanging fruit.

aiiiiiiiieeeee.
  • bitwise '&' operator used instead of boolean '&&' operator
  • '==' operator used for Long/Integer/many-other-reference-types
  • String.equals() comparing a String to a DO object
  • Enum classes with their own extra-special equals() methods. Ick.
  • Classes with equals() but no hashCode()
  • Who implements finalize()?
  • many, many more

Saturday, August 16, 2008

an afternoon I'll never get back, thanks Eclipse!

Spent Friday afternoon fighting with Eclipse. Around lunchtime, my development environment stopped working... When launched from Eclipse the app couldn't find the log4j, Spring, or Hibernate config files. After many false starts and much gnashing of teeth, finally noticed it wasn't all config files that weren't getting copied into the projects' output folders but only .xml files.

Finally managed the right phrase in Google that found me the answer: http://dev.eclipse.org/newslists/news.eclipse.platform/msg74562.html

Apparently, Eclipse plugins may update one of the preferences that tells the builders which file extensions it shouldn't copy to the build output path. To fix it, you have to go to 'Preferences... -> Java -> Compiler -> Building' and then check all the extensions entered into the Filtered Resources text field.

I'm not sure if it was a plugin change, or (probably more likely) changes I was making to my launch configurations to not run out of the 'dist' directory into which the inter-project Ant build packages everything.

I'm not sure if I should be happy that the offshore team I've now inherited at least got the Ant build right. Or, if I should be concerned about whether they've actually been running the code / configuration they think they are when running under Eclipse.

One more in a long line of complaints I've had regarding how this set of applications is configured. Normally, I'm the one on whatever-team-I'm-on pushing for extensive runtime configurability. I don't know why this project's configuration setup -- not the Spring/Hibernate, the home-grown .properties file reading and command-line handling stuff -- rubs me the wrong way. The nonsensical or nonexistent documentation, the obfuscated search paths, the unhelpful error messages, or the fragility of the whole system when you're missing one piece... all of the above?


Even with that lost afternoon, I still love Eclipse. After all, it's free. Oooh, free. One plugin that now I can't live without: Remote System Explorer.

I've got to run Windows at work, and the corporation-approved SSH client sucks balls. When you resize a terminal window... what would you expect to happen? More columns/rows? Bah! That's too obvious and old-school for Attachmate Reflection's SSH client-- instead the mother f*&#$*#&er resizes the font. Badly.

PuTTY is mediocre. The corporation's security agent can't be disabled, and it prevents the cygwin install from running completely.

Enter Remote System Explorer. Multiple terminal windows. Does the 'right' thing when resizing or scrolling. Gives me a file-tree view of the remote system via SFTP. I can edit remote files within Eclipse and it automagically saves them to the remote system.

Eclipse is a beast, and a bit of overkill for a shell window. But, I've already got it open. And, unlike my Ubuntu system running under VMWare... I don't feel like the corporation's jackbooted thugs will re-educate me if/when they find I'm using it.

conclusions

  • If you write an application where the order of command-line options matter, you're a bad person.
  • I need to use the adjective 'Diabolical' more often.

Tuesday, August 05, 2008

so many resignations, none of them mine.

Two friends at current employer resigned. Unfortunately, they were both awesome engineers, so the future looks bleak. The only silver lining is there will be less competition for the 'good' projects. I was looking forward to working with them and learning from them.

One friend at old employer resigned. Finally. Sucks for those that are left behind, but it's a great move for him.

As for me, I'm still trying to decide how long to stick it out at the current employer. Finally feel like I'm getting the hang of my current project. We're continuing to refine our processes, and we've got two new engineers that seem to be very good. Now that I've settled into the rut, I've been knocked out of it to take over as tech lead for one of the people that's leaving. More responsibility, higher profile, more interesting work... I should want that right? And not the cozy rut?

In less wishy-washy-I-am-lame news, things I'm enjoying:
I bought Toll of the Hounds a month ago (delivered right after the return from vacation). Decided I'd re-read the Malazan series from the beginning before starting on the new book. Again. I'm amazed that it's holding together on the third read through. Picking up nuances and hints I'd completely missed before. And this is the third read through. I've made it through book four, and decided to read 'A Fire Upon the Deep' as a palate cleanser.

Wednesday, July 16, 2008

another sign of the apocalypse

I was struck speechless while perusing the beer at the supermarket yesterday.

Bud & Clamato == Chelada?

It's not quite intriguing enough to want to try myself. But, inflicted upon someone else... Perhaps a six pack as a house warming gift for my enemies

Tuesday, July 15, 2008

don't eat the chicken pizza

Back from a week in Cancun! Luckily, we only got horribly sick on the last day. Just in time to fly home. Awesome.

We stayed at the Dreams Tulum Resort. For an all-inclusive, it was reasonably priced, and very nice. The staff was awesome. The food at the resort's restaurants was all very good.

The majority of the week was spent listening to downloads of the Geek Show Podcast, sipping mojitos, and staring at the ocean.

Thursday, we took a tour to Chichen Itza. Oh lord... it was a long drive from Tulum. But, the ruins were worth it. Very interesting, it was great to see the scale of things. Unfortunately, climbing the steps is now verboten.

Included in the tour package was a lunch at the buffet at the Mayaland hotel right next to the ruins. It looked risky, but we were lulled into complacency by the luck we had with our resort's food. All four of us got sick that night and the next day.

On the way back from Chichen Itza, we stopped at an amazing cenote. We only had a half hour, so we decided not to take a dip.

Wednesday, July 02, 2008

further lessons in patience

I've been tracking down why a run of the rules engine on the Windows platform produced different results for a particular bill was different from Unix.

After debugging the problem, I tell the business peeps the same thing the other 2 developers have been saying for the past month:
  • I can't reproduce that unexpected behavior on Windows or Unix, using my builds or the official release builds.
  • Also like the other 2 devs, I tell them there's a bug in the code we need to fix. The unexpected behavior is actually the desired behavior and fixing the bug will result in the desired results all the time.
Through pure chance... I realize the order in which bills work their way through the system will affect the outcome of the regression test. If bill A is part of a claim X, and claim X has one or more bills that haven't yet been reprocessed at the time bill A is pulled out of the FIFO bill processing queue... when the rules engine queries the database for the bills that are part of claim X, the engine won't retrieve the yet-to-be-processed bills.

This fits a big symptom of our inconsistent results for this and other bills. We see an unexpected result, and we try to reproduce it after-the-fact... no luck.

I can finally reproduce the original problem, but only if I run the regression test prep scripts, and then manually process a handful of bills from the claim and leave some stuck in limbo.

I naively ask the person that runs the test suite, "I think I found the problem, but to confirm I need to look at the log files from the regression test run on Windows. Do we still have the logs?"

I'm told, "I don't know, I don't think there is one. We were told that on Windows the log files aren't produced consistently."

I look into it for 5 minutes. It turns out no one bothered to configure a log file in the INI file. So, no logs were produced.

I'm dumbstruck. It's the latest in a long string of stupendous tales the previous developers have told the business people and test team. Lots of mythology around why the system acts oddly ... memory leaks, bad casts and pointer references, 'Windows can't do the log files', 'Windows sorts differently', 'You need to enter the password in all caps... No, not with the shift key, you have to enter it with CAPS LOCK'

WTF?

To be fair... I am hearing all these anecdotes second-hand and filtered through the non-technical team. But, taken as a whole I can't decide whether it lessens my confidence in the BAs and dev management, in that they couldn't successfully call the engineers on their bullshit... or if it just confirms my opinion that the engineers were lazy and outright lied about how things worked because they didn't want to take the time to correct the problems.

Saturday, June 21, 2008

triumph! ... oh, dammit

Friday was a bittersweet day.

On our horrible project-released-every-month, they are frequently late starting their regression test, and very often late with the release. In 2007, it was only released on-time twice.

As bad as that is, the good(?) thing is they don't release until their regression test says the only differences are the ones they expect. Unfortunately, that typically means a team of 12+ BA's, developers and testers work the weekend.

But not this month. We started the regression test when we were scheduled to on Monday. And, rather than posting the release on the drop-dead Monday date, we passed the release off to customer-facing site on Friday. This is the first time in years that that has happened.

The process-improvements Jeff and I have been insisting on deserve some of the credit.
Odd things. You know, completely out-of-left-field things like code-reviews and communicating. It's unclear how much of this month's success was the amount of work included in this release, and how much of it was our process changes.

Unfortunately, the day of triumph was made bittersweet by Jeff's resignation. Probably not accurate to call it a resignation, since he was a contractor and decided not to pursue extending his contract. Or, to be precise... not wait until the final day of his contract to find out if they were going to extend his contract.

I don't blame him for finding something else, and I may follow him soon enough. The monthly release meat-grinder can be made less painful... but it'll never be fun.

Wednesday, June 18, 2008

ready. set. go?

I'm coming to the realization that I was so focused on leaving the previous employer that I didn't think critically enough before accepting the offer at current employer. Beyond the many dead horses I've already beaten, there were plenty of hints to which I should have paid attention prior to accepting the offer.

More mixed feelings today after a meeting discussing our project's migration from usage under the current application framework to its replacement over the next 2-3 years. Felt good that our hard work to stabilize the crumbling infrastructure is recognized by the business people. Felt worse when I realized that 2+ years into a multi-million dollar project, they're finally discussing the actual mechanics and workflow of how both systems are going to be developed and maintained together. "Oh yeah... we want to release in October. We need to figure that out."

Nearly all the positives about the new job have been snuffed out. There is the potential for it to get better, but it's hard to hold on to the glimmer of hope. Other than feeling like I'm giving up, it's getting harder to find reasons to stay.

Friends at 4 different employers are looking for people... positions with varying levels of awesomeness. Or, I could strike out on my own and find something different.

But, what do I want? That's the million dollar question.

Someplace more engineer-y than my most recent work in content publishing or healthcare. Some of the work was great and challenging... other parts have been mind-numbing or downright eeeeeeeeviiiiiil. Not that the previous work in simulation was fluffy kittens.

Someplace that views R&D as a vital part of their business plan, rather than whining by engineers for 'fun' work. Maintenance can be fun -- debugging an issue, finding the cause of the crazy and esoteric problems is like unlocking a puzzle and can be very rewarding. But, it's not something that you look forward to doing as your sole activity for the next 3+ years.

Something more back-end rather than UI or web-applications. I'm pretty sure that new fangled interwebernet thing will never work. It's all TUBES! TUBES I TELLS YA!

It'd also be nice to land someplace where I could use Python for more than just throw-away utilities, and not be viewed as a rabble-rouser.

pink eye!

Liz's brother's family visited us for a couple days while they were in town for a funeral. Bart turned us on to one of our new favorite shows, Corner Gas.

Good silly fun. Needs more explosions.

Thursday, May 29, 2008

waaaaaah?

I was feeling good this morning. Ran the regression suite overnight through Purify, and it had found some problems. Uninitialized memory references. Who would have thought that hundreds of 5-50 field structs in a 300 kloc code base with no standard initialization routines would have this problem. Bizarre.

So far I've resisted the urge to go fix all the potential instances of the problems, and only fixed the problems exposed by the regression suite. Maybe for next month's release.

Then I spent about 4 hours in conference calls. At about the third hour in I realize the crick in my neck that I'd blamed on sleeping oddly was more in fact from the previous day's conference calls. Jeff helpfully points out over IM, "Yes, this meeting is a pain in the neck."

Then I finally get a response from technical support on 2 of my tickets, after they'd spent ~3 weeks in a black hole.
"Due to ... blahblahlbha... we cannot install open source applications on any servers."

The offending applications? Zip. Unzip. GNU Make.

I can't decide to laugh or cry. The HP-UX server in question already has the GCC toolchain, GNU tar, ant, CVS, CruiseControl and the list goes on. Not to mention widespread use of Hibernate, Spring, ehcache, and tons of Apache and Jakarta projects throughout our Java and .Net products. WTF?

I've learned my lesson. Attempts at following procedure will only be made as a last resort. I need anti-action-item-Wonder-Woman-bracelets to deflect IT-related action items onto my coworkers.

Speaking of playing dress up...

I have to keep reminding myself I have better things to do than spend $250 on t-shirts.

Sunday, May 25, 2008

further lessons in not believing anyone

More whining about work.

It's becoming increasingly apparent that the previous development staff of Project A were lazy. Definitely not stupid, only people confident in their brilliance could produce such a poorly documented and nightmarish code base.

Lazy in terms of, "Hey, that's acting weird... it seems hard to figure out too. I'll blame X and the business people will live with it."

The latest example is the supposedly 'inconsistent' regression test results. When Jeff and I joined the project, we were repeatedly warned in reverent tones of the odd non-deterministic regression test. The first time the 10k+ bills in the regression suite are run, some bills fail. If you run those bills through again, they work.

On the face of it, it seems very odd. The way it was presented to us it seemed that the failing bills were random, and on re-run the identical bill would produce different results. Various things were blamed: memory leaks, buffer overrun, bad casts.

As we dug into it, it became clear that wasn't the case:

  • the bill processing engine wasn't re-started between the failure and success.
  • for at least one large set of the failing bills, the same bills fail every time the regression test is run.
  • The regression test's preparatory script actually modifies the bill in such a way that they become illegal bills
  • the engine sees the illegal bill, and tweaks it such that works again
  • but, many of the failing bills are only partially tweaked because they have manual override codes. The partial tweaking sets the bill up to fail the first time, but corrects things enough that it will work the second time through the system.

Definitely a bug in the engine. The data that's reset by the prep script shouldn't make it behave that badly.

Saturday, May 24, 2008

python + elixir + pyyaml == yay!

And so... I'm playing with a little toy script to ease some pain at work. We've got a bunch of bills in a very large database, the bill data our application reads/modifies is split across a number of tables.

At the moment, the developers manually run SQL scripts to peek at various bits of data. Definitely a huge waste of time.

So, I'm playing with Python + Elixir + PyYAML to get a script that I can just pass in the bill ID and it'll query the 5-6 tables and serialize the bill to something more human readable. If I play my cards right, I'll never have to deal with Toad again.

Elixir works its magic and I can query my database. The autoload didn't work, and there are a couple tables with 50-100+ columns. Serializing it to YAML without repeating myself is now the trick.

I'm sure there's a better way to do this, but here's what I came up with... applied to the Elixir tutorial. Not the most exciting thing ever. But, it's the start of building a better testing framework. Doing the same for the real tables will make it much easier to compare a bill before and after running through our application.

#!/usr/bin/python

# -*- coding: latin-1 -*-



from elixir import *



from yaml import load, dump

try:

from yaml import CLoader as Loader

from yaml import CDumper as Dumper

except ImportError:

from yaml import Loader, Dumper





def _toYamlRep(ent):

"""

Given an elixir entity, query the entity's members via its __dict__

and return a dict

"""


ret = {}

for (k,v) in ent.__dict__.items():

if k.startswith('_') or k == 'row_type':

# don't print out the 'hidden' keys

continue

if v:

ret[k] = str(v)

return ret



class YamlEntity(Entity):

def toYamlRep(self):

"""

Wrap the _toYamlRep() dict in another dict, use the class' name as the header.

"""


return {self.__class__.__name__ : _toYamlRep(self) }



class Movie(YamlEntity):

title = Field(String(30))

year = Field(Integer())

description = Field(Text())

director = ManyToOne('Director')



def __repr__(self):

return '<Movie: "%s" (%d)>'%(self.title,self.year)





class Director(YamlEntity):

name = Field(String(60))

movies = OneToMany('Movie')



def __repr__(self):

return '<Director: "%s">'%(self.name)





def main():

metadata.bind = "sqlite://"

setup_all()

create_all()



rscott = Director(name="Ridley Scott")

glucas = Director(name="George Lucas")

alien = Movie(title="Alien", year=1979, director=rscott)

swars = Movie(title="Star Wars", year=1977, director=glucas)

brunner = Movie(title="Blade Runner", year=1982, director=rscott)



session.flush()



for m in Movie.query().all():

print dump(m.toYamlRep(),Dumper=Dumper,default_flow_style=False)



cleanup_all()



if __name__ == '__main__':

main()






And, here's the output:



Movie:
director: '<Director: "George Lucas">'
director_id: '2'
id: '3'
title: Star Wars
year: '1977'

Movie:
director: '<Director: "Ridley Scott">'
director_id: '1'
id: '4'
title: Alien
year: '1979'

Movie:
director: '<Director: "Ridley Scott">'
director_id: '1'
id: '5'
title: Blade Runner
year: '1982'

cruisecontrol == teh suck

Every time I try to dive into configuring CruiseControl, I hate it a little bit more.

Wednesday, May 21, 2008

can it be? finally done? ... WTF?

And now, the servers have been installed (all except for the HP-UX I may give up on).

after the following loop through the tech support desk:
  1. go to system A
  2. go to system B
  3. go to system C
  4. go to system D
  5. go to system F
  6. no, go to system C. enter text 'XYZ A'
  7. FAIL
  8. no, go to system C. enter text 'XYZ B'
  9. SUCCESS

The admin rights have been granted.

It seems obvious now, but if you're not given a definitive answer when you ask "Doesn't Umbrella Corp already have a pool of licenses we can pull from?", keep asking.

Turns out, we had the licenses all along. SWEET ZOMBIE JESUS, that was good times.

Monday, May 19, 2008

progress?

When last we saw "Steve's lame-assed adventures in IT", I was relating my struggles with our IT department. This morning's update:
  • I reply, "I don't know what group name you're asking for? How do I find that information?"
  • Tech-support person #6 replies, "Global group used by your team to access servers."
  • I reply, "I have a 'global group to access servers'? I log in using my XYZ domain account. I don't know what group you're asking for. Is the global group one of the access groups listed in service request system F?"
  • Tech-support person #6 replies, "Yes, global group is a access group."

I don't know if this is progress or not.

Further complicating matters... the process improvements Jeff and I have suggested appear to have helped us meet this month's release. The group's communication is improving, the code reviews caught some subtle problems, and it seems like meeting the deadline may not only be because of the handful of items they dropped off the release plan. And, Jeff and I are spitballing some ideas to for a unit testing framework so the developers can test their modifications without manually running SQL statements to verify the result of their changes.

It seems like we could be successful. If only we're not ground up by the monthly release chaos.

But, is it worth it? I'd hate to run away after only 2 months, but it gets very discouraging. There are at least two friends who have jobs to fill, one unknown but potentially awesome and another known and good-to-awesome.

I don't feel like I'm growing technically. But, I'm definitely growing in terms of dealing with projects, customers, and other developers onshore-and-offshore. Maybe the pain is worth it. On the plus side, Jeff and I have the complete support of the business-side to make the changes we're planning. They're suprisingly enthusiastic.

Sunday, May 18, 2008

are you there Mordac? it's me steve

Because the purpose of my life is to serve as a warning for others, please note the following:
  1. Don't spend the first 3 weeks after your initial "I need software X" request, lackadaisically monitoring the request's progress every 2-3 days. Expect to spend 1-3 hours per day following up on e-mails and on the phone.
  2. Don't bother trying to navigate the 'helpful' self-service website, or the dozen+ service request web apps. Go directly to the Help Line. While one-on-one interaction with actual people is scary, at least there will be someone to hear your plaintive cries.
  3. If tech-support person #1 helpfully forwards your e-mail to person #2 for clarification, don't assume that person #1 would take note of person #2's out-of-office reply that says they'll be out for the rest of the month.
  4. Don't assume #3 wouldn't happen twice. The same day.
  5. Assume every person 'helping' you is actually actively preventing you from reaching your
    goal through a clever misinformation campaign meant to drive you slowly insane.
As the owner of the action item, it does make me feel slightly better to be an obstruction to the people waiting on me to get this done. Their pitiful cries bring delight to my cruel, cruel heart.

Last week's status meeting:

Person Z asks, "Steve, any progress on ?"

I reply, "I've got it installed on the two wintel servers I had admin rights to. I can't find anyone to give me a straight answer on who owns the other two wintel servers, or the 'right' process for requesting admin access. I've entered a Unix service request on the fifth server, I assume someone may see it eventually."

The assorted people in the room nod sympathetically, apparently my struggles are par for the course.

Project Manager F asks, "Is there anything we can do to help?"

"Pray."


Mordac's latest trick, masquerading as other IT service members:
  • As persona #1, point me to intranet URL #1 telling me to use service request system A to apply for admin rights.
  • I submit request, push the approval through my management chain.
  • As persona #2, reject request. "System A is only to request admin access to your desktops. Not servers. You need to follow the set of instructions at this URL."
  • URL #2 describes in the vaguest way possible how to gain access to various intranet web applications through service request system B
  • I reply, "WAH? Person #1 told me this, and its clearly documented at URL #1. If that's wrong, it needs to be updated. I follow your instructions, but I don't see which access group I need to belong to gain access rights, can you clarify?"
  • Persona #2 forwards it along to persona #3.
  • Persona #3 says "You need to follow the instructions at URL #3, and use service request system C."
  • System C looks like its asking for the right information. I submit the request.
  • The request is rejected by Persona #4, "You can't do that for your specific user id. You have to enter your team's group name."
  • This is the first I've heard that my team has a group name.
  • No one on my team is aware we have a group name, or what it may be. Not even those who already have admin on those servers.
  • No reply from Persona #4 yet.

Monday, May 05, 2008

project A makes baby jesus cry

And so... for some reason I'd gotten it into my head to volunteer for Project A at work. Project A is a rules engine for processing ... things... Myself and another fellow newbie on the project (Hi Jeff!) are slowly coming up to speed on the crazy.

How crazy? Let me count the ways...
  • Release every month!
  • Without a project manager, development manager, or release manager. Who needs 'em? Crybabies.
  • We can't control the # or scope of change requests coming in each month. State regulations drive all the changes to the system. Some nebulous decision process allows us to occasionally drop some changes from the queue... but we don't know which until the deadline has been blown.
  • Crumbling 12+ year old code base. Not given love or documentation over the years. Occasionally tossed a bucket of fish heads. You can tell which dark corners survived from the halcyon days -- they use the hacked-in exception handling in what appears to be the correct way. While I appreciate the cleverness of adding exception handling to C (preprocessing macros around some setjump/longjump magic), I'd appreciate it more if anyone had bothered to write a brief document describing said 'right' way. An example in the .h file? Pshaw! That's for suckers! In the 3 weeks I've spent diving through the code to correct memory leaks, there's at least 4-5 different exception handling idioms. Many possibly broken sections of code, swallowing exceptions with no comment to explain whether the author did accidentally or on purpose. If there aren't comments, it's usually right... for certain values of right. If there are comments, the author clearly thinks exceptions either work like Java's or has no clue.
  • Functions are named as ambiguously as possible with regard to whether it returns a reference to a list that shouldn't be modified, or a copy of the list that's safe to modify and needs to be cleaned up by the caller.
  • It doesn't matter that developers create the builds pushed to customers, right?
  • Oh, and developers have been doing the source control labeling too.
  • Oh, and they're doing both of those tasks half-assed
  • The developers have been too lazy or too scared of the HP-UX makefile to allow it to work without manually copying all source files from their CVS checkout directory to the application directory.
  • You know what'd be a great idea? Basing the fancy new .Net-based Product B's rules engine on Project A. Wait... let's add a Java JNI wrapper around its re-packaged Project A. Awww, yeah... now that's good and f*^@#$ed up.
  • Cherry on top: some genius decides to branch to support Project B. "Branches are neat! Wait... branches are hard... oh well, we'll just branch this one directory that has the DAO stuff, that code definitely has to be different between Project A and B. That's what branching is for, right?" The branched directory also contains nearly identical files that will be slowly, and not-so-slowly , diverging to meet each project's mostly identical monthly release requirements. What? Labels on the branch/trunk to indicate merged code? Nah, that'd be too helpful. I should consider myself lucky that the branch has only existed for a year.
  • Oh yeah, we're releasing Project B mid-Summer. Sweet!
Somehow, I'm the calming influence on Jeff. Probably up to the point where he reads this rant. C'mon, we're in it together! You know, like Musketeers! But less flouncy.

I'm pretty sure I'm living in a sitcom written by The Daily WTF. The alternative is unthinkable.

This is Mark's cue to waltz through with a comment where he doesn't say a thing. Which I appreciate.

Saturday, April 19, 2008

more ponytails than should be allowed by law

Decided to go to the April UJUG meeting a couple nights ago. The presenters varied a little in quality, but were average to great. The topic was interesting, I would have enjoyed more "What went horribly wrong" war stories.

I thought it was telling that even though Maven was held up as a savior for many of the projects, everyone who spoke up mentioned their love/hate relationship with it. It sounded like the LDS church's team had embraced it the most, and a big part of their success with it appears to be because they've dedicated 2-3 engineers to supporting it full time within their organization.

Turnout was huge, 100+. I think I prefer the smaller, and less formal, UPyUG meetings. And it's not just because I think software developers with long hair look silly. So many ponytails at the UJUG meeting. And at least one beret.

Thursday, March 27, 2008

Spring, best thing since sliced bread?

I've been reading a lot about the Spring Framework lately. It's been aggravating me... realizing how much easier it could have made the last 2 years of my life, if I'd dug into it before starting on my last big project.

I'm enjoying Spring in Action. Only a few chapters into it. So far a good introductory book. I like the light tone of the examples, but the jokes are becoming grating.

I also picked up Expert One-on-One J2EE Development without EJB. I'm pretty sure it wins for the creepiest cover on a technical book ever. Kids, if he pulls up to your playground in a ice cream truck, RUN AWAY!

what? just because we don't have a customer we have to stop working?

The job switch has been... odd. Spent the first week and a half scrambling, reading non-stop to catch up with the rest of the team on the Java frameworks we'll be using.

Spent the second week and a half wondering what I'll be working on. The original project was put 'on hold' due to lack of business funding. We'll be shuffled off onto different projects for the time being, likely never to return to the original project.

It's good times.

Sunday, March 16, 2008

chastened by mother nature for act of hubris

After enjoying spring-ish weather, walking around the new neighborhood, and doing some cleanup in the yard... we decided to purchase some patio furniture for the back deck.

I feel I must apologize to everyone in the valley for resulting snow fall. We had at least 2-3 inches at our house.

Tuesday, February 26, 2008

adrift

Not sure why I feel more adrift now that I've not only made the decision to leave my current employer, I've done a bunch of job hunting, interviewed and accepted an offer.

I thought once I'd made my decision, things would have solidified in my mind. Maybe I'm realizing how much I'll miss my co-workers. Nothing like them being laid off, and conspiring with them against 'the man,' to bring you closer together. Their last day is Friday the 29th. My last day is the following Friday, and I'll start the new job the next Monday.

It was difficult to choose where to go. I could go someplace unknown, or I could go back to the ex-employer, or I could follow some friends that left current-employer last year. Doors #2 and #3 seem fairly recession proof; military work for the former, health for the latter.

All the reasons I had for picking door #2 were comfort-based -- very close to the new house, great friends, and interesting work. Door #3 is more of an unknown and a ton more responsibility, but with a safety net of other good friends. Ultimately picked door #3, the riskier job that could be huge career-wise.

Tuesday, February 19, 2008

eclipse + cdt + cygwin = hours of misery! hooray!

Holy dammit Christmas, every step along the way to debug a simple hello world app is an exercise in misery.

  • it appears to have found make/gcc/etc... but then won't actually run anything. Launching eclipse via a batch script that adds c:\cygwin\bin to the path before starting eclipse fixes this. It would have been nice if eclipse threw up more red flags about not finding the things it needed.
  • I get everything building, launch the debugger and it complains that it can't find the source file. This was answered by the FAQ... once I found the FAQ.
I hate that I didn't think to look for the FAQ as the first step. I hate that the top google search results were all 3 or more years out of date. I hate Eclipse's web site -- it's as bad as my employer's intranet site.

Monday, January 21, 2008

demotivation

After a Holiday season filled with indecision over staying at my employer, I got motivated again last week. I decided that there was interesting work to be done, and that I really like everyone I'm working with.

Then, the bombshell Wednesday. Lay offs. Because while we're profitable, we're not growing fast enough for our parent company. Why not outsource all our content production? All our competitors are doing it, so it must work! Quick, outsource everything in a month and a half!

Out of my group of 10 (including my manager), I'm one of 4 software developers left. The content production team was hit much worse, only 2 left out of over 20.

Now we're looking at an avalanche of legacy code maintenance work for at least the next few months as we train outsourced software developers to maintain our convoluted production system that has grown into a horrible mess over the last 10-15 years. There are a ton of services and applications that interact in odd and poorly documented ways. As if that wasn't bad enough, anything over 5 years old frequently isn't in source control.

We have an experienced set of users who know how to push the data through the tangle of applications. We'll lose that in a little over a month.

Fun times ahead.

Tuesday, January 08, 2008

house chaos, kitchen chaos, more lame book reviews

We've slowly been settling into the house. Hampered in large part by our decision to remodel the kitchen. So far, so good. It has stayed on schedule, and is turning out well.

A little over one more week of no kitchen sink or appliances to go, then we'll be able to stop living off of the kindness of friends and family, the mini-fridge and microwave, and waaaaaaay too many take-away and fast food dinners.

Further lame book reviews accumulated during the move and holidays. To further distill my always lame, and always blessedly brief reviews, they will be presented in list form.

First the Good:
  • Camouflage, by Joe Haldeman. Interesting alien(s).
  • They Call me Naughty Lola, edited by David Rose. A collection of personals from the London Review of Books. Bizarre and very funny.
  • Do Androids Dream of Electric Sheep (aka Bladerunner), Philip K. Dick.
  • Also decided to pick up some science fiction classics I had never read. All pretty fun. A Deepness in the Sky, by Vernor Vinge. Also, The Mote in God's Eye, and The Gripping Hand, by Larry Niven and Jerry Pournelle.
Next, the mediocre:
  • Storm Front, by Jim Butcher
  • The Summer Tree, The Wandering Fire, and The Darkest Road -- by Guy Gavriel Kay. Re-read these. I remember picking them up years ago because I'd heard such good reviews. I remembered not liking them much. I thought I'd give them another go since my tastes have changed quite a bit since then. Still found them very annoying. Stilted dialogue, and the whole college-kids transported into alternate medieval reality thing kept making me think Ren Faire.