sweet spot

After many self-imposed deadlines came and went, finally listed our house for sale this weekend.
We felt bad about letting the deadlines pass, but we wanted finish all the painting and staging of the house before we listed. Turns out that was a very good idea.

Listed it about 3pm Saturday. Around 5pm Saturday we got a from our realtor saying there was someone who wanted to look. Aaaiiiiieeee! We weren't ready. The house was a mess. Paint cans, cleaning supplies, unpacked rooms. A couple hours of last-minute cleanup got 90% of the rooms ready for viewing. They seemed to like the house, and also said that even with our mess, it was one of the top two homes they'd seen in the area.

Got a wake-up call Sunday from our realtor. Another group wanted a viewing. We didn't have to do much because of the previous day's frantic cleanup. We run a few errands while they look at the house with a realtor. Afterwards, we get back home to be accosted by a family in a mini-van asking about the house.

Late Sunday afternoon, I finally get a breather and mow the lawn. Interrupted once by a couple while mowing the front lawn. Interrupted a second time, and nearly had a heart attack, while mowing the backyard when a realtor waltzes through the gate with another couple in tow asking if they can view the house.

I guess we hit the sweet spot w/ regard to location, price, and size of home. We don't even have the pictures online yet (that should happen today). It's a big relief, seeing the amount of interest in the home. And, seeing that our work over the past 2 months was worth it.

If only our own house hunt was as successful. Still having found the right home after viewing probably 70+ homes. At least we're done with our own home now and we can focus on finding our new home.

update: Got a call from the lovely wife. Another realtor and their clients just dropped by "We were in the neighborhood...". 4 viewings in the 2.5 days it's been listed. Aiiiiieeeee. We're beginning to wonder when we'll have time to do the final cleaning for the open-house next weekend.

stressful vacation, house hunting, awesome basement

Just finished a week of vacation. 'Vacation' being code for fixing up the house -- finished laying the laminate floors, installing new baseboards, some new bathroom cabinets, and lots of painting.

Still not terribly near the end point. Plenty more painting and packing to do before we'd feel comfortable having potential buyers look at the house.

The lovely wife and I have been house hunting with our realtor. So many open houses, they're all a blur now.

The price on one with a super-awesome basement has finally dropped enough to be worth it. Those are indeed built-in sofas. The tales those cushions could tell.

But seriously, that house has a great location, great square footage, and the extra fantastic basement. And, the price has dropped enough that we should be able to update the kitchen and a couple other out-of-date rooms prior to moving in.

Of course, the basement would have to stay as-is. It's just too awesome.

crappy neighbor water hazard

Three times over the last couple weeks, my and one of my neighbor's backyards have flooded. The picture is the view from my yard into the neighbor's. Our house isn't that effected... but the water is right against the neighbors foundation. Not good.

Seems another neighbor can't figure out their inflatable above-ground pool.

• The first time, I think they forgot to turn on the filter and decided to the drain the funky water
• The second time, they started refilling the pool and left for the day.
  
• The third time (two days after flood #2), they hadn't tightened a coupling and didn't notice the leak.

On the plus side... I've talked to my fellow-floodee neighbor more in the last week than I have in the ~2 years they've been in their house.

On the minus side. The flooder neighbor seems more annoyed that we're upset, rather than apologetic. I understand that once the water is out there isn't much they can do about it. But, it'd be nice if they seemed slightly concerned about things they do that effect their neighbors.

mmm... pie!

A friend helped us move our furniture around for the work we're doing on the house. Last time, he and his girlfriend raved about Left Fork Grill (68 W. 3900 South, 266-4322). We had him over again this weekend to move furniture back into the finished room, and as a reward for helping us we stopped there for breakfast.

I had a breakfast sandwich (sausage and egg), the lovely wife french toast, and our friend had a fried egg sandwich. Everything was delicious. And, they made hash browns correctly -- the right ratio of crisp to tender, and not greasy.

To top it all off, they have the best pie ever. The lovely wife had banana cream, I had raspberry-rhubarb. Excellent crust, freshly made fillings. Both were great. Many others think so too.

It's shot right up to top of our list of breakfast spots -- great food, reasonable prices.

A shame that it's only open for breakfast and lunch. Pie for breakfast may become a family tradition.

finally, a chance to wear a toolbelt

We've been thinking of moving for a couple years now. But, we'd wanted to get a few things fixed up with the house. I needed to get the yard and flower beds into better shape, and we wanted to replace some of our carpet.

A few months ago, we'd bought some glueless laminate flooring at Costco while it was on sale (ended up saving around $150 because of the sale). Finally got around to starting the project last weekend. Finally realized the extent to which one of our cats ruined the carpet. Ugh.

The flooring went down quick. Not as easy as the instructions make it seem. But, it looks nice, and gave me my once-every-two-years excuse to wear a toolbelt.









And, in odd things I've stuck in my pie-hole news:

• The 'Hog Burger' at Britton's (964 East Union Square in Sandy). A burger with grilled onions nestled between a grilled-cheese sandwich with bacon, and a second grilled-cheese sandwich with tomato. The first half was good. The second half was 'what have I gotten myself into?'
  

impatient book review

I couldn't be bothered to wait the year(s?) for the U.S. release of Steven Erikson's newest Malazan book, Reaper's Gale.

After comparing the Canada/UK prices on the respective Amazon sites, and plugging in the conversion rate, it was cheaper to order from the UK and have it shipped across the Atlantic. Odd.

It was very good. Heartbreaking (and other not-so-heartbreaking) deaths of main characters. Resolution of plot lines. Very enjoyable.

I also picked up Night of Knives, by Ian Cameron Esslemont. Also set in the Malazan universe. It was pretty good, but not great. It could have used another round of editing -- typos and a couple confusing sections. But, it was fast-paced and fun.

roller derby -- violence, short skirts, beer -- what's not to like?

The lovely wife and I went to the roller derby last night. Lots of fun. Lots of violence. Lots of hot chicks in short skirts zooming around the track and knocking each other down. And beer! Mmm... Beer.

It was an All Star match between the local league and their opponents from Idaho.

Salt Lake Software Symposium, day #2

Last day. Still pretty good, but didn't rock as much as the first.

Attended a couple more of Jared Richardson's sessions: "Agile Testing Strategies", and "Software Development Techniques". Lots of good real-world examples, and lots of inspiration for changing our processes. Definitely thinking of picking up Ship It!

Neal Ford's "Pragmatic Extreme Programming" was good. But, I think it'll be easier to work in lessons from Jared's sessions (on days 1 and 2) into our existing process.

I wasn't going to attend Brian Sam-Bodden's "Complex Builds with Ant" session. From the slides, it appeared like I'd already knew the major tips on my own. But, I was wiped out by the last session. Decided it was worth going to a topic I was familiar with in order to pick an expert's brain. Only three attendees, so the session went fast and I was able to ask a lot of questions.

More sour-grapes today over all the Ruby rah-rah-rah. Again, Neal Ford declared Ruby the winner in the dynamic language race on the JVM. Neal is a very smart guy, a great presenter, and also great to talk to one-on-one. I don't have any issue with him advocating Ruby. Any language that makes developers more productive is a good thing. But, it seems disingenuous to declare a victor in a wide open race to a room full of people who have little to no experience with dynamic languages.

Personally, I prefer Python over Ruby. Others can and should disagree. Here's a fair, but slightly biased (since the author is familiar with Python), comparison of Python vs. Ruby.

I'm not sure who to blame for lack of Python excitement in the Java world (or at least at this conference). Maybe with the recent revitalization of the Jython project, it will get more Java developer's attention.

Microsoft's CLR (and the new DLR and Silverlight) seems very exciting. See here and here and here. The second link is a screencast showing interoperation between Ruby, JavaScript, Python, and VB.

All in all, the conference was very good. I'll definitely plan on going next year.

Salt Lake Software Symposium, day #1

Since this blog is all about (pick one):

• lame book reviews
• lame software conference anecdotes
• odd things I've put in my mouth
  

Here's more software conference anecdotes.

Spent today at the Salt Lake Software Symposium (aka No Fluff Just Stuff). So far, the conference is very good. Attended presentations by Neal Ford, Jared Richardson, and Brian Sletten. All great presenters, showing off great technology.

A friend has tried 2-3 times to explain Aspect Oriented Programming to me. It sounded... like more trouble than it was worth. Brian Sletten's AOP presentation was very useful in cluing me in. Seeing the AspectJ/Eclipse tools in action got me all excited to try it out. The immediate use-case I have would to remove a bunch of cut-n-paste concurrent locking boilerplate in my pipeline application. If I understood it correctly (always a questionable assumption), I can replace all that code scattered throughout my class with an aspect that'll do the

  writeLock();
  try {
       // .... do stuff....
  } finally {
       releaseWriteLock();
  }

logic all from a single place for all the setters methods in my class. And a similar readLock() aspect for my getters.

Brian also presented NetKernel. Also awesome. Still trying to get a handle on how exactly we can leverage it for our needs. His Mashup/SemanticWeb/RDF presentation was also good.

Jared Richardson's presentation Shippers Unite was very good. The best part, calling sales people 'Sales Critters'. He had lots of great advice, and great best-practice type information.

Neal Ford's SOA presentation, and his keynote after dinner were also very good. His keynote, "Polyglot Programming" was very interesting. Contrasted the increasing cruft, and complexity (and irrelevancy) of the Java language against the awesomeness of the Java platform. Described the increasing place Dynamic Languages will have within the software industry. In particular their place inside managed runtime environments (either JVM or CLR).

The only major disappointment I experienced was the (apparent) disdain the conference presenters seemed to show for Python/Jython. Lots of Groovy/Ruby/JRuby/Grails rah-rah-rah. I know Rails is the new poster child, and Ruby folks should be proud. But, to give Python/Jython/IronPython hardly a mention. And, the one time the word Python appeared on a slide, to explicitly declare Ruby the winner over Python... seems premature.

Yeah, yeah. Sour grapes. Maybe I should try Ruby before deciding I don't like it... but the Perl-like $ and @ syntax rubs me the wrong way. And, maybe I should just be happy about the rising tide of dynamically typed languages that'll lift all boats.

discrimination!

Someone brought a delicious looking treat in to work today.

The sign next to it:

APRICOT CAKE
____________

(no nuts)


And, in other lame joke news, my new favorite joke (the bottom panels) . I'm just that classy.

mmm... pie.

It's turns of phrase like this that keep me going back to Worse Than Failure :

Of course, upon his return to work, fate crapped on his hope and served him up a big, steaming slice of shattered dream pie. The familiar stench of hot computers and death descended on him while he tried to shut out the familiar fear of dying in the room and having the network absorb nutrients from his decaying body.

san francisco -- day #4 -- "You like extreme?"

Final conference sessions today. OK, but nothing fantastic.

Great lunch at House of Nanking. The waiter arrived, said "Your first time here?" When we said yes, he took our menus. No idea what the dishes were, but they were all super-excellent.

Then got a taxi to the airport. After a few minutes of silence, the driver asks, "You like extreme?"

I'm not sure I hear him right. He asks again, "You like extreme?"

Yep. I had heard him right.

Pause....... I'm not sure if I'm going to get invited to a cock fight, or what. I guess that maybe he's asking if it's OK to play his favorite thrash metal CD. "Uhhh.... extreme music?" I ask.

He explains, "No! Extreme! Like X! .... Games! Extreme. Sports."

I respond, "umm. A little?"

The rest of the ride was enclosed in blissful silence.

san francisco -- day #3

Noisey street. Some sleepy sessions. A couple awesome ones -- displaying debugging and profiling support for XQuery.

Then the final session of the day. At first, I'm thinking... blah, a mailing list archive viewer? Oh, how wrong I was. Sweet zombie Jesus, it was cool. Or maybe Jason Hunter is an great speaker. Or maybe both?

It was also very applicable to the work we're doing -- transforming and enhancing XML and non-XML assets into an normalized XML form and storing the results in MarkLogic.

san francisco -- day #2

Noisey street + hotel room close to street = no sleep.

As the opening session begins, I realize I can't focus. Then I notice the strobing migraine precursor.... Today is going to be awesome!

Interesting sessions. One of the sessions on the technical track of the conference seemed like a waste of time, but most of the others were interesting. And an added plus, presentations by the vendor and one of our competitors seem to validate our current architectural approach.

san francisco -- day #1

Flew to San Francisco for a technical conference. Flight uneventful, and in a teeny CRJ200. Only 12 and a half rows of seats.

Googled up restaurants near the hotel. I've been craving Vietnamese all the time lately, so I decided on a hole-in-the-wall place with good reviews -- Golden Flower Vietnamese. Just a couple minutes walk from the hotel.

Pretty good, but not the best I've had. The imperial rolls were greasy, and more mystery-meaty than I'd have preferred. The bun with grilled pork and shrimp was tasty.

In order of worst to first:
Pho Cali -> Golden Flower Vietnamese -> Pho Hoa -> Shanghai Cafe -> The deeee-licious versions Dien made for us over at Mark's house. Mmmmm.

Also, it's less fun to travel without coworkers. Wandering aimlessly is less fun to do on your own... it feels more like getting lost than having an adventure.

additional lame book reviews

Yes, more reviews. Why the wait? Because I finally read some books again during a few days of vacation.

Day Watch by Sergei Lukyaneko -- pretty similar to the first book, Night Watch. As good as the first, and sometimes as infuriating (wah? where'd that plot twist come from?). But pretty fun. Hopefully the new movie will be good as well.

Storm Front by Jim Butcher -- the lovely wife and I are enjoying the Dresden Files on SciFi, so I thought I'd pick up the first book in the series it was based on. Pretty good. On the fence on whether I'll pick up more.

Fiends of the Eastern Front by David Bishop -- What'll that crazy Hitler do next? Hire vampires to help win WWII, that's what. I liked the premise. I liked some scenes. But, ultimately very lame.

Did I mention it was lame? And full of typos? I usually blissfully skip right past most typos, not this time. That must mean they were 2-3x more that I missed.

Sample paraphrased ending scene of each book (in the 3-book omnibus):

books #1 and #2:
Lord Constanta (main baddie) : "Yes. You tried to kill me. Did I mention I'm completely (and lamely) invulnerable? It's because my Sire is so super bad-ass. I'll let you live to the next book, but let me tell you about my ludicrous world domination plans first...."

book #3: Yes. I'm spoiling the ending.

(further expository villains explaining their ludicrous world domination plan ... : "Yes, our plan is to darken the skies over Europe so we don't have to worry about our sunlight imposed curfew. Then we can drink blooooooooood all the time! Muauahahahhaa. No, I don't see the complication that that'll kill all life other than us... C'mon, man. We're gonna teach the stupid sun a lesson.")

And, the horrible, horrible, horrible ending:

The Sire (basically a big vampire bat you might expect to see in a Godzilla flick) : Bwaaaaah! Watch me fly outta this lake of blood. Oh nos! The day-walker hero is doing his homage to Slim Pickens in Dr. Strangelove by riding a silver-tipped V1 rocket into me. I esplode. When I gave Lord Constanta his resurrection powers, I forgot to give them to my self. Curse my strict FIFO ordering of completing my TODO list.

I was willing to give the books a little credit until the ultra-lame ending. Christ on a pogo stick, that was awful.

change the world?

I've been getting more and more excited by the OLPC project. Not only is it exciting technically, it seems like it has huge potential to change the world.

It'll take a few years before the educational goals bear fruit, but it'll have a huge technical impact much sooner. A lot of the cool technology could make its way out to non-OLPC hardware -- displays, power management, networking, security, exciting Python changes... lots of great stuff.

The Daily Python-URL pointed me here. A great introduction to the project, its goals, and the technology.

finally learned my lesson

I clearly remember thinking a year ago: "I've had incredible luck with my computer hardware. No disk, motherboard, power-supply, or other problems with any of the computers I've owned."

Since then, I've replaced the power-supply on my newest computer. TWICE. Both times, the hardware was still under warranty, so financially it hasn't been an issue. But the inconvenience of losing access to the files has been annoying.

After hearing good things about Mozy, I decided to give it a shot. So far, so good... Of course, it's only been an hour. Fingers-crossed for more hardware problems! Woo-hoo!

And, in somewhat related news, I'm finally taking the time to encrypt my sensitive files -- tax returns, contact info, schematics for man-eating robots, muffin recipes, etc. I'm giving TrueCrypt a shot. Because I'm cheap. Also, having a cross-platform encryption solution was a big plus. So far, easy to setup and use.

lesson of the day: just assume Co-Worker #1 is correct

I've been reading Java 2 Performance and Idiom Guide, it's quite good. But, I think I'm too interested in experimenting with the tweaks it suggests.

As I mentioned in my last post I'm working on a data-processing pipeline, and have spent the last couple days investigating why the performance (records/second) of a particular pipeline stage drops off like it does.

The software we're working on analyzes bibliographic records and creates a representative sample. Our users create mappings between one XML schema and another. They have to go through much fewer iterations if they've got a good sample to work with. And iterations are much shorter if they can test their mapping against a sample that's <500 records, rather than 16+ million records.

The oddly performing pipeline stage normalizes XML entities by replacing them with a numeric reference to their Unicode equivalent. Yeah, the DTD should do that... but it doesn't help when the data provider escapes the entities. Or when their DTD replaces the entity... with the entity... argh.

The graph below shows the records/second performance of three stages of a pipeline (y-axis). The x-axis is time. The yellow nodes are each file's entity-replacement performance. The blue nodes are the files getting split into 5000-record chunks and inserted into a Marklogic database. The pink nodes are each 5000-record chunk being analyzed. The first 500 or so files all have 30,000 records; the rest of the files are daily updates that vary in size -- that's why the tail end of each gets a little kooky.The initial steep drop-off for the first few files was understandable. It's one of the first stages of the pipeline, and the threads doing entity-replacement don't have to compete for I/O with other pipeline stages until the following stages have entity-replaced-output to work on. But the continued decline made me worry that there was a problem in the code.

Reviewed the code. Each file is getting processed by its own thread. Each thread should create SAX handling objects that aren't shared between threads. No static fields or other stuff to interact between threads...

But! I notice that the SAX handler uses a StringBuffer to accumulate a records-worth of text as it replaces entities. When endElement is reached it calls sbuf.setLength(0), which my fancy new book learnin' suggested could cause memory/performance problems.

And I set off on my non-adventure....
There are four series on the graph above.

1. Dark blue is the original run.
   
2. Pink is the result of replacing sbuf.setLength(0) with sbuf = new StringBuffer(). No fabulous improvement. Stupid book learnin'.
   
3. I decide that 'new StringBuffer()' meant that memory is getting re-allocated from initial the default StringBuffer size, which isn't as efficient as it could be. Yellow is the result replacing sbuf = new StringBuffer(INIT_LENGTH). INIT_LENGTH is a couple thousand bytes. Still no major improvement.
4. Re-review the code. Realize that the FileInputStream/FileOutputStream objects aren't wrapped with BufferedInputStream/BufferedOutputStream. The light blue nodes are the result of changing that... Still no major improvement. I assume this was because these input files are in Zip files, and output to GZIP files -- so were already buffered (I did increase their buffer sizes along with adding the buffering to the uncompressed-file handling).

Then I remember the comment from Co-Worker #1 when I showed the graph to her before I began my journey to nowhere: "Maybe its just file-size"

I fiddle with my statistics exporting code to include the file-size in the exported spreadsheet.
*@&#$*&!

And the KB/second graph looks like:Moral of the story: Get the most complete picture of a performance problem you can, and don't be swayed by the kooky little performance trivia you read.

a programming post, odd.

At work, I've been working on a data processing pipeline. The meta-data for the pipeline (e.g. status of pipeline phases, data locations, etc) at runtime is held in a LRU cache and flushed to persistent storage. Prior to a couple weeks ago, the storage was just a bunch of files on the filesystem. It worked, but it was pretty slow to collect aggregate information out of it -- e.g. if you wanted a global view of all status within the pipeline, you had to traverse the meta-data graph loading each individual file as the traversal progressed.

I'm allergic to relational databases, and we're already using Marklogic/XQuery to store and transform our data, so we decided to try a new implemenation of the LRU cache with the storage in Marklogic. Performance wasn't impacted much (maybe an extra couple minutes for a 1500+ file pipeline that takes 10+ hours), and now status reports take a couple seconds rather than a few minutes. Hooray!

Since that was less painful, I decided I'd also try getting some performance statistics out of the meta-data in the database. There are a few helper functions to do sum/count/average, but I wanted to avoid iterating over the list multiple times. I found some psuedocode here, and implemented an XQuery version:

(:
This is an XQuery implementation of the variance algorithm on this page:
   http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance

   credited to Donald Knuth, The Art of Computer Programming, vol 2: Seminumerical Algorithms, 3rd edn., p. 232 Boston: Addison-Wesley.

It relies on Marklogic's extensions to XQuery, so it probably won't
work on other XQuery implementations.
:)
define function putil:getSumMeanVariance($srcvals as xs:double*) as node()* {
   let $n := 0
   let $mean := 0
   let $S := 0
   let $sum := 0

   let $throwAway :=
       for $x in $srcvals
       let $delta := $x - $mean
       return (
           xdmp:set($n,$n + 1),
           xdmp:set($mean,$mean + ( $delta div $n ) ),
           xdmp:set($S, $S + $delta * ( $x - $mean )),
           xdmp:set($sum, $sum + $x )
       )
   let $variance :=
           if ($n gt 1) then
               $S div ($n - 1)
           else 0
   return (
       <count>{$n}</count>,
       <sum>{$sum}</sum>,
       <mean>{$mean}</mean>,       
       <variance>{$variance} </variance>,
       <stdev>{math:sqrt($variance)}</stdev>
   )
}


let $elapsedTimes :=
 for $pnode in collection($collName)[ .... uninteresting XPath predicate stuff here .... ]
 return $pnode/stats/time/end - $pnode/stats/time/start
return <stats>{getSumMeanVariance($elapsedTimes)}</stats>