• Seiri (organization). You need to be able to find things easily.  In software, proper naming helps you find things easily.
• Seiton (tidiness; systematize). Everything has its appropriate place.  Code should be where it is expected to be, and if not, refactor to get it there.
• Seiso (cleaning; shine). Keep the workplace clean for easier thinking (think grease, wires, etc.).  In code, leaving commented-out code and inappropriate comments increases mental clutter that has to be overcome.
• Seiketso (standardization). The group agrees on how to keep the work place clean.  Consistent coding standards and patterns apply here.
• Shutsuke (self-discipline). Having the internal discipline to follow the practices and to reflect on one’s work.

If you take a moment to really think about these parallels between the factory maintenance floor and the code we have to work in, these practices that reduce mental clutter on the factory floor apply equally well to developers working with code.  Less clutter means higher productivity and later mistakes.  For me, it was like a light bulb turned on.

One of his major points is that in coding, we spend 90% of our time reading, and only 10% of our time writing.  Thus, what we do to improve the efficiency of reading carries further weight.  Unfortunately, we are measured (subjectively or objectively) in terms of writing productivity, passing the costs on to fellow programmers later on.

It reminds me of what I wrote earlier while musing on Steve Krug’s Don’t Make Me Think as applied to web site design and usability.  By writing clean code, we require later developers to not have to think more deeply than necessary when reading and using our code by reducing misdirection and mental clutter.

I’m on page 21 of the book now, and looking forward to working through it in continuing to improve my own software craftsmanship.

Basically, fluent interfaces have setter functions return self (or this in Java).  By doing this, you can chain the setters together into what looks more like a composed english sentence instead of multiple single-line setter statements.  I find it easier to code and read code written in this manner (particularly in the jQuery style).

Sitting here watching Olympic couple’s figure skating with my wife, it seems like the perfect time to put some example ruby code together for this (as well as the Self Yield spell from Paolo Perrotta’s Metaprogramming Ruby):

class Car
    attr_accessor :speed, :wheels, :direction

    def initialize
        self
    end

    def moving_at(mph)
        self.speed = mph
        self
    end

    def with_wheel_count(count)
        self.wheels = count
        self
    end

    def heading_toward(toward)
        self.direction = toward
        self
    end

    def display
        puts "speed is #{speed}"
        puts "moving in direction #{direction}"
        puts "number of wheels is #{wheels}"
    end
end

car = Car.new.moving_at(65).heading_toward("NE").with_wheel_count(4).display

attr_accessor would not have returned the object instance, requiring a different line for each mutator.

exercises="ex1 ex2 ex3"
for exercise in exercises; do
    mkdir $home/$exercise
    filename=Target${exercise}Config.xml
done

The problem was I did not want the file to be named Targetex1Config.xml, but TargetExercise1Config.xml, respectively.  So, I wondered if it was possible to do double-dereferencing (i.e., go from exercise to ex1 to Exercise1, such as in the following:

exercises="ex1 ex2 ex3"
ex1=Exercise1
ex2=Exercise2
ex3=Exercise3
for exercise in exercises; do
    mkdir $home/$exercise
    filename=Target${${exercise}}Config.xml
done

That did not work, but a friend showed me the following use of SED with a regular expression to do the replacement:

exercises="ex1 ex2 ex3"
for exercise in exercises; do
    mkdir $home/$exercise
    newName=`echo $exercise | sed -e s/ex/Exercise/`
    filename=Target${newName}Config.xml
done

This worked like a charm (at least once I figured out that the ‘`’ above is not an apostrophe but the backward apostrophe underneath the ‘~’ key).  Later that day when I needed to copy and rename a bunch of files using a boring, repetitive pattern, I used SED to help make that easier:

for file in Provider*.xml ; do
  mv $file `echo $file | sed 's/Ex/Exercise/'` ;
done

I would like to have used Ruby to modify the contents of the files automatically using regular expressions, but Ruby is not available in this environment, and case-sensitivity in changes is an issue. Perhaps AWK can help me out with this. Either way, I plan to dig deeper into SED and AWK.

In Java, classes are defined up front, compiled into bytecode, and remain unaltered.  Every instance created from a class adheres to that class.

Of course, Ruby is different.  In Ruby, you can define a class and later redefine it (and of course, you can modify individual instances of a class so that they differ from the class):

class car
   def drive
      "I'm driving!"
  end
end

# blah blah blah

# Reopen car and monkey patch it by adding a new method
class car
  def brake
     "hitting the brakes!"
  end
end

From the Java point of view, it looks odd to repeat the class keyword twice with the same class.  But in Ruby, the class keyword simply places you back into the scope of the class you are working with (e.g., car).   There is no significant distinction in using it a second time.

What becomes more of an issue is that any plug-in or any code in your Ruby space can monkey patch a top-level entity (e.g., String) and globally change or break behavior.  This could include over-writing an existing method.  Furthermore, it can be difficult to determine what entity made the change.

There are two approaches that were highlighted from Rake and ActiveRecord in Ruby Metaprogramming for safer monkey patching.

Monkey Patch with Module

I lived in Texas for a few years, and the people were really friendly.  In their honor, I could monkey patch String as follows:

class String
    def greeting
        "howdy ya'll!"
    end
end

"abc".greeting # => howdy ya'll!

Unfortunately, there is no easy way to tell what made this change.  This would make it more explicit:

module JohnRagan
    module String
        def greeting
            "howdy ya'll!"
        end
    end
end

class String
    include JohnRagan::String
end

puts "abc".howdy  # => howdy ya'll
puts String.ancestors # "JohnRagan::String" shows up as a module

Now, the modules involved in monkey patching show up via ancestors.

Preventing Monkey Patches

Common lore is that folks in New York City are not as friendly as in Texas.  Whether or not this is actually true, let’s have some fun with it.  Let’s say a NYC developer writes a plug-in your Rails app includes, which contains the following:

class String
    def greeting
        "go screw yourself!"
    end
end

"abc".greeting # => go screw yourself!

Now, your code is suddenly rude and devoid of its previous Southern charm as this monkey patch has overridden the previous one.

While there may not much be you can do to prevent monkey patching by others, you can check to see if a method already exists before you monkey patch it yourself.  Rake has a mechanism it uses to prevent unintended monkey patching overwrites on its part.  Rake itself monkey patches the class Module, adding a method called rake_extension(method), which takes the method name, and method to be extended as a code block, and prevents the monkey patching (with a warning) if the method already exists.  I would have used it as follows:

require 'rake'
class String
    rake_extension("greeting" ) do
        def greeting
            "howdy ya'll!"
        end
    end
end

If my developer friend in New York City had already added a greeting method, this logic would have ignored my monkey patch.

class ClassFactory
  def self.create_class(new_class, *fields)
    c = Class.new do
      fields.each do |field|
        define_method field.intern do
          instance_variable_get("@#{field}")
        end
        define_method "#{field}=".intern do |arg|
          instance_variable_set("@#{field}", arg)
        end
      end
    end

    Kernel.const_set new_class, c
  end
end

ClassFactory.create_class "Car", "make", "model", "year"

new_class = Car.new
new_class.make = "Nissan"
puts new_class.make # => "Nissan"
new_class.model = "Maxima"
puts new_class.model # => "Maxima"
new_class.year = "2001"
puts new_class.year # => "2001"

In Ruby, classes are simply objects like any other, which are then assigned to a constant.  Hence, to create a new class dynamically we instantiate the class Class with Class.new, and then assign it to a constant via const_set (we invoke it on Kernel so that it is a top-level constant like any other class).  We then add the code that makes up the class in a do-end block.

In that do-end block, for each field we invoke define_method twice, first for the getter method and then the setter method with get_instance_variable and set_instance_variable, respectively.  For each field, we create the instance variables (e.g., for make, we use @make).  Note how we make use of passing the argument in for the setter.

Additionally, if I wanted to make the class a sub-class, I could have used Class.new(parent_class)

I gave a brownbag recently at work on Java Web Services.  To get that to work, I had to run utilities to generate skeletons and stubs in java source that were then compiled into byte-code.  As you can see above, no such steps are necessary as the metaprogramming is naturally built into the language.

In other words, I can create and use frameworks that leverage metaprogramming to make their use by end-users almost trivial.

Nonetheless, my love of Ruby does not cause me to knock Java.  I still enjoy coding in Java, and one should use the right tool for the job.

In order to be a better Rails and Ruby developer, I studied Pragmatic Programmer’s “Metaprogramming Ruby” (thumbs up).  By study, I mean read, took notes, created flash cards, wrote sample code and played, then finally created a suite of RUnit tests to exercise all the various capabilities.  “Mastering” Ruby metaprogramming greatly expanded my understanding of how Rails works under the covers.

I was surprised by how much Ruby metaprogramming had in common with JavaScript metaprogramming (such as dynamically defining methods).  Once again, this is one of the benefits of studying different languages – seeing the common patterns that emerge.

The next few blogs I do will go into Ruby metaprogramming.

Apparently I had missed the pragmatic part of that title.  As I am averaging three to five technical books a month right now, an investment in becoming more efficient in my learning and study would surely pay off.  So, I got the book, and I am glad I did.

Just as you want to be an efficient coder, if you are doing a lot of studying, you want to be an efficient learner.  I will talk about a few of things I found over a few blogs, but I highly recommend purchasing this book and implementing it.

In the Dreyfuss model, there are five levels of progression of mastery in terms of problem solving and the mental models you form:

• Novice.  They have little experience to rely on, but they can follow context-free rules (recipes) to guide them (think ISP help-desks and pre-canned scripts).
• Advanced Beginner.  They can break away from context-free rules a little bit and try things on their own, but they do not have a big picture understanding.
• Competent.  They have developed conceptual domain models and can work with them and solve problems.
• Proficient.  They can reflect, improve, and learn from the experience of others.  They can apply maxims (as opposed to just recipes) to problems.
• Expert.  They work from intuition instead of simply reason and are primary sources of information.

Becoming an expert is more than just collecting a body of skills and knowledge; it is about moving to intuition-based problem solving based upon those experiences.

We have a great cookbook that my wife and I pick out a new recipe from to try each week.  I take the recipe, and I follow it carefully (X amount of flour, Y amount of seasoning X, etc.).  A great chef (such as my grandmother) did not follow a recipe – it was based on intuition gained over experience (a pinch of this, adjust it this much because of the humidity, etc.).  I tried to ask my grandmother once how to make something, and she couldn’t really articulate her responses beyond “add a little bit of this” and “cook it until its ready but not too long”.  This is the typical mark of an expert.

What does this mean in terms of coding?  Well, it means becoming an expert is a little less about collecting skills and more about applying skills intuitively given a particular context.  Any fool can apply a parade of various design patterns (I recall being one of those fools 13 years ago), but an expert will intuitively apply the right pattern given a particular context.  That seriously shifted my paradigm when I read it.

Personally, I have a backlog of skills I am studying and investing in to increase my knowledge and skills portfolio.  However, I plan to spend a lot more time going forward on context.  One of the things I will soon be doing is personally reviewing my entire technical career to determine what I did and did not do right given the particular contexts I was working in.  Am I repeating any poor patterns, what should I continue, what should I drop?  I expect it to be a worthwhile exercise.

I got to the end of it, and discovered a link to Software Development Philosophies at http://en.wikipedia.org/wiki/List_of_software_development_philosophies.  Now, I can Kanban with the best of them!

A worthwhile read – check it out!

Writing a Rails app I have been working on for fun, it took longer for me to put a working regular expression together than I expected.  I had studied, summarized and even written a few short programs for regular expressions in Ruby.  However, what I hadn’t done was to go from the other direction – take a domain problem and map it back to a regular expression.  That’s when I came up with “Regular Expression Pushups”.

When I was younger, I used to be able to do a massive amount of push-ups.  Rather than continuing to increase the count, I tried to do them more quickly.  Similarly, I reviewed my notes and created thirty-three problems that would use the underlying techniques.  I would do these as quickly as possible, and then later change the questions a little to see if I could do them more quickly.  The idea was to better form the “regular expression neural networks” in my brain

I’ve gone through one round and have seen a huge improvement.  I expect the second round will go a lot faster than the first (how could it not)?!

Here are the ones I put together:

1. Find out if a certain string exists in as a substring within a document; and if so, where
2. Replace the contents of a string within a document with something else
3. Find three keys parts of a document and pull them out
4. Find the word “foo” in a sentence but it cannot be “foobar”.
5. Find a string that is a whole word only
6. Find a string that is not a case-sensitive match
7. Find the string foo or bar
8. Match foo and goo (and so forth) but not boo without using these as words in your formula
9. Find all strings that end in “oo” (3 character, and unlimited characters)
10. Find any string that ends in “oo” but boo is not valid (REPEATS #8?  Or subtle difference?  I think I reversed them)
11. See if a word matches that starts with “foo”.  Additionally, one that does not start with “foo”.
12. See if a string matches that ends with “bar”.  Additionally, one that does not end with “bar”.
13. Find a substring that begins with “foo” and ends with “bar”.
14. Find the first and last word in a sentence
15. Find the first character that is not a number or digit in a string
16. Find the first number in a string (and last number)
17. What is the text before the phrase “in the middle”?  What is the text that follows?
18. Find  “abc” or “abcabc” and so forth in the sentence
19. Find the string “abc” or “abcdef”
20. Find the string “abc” or “abc1″ or “abc11″ and so forth
21. Find all alphabetic words that end with the first “3″
22. Find the word that starts with an alphabetic character and ends with 17
23. Find the telephone number in the format 571-217-9451
24. Find a number (without commas) that is at least 5 digits
25. Find a number between 4 to 9 digits
26. Find the word that matches a sequence of 5 instances where there is one to 3 numbers and a single character
27. Find the string “abc” at the end of the string where there is a newline character
28. Given a dollar figure, return the portion without the cents.
29. Given string “#@%# 123bar 23bar 342 siojbar”, find the first word that does not have “bar” in it
30. Find all the instances of numbers greater than 5 digits
31. Find the number followed by the a space and word “bang” from “123 howdy 456 wow 789 bang”
32. Do a greedy match (from “abc!def!ghi!” get the whole thing for .+!)
33. Do a non-greedy match (from “abc!def!ghi!” get “the whole thing”abc!”" for .+!)

Suspecting a race condition, I played with some delays (for analysis only) between the child and parent deletions as well as between this service and the service invoking it.  No luck with that.  Perhaps the invoking service was taking time to release the directory resource so I tried a 15 second delay to rule that out – no luck as well.

It didn’t make sense for such a simple program.  My guess was that another process was sometimes not releasing the resource and got stuck down that path for a bit.  After a while, I decided to create a mind-map as to what was going on and what I was observing to see what would be revealed.  As suggested in the Pragmatic Wetware book by Andy Hunt that I am finishing up, after a little bit of time the R-mode of my brain took over and I found a number of things I could try.

One of those things was checking to see if the directory had any children even after deleting all of them.  Of course, this was silly because only the directory remained empty (the files were gone), and I almost skipped trying.  Much to my surprise, they were not empty.  Playing with Winscp, I discovered unexpected .nfs files were showing up as files were deleted.  Furthermore, deleting the normal children mysteriously caused these files to be created, and deleting these files caused other .nfs files to suddenly spawn into the directory.  Thus, the directories were no longer empty and could not be deleted.

I drove home thinking about this debugging incident and how to make it better and more efficient.  Here is what came to mind:

• The use of mind maps was certainly effective and something I want to continue
• Its important to challenge your assumptions and don’t get locked into them early.  Yes, it could be that another process did not release it, but there are other possibilities as well.  Using a mind map earlier would have helped, but more helpful would have been to assess how locked into my assumptions I was
• Debugging is a technical and creative endeavor.  Studying L-mode facts about the situation, and then employing R-Mode techniques earlier on would have helped
• You don’t need to stay locked on the problem until it is solved.  I should have moved on to other work and let the R-mode side of brain work on the problem in the background
• Sitting down and trying to think of all the “evil” ways that the system could be messing me up was also helpful
• Continue playing and trying things that should not happen – I am really glad I did that!

In the end, I coded my program to delete the children before the parent directory because File.delete did not get rid of non-empty directories.  Because I saw the children gone, that obvious possiblity eluded me for a bit.  That is what I am thinking about for the future.