Planning for software reuse is easy — mining is hard

Posted on Wednesday 17 September 2008

One of the best places to find reusable code is in your old projects.  However, “mining” your old projects for “reuse gems” (sorting through every VI, looking for sparkly little gems of general-purpose code that have immense value) is simply not an effective use of time or energy.

For example, if you were a miner looking for precious minerals, you would wouldn’t roam the countryside digging random holes in the ground.  Rather, you would work smart — you would try to identify a geographic location with a very high natural concentration of precious minerals (based on a variety of clues).  Guess what?  This is exactly how you should mine for reusable code in your past projects.  But, where do you start looking and what are the clues that will lead you to those reuse gems?

Here’s the secret:

While writing code, when you identify that you are writing a VI that is generic and has the potential to be reused, save it inside a sub-folder of your project folder called “Reusable VIs”.

If you adhere to this strategy, it will be very easy for you to come back later and find your reuse gems.  And, before you know it, you will have what resembles a reuse library.  Best of all, you didn’t even have to get your hands too dirty.

Note: As your reuse library grows (which it will if you use the planning techniques described above), you’ll need to start thinking about how to utilize your reuse library on multiple projects and share these VIs with other developers.  Make sure you don’t get stuck in the pitfalls of a monolithic reuse library caused by copying your reuse library from project to project or by using the same version of your reuse library on each project.  Create a VI Package and install your reuse library in your palettes using VIPM — it’s easy and simple.

I’d love to hear your feedback, so please feel free to leave a comment.  For example:

  • Do you have a folder in your project where you put new, reusable VIs?
  • Do you think that this is a good idea, or do you have a better strategy?

18 Comments for 'Planning for software reuse is easy — mining is hard'

  1.  
    crelf
    September 17, 2008 | 1:05 pm
     

    If we’re on a tight timeline and don’t have the time budget to develop for reuse during a project, we add “Reuse cadidate” text to the VI’s description – this allows the developer to continue to use a more intuative folder structure, while still tagging the VI (we use this technique for other tags too). We then use a very simple in-house tool that scans the VIs in a project for that text and generates a hyperlinked report to the source(s). It’s tempting to wait until the end of a project to run the tool to try to mine for reuse potential, but we have a team that does it every month to keep things fresh in enigneer’s minds. It also gives us an external view of reuse in a project, and also doesn’t make the final part of the project drag – just when you’re looking to the next project…

  2.  
    September 17, 2008 | 8:43 pm
     

    Hi crelf,

    It sounds like you are definitely “planning for reuse”.

    I like your idea of “tagging” VIs with annotations in the description — it’s both human-readable and machine readable. LabVIEW really lacks a standardized way of annotating code. BTW, some languages naively support annotations (e.g. Java annotation), which can be useful adding meta-data to code that can be used by development tools.

    Techniques like you describe certainly require tools to automate the process. I wanted to introduce an approach with folders because it’s a process that just about every LabVIEW developer can follow.

    You might find it interesting to know (and probably no surprise) that JKI uses the term “Reuse Candidate” too (rather than “Reusable VIs”). Our process (much like yours. I’m guessing) is a little more refined that just keeping a “reusable VIs” folder in each project. We migrate VIs out of the reuse candidates folders and move them (based on category) into a specific VI Package (e.g., “Array”, “String”, “File”, etc.). I’ll be talking more about that, soon. I’m looking forward to your feedback, in that area, too.

    Thanks,

    -Jim

  3.  
    Bob Hamburger
    September 18, 2008 | 7:27 pm
     

    Bloomy Controls (where I used to work) has a submission and vetting process for reuse VIs. A committee of CLA’s periodically reviews the candidates for appropriateness, completeness, utility, documentation, and (of course) style. Approved VIs are incorporated into the reuse library, and their names and descriptions are included into a searchable database. Perhaps things have improved since I left, but during most of my tenure at Bloomy, the reuse library was pretty much only used by the members of the review committee. Even then, people tended to only use VIs that they themselves wrote.

  4.  
    antoine châlons
    September 19, 2008 | 6:46 am
     

    I am seeing the same thing as Bob describes, it’s really hard to get people involved into submit candidate for reuse and it’s even harder to get them use reusable VIs made by someone else.
    Many people prefer to re-build their own version of the wheel…
    I’m just too lazy for that :-D

  5.  
    September 20, 2008 | 2:20 pm
     

    Hi Bob and Antoine,

    This is certainly an interesting issue that you bring up. How do you get people to use reusable code that others have developed?

    I think that the main factors are:

    • Accessibility – is it easy to get and use the reusable code?
    • Familiarity – Have people been exposed to the reuse code? They are more likely to use code that they already know how to use.
    • Quality – Is the reusable code well documented? Is it well designed? Does it work well?
    • Trust – Is the code trustworthy? Are the team members willing to use code that they didn’t write? This ties in closely with Quality

    I think that having an effective team-based reuse process requires addressing each of these facets.

    -Jim

  6.  
    Ton Plomp
    September 25, 2008 | 8:10 am
     

    Has anyone ever made a VI plugin for Google Desktop? Such a thing would be awesome in terms of mining…
    I started but never got the time (and knowledge) to finish it.

    Ton

  7.  
    September 25, 2008 | 9:49 pm
     

    Ton: I haven’t heard of such a thing (VI plugin for Google Desktop). Yes, that would be very cool!

  8.  
    Ray Robichaud
    October 3, 2008 | 8:45 am
     

    I like your secret: “While writing code, when you identify that you are writing a VI that is generic and has the potential to be reused, save it inside a sub-folder of your project folder called “Reusable VIs”.

    One item I would add is to create a sub-folder which identifies the categories so that it is easier to search through when creating the re-use library. Plus similar functionality would already be grouped together.

    I will be putting re-use libraries in place as I am now recruiting new employee(s) to do LabVIEW coding at customer site(s). The libraries will help novice programmers to progress much faster with learning, better coding practices and also provide a quick and efficient response to the client’s needs. Reducing coding time is an important aspect to be competitive in the software development world. Well documented re-use libraries should contribute to being more efficient.

    I will be learning how to properly use the VIPM and will likely carry on some discussion on LAVA conserning this.

    Good topic.. Thanks Jim!

    RayR

  9.  
    October 3, 2008 | 2:00 pm
     

    Hi Ray,

    Thanks for the feedback.

    I agree with you about organizing reusable VIs into subfolders. I had to try not to get ahead of myself and keep the article simple — there’s really a lot to cover when we’re dealing with software reuse.

    In the VI Package Guide on the JKI Software website, we have a page called “Packaging your Reuse Libraries” that has some tips on how to organize a reuse library. The first step is to organize your reusable VIs into categories that will become individual packages.

    For beginners, we try to have them just build a single package as a starting point — they can always split the package up into categorized packages, later.

    I’m happy to hear that you’re going to be spending some time learning about VIPM! I’m looking forward to chatting with you about your experiences and feedback.

    Thanks,

    -Jim

  10.  
    crelf
    October 5, 2008 | 2:05 pm
     

    Jim wrote: “The first step is to organize your reusable VIs into categories that will become individual packages.”

    We package in a way that reflects the struture of the default LabVIEW palettes (eg: we have a Programming.Numeric package, a Programming.Array package, a Programming.Dialog & User Interface package, etc), along with package icons that are the same as their counterparts in the standard palettes (with a little VIE glyph in the corner). We do this for a couple of reasons: it’s more intuative for the user, and we hope to one day be able to put our palettes directly into the exisiting palettles. Sure, occasionally we have a need to create a package that doesn’t fit within the default model (eg: we have an Addons.VIPM palette for all our VIPM pre- and post-build hook VIs), but they’re decided on a case-by-case basis.

    I think one of the reasons that we’re successful wit reuse is that we have a documented process that defines how to buuild a package the “VIE Way”, which includes where to put items – this menas that we don’t need a select group of engineers to keep control of the internal reuse package repository (we do have a repo librarian, but their role is strictly QA). Everyone is encouraged to be a different part of the process for a subimission – this limits the senior-engineer-only elitism that can occur, which really devalues reuse (as Bob suggested).

    I totally agree with Jim’s four key factors:

    Accessibility – that’s what VIPM is for. We have tried several methods to acheive this previously (including in-house tools), but VIPM is certainly the best tool hor accessibility IMHO.
    Familiarity – training, training, training. We have a TSIG (Test Software and Integration Group) meeting every week, and one of the parts of that meeting is a short session on items in our reuse library (as well as items available through OpenG). Engineers won’t use something if they either don’t know about it or don’t know how it will benefit them.
    Quality – how easy wil it be to implement in my code? Would it be easier to just make my own rather than try to understand what’s going on here?
    Trust – a lot of my job is about lowering engineering anxiety, and this is a big area of it. The question here should not be “will the reuse code be as good as if I’d written it?” Instead, it should be “does the functionality of the reuse code meet my requirements (or will with a lottle wrapping/brokering)?”

    I wasn’t planning on writing so much in this comment :)

  11.  
    October 5, 2008 | 2:29 pm
     

    Crelf,

    Wow! That’s a long comment — looks like you need your own blog ;)

    One reuse concept worth contemplating is that, in addition to highly-reliable and well designed code, there needs to be an environment where people can easily work on and share reusable nuggets of code. If we make it easy for reusable nuggets to be shared, then we’re lowering the activation energy of software reuse and promoting cross-pollination of ideas. Just as people require high-quality code that they can count on, they also need a place where they can quickly share things that are at the concept/prototype stage.

    I believe that VIPM can play a critical role in facilitating software reuse at this early stage as well. And, because VIPM is highly modular and offers configuration management capabilities, users who decide to use concept/prototype stage code in their projects do not run the risk of having changes to prototype code break their project unless they upgrade the packages that contain them.

    What this sort of process requires is a way for reuse library participants to easily know what to expect from a given reuse artifact (what is it’s development status, quality level, etc.).

    One possibility is to have two separate packages for each package category: one that is “certified” (meets organizational quality requirements) and another that is “uncertified” (does not meet certain quality requirements).

    Another possibility is to segregate the “uncertified” code into some subfolder of the package. You could make it clear that a VI is uncertified by coloring its icon and putting notices on the Front Panel and VI description.

    What are your thoughts about this? Can VIPM play an important role in facilitating the sharing and evolution of prototype reuse code in addition to mature reuse code that has gone under extensive testing, review, and other quality control processes?

    Thanks,

    -Jim

  12.  
    October 5, 2008 | 3:37 pm
     

    Crelf,

    One more thought… Regarding your comment: “Trust – a lot of my job is about lowering engineering anxiety, and this is a big area of it.”. This is a huge aspect of software engineering. The less that software engineers have to worry, the more that they can spend their energy creatively solving problems.

    -Jim

  13.  
    crelf
    October 8, 2008 | 1:06 pm
     

    Jim Kring wrote: “I believe that VIPM can play a critical role in facilitating software reuse… because VIPM is highly modular and offers configuration management capabilities…”

    I totally agree – that’s why I said this: http://jkisoft.com/vipm/customer-quotes/ :)

    Jim Kring also wrote: “(library participants to easily know what to expect) One possibility is to have two separate packages for each package category: one that is “certified” (meets organizational quality requirements) and another that is “uncertified” (does not meet certain quality requirements).”

    We (kind-of) do that with a candidate and accepted deliniation. Reuse candidates are specified so by them being placed into a designated area by the submitter – all engineers are welcome to use code in the candidate area, but at an unknown level of risk. Once the candidate is accepted to the reuse review process, it is copied to another area where it is worked on to bring it up to QA requirements, changed to be more.less generic, etc. Once it’s through that process, it is released (consequently as a VIPM package in a controlled common area).

    Then Jim Kring wrote: “Can VIPM play an important role in facilitating the sharing and evolution of prototype reuse code in addition to mature reuse code that has gone under extensive testing, review, and other quality control processes?”

    It’s certainly worth discussion, but I’m not sure how. Components/Modules/Architectures that are yet to be certified are often in a state of heavy flux, and I don’t see how VIPM’s features (in their current form) can be applied to that. That said, let’s look to the future…

  14.  
    crelf
    October 8, 2008 | 1:16 pm
     

    Jim Kring wrote: “(lowering engineering anxiety) …is a huge aspect of software engineering. The less that software engineers have to worry, the more that they can spend their energy creatively solving problems.”

    Lowering engineering anxiety is a key reason to embrace reuse. That said, the quality of the reuse component must be significantly high enough to eclipse the engineer’s natural want of doing it themselves. I’ve seen situations where an engineer is burned by one reuse component and arbitrarily decides to abandon all reuse components and code everything from scratch themselves. This is, of course, incredibly immature, but also a little understandable. What those people need to do is recognize that not all code is perfect (including their own), and that reuse is an iterative process. Plenty of effort is usually put into bringing new components into a reuse ecosystem, but an equal (if not more) amount of energy is needed to maintain and upgrade existing components (there’s less glory there, but it’s as, if not, more, important). That’s why organizations that truly want to succeed through reuse need to have a process in place to avoid these pitfalls. Otherwise the reuse effort is (after a flurry of initial excitement and effort by a champion or two) is doomed to fail. Another key factor is that the process needs to involve everyone – all of the users of the reuse components, not just a champion or two.

    I could go on, but I fear that I’m starting to evangelize :) If you truly want to hear me wax philosophically about reuse, all you need to do is find me…

  15.  
    American subprimes
    October 11, 2008 | 5:35 am
     

    the concept of “general VI to reuse in many project” is obsolete nowadays.
    This concept belongs to ‘90.
    We are in 2008… Software engineering has evolved. It’s not like you discovered America with VIPM which is a internet-updated collection of VIs
    Standard libraries exist since 1980, and LV is so lacking of libraries that VIPM seams a huge milestone!
    Good luck with your tour!

  16.  
    October 11, 2008 | 11:59 am
     

    American subprimes: How is the concept of reuse obsolete? Are you saying that software reuse is not important anymore? Are you saying that you have better ways to reuse code? What types of software engineering problems do you see as important, now days?

  17.  
    November 22, 2008 | 4:36 pm
     

    Twittered your post on Twitter – http://twitter.com/swreuse/status/1018782258

  18.  
    November 22, 2008 | 4:47 pm
     

    Software Reuse: Thanks for adding these articles to your twitter feed on software reuse. I’ll keep my eye on your feed, as it seems that you’re doing a great job finding good articles on software reuse!

Leave a comment

(required)

(required)


Information for comment users
Line and paragraph breaks are implemented automatically. Your e-mail address is never displayed. Please consider what you're posting.

Use the buttons below to customise your comment.


RSS feed for comments on this post |

 

Bad Behavior has blocked 775 access attempts in the last 7 days.