Monolithic vs. Modular Software Reuse Libraries (Part I)

Posted on Monday 11 February 2008

If you’ve gotten past the horrendously boring title of this article, you probably know a little bit about software reuse libraries.   You probably even contribute to a software reuse library (a personal reuse library or one belonging to your organization).   So, I won’t go into the benefits of code reuse and the pitfalls of reinventing the wheel -- I’ll jump right in and get to the meat of the matter.   Let’s consider two strategies for managing a software reuse library.

Monolithic vs. Modular Reuse Libraries

A monolithic reuse library is one where all components of the library are released at the same time as a single unit.  A modular reuse library is one where several individual libraries are released at different times, independently as separate and distinct units. For example, LabVIEW ships with a monolithic reuse library called vi.lib , the built-in library of VIs found beneath the <LabVIEW>\vi.lib folder.   These VIs are all kept under the same revision and release cycle as LabVIEW, itself. You'll only get new versions of these w:st="on"> VIs when LabVIEW itself is released -- it's monolithic, all or nothing. National Instruments (the makers of LabVIEW) also sells modular libraries, such as the LabVIEW add-on toolkits.   These libraries each have their own version, different from LabVIEW’s, and are installed separately.   These add-on libraries have dependencies on vi.lib VIs , but rarely have dependencies on other add-ons. zid="101">

src="http://thinkinging.com/wp-content/uploads/2008/02/el_cap.png" style="padding-top: 10px;" align="right" border="0" hspace="4" vspace="0"> zid="34"> Monolithic wins the short race

zid="84"> Most LabVIEW developers and teams find that the overhead of the modular reuse library is a show-stopper , right out of the gate.   It’s too much work compared to the much-simpler monolithic reuse library, which can be developed and deployed as a single unit. zid="43">  zid="105"> The monolithic reuse library is alluring, because it’s: zid="48"> 
  • easy to distribute – it might be delivered as a single ZIP file, or a checkout from source code control.
  • easy to keep track of which version you have – there is a single version number for the entire reuse library (perhaps a date stamp)
  • easy to manage, since there are no external dependencies – every VI that you need is kept within the reuse library, so users will never have missing VIs. zid="55">

But, there are scalability concerns

After the monolithic reuse system is up and running, several problems loom on the horizon that will slow down progress.   Soon, the reuse system might become a victim of its own success and get harder and harder to maintain as the code-base grows.

The solution is a modular system

The solution to the scalability issues (number of developers, number of project, number of reuse w:st="on"> VIs ) is to make the reuse library modular.   The only other option is to stop improving the reuse library – but, we can’t do that, of course.  In href="http://thinkinging.com/2008/06/09/monolithic-vs-modular-software-reuse-libraries-part-ii/">part II of this series, I talk about the specific href="http://thinkinging.com/2008/06/09/monolithic-vs-modular-software-reuse-libraries-part-ii/">shortcomings of the monolithic reuse library and how the modular approach can solve them. Of course, this introduces some new problems that will have to be addressed.

What is your experience?

Please share your experience with LabVIEW software reuse.  I'm curious to know the following:
  • Do you have an effective LabVIEW reuse system?
  • What does your system look like?
    • Is your system monolithic or modular?
    • How big is the library?
    • How many developers contribute the the library?
    • How do you manage your installation?
  • Are you feeling any pain points?
  • How do you plan to solve them?

style="font-style: italic;">El Capitan style="font-style: italic;" href="http://en.wikipedia.org/wiki/Image:Yosemite_El_Capitan.jpg">photo style="font-style: italic;"> courtesy of Mike Murphy.

12 Comments for 'Monolithic vs. Modular Software Reuse Libraries (Part I)'

  1.  
    Albert Geven
    February 12, 2008 | 11:05 am
     

    Hi Jim

    The perfect way to instrument drivers is not at all monolitic

  2.  
    February 12, 2008 | 11:19 am
     

    Hi Albert,

    Yes, instrument drivers are a great example of a modular component. And, one of the challenges is finding and installing them (in a way that makes them available to all developers working on a given project and also available in the palettes). Generally, instrument drivers (especially those developed by instrument vendors) do not have external dependencies, other than vi.lib.

    In the 2rd or 3rd part of this series, I’ll discuss instrument drivers as well as NI’s instrument driver installer framework.

    Thanks,

    -Jim

  3.  
    February 12, 2008 | 3:09 pm
     

    In the article a monolithic library was specified to be a library where the library is released as a single entity. However when shortcomings of monolithic library are considered, they seem to be shortcomings of non-modular architecture and not shortcomings of monolithic library.

    I don’t consider monolithic library to be a bad thing per se. Monolithic library can be built-up from modules. However the modularity is simply hidden from the user. This causes no limitations to the upgrade process of the monolithic library. However the monolithicy gives the user a guarantee that the modules of which the library is built will always work together and that the library is tested as entity prior to releasing a patch.

    I think the difference is a little bit same as with OS X and Linux. Linux is a modular architecture. Everyone can choose which packages they prefer and built a suitable Linux system of their own. With OS X everything comes in a monolithic package and it simply works. The maintenance of OS X is therefore much simpler from the user point of view, still not functionality or modularity needs to be sacrified. The problem only arises if very esoteric configurations is needed such that an old version of one module needs to be mixed with a new version of another module. Then monolithic library cannot cope with the situation.

  4.  
    Uwe Frenz
    February 13, 2008 | 2:44 am
     

    Jim,
    finally I am a single LabVIEW developer here, so my experience is somehow luimited. Anyway, here are my answers:
    I have a LabVIEW code reuse system and I use it. If it’s effective(ly), who knows?
    It is a mixed system, consisting of some modules of dedicated functionality (drivers, system and UI-tools) and a huge mix of common VIs. Those are usually simply copied to a new project folder to start the new project.
    My biggest project contains >800 subVIs, where about 400 are statically linked selfe-made VIs & Ctls and a huge more which are dynamically called or started. So the biggest ‘library’ is the project core that contains about 250 modules. We used to have a second develloper, but our coding style was not that compatible so just about some 20-30 VIs of that person are still in the libs.
    Yes, A am unsatisfied about the situation. Some problems are selfe made- mostly out of lacjk of experience. But there must be a better way of good code re-use practice with LabVIEW!
    My actual plan of resolving some of my problem is a to clean up my lbs and -maybe create some new libs. And – if my bosses accept itz, try to move ione or the other to GOOP.

    Greetings from Germany!
    – Uwe

  5.  
    Rolf Kalbermatter
    February 13, 2008 | 2:52 am
     

    I think the lines between monolithic and modular are not as clear as we all would like to make them be. Take the OpenG libraries. Some of them are almost monolithic in that they provide everything they need to operate themselves, others are more modular in that they need other libraries from the OpenG pool. But in fact I think they are all modular in that they are usually based in some ways on the “monolithic” vi.lib library.

    And the same with Instrument drivers. They are inherently self contained in that they do not need other LabVIEW libraries other than vi.lib to work and could be considered monolithic but they of course depend on a lot of things such as VISA, NI-DAQ and friends, and built in LabVIEW functions such as TCP/IP etc.

    The problem is not so much about monolithic versus modular but about clearly defined and very stable interfaces. VISA is such an interface and that makes the Instrument Drivers sort of monolithic in their own since they can simply assume that VISA will work the way it does if it is there at all.

    Once you start to change interfaces you always end up in big troubles unless you use a fully monolithic approach. And that approach does not scale at all nor is it manageable in the long run. So the problem is not so much about not using modular libraries but about making interfaces that are stable and can be used in a backwards compatible way and that is one of the real challenges to software design, since monolithic code only works well for a small and limited scope.

  6.  
    Dave Grimshire
    February 13, 2008 | 6:14 am
     

    I’m confused. What do you call a library made of Labview, C, C++, Fortran and assembler code compared to a library made only of Labviw vi’s? It seems to me the library of only Labview is monolithic.
    Modular to me means the ability to call various rouitines from the same library. It wouldn’t be modular if you made one call with different parameters.
    Anyone want to define monolithic or modular?

    Dave G.

  7.  
    Shane
    February 13, 2008 | 7:17 am
     

    I think there’s a huge grey area here.

    You can have a library which is monolithic. You can have a library made up of lots of (individually seen) monolithic parts with clearly defined interfaces. Is that modular? There’s a whole universe in between. The definition of the interfaces is probable the most important aspect. Anything which may change an interface MUST be hidden from the user, otherwise compatibility problems occur.

    I think it depends on what scale we look at the code. If we zoom in close enough, everything becomes monolithic at one stage or another. The question is how finely grained should the monolithic components be?

    Shane.

  8.  
    zen
    February 13, 2008 | 8:42 am
     

    I have a set of reusalbe sub VIs. They are basically collection of freaquently used VIs and I draw a little nice icon picture. I did not explicitly design those codes either to be monolici or modular. I do not install into LabVIEW folder but keep my library in a folder. Especially, LabVIEW 8.x has Project Manager. I usally add my folder to my projects and refer library VIs in it. I am working by myself or with only few colleagues, and distribution does not really matter to me.

    The biggest problem rises when I upgrading LabVIEW. Since my VIs depends on vi.lib and properties of front panel objects, upgrading can often cause the broken arrow. Currently I find and fix broken VIs manually. I am interested in how other people handle the situation.

  9.  
    Michael Ashe
    February 13, 2008 | 9:39 am
     

    Interesting article Jim. How to phrase this?

    There are two aspects or “domains” here, a spatial one (how the library fits together) and a temporal one (how versioning and compatibility is handled over time. In the monolithic library the spatial and temporal aspects are linked, one to one. As soon you start breaking the monolith up into modules you enable the separation of the two domains, and what was once representable with a linear model becomes more of a sparse tree hierarchy in 2D. (or even 3D+, more on that later after your next article)

    I’m assuming your second article will get into this a lot more, as you alluded. I don’t view the two patterns, mono vs module, as mutually exclusive, rather as a base root (vi.lib, etc) with branches. I have several layers to my reuse library, vi.lib, OpenG, other OpenSource and finally my own VIs from over the years. I’ve been fortunate to get the OpenG libraries and BSD open source code in general to be included in my latest customers project (by the way, thanks for the latest version of VIPM 1.1 !!). We are soon to be in the process of comverting some of the code we have developed for this customer’s project into internal OpenG style packages.

    Looking forward to your part 2!

  10.  
    February 13, 2008 | 3:49 pm
     

    Wow! I’m amazed and all the great feedback. Software reuse seems to be a hot topic, near and dear to many of us.

    Yes, this article may have oversimplified things. The terms “library” and “monolithic” can certainly signify different things to different people and there are some gray areas. By reuse library, I meant the whole body of an individual or organization’s reusable software assets. And, when I said monolithic, I meant something that is a single unit, somewhat inseparable from it’s parts, tending to be large, and having few (if any external dependencies).

    The thing that defines (to me) whether a reuse library is modular or monolithic has more to do with the release/deployment to the users. If a user of the reuse library must (or typically does out of convenience) obtain it as a single unit, then it is being treated as monolithic reuse system (all or nothing).

    Michael Ashe also brings up a good point, that there are temporal as well as spatial dimensions to the reuse model. A modular library brings with it a slew of management issues as we move towards a system of interdependent modules of varying versions. Of course, as Michael hints, a package management system can provide a great solution to these problems. And, it’s probably no mystery that’s where I’m headed with the discussion ;)

  11.  
    crelf
    February 18, 2008 | 10:20 am
     

    Looking forward to the next installment :)

  12.  
    June 10, 2008 | 8:01 pm
     

    [Update] I have just posted part II of this article, here:

    Monolithic vs. Modular Software Reuse Libraries (Part II)

    I hope it was worth the wait.

Leave a comment

(required)

(required)


Information for comment users
Line and paragraph breaks are implemented automatically. Your e-mail address is never displayed. Please consider what you're posting.

Use the buttons below to customise your comment.


RSS feed for comments on this post |

 

Bad Behavior has blocked 610 access attempts in the last 7 days.