Tuesday, May 06, 2003

The Zen of Libraries (Part I)

Copyright Notice

Copyright:Ramesh Ananthakrishnan 6th May 2003

Pilfer at your own Peril

   The time that most programmers cease using a language is when
they can no more implement painlessly their thoughts in that language using the standard
libraries provided with the language.

---The Second Rule of Language Design


When you think about the above quote, you'll see that there is a grain of
truth there. If you are a programmer you'll know what I mean. The time you decided
that you couldn't do stuff with Turbo C was when you probably wanted to add mouse
support and menu support to your programs without probably killing yourself first.
So you moved onto Win32 API's, (hard) or MFC (doubly hard), or VB (cool) but back in
VB you want to do stuff like parsing strings, and you wish you could do the entire
project in Turbo C. Or your figured out interrupts in Turbo C and are still
painfully writing programs that have mouse support in Turbo C. But you still wish
there was an easier way to program, especially manage those pesky strings in C that
cause all those crashes. Or maybe you moved onto Java, and are still waiting for
"javac" to finish the task you gave it yesterday. Anyway if you are somebody who
wants to figure out an easier way to finish your work using Turbo C, or want to know
what a library is then WELCOME!! This article is for you.

Let's start with the Strings

Strings are one of the most basic entities that a language can process. Usual
operations include the trouble of concatenating them, converting them, copying
them, and worrying about dynamically making them grow. So let's assume you have
as a rite of passage written this new excellent string class, that can do all of
the above and also dynamically grows. Wow! Now you need to use it in one of your
projects, which is a complicated sewer maintenance system. How do you do it?
An easy way would be to include the .h and .cpp files to your project and then
compile it. So let's say your sewer maintenance projects contains just two files:

1.sewer.h & sewer.cpp

then you add...

2.my_string.h & my_string.cpp to it. from the Turbo C project manager

so your code looks something like this:





So all that you have to do is compile the project and there you
are. Full project, and you used the string manipulation functions that
you have written, so you saved a lot of labour.

But now let's say in a moment of complete brainless idiocy your friend who wants these same
functions for his project decides to copy my_string.cpp but deletes the file before you even
compiled the project.
Not just to the Recycle Bin but out of your disk. What happens then?
Suppose you had to submit the project tomorrow. Suppose you were planning to sell it to Microsoft?
DOOM!!! DESPAIR!!! CURSE YOUR FRIEND!!! But actually, the moral here is to make sure that you don't
expose too much of the innards of your project.And to keep things separate. But then
how do you make sure you don't include the cpp file and yet include it's wonderful
behavior into your project? And suppose that the my_string.cpp file is so large that
it takes 6 hours for it to get compiled (not joking: 6 hours is easy compilation! IE
takes 96 hrs. to compile...). It's a terrible waste of time if you just included
my_string in another project only last TUEDAY, but have to go through the same 6 hour toil
again. So is there some way to protect your code and also reduce your work time....?

Enter Libraries (Advantage I: avoid recompiling)

The major reason libraries were introduced into C was to prevent the sort of error
that happens when you accidentally copy or delete somebody else's files when you
included the source files into your project. Especially if it is a large file that was written
over like say... the best part of a year. One that takes 24 hrs. to compile. And one which is
necessary to be used in other projects.
Apocryphal legend has it that Kernighan decided that libraries were in the C design when
somebody one day deleted all the C screen handling functions file he had written over a year
when they wanted to use it to develop a game.Without these libraries Unix could not even write
to screen. Basically Unix development just got stopped for three months while Kernighan rewrote
the damn library.
So Kernighan decided, no more does this sort of stuff happen, and thus were libraries introduced.
But as you all say, what is a library???

Well a library is a file that just consists of functions you want to use that are already
compiled and just ready, hot for being used in your code. You all know that every C program
goes through four phases preprocessing, compiling, linking and loading. You also know that
preprocessing, and compiling take up a lot of time (infact they take up a cool 90% of the source to
execution time, but that's another story....).
So in a library the functions that you have written are preprocessed and compiled and kept ready for
your use. So you are saved the trouble of compiling them over and over again, for every new project
that has to be done, thereby saving precious processor cycles. In fact all the mouse clicks and menus
that can be programmed in the simplest Windows program would take 3 days if the whole thing was
compiled from source. So that's one advantage of libraries. So what are the others....

Advantage II: Avoid reinventing the wheel. Cut your work in half

One of the most brilliant uses of libraries is that you can avoid writing code for stuff
that has already been done. Egs: When programming Windows you don't start by drawing dots
on the screen but use library routines that do that, and also draw other things like menus.
Mostly you just instruct the library to draw those menus someplace on the screen like say:
40 pixels from the left edge of the screen and 60 pixels below the top edge. But I guess you
didn't know we were doing that till now. Honest! I didn't till I thought about it when I started
writing this article. This means that there are thousands and thousands and thousands of
libraries that can do all sorts of stuff from drawing menus to other stuff. Chances are a good
bit of them may work with even Turbo C while others may require Visual C++ compiler. Don't despair
, search the web, and you can find many libraries. Which means that other people have probably
written stuff that does say 70% of your work. Believe me, you won't appreciate this till you
do your project and you have say 3 days, and it takes 2 and a half days to compile your project.
So check the web before you begin doing the next great library. Chances are it's up there and
it's better because it's been there for some time, and so many more bugs would have been fixed
or at least reported.

Advantage III: Share your work and be secure

So let's say you have the C standard libraries that does all the stuff like write to the screen,
malloc and input. Now assume that all the cpp files involved in making this library was available
to you.
Now suppose you user deleted the malloc function from the cpp file that does malloc. Suppose you
aren't even aware of it. Suppose all your programs are compile with the same cpp files,
all your programs are horribly bugged, and there is confusion all around.
To avoid this libraries are first compiled and then checked or tested. When there is reasonable surety
that the library is behaving according to specs,then the library and the header file(we'll see why later)
are sent to other people who want to use it. This makes packaging very easy. Infact it's tougher (but
we'll talk about that in another article...)
This prevents users from accidentally modifying the cpp file
and hence the entire library. This is also a security problem. You don't want malicious people to
"accidentally" change the body of the malloc function to be system("del *.*"). Think about it...
Most times it's better to distribute libraries not only to prevent malice, but to also
prevent the user from shooting himself in the foot.

NOTE:Those Unix/Linux aware may note that this is done by making the library file and the header file
as read only. This a far better security model than Windows which performs the same feat by making
libraries hidden files. And everyone knows even kids can delete hidden files. Sheesh!!! You paid
55,000 Rs. for MS Visual Studio, and now you have to bribe your Mom, Dad kid brother to make
sure they don't trash your programming tools, by deleting files they don't need. (Honest: My mom
did that to the registry one day...). I'll leave it to you guys to determine which is the better

Advantage IV: Organize your work

Infact organization of work becomes damnably simple. If you have a project you should divide it
into small libraries, each of which does independent tasks, and then call them separately from
the main program. In fact once you begin writing the main file, i.e. the file
in which main is placed you shouldn't need to compile the other libraries .cpp files again.
This means that you'll probably have to sit and think what each library should do,and what
exactly is required from your program, and what your program requires. This is a good thing,
because you are actually mapping out what calls what, and what does what and this help
enormously once you start programming. In fact this is how the design that is supposed to be
done by program analysts is done. Their job is to break the program into small functions and
determine which library does what all and all. Then the programmers code those pieces, and thus
was Rome burnt in a day.

NOTE: You'll never ever ever realize the wisdom of this till you are doing your final
project, 6 hrs. to go and you can't make head or tail of your code and just want it to work.
Organizing your program into libraries is like keeping your room organized. At first it
may be messy, but you can find your toothbrush whenever you want it, and soon it grows to
be a way of life.

So how do I start using libraries and what's the inside stuff about libraries?

So in this article we have seen why we need to use libraries. The advantages I listed were:

1. Avoid recompiling and save time

2. Avoid reinventing the wheel

3. Share your work, but protect it from doofuses

4. Organize your work

So, you ask me, if libraries are the next best thing to sliced bread how do I use them in
my project and what are they damn it! But peace, the true path to the student shall be
revealed in the next part of the article. The Zen of Libraries Part II

It's been sometime...

Yeah! it's been a long time. an awfully long time since I have updated the blog. But
sue me what! Of all the decisions I have regretted so far, the worst one has been to
join this pestillential institution of mine a.k.a Crescent and I am verily happy that
this part of my life is behind me forever (Yahoooo....!!!!Yayayaya....)! So long
you amigos.

I made one of the worst choices in life by learning Control Systems in this college
and hey did it hurt. Though I kinda like controls, it's not the kind of subject that's
taught with any skill in a cheri institution like Crescent. Well, and that intruded
with all the work I wanted to put into OS stuff. Anyway I'm glad it's over.

Motto of my life from now on: Controls is forever more a verb. Never a noun.
Never more a fucking subject.

Appologize for the break, but I shall be onto regular tech writing soon enough